paperbot · PL 论文追踪

全部论文 待读 收藏

P2 The Impact of Hyperparameters on Large Language Model Inference Performance: An Evaluation of vLLM and HuggingFace Pipelines

超参数对大语言模型推理性能的影响：vLLM 与 HuggingFace Pipeline 的评估
FSE Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering2025匹配度 2/10 · 论文研究 vLLM/HuggingFace 推理引擎的超参数与吞吐性能，属于 LLM 系统性能评测，不涉及静态分析、Agentic 程序分析、浮点程序分析或贝叶斯程序分析。它与用户关注的 PL 与 AI 交叉核心方向基本无关，仅在“机器学习系统”层面有微弱关联。引用 4 inbox
Matias Martinez
机器学习与程序语言运行时与内存管理
crossref: 2026-09-12

AI 速览

一句话：通过在 vLLM 与 HuggingFace pipelines 上评测 20 个开源大模型的吞吐，揭示超参数配置对推理性能的关键作用。

开源大语言模型让开发者能在掌控隐私与合规的前提下构建 AI 应用，但需要推理引擎将模型权重加载到 GPU 等资源并处理查询，而推理速度对实时应用至关重要。本文以吞吐量（单位时间生成的 token 数）为核心指标，对 20 个开源大语言模型在 vLLM 与 HuggingFace pipelines 两种推理库上的性能进行系统评测，考察开发者必须配置的多种超参数如何影响推理性能。结果表明，超参数优化对达到最佳性能不可或缺；在 GPU 升级或降级时重新进行超参数调优也能提升吞吐，例如在 HuggingFace pipelines 中平均分别提升 9.16% 与 13.7%。

问题
LLM 推理引擎的众多超参数需由开发者自行配置，但其对推理吞吐等性能指标的影响缺乏系统性的量化评估。
方法
在两个推理库（vLLM 与 HuggingFace pipelines）上对 20 个开源大语言模型进行吞吐量评测，分析多种可配置超参数对性能的影响，并在 GPU 升级与降级场景下进行超参数调优实验。
评估
以单位时间生成 token 数（吞吐）为指标比较不同超参数配置，结果显示超参数优化能显著提升性能，在 GPU 升级/降级后重新调参于 HuggingFace pipelines 上平均带来 9.16% 与 13.7% 的吞吐提升。
原文摘要（Abstract）

Open-source large language models (LLMs) enable developers to create AI-based solutions while maintaining control over aspects such as privacy and compliance, thereby providing governance and ownership of the model deployment process. To utilize these LLMs, inference engines are needed. These engines load the model's weights onto available resources, such as GPUs, and process queries to generate responses. The speed of inference, or performance, of the LLM is critical for real-time applications. In this paper, we analyze the performance, particularly throughput (tokens generated per unit of time), of 20 LLMs using two inference libraries: vLLM and HuggingFace's pipelines. We investigate how various hyper-parameters, which developers must configure, influence inference performance. Our results reveal the importance of hyperparameter optimization to achieve maximum performance. We demonstrate that applying hyperparameter optimization during GPU upgrades or downgrades can enhance throughput. For instance, in Hugging-Face pipelines, this approach yields average improvements of 9.16% and 13.7%, respectively.

链接与引用

DOI 原文 · PDF（开放获取） ·

BibTeX 
@inproceedings{paperbot9021,
  title = {The Impact of Hyperparameters on Large Language Model Inference Performance: An Evaluation of vLLM and HuggingFace Pipelines},
  author = {Matias Martinez},
  booktitle = {FSE},
  year = {2025},
  doi = {10.1145/3696630.3728704}
}