一句话：SimServing 通过原生执行服务框架、仅替换 GPU 计算为延迟估计器，实现可随上游演化的 LLM 服务配置调优。
LLM 服务配置空间大且影响性能，但在真实 GPU 上调优成本极高。现有模拟器重新实现服务框架的调度与内存管理等控制逻辑，形成语义副本，随上游框架演化而保真度下降；对连续八个 vLLM 版本的研究证实了该偏差。观察发现服务管理软件主导性能保真度却仅占不到 10% 执行时间，而 GPU 计算超过 90%。SimServing 因此执行原生未修改的服务软件，仅将 GPU 计算替换为可插拔延迟估计器，以执行而非建模保留控制流并自动继承上游更新。将其与粒子群优化结合调参，吞吐预测误差从 13.8% 降至 4.9%，模拟加速最高 21.18 倍，配置吞吐较基线提升最高 10.2%。
Large Language Model (LLM) services expose a large configuration space that critically affects performance, yet tuning these configurations on hardware (e.g., NVIDIA GPUs) is prohibitively expensive. Simulation offers a scalable alternative, but state-of-the-art simulators suffer from a fundamental software engineering pitfall: they re-implement the serving framework's complex control logic (e.g., request scheduling and memory management), creating a semantic copy that inevitably diverges from the rapidly evolving upstream software. Our empirical study across eight consecutive vLLM versions confirms this fidelity drift, with prediction errors widening as the framework evolves. We observe that the serving management software dominates performance fidelity yet accounts for less than 10% of total execution time, with GPU computation consuming over 90%. This asymmetry motivates SimServing, a hybrid simulation framework that executes the native, unmodified serving software while replacing GPU computation with a pluggable latency estimator. This design preserves the serving management software's control flow by executing it rather than modeling it, thereby automatically inheriting upstream updates and substantially reducing the maintenance burden associated with re-implementation. Integrating SimServing with Particle Swarm Optimization for configuration tuning, our evaluation shows that it reduces throughput prediction error from 13.8% to 4.9%, achieves up to 21.18× simulation speedup, and discovers configurations that improve throughput by up to 10.2% over existing baselines.
@inproceedings{paperbot12484,
title = {SimServing: Native-Execution Simulation for Evolution-Resilient LLM Serving Configuration Tuning},
author = {Yujie Huang and Jiazhen Gu and Zhihan Jiang and Michael R. Lyu},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3834362}
}