一句话：提出以任务原语为单位的细粒度端到端编排，将查询工作流表示为原语级数据流图，实现跨模块并行与流水线优化，最高加速 2.09 倍。
基于大语言模型（LLM）的应用同时包含 LLM 与非 LLM 组件，二者共同决定端到端延迟。已有工作主要优化 LLM 推理本身，而端到端工作流优化长期被忽视。现有框架以任务模块为粒度进行粗粒度编排，优化被限制在模块内部，导致调度次优。本文提出细粒度端到端编排：以任务原语作为基本单位，将每个查询的工作流表示为原语级数据流图。这一表示显式暴露了远为更大的设计空间，使得跨不同模块原语之间的并行化与流水线优化成为可能，并增强了调度能力以提升应用级性能。作者实现了编排框架 Ayo。在多种常见 LLM 应用上的综合实验表明，Ayo 相比现有系统可获得最高 2.09 倍的加速。
Large language model (LLM)-based applications consist of both LLM and non-LLM components, each contributing to the end-to-end latency. Despite great efforts to optimize LLM inference, end-to-end workflow optimization has been overlooked. Existing frameworks employ coarse-grained orchestration with task modules, which confines optimizations to within each module and yields suboptimal scheduling decisions. We propose fine-grained end-to-end orchestration, which utilizes task primitives as the basic units and represents each query's workflow as a primitive-level dataflow graph. This explicitly exposes a much larger design space, enables optimizations in parallelization and pipelining across primitives of different modules, and enhances scheduling to improve application-level performance. We build Ayo, a novel orchestration framework for LLM-based applications that implements this scheme. Comprehensive experiments show that Ayo can achieve up to 2.09x speedup over existing systems across various popular LLM applications.