一句话：提出弹性流水线技术，在移动端 NLP 推理中同时缓解模型加载 IO 与内存占用带来的延迟瓶颈。
移动应用越来越多采用端侧 NLP 推理以保护隐私并避免网络往返，但 NLP 模型规模空前，同时压迫延迟和内存。将整个模型常驻内存可尽早启动执行，却使单应用内存占用增加数倍，限制可服务推理次数；按需从存储加载又引入数秒 IO，远超用户可接受延迟；逐层流水线加载与计算也因 IO 与计算延迟高度偏斜而无法隐藏 IO。为此，作者提出 STI，通过弹性流水线（elastic pipelining）协调模型加载与执行，以缓解移动端延迟与内存之间的资源张力。摘要未报告具体评估结果。
Natural Language Processing (NLP) inference is seeing increasing adoption by mobile applications, where on-device inference is desirable for crucially preserving user data privacy and avoiding network roundtrips. Yet, the unprecedented size of an NLP model stresses both latency and memory, creating a tension between the two key resources of a mobile device. To meet a target latency, holding the whole model in memory launches execution as soon as possible but increases one app’s memory footprints by several times, limiting its benefits to only a few inferences before being recycled by mobile memory management. On the other hand, loading the model from storage on demand incurs IO as long as a few seconds, far exceeding the delay range satisfying to a user; pipelining layerwise model loading and execution does not hide IO either, due to the high skewness between IO and computation delays.