paperbot · PL 论文追踪

全部论文 待读 收藏

P2 From Prompts to Templates: A Systematic Prompt Template Analysis for Real-world LLMapps

从提示到模板:面向真实世界 LLM 应用的系统性提示模板分析
FSE Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering2025匹配度 2/10 · 该论文聚焦提示模板的组件归纳与指令遵循性能分析,属于提示工程/LLM应用设计研究,与用户核心方向(静态分析、形式化验证、浮点程序分析、贝叶斯程序分析与LLM交叉)无直接关联;用户明确不偏好研究目标本身是LLM或提示词技巧的工作。引用 20 inbox
Yuetian Mao, Junjie He, Chunyang Chen
crossref: 2026-09-12

AI 速览

一句话:系统分析了真实 LLM 应用中的提示模板,归纳其组件、占位符与共现模式,并检验其对指令遵循性能的影响。

大语言模型通过自然语言提示实现直观的人机交互,但提示设计的细微差异会导致输出显著不同,因此 LLM 驱动的应用(LLMapps)普遍借助提示模板来简化交互、提升可用性并支持文档分析、创意生成、代码合成等专门任务。然而当前模板设计高度依赖个人经验与反复试错,缺乏系统化方法。本文对实用 LLMapps 中的提示模板展开全面分析:作者从开源 LLMapps(包括 Uber、Microsoft 等领先企业的项目)构建了真实模板数据集,结合 LLM 驱动分析与人工复核,对模板组件和占位符进行分类,统计其分布并识别高频共现模式;进一步通过样本测试评估这些模式对 LLM 指令遵循表现的影响。研究结果为开发者提供了可操作的提示模板设计启示,有助于推动 LLMapps 在工业场景中的采用与优化。

问题
LLM 应用的提示模板设计目前主要依赖个人经验与反复试错,缺少对真实模板构成规律的系统性认识,开发者难以有依据地优化模板。
方法
从开源 LLMapps(含 Uber、Microsoft 等项目)构建真实提示模板数据集,采用 LLM 驱动分析与人工复核相结合的方式,对模板组件与占位符进行分类、分布统计并挖掘高频共现模式。
评估
通过样本测试考察所识别模式对 LLM 指令遵循性能的影响,并据此总结面向开发者的提示模板设计实践经验。
原文摘要(Abstract)

Large Language Models (LLMs) have revolutionized human-AI interaction by enabling intuitive task execution through natural language prompts. Despite their potential, designing effective prompts remains a significant challenge, as small variations in structure or wording can result in substantial differences in output. To address these challenges, LLM-powered applications (LLMapps) rely on prompt templates to simplify interactions, enhance usability, and support specialized tasks such as document analysis, creative content generation, and code synthesis. However, current practices heavily depend on individual expertise and iterative trial-and-error processes, underscoring the need for systematic methods to optimize prompt template design in LLMapps. This paper presents a comprehensive analysis of prompt templates in practical LLMapps. We construct a dataset of real-world templates from open-source LLMapps, including those from leading companies like Uber and Microsoft. Through a combination of LLM-driven analysis and human review, we categorize template components and placeholders, analyze their distributions, and identify frequent co-occurrence patterns. Additionally, we evaluate the impact of identified patterns on LLMs' instruction-following performance through sample testing. Our findings provide practical insights on prompt template design for developers, supporting the broader adoption and optimization of LLMapps in industrial settings.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot9060,
  title = {From Prompts to Templates: A Systematic Prompt Template Analysis for Real-world LLMapps},
  author = {Yuetian Mao and Junjie He and Chunyang Chen},
  booktitle = {FSE},
  year = {2025},
  doi = {10.1145/3696630.3728533}
}