paperbot · PL 论文追踪

全部论文 待读 收藏

Prompt Coach: An Empirical Evaluation of an Agentic Tutor for Learning Prompt Engineering in Software Development

Prompt Coach:面向软件开发中提示工程学习的智能体导师的实证评估
ASE Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering2026引用 0 inbox
Rohit Mehra, Kapil Singi, Vikrant Kaulgud, Vibhu Saujanya Sharma, Swapnajeet Gon Choudhury, Swati Sharma, Adam P. Burden, Majd Sakr
crossref: 2026-10-09

AI 速览

一句话:提出并实证评估 IDE 内嵌的智能体导师 Prompt Coach,用苏格拉底式提问帮助开发者提升代码生成提示词编写能力。

提示工程已成为软件开发者的关键技能,但其演化迅速、交互性强且依赖具体情境,传统学习方式难以胜任。为此作者提出 Prompt Coach(PC):一个嵌入 IDE 工作流的智能体导师,它以苏格拉底式提问引导开发者在真实编码过程中学习如何撰写高质量的代码生成提示词。PC 从多个维度评估提示质量,并结合开发者代码库与目标 LLM 的行为,给出有针对性的问题以促进自我修正。作者对 15 名职业开发者开展了早期实证研究,结合提示质量的量化打分与感知层面的质性测量。结果显示,仅经过一次 60 分钟的会话,参与者的提示质量即获得统计显著提升,且提升最大的正是开发者通常忽视的维度;同时他们报告了较高的信任度与采纳意愿,并一致认为 PC 改善了自己的提示词写作能力。

问题
提示工程是软件开发的重要技能却缺乏有效教学手段,其动态、交互、情境依赖的特性使传统学习方法难以支撑。
方法
构建嵌入 IDE 工作流的智能体导师 Prompt Coach,从多维度评估提示质量,并基于开发者代码库与目标 LLM 行为发出苏格拉底式提问以引导自我修正。
评估
对 15 名职业开发者进行单次 60 分钟会话的前后测实证研究,量化提示质量得分并结合质性感知测量(信任、采纳意愿、自评技能提升)。
原文摘要(Abstract)

Prompt engineering has emerged as a critical yet undertaught skill for software developers, one that traditional learning approaches are ill-equipped to support given its evolving, interactive, and context-dependent nature. In this paper, we introduce Prompt Coach (PC), an agentic tutor that helps developers learn how to craft high-quality code-generation prompts through Socratic guidance embedded in-flow within their IDE. PC evaluates prompt quality across multiple dimensions and surfaces targeted questions to guide self-correction, grounded in the developer's codebase and the behavior of the target LLM. We present an early empirical study with 15 professional developers combining quantitative prompt quality scoring with qualitative perception measures. Participants showed statistically significant improvements after a single 60-minute session, with the largest gains across dimensions commonly overlooked by developers. They also reported strong trust, high adoption readiness, and unanimous agreement that PC improved their prompt-writing skills.

链接与引用

DOI 原文 · arXiv · DBLP

BibTeX
@inproceedings{abs-2607-06074,
  title = {Prompt Coach: An Empirical Evaluation of an Agentic Tutor for Learning Prompt Engineering in Software Development},
  author = {Rohit Mehra and Kapil Singi and Vikrant Kaulgud and Vibhu Saujanya Sharma and Swapnajeet Gon Choudhury and Swati Sharma and Adam P. Burden and Majd Sakr},
  booktitle = {ASE},
  year = {2026},
  doi = {10.1145/3832783.3834478}
}