paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Behavior-Driven Development Meets LLMs: Do We Need Them, and Do They Work?

行为驱动开发遇上大语言模型:我们需要它们吗,它们管用吗?
ICSE Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering2026匹配度 3/10 · 该论文将 LLM 用于从 Gherkin 步骤生成 Java 胶水代码,属于测试代码生成与实证软件工程,与用户关注的静态分析、形式化验证、浮点/贝叶斯程序分析及其与 LLM 的交叉均无直接关系;虽然问题驱动的实证研究和工程落地视角与用户偏好部分契合,但研究核心是 LLM 代码生成本身,而非用 LLM/Agent 改进分析或验证。引用 1 inbox
Xinyu Shi
crossref: 2026-09-12

AI 速览

一句话:通过11个Java项目的实证研究揭示BDD胶水代码的维护负担,并提出首个从Gherkin步骤生成Java胶水代码的检索增强少样本方法AutoGlue。

行为驱动开发(BDD)通过人类可读的场景连接需求与测试代码,但编写和维护BDD场景及其step definition(胶水代码)耗时且困难。本文对11个开源Java项目开展实证研究,发现BDD制品快速膨胀且频繁修改:项目平均包含4835次测试步骤调用,其中65.1%在初次创建后被更新,由于每个测试步骤与其胶水代码紧耦合,开发人员被迫频繁同步修改。为缓解这一维护负担,作者提出AutoGlue——一个检索增强的少样本提示框架,据称是首个直接从Gherkin步骤自动生成Java胶水代码的方法。在1-shot设置下,AutoGlue取得61.6的CodeBLEU分数,表明生成的胶水代码与人类编写代码高度相似;初步结果还显示,引入项目特定上下文信息可显著提升生成质量,CodeBLEU最多提高41%。

问题
BDD场景与step definition(胶水代码)紧耦合且演化频繁,导致其编写与维护成本高,现有工作缺乏对BDD维护负担的量化理解,也缺少自动生成胶水代码的支持。
方法
对11个开源Java项目进行实证研究,量化BDD制品规模、修改频率与维护负担;进而提出AutoGlue,一个检索增强的少样本提示框架,可直接从Gherkin步骤生成Java胶水代码。
评估
在1-shot设置下评估AutoGlue生成的胶水代码与人工代码的相似度,使用CodeBLEU指标,并分析引入项目特定上下文信息对生成质量的提升效果。
原文摘要(Abstract)

Behavior-Driven Development (BDD) bridges requirements and test code through human-readable scenarios. However, writing and maintaining BDD scenarios and step definitions (glue code) can be challenging and time-consuming. To understand whether large language models (LLMs) can effectively support BDD, we conduct an empirical study on 11 open-source Java projects. We observe that BDD artifacts expand quickly and undergo frequent modifications. Because each test step is tightly coupled to its corresponding glue code, this continuous evolution makes maintenance labor-intensive. In particular, the studied projects contain an average of 4,835 test step invocations, and 65.1% of them are updated after their initial creation, which forces developers to frequently update the corresponding step definitions. Motivated by this maintenance burden, we develop AutoGlue, a retrieval-augmented few-shot prompting framework for glue code generation. To the best of our knowledge, AutoGlue is the first approach that automatically generates Java glue code directly from Gherkin steps. Our investigation shows that, in the 1-shot setting, AutoGlue achieves a CodeBLEU score of 61.6, which indicates that the generated glue code is highly similar to human-written code. Moreover, our preliminary results reveal that incorporating project-specific contextual information significantly boosts generation quality, improving the CodeBLEU score by up to 41%.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot11688,
  title = {Behavior-Driven Development Meets LLMs: Do We Need Them, and Do They Work?},
  author = {Xinyu Shi},
  booktitle = {ICSE},
  year = {2026},
  doi = {10.1145/3774748.3787745}
}