一句话：通过 542 个任务×8 轮实验揭示多轮 LLM 编程对话中的回归累积现象，并提出验证门策略加以缓解。
在 LLM 辅助软件开发中，编码常是迭代的；多轮对话里，后轮代码建议可能破坏前轮已满足的需求，本文称之为回归累积。为评估这一现象，作者从 HumanEval+ 与 MBPP+ 构建 542 个任务，将每个任务实例化为 8 轮需求演化链，在 6 个 LLM 上共评估 26,016 个轮次实例。每轮均检查当前代码建议是否仍通过早期基准测试，并对 384 个失败案例进行四人独立标注，归纳出多轮回归缺陷分类法。结果显示，所有模型均出现回归累积，43% 至 73% 的任务在整段对话中丢失先前正确行为；末轮质量普遍低于首轮，尤其在加入输入验证或更宽输入类型的轮次。人工分析表明，跨轮冲突是主要失败类别。作者进一步发现，验证门（检查新代码是否通过先前测试并触发回滚与重试）是唯一能一致提升所有模型的策略，将 DeepSeek-V3 末轮质量从 75.8% 提升至 87.9%，Llama-3.1-8B 从 31.6% 提升至 47.3%。
In LLM-assisted software development, coding is often iterative. In this paper, single-turn refers to settings where the LLM responds in one request with one code suggestion, whereas multi-turn refers to settings where the LLM responds across multiple sequential requests within the same conversation. Regression accumulation refers to a later code suggestion breaking requirements introduced in earlier turns. Reliability depends not only on satisfying the current turn, but also on preserving requirements in earlier turns. Yet, there are limited evaluations of the accumulation of regression in multi-turn coding conversations. To address this gap, we conduct an empirical study of multi-turn regression in LLM programming. We construct 542 tasks from the HumanEval+ and MBPP+ benchmarks, and instantiate each task as an 8-turn requirement-evolution chain. We evaluate six LLM models on 26,016 turn instances (542×6×8). At each turn, we check if the current code suggestion still passes earlier benchmark tests, and we analyze 384 failure cases drawn from the failure population in our 26,016 turn instances to derive and validate a taxonomy of multi-turn regression bugs through independent four-annotator labeling. Our results show that regression accumulation appears across all six models, with 43% to 73% of tasks losing previously correct behavior over the full conversation. Final-turn quality is lower than initial-turn quality across models, and especially in turns adding input validation or broader input types. Manual analysis shows that Cross-Turn Conflict, where later code conflicts with earlier requirements, is the main failure class. We further find that Verification Gate, which checks new code against prior tests and triggers rollback and retry, is the only strategy that consistently improves all models, raising final-turn quality from 75.8% to 87.9% on DeepSeek-V3 and from 31.6% to 47.3% on Llama-3.1-8B. These findings suggest that strong performance on single-turn tasks can overestimate reliability in multi-turn coding conversations. Future evaluation and tool design should test whether later code suggestions preserve requirements from earlier turns and should include Verification Gate mechanisms.
@inproceedings{abs-2607-01855,
title = {Regression Accumulation in Multi-turn LLM Programming Conversations},
author = {Yonghui (Andie) Huang and Lin Ma and Amjed Tahir and Qian Zhang and Liwen Xiao and Lysa Xiao},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3834357}
}