一句话:在 DS-1000 与 SWE-bench Lite 上比较三级提示结构,发现结构只提升可执行性与格式合规,未必提升任务正确性。
结构化提示常能提升通用 LLM 推理,但其能否迁移到代码生成仍不清楚,而编码工具又在用户请求外添加模板层。本文在 DS-1000 和 SWE-bench Lite 上比较三级自动施加的提示结构:自由形式指令、静态组件模板,以及提取并强制显式输出契约的流水线。结果显示:中等结构使代码更易执行,但并未解决更多任务;严格契约强制提升格式合规性,却显著降低功能正确性,且效果依赖底层模型。作者据此认为,对代码生成而言提示结构是调参旋钮而非最佳实践,输出契约更适合放在验证而非生成阶段。
Structured prompting routinely improves general LLM reasoning, but whether it transfers to code generation remains underexplored even as coding tools add template layers around user requests. We compare three levels of automatically imposed prompt structure, from free-form instructions over a static component template to a pipeline that extracts and enforces an explicit output contract, on DS-1000 and SWE-bench Lite. We find that moderate structure makes code more executable but does not resolve more tasks, strict contract enforcement improves format compliance while substantially reducing functional correctness, and the effect depends on the underlying model. For code generation, prompt structure is a tuning knob rather than a best practice, and output contracts belong in validation rather than generation.