一句话:提出将 LLM 嵌入经验软件工程科学方法全流程的因果推理框架 ECLAIR,并以提示设计对代码生成准确率影响的案例研究验证。
经验软件工程长期遵循科学方法,但现代软件系统的复杂性使其难以系统化实施。本文提出 ECLAIR——一个以因果推理为基础的 AI 框架,将大语言模型(LLM)嵌入科学过程的每个阶段,从假设生成到分析与解释。ECLAIR 将 LLM 视为在因果推断原则下运作的主动科学代理,并采用人机回环(human-in-the-loop)设计以防范不可靠自动推理的风险。作者通过案例研究演示该框架:考察提示设计如何影响两个 LLM 的代码生成准确率。结果显示,对两个模型而言,指令式风格、更长的少样本示例以及签名增强提示均对准确率产生小幅负向因果效应,说明因果推理为解释软件现象为何发生提供了有原则的基础。该工作提出了首个为经验软件工程科学方法嵌入 LLM 的因果基础结构化方法。
The scientific method has long guided empirical research in Software Engineering (SE), but the complexity of modern software systems often hinders its systematic application. This paper introduces ECLAIR, a causally grounded AI framework that integrates Large Language Models (LLMs) into every stage of the scientific process, from hypothesis generation to analysis and interpretation. ECLAIR treats LLMs as active scientific agents operating under the principles of causal inference, within a human-in-the-loop design that safeguards against the risks of unsound automated reasoning. We demonstrate the framework through a case study examining how prompt design influences code generation accuracy in two LLMs. Results show that, for both models, instruction-style, longer few-shot, and signature-augmented prompts yield small negative causal effects on accuracy, illustrating how causal reasoning provides a principled foundation for explaining why software phenomena occur. This study presents the first causally grounded structured methodology for embedding LLMs within the scientific method in SE, designed around the epistemological demands of empirical SE research, establishing a basis for rigorous AI-assisted research.
@inproceedings{paperbot12431,
title = {ECLAIR: A Causally-Grounded AI Framework for Scientific Discovery in Empirical Software Engineering},
author = {Alejandro Velasco and Daniel Rodriguez-Cardenas and Dipin Khati and David N. Palacio and Denys Poshyvanyk},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3834544}
}