paperbot · PL 论文追踪

全部论文 待读 收藏

P1 Assessment of Automated (Intelligent) Toolchains

自动化(智能)工具链的评估
ASE Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering2022匹配度 5/10 · 论文关注由 AI/静态分析工具组成的自动化工具链,评估其整体正确性、不确定性与误差传播,属于机器学习与程序分析交叉的评测方法,与用户对程序分析及 AI+PL 交叉的兴趣部分相关;但未涉及 LLM/Agentic 程序、浮点程序分析或贝叶斯程序分析等核心方向,匹配度中等。引用 0 inbox
Aurora Papotti
crossref: 2026-09-12

AI 速览

一句话:提出评估自动化智能工具链整体正确性、不确定性与误差传播的方法与自动化工具。

自动化智能工具链由使用 AI 或静态分析的不同工具组合而成,广泛用于自动程序修复和软件安全漏洞识别。现有研究多只评估链中单个组件并报告不确定性,缺乏对整体工具链不确定性及误差传播的度量。本文计划复现已发表案例研究以收集数据,设计重构工具链整体正确性指标或识别缺失变量的方法,并开展人类确认性实验,最后实现自动化评估工具链的制品。初步验证显示有希望的结果。

问题
多数自动化智能工具链研究只评估单个组件并报告不确定性,如何计算整体工具链上的不确定性与误差传播尚不明确。
方法
计划复现案例研究收集数据,设计重构工具链整体正确性指标或识别缺失变量的方法,开展人类确认性实验,并实现自动化评估工具。
评估
对已发表研究的初步验证显示出有希望的结果;后续将用人类确认性实验和自动化制品进一步评估该方法。
原文摘要(Abstract)

[Background:] Automated Intelligent Toolchains, which are a composition of different tools that use AI or static analysis, are widely used in software engineering to deploy automated program repair techniques, or in software security to identify vulnerabilities. [Overall Research Problem:] Most studies with automated intelligent toolchains report uncertainty and evaluations only of the individual components of the chain. How do we calculate the uncertainty and error propagation on the overall automated toolchain? [Approach:] I plan to replicate research case studies to collect data and design a methodology to reconstruct the overall correctness metrics of the toolchains, or identifying missing variables. Further confirmatory experiments with humans will be performed. Finally, I will implement an artifact to automate the overall assessment of automated toolchains. [Current Status:] A preliminary validation of published studies showed promising results.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot7193,
  title = {Assessment of Automated (Intelligent) Toolchains},
  author = {Aurora Papotti},
  booktitle = {ASE},
  year = {2022},
  doi = {10.1145/3551349.3559572}
}