一句话:提出评估自动化智能工具链整体正确性、不确定性与误差传播的方法与自动化工具。
自动化智能工具链由使用 AI 或静态分析的不同工具组合而成,广泛用于自动程序修复和软件安全漏洞识别。现有研究多只评估链中单个组件并报告不确定性,缺乏对整体工具链不确定性及误差传播的度量。本文计划复现已发表案例研究以收集数据,设计重构工具链整体正确性指标或识别缺失变量的方法,并开展人类确认性实验,最后实现自动化评估工具链的制品。初步验证显示有希望的结果。
[Background:] Automated Intelligent Toolchains, which are a composition of different tools that use AI or static analysis, are widely used in software engineering to deploy automated program repair techniques, or in software security to identify vulnerabilities. [Overall Research Problem:] Most studies with automated intelligent toolchains report uncertainty and evaluations only of the individual components of the chain. How do we calculate the uncertainty and error propagation on the overall automated toolchain? [Approach:] I plan to replicate research case studies to collect data and design a methodology to reconstruct the overall correctness metrics of the toolchains, or identifying missing variables. Further confirmatory experiments with humans will be performed. Finally, I will implement an artifact to automate the overall assessment of automated toolchains. [Current Status:] A preliminary validation of published studies showed promising results.