paperbot · PL 论文追踪

全部论文 待读 收藏

P2 TOGA

TOGA:基于神经网络的测试预言生成方法
ICSE Proceedings of the 44th International Conference on Software Engineering2022匹配度 3/10 · 该论文属于机器学习辅助软件测试中的测试预言生成,与用户核心关注的静态分析、Agentic 程序、LLM/Agent 改进静态分析/形式化验证,以及浮点程序分析和贝叶斯程序分析均无直接关系。仅在‘机器学习应用于程序语言/软件工程’这一宽泛层面相关。引用 195 inbox
Elizabeth Dinella, Gabriel Ryan, Todd Mytkowicz, Shuvendu K. Lahiri
crossref: 2026-09-12dblp: 2026-09-12arxiv: 2026-09-12

AI 速览

一句话:提出基于 Transformer 的统一神经方法 TOGA,从焦点方法上下文推断异常与断言测试预言,精度提升 33%,发现 57 个真实缺陷。

软件测试中,测试预言(通常为条件表达式)记录了单元在给定测试前缀下应有的行为,但自动合成功能测试预言十分困难,因为它必须刻画开发者意图的功能而非已实现的功能。本文提出 TOGA(Test Oracle GenerAtion),一种基于 Transformer 的统一神经方法,依据焦点方法的上下文同时推断异常型与断言型测试预言。该方法能够处理文档含糊或缺失、甚至缺少实现的单元。作者从预言推断准确率与功能缺陷发现两方面进行评估:在留出测试数据集上总体准确率达 96%,较现有预言推断方法提升 33%;将其与自动化测试生成工具 EvoSuite 集成后,在大型 Java 程序中发现了 57 个真实缺陷,其中 30 个在评估中未被任何其他自动化测试方法发现。

问题
自动合成功能测试预言很困难,因为它需要捕获开发者意图的功能而非当前实现的行为,并且要能应对文档含糊、缺失甚至实现缺失的单元。
方法
提出 TOGA,一种基于 Transformer 的统一神经方法,根据焦点方法的上下文同时推断异常型与断言型测试预言。
评估
在预言推断准确率与功能缺陷发现两方面评估:留出测试集上总体准确率达 96%,较现有方法提升 33%;与 EvoSuite 集成后在大型 Java 程序中发现 57 个真实缺陷,其中 30 个未被其他自动化测试方法发现。
原文摘要(Abstract)

Testing is widely recognized as an important stage of the software development lifecycle. Effective software testing can provide benefits such as bug finding, preventing regressions, and documentation. In terms of documentation, unit tests express a unit's intended functionality, as conceived by the developer. A test oracle, typically expressed as an condition, documents the intended behavior of a unit under a given test prefix. Synthesizing a functional test oracle is a challenging problem, as it must capture the intended functionality rather than the implemented functionality. In this paper, we propose TOGA (a neural method for Test Oracle GenerAtion), a unified transformer-based neural approach to infer both exceptional and assertion test oracles based on the context of the focal method. Our approach can handle units with ambiguous or missing documentation, and even units with a missing implementation. We evaluate our approach on both oracle inference accuracy and functional bug-finding. Our technique improves accuracy by 33% over existing oracle inference approaches, achieving 96% over-all accuracy on a held out test dataset. Furthermore, we show that when integrated with a automated test generation tool (EvoSuite), our approach finds 57 real world bugs in large-scale Java programs, including 30 bugs that are not found by any other automated testing method in our evaluation.

链接与引用

DOI 原文 · arXiv · PDF(开放获取) · DBLP

BibTeX
@inproceedings{DinellaRML22,
  title = {TOGA},
  author = {Elizabeth Dinella and Gabriel Ryan and Todd Mytkowicz and Shuvendu K. Lahiri},
  booktitle = {ICSE},
  year = {2022},
  doi = {10.1145/3510003.3510141}
}