paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Automatically Generating Precise Oracles from Structured Natural Language Specifications

从结构化自然语言规范自动生成精确的测试预言
ICSE 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE)2019匹配度 2/10 · 该论文属于软件测试与规范自动化,聚焦从结构化自然语言规范中提取测试预言并生成测试,未涉及静态分析、LLM/Agent 改进程序分析、浮点程序分析或贝叶斯程序分析等核心方向;仅在“规范驱动的验证/测试”邻域有弱关联。引用 51 inbox
Manish Motwani, Yuriy Brun
crossref: 2026-09-12dblp: 2026-09-12

AI 速览

一句话:提出 Swami,从结构化自然语言规范中提取测试预言并生成可执行测试,聚焦异常行为与边界条件。

软件规范常用自然语言描述期望行为,但这类规范难以被自动验证。本文提出 Swami,一种从结构化自然语言规范中提取测试预言(oracle)并生成可执行测试的自动化技术。Swami 重点关注异常行为与边界条件——这些情况常导致现场故障,却往往被开发者忽略而未手动编写测试。作者在官方 JavaScript 规范 ECMA-262 上评估 Swami,生成的测试有 98.4% 与规范精确一致。用 Swami 增强开发者编写的测试套件后,覆盖率得到提升,并在 Rhino 中发现 1 个此前未知缺陷和 15 个缺失的 JavaScript 特性,在 Node.js 中发现 1 个此前未知缺陷,还在 ECMA-262 规范中发现 18 处语义歧义。

问题
自然语言规范难以自动验证,且开发者常遗漏异常行为与边界条件的测试,导致现场故障。
方法
Swami 从结构化自然语言规范中自动提取测试预言并生成可执行测试,聚焦异常行为与边界条件。
评估
在 ECMA-262 上评估,98.4% 生成测试与规范精确一致;增强测试套件后在 Rhino、Node.js 中发现未知缺陷,并揭示规范歧义。
原文摘要(Abstract)

Software specifications often use natural language to describe the desired behavior, but such specifications are difficult to verify automatically. We present Swami, an automated technique that extracts test oracles and generates executable tests from structured natural language specifications. Swami focuses on exceptional behavior and boundary conditions that often cause field failures but that developers often fail to manually write tests for. Evaluated on the official JavaScript specification (ECMA-262), 98.4% of the tests Swami generated were precise to the specification. Using Swami to augment developer-written test suites improved coverage and identified 1 previously unknown defect and 15 missing JavaScript features in Rhino, 1 previously unknown defect in Node.js, and 18 semantic ambiguities in the ECMA-262 specification.

链接与引用

DOI 原文 · DBLP

BibTeX
@inproceedings{MotwaniB19,
  title = {Automatically Generating Precise Oracles from Structured Natural Language Specifications},
  author = {Manish Motwani and Yuriy Brun},
  booktitle = {ICSE},
  year = {2019},
  doi = {10.1109/icse.2019.00035}
}