paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Metamorphic Testing for Infrastructure-as-Code Engines

面向基础设施即代码引擎的蜕变测试
OOPSLA 10(OOPSLA1)2026匹配度 3/10 · 论文属于软件测试/IaC 引擎测试方向,使用 e-graph 生成等价程序并检查蜕变关系,与用户关注的静态分析、LLM/Agent、浮点程序分析或贝叶斯程序分析没有直接交集;仅在 PL/工程实现与工业落地层面边缘相关。引用 0 inbox
David Spielmann, George Zakhour, Dominik Arnold, Matteo Biagiola, Roland Meier, Guido Salvaneschi
crossref: 2026-09-10

AI 速览

一句话:提出 EMIaC,用 e-graph 生成等价 IaC 程序并检查蜕变关系,以测试 Terraform、Pulumi 等 IaC 引擎。

针对 IaC 引擎可靠性测试不足的问题,现有测试依赖手工预言且难以暴露跨多次执行的故障。EMIaC 将蜕变关系定义为 IaC 程序的图变换,通过在原始与变换程序执行间检查不变量来测试引擎。其核心创新在于把 e-graph 同时用作测试输入生成器和等价预言:e-graph 紧凑表示程序等价性,可系统生成大量等价 IaC 程序;对 43,593 个真实 Terraform 程序的分析表明依赖图通常小而稀疏,适合 e-graph。在 Pulumi、Terraform 和 OpenTofu 上评估,EMIaC 覆盖了 Terraform 的 98 条、Pulumi 的 1,313 条此前未测试语句,发现三个测试套件中未知问题,并有测试用例被合并。

问题
IaC 引擎(如 Terraform、Pulumi)是现代云基础设施的关键,但其自身正确性测试不足;现有测试依赖手工预言,难以发现跨多次执行才显现的故障。
方法
提出蜕变测试框架 EMIaC,将蜕变关系建模为 IaC 程序的图变换,并创新性地用 e-graph 作为测试输入生成器与等价预言,系统生成并校验等价 IaC 程序。
评估
分析 43,593 个真实 Terraform 程序以论证 e-graph 适用性,并在 Pulumi、Terraform、OpenTofu 上评估;结果显示覆盖 98/1,313 条此前未测试语句,发现未知问题,部分测试用例被上游合并。
原文摘要(Abstract)

Infrastructure-as-Code (IaC) engines, such as Terraform, OpenTofu, and Pulumi, automate the provisioning and management of cloud resources. They parse IaC specifications and orchestrate the required actions, making them the backbone of modern clouds, and critical to the reliability of both the underlying infrastructure and the software that depends on it. Despite this importance, this class of systems has received little attention: prior work largely targets the correctness of IaC programs rather than the IaC engines themselves. Existing test suites rely on manually written oracles and struggle to expose faults that manifest across multiple executions, leaving a significant reliability gap. We present EMI a C, a metamorphic testing framework for IaC engines. EMI a C defines metamorphic relations as graph-based transformations of IaC programs and checks invariants across executions of the original and transformed programs. A central novelty is our use of e-graphs in software testing, as both a test-input generator and an equivalence oracle. E-graphs compactly represent program equivalences, enabling the systematic generation of large spaces of equivalent IaC programs. To ground these relations, we analyze 43,593 real-world Terraform programs and show that IaC dependency graphs are typically small and sparse, making e-graphs a natural fit. Evaluating EMI a C on Pulumi, Terraform, and OpenTofu, we show that it complements existing test suites by exercising engine-critical code paths and covering 98 previously untested statements in Terraform and 1,313 in Pulumi. EMI a C also uncovers previously unknown issues in all three test suites, improving their adequacy. Three test cases have been merged into Terraform’s main branch, and Pulumi has merged a specification fix.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@article{paperbot3784,
  title = {Metamorphic Testing for Infrastructure-as-Code Engines},
  author = {David Spielmann and George Zakhour and Dominik Arnold and Matteo Biagiola and Roland Meier and Guido Salvaneschi},
  journal = {Proceedings of the ACM on Programming Languages},
  volume = {10},
  number = {OOPSLA1},
  year = {2026},
  doi = {10.1145/3798226}
}