一句话:提出 Etna,一个用于经验评估与比较基于属性的测试技术的可扩展平台,并在 Coq 与 Haskell 中开展实验。
基于属性的测试(PBT)在函数式编程中广泛应用,工具和生成策略繁多,但缺乏严谨的实证比较,用户难以选择。本文介绍 Etna:一个面向 PBT 技术的经验评估与比较平台,集成了多种流行 PBT 框架和文献中的测试负载,并具有可扩展架构,便于加入新框架与负载,同时自动处理性能测量等繁琐工作。作者用 Etna 在 Coq 和 Haskell 中对多种流行 PBT 方法进行实验,帮助用户理解最佳实践与权衡。
Property-based testing is a mainstay of functional programming, boasting a rich literature, an enthusiastic user community, and an abundance of tools — so many, indeed, that new users may have difficulty choosing. Moreover, any given framework may support a variety of strategies for generating test inputs; even experienced users may wonder which are better in a given situation. Sadly, the PBT literature, though long on creativity, is short on rigorous comparisons to help answer such questions. We present Etna, a platform for empirical evaluation and comparison of PBT techniques. Etna incorporates a number of popular PBT frameworks and testing workloads from the literature, and its extensible architecture makes adding new ones easy, while handling the technical drudgery of performance measurement. To illustrate its benefits, we use Etna to carry out several experiments with popular PBT approaches in both Coq and Haskell, allowing users to more clearly understand best practices and tradeoffs.
@article{paperbot1925,
title = {Etna: An Evaluation Platform for Property-Based Testing (Experience Report)},
author = {Jessica Shi and Alperen Keles and Harrison Goldstein and Benjamin C. Pierce and Leonidas Lampropoulos},
journal = {Proceedings of the ACM on Programming Languages},
volume = {7},
number = {ICFP},
year = {2023},
doi = {10.1145/3607860}
}