一句话：报告五个核电数值模拟程序的测试实践，发现 33 个缺陷，指出基于性质与蜕变测试最有效。
高性能数值模拟程序广泛用于在高性能计算机上模拟物理过程，以分析各类物理与工程问题，但因其高度复杂性常被视为不可测试。本文报告了作者对五个将用于核电站设计与分析的模拟程序进行测试的真实经验与教训。他们应用了五种测试方法，共发现 33 个缺陷，并发现基于性质的测试（property-based testing）与蜕变测试（metamorphic testing）是两种有效方法。然而，实践也暴露出缺乏领域知识、测试成本高、测试用例短缺、严重的 oracle 问题以及自动化支持不足等困难。结果这五个程序从软件测试角度看并未得到充分测试，许多现有测试技术与工具因可扩展性和可移植性问题而不完全适用。作者呼吁加强与其他社区的协作与交流，以推动软件测试技术的研究与应用。
High performance numerical simulation programs are widely used to simulate actual physical processes on high performance computers for the analysis of various physical and engineering problems. They are usually regarded as non-testable due to their high complexity. This paper reports our real experience and lessons learned from testing five simulation programs that will be used to design and analyze nuclear power plants. We applied five testing approaches and found 33 bugs. We found that property-based testing and metamorphic testing are two effective methods. Nevertheless, we suffered from the lack of domain knowledge, the high test costs, the shortage of test cases, severe oracle issues, and inadequate automation support. Consequently, the five programs are not exhaustively tested from the perspective of software testing, and many existing software testing techniques and tools are not fully applicable due to scalability and portability issues. We need more collaboration and communication with other communities to promote the research and application of software testing techniques.
DOI 原文 ·
@inproceedings{paperbot6043,
title = {Testing high performance numerical simulation programs: experience, lessons learned, and open issues},
author = {Xiao He and Xingwei Wang and Jia Shi and Yi Liu},
booktitle = {ISSTA},
year = {2020},
doi = {10.1145/3395363.3397382}
}