一句话:提出腐烂绿色测试问题,结合静态与动态调用点分析检测通过但断言未执行的测试,并实现 DrTest 工具。
单元测试中,绿色(通过)通常被视为被测代码正确的可靠标志;但有些绿色测试包含从未执行的断言,作者称之为“腐烂绿色测试”。这类测试比失败测试更危险:它们声称代码有效,实际并未检验有效性。论文提出结合简单静态与动态调用点分析来识别腐烂绿色测试,并考虑测试辅助方法、继承辅助方法与 trait 组合,实现了工具 DrTest。DrTest 无假阴性,但因条件使用或多测试上下文仍会产生少量假阳性。作者对 Pharo 生态成熟项目中的 19,905 个真实测试用例进行实证评估,检测出 294 个腐烂测试;其中一些已在 Pharo 中“沉睡”至少 5 年。
Unit tests are a tenant of agile programming methodologies, and are widely used to improve code quality and prevent code regression. A green (passing) test is usually taken as a robust sign that the code under test is valid. However, some green tests contain assertions that are never executed. We call such tests Rotten Green Tests. Rotten Green Tests represent a case worse than a broken test: they report that the code under test is valid, but in fact do not test that validity. We describe an approach to identify rotten green tests by combining simple static and dynamic call-site analyses. Our approach takes into account test helper methods, inherited helpers, and trait compositions, and has been implemented in a tool called DrTest. DrTest reports no false negatives, yet it still reports some false positives due to conditional use or multiple test contexts. Using DrTest we conducted an empirical evaluation of 19,905 real test cases in mature projects of the Pharo ecosystem. The results of the evaluation show that the tool is effective; it detected 294 tests as rotten—green tests that contain assertions that are not executed. Some rotten tests have been “sleeping” in Pharo for at least 5 years.