一句话:提出用非幂等结果测试提前预防 flaky 测试,避免其削弱回归测试的有效性。
回归测试能显著帮助软件开发,但 flaky 测试会在同一代码提交上看似非确定地通过或失败,从而严重削弱回归测试的有效性。已有工作识别了多类 flaky 测试、提出了检测技术,并分析了一些检测到的 flaky 测试。本文进一步提出通过 non-idempotent-outcome tests(非幂等结果测试)来预先阻止 flaky 测试:这类测试的结果在重复执行或不同执行条件下不保持幂等,可能因此暴露或触发潜在的不稳定行为。其目标是在 flaky 测试造成回归测试困扰之前进行干预。摘要未报告具体的实验设置与定量结果,但可看出该工作属于 flaky 测试的预防、识别与分析方向。
Regression testing can greatly help in software development, but it can be seriously undermined by flaky tests, which can both pass and fail, seemingly nondeterministically, on the same code commit. Flaky tests are an emerging topic in both research and industry. Prior work has identified multiple categories of flaky tests, developed techniques for detecting these flaky tests, and analyzed some detected flaky tests.