一句话:揭示现有 GoTu 估计器因覆盖率为松上界且忽略覆盖元素依赖而高估残余风险,并提出依赖感知的残余风险分析。
软件测试只能降低但不能消除缺陷,残余风险可定义为下一个测试输入发现缺陷的概率。该风险上界是发现概率(DP),即下一个测试输入覆盖新代码的概率,而 DP 又上界为覆盖率,即每个测试输入期望发现的新覆盖元素数。已有工作用 Good-Turing 估计器(GoTu)通过覆盖率估计残余风险。本文发现 GoTu 会显著高估残余风险,导致对缺陷发现的过度乐观,原因有二:覆盖率本身只是松上界,且 GoTu 忽略覆盖元素之间的依赖。为此,论文开展依赖感知的残余风险分析,旨在利用覆盖元素依赖获得更准确的残余风险估计。
However much we test a software system, some residual risk of undiscovered bugs always remains. If we model test generation as a sampling process, the residual risk can be defined as the probability that the next test input reveals a bug. This risk is upper-bounded by the discovery probability (DP), i.e., the probability that the next test input covers new code, which itself is upper-bounded by the coverage rate, i.e., the expected number of new coverage elements per test input. Prior work introduced the Good-Turing estimator (GoTu) to estimate residual risk via coverage rate. However, we find that GoTu substantially overestimates, leading to undue optimism in bug finding because (i) the coverage rate is only a loose upper bound, and (ii) GoTu ignores dependencies among coverage elements.