一句话：提出 FAST，通过引导式特征选择剪枝噪声特征来增强现有基于不确定性的 DNN 测试优先级排序，缓解高置信度错误。
DNN 测试空间巨大，测试用例优先级排序对高效测试至关重要。现有基于不确定性的排序方法难以暴露模型对错误预测仍给出高置信度的“过度自信”问题。FAST 的核心洞察是部分特征会引入噪声并影响模型输出置信度，从而导致高置信度错误。该方法量化每个特征对模型正确预测的重要性，并在推理时动态剪除噪声特征的信息，得到新的概率向量用于不确定性估计，使高置信度错误与正确分类样本更可区分。实验在多种模型结构和多个基准数据集上，与当前先进优先级排序技术进行比较，验证了 FAST 在有效性、效率和可扩展性方面的优势，并提升了 APFD 与模型增强的泛化能力。
Due to the vast testing space, the increasing demand for effective and efficient testing of deep neural networks (DNNs) has led to the development of various DNN test case prioritization techniques. However, the fact that DNNs can deliver high-confidence predictions for incorrectly predicted examples, known as the over-confidence problem, causes these methods to fail to reveal high-confidence errors. To address this limitation, in this work, we propose FAST, a method that boosts existing prioritization methods through guided FeAture SelecTion. FAST is based on the insight that certain features may introduce noise that affects the model's output confidence, thereby contributing to high-confidence errors. It quantifies the importance of each feature for the model's correct predictions, and then dynamically prunes the information from the noisy features during inference to derive a new probability vector for the uncertainty estimation. With the help of FAST, the high-confidence errors and correctly classified examples become more distinguishable, resulting in higher APFD (Average Percentage of Fault Detection) values for test prioritization, and higher generalization ability for model enhancement. We conduct extensive experiments to evaluate FAST across a diverse set of model structures on multiple benchmark datasets to validate the effectiveness, efficiency, and scalability of FAST compared to the state-of-the-art prioritization techniques.
@inproceedings{paperbot8388,
title = {FAST: Boosting Uncertainty-based Test Prioritization Methods for Neural Networks via Feature Selection},
author = {Jialuo Chen and Jingyi Wang and Xiyue Zhang and Youcheng Sun and Marta Kwiatkowska and Jiming Chen and Peng Cheng},
booktitle = {ASE},
year = {2024},
doi = {10.1145/3691620.3695472}
}