一句话：提出首个面向Web无障碍测试工具的变异分析框架Ma11y，包含25个变异算子与自动判定oracle。
尽管已有大量自动无障碍测试方案，Web无障碍问题仍普遍存在，且缺乏对现有测试工具有效性的系统评估。为此，作者提出首个变异分析框架Ma11y，用于评估Web无障碍测试工具。Ma11y定义了25个变异算子，刻意违反各类无障碍原则，并配备自动化oracle判定测试工具能否检测出变异体。作者在真实网站上评估了变异算子的实用性及框架评估工具性能的能力。结果表明，当前工具无法识别该框架注入的近50%无障碍缺陷，说明亟需更有效的无障碍测试工具；同时框架的准确性与性能也表明其可在实际场景中无缝自动化应用。
Despite the availability of numerous automatic accessibility testing solutions, web accessibility issues persist on many websites. Moreover, there is a lack of systematic evaluations of the efficacy of current accessibility testing tools. To address this gap, we present the first mutation analysis framework, called Ma11y, designed to assess web accessibility testing tools. Ma11y includes 25 mutation operators that intentionally violate various accessibility principles and an automated oracle to determine whether a mutant is detected by a testing tool. Evaluation on real-world websites demonstrates the practical applicability of the mutation operators and the framework’s capacity to assess tool performance. Our results demonstrate that the current tools cannot identify nearly 50% of the accessibility bugs injected by our framework, thus underscoring the need for the development of more effective accessibility testing tools. Finally, the framework’s accuracy and performance attest to its potential for seamless and automated application in practical settings.
@inproceedings{paperbot8560,
title = {Ma11y: A Mutation Framework for Web Accessibility Testing},
author = {Mahan Tafreshipour and Anmol Deshpande and Forough Mehralian and Iftekhar Ahmed and Sam Malek},
booktitle = {ISSTA},
year = {2024},
doi = {10.1145/3650212.3652113}
}