一句话：提出多模态 AI 智能体框架，无需代码即可模拟视障用户与 Web 界面的交互，以发现并修复无障碍问题。
静态代码分析难以发现残障用户面临的实时交互问题。本文提出一种多模态 AI 智能体框架，无需访问代码即可模拟视觉障碍用户与 Web 界面的交互。智能体使用视障用户可用的键盘导航和屏幕阅读器等模态，并通过模拟青光眼、近视等感知过滤器接收界面视觉信息，同时处理屏幕阅读器音频输出。其目标是复现残障用户在真实约束下的策略，从而更真实地评估 Web 无障碍性，定位并修复问题。作者还建议一个评估框架，用于衡量这些感知过滤器如何影响用户行为、任务成功率和 UI 可用性，旨在揭示受损感知下才显现的视觉无障碍缺陷。
Static code analysis cannot detect real-time interaction issues faced by users with disabilities. We propose a multi-modal Artificial Intelligence (AI) agent framework that simulates interactions of users with visual impairments without code access. The agent would closely simulate the experience of these users by interacting with web interfaces using the same modalities available to them, primarily keyboard navigation and screen readers. The agent perceives the interface through perceptual filters that mimic conditions including glaucoma and myopia, and handles both the altered visual input and audio output from screen readers. This approach aims to replicate the real-world constraints and strategies of users with disabilities, enabling more realistic evaluation, with the objective of identifying, locating and repairing web accessibility issues. We suggest a framework to evaluate how such filters affect user behavior, task success, and User Interface (UI) usability. Our approach aims to uncover visual accessibility flaws that become apparent under impaired perception.
DOI 原文 ·
@inproceedings{paperbot9289,
title = {Autonomous Agents for Accessibility: Simulating Visual Impairments in Web Interfaces},
author = {Juan Diego Yepes-Parra and Camilo Escobar-Velásquez},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00349}
}