一句话：提出 RealityCraft，自动从自然语言指令合成 XR 多设备交互测试脚本，解决 XR 动作空间复杂与语义鸿沟问题。
扩展现实（XR）应用市场在 2024 年已超 1650 亿美元，但其自动化测试面临独特挑战。XR 应用通过多设备协同产生具有时间精度的复杂动作空间，且自然语言指令与具体 XR 交互之间存在巨大语义鸿沟，导致面向 Android 等二维应用的测试脚本生成方法难以直接迁移。为此，本文提出 RealityCraft，旨在从自然语言指令自动合成 XR 设备交互脚本，弥合指令语义与多设备时序动作之间的差距。摘要未报告具体评估结果，但问题定义清晰，为 XR 测试自动化提供了新方向。
Extended Reality (XR) apps have become increasingly popular across different domains, evidenced by its market growing to over $165 billion in 2024. Despite their widespread usage, XR apps present new challenges for automated software testing approaches, especially for testing script generation. XR apps have complicated action space from multi-device coordination with temporal precision. Besides, natural language instructions in XR apps usually have a large semantic gap with specific XR interactions. Therefore, previous testing script generation approaches on 2D apps such as Android can hardly be adopted into XR apps.
@inproceedings{paperbot11497,
title = {RealityCraft: Automated Synthesis of Extended Reality Device Interaction Scripts from Natural Language Instructions},
author = {Shuqing Li and Yun Peng and Yige Chen and Dingyuan Xue and Yepang Liu and Michael R. Lyu},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3744916.3764549}
}