一句话：提出人在回路的测试预言学习，通过融入人工反馈以更低标注成本获得能区分机器人行为优劣的仿真测试预言。
机器人系统在高风险环境（如自动驾驶）中需要严格的安全与行为保证。现场测试昂贵且危险，基于仿真的测试可更安全、低成本地自动生成轨迹，但需要测试预言（oracle）来评估轨迹是否满足任务完成、安全、效率与可靠性等准则。监督学习训练预言准确却依赖昂贵耗时的人工标注，无监督学习无需标注但精度不足。为此，本文提出人在回路（human-in-the-loop）的测试预言学习：将人工反馈融入预言的学习与精炼过程，以更少人工投入获得能区分好坏行为的预言，并给出将该方法集成到机器人系统评估中的概念框架。
Ensuring safety and providing rigorous behavioral guarantees are critical for robotic systems operating in high-stakes environments such as autonomous driving. Field testing is common, but costly and risky. Simulation-based testing offers a safer and lower-cost alternative for automatically generating traces for analysis and performance assessment. An oracle is essential for evaluating each trace, assessing whether a robot behavior fulfills key criteria such as task completion, safety, efficiency, and reliability. Supervised learning for oracle learning is accurate but costly and time-consuming due to manual labeling, whereas unsupervised learning requires no labels but often sacrifices accuracy. To overcome these limitations, we propose human-in-the-loop oracle learning as a new approach to develop and refine oracles that are capable of distinguishing good from bad behaviors with reduced manual effort. We illustrate this approach through a conceptual framework for integrating human-in-the-loop learning into robotic system evaluation.
DOI 原文 ·
@inproceedings{paperbot9564,
title = {Human-In-The-Loop Oracle Learning for Simulation-Based Testing},
author = {Ben-Hau Chia and Eunsuk Kang and Christopher S. Timperley},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00341}
}