一句话:提出行为空间几何测试方法,通过估计 ML 组件内部行为相似性结构来定义覆盖率并指导演化后的补充测试。
在安全关键 ML 使能系统中,ML 组件内部逻辑不透明,且会因重训练或架构修改而持续演化,导致既有测试结果失效,仅靠黑盒测试不足以捕捉内部逻辑变化。论文提出行为空间测试:将 ML 组件在数据间形成的内部行为相似性表示为抽象几何空间,结合学习过程信号估计该空间几何,并据此定义测试覆盖率。当 ML 演化改变行为空间时,通过估计几何变化识别现有测试套件的覆盖缺口区域,并选择补充测试集。工作尚处方法构想阶段,未报告具体实证结果。
In safety-critical machine learning (ML)-enabled systems, the internal logic of ML component is inherently opaque, and keep evolving by retraining, or architectural modification. While these changes undermine the validity of existing test results, black-box testing alone is insufficient to capture these shifts in internal logic. We propose a novel testing approach that estimates the behavior-space geometry, an abstract geometric space representing the internal behavioral similarity that the ML component forms among data. We seek to estimate behavior-space by combining signals from the ML comopnent’s learning process, and define test coverage based on the geometry. When the behavior-space is altered by ML evolution, we plan to estimate geometric changes to detect regions of coverage gaps in the existing test suite and select supplementary test suite.