一句话：提出基于MDP的蜕变测试工具MDPMorph，可自动检测深度强化学习智能体中的故障。
本文针对深度强化学习（DRL）智能体难以自动检测故障的问题，提出蜕变测试工具MDPMorph。该工具以马尔可夫决策过程（MDP）为基础，面向DRL智能体的核心推理属性，能够从被测系统自动生成蜕变测试套件和相应变异体。MDPMorph利用测试子集和模型，通过随机梯度下降训练九条基于MDP时间特性的蜕变关系（MR）的阈值，以确定每条MR的最优阈值；随后用MR比较不同蜕变测试套件在被测模型上的执行结果，判定测试通过或失败。最后汇总执行结果计算MR的变异体检测率以验证有效性。实验表明，MDPMorph及所提MR能高效自动检测种子故障（变异体）。
We present MDPMorph, a tool for metamorphic testing of Deep Reinforcement Learning (DRL) agents. MDPMorph is based on the Markov Decision Process (MDP) and targets the core reasoning properties of DRL agents to automatically uncover potential faults. It can generate metamorphic test suites and corresponding mutants directly from the DRL system under test. MDPMorph uses a subset of the metamorphic test suite and models to train the thresholds of the nine proposed Metamorphic Relations (MRs) using stochastic gradient descent. These MRs are based on the temporal characteristics of the MDP, and the training aims to determine the optimal threshold for each MR. After obtaining the optimal threshold, MDPMorph leverages the MRs to compare the execution results of different metamorphic test suites on the model under test and reports whether each test passes or fails. Finally, by collecting the execution results, MDPMorph calculates the mutant detection rate of MR to validate its effectiveness. Experimental results show that MDPMorph and the proposed MRs are highly effective in automatically detecting seeded faults (mutants).
DOI 原文 ·
@inproceedings{paperbot9429,
title = {Metamorphic Testing of Deep Reinforcement Learning Agents with MDPMorph},
author = {Jiapeng Li and Zheng Zheng and Yuning Xing and Daixu Ren and Steven Cho and Valerio Terragni},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00381}
}