一句话:利用LLM与结构化提示自动修复深度学习库测试脚本中的无效模型。
深度学习库(如TensorFlow、PyTorch)的测试脚本常因库版本演进或测试方法自身缺陷而包含无效模型,严重影响测试效率;由于DL模型结构复杂,人工修复这些模型十分困难。本文提出一种基于大语言模型(LLM)的专用提示(prompt)策略,通过向LLM提供结构化的错误信息与模型配置,引导其生成修复无效模型的代码。实验结果显示,该策略能成功修复大多数无效模型,并有助于发现DL库测试方法中的缺陷以及版本更新导致的问题,从而提升DL模型的鲁棒性和可迁移性。
Deep learning (DL) libraries such as TensorFlow and PyTorch are widely utilized to develop machine learning models. However, when testing DL libraries, evolving library versions and flaws of testing approaches may result in invalid models within test scripts, which influence testing efficiency severely. Repairing these models can be difficult to achieve manually because of the complexity of DL models. In this paper, we propose a novel approach that utilizes a specialized prompt-based strategy with Large Language Model (LLM) to repair invalid DL models. We manually provide structured error information and model configurations to LLM, allowing it to generate code to fix invalid models. Our work shows that most invalid models can be repaired successfully with our strategy. Moreover, our approach can help to detect flaws in the DL library testing approaches and issues caused by version updates, which enhances the robustness and transferability of DL models.