一句话:提出模块化LLM变异测试框架 multiplex,支持研究者实现、配置和比较LLM变异生成技术。
变异测试通过向被测程序注入合成缺陷来评估测试套件充分性;近期LLM可生成更真实、更难杀死的变异体,但现有研究原型多为单一用途,将变异生成技术与LLM提供商和构建环境紧耦合,导致研究者需重复实现才能进行实证比较。本文提出 multiplex,一个基于LLM的模块化变异框架。它允许研究者实现自定义LLM变异生成模块,以原型化或比较技术,并配置和创建针对语言、LLM或框架的模块。论文介绍系统设计与实现,展示如何用 multiplex 进行原型化和比较,并给出实验及未来扩展。工具已开源并提供视频演示。
Mutation testing introduces synthetic defects into a program under test to assess test suite adequacy. While recent advances in large language models (LLMs) have led to more realistic and hard-to-kill mutants, the field is currently dominated by single-purpose research prototypes that tightly couple the mutant generation technique with the LLM provider and build environment, thereby requiring mutation testing researchers to re-implement techniques for empirical comparison. To address this challenge, this paper introduces multiplex, a modular LLM-based mutation framework. multiplex enables researchers to implement their own LLM-based mutant generation (MG) modules to prototype or compare techniques, as well as configure and create language, LLM or framework-specific modules for extended study. In addition to providing an overview of the system design and implementation, this paper demonstrates how researchers can use multiplex to prototype and compare techniques, along with sharing experiments conducted with multiplex and outlining future extensions. The tool is available at https://github.com/LLM-Mutation/multiplex, and a video demonstration is available at https://youtu.be/l-hj7Pmn3oo.