一句话:综述模型合并技术,并探讨其在软件工程尤其是自动程序修复中的应用与泛化性。
该技术简报聚焦模型合并(model merging)——一种无需或仅需少量额外训练即可集成语言模型、视觉模型等的技术,已被用于多任务学习、遗忘与知识迁移,并在 Open LLM 排行榜上得到验证,但在软件工程领域关注甚少。作者将介绍模型合并的核心概念与主要方法,梳理软件工程中为数不多的相关研究,并汇报其利用合并模型进行自动程序修复的实验,考察合并模型在该任务上的泛化能力。最后展望合并模型在软件工程研究中的未来方向与机会。简报面向学术界与工业界,无需特定背景即可理解。
Model merging, is a techniques that integrates AI-based models such as language or vision models using a variety of techniques, with or without additional training. Model merging has been used for multi-task learning, unlearning, and knowledge transfer. Several merged models on Open LLM leaderboards proves its success in various domains. However, less attention is paid to this technique in software engineering. In this technical briefing, we will discuss the main concepts and approaches for model merging, and then explore rare related studies in software engineering, including our experiments of using merged models for automatic program repair and generalizability of the merged model on this task. We will conclude by future directions and research opportunities of merged models in software engineering research. This briefing can attract audience from academia and industry and no specific background is needed.