一句话:通过降低能耗报告门槛、自动化模型压缩并给出选型指南,让 LLM 优化技术更易用、更节能。
大语言模型(LLM)在各类软件工程(SE)任务中展现出很高的准确率,但其高昂的计算需求与能耗带来了可持续性问题,也阻碍了它们在消费级硬件和资源受限平台上的部署。目前已存在多种优化技术,但由于在模型开发过程中应用这些技术存在较高技术难度,它们往往被开发者忽视。本项研究旨在提升优化技术的可及性,具体包括四个方面:(1) 让 LLM 的能耗报告更易获取;(2) 将优化技术流程化并自动化;(3) 为不同用例提供选择合适技术的指南;(4) 利用这些技术为 LLM 赋能的应用设计高效的 SLM 集成架构。预期贡献包括能耗测量与报告方法、自动压缩模型的工具、指导开发者完成优化流程的指南,以及基于上述结果探索压缩后 LLM 的替代部署方案。
Large Language Models (LLMs) are increasingly being applied to Software Engineering (SE) tasks, showing high accuracy in various problems. However, their high computational demands and energy consumption raise sustainability concerns and hinder their use on consumer hardware and resource-constrained platforms. Multiple optimization techniques exist, but they are often neglected due to the technical difficulty of applying them during model development. This research aims to improve the accessibility of optimization techniques by (1) making energy reporting of LLM more accessible, (2) streamlining and automating optimization techniques, (3) providing guidelines to select appropriate techniques for different use cases, and (4) exploiting these techniques to design efficient SLM ensemble architectures for LLM-enabled applications. Expected contributions include methods for measuring and reporting energy usage, tools to automate compression of models, guidelines to guide developers through the optimization process, and using these results to explore alternative deployment setups for compressed LLMs.