一句话：实证研究表明，代码感知的量化配方能让 60 亿参数代码生成模型在普通笔记本上运行，且精度与鲁棒性无明显下降。
ML 驱动的代码生成借助大规模预训练模型显著提升了开发者效率，但庞大的参数量给普通笔记本或中型服务器上的部署带来了内存、延迟、成本与碳足迹方面的沉重负担。本文提出用模型压缩中的量化技术来应对这一挑战，因为量化无需昂贵的重训练，仅把模型参数量化为低比特整数（如 int8）即可同时缩减模型体积与运行时延迟。作者在代码生成任务上从三个维度系统评估量化模型：资源占用与碳足迹、准确率、鲁棒性。通过系统性实验，他们总结出一套代码感知（code-aware）的量化配方，使 60 亿参数的模型也能在普通笔记本上运行而不显著损失准确率与鲁棒性，并发现该配方可直接迁移到代码摘要任务。
ML-powered code generation aims to assist developers to write code in a more productive manner by intelligently generating code blocks based on natural language prompts. Recently, large pretrained deep learning models have pushed the boundary of code generation and achieved impressive performance. However, the huge number of model parameters poses a significant challenge to their adoption in a typical software development environment, where a developer might use a standard laptop or mid-size server to develop code. Such large models cost significant resources in terms of memory, latency, dollars, as well as carbon footprint. Model compression is a promising approach to address these challenges. We have identified quantization as one of the most promising compression techniques for code-generation as it avoids expensive retraining costs. As quantization represents model parameters with lower-bit integer (e.g., int8), the model size and runtime latency would both benefit. We empirically evaluate quantized models on code generation tasks across different dimensions: (i) resource usage and carbon footprint, (ii) accuracy, and (iii) robustness. Through systematic experiments we find a code-aware quantization recipe that could run even a 6-billion-parameter model in a regular laptop without significant accuracy or robustness degradation. We find that the recipe is readily applicable to code summarization task as well.
@inproceedings{WeiG0ARQLKWTSAS23,
title = {Towards Greener Yet Powerful Code Generation via Quantization: An Empirical Study},
author = {Xiaokai Wei and Sujan Kumar Gonugondla and Shiqi Wang and Wasi Ahmad and Baishakhi Ray and Haifeng Qian and Xiaopeng Li and Varun Kumar and Zijian Wang and Yuchen Tian and Qing Sun and Ben Athiwaratkun and Mingyue Shang and Murali Krishna Ramanathan and Parminder Bhatia and Bing Xiang},
booktitle = {FSE},
year = {2023},
doi = {10.1145/3611643.3616302}
}