一句话：提出碳税启发的 LLM 压缩流水线 CTT，集成 NAS、结构化剪枝、量化与蒸馏，大幅降内存、提速并减碳且保持精度。
LLM 在软件工程中的广泛使用带来不可持续的计算与环境成本。本文受经济碳税启发提出 Carbon-Taxed Transformers（CTT）压缩流水线，按原则顺序集成神经架构搜索、结构化剪枝、量化与知识蒸馏，并泛化到 encoder-only、encoder-decoder 与 decoder-only 架构。在代码克隆检测、摘要与生成任务上评估，CTT 实现最高 49× 内存降低、10× 加速与 81% CO2 减排，同时保留 98% 克隆检测、89% 摘要与最高 91% 生成精度；消融研究验证流水线顺序与各组件均必要。
The growing adoption of Large Language Models in software engineering has introduced unsustainable computational and environmental costs. We propose Carbon-Taxed Transformers (CTT), a compression pipeline inspired by economic carbon taxation, integrating Neural Architecture Search, structured pruning, quantization, and knowledge distillation in a principled sequence. CTT generalizes across encoder-only, encoder-decoder, and decoder-only architectures, and is evaluated on code clone detection, summarization, and generation. Results show up to 49× memory reduction, 10× speedup, and an 81% reduction in CO2 emissions, while retaining 98% accuracy in clone detection, 89% in summarization, and up to 91% in generation. Ablation studies confirm both the pipeline ordering and each component are essential.
@inproceedings{AlamRRRS26,
title = {Carbon-Taxed Transformers: Efficient, Accurate, and Sustainable LLMs},
author = {Ajmain Inqiad Alam and Palash Roy and Chanchal Roy and Banani Roy and Kevin Schneider},
booktitle = {FSE},
year = {2026},
doi = {10.1145/3803437.3807387}
}