一句话：探索以中型骨干模型切片商业黑盒大模型，提取其专门代码能力是否可行。
大语言模型在软件工程中应用广泛，但训练高性能模型需要大量数据收集与标注人力，训练数据可能专有或仅部分开放，且依赖昂贵 GPU 集群。商业大模型具有很高知识产权价值，容易成为模仿攻击目标，但复现同等参数规模的模仿模型成本仍然很高。该论文提出一个实用且新颖的方向：用中型骨干模型对商业黑盒大语言模型进行切片，以提取其专门的代码能力。论文以可行性研究形式探讨这一思路，关注是否能在不承担从头训练或大规模同参模仿成本的情况下获得特定代码能力。摘要未报告具体实验设置、评价指标或量化结果。
Recent advances in large language models (LLMs) significantly boost their usage in software engineering. However, training a well-performing LLM demands a substantial workforce for data collection and annotation. Moreover, training datasets may be proprietary or partially open, and the process often requires a costly GPU cluster. The intellectual property value of commercial LLMs makes them attractive targets for imitation attacks, but creating an imitation model with comparable parameters still incurs high costs. This motivates us to explore a practical and novel direction: slicing commercial black-box LLMs using medium-sized backbone models.
@inproceedings{paperbot11434,
title = {On Extracting Specialized Code Abilities from Large Language Models: A Feasibility Study},
author = {Zongjie Li and Chaozheng Wang and Pingchuan Ma and Chaowei Liu and Shuai Wang and Daoyuan Wu and Cuiyun Gao and Yang Liu},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3597503.3639091}
}