一句话：提出采集IDE匿名补全日志，训练CatBoost排序模型改进代码补全，并在A/B测试中将完成补全所需输入动作从2.073降至1.832。
本文关注IDE中代码补全候选排序问题，旨在利用真实用户行为日志提升补全体验。作者提出从IntelliJ系IDE用户处匿名采集补全使用日志，设计描述补全候选及其上下文的特征，并在Early Access Program中部署日志收集。基于收集到的用户补全数据，训练CatBoost排序模型，对补全候选进行排序。评估分两部分：一是在留出的补全数据集上测试，二是在IDE中对两组用户进行独立A/B测试。结果表明，使用过去用户行为日志训练出的简单排序模型显著优于默认启发式排序，完成一次补全所需的输入动作数从2.073降低到1.832。
In this work, we propose an approach for collecting completion usage logs from the users in an IDE and using them to train a machine learning based model for ranking completion candidates. We developed a set of features that describe completion candidates and their context, and deployed their anonymized collection in the Early Access Program of IntelliJ-based IDEs. We used the logs to collect a dataset of code completions from users, and employed it to train a ranking CatBoost model. Then, we evaluated it in two settings: on a held-out set of the collected completions and in a separate A/B test on two different groups of users in the IDE. Our evaluation shows that using a simple ranking model trained on the past user behavior logs significantly improved code completion experience. Compared to the default heuristics-based ranking, our model demonstrated a decrease in the number of typing actions necessary to perform the completion in the IDE from 2.073 to 1.832.
DOI 原文 ·
@inproceedings{paperbot7086,
title = {All you need is logs: improving code completion by learning from anonymous IDE usage logs},
author = {Vitaliy Bibaev and Alexey Kalina and Vadim Lomshakov and Yaroslav Golubev and Alexander Bezzubov and Nikita Povarov and Timofey Bryksin},
booktitle = {FSE},
year = {2022},
doi = {10.1145/3540250.3558968}
}