一句话：通过分析 VS Code 市场 AI 编程助手的真实用户评论，构建用户关注点分类并给出五项改进建议。
本文研究 AI 编程助手普及背景下开发者真正看重与批评什么，以及这反映出的需求与期望。与受控实验不同，作者从 Visual Studio Code Marketplace 识别出 1,085 个 AI 编程助手，发现其仅占全部扩展 1.64%，但 90% 以上近两年发布，增长迅速。随后从 32 个具有足够安装量和评论的助手中抽样用户评论，人工标注态度并构建用户关注与反馈分类体系，得到对具体功能、担忧和整体性能的细粒度满意度洞察。结果表明用户不仅需要智能建议，还要求上下文感知、可定制且资源高效的交互。基于此，作者提出五项实践启示，指导 AI 编程助手改进以更好满足用户需求。
This paper aims to explore fundamental questions in the era when AI coding assistants like GitHub Copilot are widely adopted: what do developers truly value and criticize in AI coding assistants, and what does this reveal about their needs and expectations in real-world software development? Unlike previous studies that conduct observational research in controlled and simulated environments, we analyze extensive, first-hand user reviews of AI coding assistants, which capture developers’ authentic perspectives and experiences drawn directly from their actual day-to-day work contexts. We identify 1,085 AI coding assistants from the Visual Studio Code Marketplace. Although they only account for 1.64% of all extensions, we observe a surge in these assistants: over 90% of them are released within the past two years. We then manually analyze the user reviews sampled from 32 AI coding assistants that have sufficient installations and reviews to construct a comprehensive taxonomy of user concerns and feedback about these assistants. We manually annotate each review’s attitude when mentioning certain aspects of coding assistants, yielding nuanced insights into user satisfaction and dissatisfaction regarding specific features, concerns, and overall tool performance. Built on top of the findings-including how users demand not just intelligent suggestions but also context-aware, customizable, and resource-efficient interactions—we propose five practical implications and suggestions to guide the enhancement of AI coding assistants that satisfy user needs.
DOI 原文 ·
@inproceedings{paperbot9381,
title = {"My productivity is boosted, but …" Demystifying Users’ Perception on AI Coding Assistants},
author = {Yunbo Lyu and Zhou Yang and Jieke Shi and Jianming Chang and Yue Liu and David Lo},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00024}
}