一句话:通过 18 对软件专业人员的实验室研究,揭示 LLM 在协作软件设计中的联合使用模式、依赖差异与早期锚定效应。
已有工作多考察 LLM 在单人编码任务中的作用,对 LLM 如何影响软件工程中的协作工作了解甚少。本文聚焦软件设计,开展探索性实验室研究:18 对软件专业人员可自由使用 LLM,共同设计一个校园自行车停放应用。研究发现,共享同一 LLM 实例有助于形成共同理解,而各自并行使用不同实例有时导致“上下文漂移”;依赖程度从完全不用到把 LLM 当作信息源或内容生产者差异很大。参与者普遍会审查与反思 LLM 回复并由此获得设计洞见,但早期锚定有时限制探索。文章据此提出工具设计启示,强调在辅助设计者的同时保持以人为中心。
While much prior work examines Large Language Models (LLMs) for solo development tasks (e.g., coding), far less is known about how LLMs shape collaborative group work in software engineering. This study focuses on one such collaborative task, namely software design. It presents the results of an exploratory laboratory study of 18 pairs of software professionals who could use an LLM however they saw fit, to design a university campus bicycle parking application. Our findings reveal that introducing an LLM leads to distinct patterns of joint use: shared-instance use facilitated shared understanding, whereas parallel use across separate instances sometimes led to "context drift". We also observe wide variation in reliance, from non-use to treating the LLM as an information source or producer. Across these modes, professionals scrutinized and reflected on LLM responses, often yielding design insights; however, early anchoring sometimes curtailed exploration. We provide implications for tools to aid designers while retaining the human-centricity important to design.