一句话:通过任务内礼貌/强制语气配对扰动,测量四个 LLM 在软件工程任务中的行为稳定性与安全信号差异。
开发者日益依赖大语言模型完成编程、推理与文档撰写,但其行为在社会语用变化下是否保持稳定尚不清楚。本文开展受控的任务内扰动研究:对 60 个软件工程任务分别构造礼貌与强制语气变体,在四个已部署的 LLM 助手上共获得 480 条响应,形成 240 组任务内配对比较。作者采用经过验证的计算指标并结合针对性人工审阅,测量交际投入、情感立场、任务相关指标与安全信号(拒答、免责声明、毒性)。结果显示:礼貌提示稳定地引出更长、更积极表述的回答(+222 token,约 18%),而强制语气压缩了阐述篇幅,但审计的任务相关指标与安全信号并未出现可观察的退化。语气敏感性因领域与模型而异:伦理推理任务的情感偏移更大,GPT-4o 表现出最高的行为稳定性。作者将语气扰动解释为轻量级鲁棒性探针,而非风格噪声,主张把社会语用变异作为 LLM 辅助软件工程的一等评估维度。
Developers increasingly rely on large language models (LLMs) for programming, reasoning, and documentation, yet little is known about whether their behavior remains stable under realistic sociopragmatic variation. We conduct a controlled, within-task perturbation study pairing polite and coercive variants of 60 softwareengineering tasks across four deployed LLM assistants, yielding 480 responses (240 within-task paired comparisons). Using validated computational metrics and targeted human review, we measure communicative effort, affective stance, task-relevant indicators, and safety signals (refusals, disclaimers, toxicity). Polite prompts consistently elicit longer and more positively framed responses (+222 tokens; ≈18%), while coercive tone compresses elaboration without observable degradation in audited task-relevant indicators or safety signals. Tone sensitivity varies by domain and across models, with ethical reasoning exhibiting larger affective shifts and GPT-4o demonstrating the highest behavioral stability. We interpret tone perturbation not as stylistic noise but as a lightweight robustness probe, supporting the inclusion of socio-pragmatic variation as a first-class evaluation dimension for LLM-assisted software engineering.