paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Playing Planning Poker in Crowds: Human Computation of Software Effort Estimates

众包中的规划扑克:软件工作量估算的人类计算
ICSE 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)2021匹配度 1/10 · 该论文研究众包 Planning Poker 进行软件工作量估算,属于软件工程管理与人机计算方向,与用户关注的静态分析、LLM/Agent、浮点程序分析及贝叶斯程序分析均无直接关联。引用 16 inbox
Mohammed Alhamed, Tim Storer
crossref: 2026-09-12dblp: 2026-09-12

AI 速览

一句话:该论文设计并评估了在 Amazon Mechanical Turk 上进行的众包 Planning Poker,证明精心组织的众包可达到接近单个专家的估算精度。

软件项目中的工作量估算长期面临可靠性与成本效益的挑战。Planning Poker 在具备领域知识的团队内可产生可靠估算,但依赖专家且耗时,难以用于可能有数千待处理任务的大型或开源项目。本文研究让仅获得有限任务信息的众包工人通过 Planning Poker 给出可比准确度估算的可行性。作者设计了在 Amazon Mechanical Turk 上运行的 Crowd Planning Poker 流程,并开展了涉及 5000 多名众包工人和 39 个不同软件任务的大规模试验。结果表明,经过精心组织和筛选的众包工人能够给出与单个专家准确度相似的工作量估算。

问题
软件项目需要可靠且成本效益高的努力估算;Planning Poker 虽在专家团队中有效,但依赖专家、耗时,难以用于大规模或开源项目。
方法
论文设计并实现基于 Amazon Mechanical Turk 的 Crowd Planning Poker 流程,让只掌握有限任务信息的众包工人进行 Planning Poker 估算。
评估
通过超过 5000 名众包工人和 39 个多样软件任务的试验,将众包估算与单个专家的准确度进行比较。
原文摘要(Abstract)

Reliable cost effective effort estimation remains a considerable challenge for software projects. Recent work has demonstrated that the popular Planning Poker practice can produce reliable estimates when undertaken within a software team of knowledgeable domain experts. However, the process depends on the availability of experts and can be time-consuming to perform, making it impractical for large scale or open source projects that may curate many thousands of outstanding tasks. This paper reports on a full study to investigate the feasibility of using crowd workers supplied with limited information about a task to provide comparably accurate estimates using Planning Poker. We describe the design of a Crowd Planning Poker (CPP) process implemented on Amazon Mechanical Turk and the results of a substantial set of trials, involving more than 5000 crowd workers and 39 diverse software tasks. Our results show that a carefully organised and selected crowd of workers can produce effort estimates that are of similar accuracy to those of a single expert.

链接与引用

DOI 原文 · PDF(开放获取) · DBLP

BibTeX
@inproceedings{AlhamedS21,
  title = {Playing Planning Poker in Crowds: Human Computation of Software Effort Estimates},
  author = {Mohammed Alhamed and Tim Storer},
  booktitle = {ICSE},
  year = {2021},
  doi = {10.1109/icse43902.2021.00014}
}