一句话：通过伪精确重复实验验证 EVO 可视化能加快目标建模决策，并发现其配色方案不够直观。
高质量软件依赖精确且全面的需求。目标建模可捕捉利益相关者需求并支持权衡，但分析结果常难以解释。已有研究发现，经少量目标建模培训并使用颜色可视化 EVO 的被试，能更快做出目标建模决策且不牺牲质量。本文对 EVO 的实证证据进行稳健性检验，并质疑其原始配色选择。作者开展伪精确重复实验（n=60），改变实验地点与被试群体。即使样本异质性更高且先验需求/目标建模熟悉度更低，使用 EVO 的个体回答目标建模问题仍显著快于对照组，扩展了原结果的外部效度；但同时发现配色方案并非直观，并向目标建模社区提出建议。
The development of high-quality software depends on precise and comprehensive requirements that meet the objectives of stakeholders. Goal modeling techniques have been developed to fill this gap by capturing and analyzing stakeholders' needs and allowing them to make trade-off decisions; yet, goal modeling analysis is often difficult for stakeholders to interpret. Recent work found that when subjects are given minimal training on goal modeling and access to a color visualization, called EVO, they are able to use EVO to make goal modeling decisions faster without compromising quality. In this paper, we evaluate the robustness of the empirical evidence for EVO and question the underlying color choices made by the initial designers of EVO. We conduct a pseudo-exact replication$(n=60)$of the original EVO study, varying the experimental site and the study population. Even in our heterogeneous sample with less a priori familiarity with requirements and goal modeling, we find that individuals using EVO answered the goal-modeling questions significantly faster than those using the control, expanding the external validity of the original results. However, we find some evidence that the chosen color scheme is not intuitive and make recommendations for the goal modeling community.
DOI 原文 ·
@inproceedings{paperbot8882,
title = {Exploring the Robustness of the Effect of EVO on Intention Valuation Through Replication},
author = {Yesugen Baatartogtokh and Kaitlyn Cook and Alicia M. Grubb},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00143}
}