一句话:论文提出数据轻量级挑战,证明在百余个软件工程优化任务中,仅用少量标签的简单优化器即可达到复杂优化器约90%的效果且快得多。
当前软件工程研究普遍假设进展需要大规模数据集和CPU密集型优化器,但这一假设缺乏严格检验。本文提出“数据轻量级”挑战,形式化标签成本,构建轻量级算法库,并在100多个软件工程优化任务上系统比较简单优化器与复杂优化器。实验表明,仅使用几十个标签,简单优化器就能达到最佳报告结果约90%的性能,与SMAC、TPE、DEHB等复杂优化器相当,但运行速度快数个数量级。论文还实证分析了轻量算法在何时表现优异、何时失效,并开源了脚本与数据。
Much of current Software Engineering (SE) research assumes that progress requires massive datasets and CPU-intensive optimizers. However, has this been rigorously tested? This work bears evidence that for over 100 SE optimization tasks, with only a few dozen labels, simple optimizers achieve ≈ 90% of the best reported results, matching complex optimizers such as SMAC, TPE, and DEHB while running orders of magnitude faster. We propose the data-light challenge: when will a handful of labels suffice for SE tasks? We contribute (1) a mathematical formalization of labeling costs, (2) a library of lightweight algorithms, and (3) empirical results across 100+ SE tasks showing when lightweight algorithms excel or fail. For the purposes of open science, our scripts and data are online at https://github.com/KKGanguly/NEO.