一句话：提出黑盒框架 MobileUPReg，通过跨 OS 版本运行同一应用并比较用户可感知性能指标来检测移动 OS 性能回归。
移动 OS 更新可能引入用户可感知的性能回归，但现有检测多依赖系统级指标或特定组件，易漏掉响应变慢与 UI 卡顿。本文提出 MobileUPReg，一个黑盒框架：在不同 OS 版本下运行相同应用，提取并比较响应时间、完成时间、启动时间和丢帧等用户可感知指标，以识别真正可感知的回归。大规模评估中，其指标提取准确率高，回归检测达到 0.96 精确率、0.91 召回率和 0.93 F1，显著优于 Wilcoxon 秩和检验加 Cliff's Delta 的统计基线。该框架已部署于工业 CI，每天分析数百应用的数千录屏，并发现传统工具漏检的回归。
Mobile operating systems (OS) are frequently updated, but such updates can unintentionally degrade user experience by introducing performance regressions. Existing detection techniques often rely on system-level metrics (e.g., CPU or memory usage) or focus on specific OS components, which may miss regressions actually perceived by users—such as slower responses or UI stutters. To address this gap, we present MobileUPReg, a black-box framework for detecting user-perceived performance regressions across OS versions. MobileUPReg runs the same apps under different OS versions and compares user-perceived performance metrics—response time, finish time, launch time, and dropped frames—to identify regressions that are truly perceptible to users. In a large-scale study, MobileUPReg achieves high accuracy in extracting user-perceived metrics and detects user-perceived regressions with 0.96 precision, 0.91 recall, and 0.93 F1-score—significantly outperforming a statistical baseline using the Wilcoxon rank-sum test and Cliff’s Delta. MobileUPReg has been deployed in an industrial CI pipeline, where it analyzes thousands of screencasts across hundreds of apps daily and has uncovered regressions missed by traditional tools. These results demonstrate that MobileUPReg enables accurate, scalable, and perceptually aligned regression detection for mobile OS validation.
DOI 原文 ·
@inproceedings{paperbot9577,
title = {MobileUPReg: Identifying User-Perceived Performance Regressions in Mobile OS Versions},
author = {Wei Liu and Yi Wen Heng and Feng Lin and Tse-Hsun Peter Chen and Ahmed E. Hassan},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00308}
}