一句话：提出 profile coverage 指标，用 Android 编译 profile 衡量动态测试对真实用户行为的覆盖程度。
Android 应用日益复杂，全面动态测试不可行，而了解真实用户执行哪些方法通常需要昂贵用户研究或使用遥测。本文发现 Google Play 收集的 Cloud Profiles 等 Android 编译配置文件可作为现成的用户使用信息源：它们聚合了跨用户常执行的方法，并指导安装时 AOT 编译。作者首次深入刻画 Baseline Profiles 与 Cloud Profiles，发现前 1000 个热门应用中超过 99.89% 包含源自使用情况的 cloud profiles。基于此，论文提出 profile coverage 这一新指标，用于衡量动态测试对真实用户所交互方法的覆盖程度，支持更全面地评估自动化测试输入生成器。为测量该指标，作者实现轻量级追踪器 ProfTrace，基于 Linux 内核 uprobes，无需修改应用或系统。在 50 个热门应用上比较三种工具与无交互基线，表明 profile coverage 能揭示传统代码覆盖率遗漏的差异，例如 Candy Crush 中自动测试仅达 2.22% 方法覆盖率，但 profile coverage 达 21.39%。
The rising complexity of Android apps makes comprehensive dynamic testing infeasible, especially for third-party apps. Knowing which methods are exercised by real users typically requires costly user studies or access to usage telemetry. We show that Android’s compilation profiles, specifically Cloud Profiles collected by the Google Play Store, offer a readily available, underutilized source of such information. These operational profiles aggregate which methods are commonly executed across users and guide ahead-of-time compilation during app installation. We provide the first in-depth characterization of Baseline Profiles and Cloud Profiles and show that over 99.89% of the top 1,000 apps include usage-derived cloud profiles.Based on this insight, we introduce profile coverage, a novel metric that measures how well dynamic testing exercises the methods real users interact with. This metric builds on the idea of operational coverage and enables a more holistic evaluation of automated test input generators. To enable profile coverage measurements, we develop a lightweight tracer, ProfTrace, based on Linux kernel uprobes that requires no app or system modifications. We demonstrate its utility by comparing three tools and a no-interaction baseline on 50 popular apps, showing that profile coverage reveals differences that traditional code coverage misses. For instance, in Candy Crush, automated testing achieves only 2.22% method coverage, but 21.39% profile coverage—indicating better alignment with user behavior than traditional code coverage would suggest.
DOI 原文 ·
@inproceedings{paperbot9399,
title = {Profile Coverage: Using Android Compilation Profiles to Evaluate Dynamic Testing},
author = {Jakob Bleier and Felix Kehrer and Jürgen Cito and Martina Lindorfer},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00080}
}