一句话:揭示 Android 污点分析工具评估目标不一、基准不完整且缺乏真值,导致无法真正比较工具。
论文研究 Android 污点分析工具实证评估的可比性问题。作者指出,尽管相关工具论文都试图提供充分评估,但评估目标常不一致或不明确:被测应用描述不精确,有的仅部分覆盖公认基准,有的搜索的数据泄漏类型不同,且常缺少可供对照的 ground truth。这些限制使读者无法根据已发表结果真正比较不同工具,也无法判断工具是否兑现其宣称能力。该工作系统揭示 Android 污点分析评估实践中的缺陷,强调需要统一、可复现且带真值的评估基准与协议,以支撑工具间公平比较。
In recent years, researchers have developed a number of tools to conduct taint analysis of Android applications. While all the respective papers aim at providing a thorough empirical evaluation, comparability is hindered by varying or unclear evaluation targets. Sometimes, the apps used for evaluation are not precisely described. In other cases, authors use an established benchmark but cover it only partially. In yet other cases, the evaluations differ in terms of the data leaks searched for, or lack a ground truth to compare against. All those limitations make it impossible to truly compare the tools based on those published evaluations.