一句话：提出 PyAnalyzer，将函数、类和模块建模为一等堆对象并利用类型标注，高效高召回地提取 Python 依赖。
静态依赖提取是许多应用的基础，但 Python 的动态特性使符号级依赖解析困难，现有工具难以处理对象变化、一等公民、不同调用点和库依赖。本文提出 PyAnalyzer：将函数、类和模块统一建模为一等堆对象，传播其动态变化与类继承，以更好模拟鸭子类型等动态特性，从而在不牺牲精度的情况下获得高召回；同时利用可选类型标注作为捷径表达不同调用点并按需解析库依赖。作者在 278 个小程序、59 个真实应用和 191 个真实项目（10 MSLOC）上与 7 种先进技术比较。结果表明，PyAnalyzer 平均 F1 提高 24.7%，至少快 1.4 倍，且内存效率无明显损失。
Dependency extraction based on static analysis lays the groundwork for a wide range of applications. However, dynamic language features in Python make code behaviors obscure and nondeterministic; consequently, it poses huge challenges for static analyses to resolve symbol-level dependencies. Although prosperous techniques and tools are adequately available, they still lack sufficient capabilities to handle object changes, first-class citizens, varying call sites, and library dependencies. To address the fundamental difficulty for dynamic languages, this work proposes an effective and practical method namely PyAnalyzer for dependency extraction. PyAnalyzer uniformly models functions, classes, and modules into first-class heap objects, propagating the dynamic changes of these objects and class inheritance. This manner better simulates dynamic features like duck typing, object changes, and first-class citizens, resulting in high recall results without compromising precision. Moreover, PyAnalyzer leverages optional type annotations as a shortcut to express varying call sites and resolve library dependencies on demand. We collected two micro-benchmarks (278 small programs), two macro-benchmarks (59 real-world applications), and 191 real-world projects (10MSLOC) for comprehensive comparisons with 7 advanced techniques (i.e., Understand, Sourcetrail, Depends, ENRE19, PySonar2, PyCG, and Type4Py). The results demonstrated that PyAnalyzer achieves a high recall and hence improves the F1 by 24.7% on average, at least 1.4x faster without an obvious compromise of memory efficiency. Our work will benefit diverse client applications.
@inproceedings{paperbot11307,
title = {PyAnalyzer: An Effective and Practical Approach for Dependency Extraction from Python Code},
author = {Wuxia Jin and Shuo Xu and Dawei Chen and Jiajun He and Dinghong Zhong and Ming Fan and Hongxu Chen and Huijia Zhang and Ting Liu},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3597503.3640325}
}