一句话:提出 PyObject 状态转移模型,用符号执行跟踪引用计数生命周期,检测 Python C 扩展中的内存泄漏与释放后使用错误。
第三方 Python 模块常以 C/C++ 二进制扩展实现,其中的堆分配 PyObject 依赖 Python/C API 的引用计数机制自动回收,不当的引用计数操作会导致内存泄漏和 use-after-free,且无法靠简单的源点—汇点配对检测。现有工作主要判断原生代码返回 Python 前的最终引用计数是否正确,但很多缺陷出现在路径中间而非末尾。为此,作者提出 PyObject 状态转移模型(PSTM),记录 PyObject 的生命周期状态与引用计数值,用以刻画 Python/C API 调用与指针操作的效果;并基于该模型用符号执行跟踪状态转移,在语句触发进入错误状态时报告缺陷。该方法还扩展了程序状态以处理指针空值检查和 PyObject 智能指针。作者在 12 个开源项目上评估,在 280 条报告中检出 259 个真实问题,数量约为最先进方法的两倍;向活跃项目提交 168 个真实缺陷,其中 106 个被确认或修复。
Third-party Python modules are usually implemented as binary extensions by using native code (C/C++) to provide additional features and runtime acceleration. In native code, the heap-allocated PyObjects are managed by the reference counting mechanism provided in Python/C APIs for automatic reclaiming. Hence, improper refcount manipulations can lead to memory leaks and use-after-free problems, and cannot be detected by simply pairing the occurrence of source and sink points. To detect such problems, state-of-the-art approaches have made groundbreaking contributions to identifying inappropriate final refcount values before returning from native code to Python. However, not all problems can be exposed at the end of a path. To detect those hidden in the middle of a path in native code, it is also crucial to track the lifecycle state of PyObjects through the refcount and lifecycle operations in API calls. To achieve this goal, we propose the PyObject State Transition Model (PSTM) recording the lifecycle states and refcount values of PyObjects to describe the effects of Python/C API calls and pointer operations. We track state transitions of PyObjects with symbolic execution based on the model, and report problems when a statement triggers a transition to buggy states. The program state is also expanded to handle pointer nullity checks and smart pointers of PyObjects. We conduct experiments on 12 open-source projects and detect 259 real problems out of 280 reports, which is twice as many bugs as state-of-the-art approaches. We submit 168 real bugs to those active projects, and 106 issues are either confirmed or resolved.
@inproceedings{MaYZYZ23,
title = {Detecting Memory Errors in Python Native Code by Tracking Object Lifecycle with Reference Count},
author = {Xutong Ma and Jiwei Yan and Hao Zhang and Jun Yan and Jian Zhang},
booktitle = {ASE},
year = {2023},
doi = {10.1109/ase56229.2023.00198}
}