Operating system migration requires validating thousands of software packages for behavioral compatibility on the target distribution. Differential testing is the dominant industrial approach. OS level outputs contain many benign differences that are unrelated to package compatibility. These differences produce many false positives. They also bury real compatibility issues under heavy triage load. To address this issue, we first conduct a study of false positive root causes on industrial enterprise Linux migration data. This study yields a five-category taxonomy and a share–cost decision matrix. These findings guide our design. We propose FPSieve, a cost routed filtering pipeline for OS migration validation. FPSieve integrates rule based filtering, semantic similarity filtering, case library retrieval, package level LLM reasoning with confidence based routing, and closed loop container based verification. We evaluate FPSieve on a large scale OS migration differential testing dataset collected from industrial practice. FPSieve reduces the overall manual review queue to 5.9% of its original size while preserving 86.7% recall. It also improves review queue precision from 3.16% to 46.0%. After deduplication, FPSieve surfaced 66 compatibility issues submitted upstream. Of these, 14 received maintainer feedback, and 6 have been fixed. These results show that FPSieve can reduce false positives, reduce manual effort, and improve OS migration validation throughput.

@inproceedings{paperbot12552,
  title = {FPSieve: LLM-Powered Cascade for Filtering False Positives in OS Migration Testing},
  author = {Jie Dong and Ying Fu and Chao Hu and Yuhang Chen and Qi Xiao and Heyuan Shi},
  booktitle = {ASE},
  year = {2026},
  doi = {10.1145/3832783.3834460}
}