一句话：提出 OBsmith，用 LLM 生成程序草图并结合等价与蜕变测试，自动发现 JavaScript 混淆器的语义保持缺陷。
JavaScript 混淆器的正确性（是否保持程序语义）长期被忽视，错误变换会静默改变功能。本文提出 OBsmith：用 LLM 生成覆盖多样语法构造与边界情况的程序草图，并从真实程序抽取骨架；实例化后的程序在多种配置下混淆。测试预言包括以原程序为基准的等价性测试与基于语义保持变换的蜕变测试。在 Obfuscator.IO 与 JS-Confuser 上，用六个 LLM 生成 600 个草图、3000 余个程序、七种配置，发现 11 个未知正确性缺陷；同预算下五个主流 JS fuzzer 均未检出。消融显示各组件（除通用蜕变关系外）均有贡献。
JavaScript obfuscators are widely deployed to protect intellectual property and resist reverse engineering, yet their correctness has been largely overlooked compared to performance and resilience. Existing evaluations typically measure resistance to deobfuscation, leaving the critical question of whether obfuscators preserve program semantics unanswered. Incorrect transformations can silently alter functionality, compromise reliability, and erode security-undermining the very purpose of obfuscation. To address this gap, we present OBsmith, a novel framework to systematically test JavaScript obfuscators using large language models (LLMs). OBsmith leverages LLMs to generate program sketches-abstract templates capturing diverse language constructs, idioms, and corner cases-which are instantiated into executable programs and subjected to obfuscation under different configurations. Besides LLM-powered sketching, OBsmith also employs a second source: automatic extraction of skeletons from real programs. This extraction path enables more focused testing of project-specific features and lets developers inject domain knowledge into the resulting test cases. OBsmith uses two techniques to derive test oracles: (i) reference-oriented equivalence testing, which takes the original program as reference oracle (ground truth) and checks whether the obfuscated version preserves equivalent functionality, and (ii) metamorphic testing, which applies semantics-preserving transformations to the original program and checks if obfuscation violates expected behavior. We evaluate OBsmith on two widely used obfuscators, Obfuscator.IO and JS-Confuser, generating 600 sketches using six popular LLMs. OBsmith fills these sketches and generates over 3,000 candidate programs and obfuscates them across seven obfuscation configurations. OBsmith uncovers 11 previously unknown correctness bugs. Under an equal program budget, five general purpose state-of-the-art JavaScript fuzzers (FuzzJIT, Jsfunfuzz, Superion, DIE, Fuzzilli) failed to detect these issues, highlighting OBsmith's complementary focus on obfuscation-induced misbehavior. An ablation shows that all components except our generic MRs contribute to at least one bug class; the negative MR result suggests the need for obfuscator-specific metamorphic relations. Our results also seed a discussion on how to balance obfuscation presets and performance cost. We envision OBsmith as an important step towards automated testing and quality assurance of obfuscators and other semantic-preserving toolchains.
@article{paperbot3838,
title = {OBsmith: LLM-Powered JavaScript Obfuscator Testing},
author = {Shan Jiang and Chenguang Zhu and Sarfraz Khurshid},
journal = {Proceedings of the ACM on Programming Languages},
volume = {10},
number = {OOPSLA1},
year = {2026},
doi = {10.1145/3798204}
}