一句话：提出组合式 FHE 编译器方法 MKR，将 MVM 与卷积分解为可组合 MetaKernel，大幅减少旋转开销并加速加密推理。
在 CKKS 全同态加密（FHE）方案下进行实用的加密神经网络推理，关键在于加速矩阵向量乘法（MVM）与卷积（Conv）两类核心算子。然而现有方案（专家手工调优的库与领域特定语言）多为特定场景的临时设计，因旋转操作过多而导致显著低效。本文提出 MKR，一种新的组合式编译方法，在统一框架内为 DNN 模型优化 CKKS 下的 MVM 与 Conv 算子。MKR 将每个算子分解为可组合的单元——MetaKernel，通过水平批处理提升密文内的 SIMD 并行性，通过垂直批处理提升密文间的计算并行性。该方法还解决了此前未被处理的一系列挑战：以旋转感知的代价模型指导数据打包以降低旋转开销，同时保证高槽位利用率、对任意尺寸输入的统一处理，以及与输出张量布局的兼容。MKR 已在生产级 FHE 编译器中实现，相比最先进的 FHE 编译器，单个 MVM 与 Conv 算子推理时间加速 10.08×–185.60×，端到端推理加速 1.75×–11.84×；此外，MKR 还能同态执行此前方法无法运行的大型 DNN 模型，显著推进了 FHE 编译器的实用性。
Practical encrypted neural network inference under the CKKS fully homomorphic encryption (FHE) scheme relies heavily on accelerating two key kernel operations: Matrix-Vector Multiplication (MVM) and Convolution (Conv). However, existing solutions—such as expert-tuned libraries and domain-specific languages—are designed in an ad hoc manner, leading to significant inefficiencies caused by excessive rotations. We introduce MKR, a novel composition-based compiler approach that optimizes MVM and Conv kernel operations for DNN models under CKKS within a unified framework. MKR decomposes each kernel into composable units, called MetaKernels , to enhance SIMD parallelism within ciphertexts (via horizontal batching) and computational parallelism across them (via vertical batching). Our approach tackles previously unaddressed challenges, including reducing rotation overhead through a rotation-aware cost model for data packing, while also ensuring high slot utilization, uniform handling of inputs with arbitrary sizes, and compatibility with the output tensor layout. Implemented in a production-quality FHE compiler, MKR achieves inference time speedups of 10.08×−185.60× for individual MVM and Conv kernels and 1.75×−11.84× for end-to-end inference compared to a state-of-the-art FHE compiler. Moreover, MKR enables homomorphic execution of large DNN models, where prior methods fail, significantly advancing the practicality of FHE compilers.
@article{paperbot2899,
title = {MetaKernel: Enabling Efficient Encrypted Neural Network Inference through Unified MVM and Convolution},
author = {Peng Yuan and Yan Liu and JianXin Lai and Long Li and Tianxiang Sui and Linjie Xiao and Xiaojing Zhang and Qing Zhu and Jingling Xue},
journal = {Proceedings of the ACM on Programming Languages},
volume = {9},
number = {OOPSLA2},
year = {2025},
doi = {10.1145/3763095}
}