The integration of machine learning (ML) into OS kernels is severely hampered by the high latency of offloading to discrete GPUs (dGPUs), where data transfers across the PCIe bus can consume over 93% of the total execution time. This paper argues that for many latency-sensitive kernel tasks, the solution is not a more powerful dGPU but a fundamental shift to an I/O-efficient architecture: the integrated GPU (iGPU) found in modern APUs. We present LAIKA, a kernel-space acceleration framework that fully exploits the APU's unified memory. LAIKA combines three key mechanisms: a lightweight API proxy (AProxy) to safely bridge the kernel-user boundary, a tri-domain shared memory manager (AShm) for true zero-copy data exchange, and APU Persistent Kernel (APK) to eliminate control-path overhead. Across a suite of representative kernel workloads, our evaluation shows that by eliminating the I/O bottleneck, LAIKA slashes end-to-end inference latency by up to 9.7× and reduces total system power to as little as 28.9% of an optimized dGPU baseline. Our work identifies iGPUs as a practical and highly efficient alternative for machine learning acceleration within OS kernels.

@inproceedings{ZhuoLL026,
  title = {LAIKA: Machine Learning-Assisted In-Kernel APU Acceleration},
  author = {Haoming Zhuo and Dingding Li and Ronghua Lin and Yong Tang},
  booktitle = {ASPLOS},
  year = {2026},
  doi = {10.1145/3779212.3790181}
}