一句话：提出SOLO，利用眼动注视信息仅分割用户关注区域，并软硬件协同加速，将AR分割端到端延迟降低最多12倍。
增强现实（AR）中的图像分割需要从用户前场景中识别并分离特定物体，但计算需求大、处理延迟高，严重损害体验。本文提出 SOLO，利用自然人类眼动动态，只分割用户注视的实例。SOLO 是一个高效深度学习框架，输入高分辨率图像和用户眼图，仅分割感兴趣的实例；它还与基于显著性的传感（SBS）和 SOLO 加速器协同设计，作为 AR 设备 SoC 的插件。实验表明，与其他基线相比，端到端延迟最多降低 12 倍，显著减少图像分割的计算成本。
Augmented reality (AR) comprises groundbreaking technologies that are reshaping the landscape of human interaction. Image segmentation, which divides a user front-scene frame into more manageable parts for analysis, is of paramount importance since this technique enables AR systems to extract digital information precisely from the real world by identifying and isolating specific objects in the user's surroundings. Despite its importance, the segmentation task imposes substantial computational demands and processing delays on AR devices, significantly degrading the user experience. In this work, we aim to reduce the high computational costs of the segmentation task in AR by leveraging natural human eye dynamics and focusing on segmenting only where you look (SOLO). This involves co-optimizing image segmentation algorithms with underlying hardware for greater efficiency. We introduce SOLO algorithm, an efficient deep learning framework that takes high-resolution input images and user eye images to effectively segment only the instance of interest. Integrated with the saliency-based sensing (SBS) and SOLO accelerator as a plug-in for SoCs of the AR device, SOLO significantly lowers the computational costs for image segmentation, achieving up to a 12× reduction in end-to-end latency compared to other baselines.
@inproceedings{XiaWZ26,
title = {Segment Only Where You Look: Leveraging Human Gaze Behavior for Efficient Computer Vision Applications in Augmented Reality},
author = {Tianhua Xia and Haiyu Wang and Sai Qian Zhang},
booktitle = {ASPLOS},
year = {2026},
doi = {10.1145/3779212.3790216}
}