一句话：提出 DFVG 异构架构，把推测解码的草稿生成卸载到 FPGA、验证交给 GPU，最高取得 3.26× 加速与 5.8× 能效提升。
推测解码通过先生成草稿再验证来加速 LLM 推理，但现有系统面临三重挑战：草稿模型与验证模型的资源需求不均衡，同构部署时利用率与能效偏低；固定模式的 token 树产生大量候选却少有有效路径，导致草稿冗余；两阶段频繁交替的异步执行带来空等与回滚开销。本文提出 DFVG 异构架构，将草稿生成卸载到 FPGA、验证保留在 GPU，利用二者互补优势，并设计重叠处理器以提升吞吐；提出硬件感知的动态草稿生成，依据模型置信度并考虑硬件并行度上限，动态预测推测分支与 token 长度；设计紧耦合异构流水线与阶段解耦调度，为两阶段分配执行窗口，并配合轻量级跨设备对齐与回滚预测策略。在 OPT、LLaMA、Qwen 等主流模型上的评估显示，DFVG 相比现有方法最高取得 3.26× 加速与 5.8× 能效提升。
Speculative decoding is a promising paradigm that accelerates LLM inference by generating drafts and performing verification. However, such systems still face three major challenges: (1) The imbalance in resource requirements between draft and verification models result in low utilization and energy inefficiency when deployed together. (2) Fixed-pattern token trees produce many candidates but few valid paths, resulting in redundant drafts due to the lack of full leverage of the inherent confidence in dynamic generation. (3) Asynchronous execution with frequent alternation between the two stages suffers from idle waiting and rollback overhead. To address these issues, we propose DFVG, a heterogeneous speculative decoding architecture that offloads draft generation to FPGAs and verification to GPUs, exploiting their complementary strengths. We introduce three key contributions: (1) Heterogeneous architecture design that partitions speculative decoding into FPGA-based drafting and GPU-based verification, exploiting complementary hardware strengths with an overlap processor for high-throughput execution; (2) Hardware-aware dynamic draft generation that dynamically predicts speculative branches and token lengths based on model confidence while considering hardware parallelism limits; (3) Tightly-coupled heterogeneous pipeline with stagedecoupled scheduling that allocates execution windows between stages, combined with lightweight cross-device alignment and rollback prediction strategies. Comprehensive evaluation on mainstream models (OPT, LLaMA, Qwen) demonstrates DFVG achieves up to 3.26× speedup and 5.8× energy efficiency improvement over existing approaches. The source code at: https://github.com/ShaoqiangLu/DFVG
@inproceedings{LuWQQGDWW0026,
title = {DFVG: A Heterogeneous Architecture for Speculative Decoding with
<u>D</u>
raft-on-
<u>F</u>
PGA and
<u>V</u>
erify-on-
<u>G</u>
PU},
author = {Shaoqiang Lu and Yangbo Wei and Junhong Qian and Dongge Qin and Shiji Gao and Yizhi Ding and Qifan Wang and Chen Wu and Xiao Shi and Lei He},
booktitle = {ASPLOS},
year = {2026},
doi = {10.1145/3779212.3790153}
}