一句话：提出定制锁存器阵列与位串行计算织物，并用 tinyForge 设计空间探索优化 tinyML 架构的能量-面积权衡。
针对资源受限 IoT 上 tinyML 模型小、频率低的特点，本文提出与计算-存储织物集成的定制锁存器阵列，实现 8 μm²/B 密度与 11 fJ/B 读能耗，较综合实现提升 7 倍密度和 5 倍读能耗，并支持无需激活缓冲的数据流。通过优化二维计算阵列中脉动与组合缩放、采用位串行计算，面积降低 4.8 倍，乘累加能耗降低 2.3 倍。作者构建 tinyForge 设计空间探索，含参数化模板、功耗/面积/时延估计、数据流优化器与遗传多目标优化，以寻找 Pareto 最优架构。在 MLPerf Tiny 与 BERT-Tiny 上评估，片存权重较片外内存降低 7.5 倍单次推理能耗，定制锁存器与位串行阵列最高降低 1.8 倍能耗、2.2 倍时延和 3.7 倍面积。
The proliferation of smart IoT devices has given rise to tinyML, which deploys deep neural networks on resource-constrained systems, benefitting from custom hardware that optimizes for low silicon area and high energy efficiency amidst tinyML's characteristic small model sizes (50-500 KB) and low target frequencies (1-100 MHz). We introduce a novel custom latch array integrated with a compute memory fabric, achieving 8 μm2/B density and 11 fJ/B read energy, surpassing synthesized implementations by 7x in density and 5x in read energy. This advancement enables dataflows that do not require activation buffers, reducing memory overheads. By optimizing systolic vs. combinational scaling in a 2D compute array and using bit-serial instead of bit-parallel compute, we achieve a reduction of 4.8x in area and 2.3x in multiply-accumulate energy. To study the advantages of the proposed architecture and its performance at the system level, we architect tinyForge, a design space exploration to obtain Pareto-optimal architectures and compare the trade-offs with respect to traditional approaches. tinyForge comprises (1) a parameterized template for memory hierarchies and compute fabric, (2) estimations of power, area, and latency for hardware components, (3) a dataflow optimizer for efficient workload scheduling, (4) a genetic algorithm performing multi-objective optimization to find Pareto-optimal architectures. We evaluate the performance of our proposed architecture on all of the MLPerf Tiny Inference Benchmark workloads, and the BERT-Tiny transformer model, demonstrating its effectiveness in lowering the energy per inference while addressing the introduced area overheads. We show the importance of storing all the weights on-chip, reducing the energy per inference by 7.5x vs. utilizing off-chip memories. Finally, we demonstrate the potential of the custom latch arrays and bit-serial digital compute arrays to reduce by up to 1.8x the energy per inference, 2.2x the latency per inference, and 3.7x the silicon area.
@inproceedings{GiordanoDLM24,
title = {TinyForge: A Design Space Exploration to Advance Energy and Silicon Area Trade-offs in tinyML Compute Architectures with Custom Latch Arrays},
author = {Massimo Giordano and Rohan Doshi and Qianyun Lu and Boris Murmann},
booktitle = {ASPLOS},
year = {2024},
doi = {10.1145/3620666.3651328}
}