Increasingly sophisticated low-power microcontrollers are at the heart of millions of IoT and edge computing deployments, with developers pushing large-scale data collection, processing, and inference to end nodes. Advanced workloads on resource-constrained systems depend on emerging technologies to meet performance and lifetime demands. High-performance Non-Volatile RAMs (NVRAMs) are one such technology enabling a new class of systems previously made impossible by memory limitations, including ultra-low-power designs using program state non-volatility and sensing systems storing and processing large blocks of data. Unfortunately, existing NVRAM significantly underperforms SRAM's access latency/energy cost and flash's read performance---condemning systems dependent on NVRAM to pay a steep energy and time penalty for software execution. We observe that this performance penalty stems predominately from instruction fetches into NVRAM, which represent >75% of memory accesses in typical embedded software. To eliminate this performance bottleneck, we propose SwapRAM, a new operating model for NVRAM-based platforms which repurposes underutilized SRAM as an instruction cache, maximizing the proportion of accesses directed towards higher-performance SRAM. SwapRAM consists of a set of compile-time code transformations and a runtime management system that transparently and dynamically copies code into SRAM throughout execution, with an extensible logic to delay eviction of hot code. Across nine embedded benchmarks running on a real FRAM platform, SwapRAM's software-based design increases execution speed by up to 46% (average 26%) and reduces energy consumption by up to 36% (average 24%) compared to a baseline system using the existing hardware cache.

@inproceedings{WilliamsH24a,
  title = {A Software Caching Runtime for Embedded NVRAM Systems},
  author = {Harrison Williams and Matthew Hicks},
  booktitle = {ASPLOS},
  year = {2024},
  doi = {10.1145/3622781.3674183}
}