New Paradigms in Cache Architectures

The memory wall continues to be a major performance bottleneck for modern processors. Caching has been well recognized as a powerful solution to filter requests going to the main memory and boost performance. Unsurprisingly, the general industry trend has therefore been to increase the amount of cache that is put on die. In fact, recent advances in technologies like embedded DRAM (eDRAM) and high bandwidth memory (HBM) have even allowed building very large multi-MB to GB caches. Research in cache subsystems is driven by the conventional wisdom that increasing the hit rate in the cache subsystem improves average latency and bandwidth and hence gains performance.
At the Micro-Architecture Research Lab, Intel India, we have been fundamentally re-investigating caching, and have shown that there is more to cache design than just increasing cache hit rates. In this talk we will discuss two of our recent works in this direction.
The first part of the talk is on efficient management of large HBM or eDRAM caches. We show that blindly increasing hit rates for such large caches can in fact be detrimental to performance. Instead we develop a dynamic access partitioning scheme that sacrifices some amount of hit rate in order to tap the main memory bandwidth and increase overall performance. This work was adjudged as the best paper at HPCA 2017.
In the second part of the talk, we will do a fundamental re-analysis of the popular three level cache hierarchy and show that it is not the most efficient way of designing on-die caches. Instead we will show that at a much lower area, a two level cache hierarchy aided with a learning of the program behavior, can be more efficient. This presents interesting power-performance-area trade-offs for cache designers. This work was published at ISCA 2018.
Dr. Shankar Balachandran is a Research Scientist at the Intel Microarchitecture Research Lab at Bangalore, India. His primary area of research is in core microarchitecture, focusing on the memory and caching subsystem at present. He received his PhD in Electrical Engineering from the University of Texas at Dallas in 2005. He was on the faculty of Computer Science and Engineering at the Indian Institute of Technology Madras (IIT Madras) from 2005 to 2016, where his research focused on design automation, VLSI design, and multicore programming.