Improving Effective Bandwidth through Compiler Enhancement of Global Cache Reuse

Chen Ding and Ken Kennedy


Reusing data in cache is critical to achieving high performance on modern machines, because it reduces the impact of the latency and bandwidth limitations of direct memory access. To date, most studies of software memory hierarchy management have focused on the latency problem in loops. However, today's machines are increasingly limited by the insufficient memory bandwidth--- latency-oriented techniques are inadequate because they do not seek to minimize the total amount of memory transfer of the whole program. To address the bandwidth limitation, this paper explores the potential for global cache reuse---that is, reusing data across whole program and over the entire data. In particular, the paper investigates a two-step global strategy. The first step fuses computations on the same data to enable the caching of repeated accesses. The second step groups data used by the same computation to effect contiguous access to memory. While the first step reduces the frequency of memory access, the second step improves its efficiency. The paper demonstrates the effectiveness of this strategy and shows how to automate it in a production compiler.
Download the paper in pdf or in postscript format.

Copyright notice