Caches
What is a Cache?
A cache is a small amount of fast memory that is used to hold the data
most frequently used by the processor. Since most programs exhibit
locality of reference, caches can be effective in hiding the latency
of memory accesses. Entries in a cache have two components. A tag
which indicates which address is currently being cached in that
particular cache entry and the data which is the value that the
address contains. A memory reference checks the tags in the cache
against the address of the reference. If a match is found then the
data is returned, otherwise data gets fetched into the cache from main
memory.
Important Issues in Cache Design
- How many words in a cache line? Multi-word cache lines take
advantage of spatial locality, and do not waste too much space on
tags. However they may suffer from higher replacement rate since
there are fewer entries for the same amount of cache space.
- How many cache lines in a set? Direct-mapped caches have a
single cache line in each set. This makes searching for a cache line
easy; it is a simple indexing operation. Fully associative caches
have every cache line in the same set. Searching for a cache line
implies searching the whole cache. Set-associative caches put a small
number of cache lines in a set. Searching for a cache line implies
finding the set it is in (indexing) and then searching within the set
for the cache line itself.
- What is the replacement policy? For direct-mapped caches this is
not an issue. Each address can only be associated with a single cache
line. For associative caches LRU usually performs best but is too
expensive to implement. Random replacement is usually what is
implemented.
- What is the write policy? Writes can be either write-through or
write-back. Write-through makes replacements cheaper (since nothing
needs to be written back when a line is replaced), at the expense of
extra traffic. Write-back seems to be the policy of choice with
modern processors.
How is a cache-lookup done?
As is shown in the above figure, the low order bits are used to select
a word within a multi-word cache line. For single-word cache lines no
bits need be used for this operations. The next set of bits selects
the cache entry (a set in the cache). The final set of bits is used
to perform the tag-check against the tags in the set.
Although these operations are shown to operate using bits from the
same address implementations can actually differ. For instance many
caches use the virtual address for selecting the set and the word
within a set (indexing), but the physical address for the tag checking
(tagging). Possible combinations include a) virtually-indexed
virtually-tagged caches, b) virtually-indexed physically-tagged
caches, and c) physically0indexed physically-tagged caches.
Virtually-tagged caches introduce the aliasing problem where two
virtual address may point to the same physical address and thus should
contain the same data. If both virtual addresses are cached writes to
one need to be propagated to the other in order to maintain the cache
consistent.