Multiprocessors
Multiprocessor Classification
- SISD: Single Instruction stream, Single Data stream. This is the
classic uniprocessor.
- SIMD: Single Instruction stream, Multiple Data Stream. A set of
processors that execute in lockstep the same instructions on different
data. Examples of SIMD machines include the CM-2, the Maspar and
others.
- MIMD: Multiple Instruction Stream, Multiple Data Stream. A set
of processors that execute a different program on different data.
However since it is unintuitive to write parallel programs that
execute entirely different sequences of instructions on different
processors these machines most commonly execute the same program on
different data. However there is no need for lockstep execution of
instructions and processors could be askew in their execution. This
programming model is called SPMD (Single Program Multiple Data).
MIMD machines
The main problem that designers of parallel machines need to solve is
the way processors communicate with one another. The easiest way to
solve the problem is to assume explicit communication (i.e. the
processes use send and receive primitives to communicate with one
another). However message passing makes parallel programming
exceedingly hard so researchers have attempted to create parallel
machines that support shared memory. For small scale systems
bus-based multiprocessors are easy enough to build that they dominate
the market. Larger parallel shared memory machines are quite a bit
harder and only recently have we seen succesfull efforts in building
one.
The cache coherence problem
The cache coherence problem stems from the fact that processors copy
data in their local caches for performance purposes. Once the same
data is replicated in multiple caches a write into any of the copies
will make the other copies inconsistent. For bus-based
multiprocessors the problem can be solved using snooping. Snooping is
monitoring all activities that occur on the shared bus and modifying
cache state appropriately. The figure above shows the possible states
for a cache line and the transitions from state to state based on the
actions taken by a processor and the actions observed on the bus.
Scalable solutions cannot use snooping since scalable networks do not
provide broadcasting. Solutions to scalable shared memory machines
include directory solutions, where a portion of memory is allocated to
hold information about coherence blocks. The information includes the
state of the block and a list of sharing processors. Based on that
information a processor can enforce coherence by exchanging point to
point messages with the processors that are recorded in the sharing
list.
Types of cache misses
There are 4 types of cache misses in a parallel system:
- Cold-start misses: These are misses that are caused by the first
reference to a datum.
- True-sharing misses: These are misses to cache lines that have
been invalidated due to remote writes. Furthermore the data accessed
is that data that has been modified by the remote write(s).
- False-sharing misses: These are misses to cache lines that have
been invalidated due to remote writes. Furthermore the data accessed
is that data that has NOT been modified by the remote write(s).
- Capacity/Conflict misses: There are misses to cache lines that
have been evicted from the cache as a result of fetching other data
that replaced them.