There is a large number of performance phenomena in parallel computing, and it is virtually impossible to develop performance understanding techniques for all of them at once. To avoid this problem, we have to restrict our research to a working scenario, which must be relevant in parallel computing, and limited in scope.
We believe that the data-parallel computing paradigm fits these requirements. This paradigm can be characterized by three features [High Performance Fortran Forum1993]: (1) single-threaded control (i.e., all processes execute the same code), (2) a global name space, and (3) loosely synchronous execution. By these characteristics we mean that applications are single programs that operate on different instances of data (e.g., arrays whose elements are distributed across processors). We also call these applications SPMD (Single Program Multiple Data).
The number of products already developed (mainly compilers), active research groups [Richardson1995], and applications parallelized via this paradigm [Hawick and Havlak1995] justify our focus on performance understanding of SPMD programs. One consequence is that there are many compilers [Richardson1995] and static analysis tools [Faigin et al. 1994, Polychronopoulos et al. 1989] for data-parallel applications, and some of them are freely available.