There are scenarios other than message-passing environments where WT is an important phenomenon. In NUMA shared-memory machines, for example, we can observe waiting times in synchronization operations and in the memory hierarchy (e.g., remote memory accesses). Explanation techniques for WT in synchronization operations in this environment will be similar to the techniques used in distributed-memory machines. Waiting times in the memory hierarchy, however, can have many explanations, such as locality in the access to local data, sharing of data, data migration, and different communication network costs. We plan to evaluate the application of WTA to this environment, and adapt the technique to this scenario.
We also plan to continue the development of explanation techniques by focusing on another performance phenomenon, communication overhead, which is important in both message-passing and shared-memory systems. There are two basic questions that must be answered when understanding communication operations: (1) Why is the data necessary? and (2) Why isn't the data available locally? The first question is answered by identifying the program structures that use the data that is being transmitted. The second question is answered by verifying the history of the data, which includes previous communication operations and the data distribution.
To understand communication overhead, we start by focusing on each communication point (i.e., basic block that contains communication operations) and determining the characteristics of the data that is communicated and the communication explanations. Characteristics of the communicated data include the data structure (or portion of it), and processors involved in the communication (i.e., source, target). The explanations may be static or dynamic. Static explanations may be access patterns or data distribution. Dynamic explanations include the communication operation that previously transmitted the data to the processor that is sending it, which we call the previous transfer.
This explanation technique, which we call Communication Analysis (CA), requires some additional information for its implementation. For each code segment, we need to know the data that is accessed and must be available. For each communication operation (i.e., send or receive) we must provide, via instrumentation, the data structures that are communicated.
The starting point of CA is the trace file of the execution. We plan to traverse this trace file and for each communication operation we record a communication event that contains the processors involved, the basic blocks that contain the communication operations, the data that is communicated, and the previous transfer. As a next step, we intend to group the communication events into classes, using as our criteria the data accessed, basic blocks involved, and the class of the previous transfer. The last step of the analysis associates additional static information, such as pattern accesses and data distributions, with each communication class. The result of Communication Analysis is a set of communication classes, where each class is composed of its characteristics (i.e., data transmitted, source code, previous transfer) and explanations (static and dynamic). An example of the use of this information is to detect incompatibilities between the data distribution and the loop scheduling policy. By identifying the data structures and understanding the reasons for communication overhead, the programmer may change these implementation decisions and improve an application's performance.