We can divide the inference process of understanding the performance of parallel programs into three steps: (1) identify performance phenomena to be explained, (2) determine where these phenomena occurred, and (3) explain these phenomena, via cause-effect relations that link phenomena to implementation decisions.
In the previous sections, we described many observation and modeling techniques and found that understanding program performance using these techniques is a difficult, if not impossible, task. The difficulty is explained by the fact that these tools are very focused, and do not provide adequate information for performance understanding.
We can distinguish two dimensions of focus:
For observation techniques, we have seen that the measurements are either too abstract (i.e., summary statistics) or too detailed (i.e., execution events). Note that Paradyn [Miller et al. 1995] is an exception, since it allows hierarchical composition of events.
We can observe the same problem for understanding models generated by analytical techniques and their derived decompositions. Pure analytical models are too high level for understanding of performance behavior; models obtained by decomposition are usually simpler, but many interactions among the model pieces are not accounted when generating the model and the understanding of the application's overall behavior is very difficult.
Thus, for performance understanding, it is necessary that both observation and modeling are performed at a variable level of abstraction.
We can divide performance information into two groups according to their source: static and dynamic. We define static information as any information obtained without an application execution. On the other hand, dynamic information is defined as any performance data gathered during an application's execution.
The need to consider both static and dynamic information in performance understanding comes from the inference process itself, which links implementation decisions (static) to performance phenomena (dynamic).
In observation tools, which are based on dynamic information, it is necessary to have access to static information in order to perform the inference process. The integration of Fortran-D and Pablo [Adve et al. 1994]is an example of a tool that provides both dynamic and static information, but the understanding process is still performed by the user.
The modeling techniques presented focus on either static or dynamic information. More specifically, analytical techniques focus on dynamic information and structural techniques on static information.
These approaches are complementary in terms of model generation. Information that is usually required from the user in analytical approaches is obtained automatically in structural approaches. Sources of inaccuracies in the structural approaches are better handled in the analytical approaches, in many cases via decomposition.
For performance understanding, analytical and structural approaches are also complementary. Analytical techniques require that the user provide knowledge of the parallel system in generating models. Structural approaches can explain automatically parallel behavior, but their use for understanding is limited because their static models of behavior do not consider dynamic phenomena. Thus, structural approaches provide partial understanding of parallel program behavior, but analytical models are necessary for explaining dynamic phenomena and complete the inference process.
The focused approach adopted by many tools is not the unique reason for the difficulty in understanding performance phenomena, since these tools also lack explanation techniques. In order to overcome the first problem, we describe in the next chapter a framework for performance understanding, Carnival . In Chapter 4 we present an explanation technique developed using Carnival resources, Waiting Time Analysis, that is one example of a solution for the second problem. We also discuss the role of auxiliary tools in performance understanding by presenting the Lost Cycles Toolkit in Chapter 5.