Delta Send-Recv: Run-time Support for Dynamic Pipelining of Coarse-grained Computation and Communication

Pipelining is necessary for efficient do-across parallelism but non-trivial for MPI programs because it requires perfectly matched loop and message blocking in both the sender and the receiver code. Delta send-recv is a run-time system that uses virtual memory support to automatically divide computation and communication into increments to effect dynamic pipelining. A programmer marks the relevant computation and communication but does not restructure the computation code or compose multiple messages at the sender or the receiver. At run time, the system monitors the computation and non-blockingly sends out an increment when it has been computed. Multiple pipelines may be chained to obtain a higher speedup for more tasks.

A technical report can be downloaded here.