Introduction

Typically communication is a major bottleneck in parallel systems, and reducing communication time leads to great performance enhancement. To improve the communication architecture and the mapping of tasks to physical nodes we need to analyze the communication pattern of an MPI program. There are two major existing approaches to this problem. A lightweight lossy approach presents highly aggregated information. The other approach is a lossless technique, which stores the entire communication sequence with timing information as is. The goal of MPI trace in the approach taken by Noeth et. al [1] is to store the entire communication trace sequence and compress it. The compression is a form of real time compression and enables the trace systems to do the compression and more importantly the decompression without having to read the entire file or seeking far forward or backward into the file. This is a very important property as it enables both the tracing and the replay to be done without consuming an excess of exiting resources.

In this project I attempt to modify the existing system to accommodate a feature which was left out in the original version. In the original version they attempt only to store the sequence of MPI calls. The compression and replay systems are based on that. This however is lossy in the sense of the intermittent computation and communication wait times. These timing values can be critical to determine if load balancing is achieved in the communication. I attempt to modify the system to enable the developer to replay the actual communication with timing information. This could lead to observations which would enable developers to make the system more efficient. One could for example observe that a link is a bottleneck during a communication playback, and may redistribute workload to stagger communication requests on the link thus improving efficiency.

The objectives may be summarized as follows:

1. store timing information for every MPI call
2. extend current compression mechanism to aggregate timing information
3. output timing information in trace file(s) to allow for replay using timing information

In this paper only the changes made to the record side of the code are presented.

The layout of the paper is as follows: we review the existing code, explain its relevant architecture and explain the locations in which the requisite code changes need to be made. Finally the changes made in the code are discussed.