MPI Instructions

Instructions about available machines can be found at the Experimental machines section of the class web page.

In order to build and test your MPI programs on the experiment machines, there are a few steps you need to follow.

  1. Set up environment variables
    Several MPI versions have been installed at different directories. You need to choose the right version depending on the test machine. If you use the Bash shell or one of its variants, the following shell script should select the right MPI version. For automatic set-up, you can add the script to .bashrc in your home directory.
     
    	ARCH=`uname -m`
    
    	MPI_HOME=''
    	if [ "${ARCH}" == "i686" ]
    	then
    		# echo "Setting up x86 MPI env"
    		MPI_HOME=/u/cs458/installed/mpi/3.0.2/x86
    	elif [ "${ARCH}" == "x86_64" ]
    	then
    		# echo "Setting up x86-64 MPI env"
    		MPI_HOME=/u/cs458/installed/mpi/3.0.2/x86_64
    	else
    		# echo "Architecture ${ARCH} not recognized; cannot set up MPI env"
    		exit 1;
    	fi
    	LD_LIBRARY_PATH=${MPI_HOME}/lib:${LD_LIBRARY_PATH};
    	PATH=${MPI_HOME}/bin:${PATH}
    
    	unset MPI_HOME
    	unset ARCH
    	export LD_LIBRARY_PATH
    	export PATH
              
    If you don't want to make permanent changes to your environment variables, you can set-up a script that makes temporary changes before you run compile and run your MPI program. A related Bash script can be found in the sample MPI version of SOR at /u/cs458/apps/sor/mpi/mpi_test.sh. If you do not use Bash or you want to have full control of your shell environment, please feel free to do it in your own way. In that case, reading the above script should at least help you figure out what you need to do.

  2. Set up safe remote connection
    When running MPI on multiple machines, you need to be able to safely and instantaneously sign in to a remote server and start your application. To do this safely, we use Public Key Authentication and password-less SSH logins. If you have already set this up for your account following some online tutorial, there is nothing need to do for this step. If not, you can follow the instructions below: That 's it. Thanks to NFS, you should be able to sign in to any of the machines we use for the MPI assignment without being asked for a password. You should test this yourself; just try to sign in from cycle1 to cycle2; if you don't get asked for a password, you did it! Make sure to sign in at least once to the machines you want to use, such that the authentication routine has saved server information and you won't need to be delayed to accept it. If you don't like this setup, you can delete your ~/.ssh directory after the class.

  3. Compile/link your program
    Instead of gcc, you should use mpicc to compile and link your MPI programs. A sample Makefile can be found in the sample MPI version of SOR at /u/cs458/apps/sor/mpi/Makefile . You can follow this makefile to compile and link your own programs. Run make clean; make to compile your code.
    NOTE: if you compile your program for an x86_64 architecture, you should run it on x86_64 machines - or similarly for x86 machines. If you want to combine machines of different architectures, you must take extra set-up steps which we will not try to support; it is up to you.

  4. Run your program
    You need to create a file to indicate the available processors for running MPI programs. The file contains the names of the machines and the number of available processors on each machine. A sample file (hosts) can be found in /u/cs458/apps/sor/mpi/. For a specific example, if your system contains two machines (node17/node23) with two processors on each machine, then your file may look like this:
            
    	node17.cs.rochester.edu:2
    	node23.cs.rochester.edu:2
              
    You can run your MPI program in the following way:
    	mpirun -np 4 -machinefile hosts linux/sor
              
    Flag -np is followed by the number of MPI processes in the run. Flag -machinefile is followed by the hosts file described above.