Experiment Machines and Policies

There are several groups of machines you may use for assignments/project in this course. Note that some of these machines are heavily used for research purposes, so it is important to follow the policies when using them.

MPI Instructions

In order to build and test your MPI programs in the experiment machines, there are a few steps you need to follow.

  1. Set up environment variables
    Several MPI versions have been installed at different directories. You need to choose the right version depending on the test machine. If you use C-shell or one of its variants, the following shell script should select the right MPI version. For automatic setup, you can add the script to .cshrc in your home directory.

            # if you are using nodes
            foreach n (`seq -s " " 17 72`)
              if (`hostname` == node$n.cs.rochester.edu || `hostname` == node$n) then
                setenv MPI_HOME /u/kshen/install/mpich-1.2.6-install
              endif
            end
    
            # if you are using node4x2a
            if (`hostname` == node4x2a.cs.rochester.edu || `hostname` == node4x2a)  then
              setenv MPI_HOME /u/cs458/installed/mpi/1.2.7/x86_64
            endif
    
            setenv PATH ${MPI_HOME}/bin:${PATH}
            if ( "$?LD_LIBRARY_PATH" ) then
              setenv LD_LIBRARY_PATH ${MPI_HOME}/lib:${LD_LIBRARY_PATH}
            else
              setenv LD_LIBRARY_PATH ${MPI_HOME}/lib
            endif
              
    If you do not use C-shell or you want to have customized control of your shell environment, please feel free to do it in your own way. In that case, reading the above script should at least help you figure out what you need to do.

  2. Specify RSH connection
    When running MPI on multiple nodes, you need to specify a host file for the RSH connections which MPI needs. You need to create a .rhosts file in your home folder, specifying the host machines you will start mpirun. (Say you will start mpirun in node17 to run on node19, node20 ..., you need to add node17 to this file.) After you create the .rhosts file, remember to change the permission of the file to 600 by running chmod 600 .rhosts.

  3. Compile/link your program
    Instead of gcc, you should use mpicc to compile and link your MPI programs. A sample Makefile can be found in the sample MPI version of SOR at /u/cs458/apps/sor/mpi/. Run make clean; make to compile your code on any of the experiment platforms we are using. You can follow this makefile to compile and link your own programs.

    NOTICE: if you are testing on higher nodes (node33-node72), you should compile your program on lower nodes (node17-node28).

  4. Run your program
    You need to create a file to indicate the available processors for running MPI programs. The file contains the names of the machines and the number of available processors on each machine. A sample file (hosts) can be found in /u/cs458/apps/sor/mpi/. For a specific example, if your system contains two machines (node17/node19) with two processors on each machine, then you file may look like this:

            node17.cs.rochester.edu:2
            node19.cs.rochester.edu:2
              
    You can run your MPI program in the following way:
            mpirun -np 2 -machinefile hosts linux/sor
              
    Flag -np is followed by the number of MPI processes in the run. Flag -machinefile is followed by the hosts file described above.