When writing Makefiles, model it based on the the machine specific makefiles (e.g., Makefile.niagara1) provided as part of the 1st assignment. To use a specific makefile type make -f Makefile. < machine >. These makefiles include machine specific compiler options. For example, the niagara machine runs Solaris SunOS 5.10, which recommends that programs pass the -mcpu=v9 flag to the compiler.
The turn-in script will recursively copy all files and
subdirectories of your current directory to a location in the
course's account. Please avoid running it at your home directory or
any other directory that contains files irrelevant to the current
project. For any problems while using it please contact the TA with
an email (sending the script's output will help).You may turn-in your
project as many times as you need. However, only the MOST RECENT
submission will be graded. DO NOT FORGET to submit a README file
(text or pdf format) with your code, and include your name and e-mail
address in the README file. Check the appropriate assignment page for
information related to the contents of the README file.
INSTRUCTIONS:
*
From the grad network:
1) Go to the directory that contains
ALL the files that are related
to your current project.
2) Run the script:
/u/cs458/bin/turnin
Upon
successful completion you will get an output of the form:
File Submission completed successfully at: [CURRENT DATE]
Where CURRENT DATE represents the current date and time. You
will also receive through email a message with subject
``Submission Notification'', which will contain a listing of all
the files you submitted along with their last modification date
and size.
In this assignment you'll be working with three different programming models (1) MPI, (2) OpenMP, (3) Cilk. Besides the difference between message passing and shared memory, MPI is a library interface whereas OpenMP and Cilk consist of a set of directives that need special compiler support. Although both of them are language independent, we'll be focusing on C/C++. In this assignment, your working environment will be on any of the following department machines.
# Set the default home directories for MPI and Cilk
setenv MPI_HOME /u/{cs258 or cs458}/installed/mpi/3.0.2/x86
setenv CILK_HOME /u/{cs258 or cs458}/installed/cilk/5.4.6/cilk-5.4.6
if (`uname -i` == "x86_64") then
setenv MPI_HOME /u/cs458/installed/mpi/3.0.2/x86_64
setenv CILK_HOME /u/cs458/installed/cilk/5.4.6/cilk_x86_64-5.4.6
endif
# if your target machine is niagara1, fix the home paths on GCC, MPI, and
Cilk.
if ( `hostname` == "niagara1.cs.rochester.edu" ) then
setenv GCC_HOME /u/{cs258 or cs458}/installed/gcc/4.2.0/niagara
setenv MPI_HOME /u/{cs258 or cs458}/installed/mpi/3.0.2/niagara
setenv CILK_HOME /u/{cs258 or cs458}/installed/cilk/5.4.6/cilk_niagara-5.4.6
endif
# set MPI binary and library paths
setenv PATH ${MPI_HOME}/bin:${PATH}
if ( "$?LD_LIBRARY_PATH" ) then
setenv LD_LIBRARY_PATH ${MPI_HOME}/lib:${LD_LIBRARY_PATH}
else
setenv LD_LIBRARY_PATH ${MPI_HOME}/lib
endif
# set cilk binary and library paths
setenv PATH ${CILK_HOME}/support:${PATH}
setenv LD_LIBRARY_PATH ${CILK_HOME}/lib:${LD_LIBRARY_PATH}
Tool-hardware compatibility is an important issue. We have set up a variety of compiler-hardware setups at /u/cs2(or4)58/installed. Each specific compiler needs you to set up the PATH and LD_LIBRARY_PATH variables.
gcc supports openmp from 4.1.2 on the x86-Linux but only from 4.2 on Sparc-Solaris. 4.1.2 is the default compiler on cycle1. To use 4.2 on the sun boxes; on sync and swym it is installed under /usr/grads/share/gcc-4.2.0; on niagara1 it is installed under /u/cs4(or 2)58/installed/gcc/4.2.0/niagara. For node4x2a it is installed under /u/cs4(or 2)58/installed/gcc/4.2.0/x86_64
gcc -fopenmp -o hello hello.c
MPI is a message-passing library in which the programmer is expected to explicitly specify the communication and transfer of data between execution threads. The data communication can be implemented using a number of underlying protocols, shmem (shared memory on niagara,sync,node4x2a,discovery) or TCP/IP (cluster machines node33-64).
The MPI versions have been set up at, /u/cs(2 or 4)58/installed/mpi/3.0.2/ < machine > where < machine > is niagara or x86.
Basically, like gcc, mpicc performs both compilation and linking. A sample Makefile and program can be found at /u/cs458/apps/test_mpi. You may use the directory as a template and plug in your own programs. You also need to specify the include path to help mpicc find the header "-I/u/cs458/$MPI_HOME/include/".
Take a look at MPI tutorial from Lawrence Livermore National Lab.
mpirun -npnp specifying the number of processors to use and the machine file listing the processors to be selected from.-machinefile
NOTE: On the graduate network, any jobs submitted to the cluster nodes ranging from node33 to node64 are subject to centralized management. For detail instructions, please refer to ClusterJobs. In short, you need to wrap your jobs in a shell script and then pass it as an argument to qsub. Here's a script that could be used in submitting your MPI jobs.
#!/bin/csh source ~/.cshrc.mpi cd $PBS_O_WORKDIR #set your machinefile cat $PBS_NODEFILE > ./cluster_nodes #count how many nodes you reserve #run mpi on these nodes mpirun -np `cat ./cluster_nodes | wc -l` -machinefile ./cluster_nodes ./linux/ < your-mpi-program > > ./mpi.outNow you ssh to node64.cs.rochester.edu, enter your mpi working directory and then copy and fix run.sh. To run your mpi program, just type qsub -l nodes=4 script.sh
Cilk is a set of language extensions that allow users to specify parallelism points without worrying about threads or scheduling issues. The cilkc program that you can use to compile .cilk programs into executables.
We have installed three different versions of the cilk system: i386 /u/cs2(or 4)58/installed/cilk/5.4.6/x86 (note: on i386 machines, not x86_64 or node4x2a), niagara /u/cs2(or 4)58/installed/cilk/5.4.6/niagara, and /u/cs2(or 4)58/installed/cilk/5.4.6/x86_64/.
Here's an example to set the path.
if ( `hostname` == cycle1.cs.rochester.edu ) then
setenv CILK_HOME /u/cs458/installed/cilk/5.4.6/x86
endif
if ( `hostname` == cycle1.csug.rochester.edu ) then
setenv CILK_HOME /u/cs258/installed/cilk/5.4.6/x86
endif
if ( `hostname` == niagara1.cs.rochester.edu ) then
setenv CILK_HOME /u/cs458/installed/cilk/5.4.6/niagara/
endif
if ( `hostname` == niagara2.cs.rochester.edu ) then
setenv CILK_HOME /u/cs258/installed/cilk/5.4.6/niagara/
endif
if ( `hostname` == sync.cs.rochester.edu ) then
setenv CILK_HOME /u/cs458/installed/cilk/5.4.6/sync/
endif
setenv PATH ${CILK_HOME}/bin:${PATH}
setenv LD_LIBRARY_PATH ${CILK_HOME}/lib:${LD_LIBRARY_PATH}
Similar to gcc, cilkc takes in a number of parameters substitutes the cilkc annotations with C routines and subsequently uses the gcc to generate the executable. Refer section 2.2 (page 7) in the cilk manual Cilk Manual"
Example: cilkc -O2 hello.cilk -o helloWe have also included a cilk2c program that performs a source-source conversion of cilk programs to .c programs.