The installed base compilers on the Sun/Sparc (i.e., swym,sync,niagara1) machines are typically a few generations old. To use our tested versions please include the following blurb in your .cshrc file in your home directory. You may ask what does it do ? Essentially, it gets the compiled program to use gcc 4.1.1 libraries and binary when compiling.
if ($?prompt) then
if (`uname` == "SunOS") then
setenv PATH /usr/grads/share/gcc/bin:${PATH}
setenv LD_LIBRARY_PATH /usr/grads/share/gcc/lib:${LD_LIBRARY_PATH}
endif
endif
You might have found out by now that the niagara2 boxes lack the appropriate make, compiler and other build files. This is because the sysadmins haven't set up the default .cshrc file yet, like on the grad niagara1. Typically you will find at the top of your .cshrc the following source /usr/staff/lib/defaults/system.cshrc The system.cshrc is machine specific so Sun boxes get something different from intel.
To work around, I have installed a custom gcc-4.1.1 at ~/cs258/installed/gcc-4.1.1 I have shared it with you folks. To set up the appropriate build files, include the following in your .cshrc
if ($?prompt) then
if (`uname` == "SunOS") then
setenv PATH /u/cs258/installed/gcc-4.1.1/bin/:/usr/ccs/bin:/usr/bin:/usr/sbin/:"$PATH"
setenv LD_LIBRARY_PATH /u/cs258/installed/gcc-4.1.1/lib:/usr/staff/lib
endif
endif
(Important: Also comment out the "source /usr/staff" at the top of the .cshrc. Commenting out means prefix the statement with a "#". Why, because otherwise the rest of the .cshrc wont load)
I have also created a new makefile for niagara2. Makefile.niagara2 This file exists only for the sor app. gauss doesn't include any machine specific specifications. Although, you can include this flag if you like. This includes -mcpu=v9 option which applies the SPARC v9 ISA specific optimizations.
When writing Makefiles, model it based on the the machine specific makefiles (e.g., Makefile.niagara1/2) provided as part of the 1st assignment. To use a specific makefile type make -f Makefile. < machine >. These makefiles include machine specific compiler options. For example, our nigara's run Solaris SunOS 5.10 which recommend programs to pass the -mcpu=v9 flag to the compiler.
The turn-in script will recursively copy all files and
subdirectories of your current directory to a location in the
course's account. Please avoid running it at your home directory or
any other directory that contains files irrelevant to the current
project. For any problems while using it please contact the TA with
an email (sending the script's output will help).You may turn-in your
project as many times as you need. However, only the MOST RECENT
submission will be graded. DO NOT FORGET to submit a README file
(text or pdf format) with your code, and include your name and e-mail
address in the README file. Check the appropriate assignment page for
information related to the contents of the README file.
INSTRUCTIONS:
*
From the grad network:
1) Go to the directory that contains
ALL the files that are related
to your current project.
2) Run the script:
/u/cs458/bin/turnin
Upon
successful completion you will get an output of the form:
File Submission completed successfully at: [CURRENT DATE]
Where CURRENT DATE represents the current date and time. You
will also receive through email a message with subject
``Submission Notification'', which will contain a listing of all
the files you submitted along with their last modification date
and size.
In this assignment you'll be working with four different programming models (1) MPI, (2) OpenMP, (3) Cilk. Besides the difference of message passing and shared memory, MPI is a library interface whereas OpenMP and Cilk consists of a set of directives that need special compiler support. Although both of them are language independent we'll be mostly targeting them on C/C++. In this assignment your working environment will be on any of the following department machines.
Obviously, tool-hardware compatibility is an important issue. We have set up a variety of compiler-hardware setups at /u/cs2(or4)58/installed. Each specific compiler needs you to set up the PATH and LD_LIBRARY_PATH variables.
gcc supports openmp from 4.1.2 on the x86-Linux but only from 4.2 on Sparc-Solaris. 4.1.2 is the default compiler on cycle1. To use 4.2 on the sun boxes; sync and swym its installed under /usr/grads/share/gcc-4.2.0. . For niagara1 its installed under /u/cs4(or 2)58/installed/gcc/4.2.0/niagara. For node4x2a its installed under /u/cs4(or 2)58/installed/gcc/4.2.0/x86_64 To set-up use some script like the following. Add a specific if for each machine.
if ( `hostname` == sync.cs.rochester.edu ) then
setenv GCC_HOME /usr/grads/share/gcc-4.2.0
setenv PATH ${GCC_HOME}/bin:${PATH}
setenv LD_LIBRARY_PATH ${GCC_HOME}/lib:${LD_LIBRARY_PATH}
endif
gcc -fopenmp -o hello hello.c
MPI is a message-passing library in which the programmer is expected to explicitly specify the communication and transfer of data between execution threads. The data communication can be implemented using a number of underlying protocols, shmem (shared memory on niagara,sync,node4x2a,discovery) or TCP/IP (cluster machines node33-64).
The MPI versions have been set up at, /u/cs4(or 2)58/installed/MPI/1.2.7/ < machine > where < machine > is discovery,niagara,x86_64(node4x2a). On sync/swym filesystem you will find it at /u/cs458/installed/mpi/1.2.7/sync/ . For the node machines you will find it at /u/cs458/installed/mpi/1.2.7/nodes. UGrads and Non-CSers for your i386 machines (cycle and similar) use /u/cs258/installed/mpi/1.2.7/nodes, for niagara2 use /u/cs258/installed/mpi/1.2.7/niagara. Similar to gcc use some script like the following in your .cshrc.
foreach n (`seq -s " " 33 64`)
if (`hostname` == node$n.cs.rochester.edu) then
setenv MPI_HOME /u/cs458/installed/mpi/1.2.7/nodes
endif
end
if ( `hostname` == sync.cs.rochester.edu ) then
setenv MPI_HOME /u/cs458/installed/mpi/1.2.7/sync/
endif
setenv PATH ${MPI_HOME}/bin:${PATH}
setenv LD_LIBRARY_PATH ${MPI_HOME}/lib:${LD_LIBRARY_PATH}
Basically, like gcc, mpicc could run the errands of both compilation and linking. A sample Makefile and program can be found at /u/cs458/test_mpi. You may use the directory as a template and plug in your own programs. You also need to specify the include path to help mpicc find the header "-I/u/cs458/$MPI_HOME/include/".
Take a look at MPI tutorial from Lawrence Livermore National Lab.
mpirun -npnp specifying the number of processors to use and the machine file listing the processors to be selected from.-machinefile
NOTE: On the graduate network, any jobs submitted to the cluster nodes ranging from node33 to node64 are subject to centralized management. For detail instructions, please refer to ClusterJobs. In short, you need to wrap your jobs in a shell script and then pass it as an argument to qsub. Here's a script that could be used in submitting your MPI jobs.
#!/bin/csh source ~/.cshrc.mpi cd $PBS_O_WORKDIR #set your machinefile cat $PBS_NODEFILE > ./cluster_nodes #count how many nodes you reserve #run mpi on these nodes mpirun -np `cat ./cluster_nodes | wc -l` -machinefile ./cluster_nodes ./linux/ < your-mpi-program > > ./mpi.outNow you ssh to node64.cs.rochester.edu, enter your mpi working directory and then copy and fix run.sh. To run your mpi program, just type qsub -l nodes=4 script.sh
Cilk is a set of language extensions that allow users to specify parallelism points without worrying about threads or scheduling issues. The cilkc program that you can use to compile .cilk programs into executables.
We have installed three different versions of cilk system: i386 /u/cs4(or 2)58/installed/cilk/5.4.6/x86 (note i386 machines not x86_64 or node4x2a), Niagara /u/cs4(or 2)58/installed/cilk/5.4.6/niagara and sync (only on Grad side) /u/cs458/installed/cilk/5.4.6/sync/.
Here's an example to set the path.
if ( `hostname` == cycle1.cs.rochester.edu ) then
setenv CILK_HOME /u/cs458/installed/cilk/5.4.6/x86
endif
if ( `hostname` == cycle1.csug.rochester.edu ) then
setenv CILK_HOME /u/cs258/installed/cilk/5.4.6/x86
endif
if ( `hostname` == niagara1.cs.rochester.edu ) then
setenv CILK_HOME /u/cs458/installed/cilk/5.4.6/niagara/
endif
if ( `hostname` == niagara2.cs.rochester.edu ) then
setenv CILK_HOME /u/cs258/installed/cilk/5.4.6/niagara/
endif
if ( `hostname` == sync.cs.rochester.edu ) then
setenv CILK_HOME /u/cs458/installed/cilk/5.4.6/sync/
endif
setenv PATH ${CILK_HOME}/bin:${PATH}
setenv LD_LIBRARY_PATH ${CILK_HOME}/lib:${LD_LIBRARY_PATH}
Similar to gcc, cilkc takes in a number of parameters substitutes the cilkc annotations with C routines and subsequently uses the gcc to generate the executable. Refer section 2.2 (page 7) in the cilk manual Cilk Manual"
Example: cilkc -O2 hello.cilk -o helloWe have also included a cilk2c program that performs a source-source conversion of cilk programs to .c programs.