GitHub link

Instructions

Slides that include the design and experiment results

Reports that include the design and experiment results, more verbally.

Explain Code, refer to this diff

Makefile updated to support mpicxx compile
README.md remove the original README to point to the right documents
add-deps.sh an example to package all depedencies to run on a cluster
run_exp.sh scripts to run all experiments
run_mpi.sh script to run MPI
sync_cycle3.sh distribute data to clusters
mpi_env.csh example scripts to setup MPI environment
sync_cycle3.sh an example to sync with cluster machines
examples/mnist/lenet_train_test.prototxt network architecture to do random skip
examples/mnist/train_lenet.sh script to invoke the MPI program
examples/cifar10/cifar10_full_train_test.prototxt network architecture to do random skip
examples/cifar10/train_cifar10.sh script to invoke the MPI program
hostfile.exp there was a problem with the PBS system, so I could not use the PBS_NODEFILE to configure hosts. deprecated now
include/caffe/ParamServer.hpp The main parameter server implementation
  • Templated and Pluggable
  • spawn listener
  • update and send parameters
  • include/caffe/blob.hpp
  • src/caffe/blob.cpp
  • src/caffe/net.cpp
  • src/caffe/solver.cpp
modify the update function to update with parameter server
  • call parameterserver::Update
  • syncing before snapshotting
  • include/caffe/common.hpp
  • src/caffe/common.cpp
initialize the parameter server in the singleton
tools/caffe.cpp initialization:
  • regiester parameters
  • barrier and syncing to start from the same place