Installing Pytorch with Cuda on a 2012 Macbook Pro Retina 15

The best laptop ever produced was the 2012-2014 Macbook Pro Retina with 15 inch display. It has a Cuda-capable GPU, the NVIDIA GeForce GT 650M. This GPU has 384 cores and 1 GB of VRAM, and is cuda capability 3. Although puny by modern standards, it provides about a 4X speedup over the cpu for Pytorch, and is fine for learning Pytorch and prototyping. If you have a newer MacBook Pro you are out of luck, because it either has a Radeon GPU or none at all.

The standard Mac distribution of Pytorch does not support cuda, but it is supported if you compile Pytorch from source.  There are numerous preliminary steps and "gotchas".  Here is what you need to do.
  1. Check that you are running Mac OS X High Sierra (10.13.6).  If you have an older version, upgrade. If you have a newer version you will need to downgrade.  Apple banished Cuda compatibility with Mojave. Downgrading OS X requires creating a bootable USB memory stick installer and erasing your laptop's hard disk
  2. Check that you have installed Xcode version 9.4.1.  If you have a newer version or none at all, download it from the Apple Developer site. Rename any other version of Xcode you have installed, and then copy it to /Applications.  Open it, and under preferences, select the 9.4.1 command line tools.  Close Xcode and open a terminal.  Run
    xcode-select --install
    to reinstall the command line tools, because sometimes the Xcode application fails to install certain header files.
  3. Install the NVIDIA Quadro and Geforce OS X Driver 387.10.10.10.40.118.
  4. Install the NVIDIA Cuda driver 410.130.
  5. Add to your .profile and reboot:
    export PATH=/Developer/NVIDIA/CUDA-10.0/bin${PATH:+:${PATH}}
    export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:$DYLD_LIBRARY_PATH
  6. Install NVIDIA cuDNN 7.4
  7. Install Anaconda with Python 3.6 as default. Create an environment named ptc that includes pip, activate it, and install libraries:
    conda create --name ptc python=3.6 pip
    conda activate ptc
    conda install numpy pyyaml mkl mkl-include setuptools cmake cffi typing
    Whenever you reboot, don't forget to activate ptc again before executing the following steps.
  8. Download Pytorch version 0.4.1 source and compile it.  It is important that you set and export CMAKE_PREFIX_PATH as described. Compilation should complete with many warning messages but no errors.
    conda activate ptc
    git clone --recursive https://github.com/pytorch/pytorch
    cd pytorch
    export CMAKE_PREFIX_PATH=~/anaconda3/envs/ptc
    MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py install
  9. Install torchvision:
    pip install torchvision
  10. Reboot and test that pytorch with Cuda is working.
    conda activate ptc
    python
    import torch
    torch.cuda.is_available()
    If python does not print "true", something has gone wrong.
  11. Install ptc as a kernel for jupyter notebooks.
    conda deactivate
    conda install ipykernel
    python -m ipykernel install --user --name ptc --display-name "Python 3.6 (ptc)"
  12. When a program first invokes Cuda, the following warning will be printed, but should be ignored - Cuda will indeed work!  In order to eliminate the message, edit the file ~/anaconda3/envs/ptc/lib/python3.6/site-packages/torch/cuda/__init__.py and in the definition of the function _check_capability() eliminate the string "capability==(3,0) or".
    Found GPU0 GeForce GT 650M which is of cuda capability 3.0.
    PyTorch no longer supports this GPU because it is too old.
    The minimum cuda capability that we support is 3.5.
  13. Download pytorch examples and compare time required with and without cuda.
    git clone https://github.com/pytorch/examples
    cd examples/mnist
    conda activate ptc
    time python main.py >/dev/null
    real 1m38.430s
    user 2m6.163s
    sys 0m7.762s
    time python main.py --no-cuda >/dev/null
    real 5m47.750s
    user 37m22.609s
    sys 1m23.813s
    For the non-cuda case, user time is greater than real time because Pytorch makes use of all 8 cpu hyperthread cores.
Congratulations, you are ready to set the deep learning world on fire!

- Henry Kautz, 10 February 2019

Update May 2020: These instructions do not work for Pytorch 1.0. Furthermore, large models crash Pytorch when the GPU is enabled. The best option today is to use the latest pre-compiled CPU-only Pytorch distribution for initial development on your MacBook and employ a linux cloud-based solution for final development and training.