ML computers setup

Setting up ML computers:

• Linux distribution installed: Ubuntu 16.04

Partition made for machine learning:MachineLearning

• In Windows: Disk Management -> Resize DataStorage
• Create new ext4 patition

Mounting the partition automatically:

Get the UUID of the learning:MachineLearning partition

• sudo blkid

Add partition to fstab:

• sudo nano /etc/fstab
• Add at the bottom these two lines:
• UUID=(id of the MachineLearning partition) /media/MachineLearning rw,suid,dev,auto,user,async,exec 0 2
• UUID=(id of the DataStorage partition) /media/DataStorage ntfs-3g defaults=en_US.UTF-8 0 0

• The parameters mounting the MachineLearning partition solved this problem running caffe from that partition:
• https://github.com/rbgirshick/py-faster-rcnn/issues/162
• https://askubuntu.com/questions/678857/fstab-doesnt-mount-with-exec

Give writing permissions to new MachineLearning partition

• sudo chmod -R a+rwx /media/MachineLearning/

• Need extra space? Extending the partition

https://askubuntu.com/questions/492054/how-to-extend-my-root-partition

Installing NVIDIA Driver:

• Set Ubuntu to boot on console mode. Type:
• sudo apt-get install systemd
• sudo systemctl set-default multi-user.target
• sudo reboot now
• Login and in console mode, type:
• sudo add-apt-repository ppa:graphics-drivers/ppa
• sudo apt update
• sudo apt upgrade
• For GeForce 1070Ti (07/2018), type:
• sudo apt-get install nvidia-390
• Re-set Ubuntu to boot on graphical mode. Type:
• sudo systemctl set-default graphical.target
• sudo reboot now

Checking if Nvidia Driver is properly installed. Type:

• nvidia-smi
• nvidia-settings

Installing CUDA 9.0 for Ubuntu 16.04 (the latest version is not supported by TensorFlow):

• wget http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
• wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libcudnn7_7.0.5.15-1+cuda9.0_amd64.deb
• wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libcudnn7-dev_7.0.5.15-1+cuda9.0_amd64.deb
• wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libnccl2_2.1.4-1+cuda9.0_amd64.deb
• wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libnccl-dev_2.1.4-1+cuda9.0_amd64.deb
• sudo dpkg -i cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
• sudo dpkg -i libcudnn7_7.0.5.15-1+cuda9.0_amd64.deb
• sudo dpkg -i libcudnn7-dev_7.0.5.15-1+cuda9.0_amd64.deb
• sudo dpkg -i libnccl2_2.1.4-1+cuda9.0_amd64.deb
• sudo dpkg -i libnccl-dev_2.1.4-1+cuda9.0_amd64.deb
• sudo apt-get update
• sudo apt-get install cuda=9.0.176-1
• sudo apt-get install libcudnn7-dev
• sudo apt-get install libnccl-dev
• sudo reboot now
• export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}
• export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
• sudo nano .bashrc
• Add the two last export lines at the end of the file. Save and reboot.

Checking if CUDA is properly installed. Type:

• nvcc --version

Resources used:

• https://askubuntu.com/questions/61396/how-do-i-install-the-nvidia-drivers
• https://medium.com/@bbloks/a-machine-learning-environment-with-ubuntu-and-gpu-acceleration-in-5-steps-765608325356
• https://yangcha.github.io/CUDA90/

Other options:

NTFS fstab wizard:

• sudo apt-get install ntfs-config
• sudo ntfs-config

Format large capacity HD with fs ExFat for having access to it from Ubuntu:

• On Windows 10
• cmd
• diskpart
• select disk '#' (where # is the number of the target drive)
• list part
• select part # (where # is the number of the partition)
• format fs=exfat QUICK