Difference between revisions of "Docker"

Revision as of 14:21, 25 July 2018

ML Docker Image installed on the Interaction Station ML computers:
Deepo. It includes:

Run Deepo image with Docker:

Setting up ML computers:

Partition made for machine learning:MachineLearning

Mounting the partition automatically:

Add partition to fstab:
sudo nano /etc/fstba
Add at the bottom these two lines:
UUID=(id of the MachineLearning partition) /media/MachineLearning ext4 1 2
UUID=(id of the DataStorage partition) /media/DataStorage ntfs-3g defaults=en_US.UTF-8 0 0

Give writting permissions to new MachineLearning partition

Installing NVIDIA Driver:

Checking if Nvidia Driver is properly installed. Type:

Installing CUDA 9.0 for Ubuntu 16.04 (the latest version is not supported by TensorFlow):

wget http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libcudnn7_7.0.5.15-1+cuda9.0_amd64.deb
wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libcudnn7-dev_7.0.5.15-1+cuda9.0_amd64.deb
wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libnccl2_2.1.4-1+cuda9.0_amd64.deb
wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libnccl-dev_2.1.4-1+cuda9.0_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
sudo dpkg -i libcudnn7_7.0.5.15-1+cuda9.0_amd64.deb
sudo dpkg -i libcudnn7-dev_7.0.5.15-1+cuda9.0_amd64.deb
sudo dpkg -i libnccl2_2.1.4-1+cuda9.0_amd64.deb
sudo dpkg -i libnccl-dev_2.1.4-1+cuda9.0_amd64.deb
sudo apt-get update
sudo apt-get install cuda=9.0.176-1
sudo apt-get install libcudnn7-dev
sudo apt-get install libnccl-dev
sudo reboot now
export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
sudo nano .bashrc
Add the two last export lines at the end of the file. Save and reboot.

Checking if CUDA is properly installed. Type:

Resources used:

Installing Docker CE on Ubuntu 16.04:

sudo apt-get install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial stable"
sudo apt-get update

Installing Deepo:

Change Docker root dir using systemd

systemctl status docker.service
sudo nano /etc/default/docker
Edit ExecStart line to look like this ExecStart =/usr/bin/dockerd -g /media/MachineLearning/docker -H fd://
systemctl daemon-reload
systemctl restart docker
sudo docker info - verify the root dir has updated
https://github.com/IronicBadger/til/blob/master/docker/change-docker-root.md

Docker - clean up all the volumes

Other options:

NTFS fstab wizard:

Format large capacity HD with fs ExFat for having access to it from Ubuntu:

@@ Line 76: / Line 76: @@
 *sudo apt-get install libcudnn7-dev
 *sudo apt-get install libnccl-dev
+*sudo reboot now
+*export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}
+*export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
+*sudo nano .bashrc
+*Add the two last export lines at the end of the file. Save and reboot.
 '''Checking if CUDA is properly installed. Type:'''