Difference between revisions of "Docker"
Jump to navigation
Jump to search
| Line 80: | Line 80: | ||
*nvcc --version | *nvcc --version | ||
| − | + | '''Resources used:''' | |
*https://askubuntu.com/questions/61396/how-do-i-install-the-nvidia-drivers | *https://askubuntu.com/questions/61396/how-do-i-install-the-nvidia-drivers | ||
*https://medium.com/@bbloks/a-machine-learning-environment-with-ubuntu-and-gpu-acceleration-in-5-steps-765608325356 | *https://medium.com/@bbloks/a-machine-learning-environment-with-ubuntu-and-gpu-acceleration-in-5-steps-765608325356 | ||
| + | *https://yangcha.github.io/CUDA90/ | ||
'''Installing Docker CE on Ubuntu 16.04:''' | '''Installing Docker CE on Ubuntu 16.04:''' | ||
Revision as of 14:06, 25 July 2018
ML Docker Image installed on the Interaction Station ML computers:
Deepo. It includes:
- cudnn
- theano
- tensorflow
- sonnet
- pytorch
- keras
- lasagne
- mxnet
- cntk
- chainer
- caffe
- caffe2
- torch
Run Deepo image with Docker:
- docker run -it ufoym/deepo:gpu bash
Setting up ML computers:
- Linux distribution installed: Ubuntu 16.04
Partition made for machine learning:MachineLearning
- In Windows: Disk Management -> Resize DataStorage
- Create new ext4 patition
Mounting the partition automatically:
- Get the UUID of the learning:MachineLearning partition
- sudo blkid
- Add partition to fstab:
- sudo nano /etc/fstba
- Add at the bottom these two lines:
- UUID=(id of the MachineLearning partition) /media/MachineLearning ext4 1 2
- UUID=(id of the DataStorage partition) /media/DataStorage ntfs-3g defaults=en_US.UTF-8 0 0
Give writting permissions to new MachineLearning partition
- sudo chmod -R a+rwx /media/MachineLearning/
Installing NVIDIA Driver:
- Set Ubuntu to boot on console mode. Type:
- systemd.unit=multi-user=target
- sudo reboot now
- Login and in console mode, type:
- sudo add-apt-repository ppa:graphics-drivers/ppa
- sudo apt update
- sudo apt upgrade
- For GeForce 1070Ti (07/2018), type:
- sudo apt-get install nvidia-390
- Re-set Ubuntu to boot on graphical mode. Type:
- sudo systemctl set-default graphical.target
- sudo reboot now
Checking if Nvidia Driver is properly installed. Type:
- nvidia-smi
- nvidia-settings
Installing CUDA 9.0 for Ubuntu 16.04 (the latest version is not supported by TensorFlow):
- wget http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
- wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libcudnn7_7.0.5.15-1+cuda9.0_amd64.deb
- wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libcudnn7-dev_7.0.5.15-1+cuda9.0_amd64.deb
- wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libnccl2_2.1.4-1+cuda9.0_amd64.deb
- wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libnccl-dev_2.1.4-1+cuda9.0_amd64.deb
- sudo dpkg -i cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
- sudo dpkg -i libcudnn7_7.0.5.15-1+cuda9.0_amd64.deb
- sudo dpkg -i libcudnn7-dev_7.0.5.15-1+cuda9.0_amd64.deb
- sudo dpkg -i libnccl2_2.1.4-1+cuda9.0_amd64.deb
- sudo dpkg -i libnccl-dev_2.1.4-1+cuda9.0_amd64.deb
- sudo apt-get update
- sudo apt-get install cuda=9.0.176-1
- sudo apt-get install libcudnn7-dev
- sudo apt-get install libnccl-dev
Checking if CUDA is properly installed. Type:
- nvcc --version
Resources used:
- https://askubuntu.com/questions/61396/how-do-i-install-the-nvidia-drivers
- https://medium.com/@bbloks/a-machine-learning-environment-with-ubuntu-and-gpu-acceleration-in-5-steps-765608325356
- https://yangcha.github.io/CUDA90/
Installing Docker CE on Ubuntu 16.04:
- sudo apt-get install apt-transport-https ca-certificates curl software-properties-common
- curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
- sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial stable"
- sudo apt-get update
- More info: https://unix.stackexchange.com/questions/363048/unable-to-locate-package-docker-ce-on-a-64bit-ubuntu
Installing Deepo:
Change Docker root dir using systemd
- systemctl status docker.service
- sudo nano /etc/default/docker
- Edit ExecStart line to look like this ExecStart =/usr/bin/dockerd -g /media/MachineLearning/docker -H fd://
- systemctl daemon-reload
- systemctl restart docker
- sudo docker info - verify the root dir has updated
- https://github.com/IronicBadger/til/blob/master/docker/change-docker-root.md
Docker - clean up all the volumes
- sudo docker system prune -a -f --volumes
Other options:
NTFS fstab wizard:
- sudo apt-get install ntfs-config
- sudo ntfs-config
Format large capacity HD with fs ExFat for having access to it from Ubuntu:
- On Windows 10
- cmd
- diskpart
- select disk '#' (where # is the number of the target drive)
- list part
- select part # (where # is the number of the partition)
- format fs=exfat QUICK