Difference between revisions of "ML computers setup"

From Interaction Station Wiki
Jump to navigation Jump to search
(Created page with "=Setting up ML computers:= *Linux distribution installed: Ubuntu 16.04 ==Partition made for machine learning:MachineLearning== *In Windows: Disk Management -> Resize DataStor...")
 
 
(10 intermediate revisions by 2 users not shown)
Line 1: Line 1:
=Setting up ML computers:=
+
 
*Linux distribution installed: Ubuntu 16.04
+
 
 +
=Linux distribution installed: Ubuntu 16.04 (2017)=
  
 
==Partition made for machine learning:MachineLearning==
 
==Partition made for machine learning:MachineLearning==
Line 9: Line 10:
  
 
===Get the UUID of the learning:MachineLearning partition===
 
===Get the UUID of the learning:MachineLearning partition===
*sudo blkid
+
sudo blkid
  
 
===Add partition to fstab:===
 
===Add partition to fstab:===
*sudo nano /etc/fstab
+
sudo nano /etc/fstab
 
*Add at the bottom these two lines:
 
*Add at the bottom these two lines:
*UUID=(id of the MachineLearning partition) /media/MachineLearning rw,suid,dev,auto,user,async,exec      0      2
+
*UUID=(id of the MachineLearning partition) /media/interactionstation/MachineLearning ext4 rw,suid,dev,auto,user,async,exec      0      2
*UUID=(id of the DataStorage partition) /media/DataStorage ntfs-3g defaults=en_US.UTF-8 0 0
+
*UUID=(id of the DataStorage partition) /media/interactionstation/DataStorage ntfs-3g defaults=en_US.UTF-8 0 0
  
 
*The parameters mounting the MachineLearning partition solved this problem running caffe from that partition:
 
*The parameters mounting the MachineLearning partition solved this problem running caffe from that partition:
Line 22: Line 23:
  
 
===Give writing permissions to new MachineLearning partition===
 
===Give writing permissions to new MachineLearning partition===
*sudo chmod -R a+rwx /media/MachineLearning/
+
sudo chmod -R a+rwx /media/MachineLearning/
  
 
*Need extra space? Extending the partition
 
*Need extra space? Extending the partition
 
https://askubuntu.com/questions/492054/how-to-extend-my-root-partition
 
https://askubuntu.com/questions/492054/how-to-extend-my-root-partition
 
==Installing NVIDIA Driver:==
 
*Set Ubuntu to boot on console mode. Type:
 
*sudo apt-get install systemd
 
*sudo systemctl set-default multi-user.target
 
*sudo reboot now
 
*Login and in console mode, type:
 
*sudo add-apt-repository ppa:graphics-drivers/ppa
 
*sudo apt update
 
*sudo apt upgrade
 
*For GeForce 1070Ti (07/2018), type:
 
*sudo apt-get install nvidia-390
 
*Re-set Ubuntu to boot on graphical mode. Type:
 
*sudo systemctl set-default graphical.target
 
*sudo reboot now
 
 
'''Checking if Nvidia Driver is properly installed. Type:'''
 
*nvidia-smi
 
*nvidia-settings
 
 
==Installing CUDA 9.0 for Ubuntu 16.04 (the latest version is not supported by TensorFlow):==
 
*wget http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
 
*wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libcudnn7_7.0.5.15-1+cuda9.0_amd64.deb
 
*wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libcudnn7-dev_7.0.5.15-1+cuda9.0_amd64.deb
 
*wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libnccl2_2.1.4-1+cuda9.0_amd64.deb
 
*wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libnccl-dev_2.1.4-1+cuda9.0_amd64.deb
 
*sudo dpkg -i cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
 
*sudo dpkg -i libcudnn7_7.0.5.15-1+cuda9.0_amd64.deb
 
*sudo dpkg -i libcudnn7-dev_7.0.5.15-1+cuda9.0_amd64.deb
 
*sudo dpkg -i libnccl2_2.1.4-1+cuda9.0_amd64.deb
 
*sudo dpkg -i libnccl-dev_2.1.4-1+cuda9.0_amd64.deb
 
*sudo apt-get update
 
*sudo apt-get install cuda=9.0.176-1
 
*sudo apt-get install libcudnn7-dev
 
*sudo apt-get install libnccl-dev
 
*sudo reboot now
 
*export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}
 
*export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
 
*sudo nano .bashrc
 
*Add the two last export lines at the end of the file. Save and reboot.
 
 
'''Checking if CUDA is properly installed. Type:'''
 
*nvcc --version
 
 
'''Resources used:'''
 
*https://askubuntu.com/questions/61396/how-do-i-install-the-nvidia-drivers
 
*https://medium.com/@bbloks/a-machine-learning-environment-with-ubuntu-and-gpu-acceleration-in-5-steps-765608325356
 
*https://yangcha.github.io/CUDA90/
 
  
  
Line 79: Line 32:
  
 
'''NTFS fstab wizard:'''
 
'''NTFS fstab wizard:'''
*sudo apt-get install ntfs-config
+
sudo apt-get install ntfs-config
*sudo ntfs-config
+
sudo ntfs-config
  
 
'''Format large capacity HD with fs ExFat for having access to it from Ubuntu:'''
 
'''Format large capacity HD with fs ExFat for having access to it from Ubuntu:'''
*On Windows 10
+
*On Windows 10:
*cmd
+
cmd
*diskpart
+
diskpart
 
*select disk '#' (where # is the number of the target drive)
 
*select disk '#' (where # is the number of the target drive)
*list part
+
list part
 
*select part # (where # is the number of the partition)
 
*select part # (where # is the number of the partition)
*format fs=exfat QUICK
+
format fs=exfat QUICK
 +
 
 +
*automount external exfat drive in Ubuntu
 +
sudo apt-get install exfat-utils exfat-fuse
 +
 
 +
[[Category:AI & Machine Learning]]

Latest revision as of 13:42, 28 November 2022


Linux distribution installed: Ubuntu 16.04 (2017)

Partition made for machine learning:MachineLearning

  • In Windows: Disk Management -> Resize DataStorage
  • Create new ext4 patition

Mounting the partition automatically:

Get the UUID of the learning:MachineLearning partition

sudo blkid

Add partition to fstab:

sudo nano /etc/fstab
  • Add at the bottom these two lines:
  • UUID=(id of the MachineLearning partition) /media/interactionstation/MachineLearning ext4 rw,suid,dev,auto,user,async,exec 0 2
  • UUID=(id of the DataStorage partition) /media/interactionstation/DataStorage ntfs-3g defaults=en_US.UTF-8 0 0

Give writing permissions to new MachineLearning partition

sudo chmod -R a+rwx /media/MachineLearning/
  • Need extra space? Extending the partition

https://askubuntu.com/questions/492054/how-to-extend-my-root-partition


Other options:

NTFS fstab wizard:

sudo apt-get install ntfs-config
sudo ntfs-config

Format large capacity HD with fs ExFat for having access to it from Ubuntu:

  • On Windows 10:
cmd
diskpart
  • select disk '#' (where # is the number of the target drive)
list part
  • select part # (where # is the number of the partition)
format fs=exfat QUICK
  • automount external exfat drive in Ubuntu
sudo apt-get install exfat-utils exfat-fuse