도커와 텐서플로우 설치하기

#. 우분투를 받는다. (16.04 64bit Desktop)

http://releases.ubuntu.com/16.04/

# USB에 이미지를 쓰기 위한 프로그램을 받는다. 

https://sourceforge.net/projects/win32diskimager/?source=typ_redirect

# 설치를 한다. 

# 이제 진짜 시작이다. 

sudo apt-get update && sudo apt-get upgrade && sudo apt-get dist-upgrade

sudo apt-get install make cmake git

sudo apt-get install dkms build-essential linux-headers-generic

#아래 부분을 추가/수정한다. 

sudo gedit /etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash nomodeset"

#아래 부분을 추가/수정한다. 

sudo gedit /etc/modprobe.d/blacklist.conf

blacklist nouveau 

blacklist lbm-nouveau 

options nouveau modeset=0 

alias nouveau off 

alias lbm-nouveau off

#PC를 재시작한다. 

reboot

#Download CUDA

https://developer.nvidia.com/cuda-downloads

# 위의 사이트에서 받는다. (cuda_8.0.44_linux.run)

chmod +x cuda_8.0.44_linux.run

# Ctrl+Alt+F1 그리고 Login 

sudo service lightdm stop

sudo init 3

sudo ./cuda_8.0.44_linux.run --no-opengl-libs

sudo service lightdm start

# 아래 부분을 추가 수정한다. 

gedit ~/.profile

# CUDA

PATH="/usr/local/cuda-8.0/bin:$PATH"

sudo gedit /etc/ld.so.conf

include /usr/local/cuda-8.0/lib64

sudo ldconfig

source ~/.profile

# 아래 링크를 따라서 Secure mode를 해제한다. 

http://askubuntu.com/questions/762254/why-do-i-get-required-key-not-available-when-install-3rd-party-kernel-modules

#PC를 재시작한다. 

reboot

# 제대로 드라이버가 설치되었는지 확인한다. 

nvidia-smi 

# 도커를 설치한다. 

sudo apt-get update

sudo apt-get install apt-transport-https ca-certificates

sudo apt-key adv \

--keyserver hkp://ha.pool.sks-keyservers.net:80 \

--recv-keys 58118E89F3A912897C070ADBF76221572C52609D

echo deb https://apt.dockerproject.org/repo ubuntu-xenial main | sudo tee /etc/apt/sources.list.d/docker.list

sudo apt-get update

apt-cache policy docker-engine

sudo apt-get update

sudo apt-get install linux-image-extra-$(uname -r) linux-image-extra-virtual

sudo apt-get update

sudo apt-get install docker-engine

sudo service docker start

sudo docker run hello-world

sudo usermod -aG docker $USER

# 도커가 제대로 설치되었는지 확인한다. 

id sj

: This is just for checking. If you see 999 (docker), you are good to go.

#INSTALL NVIDIA DOCKER

wget -P /tmp https://github.com/.../nvidia-docker_1.0.0.rc.3-1_amd64.deb

sudo dpkg -i /tmp/nvidia-docker*.deb && rm /tmp/nvidia-docker*.deb

#PC를 재시작한다. 

reboot

nvidia-docker run --rm nvidia/cuda nvidia-smi

# 도커 설치에 사용할 폴더를 만들고 이동한다. 

mkdir Workspace

cd Workspace

https://github.com/.../9a765d6168.../tensorflow/tools/docker

에서 jupyter_notebook_config.py 와 run_jupyter.sh 파일을 받는다. 

# DockerFile을 만든다. (아래를 복사 붙여넣기)

FROM nvidia/cuda:8.0-cudnn5-devel

MAINTAINER Craig Citro <craigcitro@google.com>

# Pick up some TF dependencies

RUN apt-get update && apt-get install -y --no-install-recommends \

build-essential \

curl \

git \

swig \

libcurl3-dev \

libfreetype6-dev \

libpng12-dev \

libzmq3-dev \

libtiff5-dev libjpeg8-dev \

liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python-tk \

pkg-config \

python \

python-dev \

rsync \

software-properties-common \

unzip \

zip \

zlib1g-dev \

&& \

apt-get clean && \

rm -rf /var/lib/apt/lists/*

RUN curl -fSsL -O https://bootstrap.pypa.io/get-pip.py && \

python get-pip.py && \

rm get-pip.py

RUN pip --no-cache-dir install \

ipykernel \

jupyter \

Pillow \

matplotlib \

numpy \

scipy \

sklearn \

&& \

python -m ipykernel.kernelspec

# Set up Bazel.

# We need to add a custom PPA to pick up JDK8, since trusty doesn't

# have an openjdk8 backport. openjdk-r is maintained by a reliable contributor:

# Matthias Klose (https://launchpad.net/~doko). It will do until

# we either update the base image beyond 14.04 or openjdk-8 is

# finally backported to trusty; see e.g.

# https://bugs.launchpad.net/trusty-backports/+bug/1368094

#

# RUN add-apt-repository -y ppa:openjdk-r/ppa && \

# apt-get update && \

# apt-get install -y --no-install-recommends openjdk-8-jdk openjdk-8-jre-headless && \

# apt-get clean && \

# rm -rf /var/lib/apt/lists/*

RUN add-apt-repository ppa:webupd8team/java && \

apt-get update && \

echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections && \

apt-get install -y oracle-java8-installer && \

apt-get clean && \

rm -rf /var/lib/apt/lists/*

# Running bazel inside a `docker build` command causes trouble, cf:

# https://github.com/bazelbuild/bazel/issues/134

# The easiest solution is to set up a bazelrc file forcing --batch.

RUN echo "startup --batch" >>/root/.bazelrc

# Similarly, we need to workaround sandboxing issues:

# https://github.com/bazelbuild/bazel/issues/418

RUN echo "build --spawn_strategy=standalone --genrule_strategy=standalone" \

>>/root/.bazelrc

ENV BAZELRC /root/.bazelrc

# Install the most recent bazel release.

ENV BAZEL_VERSION 0.4.1

WORKDIR /

RUN mkdir /bazel && \

cd /bazel && \

curl -fSsL -O https://github.com/.../bazel-$BAZEL_VERSION-installer... && \

curl -fSsL -o /bazel/LICENSE.txt https://raw.githubusercontent.com/.../master/LICENSE.txt && \

chmod +x bazel-*.sh && \

./bazel-$BAZEL_VERSION-installer-linux-x86_64.sh && \

cd / && \

rm -f /bazel/bazel-$BAZEL_VERSION-installer-linux-x86_64.sh

# Download and build TensorFlow.

RUN git clone -b r0.12 --single-branch https://github.com/tensorflow/tensorflow.git && \

cd tensorflow && \

git checkout r0.12

WORKDIR /tensorflow

# Configure the build for our CUDA configuration.

ENV LD_LIBRARY_PATH /usr/local/cuda/extras/CUPTI/lib64:$LD_LIBRARY_PATH

ENV TF_NEED_CUDA 1

ENV TF_CUDA_COMPUTE_CAPABILITIES=3.0,3.5,5.2

RUN tensorflow/tools/ci_build/builds/configured GPU \

bazel build -c opt --config=cuda tensorflow/tools/pip_package:build_pip_package && \

bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/pip && \

pip install --upgrade /tmp/pip/tensorflow-*.whl && \

rm -rf /tmp/pip && \

rm -rf /root/.cache

# Clean up pip wheel and Bazel cache when done.

# Set up our notebook config.

# COPY jupyter_notebook_config.py /root/.jupyter/

# Copy sample notebooks.

# COPY notebooks /notebooks

# Jupyter has issues with being run directly:

# https://github.com/ipython/ipython/issues/7062

# We just add a little wrapper script.

# COPY run_jupyter.sh /

# MY MODIFICATION

# FOR GUI USAGE

RUN sed 's/main$/main universe/' -i /etc/apt/sources.list && \

apt-get update && \

apt-get install -y libxext-dev libxrender-dev libxtst-dev && \

apt-get clean && \

rm -rf /var/lib/apt/lists/* && \

rm -rf /tmp/*

RUN addgroup --gid 1000 sj

RUN adduser -uid 1000 -gid 1000 sj

RUN usermod -aG sudo sj

RUN cp /root/.bashrc /home/sj

RUN chown sj:sj /home/sj/.bashrc

# RUN echo "\n# FOR CUDA AND cudnn\n" >> /home/sj/.bashrc

# RUN echo "export LD_LIBRARY_PATH="/usr/local/nvidia/lib:/usr/local/nvidia/lib64:$LD_LIBRARY_PATH"" >> /home/sj/.bashrc

# RUN echo "export PATH="/usr/local/nvidia/bin:/usr/local/cuda/bin:$PATH"" >> /home/sj/.bashrc

COPY jupyter_notebook_config.py /home/sj/.jupyter/

COPY run_jupyter.sh /home/sj

RUN chown sj:sj /home/sj/.jupyter/jupyter_notebook_config.py

RUN chown sj:sj /home/sj/run_jupyter.sh

# TensorBoard

EXPOSE 6006

# IPython

EXPOSE 8888

WORKDIR "/home/sj/notebooks"

# CMD ["/run_jupyter.sh"]

# 도커를 빌드한다. 

nvidia-docker build --tag sj/tf0.12rc0 .

(When doing this, you should have 'jupyter_notebook_config.py' and 'run_jupyter.sh' 

which can be found in tf sites (unchanged).)

오래 기다린다. 또 기다린다. 오래 걸리는 이유는 Bazel build 때문이다. 

# 잘 만들어졌는지 확인한다. 

docker images

:  If you see sj/tf0.12rc0 , then you are good to go!

# 이리 만들어놓은 도커 컨테이너를 여는 파일을 부른다. 

sh container.sh

nvidia-docker run --name sj.tensorflow \

-p 8888:8888 \

-p 6006:6006 \

-it \

-e DISPLAY=$DISPLAY \

-v '/home/sj/code/notebooks_docker:/home/sj/notebooks' \

-v '/home/sj/code/PycharmProjects_docker:/home/sj/PycharmProjects' \

-v '/tmp/.X11-unix:/tmp/.X11-unix' \

sj/tf0.12rc0 /bin/bash

In case of seeing The name "/sj.tensorflow" is already in use by container ..

docker rm /sj.tensorflow

# 도커 환경에 들어왔다면, 

passwd root

passwd sj

: sj는 내 아이디이다. 

su sj

echo $LD_LIBRARY_PATH

cd ~

# 쥬피터 환경 설정하고 연다. 

sudo chown -R sj:sj .jupyter/

cd notebooks/

sh ../run_jupyter.sh

# 이렇게 접속할 수 있다. (혹은 외부에선 147.46.xx.xx:8888)

http://localhost:8888

# 비밀번호를 설정한다. 

export PASSWORD='ADSL'

# 파이참을 받아서 설치한다. 

https://www.jetbrains.com/pycharm/download/#section=windows

# 다운 링크를 복사한다. (wget으로 받을 것이다.)

cd ~

mkdir downloads

cd downloads

# 받고, 압출 풀고, 실행한다. 

wget https://download.jetbrains.com/.../pycharm-community-2016...

tar xfz pycharm-community-2016.3.tar.gz

cd pycharm-community-2016.3/bin/

sh pycharm.sh

# 혹시 다른 창에서 새롭게 열려있는 도커를 실행하고 싶다면

docker exec -it sj.tensorflow /bin/bash

# 외부에서 현재 PC로 접속할 수 있게 포트를 열어 놓는다. 

sudo apt-get install openssh-server

sudo ufw allow 22

# 현재까지 만들어진 컨테이너를 저장한다. (혹시 뻑나면 이걸로 백업하면 된다.)

docker commit -m 'v0.1' sj.tensorflow sj/tf0.12rc0:v0.1

# 확인해본다. 

docker images

: 아래와 같은 것이 보이면 된 것이다. 

sj/tf0.12rc0 v0.1 854a54af00ce 22 seconds ago