NanoPi M4V2 : TensorFlow 2.2.0 (aarch64) 빌드 및 설치 (Python 3.7)

출처

• 강력추천 : Install TensorFlow 2.1.0 on Raspberry Pi 4 - Q-engineering
• Raspberry PI 에서 docker 설치와 실행하기 - Seongho's SE & IT Life
• Raspberry Pi의 소스에서 빌드 | TensorFlow
• 런타임 버전 목록 | AI Platform | Google Cloud
• トマト農家のロボット創り　Robot creation by tomato farmer: Build Bazel(0.29.0) on Raspberry Pi 4
• 라즈베리파이3 B+ 설치가이드(14) swap 사이즈 설정 - Geek’s Voyage
• PINTO0309/Tensorflow-bin: Prebuilt binary with Tensorflow Lite enabled (native build). For RaspberryPi / Jetson Nano. And, solved Tensorflow issues #15062,#21574,#21855,#23082,#25120,#25748,#29617,#29704,#30359.
• 리눅스 CPU 정보 확인하기
• ERROR: Cannot uninstall 'wrapt'. during upgrade · Issue #30191 · tensorflow/tensorflow

리눅스 정보 확인

CPU 정보 확인

$ cat /proc/cpuinfo
processor       : 0
BogoMIPS        : 48.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd03
CPU revision    : 4
 
 ... 생략 ...
 
Serial          : ca0a2ef4e6xxxxxx

instruction set architecture (ISA)

$ uname -i
aarch64

커널에 대한 정보

$ uname -a
Linux NanoPi-M4v2 4.4.179 #1 SMP Tue Mar 17 16:51:26 CST 2020 aarch64 aarch64 aarch64 GNU/Linux

OS버전에 대한 정보

$ cat /etc/issue
Ubuntu 18.04.4 LTS \n \l

논리 코어 수 확인

$ grep -c processor /proc/cpuinfo
6

메모리, swap 사이즈 확인

$ free -h
              total        used        free      shared  buff/cache   available
Mem:           3.8G         76M        2.9G        8.8M        803M        3.6G
Swap:            0B          0B          0B

swap 사이즈 설정 (tensorflow 빌드시 4G가 이상 필요)

현재 swap 사이즈 확인

$ free -h
              total        used        free      shared  buff/cache   available
Mem:           3.8G        1.1G        2.3G        8.8M        391M        2.6G
Swap:            0B          0B          0B
$ sudo swapon -s
$ sudo dd if=/dev/zero of=/mnt/2G.swap bs=1M count=2048 oflag=append conv=notrunc
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 27.6497 s, 77.7 MB/s
$ sudo mkswap /mnt/2G.swap
mkswap: /mnt/2G.swap: insecure permissions 0644, 0600 suggested.
Setting up swapspace version 1, size = 2 GiB (2147479552 bytes)
no label, UUID=0fb271af-7185-4636-91f8-49ac5de6451f
$ sudo swapon /mnt/2G.swap
swapon: /mnt/2G.swap: insecure permissions 0644, 0600 suggested.
$ free -h
              total        used        free      shared  buff/cache   available
Mem:           3.8G        1.1G        207M        8.8M        2.4G        2.5G
Swap:          2.0G          0B        2.0G
$ sudo swapon -s
Filename                                Type            Size    Used    Priority
/mnt/2G.swap

Bazel 설치

출처 : トマト農家のロボット創り　Robot creation by tomato farmer: Build Bazel(0.29.0) on Raspberry Pi 4

javac 1.8 버전으로 변경(심볼릭 링크 변경)

$ sudo update-alternatives --config javac
There are 2 choices for the alternative javac (providing /usr/bin/javac).
 
  Selection    Path                                          Priority   Status
------------------------------------------------------------
* 0            /usr/lib/jvm/java-11-openjdk-armhf/bin/javac   1111      auto mode
  1            /usr/lib/jvm/java-11-openjdk-armhf/bin/javac   1111      manual mode
  2            /usr/lib/jvm/java-8-openjdk-armhf/bin/javac    1081      manual mode
 
Press  to keep the current choice[*], or type selection number: 2
update-alternatives: using /usr/lib/jvm/java-8-openjdk-armhf/bin/javac to provide /usr/bin/javac (javac) in manual mode
$ sudo update-alternatives --config javac
There are 2 choices for the alternative javac (providing /usr/bin/javac).
 
  Selection    Path                                          Priority   Status
------------------------------------------------------------
  0            /usr/lib/jvm/java-11-openjdk-armhf/bin/javac   1111      auto mode
  1            /usr/lib/jvm/java-11-openjdk-armhf/bin/javac   1111      manual mode
* 2            /usr/lib/jvm/java-8-openjdk-armhf/bin/javac    1081      manual mode

Bazel 다운로드 및 설치

$ mkdir bazel
$ cd bazel
$ wget https://github.com/bazelbuild/bazel/releases/download/2.0.0/bazel-2.0.0-dist.zip
$ unzip bazel-2.0.0-dist.zip
$ sudo env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" BAZEL_JAVAC_OPTS="-J-Xms384m -J-Xmx1024m" bash ./compile.sh
$ sudo cp output/bazel /usr/local/bin/bazel

Tensorflow 설치

필수 패키지 설치

$ sudo apt-get install curl libcurl3-dev wget zip unzip git vim build-essential ca-certificates ca-certificates-java pkg-config rsync libatlas-base-dev liblapack-dev libpng-dev libjpeg-dev gfortran python2.7 python-dev python-setuptools gdb openjdk-8-jdk openjdk-8-jre-headless g++ zlib1g-dev libio-all-perl libsnappy-dev libyaml-0-2 python-opencv libopenblas-dev libhdf5-dev

Python 설치

$ sudo apt install python-dev python-pip python3-dev python3-pip

필수 라이브러리 설치

$ sudo -H pip3 install -U --user pip six numpy wheel setuptools mock future>=0.17.1
$ sudo -H pip3 install -U --user keras_applications==1.0.6 --no-deps
$ sudo -H pip3 install -U --user keras_preprocessing==1.0.5 --no-deps

Tensorflow 소스 다운로드

$ wget https://github.com/tensorflow/tensorflow/archive/v2.2.0.tar.gz
$ tar -xvf  v2.2.0.tar.gz
$ cd tensorflow-2.2.0/

빌드 구성

Do you wish to build TensorFlow with XLA JIT support? --> n 으로 설정

$ ./configure 
You have bazel 2.0.0- (@non-git) installed.
Please specify the location of python. [Default is /usr/bin/python]: /usr/local/bin/python3
 
 
Found possible Python library paths:
  /usr/local/lib/python3.7/site-packages
Please input the desired Python library path to use.  Default is [/usr/local/lib/python3.7/site-packages]
 
Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: 
No OpenCL SYCL support will be enabled for TensorFlow.
 
Do you wish to build TensorFlow with ROCm support? [y/N]: 
No ROCm support will be enabled for TensorFlow.
 
Do you wish to build TensorFlow with CUDA support? [y/N]: 
No CUDA support will be enabled for TensorFlow.
 
Do you wish to download a fresh release of clang? (Experimental) [y/N]: 
Clang will not be downloaded.
 
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]: 
 
 
Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: 
Not configuring the WORKSPACE for Android builds.
 
Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
        --config=mkl            # Build with MKL support.
        --config=monolithic     # Config for mostly static monolithic build.
        --config=ngraph         # Build with Intel nGraph support.
        --config=numa           # Build with NUMA support.
        --config=dynamic_kernels        # (Experimental) Build kernels into separate shared objects.
        --config=v2             # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
        --config=noaws          # Disable AWS S3 filesystem support.
        --config=nogcp          # Disable GCP support.
        --config=nohdfs         # Disable HDFS support.
        --config=nonccl         # Disable NVIDIA NCCL support.
Configuration finished

빌드

$ sudo bazel --host_jvm_args=-Xmx1624m build --config=noaws --config=v2 --local_cpu_resources=1 --verbose_failures //tensorflow/tools/pip_package:build_pip_package

파이썬 라이브러리 만들기

$ sudo bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
Mon May 11 15:57:13 UTC 2020 : === Preparing sources in dir: /tmp/tmp.y2XMersE47
~/tensorflow-2.2.0 ~/tensorflow-2.2.0
~/tensorflow-2.2.0
~/tensorflow-2.2.0/bazel-bin/tensorflow/tools/pip_package/build_pip_package.runfiles/org_tensorflow ~/tensorflow-2.2.0
~/tensorflow-2.2.0
/tmp/tmp.y2XMersE47/tensorflow/include ~/tensorflow-2.2.0
~/tensorflow-2.2.0
Mon May 11 15:57:49 UTC 2020 : === Building wheel
warning: no files found matching 'README'
warning: no files found matching '*.pyd' under directory '*'
warning: no files found matching '*.pyi' under directory '*'
warning: no files found matching '*.pd' under directory '*'
warning: no files found matching '*.dylib' under directory '*'
warning: no files found matching '*.dll' under directory '*'
warning: no files found matching '*.lib' under directory '*'
warning: no files found matching '*.csv' under directory '*'
warning: no files found matching '*.h' under directory 'tensorflow/include/tensorflow'
warning: no files found matching '*.proto' under directory 'tensorflow/include/tensorflow'
warning: no files found matching '*' under directory 'tensorflow/include/third_party'
Mon May 11 15:59:22 UTC 2020 : === Output wheel file is in: /tmp/tensorflow_pkg

tensorflow 2.2.0 파이썬 페키지 설치

$ cd /tmp/tensorflow_pkg/
$ sudo pip3 install --upgrade tensorflow-2.2.0-cp37-cp37m-linux_aarch64.whl 
WARNING: The directory '/home/pi/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Processing ./tensorflow-2.2.0-cp37-cp37m-linux_aarch64.whl
Requirement already satisfied, skipping upgrade: numpy<2.0,>=1.16.0 in /home/pi/.local/lib/python3.7/site-packages (from tensorflow==2.2.0) (1.18.4)
Collecting tensorboard<2.3.0,>=2.2.0
  Downloading tensorboard-2.2.1-py3-none-any.whl (3.0 MB)
     |████████████████████████████████| 3.0 MB 4.5 MB/s 
Collecting h5py<2.11.0,>=2.10.0
  Downloading h5py-2.10.0.tar.gz (301 kB)
     |████████████████████████████████| 301 kB 5.4 MB/s 
Collecting astunparse==1.6.3
  Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting opt-einsum>=2.3.2
  Downloading opt_einsum-3.2.1-py3-none-any.whl (63 kB)
     |████████████████████████████████| 63 kB 4.2 MB/s 
Requirement already satisfied, skipping upgrade: wheel>=0.26; python_version >= "3" in /usr/local/lib/python3.7/site-packages (from tensorflow==2.2.0) (0.34.2)
Collecting wrapt>=1.11.1
  Downloading wrapt-1.12.1.tar.gz (27 kB)
Collecting absl-py>=0.7.0
  Downloading absl-py-0.9.0.tar.gz (104 kB)
     |████████████████████████████████| 104 kB 6.4 MB/s 
Requirement already satisfied, skipping upgrade: six>=1.12.0 in /home/pi/.local/lib/python3.7/site-packages (from tensorflow==2.2.0) (1.14.0)
Collecting tensorflow-estimator<2.3.0,>=2.2.0
  Downloading tensorflow_estimator-2.2.0-py2.py3-none-any.whl (454 kB)
     |████████████████████████████████| 454 kB 5.0 MB/s 
Collecting gast==0.3.3
  Downloading gast-0.3.3-py2.py3-none-any.whl (9.7 kB)
Collecting termcolor>=1.1.0
  Downloading termcolor-1.1.0.tar.gz (3.9 kB)
Collecting grpcio>=1.8.6
  Downloading grpcio-1.28.1.tar.gz (19.5 MB)
     |████████████████████████████████| 19.5 MB 17.1 MB/s 
Collecting scipy==1.4.1; python_version >= "3"
  Downloading scipy-1.4.1.tar.gz (24.6 MB)
     |████████████████████████████████| 24.6 MB 14.0 MB/s 
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting google-pasta>=0.1.8
  Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
     |████████████████████████████████| 57 kB 4.5 MB/s 
Collecting protobuf>=3.8.0
  Downloading protobuf-3.11.3-py2.py3-none-any.whl (434 kB)
     |████████████████████████████████| 434 kB 6.5 MB/s 
Collecting keras-preprocessing>=1.1.0
  Downloading Keras_Preprocessing-1.1.0-py2.py3-none-any.whl (41 kB)
     |████████████████████████████████| 41 kB 2.2 MB/s 
Collecting markdown>=2.6.8
  Downloading Markdown-3.2.2-py3-none-any.whl (88 kB)
     |████████████████████████████████| 88 kB 5.9 MB/s 
Collecting requests<3,>=2.21.0
  Downloading requests-2.23.0-py2.py3-none-any.whl (58 kB)
     |████████████████████████████████| 58 kB 3.9 MB/s 
Collecting google-auth<2,>=1.6.3
  Downloading google_auth-1.14.2-py2.py3-none-any.whl (89 kB)
     |████████████████████████████████| 89 kB 5.5 MB/s 
Requirement already satisfied, skipping upgrade: setuptools>=41.0.0 in /usr/local/lib/python3.7/site-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow==2.2.0) (46.1.3)
Collecting werkzeug>=0.11.15
  Downloading Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
     |████████████████████████████████| 298 kB 6.8 MB/s 
Collecting tensorboard-plugin-wit>=1.6.0
  Downloading tensorboard_plugin_wit-1.6.0.post3-py3-none-any.whl (777 kB)
     |████████████████████████████████| 777 kB 5.3 MB/s 
Collecting google-auth-oauthlib<0.5,>=0.4.1
  Downloading google_auth_oauthlib-0.4.1-py2.py3-none-any.whl (18 kB)
Collecting importlib-metadata; python_version < "3.8"
  Downloading importlib_metadata-1.6.0-py2.py3-none-any.whl (30 kB)
Collecting idna<3,>=2.5
  Downloading idna-2.9-py2.py3-none-any.whl (58 kB)
     |████████████████████████████████| 58 kB 4.9 MB/s 
Collecting chardet<4,>=3.0.2
  Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
     |████████████████████████████████| 133 kB 6.2 MB/s 
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
  Downloading urllib3-1.25.9-py2.py3-none-any.whl (126 kB)
     |████████████████████████████████| 126 kB 6.6 MB/s 
Collecting certifi>=2017.4.17
  Downloading certifi-2020.4.5.1-py2.py3-none-any.whl (157 kB)
     |████████████████████████████████| 157 kB 4.4 MB/s 
Collecting rsa<4.1,>=3.1.4
  Downloading rsa-4.0-py2.py3-none-any.whl (38 kB)
Collecting pyasn1-modules>=0.2.1
  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
     |████████████████████████████████| 155 kB 6.3 MB/s 
Collecting cachetools<5.0,>=2.0.0
  Downloading cachetools-4.1.0-py3-none-any.whl (10 kB)
Collecting requests-oauthlib>=0.7.0
  Downloading requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting zipp>=0.5
  Downloading zipp-3.1.0-py3-none-any.whl (4.9 kB)
Collecting pyasn1>=0.1.3
  Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
     |████████████████████████████████| 77 kB 5.3 MB/s 
Collecting oauthlib>=3.0.0
  Downloading oauthlib-3.1.0-py2.py3-none-any.whl (147 kB)
     |████████████████████████████████| 147 kB 6.2 MB/s 
Building wheels for collected packages: h5py, wrapt, absl-py, termcolor, grpcio, scipy
  Building wheel for h5py (setup.py) ... done
  Created wheel for h5py: filename=h5py-2.10.0-cp37-cp37m-linux_aarch64.whl size=4023366 sha256=0ca490b9297ddd162e411faf2de0b19d035dec714ac002c5518ff911649d7b00
  Stored in directory: /tmp/pip-ephem-wheel-cache-9x9ib2b3/wheels/58/e7/f7/d9385040f9b6919de4a5c05625c4e655999412f12c96d3c7c3
  Building wheel for wrapt (setup.py) ... done
  Created wheel for wrapt: filename=wrapt-1.12.1-cp37-cp37m-linux_aarch64.whl size=71539 sha256=c4e9c727e5a12d10a2f4e4507fec751d4c4f126b6448f5edcd8eb49ac09b3927
  Stored in directory: /tmp/pip-ephem-wheel-cache-9x9ib2b3/wheels/62/76/4c/aa25851149f3f6d9785f6c869387ad82b3fd37582fa8147ac6
  Building wheel for absl-py (setup.py) ... done
  Created wheel for absl-py: filename=absl_py-0.9.0-py3-none-any.whl size=121931 sha256=48bc76f542213bc268a2f25ce5139a3817f9c6c859cc5a8171d7d943a5379f87
  Stored in directory: /tmp/pip-ephem-wheel-cache-9x9ib2b3/wheels/cc/af/1a/498a24d0730ef484019e007bb9e8cef3ac00311a672c049a3e
  Building wheel for termcolor (setup.py) ... done
  Created wheel for termcolor: filename=termcolor-1.1.0-py3-none-any.whl size=4830 sha256=e108924f90b631bff23c0252b560510a3dce6d76a436ffc34d3a52fcd071e5fd
  Stored in directory: /tmp/pip-ephem-wheel-cache-9x9ib2b3/wheels/3f/e3/ec/8a8336ff196023622fbcb36de0c5a5c218cbb24111d1d4c7f2
  Building wheel for grpcio (setup.py) ... done
  Created wheel for grpcio: filename=grpcio-1.28.1-cp37-cp37m-linux_aarch64.whl size=21721618 sha256=1c35d05a146e98a5b0fd24d9a8b9583176cb0b993c6f019ae0a43262f94c2164
  Stored in directory: /tmp/pip-ephem-wheel-cache-9x9ib2b3/wheels/1a/57/84/d91cbd4882f6ae0adc3acba56cbf0c2c75e0263118614d1fcf
  Building wheel for scipy (PEP 517) ... done
  Created wheel for scipy: filename=scipy-1.4.1-cp37-cp37m-linux_aarch64.whl size=45319698 sha256=6b3332b029f955a76cf77f18e6c732adb593153ff2fc913dae122ab5243c6834
  Stored in directory: /tmp/pip-ephem-wheel-cache-9x9ib2b3/wheels/ff/bd/f9/ac5a9a019da5be2072e8a283f36d28b219762cebdd21ff3c68
Successfully built h5py wrapt absl-py termcolor grpcio scipy
Installing collected packages: zipp, importlib-metadata, markdown, idna, chardet, urllib3, certifi, requests, pyasn1, rsa, pyasn1-modules, cachetools, google-auth, werkzeug, tensorboard-plugin-wit, grpcio, absl-py, oauthlib, requests-oauthlib, google-auth-oauthlib, protobuf, tensorboard, h5py, astunparse, opt-einsum, wrapt, tensorflow-estimator, gast, termcolor, scipy, google-pasta, keras-preprocessing, tensorflow
  Attempting uninstall: keras-preprocessing
    Found existing installation: Keras-Preprocessing 1.0.5
    Uninstalling Keras-Preprocessing-1.0.5:
      Successfully uninstalled Keras-Preprocessing-1.0.5
Successfully installed absl-py-0.9.0 astunparse-1.6.3 cachetools-4.1.0 certifi-2020.4.5.1 chardet-3.0.4 gast-0.3.3 google-auth-1.14.2 google-auth-oauthlib-0.4.1 google-pasta-0.2.0 grpcio-1.28.1 h5py-2.10.0 idna-2.9 importlib-metadata-1.6.0 keras-preprocessing-1.1.0 markdown-3.2.2 oauthlib-3.1.0 opt-einsum-3.2.1 protobuf-3.11.3 pyasn1-0.4.8 pyasn1-modules-0.2.8 requests-2.23.0 requests-oauthlib-1.3.0 rsa-4.0 scipy-1.4.1 tensorboard-2.2.1 tensorboard-plugin-wit-1.6.0.post3 tensorflow-2.2.0 tensorflow-estimator-2.2.0 termcolor-1.1.0 urllib3-1.25.9 werkzeug-1.0.1 wrapt-1.12.1 zipp-3.1.0

버전 확인

$ python3
Python 3.7.7 (default, May  8 2020, 17:06:24) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> tf.__version__
'2.2.0'

C, C++ 라이브러리 설치

라이브러리 만들기

$ sudo bazel --host_jvm_args=-Xmx1624m build --config=noaws --config=v2 --local_cpu_resources=1 --verbose_failures //tensorflow/tools/lib_package:libtensorflow
 
 ... 생략 ...
 
INFO: Analyzed target //tensorflow/tools/lib_package:libtensorflow (4 packages loaded, 42 targets configured).
INFO: Found 1 target...
Target //tensorflow/tools/lib_package:libtensorflow up-to-date:
  bazel-bin/tensorflow/tools/lib_package/libtensorflow.tar.gz
INFO: Elapsed time: 77.540s, Critical Path: 73.82s
INFO: 8 processes: 8 local.
INFO: Build completed successfully, 20 total actions

라이브러리 설치

$ sudo tar -C /usr/local -xzf bazel-bin/tensorflow/tools/lib_package/libtensorflow.tar.gz

Linker 설정

$ sudo ldconfig

예제 hello.c

#include <stdio.h>
#include <tensorflow/c/c_api.h>

int main() {
  printf("TensorFlow C library version %s\n", TF_Version());
  return 0;
}

예제 컴파일

$ gcc hello.c -ltensorflow -o hello

예제 실행

$ ./hello
TensorFlow C library version 2.2.0