[TX2] Tensorflow 1.12.0 在 Jetson TX2 上的编译

2018年12月26日

系统环境
Ubuntu 16.04
Jetpack 3.2.1 on TX2 [Link](with CUDA 9.0 cuDNN 7.0.5)

1、编译准备
1）配置环境

export LD_LIBRARY_PATH=/usr/local/cuda/extras/CUPTI/lib64:$LD_LIBRARY_PATH

1	export LD_LIBRARY_PATH=/usr/local/cuda/extras/CUPTI/lib64:$LD_LIBRARY_PATH

2）安装依赖
Java

sudo apt-get install openjdk-8-jdk

1	sudo apt-get install openjdk-8-jdk

Bazel (Tensorflow 使用 Bazel 0.15 编译，因此这里下载 0.15.2 版本，详情参见这里)

cd ~/Downloads
wget https://github.com/bazelbuild/bazel/releases/download/0.15.2/bazel-0.15.2-dist.zip
mkdir -p ~/bazel
cd ~/bazel
unzip ~/Downloads/bazel-0.15.2-dist.zip -d bazel-0.15.2-dist
cd bazel-0.15.2-dist
./compile.sh
sudo cp output/bazel /usr/local/bin
bazel help

cd ~/Downloads

wget https://github.com/bazelbuild/bazel/releases/download/0.15.2/bazel-0.15.2-dist.zip

mkdir -p ~/bazel

cd ~/bazel

unzip ~/Downloads/bazel-0.15.2-dist.zip -d bazel-0.15.2-dist

cd bazel-0.15.2-dist

./compile.sh

sudo cp output/bazel /usr/local/bin

bazel help

2、编译 Tensorflow
1）下载源码
运行如下命令下载 tensorflow 1.12.0 版本源码：

cd ~/Downloads
wget https://github.com/tensorflow/tensorflow/archive/v1.12.0.tar.gz
tar xzvf ./v1.12.0.tar.gz
cd tensorflow-1.12.0

cd ~/Downloads

wget https://github.com/tensorflow/tensorflow/archive/v1.12.0.tar.gz

tar xzvf ./v1.12.0.tar.gz

cd tensorflow-1.12.0

2）修改源码
为了在 TX2 上编译通过，需要 apply this patch
具体说明如下：

2.1）打开文件：tensorflow-1.12.0/tensorflow/contrib/lite/kernels/internal/BUILD
查找：

NEON_FLAGS_IF_APPLICABLE = select({
    ":arm": [
        "-O3",
        "-mfpu=neon",
    ],
    ":armeabi-v7a": [
        "-O3",
        "-mfpu=neon",
    ],
    ":armv7a": [
        "-O3",
        "-mfpu=neon",
    ],
    "//conditions:default": [
        "-O3",
    ],
})

NEON_FLAGS_IF_APPLICABLE = select({

":arm": [

"-O3",

"-mfpu=neon",

":armeabi-v7a": [

"-O3",

"-mfpu=neon",

":armv7a": [

"-O3",

"-mfpu=neon",

"//conditions:default": [

"-O3",

})

修改成：

NEON_FLAGS_IF_APPLICABLE = select({
    ":arm": [
        "-O3",
    ],
    ":armeabi-v7a": [
        "-O3",
        "-mfpu=neon",
    ],
    ":armv7a": [
        "-O3",
        "-mfpu=neon",
    ],
    "//conditions:default": [
        "-O3",
    ],
})

NEON_FLAGS_IF_APPLICABLE = select({

":arm": [

"-O3",

":armeabi-v7a": [

"-O3",

"-mfpu=neon",

":armv7a": [

"-O3",

"-mfpu=neon",

"//conditions:default": [

"-O3",

})

2.2）打开文件：tensorflow-1.12.0/third_party/aws.BUILD
查找：

        "//conditions:default": [],

1	"//conditions:default": [],

修改成：

        "//conditions:default": glob([
            "aws-cpp-sdk-core/source/platform/linux-shared/*.cpp",
	]),

"//conditions:default": glob([

"aws-cpp-sdk-core/source/platform/linux-shared/*.cpp",

]),

3）编译配置

./configure

1	./configure

其中会提示若干选项，请参考如下进行选择：

Found possible Python library paths:
  /usr/local/lib/python3.5/dist-packages
  /usr/lib/python3/dist-packages
Please input the desired Python library path to use.  Default is [/usr/local/lib/python3.5/dist-packages]
   
Do you wish to build TensorFlow with jemalloc as malloc support? [Y/n]: 
jemalloc as malloc support will be enabled for TensorFlow.
   
Do you wish to build TensorFlow with Google Cloud Platform support? [Y/n]: n
No Google Cloud Platform support will be enabled for TensorFlow.
   
Do you wish to build TensorFlow with Hadoop File System support? [Y/n]: n
No Hadoop File System support will be enabled for TensorFlow.
   
Do you wish to build TensorFlow with Amazon S3 File System support? [Y/n]: n
No Amazon S3 File System support will be enabled for TensorFlow.
   
Do you wish to build TensorFlow with Apache Kafka Platform support? [Y/n]: n
No Apache Kafka Platform support will be enabled for TensorFlow.
   
Do you wish to build TensorFlow with XLA JIT support? [y/N]: 
No XLA JIT support will be enabled for TensorFlow.
   
Do you wish to build TensorFlow with GDR support? [y/N]: 
No GDR support will be enabled for TensorFlow.
   
Do you wish to build TensorFlow with VERBS support? [y/N]: 
No VERBS support will be enabled for TensorFlow.
   
Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: 
No OpenCL SYCL support will be enabled for TensorFlow.
   
Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.
   
Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 9.0]: 
   
   
Please specify the location where CUDA 9.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: 
   
   
Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7.0]: 7.1.5
   
Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:/usr/lib/aarch64-linux-gnu
   
   
Do you wish to build TensorFlow with TensorRT support? [y/N]: y
TensorRT support will be enabled for TensorFlow.
   
Please specify the location where TensorRT is installed. [Default is /usr/lib/aarch64-linux-gnu]:
   
   
Please specify the NCCL version you want to use. [Leave empty to default to NCCL 1.3]: 
   
   
Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size. [Default is: 3.5,5.2]6.2
   
   
Do you want to use clang as CUDA compiler? [y/N]: 
nvcc will be used as CUDA compiler.
   
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: 
   
   
Do you wish to build TensorFlow with MPI support? [y/N]: 
No MPI support will be enabled for TensorFlow.
   
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]: 
   
   
Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: 
Not configuring the WORKSPACE for Android builds.
   
Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See tools/bazel.rc for more details.
	--config=mkl         	# Build with MKL support.
	--config=monolithic  	# Config for mostly static monolithic build.
Configuration finished

Found possible Python library paths:

/usr/local/lib/python3.5/dist-packages

/usr/lib/python3/dist-packages

Please input the desired Python library path to use. Default is [/usr/local/lib/python3.5/dist-packages]

Do you wish to build TensorFlow with jemalloc as malloc support? [Y/n]:

jemalloc as malloc support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Google Cloud Platform support? [Y/n]: n

No Google Cloud Platform support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Hadoop File System support? [Y/n]: n

No Hadoop File System support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Amazon S3 File System support? [Y/n]: n

No Amazon S3 File System support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Apache Kafka Platform support? [Y/n]: n

No Apache Kafka Platform support will be enabled for TensorFlow.

Do you wish to build TensorFlow with XLA JIT support? [y/N]:

No XLA JIT support will be enabled for TensorFlow.

Do you wish to build TensorFlow with GDR support? [y/N]:

No GDR support will be enabled for TensorFlow.

Do you wish to build TensorFlow with VERBS support? [y/N]:

No VERBS support will be enabled for TensorFlow.

Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]:

No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: y

CUDA support will be enabled for TensorFlow.

Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 9.0]:

Please specify the location where CUDA 9.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:

Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7.0]: 7.1.5

Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:/usr/lib/aarch64-linux-gnu

Do you wish to build TensorFlow with TensorRT support? [y/N]: y

TensorRT support will be enabled for TensorFlow.

Please specify the location where TensorRT is installed. [Default is /usr/lib/aarch64-linux-gnu]:

Please specify the NCCL version you want to use. [Leave empty to default to NCCL 1.3]:

Please specify a list of comma-separated Cuda compute capabilities you want to build with.

You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.

Please note that each additional compute capability significantly increases your build time and binary size. [Default is: 3.5,5.2]6.2

Do you want to use clang as CUDA compiler? [y/N]:

nvcc will be used as CUDA compiler.

Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:

Do you wish to build TensorFlow with MPI support? [y/N]:

No MPI support will be enabled for TensorFlow.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:

Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]:

Not configuring the WORKSPACE for Android builds.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See tools/bazel.rc for more details.

--config=mkl # Build with MKL support.

--config=monolithic # Config for mostly static monolithic build.

Configuration finished

3）编译
由于 TX2 上面内存只有 8G，因此编译 C Lib 时需要限制内存使用不超过8G，如果直接按照官方的方式是会报错的，当然你也可以设置 swap 来解决。我这里直接设置为4G，你可以根据需要进行调整，但是最好不要超过8G：

 bazel test --config opt --local_resources 4096,2.0,1.0 //tensorflow/tools/lib_package:libtensorflow_test
bazel build --config opt --local_resources 4096,2.0,1.0 //tensorflow/tools/lib_package:libtensorflow

1 2	bazel test --config opt --local_resources 4096,2.0,1.0 //tensorflow/tools/lib_package:libtensorflow_test bazel build --config opt --local_resources 4096,2.0,1.0 //tensorflow/tools/lib_package:libtensorflow

这一过程大约需要5、6个小时，如果最后提示编译成功，则会在如下目录生成 libtensorflow.tar.gz 文件
bazel-bin/tensorflow/tools/lib_package/libtensorflow.tar.gz

至此就完成了 TX2 上 Tensorflow 版本的编译。

下面是包含 TX2 在内的几个常用 Tensorflow C Lib 地址，不想编译的直接下载我这里的网盘文件即可：

TensorFlow C library	CUDA	cuDNN	URL
Linux x86_64 CPU	x	x	https://pan.baidu.com/s/1FDdXCgtJJlDJP8ziDs6dow
Linux x86_64 GPU	9.0	7.x	https://pan.baidu.com/s/1qxDntkQ-rcgvp1xxrSKW0w
macOS CPU	x	x	https://pan.baidu.com/s/1F6NdNtCxg11P_EpEdsqttA
Linux aarch64 GPU (TX2)	9.0	7.0.5	https://pan.baidu.com/s/1mI76203wY9Nd5US4sH5-pg

参考文献
[1] https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/lib_package/README.md
[2] https://jkjung-avt.github.io/build-tensorflow-1.8.0/
[3] https://github.com/peterlee0127/tensorflow-nvJetson

About The Author

skylook

增强现实、图像识别技术爱好者。

技术刘

[TX2] Tensorflow 1.12.0 在 Jetson TX2 上的编译

About The Author

skylook

Add a Comment

Related Posts

About The Author

skylook

Add a Comment