centos编译tensorflow

安装好编译工具后，我们使用git将tf的代码clone到服务器。

1. 编译配置

然后进入 tensorflow的目录。

运行configure文件：

./configure，根据个人的需要进行tf编译的配置。这里要求系统需要预装python 和c++,这里做好是使用python3.x,2.7编译会失败

如果没有请进行安装：

yum install gcc-c++ libstdc++-devel

2. 编译

bazel build -c opt --verbose_failures --jvmopt="-server -Xms1024m" //tensorflow/tools/pip_package:build_pip_package

会遇到很多的错误，有很多依赖库没安装会导致编译错误，需要进行安装

依赖的python库有numpy，keras

依赖的工具有：patch

编译完成了

INFO: Analysed target //tensorflow/tools/pip_package:build_pip_package (0 packages loaded, 0 targets configured).

INFO: Found 1 target...

Target //tensorflow/tools/pip_package:build_pip_package up-to-date:

bazel-bin/tensorflow/tools/pip_package/build_pip_package

INFO: Elapsed time: 18.080s, Critical Path: 4.67s, Remote (0.00% of the time): [queue: 0.00%, setup: 0.00%, process: 0.00%]

INFO: 26 processes: 26 local.

INFO: Build completed successfully, 44 total actions

3. 构建安装包，可以指定想要存放安装包的位置

bazel-bin/tensorflow/tools/pip_package/build_pip_package pack_path

这里我们放到tf源码的tmp目录下

bazel-bin/tensorflow/tools/pip_package/build_pip_package /data/tensorflow/tmp/

完成后可以看到wheel安装包

使用pip安装既可;

pip install tensorflow-1.12.0rc0-cp36-cp36m-linux_x86_64.whl

然后我们就可以测试:

import tensorflow as tf

hello = tf.constant("Hello, TensorFlow!")

sess = tf.Session()

print(sess.run(hello))

错误集锦：

1.ERROR: /data/tensorflow/tensorflow/lite/toco/python/BUILD:15:1: C++ compilation of rule '//tensorflow/lite/toco/python:toco_python_api' failed (Exit 1): gcc failed: error executing command

(cd /root/.cache/bazel/_bazel_root/48b0697df5815d95b49fd98c3083955b/execroot/org_tensorflow && \

exec env - ......

编译输出如下：

\/root/.cache/bazel/_bazel_root/48b0697df5815d95b49fd98c3083955b/external/protobuf_archive/BUILD:640:1: C++ compilation of rule '@protobuf_archive//:python/google/protobuf/pyext/_message.so' failed (Exit 1)

external/protobuf_archive/python/google/protobuf/pyext/descriptor.cc:33:20: fatal error: Python.h: No such file or directory

#include <Python.h>

#include "Python.h"

解决: 安装python-devel

sudo yum -y install python-devel

但是以上似乎并不能解决问题：我们换成如下命令，打印详细的编译日志：

bazel build -c opt --verbose_failures //tensorflow/tools/pip_package:build_pip_package

ERROR: /data/tensorflow/tensorflow/lite/toco/python/BUILD:15:1: C++ compilation of rule '//tensorflow/lite/toco/python:toco_python_api' failed (Exit 1): gcc failed: error executing command

(cd /root/.cache/bazel/_bazel_root/48b0697df5815d95b49fd98c3083955b/execroot/org_tensorflow && \

exec env - \

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/bin:/root/bin \

PWD=/proc/self/cwd \

/usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -B/usr/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections '-std=c++0x' -MD -MF bazel-out/host/bin/tensorflow/lite/toco/python/_objs/toco_python_api/toco_python_api.pic.d '-frandom-seed=bazel-out/host/bin/tensorflow/lite/toco/python/_objs/toco_python_api/toco_python_api.pic.o' -fPIC -D__CLANG_SUPPORT_DYN_ANNOTATION__ -DEIGEN_MPL2_ONLY '-DEIGEN_MAX_ALIGN_BYTES=64' '-DEIGEN_HAS_TYPE_TRAITS=0' -DTF_USE_SNAPPY -iquote . -iquote bazel-out/host/genfiles -iquote bazel-out/host/bin -iquote external/local_config_python -iquote bazel-out/host/genfiles/external/local_config_python -iquote bazel-out/host/bin/external/local_config_python -iquote external/bazel_tools -iquote bazel-out/host/genfiles/external/bazel_tools -iquote bazel-out/host/bin/external/bazel_tools -iquote external/com_google_absl -iquote bazel-out/host/genfiles/external/com_google_absl -iquote bazel-out/host/bin/external/com_google_absl -iquote external/nsync -iquote bazel-out/host/genfiles/external/nsync -iquote bazel-out/host/bin/external/nsync -iquote external/eigen_archive -iquote bazel-out/host/genfiles/external/eigen_archive -iquote bazel-out/host/bin/external/eigen_archive -iquote external/local_config_sycl -iquote bazel-out/host/genfiles/external/local_config_sycl -iquote bazel-out/host/bin/external/local_config_sycl -iquote external/gif_archive -iquote bazel-out/host/genfiles/external/gif_archive -iquote bazel-out/host/bin/external/gif_archive -iquote external/jpeg -iquote bazel-out/host/genfiles/external/jpeg -iquote bazel-out/host/bin/external/jpeg -iquote external/protobuf_archive -iquote bazel-out/host/genfiles/external/protobuf_archive -iquote bazel-out/host/bin/external/protobuf_archive -iquote external/com_googlesource_code_re2 -iquote bazel-out/host/genfiles/external/com_googlesource_code_re2 -iquote bazel-out/host/bin/external/com_googlesource_code_re2 -iquote external/farmhash_archive -iquote bazel-out/host/genfiles/external/farmhash_archive -iquote bazel-out/host/bin/external/farmhash_archive -iquote external/fft2d -iquote bazel-out/host/genfiles/external/fft2d -iquote bazel-out/host/bin/external/fft2d -iquote external/highwayhash -iquote bazel-out/host/genfiles/external/highwayhash -iquote bazel-out/host/bin/external/highwayhash -iquote external/zlib_archive -iquote bazel-out/host/genfiles/external/zlib_archive -iquote bazel-out/host/bin/external/zlib_archive -iquote external/gemmlowp -iquote bazel-out/host/genfiles/external/gemmlowp -iquote bazel-out/host/bin/external/gemmlowp -iquote external/arm_neon_2_x86_sse -iquote bazel-out/host/genfiles/external/arm_neon_2_x86_sse -iquote bazel-out/host/bin/external/arm_neon_2_x86_sse -iquote external/flatbuffers -iquote bazel-out/host/genfiles/external/flatbuffers -iquote bazel-out/host/bin/external/flatbuffers -isystem external/local_config_python/python_include -isystem bazel-out/host/genfiles/external/local_config_python/python_include -isystem bazel-out/host/bin/external/local_config_python/python_include -isystem external/nsync/public -isystem bazel-out/host/genfiles/external/nsync/public -isystem bazel-out/host/bin/external/nsync/public -isystem external/eigen_archive -isystem bazel-out/host/genfiles/external/eigen_archive -isystem bazel-out/host/bin/external/eigen_archive -isystem external/gif_archive/lib -isystem bazel-out/host/genfiles/external/gif_archive/lib -isystem bazel-out/host/bin/external/gif_archive/lib -isystem external/protobuf_archive/src -isystem bazel-out/host/genfiles/external/protobuf_archive/src -isystem bazel-out/host/bin/external/protobuf_archive/src -isystem external/farmhash_archive/src -isystem bazel-out/host/genfiles/external/farmhash_archive/src -isystem bazel-out/host/bin/external/farmhash_archive/src -isystem external/zlib_archive -isystem bazel-out/host/genfiles/external/zlib_archive -isystem bazel-out/host/bin/external/zlib_archive -isystem tensorflow/lite/schema -isystem bazel-out/host/genfiles/tensorflow/lite/schema -isystem bazel-out/host/bin/tensorflow/lite/schema -isystem external/flatbuffers/include -isystem bazel-out/host/genfiles/external/flatbuffers/include -isystem bazel-out/host/bin/external/flatbuffers/include -g0 -g0 -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c tensorflow/lite/toco/python/toco_python_api.cc -o bazel-out/host/bin/tensorflow/lite/toco/python/_objs/toco_python_api/toco_python_api.pic.o)

In file included from tensorflow/lite/toco/python/toco_python_api.cc:20:0:

./tensorflow/lite/toco/python/toco_python_api.h:18:20: fatal error: Python.h: No such file or directory

#include <Python.h>

以上问题可能是由于内存不足引起的，我们增加swap空间：

Ubuntu下:

# 生成swap镜像文件

sudo dd if=/dev/zero of=/mnt/512Mb.swap bs=1M count=512

# 对该镜像文件格式化

sudo mkswap /mnt/512Mb.swap

# 挂载该镜像文件

sudo swapon /mnt/512Mb.swap

centos:

. 检查是否配置swap：

swapon -s

或者free -m

total used free shared buff/cache available

Mem: 15885 1539 13593 8 752 14020

Swap: 0 0 0

. 查看系统存储

df -h

尝试指定jvm参数：

bazel build -c opt --verbose_failures --jvmopt="-server -Xms1024m" //tensorflow/tools/pip_package:build_pip_package

同样会出现该错误，接下来我们换成python3.6 试一下，使用anaconda创建一个python3.6的环境，激活环境并重新运行上述命令：

2. from tensorflow.python.keras.preprocessing.sequence import _remove_long_seq

File "/root/.cache/bazel/_bazel_root/48b0697df5815d95b49fd98c3083955b/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api_1_tf_python_api_gen_v1.runfiles/org_tensorflow/tensorflow/python/keras/preprocessing/__init__.py", line 21, in <module>

import keras_preprocessing

ModuleNotFoundError: No module named 'keras_preprocessing'

尝试安装keras，pip install keras，重新开始编译