1、serving分为 gpu版和cpu版、gpu版需要 navida显卡

2、cpu版本：

docker pull tensorflow/serving
sudo docker run -p 8500:8500 -p 8501:8501 \                                                                
--mount type=bind,source=/Users/dequan/tensorflow/models,target=/models \
-t  emacski/tensorflow-serving:latest \
--enable_batching=true \
--model_config_file=/models/models.config &

3、gpu版本

docker run --runtime=nvidia -d --name tensorflow-serving  -p 8500:8500 -p  8501:8501 --mount type=bind,source=/home/tensorflow/models,target=/models -t tensorflow/serving:2.11.0-gpu --enable_batching=true --model_config_file=/models/models.config 

建议centos、debian、ubuntu 系统

3.1、安装 NVIDIA driver

参考 https://docs.nvidia.com/datacenter/tesla/tesla-installation-notes/index.html#runfile

1）、安装前验证
#看硬件是否支持 nvida
lspci | grep -i nvidia
#看是不是64位操作系统
uname -m && cat /etc/*release
# 查看是否支持gcc
gcc --version
# Verify the System has the Correct Kernel Headers and Development Packages Installed
uname -r
#The kernel headers and development packages for the currently running kernel can be installed with: centos7
sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r)

2)、centos7 package安装

#安装依赖
sudo dnf install -y tar bzip2 make automake gcc gcc-c++ pciutils elfutils-libelf-devel libglvnd-devel iptables firewalld vim bind-utils wget

# EPEL for DKMS
sudo yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

#Install the CUDA repository public GPG key.
distribution=rhel7

#Setup the CUDA network repository.
ARCH=$( /bin/arch )
sudo yum-config-manager --add-repo http://developer.download.nvidia.com/compute/cuda/repos/$distribution/${ARCH}/cuda-$distribution.repo

#The NVIDIA driver requires that the kernel headers and development packages for the running version of the kernel be installed at the time of the driver installation, as well whenever the driver is rebuilt. For example, if your system is running kernel version 4.4.0, the 4.4.0 kernel headers and development packages must also be installed.
For CentOS 7, ensure that the system has the correct Linux kernel sources from the CentOS repositories:
$ sudo yum install -y kernel-devel-$(uname -r) kernel-headers-$(uname -r)

#Update the repository cache and install the driver using the nvidia-driver-latest-dkms meta-package.
$ sudo yum clean expire-cache
$ sudo yum install -y nvidia-driver-latest-dkms

参考https://github.com/NVIDIA/nvidia-docker

https://blog.csdn.net/qq_36287702/article/details/122478012

curl http://localhost:8501/v1/models/number/metadata #查看接口详细信息
curl http://localhost:8501/v1/models/number #查看接口状态
curl -d '{"instances": [1.0, 2.0, 5.0]}' \
  -X POST http://localhost:8501/v1/models/half_plus_two:predict #请求接口， instances 里可以放多个对象