1、环境
IP 服务 主机名
192.168.3.204 Swarm管理节点 controler
192.168.3.202 Swarm的node节点 node2
192.168.3.201 Swarm的node节点 node1
2、配置docker yum源（三台机器）
cd /etc/yum.repos.d/
touch docker.repo
vim docker.repo
添加内容如下：*
[dockerrepo]
name=Docker Repository
baseurl=https://yum.dockerproject.org/repo/main/centos/$releasever/
enabled=1
gpgcheck=1
gpgkey=https://yum.dockerproject.org/gpg
:wq保存退出
3、配置hosts表（三台机器）
vim /etc/hosts
添加内容如下：
192.168.3.204 controller
192.168.3.201 node1
192.168.3.202 node2
4、配置manager节点（192.168.3.204）
4.1、删除旧版本的环境
yum remove docker docker-client docker-client-latest docker-common docker-latest docker-latest-logrotate docker-logrotate docker-selinux docker-engine-selinux docker-engine
4.2、查看可用的docker版本
[root@controller lib]# yum list docker-engine.x86_64 --showduplicates |sort -r
Loading mirror speeds from cached hostfile
Loaded plugins: fastestmirror, langpacks
docker-engine.x86_64 1.9.1-1.el7.centos dockerrepo
docker-engine.x86_64 1.9.0-1.el7.centos dockerrepo
docker-engine.x86_64 1.8.3-1.el7.centos dockerrepo
docker-engine.x86_64 1.8.2-1.el7.centos dockerrepo
docker-engine.x86_64 1.8.1-1.el7.centos dockerrepo
docker-engine.x86_64 1.8.0-1.el7.centos dockerrepo
docker-engine.x86_64 1.7.1-1.el7.centos dockerrepo
docker-engine.x86_64 17.05.0.ce-1.el7.centos dockerrepo
docker-engine.x86_64 17.04.0.ce-1.el7.centos dockerrepo
docker-engine.x86_64 17.03.1.ce-1.el7.centos dockerrepo
docker-engine.x86_64 17.03.0.ce-1.el7.centos dockerrepo
docker-engine.x86_64 1.7.0-1.el7.centos dockerrepo
docker-engine.x86_64 1.13.1-1.el7.centos dockerrepo
docker-engine.x86_64 1.13.0-1.el7.centos dockerrepo
docker-engine.x86_64 1.12.6-1.el7.centos dockerrepo
docker-engine.x86_64 1.12.5-1.el7.centos dockerrepo
docker-engine.x86_64 1.12.4-1.el7.centos dockerrepo
docker-engine.x86_64 1.12.3-1.el7.centos dockerrepo
docker-engine.x86_64 1.12.2-1.el7.centos dockerrepo
docker-engine.x86_64 1.12.1-1.el7.centos dockerrepo
docker-engine.x86_64 1.12.0-1.el7.centos dockerrepo
docker-engine.x86_64 1.11.2-1.el7.centos dockerrepo
docker-engine.x86_64 1.11.1-1.el7.centos dockerrepo
docker-engine.x86_64 1.11.0-1.el7.centos dockerrepo
docker-engine.x86_64 1.10.3-1.el7.centos dockerrepo
docker-engine.x86_64 1.10.2-1.el7.centos dockerrepo
docker-engine.x86_64 1.10.1-1.el7.centos dockerrepo
docker-engine.x86_64 1.10.0-1.el7.centos dockerrepo
Available Packages
4.3、安装稳定的版本1.13.1
[root@controller lib]# yum -y install docker-engine-1.13.1-1.el7.centos
启动docker服务并设置开机自启动
systemctl start docker
systemctl enable docker
4.4、下载swarm镜像
[root@controller ~]# docker pull swarm
Using default tag: latest
latest: Pulling from library/swarm
Digest: sha256:406022f04a3d0c5ce4dbdb60422f24052c20ab7e6d41ebe5723aa649c3833975
Status: Downloaded newer image for swarm:latest
[root@controller ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
swarm latest ff454b4a0e84 7 weeks ago 12.7 MB
4.5、创建swarm
[root@controller ~]# docker swarm init --advertise-addr 192.168.3.204
Swarm initialized: current node (r344b4bz1ug3bnw6olsxzqtis) is now a manager.
To add a worker to this swarm, run the following command:
docker swarm join \
--token SWMTKN-1-1kvrsn7j74ie7fqxzxb0wndk76gjnwdyc6jnhe4771h55edgzw-20spfbubw7ckh1w0w3plvsak3 \
192.168.3.204:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
注：上面的token值要保存，其他节点均通过它加入集群
4.6、查看集群信息
[root@controller ~]# docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 1
Server Version: 1.13.1
Storage Driver: overlay
Backing Filesystem: xfs
Supports d_type: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: active
NodeID: r344b4bz1ug3bnw6olsxzqtis
Is Manager: true
ClusterID: p5r0a72jdoq9cztshyf0bvfye
Managers: 1
Nodes: 1
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 3
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Node Address: 192.168.3.204
Manager Addresses:
192.168.3.204:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1
runc version: 9df8b306d01f59d3a8029be411de015b7304dd8f
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 3.10.0-327.18.2.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.551 GiB
Name: controller
ID: 5E6V:BHYM:3L74:EYS3:ZUID:INQP:DINI:U7I6:QKJZ:UCYY:QZJF:GXGN
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
4.7、查看集群节点信息
[root@controller ~]# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
r344b4bz1ug3bnw6olsxzqtis * controller Ready Active Leader
5、配置节点（192.168.3.202,192.168.3.201）
5.1、安装docker
yum -y install docker-engine-1.13.1-1.el7.centos
启动docker
systemctl start docker
开机自启动
systemctl enable docker
5.2、加入到集群
docker swarm join --token SWMTKN-1-1kvrsn7j74ie7fqxzxb0wndk76gjnwdyc6jnhe4771h55edgzw-20spfbubw7ckh1w0w3plvsak3 192.168.3.204:2377
5.3、查看docker信息（两个节点一样执行，我这里只贴出在node2执行的信息）
[root@node2 ~]# docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 1.13.1
Storage Driver: overlay
Backing Filesystem: xfs
Supports d_type: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: active
NodeID: pdqau3dpxtgnr6gg78zbt7nnz
Is Manager: false
Node Address: 192.168.3.202
Manager Addresses:
192.168.3.204:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1
runc version: 9df8b306d01f59d3a8029be411de015b7304dd8f
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 3.10.0-229.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.797 GiB
Name: node2
ID: Q7VU:TMEP:NCO6:YVXC:O6U2:2DON:5RUL:VGJ4:D2TA:345M:I4Q6:UHN5
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: bridge-nf-call-ip6tables is disabled
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
5.4、在集群节点（192.168.3.204）查看节点信息
[root@controller ~]# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
hjba4ers0ahzbdvggajc3b73s node1 Ready Active
pdqau3dpxtgnr6gg78zbt7nnz node2 Ready Active
r344b4bz1ug3bnw6olsxzqtis * controller Ready Active Leader
6、在swarm中部署nginx服务
6.1、创建覆盖网络（保障不同主机容器互通）
[root@controller ~]# docker network create -d overlay nginx
692ud2yyqimkuq6rufpnwnt05
[root@controller ~]# docker network ls
NETWORK ID NAME DRIVER SCOPE
27940e4bc00a bridge bridge local
cd675706a24e docker_gwbridge bridge local
ec66ee0e74b1 host host local
l8d4j4w5o52u ingress overlay swarm
692ud2yyqimk nginx overlay swarm
3a44f24d7c1f none null local
6.2、创建服务
创建：
[root@controller ~]# docker service create --replicas 1 --network nginx --name MyNginx -p 80:80 nginx
8sk44dipvooy4sb2e7e4l8wb1
查看服务
[root@controller ~]# docker service ls
ID NAME MODE REPLICAS IMAGE
8sk44dipvooy MyNginx replicated 1/1 nginx:latest
查看摘要信息：
[root@controller ~]# docker service inspect --pretty MyNginx
ID: 8sk44dipvooy4sb2e7e4l8wb1
Name: MyNginx
Service Mode: Replicated
Replicas: 1
Placement:
UpdateConfig:
Parallelism: 1
On failure: pause
Max failure ratio: 0
ContainerSpec:
Image: nginx:latest@sha256:d85914d547a6c92faa39ce7058bd7529baacab7e0cd4255442b04577c4d1f424
Resources:
Networks: nginx
Endpoint Mode: vip
Ports:
PublishedPort 80
Protocol = tcp
TargetPort = 80
6.3、查看服务的运行情况
[root@controller ~]# docker service ps MyNginx
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
khvp4r03de4f MyNginx.1 nginx:latest node2 Running Running about a minute ago
6.4、访问服务
浏览器或者curl访问服务http://192.168.3.204
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
7、Service的复制功能
7.1、增加
设置5个副本
[root@controller ~]# docker service scale MyNginx=5
MyNginx scaled to 5
[root@controller ~]# docker service ps MyNginx
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
khvp4r03de4f MyNginx.1 nginx:latest node2 Running Running 12 minutes ago
wecu7jye2mg3 MyNginx.2 nginx:latest controller Running Preparing 5 seconds ago
5a6rp8c6arvt MyNginx.3 nginx:latest node2 Running Running 4 seconds ago
68lu0fpg75on MyNginx.4 nginx:latest node1 Running Preparing 5 seconds ago
todza2i5lnnt MyNginx.5 nginx:latest controller Running Preparing 5 seconds ago
7.2、减少
变更为1个副本
[root@controller ~]# docker service scale MyNginx=1
MyNginx scaled to 1
[root@controller ~]# docker service ls
ID NAME MODE REPLICAS IMAGE
8sk44dipvooy MyNginx replicated 1/1 nginx:latest
[root@controller ~]# docker service ps MyNginx
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
exoh860irad5 MyNginx.1 nginx:latest node1 Running Running 4 minutes ago
8、使用volume
8.1、查看volume
[root@controller ~]# docker volume ls
DRIVER VOLUME NAME
8.2、创建volume
[root@controller ~]# docker volume create --name nginx
nginx
[root@controller ~]# docker volume ls
DRIVER VOLUME NAME
local nginx
8.3、创建服务并指定挂载
src是volume名称
dst是挂载点（也就是容器内的路径，如果不存在会自动创建）
注：若指定的是本机的路径，需要把type改成bind, src是本机的路径
[root@controller ~]# docker service create --replicas 2 --mount type=volume,src=nginx,dst=/farmer --name ng nginx
r7bld5xg9cy6285z6207wlly2
[root@controller ~]# docker service ls
ID NAME MODE REPLICAS IMAGE
9et4o10fn0w2 MyNginx replicated 2/2 nginx:latest
r7bld5xg9cy6 ng replicated 2/2 nginx:latest
[root@controller ~]# docker service ps ng
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
190gm96z6brc ng.1 nginx:latest node2 Running Running 25 seconds ago
093pql11ttdm ng.2 nginx:latest node1 Running Running 25 seconds ago
9、创建nginx服务和负载均衡的测试
9.1、创建volume名称为modules
docker volume create --name modules
注：因为/etc/nginx/mudules是软连接到/usr/lib/nginx/modules，一旦将/etc/nginx挂载到本地，这个目录将会失效，所以需要在modules也挂到本地
lrwxrwxrwx 1 root root 22 Jul 24 21:02 modules -> /usr/lib/nginx/modules
9.2、创建软连接（所有节点，192.168.3.204,192.168.3.202,192.168.3.201）
mkdir /usr/lib/nginx
ln -s /var/lib/docker/volumes/modules/_data/modules/ /usr/lib/nginx/modules
9.3、创建test服务
[root@controller ~]# docker service create --replicas 2 --mount type=volume,src=nginx,dst=/etc/nginx/ --mount type=volume,dst=/usr/lib/nginx/,src=modules --name test -p 8880:80 nginx
9mt9wuabsluykwfh2j7ez9gut
[root@controller ~]# docker service ps test
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
hwp14enb23gz test.1 nginx:latest node1 Running Running 6 seconds ago
t6mpq6sbongg test.2 nginx:latest controller Running Running 6 seconds ago
9.4、配置HTML服务（三个节点，192.168.3.201,192.168.3.202,192.168.3.204）
cd /var/lib/docker/volumes/nginx/_data
mkdir html
vim conf.d/default.conf
修改如下：
location / {
root /etc/nginx/html;
index index.html index.htm;
}
注：这里用/etc/nginx/html路径是因为在docker里面路径是这个
我把创建的新的html当做nginx的root目录
9.4.1、管理节点
cd /var/lib/docker/volumes/nginx/_data/html
touch index.html
添加内容：
<h1>Controller</h1>
9.4.1、node1节点
cd /var/lib/docker/volumes/nginx/_data/html
touch index.html
添加内容：
<h1>Node1</h1>
9.4.2、node2节点
cd /var/lib/docker/volumes/nginx/_data/html
touch index.html
添加内容：
<h1>Node2</h1>
9.5、重启nginx服务
当前服务在node1和controller节点运行，故而只在这两个节点执行（机器环境不同，所在的节点不同，请看清自己的节点）
9.5.1、node1节点
[root@node1 ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
33b4e08c57d0 nginx@sha256:d85914d547a6c92faa39ce7058bd7529baacab7e0cd4255442b04577c4d1f424 "nginx -g 'daemon ..." 4 hours ago Up 4 hours 80/tcp test.1.ov76pdo715cc1f24i5vfgu6si
docker exec -ti 33b4e08c57d0 /bin/bash
/etc/init.d/nginx reload
9.5.2、controller节点
[root@controller _data]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e54913d64ce9 nginx@sha256:d85914d547a6c92faa39ce7058bd7529baacab7e0cd4255442b04577c4d1f424 "nginx -g 'daemon ..." 3 days ago Up 3 days 80/tcp test.2.t6mpq6sbongg5rq8rd4lofp6u
[root@controller _data]# docker exec -ti e54913d64ce9 /bin/bash
root@e54913d64ce9:/# /etc/init.d/nginx reload
9.6、测试服务（服务是否正常，是否有负载）
注：测试过程中有时候多curl几次才会负载到另外的一个节点
[root@mysql76 ~]# curl http://192.168.3.204:8880/
<h1>Node1</h1>
[root@mysql76 ~]# curl http://192.168.3.204:8880/
<h1>Controller</h1>
9.7、测试集群
在节点1执行
docker stop 33b4e08c57d0
查看集群服务
[root@controller _data]# docker service ps test
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
ov76pdo715cc test.1 nginx:latest node2 Running Running 1 hours ago
qoyuk652tky2 \_ test.1 nginx:latest node1 Shutdown Complete 1 hours ago
t6mpq6sbongg test.2 nginx:latest controller Running Running 3 days ago
测试集群服务（服务是否正常，是否有负载）
注：测试过程中有时候多curl几次才会负载到另外的一个节点
[root@mysql76 ~]# curl http://192.168.3.204:8880/
<h1>Node2</h1>
[root@mysql76 ~]# curl http://192.168.3.204:8880/
<h1>Controller</h1>