背景：因为kubernetes中所有服务信息存储在etcd中，所以要保证etcd的高可用。

环境：
192.168.40.50   local-master
192.168.40.51   local-node1   
192.168.40.52   local-node2     
版本：etcd-3.1.0-2.el7.x86_64

1.修改etcd配置文件

[root@local-master docker]# cat /etc/etcd/etcd.conf |grep -v "^#"
ETCD_NAME=etcd0
ETCD_DATA_DIR="/var/lib/etcd/etcd0.etcd"
ETCD_LISTEN_PEER_URLS="http://local-master:2380"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://local-master:2380"
ETCD_INITIAL_CLUSTER="etcd0=http://local-master:2380,etcd1=http://local-node1:2380,etcd2=http://local-n
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://local-master:2379"

[root@local-node1 ~]# cat /etc/etcd/etcd.conf |grep -v "^#"
ETCD_NAME=etcd1
ETCD_DATA_DIR="/var/lib/etcd/etcd1.etcd"
ETCD_LISTEN_PEER_URLS="http://local-node1:2380"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://local-node1:2380"
ETCD_INITIAL_CLUSTER="etcd0=http://local-master:2380,etcd1=http://local-node1:2380,etcd2=http://local-node2:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://local-node1:2379"

[root@local-node2 ~]#  cat /etc/etcd/etcd.conf |grep -v "^#"
ETCD_NAME=etcd2
ETCD_DATA_DIR="/var/lib/etcd/etcd2.etcd"
ETCD_LISTEN_PEER_URLS="http://local-node2:2380"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://local-node2:2380"
ETCD_INITIAL_CLUSTER="etcd0=http://local-master:2380,etcd1=http://local-node1:2380,etcd2=http://local-node2:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://local-node2:2379"

三台机同时启动etcd
systemctl restart etcd && systemctl enable etcd

如果启动不成功，配置文件没有错。可以修改启动文件/usr/lib/systemd/system/etcd.service 。

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
User=etcd
# set GOMAXPROCS to number of processors
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/bin/etcd --name=\"${ETCD_NAME}\" --data-dir=\"${ETCD_DATA_DIR}\" --listen-client-urls=\"${ETCD_LISTEN_CLIENT_URLS}\" --listen-client-urls=\"${ETCD_LISTEN_CLIENT_URLS}\" --advertise-client-urls=\"${ETCD_ADVERTISE_CLIENT_URLS}\" --initial-cluster-token=\"${ETCD_INITIAL_CLUSTER_TOKEN}\" --initial-cluster=\"${ETCD_INITIAL_CLUSTER}\" --initial-cluster-state=\"${ETCD_INITIAL_CLUSTER_STATE}\""
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

2. 修改calico网络配置

vim /etc/calico/calico.env
ETCD_ENDPOINTS=http://local-master:2379,http://local-node1:2379,http://local-node2:2379

vim /etc/cni/net.d/10-calico.conf 
{
    "name": "calico-k8s-network",
    "type": "calico",
    "etcd_endpoints": "http://local-master:2379,http://local-node1:2379,http://local-node2:2379",
    "log_level": "info",
    "ipam": {
        "type": "calico-ipam"
    },
    "policy": {
        "type": "k8s"
    }

export ETCD_AUTHORITY=local-master:2379,local-node1:2379,local-node2:2379
systemctl restart calico-node

3.修改k8s配置文件

vi /etc/kubernetes/apiserver
KUBE_ETCD_SERVERS="--etcd-servers=http://192.168.40.50:2379,http://192.168.40.51:2379,http://192.168.40.52:2379"

重启master各个组件

systemctl restart etcd  kube-apiserver kube-controller-manager kube-scheduler
systemctl status etcd  kube-apiserver kube-controller-manager kube-scheduler

重启node各个组件

systemctl restart etcd docker kubelet  kube-proxy
systemctl status etcd docker kubelet  kube-proxy

4.检查

查看etcd的集群状态

[root@local-master docker]# etcdctl  member list
45ea5f54d4995521: name=etcd2 peerURLs=http://local-node2:2380 clientURLs=http://local-node2:2379 isLeader=false
a9a9c48b6dc0323e: name=etcd0 peerURLs=http://local-master:2380 clientURLs=http://local-master:2379 isLeader=true
d7b06390cd6b02b8: name=etcd1 peerURLs=http://local-node1:2380 clientURLs=http://local-node1:2379 isLeader=false

查看集群健康状态

[root@local-master docker]# etcdctl  cluster-health
member 45ea5f54d4995521 is healthy: got healthy result from http://local-node2:2379
member a9a9c48b6dc0323e is healthy: got healthy result from http://local-master:2379
member d7b06390cd6b02b8 is healthy: got healthy result from http://local-node1:2379
cluster is healthy

kubectl get nodes
查看k8s是否获取到etcd中的资源信息、

注意

在master主机上面kubernetes的进程不要依赖etcd启动，不然master主机上的etcd进程停止后，整个k8s集群会调度不起来

参考
https://mritd.me/2016/09/01/Etcd-%E9%9B%86%E7%BE%A4%E6%90%AD%E5%BB%BA/#43%E6%B5%8B%E8%AF%95
http://www.pangxie.space/docker/702
https://github.com/kubernetes/kubernetes/tree/master/test/fixtures/doc-yaml/admin/high-availability