kubernetes版本的升级主要是升级kubeadm、kubectl和kubelet，升级应遵循以下原则：
1、apiserver保护版本最大，其它功能最多比apiserver小一个小版本号。
2、不能跨小版本升级。
0、使用下列命令列出可用版本
yum list --showduplicates kubeadm --disableexcludes=kubernetes
操作系统CentOS7，这次升级从 1.15.0 升级到 1.16.0。
1、升级kubeadm
yum install -y kubeadm-1.16.0-0 --disableexcludes=kubernetes
kubectl drain k8s-master --ignore-daemonsets --force
kubeadm upgrade plan
kubeadm upgrade apply v1.16.0
kubectl uncordon k8s-master
yum install -y kubelet-1.16.0-0 kubectl-1.16.0-0 --disableexcludes=kubernetes
systemctl daemon-reload
systemctl restart kubelet
2、k8s-master升级后看到k8s-master处于NotReady状态，查看日志:journalctl -f -u kubelet，服务一直提示cni网络组件有问题
[root@k8s-master CNI]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master NotReady master 385d v1.16.0
k8s-node1 Ready node 385d v1.15.0
k8s-node2 Ready node 20h v1.15.0
journalctl -f -u kubelet
3、删除原有flannel重新创建下试试：
[root@k8s-master CNI]# kubectl delete -f https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml
##删除可能会报错：
kubectl get DaemonSet -n kube-system
##删除里面的关于flannel的DaemonSet：
kubectl delete XXX -n kube-system
[root@k8s-master CNI]# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
unable to recognize "kube-flannel.yml": no matches for kind "DaemonSet" in version "extensions/v1beta1"
unable to recognize "kube-flannel.yml": no matches for kind "DaemonSet" in version "extensions/v1beta1"
unable to recognize "kube-flannel.yml": no matches for kind "DaemonSet" in version "extensions/v1beta1"
unable to recognize "kube-flannel.yml": no matches for kind "DaemonSet" in version "extensions/v1beta1"
unable to recognize "kube-flannel.yml": no matches for kind "DaemonSet" in version "extensions/v1beta1"
4、换个flannel链接试试：
[root@k8s-master CNI]# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel configured
clusterrolebinding.rbac.authorization.k8s.io/flannel unchanged
serviceaccount/flannel unchanged
configmap/kube-flannel-cfg unchanged
daemonset.apps/kube-flannel-ds-amd64 created
daemonset.apps/kube-flannel-ds-arm64 created
daemonset.apps/kube-flannel-ds-arm created
daemonset.apps/kube-flannel-ds-ppc64le created
daemonset.apps/kube-flannel-ds-s390x created
5、我们查看k8s-master已经变成Ready状态
[root@k8s-master CNI]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 385d v1.16.0
k8s-node1 Ready node 385d v1.15.0
k8s-node2 Ready node 20h v1.15.0
6、升级业务节点（同样适用于剩余的其它控制节点）
yum install -y kubeadm-1.16.0-0 --disableexcludes=kubernetes
#在master节点执行
kubectl drain k8s-node-a --ignore-daemonsets --force
kubeadm upgrade node
yum install -y kubelet-1.16.0-0 kubectl-1.16.0-0 --disableexcludes=kubernetes
systemctl daemon-reload
systemctl restart kubelet
#在master节点执行
kubectl uncordon k8s-node-a
