ubuntu 18.04.4 kubeadmin 安装单服务器k8s集群实践

参见 https://www.jianshu.com/p/13b541cc669b

1. 网络配置准备

https://manpages.ubuntu.com/manpages/bionic/man5/modules.5.html
https://manpages.ubuntu.com/manpages/bionic/man5/modprobe.d.5.html

test@k8s_single:/etc$ sudo sh -c 'echo "br_netfilter" >> /etc/modules'
test@k8s_single:/etc$ sudo reboot

test@k8s_single:~$ lsmod |grep br_netfilter
br_netfilter           24576  0
bridge                155648  1 br_netfilter

2. 安装Docker

官网文档

卸载旧版本

sudo apt-get remove docker docker-engine docker.io containerd runc

设置repository

sudo apt-get update
sudo apt-get install  apt-transport-https  ca-certificates  curl  gnupg-agent  software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

安装指定版本
安装默认版本使用命令sudo apt-get install docker-ce docker-ce-cli containerd.io
查看版本apt-cache madison docker-ce

sudo apt-get update
apt-cache madison docker-ce
sudo apt-get install docker-ce=5:18.09.9~3-0~ubuntu-bionic docker-ce-cli=5:18.09.9~3-0~ubuntu-bionic containerd.io

测试

test@k8s_single:~$ sudo docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

3. 安装 kubeadm, kubelet and kubectl

以root执行

sudo apt-get update && apt-get install -y apt-transport-https
curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -

cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF

apt-get update

查看安装版本，我们需要安装1.16.3版本，以与云服务商保持一致

apt-cache madison kubeadm
apt-cache madison kubelet
apt-cache madison kubectl 
apt-get install kubeadm=1.16.3-00 kubelet=1.16.3-00 kubectl=1.16.3-00

检查kubelet服务是否启动，并设置开机启动

test@k8s_single:~$ sudo systemctl is-active kubelet
activating
test@k8s_single:~$ sudo systemctl is-enabled kubelet
enabled

4. 初始化 control-plane 节点

kubeadm-init 官方文档
--control-plane-endpoint 如果需要升级为HA，需要配置
--kubernetes-version 指定k8s版本
--image-repository 指定仓库（google被墙了哎~）
--pod-network-cidr 指定pod网段
--service-cidr 指定service网段， Default: "10.96.0.0/12"
--service-dns-domain 指定service默认域名
--v 日志输出级别

主机名规范
nodeRegistration.name: Invalid value: "k8s_single": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is 'a-z0-9?(.a-z0-9?)*')

必须关闭swap

sudo kubeadm init  \
--image-repository=registry.aliyuncs.com/google_containers \
--kubernetes-version=1.16.3 \
--control-plane-endpoint="cp:6443" \
--pod-network-cidr=172.16.0.0/16 \
--service-dns-domain=rha.local

5. 配置使用kubectl

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

验证

test@singlek8s:~$ kubectl get nodes
NAME        STATUS     ROLES    AGE    VERSION
singlek8s   NotReady   master   4m8s   v1.16.3

test@singlek8s:~$ kubectl get services
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   4m22s

test@singlek8s:~$ kubectl get pods
No resources found in default namespace.

test@singlek8s:~$ kubectl get pods --all-namespaces
NAMESPACE     NAME                                READY   STATUS    RESTARTS   AGE
kube-system   coredns-58cc8c89f4-44fxc            0/1     Pending   0          17h
kube-system   coredns-58cc8c89f4-xsr4k            0/1     Pending   0          17h
kube-system   etcd-singlek8s                      1/1     Running   0          17h
kube-system   kube-apiserver-singlek8s            1/1     Running   0          17h
kube-system   kube-controller-manager-singlek8s   1/1     Running   0          17h
kube-system   kube-proxy-vvgkh                    1/1     Running   0          17h
kube-system   kube-scheduler-singlek8s            1/1     Running   0          17h

初始化成功后的运行实例

test@singlek8s:~$ sudo docker images
REPOSITORY                                                        TAG                 IMAGE ID            CREATED             SIZE
registry.aliyuncs.com/google_containers/kube-proxy                v1.16.3             9b65a0f78b09        6 months ago        86.1MB
registry.aliyuncs.com/google_containers/kube-apiserver            v1.16.3             df60c7526a3d        6 months ago        217MB
registry.aliyuncs.com/google_containers/kube-controller-manager   v1.16.3             bb16442bcd94        6 months ago        163MB
registry.aliyuncs.com/google_containers/kube-scheduler            v1.16.3             98fecf43a54f        6 months ago        87.3MB
registry.aliyuncs.com/google_containers/etcd                      3.3.15-0            b2756210eeab        9 months ago        247MB
registry.aliyuncs.com/google_containers/coredns                   1.6.2               bf261d157914        9 months ago        44.1MB
registry.aliyuncs.com/google_containers/pause                     3.1                 da86e6ba6ca1        2 years ago         742kB

test@singlek8s:~$ sudo docker ps
CONTAINER ID        IMAGE                                               COMMAND                  CREATED             STATUS              PORTS               NAMES
bd1684f8f8b0        9b65a0f78b09                                        "/usr/local/bin/kube…"   16 hours ago        Up 16 hours                             k8s_kube-proxy_kube-proxy-vvgkh_kube-system_269b5059-d4f6-4ec0-b308-023d06bd2274_0
918ee5334ad6        registry.aliyuncs.com/google_containers/pause:3.1   "/pause"                 16 hours ago        Up 16 hours                             k8s_POD_kube-proxy-vvgkh_kube-system_269b5059-d4f6-4ec0-b308-023d06bd2274_0
71ef90f16d8f        b2756210eeab                                        "etcd --advertise-cl…"   16 hours ago        Up 16 hours                             k8s_etcd_etcd-singlek8s_kube-system_07dc93a617eee4df326d01a7a21c55f4_0
505f6328202b        98fecf43a54f                                        "kube-scheduler --au…"   16 hours ago        Up 16 hours                             k8s_kube-scheduler_kube-scheduler-singlek8s_kube-system_f48641826bbe4a7f22cd206f2178ae9e_0
8aa2734e242d        df60c7526a3d                                        "kube-apiserver --ad…"   16 hours ago        Up 16 hours                             k8s_kube-apiserver_kube-apiserver-singlek8s_kube-system_4cbaa4c6a36c32a12652755a08b4bdcf_0
94cea3cb3928        bb16442bcd94                                        "kube-controller-man…"   16 hours ago        Up 16 hours                             k8s_kube-controller-manager_kube-controller-manager-singlek8s_kube-system_4836eaa6d5f0cee1d4aae17d1eed2da7_0
92cf325725d6        registry.aliyuncs.com/google_containers/pause:3.1   "/pause"                 16 hours ago        Up 16 hours                             k8s_POD_kube-scheduler-singlek8s_kube-system_f48641826bbe4a7f22cd206f2178ae9e_0
3684c0b65239        registry.aliyuncs.com/google_containers/pause:3.1   "/pause"                 16 hours ago        Up 16 hours                             k8s_POD_kube-controller-manager-singlek8s_kube-system_4836eaa6d5f0cee1d4aae17d1eed2da7_0
379cdaa46a03        registry.aliyuncs.com/google_containers/pause:3.1   "/pause"                 16 hours ago        Up 16 hours                             k8s_POD_kube-apiserver-singlek8s_kube-system_4cbaa4c6a36c32a12652755a08b4bdcf_0
de5f1ac61d67        registry.aliyuncs.com/google_containers/pause:3.1   "/pause"                 16 hours ago        Up 16 hours                             k8s_POD_etcd-singlek8s_kube-system_07dc93a617eee4df326d01a7a21c55f4_0

etcd

CONTAINER ID: 71ef90f16d8f
IMAGE: etcd
NAMES: k8s_etcd_etcd-singlek8s_kube-system_07dc93a617eee4df326d01a7a21c55f4_0
COMMAND: etcd --advertise-client-urls=https://10.0.31.49:2379 --cert-file=/etc/kubernetes/pki/etcd/server.crt --client-cert-auth=true --data-dir=/var/lib/etcd --initial-advertise-peer-urls=https://10.0.31.49:2380 --initial-cluster=singlek8s=https://10.0.31.49:2380 --key-file=/etc/kubernetes/pki/etcd/server.key --listen-client-urls=https://127.0.0.1:2379,https://10.0.31.49:2379 --listen-metrics-urls=http://127.0.0.1:2381 --listen-peer-urls=https://10.0.31.49:2380 --name=singlek8s --peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt --peer-client-cert-auth=true --peer-key-file=/etc/kubernetes/pki/etcd/peer.key --peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt --snapshot-count=10000 --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
&
CONTAINER ID: de5f1ac61d67
IMAGE: pause
NAMES: k8s_POD_etcd-singlek8s_kube-system_07dc93a617eee4df326d01a7a21c55f4_0
COMMAND: /pause

kube-proxy

CONTAINER ID: bd1684f8f8b0
IMAGE: kube-proxy
NAMES: k8s_kube-proxy_kube-proxy-vvgkh_kube-system_269b5059-d4f6-4ec0-b308-023d06bd2274_0
COMMAND: /usr/local/bin/kube-proxy --config=/var/lib/kube-proxy/config.conf --hostname-override=singlek8s
&
CONTAINER ID: 918ee5334ad6
IMAGE: pause
NAMES: k8s_POD_kube-proxy-vvgkh_kube-system_269b5059-d4f6-4ec0-b308-023d06bd2274_0
COMMAND: /pause

kube-scheduler

CONTAINER ID: 505f6328202b
IMAGE: kube-scheduler
NAMES: k8s_kube-scheduler_kube-scheduler-singlek8s_kube-system_f48641826bbe4a7f22cd206f2178ae9e_0
COMMAND: kube-scheduler --authentication-kubeconfig=/etc/kubernetes/scheduler.conf --authorization-kubeconfig=/etc/kubernetes/scheduler.conf --bind-address=127.0.0.1 --kubeconfig=/etc/kubernetes/scheduler.conf --leader-elect=true
&
CONTAINER ID: 92cf325725d6
IMAGE: pause
NAMES: k8s_POD_kube-scheduler-singlek8s_kube-system_f48641826bbe4a7f22cd206f2178ae9e_0
COMMAND: /pause

kube-apiserver

CONTAINER ID: 8aa2734e242d
IMAGE: kube-apiserver
NAMES: k8s_kube-apiserver_kube-apiserver-singlek8s_kube-system_4cbaa4c6a36c32a12652755a08b4bdcf_0
COMMAND: kube-apiserver --advertise-address=10.0.31.49 --allow-privileged=true --authorization-mode=Node,RBAC --client-ca-file=/etc/kubernetes/pki/ca.crt --enable-admission-plugins=NodeRestriction --enable-bootstrap-token-auth=true --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key --etcd-servers=https://127.0.0.1:2379 --insecure-port=0 --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key --requestheader-allowed-names=front-proxy-client --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6443 --service-account-key-file=/etc/kubernetes/pki/sa.pub --service-cluster-ip-range=10.96.0.0/12 --tls-cert-file=/etc/kubernetes/pki/apiserver.crt --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
&
CONTAINER ID: 379cdaa46a03
IMAGE: pause
NAMES: k8s_POD_kube-apiserver-singlek8s_kube-system_4cbaa4c6a36c32a12652755a08b4bdcf_0
COMMAND: /pause

kube-controller-manager

CONTAINER ID: 94cea3cb3928
IMAGE: kube-controller-manager
NAMES: k8s_kube-controller-manager_kube-controller-manager-singlek8s_kube-system_4836eaa6d5f0cee1d4aae17d1eed2da7_0
COMMAND: kube-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/etc/kubernetes/controller-manager.conf --authorization-kubeconfig=/etc/kubernetes/controller-manager.conf --bind-address=127.0.0.1 --client-ca-file=/etc/kubernetes/pki/ca.crt --cluster-cidr=172.16.0.0/16 --cluster-signing-cert-file=/etc/kubernetes/pki/ca.crt --cluster-signing-key-file=/etc/kubernetes/pki/ca.key --controllers=*,bootstrapsigner,tokencleaner --kubeconfig=/etc/kubernetes/controller-manager.conf --leader-elect=true --node-cidr-mask-size=24 --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt --root-ca-file=/etc/kubernetes/pki/ca.crt --service-account-private-key-file=/etc/kubernetes/pki/sa.key --service-cluster-ip-range=10.96.0.0/12 --use-service-account-credentials=true
&
CONTAINER ID: 3684c0b65239
IMAGE: pause
NAMES: k8s_POD_kube-controller-manager-singlek8s_kube-system_4836eaa6d5f0cee1d4aae17d1eed2da7_0
COMMAND: /pause

6. 安装Pod网络插件

test@singlek8s:~$ kubectl apply -f https://docs.projectcalico.org/v3.14/manifests/calico.yaml
configmap/calico-config created
customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created
clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrole.rbac.authorization.k8s.io/calico-node created
clusterrolebinding.rbac.authorization.k8s.io/calico-node created
daemonset.apps/calico-node created
serviceaccount/calico-node created
deployment.apps/calico-kube-controllers created
serviceaccount/calico-kube-controllers created

检查。coredns运行正常即部署成功。

test@singlek8s:~$ kubectl get pods --all-namespaces
NAMESPACE     NAME                                       READY   STATUS     RESTARTS   AGE
kube-system   calico-kube-controllers-77d6cbc65f-gvlx8   0/1     Pending    0          47s
kube-system   calico-node-dr28q                          0/1     Init:2/3   0          47s
kube-system   coredns-58cc8c89f4-44fxc                   0/1     Pending    0          16h
kube-system   coredns-58cc8c89f4-xsr4k                   0/1     Pending    0          16h
kube-system   etcd-singlek8s                             1/1     Running    0          16h
kube-system   kube-apiserver-singlek8s                   1/1     Running    0          16h
kube-system   kube-controller-manager-singlek8s          1/1     Running    0          16h
kube-system   kube-proxy-vvgkh                           1/1     Running    0          16h
kube-system   kube-scheduler-singlek8s                   1/1     Running    0          16h

... output omitted ...

test@singlek8s:~$ kubectl get pods --all-namespaces
NAMESPACE     NAME                                       READY   STATUS    RESTARTS   AGE
kube-system   calico-kube-controllers-77d6cbc65f-gvlx8   1/1     Running   0          2m7s
kube-system   calico-node-dr28q                          1/1     Running   0          2m7s
kube-system   coredns-58cc8c89f4-44fxc                   1/1     Running   0          16h
kube-system   coredns-58cc8c89f4-xsr4k                   1/1     Running   0          16h
kube-system   etcd-singlek8s                             1/1     Running   0          16h
kube-system   kube-apiserver-singlek8s                   1/1     Running   0          16h
kube-system   kube-controller-manager-singlek8s          1/1     Running   0          16h
kube-system   kube-proxy-vvgkh                           1/1     Running   0          16h
kube-system   kube-scheduler-singlek8s                   1/1     Running   0          16h

检查下载的容器镜像

test@singlek8s:~$ sudo docker images
REPOSITORY                                                        TAG                 IMAGE ID            CREATED             SIZE
calico/node                                                       v3.14.1             04a9b816c753        4 days ago          263MB
calico/pod2daemon-flexvol                                         v3.14.1             7f93af2e7e11        4 days ago          112MB
calico/cni                                                        v3.14.1             35a7136bc71a        4 days ago          225MB
calico/kube-controllers                                           v3.14.1             ac08a3af350b        4 days ago          52.8MB
registry.aliyuncs.com/google_containers/kube-apiserver            v1.16.3             df60c7526a3d        6 months ago        217MB
registry.aliyuncs.com/google_containers/kube-proxy                v1.16.3             9b65a0f78b09        6 months ago        86.1MB
registry.aliyuncs.com/google_containers/kube-controller-manager   v1.16.3             bb16442bcd94        6 months ago        163MB
registry.aliyuncs.com/google_containers/kube-scheduler            v1.16.3             98fecf43a54f        6 months ago        87.3MB
registry.aliyuncs.com/google_containers/etcd                      3.3.15-0            b2756210eeab        9 months ago        247MB
registry.aliyuncs.com/google_containers/coredns                   1.6.2               bf261d157914        9 months ago        44.1MB
registry.aliyuncs.com/google_containers/pause                     3.1                 da86e6ba6ca1        2 years ago         742kB

检查运行的容器

test@singlek8s:~$ sudo docker ps
CONTAINER ID        IMAGE                                               COMMAND                  CREATED             STATUS              PORTS               NAMES
608678f945ae        calico/kube-controllers                             "/usr/bin/kube-contr…"   6 minutes ago       Up 6 minutes                            k8s_calico-kube-controllers_calico-kube-controllers-77d6cbc65f-gvlx8_kube-system_0f0e6d8b-2bfa-49e0-84de-274fa2986e83_0
a44cb8ee73ef        bf261d157914                                        "/coredns -conf /etc…"   6 minutes ago       Up 6 minutes                            k8s_coredns_coredns-58cc8c89f4-xsr4k_kube-system_ae7cb050-500d-4099-8a9d-b72f19248b57_0
7d8b43b4a6fa        registry.aliyuncs.com/google_containers/pause:3.1   "/pause"                 6 minutes ago       Up 6 minutes                            k8s_POD_coredns-58cc8c89f4-xsr4k_kube-system_ae7cb050-500d-4099-8a9d-b72f19248b57_18
bc035642c4a3        bf261d157914                                        "/coredns -conf /etc…"   6 minutes ago       Up 6 minutes                            k8s_coredns_coredns-58cc8c89f4-44fxc_kube-system_f466fca9-56e5-468b-9955-75462040b7b9_0
b153bb8f3801        registry.aliyuncs.com/google_containers/pause:3.1   "/pause"                 6 minutes ago       Up 6 minutes                            k8s_POD_calico-kube-controllers-77d6cbc65f-gvlx8_kube-system_0f0e6d8b-2bfa-49e0-84de-274fa2986e83_16
4da55b16e269        calico/node                                         "start_runit"            6 minutes ago       Up 6 minutes                            k8s_calico-node_calico-node-dr28q_kube-system_27f2dc0e-6784-4701-8fa5-9f42d5b78f7b_0
052dc4939146        registry.aliyuncs.com/google_containers/pause:3.1   "/pause"                 6 minutes ago       Up 6 minutes                            k8s_POD_coredns-58cc8c89f4-44fxc_kube-system_f466fca9-56e5-468b-9955-75462040b7b9_17
57d62f467406        registry.aliyuncs.com/google_containers/pause:3.1   "/pause"                 8 minutes ago       Up 8 minutes                            k8s_POD_calico-node-dr28q_kube-system_27f2dc0e-6784-4701-8fa5-9f42d5b78f7b_0
bd1684f8f8b0        9b65a0f78b09                                        "/usr/local/bin/kube…"   17 hours ago        Up 17 hours                             k8s_kube-proxy_kube-proxy-vvgkh_kube-system_269b5059-d4f6-4ec0-b308-023d06bd2274_0
918ee5334ad6        registry.aliyuncs.com/google_containers/pause:3.1   "/pause"                 17 hours ago        Up 17 hours                             k8s_POD_kube-proxy-vvgkh_kube-system_269b5059-d4f6-4ec0-b308-023d06bd2274_0
71ef90f16d8f        b2756210eeab                                        "etcd --advertise-cl…"   17 hours ago        Up 17 hours                             k8s_etcd_etcd-singlek8s_kube-system_07dc93a617eee4df326d01a7a21c55f4_0
505f6328202b        98fecf43a54f                                        "kube-scheduler --au…"   17 hours ago        Up 17 hours                             k8s_kube-scheduler_kube-scheduler-singlek8s_kube-system_f48641826bbe4a7f22cd206f2178ae9e_0
8aa2734e242d        df60c7526a3d                                        "kube-apiserver --ad…"   17 hours ago        Up 17 hours                             k8s_kube-apiserver_kube-apiserver-singlek8s_kube-system_4cbaa4c6a36c32a12652755a08b4bdcf_0
94cea3cb3928        bb16442bcd94                                        "kube-controller-man…"   17 hours ago        Up 17 hours                             k8s_kube-controller-manager_kube-controller-manager-singlek8s_kube-system_4836eaa6d5f0cee1d4aae17d1eed2da7_0
92cf325725d6        registry.aliyuncs.com/google_containers/pause:3.1   "/pause"                 17 hours ago        Up 17 hours                             k8s_POD_kube-scheduler-singlek8s_kube-system_f48641826bbe4a7f22cd206f2178ae9e_0
3684c0b65239        registry.aliyuncs.com/google_containers/pause:3.1   "/pause"                 17 hours ago        Up 17 hours                             k8s_POD_kube-controller-manager-singlek8s_kube-system_4836eaa6d5f0cee1d4aae17d1eed2da7_0
379cdaa46a03        registry.aliyuncs.com/google_containers/pause:3.1   "/pause"                 17 hours ago        Up 17 hours                             k8s_POD_kube-apiserver-singlek8s_kube-system_4cbaa4c6a36c32a12652755a08b4bdcf_0
de5f1ac61d67        registry.aliyuncs.com/google_containers/pause:3.1   "/pause"                 17 hours ago        Up 17 hours                             k8s_POD_etcd-singlek8s_kube-system_07dc93a617eee4df326d01a7a21c55f4_0

calico

CONTAINER ID: 4da55b16e269
IMAGE: calico/node
NAMES: k8s_calico-node_calico-node-dr28q_kube-system_27f2dc0e-6784-4701-8fa5-9f42d5b78f7b_0
COMMAND: start_runit
&
CONTAINER ID: 57d62f467406
IMAGE: pause
NAMES: k8s_POD_calico-node-dr28q_kube-system_27f2dc0e-6784-4701-8fa5-9f42d5b78f7b_0
COMMAND: /pause

calico

CONTAINER ID: 608678f945ae
IMAGE: calico/kube-controllers
NAMES: k8s_calico-kube-controllers_calico-kube-controllers-77d6cbc65f-gvlx8_kube-system_0f0e6d8b-2bfa-49e0-84de-274fa2986e83_0
COMMAND: /usr/bin/kube-controllers
&
CONTAINER ID: b153bb8f3801
IMAGE: pause
NAMES: k8s_POD_calico-kube-controllers-77d6cbc65f-gvlx8_kube-system_0f0e6d8b-2bfa-49e0-84de-274fa2986e83_16
COMMAND: /pause

coredns

CONTAINER ID: a44cb8ee73ef
IMAGE: coredns
NAMES: k8s_coredns_coredns-58cc8c89f4-xsr4k_kube-system_ae7cb050-500d-4099-8a9d-b72f19248b57_0
COMMAND: /coredns -conf /etc/coredns/Corefile
&
CONTAINER ID: 7d8b43b4a6fa
IMAGE: pause
NAMES: k8s_POD_coredns-58cc8c89f4-xsr4k_kube-system_ae7cb050-500d-4099-8a9d-b72f19248b57_18
COMMAND: /pause

coredns

CONTAINER ID: bc035642c4a3
IMAGE: coredns
NAMES: k8s_coredns_coredns-58cc8c89f4-44fxc_kube-system_f466fca9-56e5-468b-9955-75462040b7b9_0
COMMAND: /coredns -conf /etc/coredns/Corefile
&
CONTAINER ID: 052dc4939146
IMAGE: pause
NAMES: k8s_POD_coredns-58cc8c89f4-44fxc_kube-system_f466fca9-56e5-468b-9955-75462040b7b9_17
COMMAND: /pause

test@singlek8s:~$ kubectl get nodes
NAME        STATUS   ROLES    AGE   VERSION
singlek8s   Ready    master   17h   v1.16.3

7. Control Plane node isolation 节点隔离

不做节点隔离配置的失败范例：

test@singlek8s:~$ kubectl create deployment kubernetes-bootcamp --image=10.0.31.201/k8s.gcr.io/google-samples/kubernetes-bootcamp:v1 
deployment.apps/kubernetes-bootcamp created
test@singlek8s:~$ kubectl get pods
NAME                                   READY   STATUS    RESTARTS   AGE
kubernetes-bootcamp-6c599d98b4-wnnrs   0/1     Pending   0          5m55s
test@singlek8s:~$ kubectl get deployments
NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
kubernetes-bootcamp   0/1     1            0           5m56s

查看报错：

test@singlek8s:~$ kubectl describe pod kubernetes-bootcamp-6c599d98b4-wnnrs
Name:           kubernetes-bootcamp-6c599d98b4-wnnrs
Namespace:      default
Priority:       0
Node:           <none>
Labels:         app=kubernetes-bootcamp
                pod-template-hash=6c599d98b4
Annotations:    <none>
Status:         Pending
IP:             
IPs:            <none>
Controlled By:  ReplicaSet/kubernetes-bootcamp-6c599d98b4
Containers:
  kubernetes-bootcamp:
    Image:        10.0.31.201/k8s.gcr.io/google-samples/kubernetes-bootcamp:v1
    Port:         <none>
    Host Port:    <none>
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-p96bk (ro)
Conditions:
  Type           Status
  PodScheduled   False 
Volumes:
  default-token-p96bk:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-p96bk
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason            Age                 From               Message
  ----     ------            ----                ----               -------
  Warning  FailedScheduling  56s (x17 over 22m)  default-scheduler  0/1 nodes are available: 1 node(s) had taints that the pod didn't tolerate.

0/1 nodes are available: 1 node(s) had taints that the pod didn't tolerate.

设置节点隔离

test@singlek8s:~$ kubectl describe node singlek8s
Name:               singlek8s
... output omitted ...
Taints:             node-role.kubernetes.io/master:NoSchedule
... output omitted ...

test@singlek8s:~$ kubectl taint nodes --all node-role.kubernetes.io/master-
node/singlek8s untainted

test@singlek8s:/etc/docker$ kubectl run --image=nginx nginx-app --port=80
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
deployment.apps/nginx-app created

test@singlek8s:/etc/docker$ kubectl get pods
NAME                                   READY   STATUS    RESTARTS   AGE
kubernetes-bootcamp-6c599d98b4-wnnrs   1/1     Running   0          141m
nginx-app-69ff7df578-rlmtp             1/1     Running   0          101s

test@singlek8s:/etc/docker$ kubectl get deployments
NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
kubernetes-bootcamp   1/1     1            1           141m
nginx-app             1/1     1            1           116s

8. 添加节点

单节点cluster无需

ubuntu 18.04.4 kubeadmin 安装单服务器k8s集群 实践