部署CalicoNode（所有节点）报错：
Oct 10 21:04:55 smallsoup docker[76907]: bird: BGP: Unexpected connect from unknown address 192.168.1.104 (port 58153)
Oct 10 21:04:59 smallsoup docker[76907]: bird: BGP: Unexpected connect from unknown address 192.168.1.104 (port 40415)
Oct 10 21:05:00 smallsoup docker[76907]: 2018-10-10 13:05:00.213 [INFO][87] int_dataplane.go 690: Applying dataplane updates
Oct 10 21:05:00 smallsoup docker[76907]: 2018-10-10 13:05:00.213 [INFO][87] ipsets.go 224: Asked to resync with the dataplane on next update. family="inet"
Oct 10 21:05:00 smallsoup docker[76907]: 2018-10-10 13:05:00.213 [INFO][87] ipsets.go 255: Resyncing ipsets with dataplane. family="inet"
Oct 10 21:05:00 smallsoup docker[76907]: 2018-10-10 13:05:00.218 [INFO][87] ipsets.go 297: Finished resync family="inet" numInconsistenciesFound=0 resyncDuration=4.542824ms
Oct 10 21:05:00 smallsoup docker[76907]: 2018-10-10 13:05:00.218 [INFO][87] int_dataplane.go 705: Finished applying updates to dataplane. msecToApply=5.302726
查看节点运行情况，
$ calicoctl node status
Calico process is running.
IPv4 BGP status
+---------------+-------------------+-------+----------+--------------------------------+
| PEER ADDRESS | PEER TYPE | STATE | SINCE | INFO |
+---------------+-------------------+-------+----------+--------------------------------+
| 192.168.1.103 | node-to-node mesh | start | 00:01:41 | Active Socket: Connection |
| | | | | closed |
+---------------+-------------------+-------+----------+--------------------------------+
IPv6 BGP status
No IPv6 peers found.
节点信息如下：
系统类型 IP地址 节点角色 CPU Memory Hostname
Centos7.2 192.168.1.103 master 1 2G server01
Centos7.2 192.168.1.102 worker 1 2G server02
Centos7.2 192.168.1.104 worker 1 2G server03
是因为/lib/systemd/system/kube-calico.service没ExecStart后面没加--IP参数，值为当前节点IP，应该是这样：
[Unit]
Description=calico node
After=docker.service
Requires=docker.service
[Service]
User=root
PermissionsStartOnly=true
ExecStart=/usr/bin/docker run --net=host --privileged --name=calico-node \
-e ETCD_ENDPOINTS=http://192.168.1.103:2379 \
-e CALICO_LIBNETWORK_ENABLED=true \
-e CALICO_NETWORKING_BACKEND=bird \
-e CALICO_DISABLE_FILE_LOGGING=true \
-e CALICO_IPV4POOL_CIDR=172.20.0.0/16 \
-e CALICO_IPV4POOL_IPIP=off \
-e FELIX_DEFAULTENDPOINTTOHOSTACTION=ACCEPT \
-e FELIX_IPV6SUPPORT=false \
-e FELIX_LOGSEVERITYSCREEN=info \
-e FELIX_IPINIPMTU=1440 \
-e FELIX_HEALTHENABLED=true \
-e IP=192.168.1.103 \
-v /var/run/calico:/var/run/calico \
-v /lib/modules:/lib/modules \
-v /run/docker/plugins:/run/docker/plugins \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /var/log/calico:/var/log/calico \
registry.cn-hangzhou.aliyuncs.com/imooc/calico-node:v2.6.2
ExecStop=/usr/bin/docker rm -f calico-node
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
结果：
[root@Server02 kubernetes-starter]# calicoctl node status
Calico process is running.
IPv4 BGP status
+---------------+-------------------+-------+----------+-------------+
| PEER ADDRESS | PEER TYPE | STATE | SINCE | INFO |
+---------------+-------------------+-------+----------+-------------+
| 192.168.1.102 | node-to-node mesh | up | 13:05:42 | Established |
| 192.168.1.103 | node-to-node mesh | up | 13:10:56 | Established |
+---------------+-------------------+-------+----------+-------------+
IPv6 BGP status
No IPv6 peers found.
k8s集群搭建好以后，小试牛刀：
[root@smallsoup net.d]# kubectl run kubernetes-bootcamp --image=jocatalin/kubernetes-bootcamp:v1 --port=8080
deployment "kubernetes-bootcamp" created
[root@smallsoup net.d]#
[root@smallsoup net.d]# kubectl get deployments
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
kubernetes-bootcamp 1 1 1 1 4m
[root@smallsoup net.d]#
[root@smallsoup net.d]#
[root@smallsoup net.d]# kubectl get pods
NAME READY STATUS RESTARTS AGE
kubernetes-bootcamp-6b7849c495-vrcc2 1/1 Running 0 6m
[root@smallsoup net.d]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
kubernetes-bootcamp-6b7849c495-vrcc2 1/1 Running 0 6m 172.20.173.192 192.168.1.103
[root@smallsoup net.d]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
56165c91587d jocatalin/kubernetes-bootcamp "/bin/sh -c 'node se…" 5 minutes ago Up 5 minutes k8s_kubernetes-bootcamp_kubernetes-bootcamp-6b7849c495-vrcc2_default_878e7b6f-cc92-11e8-bd34-000c295db389_0
73ece5042794 registry.cn-hangzhou.aliyuncs.com/imooc/pause-amd64:3.0 "/pause" 7 minutes ago Up 7 minutes k8s_POD_kubernetes-bootcamp-6b7849c495-vrcc2_default_878e7b6f-cc92-11e8-bd34-000c295db389_0
a9b502648a37 registry.cn-hangzhou.aliyuncs.com/imooc/calico-node:v2.6.2 "start_runit" 41 minutes ago Up 40 minutes calico-node
[root@smallsoup net.d]#
[root@smallsoup net.d]# kubectl proxy
Starting to serve on 127.0.0.1:8001
[root@smallsoup ~]# curl http://127.0.0.1:8001/api/v1/proxy/namespaces/default/pods/kubernetes-bootcamp-6b7849c495-vrcc2/
Hello Kubernetes bootcamp! | Running on: kubernetes-bootcamp-6b7849c495-vrcc2 | v=1
[root@smallsoup ~]#
[root@smallsoup ~]# kubectl scale deploy kubernetes-bootcamp --replicas=4
deployment "kubernetes-bootcamp" scaled
[root@smallsoup ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
kubernetes-bootcamp-6b7849c495-mpp5k 1/1 Running 0 4m 172.20.173.193 192.168.1.103
kubernetes-bootcamp-6b7849c495-nbbk5 1/1 Running 0 4m 172.20.0.192 192.168.1.102
kubernetes-bootcamp-6b7849c495-tgscr 1/1 Running 0 4m 172.20.0.128 192.168.1.102
kubernetes-bootcamp-6b7849c495-vrcc2 1/1 Running 0 32m 172.20.173.192 192.168.1.103
[root@smallsoup ~]#
[root@smallsoup ~]# kubectl get deploy
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
kubernetes-bootcamp 4 4 4 4 32m
[root@smallsoup ~]# kubectl scale deploy kubernetes-bootcamp --replicas=2
deployment "kubernetes-bootcamp" scaled
[root@smallsoup ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
kubernetes-bootcamp-6b7849c495-mpp5k 1/1 Running 0 5m 172.20.173.193 192.168.1.103
kubernetes-bootcamp-6b7849c495-nbbk5 1/1 Terminating 0 5m 172.20.0.192 192.168.1.102
kubernetes-bootcamp-6b7849c495-tgscr 1/1 Terminating 0 5m 172.20.0.128 192.168.1.102
kubernetes-bootcamp-6b7849c495-vrcc2 1/1 Running 0 34m 172.20.173.192 192.168.1.103
[root@smallsoup ~]#
[root@smallsoup ~]# kubectl describe deploy
Name: kubernetes-bootcamp
Namespace: default
CreationTimestamp: Wed, 10 Oct 2018 21:43:52 +0800
Labels: run=kubernetes-bootcamp
Annotations: deployment.kubernetes.io/revision=1
Selector: run=kubernetes-bootcamp
Replicas: 2 desired | 2 updated | 2 total | 2 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 1 max unavailable, 1 max surge
Pod Template:
Labels: run=kubernetes-bootcamp
Containers:
kubernetes-bootcamp:
Image: jocatalin/kubernetes-bootcamp:v1
Port: 8080/TCP
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
OldReplicaSets: <none>
NewReplicaSet: kubernetes-bootcamp-6b7849c495 (2/2 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 38m deployment-controller Scaled up replica set kubernetes-bootcamp-6b7849c495 to 1
Normal ScalingReplicaSet 10m deployment-controller Scaled up replica set kubernetes-bootcamp-6b7849c495 to 4
Normal ScalingReplicaSet 4m deployment-controller Scaled down replica set kubernetes-bootcamp-6b7849c495 to 2
[root@smallsoup ~]#
[root@smallsoup ~]# kubectl set image deploy kubernetes-bootcamp kubernetes-bootcamp=jocatalin/kubernetes-bootcamp:v2
deployment "kubernetes-bootcamp" image updated
[root@smallsoup ~]# kubectl rollout status deploy kubernetes-bootcamp
Waiting for rollout to finish: 1 old replicas are pending termination...
Waiting for rollout to finish: 1 old replicas are pending termination...
Waiting for rollout to finish: 1 old replicas are pending termination...
Waiting for rollout to finish: 1 of 2 updated replicas are available...
^C[root@smallsoup ~]#
[root@smallsoup ~]# kubectl rollout status deploy kubernetes-bootcamp
Waiting for rollout to finish: 1 of 2 updated replicas are available...
[root@smallsoup ~]# kubectl describe deploy
Name: kubernetes-bootcamp
Namespace: default
CreationTimestamp: Wed, 10 Oct 2018 21:43:52 +0800
Labels: run=kubernetes-bootcamp
Annotations: deployment.kubernetes.io/revision=2
Selector: run=kubernetes-bootcamp
Replicas: 2 desired | 2 updated | 2 total | 2 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 1 max unavailable, 1 max surge
Pod Template:
Labels: run=kubernetes-bootcamp
Containers:
kubernetes-bootcamp:
Image: jocatalin/kubernetes-bootcamp:v2
Port: 8080/TCP
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
OldReplicaSets: <none>
NewReplicaSet: kubernetes-bootcamp-7689dc585d (2/2 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 41m deployment-controller Scaled up replica set kubernetes-bootcamp-6b7849c495 to 1
Normal ScalingReplicaSet 13m deployment-controller Scaled up replica set kubernetes-bootcamp-6b7849c495 to 4
Normal ScalingReplicaSet 7m deployment-controller Scaled down replica set kubernetes-bootcamp-6b7849c495 to 2
Normal ScalingReplicaSet 2m deployment-controller Scaled up replica set kubernetes-bootcamp-7689dc585d to 1
Normal ScalingReplicaSet 2m deployment-controller Scaled down replica set kubernetes-bootcamp-6b7849c495 to 1
Normal ScalingReplicaSet 2m deployment-controller Scaled up replica set kubernetes-bootcamp-7689dc585d to 2
Normal ScalingReplicaSet 1m deployment-controller Scaled down replica set kubernetes-bootcamp-6b7849c495 to 0
[root@smallsoup ~]#
[root@smallsoup ~]# kubectl rollout status deploy kubernetes-bootcamp
deployment "kubernetes-bootcamp" successfully rolled out
[root@smallsoup ~]#
#如果一不小心版本写错了，可以回滚：
[root@smallsoup ~]# kubectl set image deploy kubernetes-bootcamp kubernetes-bootcamp=jocatalin/kubernetes-bootcamp:v20deployment "kubernetes-bootcamp" image updated
[root@smallsoup ~]#
[root@smallsoup ~]# kubectl rollout status deploy kubernetes-bootcamp
Waiting for rollout to finish: 1 old replicas are pending termination...
^C[root@smallsoup ~]# kubectl rollout undo deploy kubernetes-bootcamp
deployment "kubernetes-bootcamp"
[root@smallsoup ~]#
[root@smallsoup services]# vim nginx-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
[root@smallsoup services]# kubectl create -f nginx-pod.yaml
pod "nginx" created
[root@smallsoup services]# kubectl get pods
NAME READY STATUS RESTARTS AGE
kubernetes-bootcamp-7689dc585d-45w6d 1/1 Running 0 5m
kubernetes-bootcamp-7689dc585d-zxxdf 1/1 Running 0 10m
nginx 0/1 ContainerCreating 0 5s
[root@smallsoup services]# kubectl get deploy
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
kubernetes-bootcamp 2 2 2 2 49m
[root@smallsoup services]# kubectl get pods
NAME READY STATUS RESTARTS AGE
kubernetes-bootcamp-7689dc585d-45w6d 1/1 Running 0 12m
kubernetes-bootcamp-7689dc585d-zxxdf 1/1 Running 0 16m
nginx 1/1 Running 0 6m
[root@smallsoup net.d]# kubectl proxy
Starting to serve on 127.0.0.1:8001
[root@smallsoup services]# curl http://127.0.0.1:8001/api/v1/proxy/namespaces/default/pods/nginx/
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
创建deployment：
[root@smallsoup services]# vim nginx-deployments.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 2
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
[root@smallsoup services]# kubectl create -f nginx-deployments.yaml
deployment "nginx-deployment" created
[root@smallsoup services]# kubectl get deploy
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
kubernetes-bootcamp 2 2 2 2 1h
nginx-deployment 2 2 2 2 3m
[root@smallsoup services]# kubectl get pods -owide
NAME READY STATUS RESTARTS AGE IP NODE
kubernetes-bootcamp-7689dc585d-45w6d 1/1 Running 0 23m 172.20.173.197 192.168.1.103
kubernetes-bootcamp-7689dc585d-zxxdf 1/1 Running 0 27m 172.20.0.129 192.168.1.102
nginx 1/1 Running 0 17m 172.20.0.132 192.168.1.102
nginx-deployment-6c54bd5869-94bgh 1/1 Running 0 3m 172.20.0.133 192.168.1.102
nginx-deployment-6c54bd5869-js2r8 1/1 Running 0 3m 172.20.173.198 192.168.1.103
[root@smallsoup services]# kubectl get pods -o wide -l app=nginx
NAME READY STATUS RESTARTS AGE IP NODE
nginx-deployment-6c54bd5869-94bgh 1/1 Running 0 4m 172.20.0.133 192.168.1.102
nginx-deployment-6c54bd5869-js2r8 1/1 Running 0 4m 172.20.173.198 192.168.1.103
#Services
[root@smallsoup kubernetes-starter]# kubectl expose deploy kubernetes-bootcamp --type="NodePort" --target-port=8080 --port=80
service "kubernetes-bootcamp" exposed
[root@smallsoup kubernetes-starter]# kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.68.0.1 <none> 443/TCP 15h
kubernetes-bootcamp NodePort 10.68.194.69 <none> 80:27543/TCP 16s
[root@smallsoup kubernetes-starter]#
[root@smallsoup kubernetes-starter]# netstat -ntlp | grep 27543
tcp6 0 0 :::27543 :::* LISTEN 90067/kube-proxy
[root@smallsoup kubernetes-starter]# curl 192.168.1.103:27543
Hello Kubernetes bootcamp! | Running on: kubernetes-bootcamp-7689dc585d-45w6d | v=2
[root@smallsoup kubernetes-starter]# curl 192.168.1.102:27543
Hello Kubernetes bootcamp! | Running on: kubernetes-bootcamp-7689dc585d-45w6d | v=2
[root@smallsoup kubernetes-starter]# curl 192.168.1.104:27543
curl: (7) Failed connect to 192.168.1.104:27543; Connection refused
[root@smallsoup kubernetes-starter]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
kubernetes-bootcamp-7689dc585d-45w6d 1/1 Running 0 52m 172.20.173.197 192.168.1.103
kubernetes-bootcamp-7689dc585d-zxxdf 1/1 Running 0 57m 172.20.0.129 192.168.1.102
nginx 1/1 Running 0 47m 172.20.0.132 192.168.1.102
nginx-deployment-6c54bd5869-94bgh 1/1 Running 0 33m 172.20.0.133 192.168.1.102
nginx-deployment-6c54bd5869-js2r8 1/1 Running 0 33m 172.20.173.198 192.168.1.103
[root@smallsoup kubernetes-starter]# docker ps -a|grep bootcamp
230582f46c49 jocatalin/kubernetes-bootcamp "/bin/sh -c 'node se…" About an hour ago Up About an hour k8s_kubernetes-bootcamp_kubernetes-bootcamp-7689dc585d-45w6d_default_a7c9c95c-cc98-11e8-bd34-000c295db389_0
48c8ca188409 registry.cn-hangzhou.aliyuncs.com/imooc/pause-amd64:3.0 "/pause" About an hour ago Up About an hour k8s_POD_kubernetes-bootcamp-7689dc585d-45w6d_default_a7c9c95c-cc98-11e8-bd34-000c295db389_0
[root@smallsoup kubernetes-starter]# docker exec -ti 230582f46c49 bash
root@kubernetes-bootcamp-7689dc585d-45w6d:/# curl 10.68.194.69
Hello Kubernetes bootcamp! | Running on: kubernetes-bootcamp-7689dc585d-45w6d | v=2
root@kubernetes-bootcamp-7689dc585d-45w6d:/#
[root@smallsoup services]# vim nginx-service.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
ports:
- port: 8080
targetPort: 80
nodePort: 20000
selector:
app: nginx
type: NodePort
[root@smallsoup services]# kubectl create -f nginx-service.yaml
service "nginx-service" created
[root@smallsoup services]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.68.0.1 <none> 443/TCP 15h
kubernetes-bootcamp NodePort 10.68.194.69 <none> 80:27543/TCP 10m
nginx-service NodePort 10.68.80.238 <none> 8080:20000/TCP 2m
[root@smallsoup services]# curl 10.68.80.238:8080
[root@smallsoup services]# curl 192.168.1.103:20000
[root@smallsoup services]# curl 192.168.1.102:20000
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
dns服务：
[root@smallsoup kubernetes-starter]# kubectl create -f target/services/kube-dns.yaml
configmap "kube-dns" created
serviceaccount "kube-dns" created
service "kube-dns" created
deployment "kube-dns" created
[root@smallsoup kubernetes-starter]#
[root@smallsoup kubernetes-starter]# kubectl -n kube-system get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 10.68.0.2 <none> 53/UDP,53/TCP 48s
[root@smallsoup kubernetes-starter]#
[root@smallsoup kubernetes-starter]# kubectl -n kube-system get deploy
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
kube-dns 1 1 1 1 1m
[root@smallsoup kubernetes-starter]# kubectl -n kube-system get pods
NAME READY STATUS RESTARTS AGE
kube-dns-74cdd9c8-smtld 3/3 Running 0 1m
[root@smallsoup kubernetes-starter]# kubectl -n kube-system get pods -owide
NAME READY STATUS RESTARTS AGE IP NODE
kube-dns-74cdd9c8-smtld 3/3 Running 0 1m 172.20.173.199 192.168.1.103
验证域名解析：
root@kubernetes-bootcamp-7689dc585d-45w6d:/#
root@kubernetes-bootcamp-7689dc585d-45w6d:/#
root@kubernetes-bootcamp-7689dc585d-45w6d:/# cat /etc/resolv.conf
nameserver 10.68.0.2
search default.svc.cluster.local. svc.cluster.local. cluster.local. DHCP HOST
options ndots:5
root@kubernetes-bootcamp-7689dc585d-45w6d:/#
root@kubernetes-bootcamp-7689dc585d-45w6d:/# curl nginx-service:8080
