前提
- centos 7.2/7.3
- ansible 2.2.1及以上
- ssh免密登录（建立操作服务器到目标服务器）
- /etc/resolv.conf 中 nameserver 不超过3个
- 不要使用系统默认的docker，增加如下repo
[xzx@cloud4ourself-kgo1 ~]$ cat /etc/yum.repos.d/docker.repo
[dockerrepo]
name=Docker Repository
baseurl=https://yum.dockerproject.org/repo/main/centos/7
enabled=1
gpgcheck=1
gpgkey=https://yum.dockerproject.org/gpg
yum install docker-engine -y
参考https://yq.aliyun.com/articles/29941 增加阿里云加速器，否则安装过程时间很长
- 升级jinja2
yum install -y python-netaddr
pip install --upgrade Jinja2
- 使用可更新的yum源（如：https://mirrors.aliyun.com/centos/7/）
安装
- 下载
该库已经将gcr.io 和 部分 quay.io 的源改为docker.io，这样可以利用阿里云加速，解决无法访问问题
git clone https://github.com/4admin2root/kargo
修改ansible资产文件
inventory/inventory.example：将主机kgo[1-4]改为实际情况执行安装
ansible-playbook -i inventory/inventory.example cluster.yml -vvvv
- 结果
result.png
测试
kubectl create -f https://raw.githubusercontent.com/4admin2root/daocloud/master/my-calc.yaml
kubectl create -f https://raw.githubusercontent.com/4admin2root/daocloud/master/my-frontend.yaml
kubectl run busybox --image=busybox -- sleep 3600
kubectl exec -it `kubectl get pod |grep busybox|awk '{print $1}'` -- sh
/ # nslookup my-frontend-service
Server: 10.233.0.2
Address 1: 10.233.0.2
Name: my-frontend-service
Address 1: 10.233.40.222
/ # wget --post-data="x=1&y=2&submit=addition" -O foo http://my-frontend-service:5000/add
Connecting to my-frontend-service:5000 (10.233.40.222:5000)
foo 100% |*********************************************************************************************************************| 232 0:00:00 ETA
/ # cat foo
<html>
<body>
<div>
<form method="post" action="/add">
<input type="text" name="x" size="2"/>
<input type="text" name="y" size="2"/>
<input type="submit" value="addition"/>
</form>
<p>Answer : 1 + 2 = 3
</p>
</div>
</body>
/ #
问题
- 报错
TASK [vault : test] ************************************************************
task path: /usr/local/lvzj/github/kargo/roles/vault/tasks/shared/check_vault.yml:27
Monday 20 March 2017 17:47:59 +0800 (0:00:00.034) 0:00:37.293 **********
TASK [vault : debug] ***********************************************************
task path: /usr/local/lvzj/github/kargo/roles/vault/tasks/shared/check_vault.yml:31
Monday 20 March 2017 17:47:59 +0800 (0:00:00.034) 0:00:37.328 **********
TASK [vault : check_vault | Set fact about the Vault cluster's initialization state] ***
task path: /usr/local/lvzj/github/kargo/roles/vault/tasks/shared/check_vault.yml:34
Monday 20 March 2017 17:47:59 +0800 (0:00:00.037) 0:00:37.365 **********
fatal: [kg2]: FAILED! => {
"failed": true,
"msg": "'dict object' has no attribute 'vault'"
}
fatal: [kg1]: FAILED! => {
"failed": true,
"msg": "'dict object' has no attribute 'vault'"
}
解决办法：升级ansible的版本
- 如使用ceph 会存在rbd报错
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
16h 1m 498 {kubelet cloud4ourself-kg3} Warning FailedMount MountVolume.SetUp failed for volume "kubernetes.io/rbd/a1211922-0ee9-11e7-8784-fa163e2dc2d7-pvc-9b0b20fa-0ee1-11e7-8784-fa163e2dc2d7" (spec.Name: "pvc-9b0b20fa-0ee1-11e7-8784-fa163e2dc2d7") pod "a1211922-0ee9-11e7-8784-fa163e2dc2d7" (UID: "a1211922-0ee9-11e7-8784-fa163e2dc2d7") with: rbd: failed to modprobe rbd error:exit status 1
16h 26s 447 {kubelet cloud4ourself-kg3} Warning FailedMount Unable to mount volumes for pod "mysql-1_default(a1211922-0ee9-11e7-8784-fa163e2dc2d7)": timeout expired waiting for volumes to attach/mount for pod "default"/"mysql-1". list of unattached/unmounted volumes=[data]
16h 26s 447 {kubelet cloud4ourself-kg3} Warning FailedSync Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "default"/"mysql-1". list of unattached/unmounted volumes=[data]
需要在所有节点配置安装ceph-common，并配置/etc/ceph/ keyring 和 ceph.conf
还需改变kubelet 容器的启动参数，sudo vim /usr/local/bin/kubelet
增加
-v /sbin/modprobe:/sbin/modprobe:ro \
-v /lib/modules:/lib/modules:ro \
-v /etc/ceph:/etc/ceph:ro \
安装失败手动清理
rm -rf /etc/kubernetes/
rm -rf /var/lib/kubelet
rm -rf /var/lib/etcd
rm -rf /usr/local/bin/kubectl
rm -rf /etc/systemd/system/calico-node.service
rm -rf /etc/systemd/system/kubelet.service
systemctl stop etcd.service
systemctl disable etcd.service
systemctl stop calico-node.service
systemctl disable calico-node.service
service docker restart
参考
https://github.com/kubernetes-incubator/kargo#requirements
https://www.xf80.com/2017/03/13/kargo-kubernetes-1.5.4/
https://github.com/kubernetes/kubernetes/issues/23924
