问题

microk8s安装过程中,部分镜像需要从google的镜像仓库拉取,但是国内无法访问其镜像仓库, 故需要手动获取镜像再自行安装(从官方提供的google mirror仓库获取)

microk8s不是使用的宿主机器的docker进程, 故不能简单的把自己获取的镜像重新tag来完成安装; 需要导出之后然后使用microk8s提供的镜像管理功能进行导入

解决原理

获取到你需要的镜像名称和版本之后 (参见后面的排查技巧)

docker pull mirrorgooglecontainers/$imageName:$imageVersion
docker tag  mirrorgooglecontainers/$imageName:$imageVersion k8s.gcr.io/$imageName:$imageVersion
docker save k8s.gcr.io/$imageName:$imageVersion > $imageName.tar
microk8s.ctr -n k8s.io image import $imageName.tar

示例步骤

视你开启的插件而言,需要手动安装需要的镜像, 以我为例, 需要如下这些(注意版本可能不一样)

k8s.gcr.io/pause:3.1
k8s.gcr.io/heapster-influxdb-amd64:v1.3.3
k8s.gcr.io/heapster-grafana-amd64:v4.4.3
k8s.gcr.io/heapster-amd64:v1.5.2
k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3

gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.7
gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.7
gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.7
安装脚本如下(可以根据原理做成通用脚本再传参安装):

docker pull mirrorgooglecontainers/pause:3.1
docker pull mirrorgooglecontainers/heapster-influxdb-amd64:v1.3.3
docker pull mirrorgooglecontainers/heapster-grafana-amd64:v4.4.3
docker pull mirrorgooglecontainers/kubernetes-dashboard-amd64:v1.8.3
docker pull mirrorgooglecontainers/heapster-amd64:v1.5.2
docker pull mirrorgooglecontainers/k8s-dns-dnsmasq-nanny-amd64:1.14.7
docker pull mirrorgooglecontainers/k8s-dns-kube-dns-amd64:1.14.7
docker pull mirrorgooglecontainers/k8s-dns-sidecar-amd64:1.14.7

docker tag mirrorgooglecontainers/pause:3.1 k8s.gcr.io/pause:3.1
docker tag mirrorgooglecontainers/heapster-influxdb-amd64:v1.3.3 k8s.gcr.io/heapster-influxdb-amd64:v1.3.3
docker tag mirrorgooglecontainers/heapster-grafana-amd64:v4.4.3 k8s.gcr.io/heapster-grafana-amd64:v4.4.3
docker tag mirrorgooglecontainers/kubernetes-dashboard-amd64:v1.8.3 k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3
docker tag mirrorgooglecontainers/heapster-amd64:v1.5.2 k8s.gcr.io/heapster-amd64:v1.5.2
docker tag mirrorgooglecontainers/k8s-dns-dnsmasq-nanny-amd64:1.14.7 gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.7
docker tag mirrorgooglecontainers/k8s-dns-kube-dns-amd64:1.14.7 gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.7
docker tag mirrorgooglecontainers/k8s-dns-sidecar-amd64:1.14.7 gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.7

docker save k8s.gcr.io/pause > pause.tar
docker save k8s.gcr.io/heapster-influxdb-amd64 > heapster-influxdb-amd64.tar
docker save k8s.gcr.io/heapster-grafana-amd64 > heapster-grafana-amd64.tar
docker save k8s.gcr.io/kubernetes-dashboard-amd64 > kubernetes-dashboard-amd64.tar
docker save k8s.gcr.io/heapster-amd64 > heapster-amd64.tar
docker save gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64 > k8s-dns-dnsmasq-nanny-amd64.tar
docker save gcr.io/google_containers/k8s-dns-kube-dns-amd64 > k8s-dns-kube-dns-amd64.tar
docker save gcr.io/google_containers/k8s-dns-sidecar-amd64 > k8s-dns-sidecar-amd64.tar

microk8s.ctr -n k8s.io image import pause.tar
microk8s.ctr -n k8s.io image import heapster-influxdb-amd64.tar
microk8s.ctr -n k8s.io image import heapster-grafana-amd64.tar
microk8s.ctr -n k8s.io image import kubernetes-dashboard-amd64.tar
microk8s.ctr -n k8s.io image import heapster-amd64.tar
microk8s.ctr -n k8s.io image import k8s-dns-dnsmasq-nanny-amd64.tar
microk8s.ctr -n k8s.io image import k8s-dns-kube-dns-amd64.tar
microk8s.ctr -n k8s.io image import k8s-dns-sidecar-amd64.tar

排查技巧

遇到问题时先通过 microk8s.kubectr get pods 查看是否是RUNNING状态, 如果不是,先按照上述方法解决镜像问题

查看具体的镜像版本时候可以通过 kubectl get pods --namespace=kube-system -o json |grep message直接过滤出错误消息

注意namespace的选择, 特别是你自己定义了namespace之后

配置 microk8s 内置 docker 的 registry.mirrors

编辑 /var/snap/microk8s/current/args/containerd-template.toml 文件
在 endpoint 添加新的国内 registry.mirrors ，如 "https://docker.mirrors.ustc.edu.cn"

...
[plugins.cri.registry]
      [plugins.cri.registry.mirrors]
        [plugins.cri.registry.mirrors."docker.io"]
          endpoint = [
                "https://docker.mirrors.ustc.edu.cn",
                "https://hub-mirror.c.163.com",
                "https://mirror.ccs.tencentyun.com",
                "https://registry-1.docker.io"
          ]

然后，先停止 microk8s，再启动 microk8s

sudo microk8s stop
sudo microk8s start

启动 microk8s dns

这个插件很重要，装不上，就不用玩k8s了。

$ sudo microk8s.enable dns
$ sudo kubectl get pods -n kube-system

NAME                      READY   STATUS              RESTARTS   AGE
coredns-87fcbc976-pckpj   0/1     ContainerCreating   0          60m
$ sudo kubectl describe pods coredns-87fcbc976-pckpj -n kube-system
...
Warning  FailedCreatePodSandBox  58s                    
kubelet, doudong   Failed to create pod sandbox: rpc error: code = Unknown desc = failed to 
get sandbox image "k8s.gcr.io/pause:3.1": failed to pull image "k8s.gcr.io/pause:3.1": failed to resolve image "k8s.gcr.io/pause:3.1": no available registry endpoint: 
failed to do request: Head "https://k8s.gcr.io/v2/pause/manifests/3.1": dial tcp 64.233.189.82:443: i/o timeout

发现失败了，无法拉取镜像，按照上面提供的方法，拉取一下对应的镜像版本解决这个问题。

microk8s 启动 istio

$ sudo microk8s.enable istio
Enabling Istio
Fetching istioctl version v1.3.4.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   635  100   635    0     0    292      0  0:00:02  0:00:02 --:--:--   292
  7 36.3M    7 2667k    0     0  23557      0  0:26:59  0:01:55  0:25:04 25609
$ sudo kubectl get pods -n istio-system
NAME                                      READY   STATUS      RESTARTS   AGE
grafana-78758665bb-5vp4j                  1/1     Running     0          9h
istio-citadel-799bf7ddb9-2v2mm            1/1     Running     1          9h
istio-egressgateway-78d46d89bf-w44bp      1/1     Running     1          9h
istio-galley-cbf67694f-4lfnp              1/1     Running     1          9h
istio-grafana-post-install-1.3.4-8wpbm    0/1     Completed   0          9h
istio-ingressgateway-5d86f5f694-2fhnl     1/1     Running     1          9h
istio-pilot-5867df8f85-hjj69              2/2     Running     2          9h
istio-policy-54d878f564-q6p42             2/2     Running     8          9h
istio-security-post-install-1.3.4-d2bw2   0/1     Completed   0          9h
istio-sidecar-injector-66bc6cf76b-4nzbc   1/1     Running     2          9h
istio-telemetry-647b9c8ccd-mzs4v          2/2     Running     9          9h
istio-tracing-6cb4f885d4-5tv7g            1/1     Running     1          9h
kiali-68d8fc4d54-p8q8n                    1/1     Running     1          9h
prometheus-5c8fb8f746-jk89s               1/1     Running     1          9h

两个Completed 是正常的情况。

containerd-env环境代理

${SNAP_DATA}/args/containerd-env
（通常是 /var/snap/microk8s/current/args/containerd-env）
加入两行

HTTPS_PROXY=<你的代理地址:端口>
HTTP_PROXY=<你的代理地址:端口>

然后重启microk8s

snap代理

sudo vim /lib/systemd/system/snapd.service

#在service 下面添加两行
[Service]
Environment=http_proxy=http://192.168.10.173:8888

Environment=https_proxy=http://192.168.10.173:8888

参考:
microk8s安装过程中遇到的几个问题
 教你一步一步用 ubuntu / microk8s 安装 istio （国内源）

microk8s安装过程中遇到的问题总结