概述
在指定 pod 拉取外网镜像重定向到内网镜像仓库经常出现拉取外网镜像失败的问题，这个问题在离线安装operatorhub尤其明显，主要原因还是在于最终生效的 Machineconfig 配置以及 Image policies 策略。
介绍
离线安装ocp4.3结束后
在离线安装ocp4后会生成两个imagecontentsourcepolicies.operator.openshift.io，
# oc get imagecontentsourcepolicies.operator.openshift.io
NAME AGE
example 8d
image-policy-0 10d
image-policy-1 10d
查看image-policy-0和image-policy-1的内容，可以看到两个外网和内网两个镜像仓库的映射关系：
[root@registry ~]# oc describe imagecontentsourcepolicies.operator.openshift.io image-policy-0
Name: image-policy-0
Namespace:
Labels: <none>
Annotations: <none>
API Version: operator.openshift.io/v1alpha1
Kind: ImageContentSourcePolicy
Metadata:
Creation Timestamp: 2020-04-22T08:35:06Z
Generation: 1
Resource Version: 554
Self Link: /apis/operator.openshift.io/v1alpha1/imagecontentsourcepolicies/image-policy-0
UID: 266570fb-f9ae-4e48-84fd-6fba137a642e
Spec:
Repository Digest Mirrors:
Mirrors:
registry.ocp4.offline.zsp.com:5000/ocp4/openshift4
Source: quay.io/openshift-release-dev/ocp-release
Events: <none>
[root@registry ~]# oc describe imagecontentsourcepolicies.operator.openshift.io image-policy-1
Name: image-policy-1
Namespace:
Labels: <none>
Annotations: <none>
API Version: operator.openshift.io/v1alpha1
Kind: ImageContentSourcePolicy
Metadata:
Creation Timestamp: 2020-04-22T08:35:06Z
Generation: 1
Resource Version: 559
Self Link: /apis/operator.openshift.io/v1alpha1/imagecontentsourcepolicies/image-policy-1
UID: ac03f2de-4944-44da-9ced-1e7ceea34d91
Spec:
Repository Digest Mirrors:
Mirrors:
registry.ocp4.offline.zsp.com:5000/ocp4/openshift4
Source: quay.io/openshift-release-dev/ocp-v4.0-art-dev
Events: <none>
这两个imagecontentsourcepolicies是由我们离线安装时在install-config.yaml里面定义的以下字段生成：
imageContentSources:
- mirrors:
- registry.ocp4.poc.com:5000/ocp4/openshift4
source: quay.io/openshift-release-dev/ocp-release
- mirrors:
- registry.ocp4.poc.com:5000/ocp4/openshift4
source: quay.io/openshift-release-dev/ocp-v4.0-art-dev
因为这些参数最终是要给每个节点来使用的，作为配置存在节点上，ocp4节点的管理由machine operator，所以我们需要查看 machineconfig :
# oc get machineconfig
NAME GENERATEDBYCONTROLLER IGNITIONVERSION AGE
00-master 5803275f05729eef2a5affc1ad437235c6981f68 2.2.0 10d
00-master-test 5803275f05729eef2a5affc1ad437235c6981f68 2.2.0 7d3h
00-worker 5803275f05729eef2a5affc1ad437235c6981f68 2.2.0 10d
01-master-container-runtime 5803275f05729eef2a5affc1ad437235c6981f68 2.2.0 10d
01-master-kubelet 5803275f05729eef2a5affc1ad437235c6981f68 2.2.0 10d
01-worker-container-runtime 5803275f05729eef2a5affc1ad437235c6981f68 2.2.0 10d
01-worker-kubelet 5803275f05729eef2a5affc1ad437235c6981f68 2.2.0 10d
99-master-12cf926e-f293-45ce-9bb4-21aef7eec4cf-registries 5803275f05729eef2a5affc1ad437235c6981f68 2.2.0 10d
99-master-ssh 2.2.0 10d
99-worker-73f736fe-bf60-4834-befe-b7b44a343572-registries 5803275f05729eef2a5affc1ad437235c6981f68 2.2.0 10d
99-worker-ssh 2.2.0 10d
rendered-master-30a5781e8ac597ecabb1205ad8d7d82c 5803275f05729eef2a5affc1ad437235c6981f68 2.2.0 7d3h
rendered-master-51788310d621218c658311e34fd3aac5 5803275f05729eef2a5affc1ad437235c6981f68 2.2.0 8d
rendered-master-88b3e65881e31a9a0915e6bf27dea73d ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 10d
rendered-master-9a83f11058611f346f2b5539a98d59c7 f6d1fe753cbcecb3aa1c2d3d3edd4a5d04ffca54 2.2.0 9d
rendered-master-cf4df21382f9c71e79aa98b7a3cf2f8f f6d1fe753cbcecb3aa1c2d3d3edd4a5d04ffca54 2.2.0 8d
rendered-worker-a778c09867f0b7889b4a6f034e113216 ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 10d
rendered-worker-ae3b3ff2d7103312b48841d9471539d2 f6d1fe753cbcecb3aa1c2d3d3edd4a5d04ffca54 2.2.0 9d
rendered-worker-cb525d7b2cf5fc2518d2656cc4a4359d 5803275f05729eef2a5affc1ad437235c6981f68 2.2.0 8d
rendered-worker-d0aec4b2f11f239256493436faa37927 f6d1fe753cbcecb3aa1c2d3d3edd4a5d04ffca54 2.2.0 8d
对应两个imagecontentsourcepolicies的是99-master-12cf926e-f293-45ce-9bb4-21aef7eec4cf-registries和99-worker-73f736fe-bf60-4834-befe-b7b44a343572-registries，查看一下内容：
# oc get machineconfig 99-master-12cf926e-f293-45ce-9bb4-21aef7eec4cf-registries -oyaml
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
annotations:
machineconfiguration.openshift.io/generated-by-controller-version: 5803275f05729eef2a5affc1ad437235c6981f68
creationTimestamp: "2020-04-22T08:44:18Z"
generation: 2
labels:
machineconfiguration.openshift.io/role: master
name: 99-master-12cf926e-f293-45ce-9bb4-21aef7eec4cf-registries
ownerReferences:
- apiVersion: config.openshift.io/v1
kind: Image
name: cluster
uid: ef354909-71a2-4dd6-8a88-97c167d52524
spec:
config:
ignition:
config: {}
security:
tls: {}
timeouts: {}
version: 2.2.0
networkd: {}
passwd: {}
storage:
files:
- contents:
source: data:text/plain,unqualified-search-registries%20%3D%20%5B%22registry.access.redhat.com%22%2C%20%22docker.io%22%5D%0A%0A%5B%5Bregistry%5D%5D%0A%20%20prefix%20%3D%20%22%22%0A%20%20location%20%3D%20%22quay.io%2Fopenshift-release-dev%2Focp-release%22%0A%20%20mirror-by-digest-only%20%3D%20true%0A%0A%20%20%5B%5Bregistry.mirror%5D%5D%0A%20%20%20%20location%20%3D%20%22registry.ocp4.offline.zsp.com%3A5000%2Focp4%2Fopenshift4%22%0A%0A%5B%5Bregistry%5D%5D%0A%20%20prefix%20%3D%20%22%22%0A%20%20location%20%3D%20%22quay.io%2Fopenshift-release-dev%2Focp-v4.0-art-dev%22%0A%20%20mirror-by-digest-only%20%3D%20true%0A%0A%20%20%5B%5Bregistry.mirror%5D%5D%0A%20%20%20%20location%20%3D%20%22registry.ocp4.offline.zsp.com%3A5000%2Focp4%2Fopenshift4%22%0A
verification: {}
filesystem: root
mode: 420
path: /etc/containers/registries.conf
systemd: {}
fips: false
kernelArguments: null
kernelType: ""
osImageURL: ""
将source字段的内容进行url解码，可以看到以下内容，这文件的在节点上的路径为/etc/containers/registries.conf：
unqualified-search-registries = ["registry.access.redhat.com", "docker.io"]
[[registry]]
prefix = ""
location = "quay.io/openshift-release-dev/ocp-release"
mirror-by-digest-only = true
[[registry.mirror]]
location = "registry.ocp4.offline.zsp.com:5000/ocp4/openshift4"
[[registry]]
prefix = ""
location = "quay.io/openshift-release-dev/ocp-v4.0-art-dev"
mirror-by-digest-only = true
[[registry.mirror]]
location = "registry.ocp4.offline.zsp.com:5000/ocp4/openshift4"
在有多个MachineConfig中定义相同配置文件，则最后一个生效，如果要看最后综合的MachineConfig，则需要看对应的rendered*的MachineConfig，最终生效的MachineConfig时间距离最近，如果不确定可以把所有的rendered删了，过一会儿会自动生成，
手动创建MachineConfig
以上演示的集群没有多个相同/etc/containers/registries.conf的MachineConfig，所以我将切换一个集群，并且我会创建一个ImageContentSourcePolicy，因为默认的被我删了，但是最终也是在99-XXXX-XXXX-registries里面配置。
apiVersion: operator.openshift.io/v1alpha1
kind: ImageContentSourcePolicy
metadata:
name: example
spec:
repositoryDigestMirrors:
- mirrors:
- registry.ocp4.poc.com:5000/ocp4/ose-elasticsearch-operator
source: registry.redhat.io/openshift4/ose-elasticsearch-operator
创建
oc create -f imagecontentsourcepolicy
imagecontentsourcepolicy.operator.openshift.io/example created
查看创建的imagecontentsourcepolicy.内容
oc get imagecontentsourcepolicy.operator.openshift.io/example -oyaml
apiVersion: operator.openshift.io/v1alpha1
kind: ImageContentSourcePolicy
metadata:
creationTimestamp: "2020-05-02T13:53:12Z"
generation: 1
name: example
resourceVersion: "2161933"
selfLink: /apis/operator.openshift.io/v1alpha1/imagecontentsourcepolicies/example
uid: 46876e0d-c892-4a75-a640-a6806072f985
spec:
repositoryDigestMirrors:
- mirrors:
- registry.ocp4.poc.com:5000/ocp4/ose-elasticsearch-operator
source: registry.redhat.io/openshift4/ose-elasticsearch-operator
- 查看machineconfig
# oc get machineconfig
NAME GENERATEDBYCONTROLLER IGNITIONVERSION CREATED
00-master ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 3d10h
00-worker ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 3d10h
01-master-container-runtime ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 3d10h
01-master-kubelet ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 3d10h
01-worker-container-runtime ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 3d10h
01-worker-kubelet ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 3d10h
99-master-5907d758-226d-4a6f-89da-20a1100f288c-registries ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 148m
99-master-container-registry-conf 2.2.0 44h
99-master-ssh 2.2.0 3d10h
99-worker-a59b993d-7ded-4e15-89c4-72a4980df15a-registries ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 148m
99-worker-ssh 2.2.0 3d10h
rendered-master-3bb8239d0ca28cfdc00382d5c5a0a79f ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 147m
rendered-master-7402c8f74c8a28c7f995c038d956aef1 ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 149m
rendered-worker-1bf4fc757fba4f5450d39f5b4ae73a8b ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 147m
rendered-worker-cb7b84e687a34004966348a5c912f340 ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 77m
rendered-worker-eec15a28849de7ad99ea8e3f95b3d9a4 ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 149m
rendered-worker-f080151f04d85d01a3dc5a44ca75fe98 ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 128m
查看对应的machineconfig内容
# oc get machineconfig 99-worker-a59b993d-7ded-4e15-89c4-72a4980df15a-registries -oyaml
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
annotations:
machineconfiguration.openshift.io/generated-by-controller-version: ab4d62a3bf3774b77b6f9b04a2028faec1568aca
creationTimestamp: "2020-05-02T11:26:02Z"
generation: 3
labels:
machineconfiguration.openshift.io/role: worker
name: 99-worker-a59b993d-7ded-4e15-89c4-72a4980df15a-registries
ownerReferences:
- apiVersion: config.openshift.io/v1
kind: Image
name: cluster
uid: 799c6b45-5743-452a-99f8-c93db41fb959
resourceVersion: "2161935"
selfLink: /apis/machineconfiguration.openshift.io/v1/machineconfigs/99-worker-a59b993d-7ded-4e15-89c4-72a4980df15a-registries
uid: 12f5b91a-84ec-4b24-8b06-7e77e741c8b7
spec:
config:
ignition:
config: {}
security:
tls: {}
timeouts: {}
version: 2.2.0
networkd: {}
passwd: {}
storage:
files:
- contents:
source: data:text/plain,unqualified-search-registries%20%3D%20%5B%22registry.access.redhat.com%22%2C%20%22docker.io%22%5D%0A%0A%5B%5Bregistry%5D%5D%0A%20%20prefix%20%3D%20%22%22%0A%20%20location%20%3D%20%22registry.redhat.io%2Fopenshift4%2Fose-elasticsearch-operator%22%0A%20%20mirror-by-digest-only%20%3D%20true%0A%0A%20%20%5B%5Bregistry.mirror%5D%5D%0A%20%20%20%20location%20%3D%20%22registry.ocp4.poc.com%3A5000%2Focp4%2Fose-elasticsearch-operator%22%0A
verification: {}
filesystem: root
mode: 420
path: /etc/containers/registries.conf
systemd: {}
fips: false
kernelArguments: null
osImageURL: ""
解码source的内容:
unqualified-search-registries = ["registry.access.redhat.com", "docker.io"]
[[registry]]
prefix = ""
location = "registry.redhat.io/openshift4/ose-elasticsearch-operator"
mirror-by-digest-only = true
[[registry.mirror]]
location = "registry.ocp4.poc.com:5000/ocp4/ose-elasticsearch-operator"
- 我手动创建了一个叫
99-master-container-registry-conf的machineconfig，查看内容：
# oc get machineconfig 99-master-container-registry-conf -oyaml
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
generation: 2
labels:
machineconfiguration.openshift.io/role: master
name: 99-master-container-registry-conf
spec:
config:
ignition:
version: 2.2.0
storage:
files:
- contents:
source: data:text/plain;charset=utf-8;base64,W1tyZWdpc3RyeV1dCiAgbG9jYXRpb24gPSAicXVheS5pby9vcGVuc2hpZnQtcmVsZWFzZS1kZXYvb2NwLXJlbGVhc2UiCiAgaW5zZWN1cmUgPSBmYWxzZQogIGJsb2NrZWQgPSBmYWxzZQogIG1pcnJvci1ieS1kaWdlc3Qtb25seSA9IHRydWUKICBwcmVmaXggPSAiIgoKICBbW3JlZ2lzdHJ5Lm1pcnJvcl1dCiAgICBsb2NhdGlvbiA9ICJyZWdpc3RyeS5vY3A0LnBvYy5jb206NTAwMC9vY3A0L29wZW5zaGlmdDQiCiAgICBpbnNlY3VyZSA9IGZhbHNlCgpbW3JlZ2lzdHJ5XV0KICBsb2NhdGlvbiA9ICJxdWF5LmlvL29wZW5zaGlmdC1yZWxlYXNlLWRldi9vY3AtdjQuMC1hcnQtZGV2IgogIGluc2VjdXJlID0gZmFsc2UKICBibG9ja2VkID0gZmFsc2UKICBtaXJyb3ItYnktZGlnZXN0LW9ubHkgPSB0cnVlCiAgcHJlZml4ID0gIiIKCiAgW1tyZWdpc3RyeS5taXJyb3JdXQogICAgbG9jYXRpb24gPSAicmVnaXN0cnkub2NwNC5wb2MuY29tOjUwMDAvb2NwNC9vcGVuc2hpZnQ0IgogICAgaW5zZWN1cmUgPSBmYWxzZQ==
verification: {}
filesystem: root
mode: 420
path: /etc/containers/registries.conf
因为这个文件我是用base64加密的，所以要解码source内容：
[[registry]]
location = "quay.io/openshift-release-dev/ocp-release"
insecure = false
blocked = false
mirror-by-digest-only = true
prefix = ""
[[registry.mirror]]
location = "registry.ocp4.poc.com:5000/ocp4/openshift4"
insecure = false
[[registry]]
location = "quay.io/openshift-release-dev/ocp-v4.0-art-dev"
insecure = false
blocked = false
mirror-by-digest-only = true
prefix = ""
[[registry.mirror]]
location = "registry.ocp4.poc.com:5000/ocp4/openshift4"
insecure = false
其中这个时候有三个MachineConfig定义了/etc/containers/registries.conf文件内容
01-master-container-runtime ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 3d10h
99-master-5907d758-226d-4a6f-89da-20a1100f288c-registries ab4d62a3bf3774b77b6f9b04a2028faec1568aca 2.2.0 148m
99-master-container-registry-conf 2.2.0 44h
01-master-container-runtime 内容关于/etc/containers/registries.conf就是默认的，很少
# oc get machineconfig 01-master-container-runtime -oyaml
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
annotations:
name: 01-master-container-runtime
spec:
config:
storage:
files:
- contents:
source: data:,unqualified-search-registries%20%3D%20%5B'registry.access.redhat.com'%2C%20'docker.io'%5D%0A
verification: {}
filesystem: root
mode: 420
path: /etc/containers/registries.conf
查看
# oc get machineconfig rendered-master-7402c8f74c8a28c7f995c038d956aef1 -oyaml
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
annotations:
machineconfiguration.openshift.io/generated-by-controller-version: ab4d62a3bf3774b77b6f9b04a2028faec1568aca
creationTimestamp: "2020-05-02T11:24:07Z"
generation: 1
name: rendered-master-7402c8f74c8a28c7f995c038d956aef1
···············································································································
- contents:
source: data:,unqualified-search-registries%20%3D%20%5B'registry.access.redhat.com'%2C%20'docker.io'%5D%0A
verification: {}
filesystem: root
mode: 420
path: /etc/containers/registries.conf
···············································································································
- contents:
source: data:text/plain,unqualified-search-registries%20%3D%20%5B%22registry.access.redhat.com%22%2C%20%22docker.io%22%5D%0A%0A%5B%5Bregistry%5D%5D%0A%20%20prefix%20%3D%20%22%22%0A%20%20location%20%3D%20%22registry.redhat.io%2Fopenshift4%2Fose-elasticsearch-operator%22%0A%20%20mirror-by-digest-only%20%3D%20true%0A%0A%20%20%5B%5Bregistry.mirror%5D%5D%0A%20%20%20%20location%20%3D%20%22registry.ocp4.poc.com%3A5000%2Focp4%2Fose-elasticsearch-operator%22%0A
verification: {}
filesystem: root
mode: 420
path: /etc/containers/registries.conf
···············································································································
- contents:
source: data:text/plain;charset=utf-8;base64,W1tyZWdpc3RyeV1dCiAgbG9jYXRpb24gPSAicXVheS5pby9vcGVuc2hpZnQtcmVsZWFzZS1kZXYvb2NwLXJlbGVhc2UiCiAgaW5zZWN1cmUgPSBmYWxzZQogIGJsb2NrZWQgPSBmYWxzZQogIG1pcnJvci1ieS1kaWdlc3Qtb25seSA9IHRydWUKICBwcmVmaXggPSAiIgoKICBbW3JlZ2lzdHJ5Lm1pcnJvcl1dCiAgICBsb2NhdGlvbiA9ICJyZWdpc3RyeS5vY3A0LnBvYy5jb206NTAwMC9vY3A0L29wZW5zaGlmdDQiCiAgICBpbnNlY3VyZSA9IGZhbHNlCgpbW3JlZ2lzdHJ5XV0KICBsb2NhdGlvbiA9ICJxdWF5LmlvL29wZW5zaGlmdC1yZWxlYXNlLWRldi9vY3AtdjQuMC1hcnQtZGV2IgogIGluc2VjdXJlID0gZmFsc2UKICBibG9ja2VkID0gZmFsc2UKICBtaXJyb3ItYnktZGlnZXN0LW9ubHkgPSB0cnVlCiAgcHJlZml4ID0gIiIKCiAgW1tyZWdpc3RyeS5taXJyb3JdXQogICAgbG9jYXRpb24gPSAicmVnaXN0cnkub2NwNC5wb2MuY29tOjUwMDAvb2NwNC9vcGVuc2hpZnQ0IgogICAgaW5zZWN1cmUgPSBmYWxzZQ==
verification: {}
filesystem: root
mode: 420
path: /etc/containers/registries.conf
···············································································································
按照顺序最后一个生效，进入节点的/etc/containers/registries.conf就可以确认了
[core@master00 ~]$ cat /etc/containers/registries.conf
[[registry]]
location = "quay.io/openshift-release-dev/ocp-release"
insecure = false
blocked = false
mirror-by-digest-only = true
prefix = ""
[[registry.mirror]]
location = "registry.ocp4.poc.com:5000/ocp4/openshift4"
insecure = false
[[registry]]
location = "quay.io/openshift-release-dev/ocp-v4.0-art-dev"
insecure = false
blocked = false
mirror-by-digest-only = true
prefix = ""
[[registry.mirror]]
location = "registry.ocp4.poc.com:5000/ocp4/openshift4"
insecure = false
手动验证，进入节点，切换到root或者使用sudo执行podman login登录本地仓库，然后拉取外网镜像，就能拉取了，可以在podman后面加上--log-level debug查看过程，如果拉取失败，则有以下几种原因，
1、内部镜像仓库https证书错误。
2、内部镜像仓库登录失败，密码错误等。
3、内部镜像仓库没有对应的镜像。
以上三种情况发生时则会去外网拉取镜像，不能联网的话就会报错。
默认节点的镜像仓库https证书路径：
/etc/pki/ca-trust/source/anchors/openshift-config-user-ca-bundle.crt
默认节点的镜像仓库登录文件路径：
cat /var/lib/kubelet/config.json
{"auths":{"registry.ocp4.poc.com:5000":{"auth":"YWRtaW46YWRtaW4=","email":"you@example.com"}}}
使用podman拉取镜像带上密码文件：
sudo podman pull --authfile /var/lib/kubelet/config.json quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6f2788587579df093bf990268390f7f469cce47e45a7f36b04b74aa00d1dd9e0
关于/etc/containers/registries.conf内的配置
mirror-by-digest-only这个配置很重要，意思是说如果这个值为true，那么你拉取镜像的时候的时候必须要用sha256，如果用tag就去外网拉取镜像了，如果用sha256就必须保证本地的镜像仓库必须有对应sha256的镜像，如果你是用docker save或者podman save从其他机器拿来的，再push到本地镜像仓库，那么sha256基本就变了，所以这个要注意，尤其ocp4离线安装的时候，都是用sha256来拉取镜像。
mirror-by-digest-only这个值我查了openshift文档没法在ImageContentSourcePolicy里面定义，所以只能通过手动创建MachineConfig导入了。
- 官方解释：
mirror-by-digest-only : true or false. If true, mirrors will only be used during pulling if the image reference includes a digest. Referencing an image by digest ensures that the same is always used (whereas referencing an image by a tag may cause different registries to return different images if the tag mapping is out of sync).
https://github.com/containers/image/blob/master/docs/containers-registries.conf.5.md
关于配置本地镜像仓库证书和登录的问题参考以下链接
https://docs.openshift.com/container-platform/4.3/openshift_images/image-configuration.html
https://docs.openshift.com/container-platform/4.3/openshift_images/managing_images/using-image-pull-secrets.html
参考链接
https://docs.openshift.com/container-platform/4.3/openshift_images/image-configuration.html
https://access.redhat.com/solutions/4838051
