一、Red Hat OpenShift Service Mesh 安装概览
Red Hat OpenShift Service Mesh 安装过程中会创建两个不同的 project (namespace)
istio-operator project (1 pod)
istio-system project (17 pods)
你首先需要创建一个Kubernetes operator。这个operator定义并监控着一个custom resource。operator用于部署、升级、删除Service Mesh的组件。
取决于你如何定义这个custom resource文件，当安装Service Mesh时，你可以选择安装以下一个或多个以下组件
Istio - 基于开源 Istio 项目, 让你能连接、安全保护、控制、观察组成你的应用的各个微服务
Jaeger - 基于开源 Jaeger 项目, 让你能在一个复杂的分布式系统中，通过调用链跟踪来监控和排查事务问题
Kiali - 基于开源 Kiali 项目, Kiali 为你的 Service Mesh 提供了可观察的特性，使用 Kiali 能让你可视化地配置、监控流量、可视化地分析调用链。
Launcher - 基于开源 fabric8 社区, fabric8是一个开源集成开发平台，为基于Kubernetes和Jenkins的微服务提供持续发布。
在 operator 的安装过程中，会创建一个 Ansible job， Ansible job 会启动一个 Ansible playbook，Ansible playbook 会自动执行以下步骤，并配置好各个组件。
1.1 创建 istio-system namespace
1.2 创建 openshift-ansible-istio-installer-job，它会安装以下组件:
Istio components: istio-citadel、istio-egressgateway、istio-galley、istio-ingressgateway、istio-pilot、istio-policy、istio-sidecar-injector、istio-statsd-prom-bridge、istio-telemetry
Elasticsearch
Grafana
Jaeger components: jaeger-agent、jaeger-collector、jaeger-query
Kiali components (如果在 custom resource 文件中配置了 Kiali 的话): Kiali
Prometheus
1.3 执行 launcher 的配置任务 (如果在 custom resource 文件中配置了 launcher 的话):
1.3.1 创建一个 devex 项目并安装 Fabric8 launcher 到 devex project中.
1.3.2 将集群管理角色添加到在自定义资源文件的启动程序参数中指定的openshift容器平台用户。
二、安装前准备
2.1 如果离线安装需要下载的镜像包括
docker pull registry.access.redhat.com/openshift-istio-tech-preview/istio-operator:0.6.0
docker pull registry.access.redhat.com/openshift-istio-tech-preview/openshift-ansible:0.6.0
docker pull registry.access.redhat.com/openshift-istio-tech-preview/citadel:0.6.0
docker pull registry.access.redhat.com/openshift-istio-tech-preview/proxyv2:0.6.0
docker pull registry.access.redhat.com/openshift-istio-tech-preview/pilot:0.6.0
docker pull registry.access.redhat.com/openshift-istio-tech-preview/mixer:0.6.0
docker pull registry.access.redhat.com/openshift-istio-tech-preview/galley:0.6.0
docker pull registry.access.redhat.com/openshift-istio-tech-preview/sidecar-injector:0.6.0
docker pull registry.access.redhat.com/openshift-istio-tech-preview/proxy-init:0.6.0
docker pull registry.access.redhat.com/openshift-istio-tech-preview/kiali:0.11.0
docker pull registry.access.redhat.com/distributed-tracing-tech-preview/jaeger-elasticsearch:5.6.10
docker pull registry.access.redhat.com/distributed-tracing-tech-preview/jaeger-agent:1.8.1
docker pull registry.access.redhat.com/distributed-tracing-tech-preview/jaeger-collector:1.8.1
docker pull registry.access.redhat.com/distributed-tracing-tech-preview/jaeger-query:1.8.1
docker pull grafana/grafana:5.4.2
docker pull docker.io/prom/prometheus:v2.3.1
2.2 更新 Openshift 各节点配置
2.2.1 在每台机器新建立一个/etc/sysctl.d/99-elasticsearch.conf文件，添加
vm.max_map_count = 262144
2.2.2 在每个 Node 上执行命令
$ sysctl vm.max_map_count=262144
三、安装 Service Mesh
3.1 创建 CUSTOM RESOURCE 文件
包含所有组件的 istio-installation.yaml：
apiVersion: "istio.openshift.com/v1alpha1"
kind: "Installation"
metadata:
name: "istio-installation"
spec:
deployment_type: openshift
istio:
authentication: true
community: false
prefix: openshift-istio-tech-preview/ #镜像前缀
version: 0.6.0 #镜像tag
jaeger:
prefix: distributed-tracing-tech-preview/
version: 1.8.1
elasticsearch_memory: 1Gi
kiali:
username: username #kiali管理界面的登录名
password: password #kiali管理界面的登录密码
prefix: openshift-istio-tech-preview/
version: 0.11.0
launcher: #内网中干掉
openshift:
user: user
password: password
github:
username: username
token: token
catalog:
filter: booster.mission.metadata.istio
branch: v71
repo: https://github.com/fabric8-launcher/launcher-booster-catalog.git
注：如果在内网环境安装，由于无法访问外网的 Github，可以不用安装 launcher 组件，在 custom resource 文件中去掉 launcher 相关的所有配置即可。
最小化的 istio-installation.yaml
apiVersion: "istio.openshift.com/v1alpha1"
kind: "Installation"
metadata:
name: "istio-installation"
3.2 安装 operator
Service Mesh 安装过程中引入了 kubernetes operator 来管理 istio-system namespace 内 control plane 的安装。此 operator 定义了监视 control plane 的部署、更新和删除相关的自定义资源。
istio_product_operator_template.yaml :
apiVersion: v1
kind: Template
metadata:
name: istio-operator-job
parameters:
- displayName: Master Public URL
description: The public URL for master
name: OPENSHIFT_ISTIO_MASTER_PUBLIC_URL
value: https://127.0.0.1:8443
- displayName: OpenShift Release
description: The version of the OpenShift release.
name: OPENSHIFT_RELEASE
value: v3.11.0
required: true
- displayName: Istio Operator Namespace
description: The namespace for the Istio operator
name: OPENSHIFT_ISTIO_OPERATOR_NAMESPACE
value: istio-operator
required: true
- displayName: Default Prefix
description: The default image prefix for istio deployments
name: OPENSHIFT_ISTIO_PREFIX
value: openshift-istio-tech-preview/
- displayName: Default Version
description: The default image version for istio deployments
name: OPENSHIFT_ISTIO_VERSION
value: 0.6.0
- displayName: Default Deployment Type
description: The default deployment type for istio deployments
name: OPENSHIFT_DEPLOYMENT_TYPE
value: openshift
objects:
- kind: CustomResourceDefinition
apiVersion: apiextensions.k8s.io/v1beta1
metadata:
name: installations.istio.openshift.com
spec:
group: istio.openshift.com
names:
kind: Installation
plural: installations
singular: installation
scope: Namespaced
version: v1alpha1
- kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: istio-operator
rules:
- apiGroups:
- istio.openshift.com
resources:
- "*"
verbs:
- "*"
- apiGroups:
- ""
resources:
- pods
- services
- endpoints
- persistentvolumeclaims
- events
- configmaps
- secrets
- securitycontextconstraints
verbs:
- "*"
- apiGroups:
- apps
resources:
- deployments
- daemonsets
- replicasets
- statefulsets
verbs:
- "*"
- kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: default-account-istio-operator
subjects:
- kind: ServiceAccount
namespace: ${OPENSHIFT_ISTIO_OPERATOR_NAMESPACE}
name: default
roleRef:
kind: Role
name: istio-operator
apiGroup: rbac.authorization.k8s.io
- kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: default-account-istio-operator-cluster-role-binding
subjects:
- kind: ServiceAccount
namespace: ${OPENSHIFT_ISTIO_OPERATOR_NAMESPACE}
name: default
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
- kind: Deployment
apiVersion: apps/v1
metadata:
name: istio-operator
namespace: ${OPENSHIFT_ISTIO_OPERATOR_NAMESPACE}
spec:
replicas: 1
selector:
matchLabels:
name: istio-operator
template:
metadata:
labels:
name: istio-operator
spec:
containers:
- name: istio-operator
image: ${OPENSHIFT_ISTIO_PREFIX}istio-operator:${OPENSHIFT_ISTIO_VERSION}
ports:
- containerPort: 60000
name: metrics
command:
- istio-operator
args:
- "--release=${OPENSHIFT_RELEASE}"
- "--masterPublicURL=${OPENSHIFT_ISTIO_MASTER_PUBLIC_URL}"
- "--istioPrefix=${OPENSHIFT_ISTIO_PREFIX}"
- "--istioVersion=${OPENSHIFT_ISTIO_VERSION}"
- "--deploymentType=${OPENSHIFT_DEPLOYMENT_TYPE}"
imagePullPolicy: IfNotPresent
env:
- name: WATCH_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: OPERATOR_NAME
value: "istio-operator"
3.3 安装 operator
以下命令将安装 Service Mesh operator 到 Openshift 容器平台中。在集群任意一个节点执行。
$ oc new-project istio-operator
$ oc new-app -f istio_product_operator_template.yaml --param=OPENSHIFT_ISTIO_MASTER_PUBLIC_URL=<master public url>
3.4 验证 operator 是否安装成功
先前的命令创建了一个 deployment 资源 到 istio-operator 这个 project 里，同时运行了 operator 通过 custom resource 来管理 Red Hat OpenShift Service Mesh 中 control plane 的状态。
为了验证 operator 是否正确安装，执行以下命令观察日志：
$ oc logs -n istio-operator $(oc -n istio-operator get pods -l name=istio-operator --output=jsonpath={.items..metadata.name})
执行以上命令后，如果出现类似以下的结果，说明安装正确：
time="2018-08-31T17:42:39Z" level=info msg="Go Version: go1.9.4"
time="2018-08-31T17:42:39Z" level=info msg="Go OS/Arch: linux/amd64"
time="2018-08-31T17:42:39Z" level=info msg="operator-sdk Version: 0.0.5+git"
time="2018-08-31T17:42:39Z" level=info msg="Metrics service istio-operator created"
time="2018-08-31T17:42:39Z" level=info msg="Watching resource istio.openshift.com/v1alpha1, kind Installation, namespace istio-operator, resyncPeriod 0"
time="2018-08-31T17:42:39Z" level=info msg="Installing istio for Installation istio-installation"
3.5 部署 control plane
$ oc create -f istio-installation.yaml -n istio-operator #istio-installation.yaml的内容见上
观察安装过程中 pods 的状态：
$ oc get pods -n istio-system -w
3.6 验证 control plane 的安装
$ oc get pods -n istio-system
如果出现和下面一样的内容，说明安装成功：
NAME READY STATUS RESTARTS AGE
elasticsearch-0 1/1 Running 0 2m
grafana-6d5c5477-k7wrh 1/1 Running 0 2m
istio-citadel-6f9c778bb6-q9tg9 1/1 Running 0 3m
istio-egressgateway-957857444-2g84h 1/1 Running 0 3m
istio-galley-c47f5dffc-dm27s 1/1 Running 0 3m
istio-ingressgateway-7db86747b7-s2dv9 1/1 Running 0 3m
istio-pilot-5646d7786b-rh54p 2/2 Running 0 3m
istio-policy-7d694596c6-pfdzt 2/2 Running 0 3m
istio-sidecar-injector-57466d9bb-4cjrs 1/1 Running 0 3m
istio-statsd-prom-bridge-7f44bb5ddb-6vx7n 1/1 Running 0 3m
istio-telemetry-7cf7b4b77c-p8m2k 2/2 Running 0 3m
jaeger-agent-5mswn 1/1 Running 0 2m
jaeger-collector-9c9f8bc66-j7kjv 1/1 Running 0 2m
jaeger-query-fdc6dcd74-99pnx 1/1 Running 0 2m
kiali-779bcc566f-qqt65 1/1 Running 0 2m
openshift-ansible-istio-installer-job-f8n9g 0/1 Completed 0 7m
prometheus-84bd4b9796-2vcpc 1/1 Running 0 3m
如果在 custom resource 文件中配置了 launcher，查看 devex project 中的 容器状态会出现下面内容：
$ oc get pods -n devex
NAME READY STATUS RESTARTS AGE
configmapcontroller-1-8rr6w 1/1 Running 0 1m
launcher-backend-2-2wg86 1/1 Running 0 1m
launcher-frontend-2-jxjsd 1/1 Running 0 1m
四、在 Openshift Service Mesh 部署应用需要满足的要求
4.1 为应用的 service account 配置 SCC （Security Context Constraints，安全上下文 ）
oc adm policy add-scc-to-user anyuid -z <service account> -n <namespace>
oc adm policy add-scc-to-user privileged -z <service account> -n <namespace>
4.2 更新 Openshift Master节点的配置
保证以下操作在每个 Openshift Container Platform 的 master 节点中都执行：
4.2.1 切换目录到包含 master configuration 文件的路径下(for example, /etc/origin/master/master-config.yaml).
$ cd /etc/origin/master
4.2.2 创建master-config.patch，包含以下内容：
admissionConfig:
pluginConfig:
MutatingAdmissionWebhook:
configuration:
apiVersion: apiserver.config.k8s.io/v1alpha1
kubeConfigFile: /dev/null
kind: WebhookAdmission
ValidatingAdmissionWebhook:
configuration:
apiVersion: apiserver.config.k8s.io/v1alpha1
kubeConfigFile: /dev/null
kind: WebhookAdmission
4.2.3 在该目录下执行以下命令，来修改 master-config.yaml 文件：
$ cp -p master-config.yaml master-config.yaml.prepatch
$ oc ex config patch master-config.yaml.prepatch -p "$(cat master-config.patch)" > master-config.yaml
$ /usr/local/bin/master-restart api && /usr/local/bin/master-restart controllers
4.3 如何为应用配置 sidecar 的自动注入
相当简单，只需要在应用的部署yaml文件中，为 annotation 添加 sidecar.istio.io/inject 属性，并置为 true
例如：
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: sleep
spec:
replicas: 1
template:
metadata:
annotations:
sidecar.istio.io/inject: "true"
labels:
app: sleep
spec:
containers:
- name: sleep
image: tutum/curl
command: ["/bin/sleep","infinity"]
imagePullPolicy: IfNotPresent
至此，Openshift 3.11 环境下安装 istio 及相关配套组件的步骤全部结束，下一章进入 Service Mesh 示例工程 -- bookInfo 的安装教程
参考：https://docs.openshift.com/container-platform/3.11/servicemesh-install/servicemesh-install.html
