前言
本文Harbor高可用依照Harbor官网部署，主要思路如下，大家可以根据具体情况选择搭建。
- 部署Postgresql高可用集群。（本文选用Stolon进行管理，请查看文章《kubernetes下Stolon部署高可用Postgresql》）
- 部署redis高可用集群。（本文选用Helm对redis进行高可用部署，请查看文章《kubernetes部署高可用redis》，该文以整理好redis编排文件可直接使用）
- 部署Harbor高可用集群。（本文主要阐述Harbor的高可用部署，为《kubernetes搭建Harbor无坑及Harbor仓库同步》补充部分，请先行阅读）
一、Harbor部署前准备
本文仅说明高可用配置，其余部署请查看《kubernetes搭建Harbor无坑及Harbor仓库同步》
.安装方式
-
helm安装
-
直接使用博主整理好的编排文件安装（通过Helm生成）
1.helm安装
安装Helm请查看《kubernetes搭建Harbor无坑及Harbor仓库同步》，其中包含Helm安装。
1.1.下载 harbor-helm
git clone https://github.com/goharbor/harbor-helm.git
cd XXX/harbor-helm
1.2.修改value.yaml
database的Postgresql配置
database:
# if external database is used, set "type" to "external"
# and fill the connection informations in "external" section
type: external
internal:
image:
repository: goharbor/harbor-db
tag: v1.8.2-dev
# The initial superuser password for internal database
password: "changeit"
# resources:
# requests:
# memory: 256Mi
# cpu: 100m
nodeSelector: {}
tolerations: []
affinity: {}
external:
host: "stolon-proxy-service" #管理postgresql的stolon的service，因为都在Pod中可相互访问
port: "5432"
username: "postgres"
password: "password1"
coreDatabase: "registry"
clairDatabase: "clair"
notaryServerDatabase: "notaryserver"
notarySignerDatabase: "notarysigner"
sslmode: "disable"
## Additional deployment annotations
podAnnotations: {}
redis的配置
redis:
# if external Redis is used, set "type" to "external"
# and fill the connection informations in "external" section
type: external
internal:
image:
repository: goharbor/redis-photon
tag: v1.8.2-dev
# resources:
# requests:
# memory: 256Mi
# cpu: 100m
nodeSelector: {}
tolerations: []
affinity: {}
external:
host: "10.8.4.133" #haproxy的地址通过haproxy管理redis集群
port: "6379"
# The "coreDatabaseIndex" must be "0" as the library Harbor
# used doesn't support configuring it
coreDatabaseIndex: "0"
jobserviceDatabaseIndex: "1"
registryDatabaseIndex: "2"
chartmuseumDatabaseIndex: "3"
password: ""
修改Harbor其他组件replicas(副本数)
# 例如nginx的副本数更改
nginx:
image:
repository: goharbor/nginx-photon
tag: v1.8.2-dev
replicas: 3
1.3.准备Harbor所需的registry、notarysigner、notaryserver、clair数据库，Harbor会自动在其中建表。
执行sql语句脚本，供stolon-init-database-job.yaml使用
cat <<EOF > ./postgresql.sh
#!/bin/bash
host="stolon-proxy-service"
user="postgres"
db="postgres"
export PGPASSWORD="password1"
args=(
# force postgres to not use the local unix socket (test "external" connectibility)
--host "$host"
--username "$user"
--dbname "$db"
--quiet --no-align --tuples-only
)
if select="$(echo 'SELECT 1' | psql "${args[@]}")" && [ "$select" = '1' ]; then
echo "====notaryserver==database==creating===="
psql -h stolon-proxy-service -p 5432 -U postgres -f "/docker-entrypoint-initdb.d/initial-notaryserver.sql"
echo "====notarysigner==database==creating===="
psql -h stolon-proxy-service -p 5432 -U postgres -f "/docker-entrypoint-initdb.d/initial-notarysigner.sql"
echo "====registry==database==creating===="
psql -h stolon-proxy-service -p 5432 -U postgres -f "/docker-entrypoint-initdb.d/initial-registry.sql"
echo "====clair==database==creating===="
psql -h stolon-proxy-service -p 5432 -U postgres -f "/docker-entrypoint-initdb.d/initial-clair.sql"
exit 0
fi
exit 1
EOF
创建registry数据库
cat <<EOF > ./initial-registry.sql
CREATE DATABASE registry ENCODING 'UTF8';
\c registry;
CREATE TABLE schema_migrations(version bigint not null primary key, dirty boolean not null);
EOF
创建notaryserver数据库
cat <<EOF > ./initial-notaryserver.sql
CREATE DATABASE notaryserver;
CREATE USER server;
alter user server with encrypted password 'password';
GRANT ALL PRIVILEGES ON DATABASE notaryserver TO server;
EOF
创建notarysigner数据库
cat <<EOF > ./initial-notarysigner.sql
CREATE DATABASE notarysigner;
CREATE USER signer;
alter user signer with encrypted password 'password';
GRANT ALL PRIVILEGES ON DATABASE notarysigner TO signer;
EOF
创建clair数据库
cat <<EOF > ./initial-clair.sql
CREATE DATABASE clair;
EOF
创建一个job的yaml（stolon-init-database-job.yaml），用于创建数据库，注意更改脚本的挂载位置，并复制脚本到各个节点或为node和yaml加上nodeselect标签，只在当前标签node下复制脚本
apiVersion: batch/v1
kind: Job
metadata:
name: stolon-init-database-job
spec:
template:
spec:
containers:
- name: stolon-proxy
image: sorintlab/stolon:master-pg10
command:
- "/bin/bash"
- "/docker-entrypoint-initdb.d/postgresql.sh"
volumeMounts:
- mountPath: /docker-entrypoint-initdb.d
name: database
restartPolicy: OnFailure #失败重启
volumes:
- name: database
hostPath:
path: /root/tmp/harbor/stolon/examples/kubernetes/sql
activeDeadlineSeconds: 600 #10分钟没有complete，不再重启并移除Pod
1.3.部署Postgresql、redis
- 按照《kubernetes下Stolon部署高可用Postgresql》部署Postgresql，注意加入stolon-init-database-job.yaml。
- 按照《kubernetes部署高可用redis》部署redis，之后用haproxy管理redis集群（不可直接使用redis的service暴露，service会访问到slave节点，redis副本是只读不可写的，在harbor中会有报错）
- 部署haproxy
- 安装haproxy
yum -y install haproxy
cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg-back
vim /etc/haproxy/haproxy.cfg - 加入配置
- 安装haproxy
frontend ft_redis
bind 0.0.0.0:6379 name redis
default_backend bk_redis
backend bk_redis
option tcp-check
tcp-check connect
tcp-check send PING\r\n
tcp-check expect string +PONG
tcp-check send info\ replication\r\n
tcp-check expect string role:master
tcp-check send QUIT\r\n
tcp-check expect string +OK
server R1 redis-0.redis-headless.default.svc.cluster.local:6379 check inter 1s
server R2 redis-1.redis-headless.default.svc.cluster.local:6379 check inter 1s
server R3 redis-2.redis-headless.default.svc.cluster.local:6379 check inter 1s
listen admin_stats
stats enable
bind *:9090
mode http
option httplog
log global
maxconn 10
stats refresh 30s
stats uri /admin
stats realm haproxy
stats auth admin:admin
stats hide-version
stats admin if TRUE
systemctl start haproxy && systemctl enable haproxy && systemctl status haproxy
访问 harbor节点Ip:9090/admin如图所示，便成功
redis
⚠️k8s master节点高可用可阅读《haproxy+keepalive实现master集群高可用》
1.4.部署Harbor
安装harbor并将日志写入文件，可编辑文件保留.yaml编排文件，以便以后使用
helm install . --debug --name hub |sed 'w harbor.yaml'
或执行以下命令，编排chart不执行，作用生成编排文件，删除多余部分，进行使用
helm install . --debug --dry-run --name hub |sed 'w harbor.yaml'
2.通过整理好的编排文件执行
链接:https://pan.baidu.com/s/1cr1fnWGHc-70HAxx1YH4kg 密码:21a8
直接使用这个编排文件可能会有问题，最好勤劳以下使用helm跑，也可避免更改配置遗漏或错误的问题，适用用于实验，如若搭建请注意修改Volum、requestsource等Pod设置
