vault+k8s
vault-agent workflow
我们通过vault的secret（version1 kv）backend来进行管理secret中的key/value，通过我们的vault-agent来动态（每隔SECRET_REFRESH_TIME秒来获取这些既定路径的secret，并写到K8S的secret中。
比如我们写入以下key/value
vault write secret/projects/georgesreinc-test/services/foo/defaults/test1 value=value1
vault write secret/projects/georgesreinc-test/services/bar/defaults/test1 value=value1
vault write secret/projects/georgesreinc-test/services/bar/defaults/test2 value=value2
vault write secret/projects/georgesreinc-test/services/baz/namespaces/ops/apikey value=secret
kubectl create namespace ops
kubectl create namespace stable
kubectl create namespace all-apps
kubectl create namespace foo
kubectl create namespace wrong
kubectl apply -f - <<-EOF
---
apiVersion: v1
kind: ConfigMap
metadata:
name: vault-agent.app-config
namespace: ops
data:
apps: |-
- baz
---
apiVersion: v1
kind: ConfigMap
metadata:
name: vault-agent.app-config
namespace: all-apps
data:
apps: ALL
---
apiVersion: v1
kind: ConfigMap
metadata:
name: vault-agent.app-config
namespace: foo
data:
apps: foo
---
apiVersion: v1
kind: ConfigMap
metadata:
name: vault-agent.app-config
namespace: wrong
data:
apps: |-
- baz
- ALL
EOF
我们的k8s中自动创建出来的secret条目如下所示：
kubectl get secret --all-namespaces | grep service-secrets should yield something that looks like:
all-apps bar.service-secrets Opaque 2 19m
all-apps foo.service-secrets Opaque 1 19m
foo foo.service-secrets Opaque 1 19m
ops baz.service-secrets Opaque 1 19m
stable bar.service-secrets Opaque 2 19m
stable foo.service-secrets Opaque 1 19m
wrong bar.service-secrets Opaque 2 19m
wrong foo.service-secrets Opaque 1 19m
vault-agent基本上可以实现：
- 控制某个secret只能创建在某个/些指定的namespaces中
- 动态更新secret中的key/value值
- 每隔SECRET_REFRESH_TIME秒更新一次secret
先决条件
- 已经存在了一个k8s（本文用docker for mac自带的Kubernetes，设置的教程见https://docs.docker.com/docker-for-mac/#kubernetes）
- k8s已经安装集成helm
- 已经存在了一个vault（我们会在mac本地系统起一个测试vault）
设置用于测试的vault server
符合生产环境的vault搭建会写在以后的文章中，本文直接在mac上启动一个dev版本的vault server并且做最简单的设置
我们按照官方的学习文档来启动一个dev vault server，简单的步骤如下：
https://learn.hashicorp.com/vault/getting-started/dev-server
安装vault server
brew install vault启动vault server
georgehe@Sha-51664-Mbp ~ vault server -dev -dev-listen-address=0.0.0.0:8200
==> Vault server configuration:
Api Address: http://0.0.0.0:8200
Cgo: disabled
Cluster Address: https://0.0.0.0:8201
Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
Log Level: info
Mlock: supported: false, enabled: false
Storage: inmem
Version: Vault v1.1.2
Version Sha: 0082501623c0b704b87b1fbc84c2d725994bac54
WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory
and starts unsealed with a single unseal key. The root token is already
authenticated to the CLI, so you can immediately begin using Vault.
You may need to set the following environment variable:
$ export VAULT_ADDR='http://0.0.0.0:8200'
The unseal key and root token are displayed below in case you want to
seal/unseal the Vault or re-authenticate.
Unseal Key: x4np0Bh3/h5xfxw1hqV0JCdz7QUbpVu90G4YJZ1GdSo=
Root Token: s.w8c22AWPXvyqS7d9cZDdw9Lu
Development mode should NOT be used in production installations!
==> Vault server started! Log data will stream in below:
- 访问vault server
我们要得到mac的IP地址（DHCP）
ifconfig en0 | grep "inet "
inet 192.168.31.185 netmask 0xffffff00 broadcast 192.168.31.255
我们的vault访问地址为：http://192.168.31.185:8200
- 测试vault的可用性，及写入两个用于测试的k/v
$ export VAULT_ADDR=http://192.168.31.185:8200
$ vault status
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed false
Total Shares 1
Threshold 1
Version 1.1.2
Cluster Name vault-cluster-79c802fc
Cluster ID 26bef5d1-febd-0319-6170-07808889b9b7
HA Enabled false
$ vault login token=s.w8c22AWPXvyqS7d9cZDdw9Lu
$ vault secrets disable secret
$ vault secrets enable -path=secret kv
$ vault write secret/projects/georgeinc/services/test-app/defaults/key1 value=value1
$ vault list secret/projects/georgeinc/services/test-app/defaults/
Keys
----
key1
$ vault read secret/projects/georgeinc/services/test-app/defaults/key1
Key Value
--- -----
refresh_interval 768h
value value1
Using dev mode with KV v1 by default #111
我们用的dev模式，默认secret是version2版本，所以我们先disable secret再重新启用version1的secret backend
后面我们期待生成的secret名字为test-app.service-secrets，里面有key1=value1的内容，此secret将默认出现在所有的namespaces下。
使用helm安装vault
- 加载
george-srehelm repo
helm repo add george-sre 'https://raw.githubusercontent.com/george-sre/helm-repo-in-github/master/'
helm repo update
helm search vault-agent
- 安装
vault-agentChart
helm install --debug george-sre/vault-agent \
--name vault-agent \
--set project=georgeinc \
--set vault_addr=http://192.168.31.185:8200 \
--set vault_token=s.w8c22AWPXvyqS7d9cZDdw9Lu
- 验证helm的安装
helm list
NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
vault-agent 1 Mon May 27 15:39:56 2019 DEPLOYED vault-agent-0.1.0 1.0 default
kubectl get pod
NAME READY STATUS RESTARTS AGE
vault-agent-77964b9fbb-xfbkt 1/1 Running 0 34s
- 验证secret的创建
kubectl get secret --all-namespaces | grep test-app
docker test-app.service-secrets Opaque 1 14m
kubectl get secret test-app.service-secrets -n docker -o yaml --export
apiVersion: v1
data:
key1: dmFsdWUx
kind: Secret
metadata:
creationTimestamp: null
labels:
app: test-app
name: test-app.service-secrets
selfLink: /api/v1/namespaces/docker/secrets/test-app.service-secrets
type: Opaque
更多关于vault-agent的用法，请移步https://github.com/george-sre/vault-agent
链接
- https://github.com/george-sre/vault-agent
- https://github.com/george-sre/helm-repo-in-github
- https://learn.hashicorp.com/vault/getting-started/dev-server
云平台开发运维解决方案@george.sre
GitHub: https://github.com/george-sre
欢迎交流～
