动态存储简介
虽然K8S提供了PVC的方式进行存储的便利性，但是PV的创建还是要手工的，使用起来不是很方便，在k8s 1.4以后，kubernetes提供了一种更加方便的动态创建PV的方式，即StorageClass。使用StorageClass时无需预先创建固定大小的PV来等待使用者创建PVC使用，而是直接创建PVC即可使用。
本文主要使用Rook搭建一个ceph集群，然后使用StorageClass来直接创建PVC。
1 PV
PV全称叫做Persistent Volume，持久化存储卷。它是用来描述或者说用来定义一个存储卷的，这个通常都是有运维或者数据存储工程师来定义。比如下面我们定义一个NFS类型的PV：
accessModes：支持三种类型
- ReadWriteMany 多路读写，卷能被集群多个节点挂载并读写
- ReadWriteOnce 单路读写，卷只能被单一集群节点挂载读写
- ReadOnlyMany 多路只读，卷能被多个集群节点挂载且只能读
2 PVC
PVC是用来描述希望使用什么样的或者说是满足什么条件的存储，它的全称是Persistent Volume Claim，也就是持久化存储声明。开发人员使用这个来描述该容器需要一个什么存储。比如下面使用NFS的PVC：
这个PVC就会和上面的PV进行绑定，为什么呢？它有一些原则：
- PV和PVC中的spec关键字段要匹配，比如存储（storage）大小。
- PV和PVC中的storageClassName字段必须一致，这个后面再说。
3 StorageClass
StorageClassPV就是动态供给创建PV模板,是运维人员来创建的，开发操作PVC，可是大规模集群中可能会有很多PV，如果这些PV都需要运维手动来处理这也是一件很繁琐的事情，所以就有了动态供给概念，也就是Dynamic Provisioning。而我们上面的创建的PV都是静态供给方式，也就是Static Provisioning。
创建StorageClass里面需要定义PV属性比如存储类型、大小等；另外创建这种PV需要用到存储插件。最终效果是，用户提交PVC，里面指定存储类型，如果符合我们定义的StorageClass，则会为其自动创建PV并进行绑定。
image
Ceph 动态存储
Rook Ceph 环境搭建： https://www.cnblogs.com/vpc123/articles/14397126.html
# 进入客户端控制台
kubectl -n rook-ceph exec -it $(kubectl -n rook-ceph get pod -l "app=rook-ceph-tools" -o jsonpath='{.items[0].metadata.name}') bash
# 工具箱中的所有可用工具命令均已准备就绪，可满足您的故障排除需求
ceph status
ceph osd status
ceph df
rados df
1 Ceph 动态 StorageClass
storageclass.yaml 内容如下:
apiVersion: ceph.rook.io/v1
kind: CephBlockPool
metadata:
name: replicapool
namespace: rook-ceph
spec:
failureDomain: host
replicated:
size: 3
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: rook-ceph-block
# Change "rook-ceph" provisioner prefix to match the operator namespace if needed
provisioner: rook-ceph.rbd.csi.ceph.com
parameters:
# clusterID is the namespace where the rook cluster is running
clusterID: rook-ceph
# Ceph pool into which the RBD image shall be created
pool: replicapool
# RBD image format. Defaults to "2".
imageFormat: "2"
# RBD image features. Available for imageFormat: "2". CSI RBD currently supports only `layering` feature.
imageFeatures: layering
# The secrets contain Ceph admin credentials.
csi.storage.k8s.io/provisioner-secret-name: rook-csi-rbd-provisioner
csi.storage.k8s.io/provisioner-secret-namespace: rook-ceph
csi.storage.k8s.io/node-stage-secret-name: rook-csi-rbd-node
csi.storage.k8s.io/node-stage-secret-namespace: rook-ceph
# Specify the filesystem type of the volume. If not specified, csi-provisioner
# will set default as `ext4`.
csi.storage.k8s.io/fstype: xfs
# Delete the rbd volume when a PVC is deleted
reclaimPolicy: Delete
资源检查:
# 创建动态资源池
kubectl create -f storageclass.yaml
# 存储资源检查
[root@iZbp1dnzy9ygt2oosvzzzcZ ceph-demo]# kubectl -n rook-ceph get cephblockpools.ceph.rook.io
NAME AGE
replicapool 38m
[root@iZbp1dnzy9ygt2oosvzzzcZ ceph-demo]# kubectl -n rook-ceph get storageclasses.storage.k8s.io
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
local-path (default) rancher.io/local-path Delete WaitForFirstConsumer false 21h
rook-ceph-retain-bucket ceph.rook.io/bucket Retain Immediate false 160m
rook-ceph-block rook-ceph.rbd.csi.ceph.com Delete Immediate false 38m
2 Pod 实例
# 案例资源获取
wget https://raw.githubusercontent.com/rook/rook/release-1.2/cluster/examples/kubernetes/mysql.yaml
wget https://raw.githubusercontent.com/rook/rook/release-1.2/cluster/examples/kubernetes/wordpress.yam
# 资源部署
kubectl apply -f mysql.yaml
kubectl apply -f wordpress.yaml
# 查看存储资源
[root@iZbp1dnzy9ygt2oosvzzzcZ ceph-demo]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
myclaim Bound pvc-d63e22c9-c47c-4659-b467-d690431fa76f 2Gi RWO rook-ceph-block 78m
mysql-pv-claim Bound pvc-6b792873-c281-4541-ba53-e7eb0e2b36f5 20Gi RWO rook-ceph-block 41m
wp-pv-claim Bound pvc-9de3bbb1-9567-4508-86c8-744eeb660c54 20Gi RWO rook-ceph-block 39m
3 资源清理
kubectl delete -f wordpress.yaml
kubectl delete -f mysql.yaml
kubectl delete -n rook-ceph cephblockpools.ceph.rook.io replicapool
kubectl delete storageclass rook-ceph-block
存储解析
mysql.yaml 解析:
apiVersion: v1
kind: Service
metadata:
name: wordpress-mysql
labels:
app: wordpress
spec:
ports:
- port: 3306
selector:
app: wordpress
tier: mysql
clusterIP: None
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysql-pv-claim
labels:
app: wordpress
spec:
storageClassName: rook-ceph-block
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: wordpress-mysql
labels:
app: wordpress
tier: mysql
spec:
selector:
matchLabels:
app: wordpress
tier: mysql
strategy:
type: Recreate
template:
metadata:
labels:
app: wordpress
tier: mysql
spec:
containers:
- image: mysql:5.6
name: mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: changeme
ports:
- containerPort: 3306
name: mysql
volumeMounts:
- name: mysql-persistent-storage
mountPath: /var/lib/mysql
volumes:
- name: mysql-persistent-storage
persistentVolumeClaim:
claimName: mysql-pv-claim
1 查看动态 sc
[root@iZbp1dnzy9ygt2oosvzzzcZ ceph-demo]# kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
local-path (default) rancher.io/local-path Delete WaitForFirstConsumer false 21h
rook-ceph-retain-bucket ceph.rook.io/bucket Retain Immediate false 171m
rook-ceph-block rook-ceph.rbd.csi.ceph.com Delete Immediate false 49m
2 申请 pvc
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysql-pv-claim
labels:
app: wordpress
spec:
storageClassName: rook-ceph-block
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
image
2 绑定 pvc
apiVersion: apps/v1
kind: Deployment
metadata:
name: wordpress-mysql
labels:
app: wordpress
tier: mysql
spec:
selector:
matchLabels:
app: wordpress
tier: mysql
strategy:
type: Recreate
template:
metadata:
labels:
app: wordpress
tier: mysql
spec:
containers:
- image: mysql:5.6
name: mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: changeme
ports:
- containerPort: 3306
name: mysql
volumeMounts:
- name: mysql-persistent-storage
mountPath: /var/lib/mysql
volumes:
- name: mysql-persistent-storage
persistentVolumeClaim:
claimName: mysql-pv-claim
image
拓展阅读
后续的文章将会从云原生数据持久化的各个方面进行展开，存储的落地探讨：
image
