K8s对于存储解耦的设计是,pv交给存储管理员来管理,我们只管用pvc来消费就好,但这里我们实际还是得一起管理pv和pvc,在实际工作中,我们(存储管理员)可以提前配置好pv的动态供给StorageClass,来根据pvc的消费动态生成pv。
利用nfs-client-provisioner来生成一个基于nfs的StorageClass,部署配置yaml配置如下,保持为nfs-sc.yaml:
cat nfs-sc.yamlapiVersion: v1kind: ServiceAccountmetadata:name: nfs-client-provisionernamespace: kube-system---kind: ClusterRoleapiVersion: rbac.authorization.k8s.io/v1metadata:name: nfs-client-provisioner-runnerrules:- apiGroups: [""]resources: ["persistentvolumes"]verbs: ["get", "list", "watch", "create", "delete"]- apiGroups: [""]resources: ["persistentvolumeclaims"]verbs: ["get", "list", "watch", "update"]- apiGroups: ["storage.k8s.io"]resources: ["storageclasses"]verbs: ["get", "list", "watch"]- apiGroups: [""]resources: ["events"]verbs: ["list", "watch", "create", "update", "patch"]- apiGroups: [""]resources: ["endpoints"]verbs: ["get", "list", "watch", "create", "update", "patch"]---kind: ClusterRoleBindingapiVersion: rbac.authorization.k8s.io/v1metadata:name: run-nfs-client-provisionersubjects:- kind: ServiceAccountname: nfs-client-provisionernamespace: kube-systemroleRef:kind: ClusterRolename: nfs-client-provisioner-runnerapiGroup: rbac.authorization.k8s.io---kind: DeploymentapiVersion: apps/v1metadata:name: nfs-provisioner-01namespace: kube-systemspec:replicas: 1strategy:type: Recreateselector:matchLabels:app: nfs-provisioner-01template:metadata:labels:app: nfs-provisioner-01spec:serviceAccountName: nfs-client-provisionercontainers:- name: nfs-client-provisionerimage: jmgao1983/nfs-client-provisioner:latestimagePullPolicy: IfNotPresentvolumeMounts:- name: nfs-client-rootmountPath: /persistentvolumesenv:- name: PROVISIONER_NAMEvalue: nfs-provisioner-01 # 此处供应者名字供storageclass调用- name: NFS_SERVERvalue: 10.0.1.201 # 填入NFS的地址- name: NFS_PATHvalue: /nfs_dir # 填入NFS挂载的目录volumes:- name: nfs-client-rootnfs:server: 10.0.1.201 # 填入NFS的地址path: /nfs_dir # 填入NFS挂载的目录---apiVersion: storage.k8s.io/v1kind: StorageClassmetadata:name: nfs-bogeprovisioner: nfs-provisioner-01# Supported policies: Delete、 Retain , default is DeletereclaimPolicy: Retain
开始创建这个StorageClass:
# kubectl apply -f nfs-sc.yamlserviceaccount/nfs-client-provisioner createdclusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-runner createdclusterrolebinding.rbac.authorization.k8s.io/run-nfs-client-provisioner createddeployment.apps/nfs-provisioner-01 createdorageclass.storage.k8s.io/nfs-boge created# 注意这个是在放kube-system的namespace下面,这里面放置一些偏系统类的服务# kubectl -n kube-system get pod -wNAME READY STATUS RESTARTS AGEcalico-kube-controllers-7fdc86d8ff-dpdm5 1/1 Running 1 24hcalico-node-8jcp5 1/1 Running 1 24hcalico-node-m92rn 1/1 Running 1 24hcalico-node-xg5n4 1/1 Running 1 24hcalico-node-xrfqq 1/1 Running 1 24hcoredns-d9b6857b5-5zwgf 1/1 Running 1 24hmetrics-server-869ffc99cd-wfj44 1/1 Running 2 24hnfs-provisioner-01-5db96d9cc9-qxlgk 0/1 ContainerCreating 0 9snfs-provisioner-01-5db96d9cc9-qxlgk 1/1 Running 0 21s# StorageClass已经创建好了# kubectl get scNAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGEnfs-boge nfs-provisioner-01 Retain Immediate false 37s
我们来基于StorageClass创建一个pvc,看看动态生成的pv是什么效果:
# vim pvc-sc.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: pvc-sc
spec:
storageClassName: nfs-boge
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Mi
# kubectl apply -f pvc-sc.yaml
persistentvolumeclaim/pvc-sc created
# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvc-sc Bound pvc-63eee4c7-90fd-4c7e-abf9-d803c3204623 1Mi RWX nfs-boge 3s
pvc1 Bound pv1 1Gi RWO nfs 24m
# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv1 1Gi RWO Recycle Bound default/pvc1 nfs 49m
pvc-63eee4c7-90fd-4c7e-abf9-d803c3204623 1Mi RWX Retain
我们修改下nginx的yaml配置,将pvc的名称换成上面的pvc-sc:
# vim nginx.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx
volumeMounts: # 我们这里将nginx容器默认的页面目录挂载
- name: html-files
mountPath: "/usr/share/nginx/html"
volumes:
- name: html-files
persistentVolumeClaim:
claimName: pvc-sc
# kubectl apply -f nginx.yaml
service/nginx unchanged
deployment.apps/nginx configured
# 这里注意下,因为是动态生成的pv,所以它的目录基于是一串随机字符串生成的,这时我们直接进到pod内来创建访问页面
# kubectl exec -it nginx-57cdc6d9b4-n497g -- bash
root@nginx-57cdc6d9b4-n497g:/# echo 'storageClass used' > /usr/share/nginx/html/index.html
root@nginx-57cdc6d9b4-n497g:/# exit
# curl 10.68.238.54
storageClass used
# 我们看下NFS挂载的目录
# ll /nfs_dir/
total 0
drwxrwxrwx 2 root root 24 Nov 27 17:52 default-pvc-sc-pvc-63eee4c7-90fd-4c7e-abf9-d803c3204623
drwxr-xr-x 2 root root 6 Nov 27 17:25 pv1
