一、数据存储概述

在 Docker 中就有数据卷的概念，当容器删除时，数据也一起会被删除，想要持久化使用数据，需要把主机上的目录挂载到 Docker 中去，在 K8S 中，数据卷是通过 Pod 实现持久化的，如果 Pod 删除，数据卷也会一起删除，k8s 的数据卷是 docker 数据卷的扩展，K8S 适配各种存储系统，包括本地存储 EmptyDir,HostPath,网络存储 NFS,GlusterFS,PV/PVC 等，下面就详细介绍下 K8S 的存储如何实现。

二、查看可用的存储

使用以下命令，可以看到Kubernetes中可用的存储：

  1. $ kubectl explain pods.spec.volumes
  2. KIND:     Pod
  3. VERSION:  v1
  5. RESOURCE: volumes <[]Object>
  7. DESCRIPTION:
  8.      List of volumes that can be mounted by containers belonging to the pod.    
  9.      More info: https://kubernetes.io/docs/concepts/storage/volumes
  11.      Volume represents a named volume in a pod that may be accessed by any      
  12.      container in the pod.
  14. FIELDS:
  15.    awsElasticBlockStore <Object>
  16.      AWSElasticBlockStore represents an AWS Disk resource that is attached to a 
  17.      kubelet's host machine and then exposed to the pod. More info:
  18.      https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore   
  20.    azureDisk    <Object>
  21.      AzureDisk represents an Azure Data Disk mount on the host and bind mount to
  22.      the pod.
  24.    azureFile    <Object>
  25.      AzureFile represents an Azure File Service mount on the host and bind mount
  26.      to the pod.
  28.    cephfs       <Object>
  29.      CephFS represents a Ceph FS mount on the host that shares a pod's lifetime
  31.    cinder       <Object>
  32.      Cinder represents a cinder volume attached and mounted on kubelets host
  33.      machine. More info: https://examples.k8s.io/mysql-cinder-pd/README.md
  35.    configMap    <Object>
  36.      ConfigMap represents a configMap that should populate this volume
  38.    csi  <Object>
  39.      CSI (Container Storage Interface) represents storage that is handled by an
  40.      external CSI driver (Alpha feature).
  42.    downwardAPI  <Object>
  43.      DownwardAPI represents downward API about the pod that should populate this
  44.      volume
  46.    emptyDir     <Object>
  47.      EmptyDir represents a temporary directory that shares a pod's lifetime.
  48.      More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir
  50.    fc   <Object>
  51.      FC represents a Fibre Channel resource that is attached to a kubelet's host
  52.      machine and then exposed to the pod.
  54.    flexVolume   <Object>
  55.      FlexVolume represents a generic volume resource that is
  56.      provisioned/attached using an exec based plugin.
  58.    flocker      <Object>
  59.      Flocker represents a Flocker volume attached to a kubelet's host machine.
  60.      This depends on the Flocker control service being running
  62.    gcePersistentDisk    <Object>
  63.      GCEPersistentDisk represents a GCE Disk resource that is attached to a
  64.      kubelet's host machine and then exposed to the pod. More info:
  65.      https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk
  67.    gitRepo      <Object>
  68.      GitRepo represents a git repository at a particular revision. DEPRECATED:
  69.      GitRepo is deprecated. To provision a container with a git repo, mount an
  70.      EmptyDir into an InitContainer that clones the repo using git, then mount
  71.      the EmptyDir into the Pod's container.
  73.    glusterfs    <Object>
  74.      Glusterfs represents a Glusterfs mount on the host that shares a pod's
  75.      lifetime. More info: https://examples.k8s.io/volumes/glusterfs/README.md
  77.    hostPath     <Object>
  78.      HostPath represents a pre-existing file or directory on the host machine
  79.      that is directly exposed to the container. This is generally used for
  80.      system agents or other privileged things that are allowed to see the host
  81.      machine. Most containers will NOT need this. More info:
  82.      https://kubernetes.io/docs/concepts/storage/volumes#hostpath
  84.    iscsi        <Object>
  85.      ISCSI represents an ISCSI Disk resource that is attached to a kubelet's
  86.      host machine and then exposed to the pod. More info:
  87.      https://examples.k8s.io/volumes/iscsi/README.md
  89.    name <string> -required-
  90.      Volume's name. Must be a DNS_LABEL and unique within the pod. More info:
  91.      https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
  93.    nfs  <Object>
  94.      NFS represents an NFS mount on the host that shares a pod's lifetime More
  95.      info: https://kubernetes.io/docs/concepts/storage/volumes#nfs
  97.    persistentVolumeClaim        <Object>
  98.      PersistentVolumeClaimVolumeSource represents a reference to a
  99.      PersistentVolumeClaim in the same namespace. More info:
  100.      https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims
  102.    photonPersistentDisk <Object>
  103.      PhotonPersistentDisk represents a PhotonController persistent disk attached
  104.      and mounted on kubelets host machine
  106.    portworxVolume       <Object>
  107.      PortworxVolume represents a portworx volume attached and mounted on
  108.      kubelets host machine
  110.    projected    <Object>
  111.      Items for all in one resources secrets, configmaps, and downward API
  113.    quobyte      <Object>
  114.      Quobyte represents a Quobyte mount on the host that shares a pod's lifetime
  116.    rbd  <Object>
  117.      RBD represents a Rados Block Device mount on the host that shares a pod's
  118.      lifetime. More info: https://examples.k8s.io/volumes/rbd/README.md
  120.    scaleIO      <Object>
  121.      ScaleIO represents a ScaleIO persistent volume attached and mounted on
  122.      Kubernetes nodes.
  124.    secret       <Object>
  125.      Secret represents a secret that should populate this volume. More info:
  126.      https://kubernetes.io/docs/concepts/storage/volumes#secret
  128.    storageos    <Object>
  129.      StorageOS represents a StorageOS volume attached and mounted on Kubernetes
  130.      nodes.
  132.    vsphereVolume        <Object>
  133.      VsphereVolume represents a vSphere volume attached and mounted on kubelets
  134.      host machine

三、相关概念

持久卷

管理存储和管理计算有着明显的不同。PersistentVolume子系统给用户和管理员提供了一套API，从而抽象出存储是如何提供和消耗的细节。在这里，我们介绍两种新的API资源：PersistentVolume（简称PV）和PersistentVolumeClaim（简称PVC）。

容量

一般来说，PV会指定存储的容量，使用PV的 capacity 属性来设置。

当前，存储大小是唯一能被设置或请求的资源。未来可能包含IOPS，吞吐率等属性。

访问模式

PV可以使用存储资源提供商支持的任何方法来映射到host中。如下的表格中所示，提供商有着不同的功能，每个PV的访问模式被设置为卷支持的指定模式。比如，NFS可以支持多个读/写的客户端，但可以在服务器上指定一个只读的NFS PV。每个PV有它自己的访问模式。

访问模式包括：

  • ReadWriteOnce —— 该volume只能被单个节点以读写的方式映射
  • ReadOnlyMany —— 该volume可以被多个节点以只读方式映射
  • ReadWriteMany —— 该volume只能被多个节点以读写的方式映射

在CLI中，访问模式可以简写为：

  • RWO - ReadWriteOnce
  • ROX - ReadOnlyMany
  • RWX - ReadWriteMany
Volume Plugin ReadWriteOnce ReadOnlyMany ReadWriteMany
HostPath ✓ - -
NFS ✓ ✓ ✓
AWSElasticBlockStore ✓ - -
AzureFile ✓ ✓ ✓
AzureDisk ✓ - -
CephFS ✓ ✓ ✓
Cinder ✓ - -
FC ✓ ✓ -
FlexVolume ✓ ✓ -
Flocker ✓ - -
GCEPersistentDisk ✓ ✓ -
Glusterfs ✓ ✓ ✓
iSCSI ✓ ✓ -
PhotonPersistentDisk ✓ - -
Quobyte ✓ ✓ ✓
RBD ✓ ✓ -
VsphereVolume ✓ - -
PortworxVolume ✓ - ✓
ScaleIO ✓ ✓ -

存储类

一个PV可以有一种class，通过设置storageClassName属性来选择指定的StorageClass。有指定class的PV只能绑定给请求该class的PVC。没有设置storageClassName属性的PV只能绑定给未请求class的PVC。

回收策略

当前的回收策略有：

  • Retain: 允许用户手动回收
  • Recycle: 删除 PV 上的数据 (“rm -rf /thevolume/*”)
  • Delete: 删除 PV

当前，只有NFS和HostPath支持回收利用，AWS EBS，GCE PD，Azure Disk，or OpenStack Cinder卷支持删除操作。

卷所处阶段

一个volume卷处于以下几个阶段之一：

  • Available: 空闲的资源，未绑定给PVC
  • Bound: 绑定给了某个PVC
  • Released: PVC已经删除了，但是PV还没有被集群回收
  • Failed: PV在自动回收中失败了

参考资料