Persistent Storage

In Kubernetes, there are many types of volumes and persistent storage is just one type of volume. It has multiple concepts which represent the entire storage lifecycle.

This type of volume supports static or dynamic provisioning. Which means, the volume provisioning is fully abstracted to the user.

1. plugin - It's just the interfaces defined by Kubernetes which must be implemented by any storage provider to offer volume to the cluster. There are many types of plugin and most of them are implemented natively within the kubelet itself. Only CSI plugin was introduced where the implementations come externally.
2. Driver - It's the controller that implements the plugin specification. It's running mostly as controller on admin layer and the node driver is running as a daemonSet on each node.
3. Storage Class - Represents different types of storage available. Depending on storage class requested, different provisioners will handle the request.
4. Persistent Volume Claim - Request for storage by a user. This is the volume type requested by the user.
5. Persistent Volume - Actual storage resource created on the infrastructure. Every persistent volume claim will be bound to a persistent volume.
6. VolumeClaimTemplate - Used in StatefulSets to create a persistent volume claim for each pod.

volume plugins vs drivers

Plugins are just pure interfaces and drivers are the implementation. This is exactly why there are multiple types of plugin such as CSI, NFS, local, etc. Drivers for some are already part of vanilla Kubernetes.

PVC vs VolumeClaimTemplate

When we've a deployment with 3 replicas, we can use a single Persistent Volume Claim (PVC) to request storage for all 3 pods.

But in case of StatefulSets, we need to use a VolumeClaimTemplate. In this case, each pod will get it's own Persistent Volume Claim (PVC) using the requested template.

CSI Plugin Workflow

PV is dynamically created

When a Persistent Volume Claim (PVC) is created, the PVC controller creates the PV resource by itself. The user need not create a Persistent Volume (PV) manually.

In cases where the storage provisioning isn't automated, somebody will have to create the storage manually and then create the physical volume, PV and link it to the PVC.

Storage Class

This is similar to what we saw in ingress. Every storage class represents a different provisioner.

Default storage class

Every cluster has a default storage class. Every PVC created without specifying a storage class will use the default storage class.

Access Modes

Storage access modes defines what kind of access is required for the volume. It's the storage class and the access modes which determines the provisioner that will handle the request.

Access modes are for Nodes

Access modes aren't for Pods. It's at node level.

For example, if a volume is mounted on a node in ReadWriteOnce mode, then it's mounted on a single node and all pods on that node can access it. But if we use the RWOP mode, then the volume can be also used by only one pod at a time.

Volume Mount Location

The CSI specifies where the volume must be mounted on the host. Kubelet expects the volume at that very specific location after the volume provisioning is complete.

volume path example

This is defined by the specifications - /var/lib/kubelet/pods/${pod-uid}/volumes/${plugin-name}/${volume-name}/

Different types of volumes

1. emptyDir - Means the directory is empty when started and also is emptied/deleted when the pod is deleted on the node. It can also be memory backed storage.
2. hostPath - The data in the volume is specific to the host. When a pod is scheduled to another host, then it sees completely different data volume on that host.
3. persistentVolumeClaim - Long-term storage. The volume and its data survives pod restarts.
4. configMap - injects configMap data as files.
5. secret - Injects secrets as files or environment variables.
6. downwardAPI - Pod metadata is provided as files inside the pod.
7. NFS, iSCSI, local, many more.