Persistent Storage

Kubernetes has many volume types. Persistent storage is just one of them. It has several concepts that cover the full storage lifecycle.

This volume type supports static or dynamic provisioning. The provisioning is fully hidden from the user.

1. plugin - These are the interfaces Kubernetes defines. Any storage provider must implement them to offer a volume to the cluster. Many plugin types exist. Most are built into the kubelet. Only the CSI plugin lets the implementation come from outside.
2. Driver - This is the controller that implements the plugin. It mostly runs as a controller on the admin layer. The node driver runs as a daemonSet on each node.
3. Storage Class - Represents different types of storage available. Depending on storage class requested, different provisioners will handle the request.
4. Persistent Volume Claim - Request for storage by a user. This is the volume type requested by the user.
5. Persistent Volume - Actual storage resource created on the infrastructure. Every persistent volume claim will be bound to a persistent volume.
6. VolumeClaimTemplate - Used in StatefulSets to create a persistent volume claim for each pod.

volume plugins vs drivers

Plugins are pure interfaces. Drivers are the implementation. This is why there are many plugin types, such as CSI, NFS, and local. Some drivers ship with vanilla Kubernetes.

PVC vs VolumeClaimTemplate

Say you have a deployment with 3 replicas. You can use one Persistent Volume Claim (PVC) for all 3 pods.

For StatefulSets, though, you use a VolumeClaimTemplate. Then each pod gets its own PVC from the template.

CSI Plugin Workflow

PV is dynamically created

When a PVC is created, the PVC controller creates the PV resource itself. The user need not create a Persistent Volume (PV) by hand.

If provisioning isn't automated, someone must create the storage by hand. Then they create the PV and link it to the PVC.

Storage Class

This is similar to what we saw in ingress. Every storage class represents a different provisioner.

Default storage class

Every cluster has a default storage class. Every PVC created without specifying a storage class will use the default storage class.

Access Modes

Access modes define what kind of access the volume needs. The storage class and the access modes pick the provisioner that handles the request.

Access modes are for Nodes

Access modes aren't for Pods. They're at the node level.

For example, mount a volume on a node in ReadWriteOnce mode. Then it's on one node, and all pods on that node can use it. With RWOP mode, only one pod can use it at a time.

Volume Mount Location

The CSI sets where the volume must be mounted on the host. Kubelet expects the volume at that exact spot once provisioning is done.

volume path example

This is defined by the specifications - /var/lib/kubelet/pods/${pod-uid}/volumes/${plugin-name}/${volume-name}/

Different types of volumes

1. emptyDir - The directory is empty at start. It's emptied when the pod is deleted on the node. It can also be memory backed.
2. hostPath - The data is tied to the host. If a pod moves to another host, it sees a different data volume there.
3. persistentVolumeClaim - Long-term storage. The volume and its data survives pod restarts.
4. configMap - injects configMap data as files.
5. secret - Injects secrets as files or environment variables.
6. downwardAPI - Pod metadata is provided as files inside the pod.
7. NFS, iSCSI, local, many more.