Volumes & Persistent Storage

Quick check: if a container restarts right now, does the file it just wrote survive? What if the whole Pod gets rescheduled to a different node? The answers are "sometimes" and "no" — knowing exactly why is the line between a toy app and one that can run a real database.

Containers forget everything

A container's filesystem is a thin, writable layer on top of the read-only image it was built from. Anything a process writes — a log, an upload, a database file — lands there, tied to the container's lifecycle, not the Pod's.

Restart the container and that layer is thrown away and rebuilt from the image. Delete the Pod and it's gone completely: new Pod, new empty layers, no memory of what came before. Fine for a stateless web server. A dealbreaker for Postgres, or anything storing user uploads.

Kubernetes' fix is the Volume: storage attached to a Pod that outlives that disposable layer. Not all volumes are equally durable — picking the wrong kind is a classic way to lose data. Here they are, least to most durable.

emptyDir: scratch space that lives with the Pod

An emptyDir is a directory that starts empty when the Pod is created and lasts exactly as long as the Pod — no longer, no shorter. A container crash and kubelet restart leaves it untouched, because the Pod itself never went away. Delete the Pod, or have it rescheduled elsewhere, and the emptyDir goes with it.

Remember the sidecar pattern from Post 3: a main container writing logs, a helper container shipping them. That shared volume is a textbook emptyDir — both containers mount it at different paths and pass data through it. It also works as plain scratch space you're happy to lose when the job ends.

apiVersion: v1
kind: Pod
metadata:
  name: log-shipper
spec:
  containers:
    - name: app
      image: myapp:latest
      volumeMounts:
        - name: log-volume
          mountPath: /var/log/app
    - name: log-forwarder
      image: fluent-bit:latest
      volumeMounts:
        - name: log-volume
          mountPath: /var/log/app
  volumes:
    - name: log-volume
      emptyDir: {}

Both containers reference the same volume name, log-volume, and mount it wherever they need it. That's the whole mechanism: no storage class, no external disk, just a directory the kubelet carves out on the node for the Pod's lifetime.

hostPath: tempting, and mostly a trap

A hostPath volume mounts a specific path from the node's own filesystem straight into your Pod. It looks like it solves persistence, since that directory doesn't disappear when the Pod dies. True — but only on that one node.

Say your Pod writes to /data/mydb via a hostPath on node worker-1, then gets deleted and recreated. If the scheduler places the new Pod on worker-2, it has no idea the data ever existed — different machine, its own empty /data/mydb. The data isn't lost, just stranded, unless you pin the Pod to that exact node — which fights everything Kubernetes is good at: flexible scheduling and self-healing.

So hostPath is the wrong tool for application data. You'll mostly see it used for node-level system tooling, like a monitoring agent reading /var/log on that specific node, where "this node's own files" is the point, not a bug.

PersistentVolume and PersistentVolumeClaim: the real pattern

This is the pair of objects you reach for whenever a Pod needs storage that outlives it, no matter which node it lands on. Kubernetes splits the job into two roles.

A PersistentVolume (PV) is an actual piece of storage attached to the cluster — its own object, independent of any Pod. It could be a cloud disk (a GCP Persistent Disk on GKE), an NFS share, or another backend.

A PersistentVolumeClaim (PVC) is what your Pod actually references: a request, not a specific disk. It says "I need 1Gi of storage, ReadWriteOnce access." You never name which PV to use — Kubernetes matches your claim to one that fits, same as the storage-unit company handing you a unit matching your claim ticket. Your Pod mounts the PVC, never the PV, and stays ignorant of what's really backing it.

Here's a minimal PVC:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: db-data-claim
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: standard

And a Pod that mounts it, using persistentVolumeClaim in place of emptyDir in the volumes list:

apiVersion: v1
kind: Pod
metadata:
  name: postgres
spec:
  containers:
    - name: postgres
      image: postgres:16
      volumeMounts:
        - name: db-data
          mountPath: /var/lib/postgresql/data
  volumes:
    - name: db-data
      persistentVolumeClaim:
        claimName: db-data-claim

Notice the shape: the volume block still has a name (db-data) that volumeMounts refers to, same mechanism as emptyDir — it just points at a claimName instead of declaring storage inline.

StorageClass: how the PV actually gets made

Who creates the PV? An admin can pre-create them by hand, sized and ready, and the PVC binds to one sitting around — static provisioning, tedious at scale.

In practice, most clusters use dynamic provisioning through a StorageClass: a template telling Kubernetes how to create storage on demand — which backend, what disk type. When your PVC names a storageClassName (like standard above) and no matching PV exists, the provisioner creates one automatically — on GKE, a real GCP Persistent Disk appears in your project just because you applied a PVC. Most managed clusters ship with a default StorageClass, so you can often skip storageClassName and fall back to it.

Trying it yourself

Apply the PVC:

$ kubectl apply -f pvc.yaml
persistentvolumeclaim/db-data-claim created

$ kubectl get pvc
NAME             STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
db-data-claim    Bound    pvc-8f2c1a3e-9b7d-4e21-9a5f-1c2d3e4f5a6b   1Gi        RWO            standard       9s

$ kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                    STORAGECLASS   AGE
pvc-8f2c1a3e-9b7d-4e21-9a5f-1c2d3e4f5a6b   1Gi        RWO            Delete           Bound    default/db-data-claim    standard       9s

The PV's name is auto-generated and ugly — you were never meant to name or care about it. STATUS Bound on both objects is what you want: claim and volume are linked.

If something's off, describe is where you go — same habit as debugging Pods in Post 3:

$ kubectl describe pvc db-data-claim
Name:          db-data-claim
Namespace:     default
StorageClass:  standard
Status:        Bound
Volume:        pvc-8f2c1a3e-9b7d-4e21-9a5f-1c2d3e4f5a6b
Capacity:      1Gi
Access Modes:  RWO
Events:
  Type    Reason                 Age   From                         Message
  ----    ------                 ----  ----                         -------
  Normal  ProvisioningSucceeded  10s   persistentvolume-controller  Successfully provisioned volume pvc-8f2c1a3e-9b7d-4e21-9a5f-1c2d3e4f5a6b

Next up: Namespaces and cluster organization — how to carve one cluster into isolated slices for different teams, environments, and projects.