Volumes & Persistent Data
Volumes & Persistent Data
A container's writable layer is ephemeral by design. The moment you run docker rm, every byte written inside that layer vanishes. For stateless web servers that is a feature — identical containers, zero state drift. For anything that generates durable data — databases, file uploads, ML checkpoints, audit logs — it is a production landmine. This lesson draws the sharp line between bind mounts and Docker volumes, explains the data lifecycle behind each, and walks through the patterns that Google, Netflix, and similar shops use to keep stateful containers sane at scale.
The Writable Layer Problem
Docker images are composed of read-only layers stacked via a union filesystem (overlay2 on modern Linux). When a container starts, Docker adds one thin, writable layer on top. Writes go there and nowhere else. Three consequences follow:
- Data loss on removal.
docker rmdeletes the writable layer unconditionally. - No sharing. Two containers from the same image each get their own isolated writable layer — they cannot see each other's writes.
- I/O overhead. The union filesystem copy-on-write mechanism is slower than native disk; intensive random I/O (databases) suffers measurably.
The solution is to mount storage from outside the container into a path inside it. Docker provides two primary mechanisms: bind mounts and named volumes. A third, tmpfs mounts, keeps data in RAM and is covered at the end.
Named Volumes — The Production Default
A named volume is a chunk of host storage that Docker itself creates and manages, typically under /var/lib/docker/volumes/<name>/_data on Linux. You reference it by name, not by path. Docker handles creation, ownership, and lifecycle independently of any single container.
docker stop, docker rm, and even image upgrades. They are the right default for any database, cache store, or upload directory in production.
Volume drivers extend this model. The default local driver writes to the host disk. In cloud environments you swap it for a driver that mounts NFS, AWS EFS, or Azure Files, giving containers the same named-volume API while the backend is shared network storage:
Bind Mounts — For Development and Config Injection
A bind mount maps an absolute path on the host directly into the container. Docker does no management — the directory must already exist, and the container gains full read/write access to whatever is there.
The canonical production use cases for bind mounts are: injecting TLS certificates from a secrets manager that writes to a host path, mounting /var/run/docker.sock into CI agent containers (with extreme care), and sharing build artifacts between containers in a multi-stage pipeline running on the same host.
Data Lifecycle: Volume vs Container
Understanding what owns the lifecycle of data is the core mental model. A named volume's lifecycle is completely decoupled from any container:
- Container deleted → volume persists.
- Container upgraded (new image) → mount the same volume → data carries over automatically.
- Two containers can mount the same volume simultaneously — useful for read replicas or sidecar log shippers, but dangerous if both write the same files.
- Volume deleted explicitly with
docker volume rm→ data is gone permanently.
docker volume prune periodically on build hosts to reclaim disk from volumes no longer attached to any container. On production hosts, never prune without a confirmed backup — unnamed (anonymous) volumes from old containers are also swept.
Anonymous Volumes and the --volumes-from Pattern
When a Dockerfile declares a VOLUME instruction or you use -v /some/path without a name, Docker creates an anonymous volume — a named volume with a UUID as its name. It behaves identically to a named volume but is harder to manage because the name is opaque. Avoid anonymous volumes in anything you operate; always give volumes explicit names.
The legacy --volumes-from flag mounts all volumes from one container into another. It was common in the data-container pattern (pre-Compose era) and is still occasionally seen in backup sidecars:
tmpfs Mounts — Ephemeral In-Memory Storage
A tmpfs mount exists only in the host kernel's RAM and is never written to disk. It disappears when the container stops. Use it for sensitive scratch data (session tokens, decrypted secrets) that must not appear in logs or on disk:
Production Patterns at Scale
In a Kubernetes or Docker Swarm cluster, named volumes backed by the local driver do not work for multi-node deployments — each node has its own disk and there is no synchronization. The solutions used in practice:
- Cloud block storage (AWS EBS, GCP PD). Single-attach; the volume follows the pod/container to whichever node it schedules on, via a CSI driver (Kubernetes) or volume plugin (Swarm). Lowest latency; best for databases.
- Shared network filesystems (EFS, Azure Files, NFS). Multi-attach; any node can mount the same filesystem simultaneously. Good for shared assets like user uploads or ML model weights.
- Object storage (S3, GCS) via FUSE mounts or SDK. Best for large, infrequently-accessed blobs. Not a block device — wrong choice for a transactional database.