Docker & Containerization

Docker Compose

18 min Lesson 8 of 30

Docker Compose

A production application is almost never a single container. A typical web service runs at minimum three processes — a backend API, a relational database, and a cache — plus possibly a background worker, a message broker, and a reverse proxy. Running each with a bare docker run command means manually wiring networks, volumes, environment variables, and startup order. Docker Compose solves this by letting you declare the entire stack as a single YAML file and control it with one command.

What Compose Actually Does

Compose reads a docker-compose.yml (or compose.yaml — both are recognized) and translates it into a coordinated set of docker API calls: it creates a dedicated bridge network, pulls or builds each image, starts containers in dependency order, mounts volumes, and wires environment variables. Critically, it gives every service a DNS name matching its service key, so your API container reaches the database simply by hostname db — no IP management, no service discovery infrastructure needed for local development.

Compose V2 is the current standard. The old docker-compose Python binary (V1) is end-of-life. Modern Docker Desktop and Docker Engine ship docker compose (space, no hyphen) as a CLI plugin. Use docker compose in all new work; V1 syntax is largely compatible but the plugin is faster and maintained.

Anatomy of a Compose File

The following file defines a three-tier web stack: a Node.js API, a PostgreSQL database, and a Redis cache. Read every section carefully — each field maps to a docker run flag you already know.

# compose.yaml  —  a three-tier web stack

services:

  api:
    build:
      context: .
      target: production          # multi-stage build target
    image: myapp/api:dev
    ports:
      - "3000:3000"
    environment:
      DATABASE_URL: postgres://app:secret@db:5432/appdb
      REDIS_URL: redis://cache:6379
    depends_on:
      db:
        condition: service_healthy   # wait for the health check, not just start
      cache:
        condition: service_started
    restart: unless-stopped
    networks:
      - backend

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: appdb
    volumes:
      - pg_data:/var/lib/postgresql/data   # named volume = survives compose down
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d appdb"]
      interval: 5s
      timeout: 3s
      retries: 5
    networks:
      - backend

  cache:
    image: redis:7-alpine
    command: redis-server --save 60 1 --loglevel warning
    volumes:
      - redis_data:/data
    networks:
      - backend

volumes:
  pg_data:
  redis_data:

networks:
  backend:
    driver: bridge

Key design choices in this file:

depends_on with condition: service_healthy — without this, the API starts immediately after the container starts, not after PostgreSQL is actually ready to accept connections. That race condition causes startup failures in CI pipelines every single day.
Named volumes (pg_data, redis_data) instead of bind mounts for database data — they survive docker compose down but are destroyed by docker compose down -v. Never store database files in a bind mount; it creates file-permission mismatches between the host and container UID.
Explicit network — isolating services on a named network means they cannot reach containers in other Compose projects by accident, a real security boundary in shared environments.

The three services share the backend bridge network; DNS names match service keys. Named volumes persist database data independently of container lifecycle.

Essential Commands

Most day-to-day Compose work uses a handful of commands:

# Start all services in the background (detached)
docker compose up -d

# Rebuild images and restart (after Dockerfile changes)
docker compose up -d --build

# Follow logs from all services
docker compose logs -f

# Follow logs from one service only
docker compose logs -f api

# List running services and their status
docker compose ps

# Run a one-off command inside a service container
docker compose exec api sh

# Scale a stateless service to 3 replicas (port conflict: remove host port mapping first)
docker compose up -d --scale api=3

# Stop and remove containers, networks (volumes preserved)
docker compose down

# Stop and remove containers, networks, AND named volumes  —  destroys DB data
docker compose down -v

docker compose down -v destroys all named volumes. Running this against a stack with a real database wipes all data permanently. Always confirm which environment you are in before adding -v. On CI this is fine; on a staging environment it is catastrophic.

Environment Files and Variable Substitution

Hard-coding secrets in compose.yaml is a bad practice even for local development. Compose automatically loads a .env file from the project directory, and you can reference its variables using ${VAR} syntax:

# .env  (gitignored)
POSTGRES_PASSWORD=localdevpassword
API_TAG=latest
APP_PORT=3000

# compose.yaml referencing .env variables
services:
  api:
    image: myapp/api:${API_TAG}
    ports:
      - "${APP_PORT}:3000"
  db:
    environment:
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}

For multiple environments you can maintain separate override files and merge them at runtime:

compose.yaml — base definitions, committed to git
compose.override.yaml — auto-merged by Compose when present; ideal for local dev extras like volume bind-mounts of source code and debug ports
compose.ci.yaml — merge explicitly in CI with docker compose -f compose.yaml -f compose.ci.yaml up -d

Profiles: Conditional Services

Profiles let you define services that are not started by default and only activated on demand. This is the clean solution to the "I only want the monitoring stack in CI" or "run the seeder only once" problem:

services:
  api:
    build: .
    # no profile = always started

  db:
    image: postgres:16-alpine
    # no profile = always started

  seed:
    image: myapp/api:dev
    command: node scripts/seed.js
    depends_on:
      db:
        condition: service_healthy
    profiles:
      - tools        # only started when --profile tools is passed

  prometheus:
    image: prom/prometheus:latest
    profiles:
      - monitoring

# Usage:
#   docker compose up -d                         # starts api + db only
#   docker compose --profile tools run seed      # also runs seed
#   docker compose --profile monitoring up -d    # adds prometheus

Use profiles for database admin UIs. Services like Adminer or pgAdmin are invaluable locally but should never run in CI or production. Put them behind a debug profile — they start on demand and are invisible otherwise. This pattern eliminates the "oops, I left Adminer exposed" class of security incident.

Local Dev Stacks: Bind Mounts for Hot Reload

For active development you almost always want your source code mounted into the container so the process sees file changes without rebuilding. Use compose.override.yaml for this so the bind mount never reaches CI or production:

# compose.override.yaml  —  auto-merged locally, not committed (or gitignored)

services:
  api:
    build:
      target: development     # use the dev stage with dev dependencies
    volumes:
      - .:/app                # bind-mount source
      - /app/node_modules     # anonymous volume prevents host node_modules shadowing container
    command: npm run dev      # nodemon / ts-node-dev watches for changes
    environment:
      NODE_ENV: development
    ports:
      - "9229:9229"           # Node.js debugger port

The /app/node_modules anonymous volume is a well-known pattern: it prevents the host's node_modules (built for macOS/Windows) from shadowing the container's (built for Linux), which causes native binary failures. The same pattern applies to Python's __pycache__, Ruby's bundle, and Go's module cache.

Production Considerations

Compose is excellent for local development and small-scale deployments (a single VM running a complete stack). At production scale, teams graduate to Kubernetes. However, Docker Compose is still used in production at many companies for:

Single-server deployments with docker compose up -d managed by a CI pipeline
Integration test environments in CI — spinning up a real database and cache instead of mocking them
Internal tooling stacks that do not justify Kubernetes overhead

When running Compose in CI, always pin image tags to digests or specific versions (never latest) so your tests are reproducible. Use --wait (docker compose up -d --wait) in CI — it blocks until all services with health checks report healthy before the next pipeline step runs.