Web Servers & Reverse Proxies

Reverse Proxying & Upstreams

18 min Lesson 4 of 28

Reverse Proxying & Upstreams

At large-scale companies, Nginx almost never serves application responses directly — it acts as the front door that terminates client connections and forwards requests to a pool of application servers running behind it. This is reverse proxying: the client talks to Nginx, Nginx talks to your app, and the client never sees the app directly. Everything from Google's front-end infrastructure to Netflix's edge layer follows this pattern.

Mastering proxy_pass and upstream blocks is the single most important Nginx skill for a DevOps engineer. You will use it for Node.js apps, Python/Django/Flask services, Laravel/PHP-FPM via FastCGI, gRPC back-ends, microservice meshes, and websocket servers.

proxy_pass: The Core Directive

proxy_pass tells Nginx where to forward a matched request. When Nginx receives a connection it:

Reads the full request headers from the client (or the first buffer-worth of body).
Opens a TCP connection to the upstream target (or reuses a keepalive connection from its pool).
Forwards the reconstructed request, adding proxy headers.
Streams the upstream response back to the client, buffering or passing through depending on your config.

The trailing slash in proxy_pass matters enormously. Without a trailing slash, the full URI (including the location prefix) is sent upstream. With a trailing slash (or any path component), Nginx strips the matched prefix and appends the remainder.

server {
    listen 80;
    server_name api.example.com;

    # WITHOUT a trailing slash:
    # GET /api/users  →  upstream receives  /api/users
    location /api/ {
        proxy_pass http://127.0.0.1:3000;
    }

    # WITH a trailing slash (URI rewriting):
    # GET /api/users  →  upstream receives  /users
    location /api/ {
        proxy_pass http://127.0.0.1:3000/;
    }
}

Production pitfall — URI stripping surprises: The trailing-slash behavior catches engineers constantly. If your Node.js or Django app expects paths without the /api prefix, you need the trailing slash. If it expects the full path, omit it. Mismatches produce 404s that look like Nginx routing errors but are actually the app rejecting the rewritten URI. Always test with curl -v and inspect what the app actually receives.

Essential Proxy Headers

When Nginx forwards a request, it establishes a new TCP connection to the upstream. By default the upstream sees Nginx's loopback IP as the client — not the real visitor's IP. You must explicitly pass the original context in headers.

location / {
    proxy_pass http://127.0.0.1:8080;

    # Pass the real client IP so app logs and rate-limiters see it
    proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header X-Real-IP         $remote_addr;

    # Tell the app what protocol the client used (http or https)
    proxy_set_header X-Forwarded-Proto $scheme;

    # Pass the original Host header so the app generates correct URLs
    proxy_set_header Host              $host;

    # Nginx uses HTTP/1.0 to upstreams by default — force 1.1 for keepalive + chunked
    proxy_http_version 1.1;

    # Required when using keepalive upstream connections
    proxy_set_header Connection "";
}

X-Forwarded-For vs X-Real-IP: X-Forwarded-For is a comma-separated list that grows as the request passes through proxies (client, proxy1, proxy2). Your application should read the leftmost IP it trusts — not the rightmost, which an attacker can spoof. X-Real-IP is a single-value header set by Nginx to $remote_addr (the IP of the connection Nginx received, which is the real client IP when Nginx is the first proxy). In most single-layer setups, X-Real-IP is simpler to use in application code.

Upstream Pools: Scaling Beyond One Server

A single app process is a single point of failure. The upstream block defines a named pool of backend servers that Nginx distributes traffic across. All the load balancing, health-checking, and failover logic lives here.

Nginx upstream pool distributing client requests across three application server instances.

# Define the pool — Nginx can load-balance across all listed servers
upstream app_servers {
    # Default algorithm: round-robin (each request goes to the next server)
    server 10.0.1.10:8000;
    server 10.0.1.11:8000;
    server 10.0.1.12:8000;

    # Keep 32 idle connections open per worker to each upstream
    # (avoids TCP handshake overhead on every request)
    keepalive 32;
}

server {
    listen 80;
    server_name app.example.com;

    location / {
        proxy_pass         http://app_servers;   # name matches upstream block
        proxy_http_version 1.1;
        proxy_set_header   Connection        "";
        proxy_set_header   Host              $host;
        proxy_set_header   X-Real-IP         $remote_addr;
        proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto $scheme;

        # How long to wait for the upstream to accept the connection
        proxy_connect_timeout 5s;
        # How long to wait for the upstream to send response headers
        proxy_read_timeout    60s;
        # How long to wait for the upstream to receive the request body
        proxy_send_timeout    60s;
    }
}

Upstream Server Parameters

Each server directive inside upstream accepts optional parameters that give you fine-grained control over traffic distribution and failure handling:

weight=N — send N times more requests to this server than weight-1 servers. Use for heterogeneous hardware.
max_fails=N — mark the server unavailable after N consecutive failures within the fail_timeout window (default: 1 fail, 10 s window).
fail_timeout=Xs — how long the server stays marked unavailable before Nginx retries it.
backup — only use this server when all primary servers are down. Classic hot-standby pattern.
down — permanently mark this server as offline (useful during rolling deploys without editing the live config).

upstream app_servers {
    server 10.0.1.10:8000 weight=3;             # primary, high-spec host
    server 10.0.1.11:8000 weight=1;             # primary, lower-spec host
    server 10.0.1.12:8000 backup;               # only used if both primaries fail
    server 10.0.1.13:8000 max_fails=3 fail_timeout=30s;  # custom failure window

    keepalive 32;
}

Production tip — upstream keepalive is not optional at scale: Without keepalive, Nginx opens a brand-new TCP connection to the upstream for every proxied request. At 1000 req/s that is 1000 TCP handshakes per second: latency spikes and file-descriptor exhaustion follow. Set keepalive 32 (or higher based on your concurrency) and always pair it with proxy_http_version 1.1 and proxy_set_header Connection "" — the latter clears the Connection: close header that HTTP/1.0 sends by default, which would otherwise close the connection after every response.

WebSocket Proxying

WebSockets start as an HTTP/1.1 connection and then perform an upgrade handshake — the client sends Connection: Upgrade and Upgrade: websocket, and the server responds with 101 Switching Protocols. After that, the connection becomes a persistent, full-duplex TCP channel that lives for minutes or hours.

By default Nginx is a request-response proxy. It drops the Upgrade header and closes the connection, breaking WebSockets completely. You must explicitly map and forward the upgrade headers:

http {
    # map lets Nginx conditionally set the Upgrade header.
    # If the client sends an Upgrade header, pass it through; otherwise send empty string.
    map $http_upgrade $connection_upgrade {
        default   upgrade;
        ""        close;
    }

    server {
        listen 80;
        server_name ws.example.com;

        location /ws/ {
            proxy_pass          http://websocket_backend;

            # The two headers that make WebSocket upgrades work
            proxy_http_version  1.1;
            proxy_set_header    Upgrade    $http_upgrade;
            proxy_set_header    Connection $connection_upgrade;

            # WebSocket connections are long-lived — raise the read timeout
            # 0 = no timeout (use with caution; prefer a large value like 1h)
            proxy_read_timeout  3600s;

            proxy_set_header    Host             $host;
            proxy_set_header    X-Real-IP        $remote_addr;
            proxy_set_header    X-Forwarded-For  $proxy_add_x_forwarded_for;
        }
    }

    upstream websocket_backend {
        # For WebSockets, ip_hash keeps a client pinned to the same server
        # (important if the app stores session state in-process)
        ip_hash;
        server 10.0.1.20:3000;
        server 10.0.1.21:3000;
    }
}

Sticky sessions vs stateless design: ip_hash pins each client IP to the same upstream for the duration of the connection. This solves in-process WebSocket state but breaks geographic load balancing and fails over poorly (if a node dies, all its pinned clients reconnect and may hit a cold server). The production-grade solution is to externalize session state into Redis (socket.io adapter, Action Cable, etc.) so any upstream can handle any connection — then use plain round-robin.

Combining Static and Proxied Locations

A complete production server block typically combines both patterns: Nginx serves static assets at wire speed and proxies everything else to the application. This is the pattern you will see in virtually every Laravel, Rails, Django, and Next.js deployment guide:

Static assets (CSS, JS, images): served directly by Nginx with long cache headers.
API and page routes: proxied to the upstream pool.
WebSocket endpoint (/ws/): proxied with upgrade headers and long read timeout.
Health check endpoint: proxied to the app or answered directly by Nginx.

This separation means your app servers spend zero CPU on files that never change — a meaningful difference at scale when static assets account for 80–90 % of request volume.