HTTP & HTTPS

HTTP & HTTPS

HTTP (HyperText Transfer Protocol) is the application-layer protocol that powers virtually every client-server interaction on the web. Every API call, every page load, every microservice request travels over HTTP. As a system designer, you need to understand its structure deeply — not as trivia, but because every performance, caching, and security decision you make depends on it.

The Request / Response Model

HTTP is a stateless, text-based, request–response protocol. A client sends a request; a server sends exactly one response. The connection carries no memory of previous exchanges — any state must be managed explicitly (cookies, tokens, server-side sessions).

Every HTTP request has three parts:

1. Request line — method, path, and protocol version (e.g. GET /api/users HTTP/1.1)
2. Headers — metadata key-value pairs (host, content type, auth token, caching hints, etc.)
3. Body — optional payload (present for POST, PUT, PATCH; absent for GET, DELETE)

Every HTTP response mirrors this structure: a status line, headers, and an optional body.

HTTP Methods — Semantics Matter

HTTP defines a small vocabulary of methods (verbs). Using the right verb is not just convention — it controls caching, proxying, browser behaviour, and idempotency guarantees that matter at scale.

• GET — Retrieve a resource. Safe (no side effects) and idempotent (repeat calls return the same result). Responses may be cached by browsers, CDNs, and reverse proxies.
• POST — Submit data to create a resource or trigger an action. Neither safe nor idempotent — duplicate calls can create duplicate records. Never cached by default.
• PUT — Replace a resource in full. Idempotent — calling it twice has the same effect as once. The client sends the complete representation.
• PATCH — Partially update a resource. Idempotent in practice (when applied correctly). Useful when payloads are large and only a field or two changes.
• DELETE — Remove a resource. Idempotent — deleting something twice should not be an error.
• HEAD — Like GET but returns only headers, no body. Used to check whether a resource exists or has changed (ETags, Last-Modified) without downloading the full body.
• OPTIONS — Ask the server what methods and headers it accepts. CORS preflight requests use OPTIONS automatically.

Status Codes — The Server's Answer

Status codes are three-digit numbers grouped by the hundreds. Every system designer should know the most important ones cold, because they drive how clients, load balancers, and monitoring systems behave.

• 1xx — Informational: 100 Continue (keep sending the body), 101 Switching Protocols (WebSocket upgrade).
• 2xx — Success:
  • 200 OK — Generic success with a body.
  • 201 Created — Resource was created; the Location header should point to it.
  • 204 No Content — Success but no body (common for DELETE).
• 3xx — Redirection:
  • 301 Moved Permanently — Cached forever by clients; use for URL migrations.
  • 302 Found — Temporary redirect; clients re-check each time.
  • 304 Not Modified — The resource has not changed since the client's cached copy (ETags / Last-Modified); the body is omitted entirely, saving bandwidth.
• 4xx — Client Error:
  • 400 Bad Request — Malformed input.
  • 401 Unauthorized — Not authenticated.
  • 403 Forbidden — Authenticated but not authorized.
  • 404 Not Found — Resource does not exist.
  • 409 Conflict — State conflict (e.g. duplicate creation).
  • 429 Too Many Requests — Rate limited. Should include a Retry-After header.
• 5xx — Server Error:
  • 500 Internal Server Error — Generic unhandled exception.
  • 502 Bad Gateway — The upstream returned an invalid response (common when a backend pod crashes).
  • 503 Service Unavailable — Server is overloaded or in maintenance; a load balancer removes it from rotation.
  • 504 Gateway Timeout — The upstream did not respond in time.

Key Headers for System Designers

Headers carry the metadata that makes HTTP powerful beyond simple data transfer. The most important ones for system design:

• Content-Type / Accept — Negotiate the format (JSON, protobuf, XML). Set these correctly or your reverse proxy may not parse the body.
• Authorization — Carries bearer tokens (JWTs), API keys, or Basic auth. Never log this header.
• Cache-Control — Directives like max-age=3600, no-store, public, private control whether CDNs and browsers cache the response.
• ETag / If-None-Match — Conditional requests: the server provides a version token; the client sends it back; a 304 saves bandwidth if unchanged.
• X-Request-ID / Trace-ID — Correlation IDs for distributed tracing. Generate one at the edge and pass it through every downstream call.
• Retry-After — Tells clients when to retry after a 429 or 503. Prevents thundering herds after outages.

What HTTPS Adds: TLS

HTTPS is HTTP carried over TLS (Transport Layer Security). TLS adds three guarantees that plain HTTP entirely lacks:

1. Encryption — All data is encrypted in transit. An attacker on the same Wi-Fi or ISP network cannot read the request or response body, headers, or URL path (though the domain name is visible in TLS SNI).
2. Authentication — The server presents an X.509 certificate signed by a Certificate Authority (CA) that the client trusts. This proves the server is who it claims to be, preventing DNS-spoofing and MITM attacks.
3. Integrity — A Message Authentication Code (MAC) on every TLS record ensures that data cannot be tampered with in transit without detection.

TLS Overhead — What It Actually Costs

A common concern is TLS performance overhead. In practice, TLS 1.3 (2018) reduced this to near zero for established connections:

• Handshake cost: TLS 1.3 completes in one round-trip (1-RTT) for new connections, versus two RTTs for TLS 1.2. Session resumption with 0-RTT is possible for repeated connections (though it has replay-attack caveats).
• CPU cost: Modern CPUs include AES-NI hardware acceleration. Nginx on commodity hardware can handle ~10,000 TLS handshakes per second and the symmetric encryption overhead on data transfer is negligible.
• Latency: The handshake adds ~1 network RTT. On a 50 ms RTT link, that is 50 ms of extra latency for the very first request to a new server. HTTP/2 and persistent connections amortise this across many requests.

HTTP/1.1 vs HTTP/2 vs HTTP/3

The version of HTTP in use has a large impact on latency and throughput:

• HTTP/1.1 — One request per connection at a time (head-of-line blocking). Workaround: browsers open 6 parallel connections per origin. Keep-Alive reuses connections, but pipelining never worked well in practice.
• HTTP/2 — Multiplexes multiple streams over a single TCP connection, with header compression (HPACK) and server push. Eliminates HTTP-level head-of-line blocking, but TCP-level blocking still occurs on packet loss. Default in most modern load balancers today.
• HTTP/3 — Runs over QUIC (UDP-based) instead of TCP. Eliminates TCP head-of-line blocking entirely. Especially beneficial on lossy connections (mobile). Requires UDP to be open through firewalls. Increasingly deployed at the edge (Cloudflare, Google).

Putting It Together: HTTPS in a Real Architecture

In a production system, HTTPS works as follows: a client makes a request to your domain. DNS resolves to a CDN or load balancer IP. The CDN/LB terminates TLS — it holds your certificate and private key. It then forwards the decrypted HTTP request to your backend over an internal, trusted network (often using HTTP/2). The backend sends a response back; the LB re-encrypts it (or sends it as-is if you use end-to-end TLS) and returns it to the client. Certificate rotation, OCSP stapling, and HSTS (HTTP Strict Transport Security) headers ensure the chain of trust is maintained continuously.

Understanding this stack — from the raw request line, through headers, status codes, TLS, and version negotiation — is the foundation for every other networking concept in this course: API gateways, WebSockets, and gRPC all build directly on top of it.