Load balancing and upstreams

Upstream blocks, round-robin, least_conn and ip_hash, weights, backup servers, passive and active health checks, and draining a node without dropping requests.

Upstream blocks and algorithms

upstream app {
    least_conn;                       # send to the node with the fewest active requests
    server 10.0.0.11:8080 weight=3;
    server 10.0.0.12:8080;
    server 10.0.0.13:8080 max_fails=2 fail_timeout=10s;
    server 10.0.0.14:8080 backup;     # only used when the others are unavailable
    keepalive 32;                     # pooled connections to the upstreams
}

server {
    listen 443 ssl;
    server_name api.example.com;

    location / {
        proxy_pass http://app;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_connect_timeout 3s;
        proxy_next_upstream error timeout http_502 http_503;
    }
}

proxy_next_upstream retries on another node, but only for requests nginx knows are safe to repeat. A non-idempotent POST is not retried by default, and configuring it to be is a decision about duplicate writes, not a performance tweak.

Health checks

# passive: nginx observes real traffic and ejects a node after failures
upstream app {
    server 10.0.0.11:8080 max_fails=3 fail_timeout=30s;
    server 10.0.0.12:8080 max_fails=3 fail_timeout=30s;
}

# active: nginx actively probes every node, available in nginx Plus
upstream api {
    zone api 64k;
    server 10.0.0.11:8080;
    server 10.0.0.12:8080;
    health_check interval=5s fails=2 passes=2 uri=/health match=healthy;
}
match healthy {
    status 200;
    body ~ "ok";
}

• Passive checks need proxy_next_upstream to be useful; without it a failed node produces errors instead of traffic moving elsewhere.
• After fail_timeout nginx tries the node again with a single request. A node that is still down counts as failing again, so recovery costs only one request in 30 seconds.
• proxy_connect_timeout should be short — a few seconds. Long connect timeouts turn a dead node into slow responses rather than fast failures.
• Health endpoints should check downstream dependencies, not just return 200 unconditionally.

Deploying and draining safely

# drain: keep serving existing requests but send new ones elsewhere.
# Mark the node down and reload; in-flight requests are not interrupted by a reload.
upstream app {
    server 10.0.0.11:8080 down;
    server 10.0.0.12:8080;
}

# then, after the connections have drained, deploy that node
# nginx -s reload   -> graceful handover, old workers finish their requests

FAQ

Related

TLS hardening and certificates with Let's Encrypt Performance tuning: workers, buffers and timeouts

Last refreshed 2026-09-18.