Nginx Tuning for High Traffic

Size worker_connections for two sockets per request and keep a right-sized upstream keepalive pool, so nginx stops dropping connections at peak.

Published

Applies to nginx 1.18 and later; upstream keepalive is on by default from 1.29.7

A stock nginx reverse proxy runs out of two things in a traffic spike: connection slots per worker, and local ports for upstream connections. You size both for your own peak below, then prove the change on staging before you ship it. The kernel side of the same problem lives in Linux network tuning for web servers.

What breaks at peak

Worker connections run out. worker_connections caps the open connections per worker process. The cap covers client connections and upstream connections together, so each proxied request holds two slots. Idle upstream keepalive connections hold slots too. The official image ships with 1024 per worker.

Past the cap, nginx fails in two places. On the accept side it logs worker_connections are not enough and closes the new client socket, so the client sees a reset. On the upstream side it logs the line below and answers the client with a 500:

1024 worker_connections are not enough while connecting to upstream

Upstream connections churn. Before nginx 1.29.7, which covers 1.28 stable and older, the proxy speaks HTTP/1.0 to the upstream and sends Connection: close. Each request opens a new TCP connection to your app and closes it. Usually the backend closes first and holds TIME_WAIT. When nginx finishes reading the body before the backend’s FIN arrives, nginx closes first; on a busy proxy without keepalive that’s the usual route to error 99.

A small keepalive pool exhausts ports on the proxy. Upstream keepalive N caps the idle connections each worker caches. It doesn’t cap the total. If more requests are in flight than the pool holds, nginx opens extra connections. Once those responses finish, nginx closes the extras from its own side and holds TIME_WAIT for 60 seconds per socket. At a high request rate the proxy runs out of ephemeral ports. nginx logs this at [crit] and returns 502 Bad Gateway, or the request hangs until a timeout:

connect() to 10.0.3.7:8080 failed (99: Cannot assign requested address) while connecting to upstream

That’s the glibc wording, which Debian and Ubuntu builds print. Alpine images use musl and print (99: Address not available) instead. Search your log for '\(99: ' to catch both.

nginx 1.29.7 changed the defaults. It speaks HTTP/1.1 to upstreams, stops sending the Connection header and applies keepalive 32 local; to each upstream block. That ends the churn. A pool of 32 is small for a busy proxy, though, so an upgrade can drop you into the small-pool trap. Run nginx -v to see which defaults you have.

The nginx config for high traffic

The config comes in two files. The first is a full nginx.conf. It keeps the include lines that Debian and Ubuntu packages rely on, so conf.d/*.conf, sites-enabled/* and dynamic modules still load. If your nginx.conf carries other changes, merge the changed lines instead of replacing the file.

# /etc/nginx/nginx.conf
user  nginx;                     # www-data on Debian and Ubuntu packages
worker_processes  4;             # match the CPUs nginx may use
worker_rlimit_nofile  65535;

error_log  /var/log/nginx/error.log  warn;
pid        /run/nginx.pid;

include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections  16384;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    access_log  /var/log/nginx/access.log  combined  buffer=64k  flush=5s;

    sendfile            on;
    tcp_nopush          on;
    keepalive_timeout   30s;
    keepalive_requests  10000;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

The second file holds the upstream and the proxy. Replace 127.0.0.1:8080 with your app’s address.

# /etc/nginx/conf.d/app.conf
upstream app {
    server 127.0.0.1:8080;       # your app
    keepalive 64;                # idle connections per worker; size it below
    keepalive_requests 10000;
    keepalive_timeout 30s;       # keep below the backend's idle timeout
}

server {
    listen 80 backlog=4096 reuseport;

    location / {
        proxy_pass http://app;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Both files pass nginx -t on the official 1.27, 1.30 stable and 1.31 mainline images, and on Ubuntu’s packaged nginx once you change user nginx; to user www-data;. Remove the stock default site, conf.d/default.conf on the official image or sites-enabled/default on Debian and Ubuntu. Otherwise it keeps answering on port 80.

Workers. Set worker_processes to the CPUs nginx can use. Under a container CPU limit, auto counts the host’s cores, so a pod limited to 2 CPUs on a 16-core node starts 16 workers. Use auto on a VM where nginx owns the box.

Worker connections. Size worker_connections to at least 2 × peak concurrent clients per worker, plus the keepalive pool. With 20,000 clients across 4 workers, that’s 5,000 per worker, so 10,000 plus the pool. Round up, since an unused slot costs little memory.

File descriptors. Each connection uses a file descriptor, and so do log files. Set worker_rlimit_nofile above worker_connections. The kernel’s fs.nr_open still caps it, and the Linux recipe covers that limit.

Access log buffering. buffer=64k flush=5s batches log writes instead of one write() per request. The combined format is built in, so you don’t need a log_format line.

Client keepalive. keepalive_timeout 30s frees idle client connections sooner. The compiled default is 75 seconds, and the packaged nginx.conf sets 65. keepalive_requests 10000 lets a busy client reuse one connection for longer.

Upstream keepalive. proxy_http_version 1.1 and proxy_set_header Connection "" turn on connection reuse to the upstream. On 1.29.7 and later they match the defaults, and keeping them makes the config safe on older builds.

Pool size. Size keepalive from your traffic:

in-flight upstream requests per worker ≈ requests per second × upstream latency (s) ÷ workers

At 4,000 requests a second, 50 ms upstream latency and 4 workers, that’s 4,000 × 0.05 ÷ 4 = 50. Set the pool a bit above your peak figure, such as 64. I drop the local flag from the 1.29.7 default so every location that proxies to app draws from one shared pool, which is what this sizing assumes.

I size the pool against the backend before I size it against traffic. The pool can hold workers × keepalive connections open to your app, and each app server takes only so many. Puma has its thread count and Gunicorn’s gthread workers have theirs. Gunicorn’s sync workers don’t support keep-alive at all, so a pool in front of them gains nothing. A pool of thousands per worker can swamp a backend that serves a few hundred at once.

Idle timeouts. Keep the upstream keepalive_timeout below the backend’s own idle timeout. On current Node.js releases, server.keepAliveTimeout defaults to 5 seconds. If the backend closes a connection as nginx reuses it, nginx logs one of these:

upstream prematurely closed connection while reading response header from upstream
recv() failed (104: Connection reset by peer) while reading response header from upstream

nginx retries an idempotent request on another connection after that failure. It won’t retry a POST or PATCH, so the 502s cluster on writes. Either raise the backend’s timeout above nginx’s or set nginx’s below it. The Node.js cluster mode recipe sets the Node side.

Listen backlog. backlog=4096 lets the kernel queue more connections waiting for a worker to accept them during a burst. nginx’s default on Linux is 511. The kernel caps it at net.core.somaxconn, which is 4096 on kernel 5.4 and later and 128 before that.

reuseport. reuseport gives each worker its own listening socket, so the kernel spreads new connections across workers. Set it on one listen per address and port.

What else to tune. Each of these changes behavior under load, so test them one at a time:

  • gzip trades proxy CPU for bandwidth; set gzip_comp_level low and skip types that are already compressed.
  • open_file_cache saves open() and stat() calls when nginx serves static files itself.
  • ssl_session_cache and HTTP/2 cut handshakes when nginx terminates TLS.
  • proxy_buffers and proxy_buffer_size decide whether large responses spill to temp files on disk.
  • limit_req sheds abusive clients before they reach your app.
  • proxy_cache serves repeat responses without touching the upstream at all.

ingress-nginx. The Kubernetes ingress-nginx controller sets the same limits through its ConfigMap: max-worker-connections, upstream-keepalive-connections and upstream-keepalive-timeout. The Kubernetes project retired ingress-nginx in March 2026, so it gets no more releases or security fixes. If you still run it, plan a move to a Gateway API controller.

Test it on your system

Point the test at a staging copy with production-like instance sizes and the same nginx version. If TLS terminates on a load balancer in front of nginx, test nginx itself over plain HTTP on port 80 from inside the network. Run the old config and the new one under the same load, at your expected peak and then above it.

wrk holds a fixed number of open connections. Raise the load generator’s own file limit first, or wrk fails before nginx does:

ulimit -n 65535
wrk -t8 -c8000 -d120s --latency http://nginx-staging.internal/your-busiest-endpoint

wrk and k6’s default executor are closed-loop tools. As the server slows down, they send fewer requests and the latency figures look better than reality. Failed requests that return fast also pull the percentiles down. For p99, hold a fixed arrival rate with wrk2 -R or k6’s constant-arrival-rate executor:

import http from 'k6/http';

export const options = {
  scenarios: {
    peak: {
      executor: 'constant-arrival-rate',
      rate: 4000, timeUnit: '1s', duration: '5m',
      preAllocatedVUs: 2000, maxVUs: 10000,
    },
  },
  thresholds: { http_req_failed: ['rate<0.001'], http_req_duration: ['p(99)<500'] },
};

export default function () {
  http.get('http://nginx-staging.internal/your-busiest-endpoint');
}

Set rate to your own peak and the thresholds to your own targets. I watch the error log before any dashboard, because these lines name the limit you hit:

tail -f /var/log/nginx/error.log | grep -E 'worker_connections are not enough|\(99: |prematurely closed|\(104: '
watch -n2 'ss -Htan state time-wait | wc -l'

Count 500, 502 and 504 responses in the access log or your load tool as well. Decide your targets before the run. The new config passes when the error log stays free of those lines and 5xx rates sit inside your error budget. TIME_WAIT on the proxy should level off well below the size of ip_local_port_range. At a fixed arrival rate, p99 should land under your latency target.

If TIME_WAIT keeps climbing toward the port range, the pool is too small for the load. I cover how I pick the arrival rate and latency budget in Find your p99 latency limits before launch day.

Keep it healthy

Expose stub_status on a local port and scrape it with nginx-prometheus-exporter. Save this as /etc/nginx/conf.d/status.conf, which loads inside the http {} block:

server {
    listen 127.0.0.1:8081;
    location = /stub_status { stub_status; }
}

nginx_connections_active counts client connections only. A proxied request uses two worker slots, so the alert threshold is workers × worker_connections ÷ 2 × 0.8. For 4 workers at 16,384, that’s 26,214.

groups:
  - name: nginx-capacity
    rules:
      - alert: NginxDroppingConnections
        expr: rate(nginx_connections_accepted[5m]) - rate(nginx_connections_handled[5m]) > 0
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "nginx on {{ $labels.instance }} is dropping client connections"
          description: "A worker ran out of slots on the accept side. Check the error log for worker_connections lines, then raise worker_connections or add workers."
      - alert: NginxConnectionsNearLimit
        # 4 workers x 16384 worker_connections / 2 slots per proxied request x 80%
        expr: nginx_connections_active > 26214
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "nginx on {{ $labels.instance }} is near its worker_connections limit"
          description: "Client connections passed 80% of what the workers can proxy. Raise worker_connections and worker_rlimit_nofile, or add capacity, before the next peak."

Accepted minus handled stays at zero until a worker runs out of slots on the accept side. It misses slot exhaustion while connecting upstream, which shows only as 500s and a log line. Alert on these lines in Loki or your log tool to cover that case and the port and timeout failures:

worker_connections are not enough while connecting to upstream
(99: Cannot assign requested address) while connecting to upstream
(99: Address not available) while connecting to upstream
upstream prematurely closed connection

The TIME_WAIT, listen queue and port alerts sit in the Linux recipe.

Roll back

Copy both files before you change anything:

cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak
cp /etc/nginx/conf.d/app.conf /etc/nginx/conf.d/app.conf.bak 2>/dev/null

To roll back, restore the copies, test them, then reload:

cp /etc/nginx/nginx.conf.bak /etc/nginx/nginx.conf
cp /etc/nginx/conf.d/app.conf.bak /etc/nginx/conf.d/app.conf 2>/dev/null
nginx -t && nginx -s reload

A reload starts new workers on the restored config and lets the old workers finish their open requests. If the new config is causing errors, such as backend 502s after a timeout change, reload the old one now. Any other revert can wait for a quiet hour, because the old config hits the same limits at the same load. When one setting misbehaves, revert that line alone. A wrong pool size or idle timeout tends to show up at the next traffic peak.

More recipes