A Node.js process runs your JavaScript on one thread, so it uses about one CPU core. Garbage collection and libuv’s thread pool add a little on top. Put that process in a 4-CPU container and most of the CPU you pay for sits idle while requests queue. Pick where the extra processes live first, then set them up so a crash or a deploy doesn’t drop requests.
What breaks at peak
The event loop runs one callback at a time. When requests arrive faster than one core can serve them, they wait inside the process. Event loop lag climbs first, p99 latency follows, and then clients start to time out.
Your autoscaler can miss it. A Horizontal Pod Autoscaler divides CPU usage by the pod’s CPU request, not its limit. A pod that requests 4 CPUs and runs one busy Node process reads about 25%. With a 70% target, the HPA sees a quiet pod and keeps the replica count. On your dashboards you’ll find high event loop lag next to a container that uses one core of four.
CPU-heavy endpoints gain the most from more processes: JSON serialization, server-side rendering, crypto and image work. Endpoints that wait on a database or another API gain less, because the queue sits in that dependency.
The Node.js cluster setup
Decide where the extra processes live before you write any code.
- On Kubernetes, prefer one process per pod. Set a CPU request of 1 and let the HPA scale the pod count. The kubelet sees every crash, probes check the process that serves traffic, and the CPU percentage means what the HPA assumes. This is my default, and I only reach for cluster mode when per-pod overhead starts to hurt.
- Use
node:clusteron VMs, or when per-pod overhead makes many small pods costly. Sidecars, agents and per-pod connection pools add up. PM2 does the same job withpm2 start app.js -i 4, orpm2-runtimeinside a container. Pass a number, because-i maxcan count the host’s cores. - Skip cluster under a CPU limit below 2. A pod with a
500mlimit gets half a core of CPU time whatever the worker count. Extra workers add memory and context switches and no throughput. - Use worker threads for one CPU-heavy path. If one endpoint renders PDFs or resizes images, move that work to
worker_threads. A pool such as piscina manages the threads and keeps the event loop free.
Whichever you pick, give the CPU limit some room. GC and libuv threads burst above one core for short spells, and a hard limit turns each burst into throttling. For one process per pod I set a request of 1 and a limit slightly above it, such as 1200m, then watch nr_throttled under load.
Check what your worker count reads. I run this check first on any cluster setup, because a wrong worker count undoes everything else. os.availableParallelism() reads the cgroup CPU quota from Node 22.12, 23.1 and 24.0, which ship libuv 1.49 or later. It rounds down, so a 1500m limit gives 1. A limit below one CPU reads as 1 from Node 24.16, with libuv 1.52, and older versions fall back to the host’s core count. Node 20, and os.cpus().length on any version, return every core on the host. With a request and no limit there’s no quota to read, so you get the node’s full core count. Check inside the running container:
kubectl exec deploy/api -- node -p "require('node:os').availableParallelism()"
kubectl exec deploy/api -- cat /sys/fs/cgroup/cpu.max
# on cgroup v1 nodes, read these two files instead
kubectl exec deploy/api -- cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us /sys/fs/cgroup/cpu/cpu.cfs_period_us
On cgroup v2, cpu.max prints the quota and the period, such as 400000 100000 for 4 CPUs, and max 100000 means no limit. On cgroup v1, divide cpu.cfs_quota_us by cpu.cfs_period_us, where -1 means no limit. If Node’s answer and the quota disagree, set WORKERS yourself.
The primary. cluster.js forks the workers and gives each one a stable index. It re-forks a crashed worker into the same slot with backoff, serves one /metrics for the whole pod and drains workers on SIGTERM. It needs prom-client:
// cluster.js
const cluster = require('node:cluster');
const http = require('node:http');
const os = require('node:os');
const client = require('prom-client');
const WORKERS = Number(process.env.WORKERS) || os.availableParallelism();
const GRACE_MS = Number(process.env.SHUTDOWN_GRACE_MS) || 20000;
const METRICS_PORT = Number(process.env.METRICS_PORT) || 9100;
// Create it in the primary and in every worker: workers answer the primary's
// metrics requests through the listeners it sets up.
const aggregator = new client.AggregatorRegistry();
if (cluster.isPrimary) {
let stopping = false;
let restartDelay = 1000;
let lastCrash = 0;
// Live workers and their stable index 0..WORKERS-1. cluster.workers drops a
// worker as soon as disconnect() starts, so track them here instead.
const slots = new Map();
const reforks = new client.Counter({
name: 'nodejs_cluster_worker_reforks_total',
help: 'Workers the primary re-forked after a crash',
});
const fork = (index) => slots.set(cluster.fork({ WORKER_INDEX: String(index) }), index);
console.log(`primary ${process.pid}: starting ${WORKERS} workers ` +
`(availableParallelism=${os.availableParallelism()})`);
for (let i = 0; i < WORKERS; i++) fork(i);
cluster.on('exit', (worker, code, signal) => {
const index = slots.get(worker);
slots.delete(worker);
if (stopping) {
if (slots.size === 0) process.exit(0);
return;
}
if (worker.exitedAfterDisconnect) return; // a planned stop, not a crash
const now = Date.now();
restartDelay = now - lastCrash > 60000 ? 1000 : Math.min(restartDelay * 2, 30000);
lastCrash = now;
reforks.inc();
console.error(`worker ${index} (pid ${worker.process.pid}) died ` +
`(${signal || code}), restarting in ${restartDelay} ms`);
setTimeout(() => { if (!stopping) fork(index); }, restartDelay);
});
// One /metrics for the whole pod: every worker's series plus the primary's counter.
http.createServer(async (req, res) => {
try {
const workers = await aggregator.clusterMetrics();
const primary = await client.register.metrics();
res.writeHead(200, { 'Content-Type': aggregator.contentType });
res.end(workers + '\n' + primary);
} catch (err) {
res.writeHead(500);
res.end(err.message);
}
}).listen(METRICS_PORT);
const shutdown = () => {
if (stopping) return;
stopping = true;
if (slots.size === 0) process.exit(0);
for (const w of slots.keys()) w.disconnect(); // stop new connections, drain the rest
setTimeout(() => {
for (const w of slots.keys()) w.process.kill('SIGKILL');
process.exit(1);
}, GRACE_MS).unref();
};
process.on('SIGTERM', shutdown);
process.on('SIGINT', shutdown);
} else {
require('./server.js');
}
The worker. server.js runs inside each worker. It also runs alone with node server.js, and then it serves /metrics on the same port:
// server.js
const cluster = require('node:cluster');
const http = require('node:http');
const client = require('prom-client');
const app = require('./app'); // your request handler, e.g. an Express app
// Index 0..WORKERS-1 from cluster.js, or 0 when you run `node server.js` alone.
client.register.setDefaultLabels({ worker: process.env.WORKER_INDEX ?? '0' });
client.collectDefaultMetrics();
const server = http.createServer(app);
server.listen(Number(process.env.PORT) || 3000);
if (cluster.isWorker) {
// The primary drives shutdown. Ignore signals sent to the whole process
// group (tini -g, Ctrl-C), or they kill this worker mid-request.
process.on('SIGTERM', () => {});
process.on('SIGINT', () => {});
// The primary's disconnect() stops new connections to this worker.
// server.close() waits for in-flight requests. Close DB pools, then exit.
cluster.worker.on('disconnect', () => server.close(() => process.exit(0)));
} else {
// Standalone: serve /metrics on the same port the primary would use.
const metrics = http.createServer(async (req, res) => {
res.writeHead(200, { 'Content-Type': client.register.contentType });
res.end(await client.register.metrics());
}).listen(Number(process.env.METRICS_PORT) || 9100);
process.on('SIGTERM', () => {
metrics.close();
server.close(() => process.exit(0));
});
}
- The
workerlabel comes fromWORKER_INDEX, a slot number from 0 toWORKERS - 1. A re-forked worker takes its old slot, so your series keep their names.cluster.worker.idgrows on every fork and doesn’t exist when you runserver.jsalone. AggregatorRegistryaverages event loop lag across workers that share the same labels. Without theworkerlabel, one stuck worker hides behind three idle ones.- A crashed worker comes back after 1 second. If crashes keep coming within a minute, the delay doubles up to 30 seconds.
nodejs_cluster_worker_reforks_totalcounts each re-fork, because the pod’s restart counter stays flat. - Run
node cluster.jsas the container command, notnpm start, because npm can swallow SIGTERM. Node as PID 1 handles SIGTERM here, but it won’t reap orphaned grandchildren. Put tini in the image as the entrypoint, or useinit: truein Docker Compose.
Fit the drain inside the pod’s grace period. Kubernetes counts the preStop hook and your drain against terminationGracePeriodSeconds. Keep the preStop sleep plus SHUTDOWN_GRACE_MS below it, or the kubelet kills the pod mid-drain. The sleep gives the load balancer time to stop sending new requests. Changed lines only, in the Deployment’s pod spec:
terminationGracePeriodSeconds: 30
containers:
- name: app
command: ["node", "cluster.js"]
env:
- name: WORKERS
value: "4"
- name: SHUTDOWN_GRACE_MS
value: "20000" # 5 s preStop + 20 s drain stays under 30 s
- name: NODE_OPTIONS
value: "--max-old-space-size=384"
ports:
- name: metrics
containerPort: 9100
resources:
requests:
cpu: "4"
memory: 2Gi
limits:
cpu: "4500m" # availableParallelism() still reads 4
memory: 2Gi
lifecycle:
preStop:
exec:
command: ["sleep", "5"]
Cap the heap per worker. Each worker is a full Node process with its own V8 heap. V8 sizes the default heap limit from the memory it sees, which is the container limit on recent Node. On the versions I checked, Node 20 to 24, that comes out near half the limit, with a cap around 2 GB. Four workers can each plan to use half the pod. Check the real figure inside your pod:
kubectl exec deploy/api -- node -p "v8.getHeapStatistics().heap_size_limit / 1048576"
Then set the old-space limit per worker:
--max-old-space-size ≈ (container memory limit − overhead) ÷ WORKERS
Overhead covers the primary, each worker’s young generation, Buffers and native memory outside the heap. Start with about a quarter of the limit and adjust from what you measure. Set it through NODE_OPTIONS, which every worker inherits. The example above fits the rule: 2 GiB, a quarter held back, four workers at 384 MB each.
Watch how connections spread. Cluster hands each new connection to a worker in turn. Every request on that connection then stays with that worker. A load balancer or an nginx upstream pool often holds a few long keepalive or HTTP/2 connections. Those few connections can pin most traffic to one or two workers. Check the per-worker lag under load. If it’s uneven, cap requests per upstream connection with nginx’s keepalive_requests, as in the nginx high-traffic recipe.
Plan for Socket.IO. Long-polling sends several HTTP requests per session, and they all need the same worker. You can set clients to the websocket transport only, which removes the need for sticky sessions. Broadcasts and rooms still span workers, so you need @socket.io/cluster-adapter either way. Keep long-polling and you also need @socket.io/sticky.
Move shared state out of the process. In-process sessions, caches and rate-limit counters live in one worker’s memory. A user’s session exists in one worker. Each worker counts its own rate limit, so the real limit multiplies by WORKERS. Move that state to Redis, and size Redis with the Redis maxmemory and eviction recipe.
Count your database connections. Each worker opens its own pool, so four workers with a pool of 10 hold 40 connections per pod. Multiply by your replica count before you deploy. If the total nears PostgreSQL’s max_connections, put PgBouncer in front of PostgreSQL.
Test it on your system
Run the same load against WORKERS=1 and then WORKERS=N on staging, with the same pod CPU and memory limits as production. Use an open-model tool that sends requests at a fixed arrival rate. Closed-loop tools such as wrk or autocannon wait for each reply before sending the next request. They slow down with the server and hide the queue you’re looking for.
This k6 script holds a fixed rate. Set rate to your peak and the thresholds to your own SLO:
import http from 'k6/http';
export const options = {
scenarios: {
peak: {
executor: 'constant-arrival-rate',
rate: 500, // your peak requests per second
timeUnit: '1s',
duration: '10m',
preAllocatedVUs: 200,
maxVUs: 1000,
},
},
thresholds: {
http_req_failed: ['rate<0.01'],
http_req_duration: ['p(99)<500'],
},
};
export default function () {
http.get(`${__ENV.TARGET}/api/products`);
}
Run it with k6 run -e TARGET=https://staging.example.com peak.js. With wrk2, wrk -t4 -c200 -d10m -R500 --latency <url> does the same job. Point the test at the endpoints your peak traffic hits, not a health check. Send it through the same load balancer or ingress as production, so the connection pattern matches.
While it runs, watch these:
-
Event loop lag per worker. The primary’s
/metricsholds one series per worker. Pull it from inside the pod:kubectl exec deploy/api -- node -e "fetch('http://localhost:9100/metrics').then(r => r.text()).then(console.log)" | grep '^nodejs_eventloop_lag_p99' -
CPU per worker. Run
topinside the container. Every worker should carry load, and none should sit at 100% while the rest idle. -
Throttling.
cat /sys/fs/cgroup/cpu.statshowsnr_periodsandnr_throttled. Compare the two before and after the run. -
Errors and timeouts from k6 and from your app logs.
Halfway through, test the failure paths too. Kill one worker with kill -9 on its PID. The log should show the re-fork, and nodejs_cluster_worker_reforks_total should go up by one. Then roll the Deployment while k6 runs. Your error threshold should hold through both.
Write the pass bar from your SLO before the run. For example, p99 latency and error rate stay under your thresholds at peak rate. Event loop p99 stays under 100 ms on every worker, and throttled periods stay a small share of the total. Then raise the rate in 25% steps until a threshold breaks, so you know your headroom.
Keep it healthy
These rules use prom-client’s default metrics from the primary’s /metrics, plus cAdvisor and kube-state-metrics. They assume your scrape config adds namespace and pod labels and that the container is named app:
groups:
- name: nodejs-workers
rules:
- alert: NodeEventLoopLagHigh
expr: max by (namespace, pod) (nodejs_eventloop_lag_p99_seconds) > 0.1
for: 5m
labels:
severity: warning
annotations:
summary: "Event loop p99 lag above 100 ms on {{ $labels.pod }}"
description: "One worker or more is queuing callbacks. Check CPU per worker and throttling, then look for a slow synchronous path or uneven connection spread."
- alert: NodeBusyOnOneCore
expr: |
(max by (namespace, pod) (nodejs_eventloop_lag_p99_seconds) > 0.1)
and on (namespace, pod)
(sum by (namespace, pod) (rate(container_cpu_usage_seconds_total{container="app"}[5m])) < 1.2)
and on (namespace, pod)
(sum by (namespace, pod) (kube_pod_container_resource_requests{resource="cpu", container="app"}) >= 2)
for: 10m
labels:
severity: warning
annotations:
summary: "{{ $labels.pod }} lags on about one core of a larger CPU request"
description: "The pod runs one busy event loop. Check that WORKERS matches the CPU request and that availableParallelism() reads the cgroup quota."
- alert: NodeCPUThrottled
expr: |
sum by (namespace, pod) (rate(container_cpu_cfs_throttled_periods_total{container="app"}[5m]))
/ sum by (namespace, pod) (rate(container_cpu_cfs_periods_total{container="app"}[5m]))
> 0.25
for: 15m
labels:
severity: warning
annotations:
summary: "{{ $labels.pod }} is CPU-throttled in over 25% of periods"
description: "Workers, GC and libuv threads burst past the CPU limit. Raise the limit a little above the request, or run one worker fewer."
- alert: NodeWorkersReforking
expr: increase(nodejs_cluster_worker_reforks_total[15m]) > 2
for: 1m
labels:
severity: critical
annotations:
summary: "Node workers on {{ $labels.pod }} keep crashing"
description: "The primary re-forked workers more than twice in 15 minutes. Read the worker crash logs; check for heap OOM against --max-old-space-size."
- alert: NodePodRestarting
expr: increase(kube_pod_container_status_restarts_total{container="app"}[30m]) > 2
for: 1m
labels:
severity: warning
annotations:
summary: "{{ $labels.pod }} restarted more than twice in 30 minutes"
description: "Check the last termination reason. OOMKilled means the worker heaps add up past the memory limit; lower --max-old-space-size or WORKERS."
Set the lag and throttling thresholds from your own baseline. NodeBusyOnOneCore is the one I’d keep even after the rollout, because a later image change can drop WORKERS back to one without anyone noticing.
Roll back
Save the current Deployment first with kubectl get deploy api -o yaml > api-before.yaml. Then pick the smallest step back:
- Set
WORKERS=1. The primary runs one worker, which behaves like a single process, and you need no code change. - Change the start command back to
node server.jsto remove cluster mode. The SIGTERM handler inserver.jskeeps shutdown clean, and/metricsstays on the same port. - Run
kubectl rollout undo deploy/apito return to the previous spec in one step.
If workers crash in a loop or pods hit OOMKilled, roll back now. The same goes for a feature that broke because it kept state in memory. Move that state to Redis before you try again. If you’re reverting for any other reason, wait for a quiet hour. Keep any higher CPU or memory limits until the rollback is live, then lower them.
More recipes
- Node.js Event Loop Lag Monitoring with Prometheus
Alert on prom-client event loop lag, loop utilization and failed scrapes, so a blocked Node.js event loop pages you before your health checks start failing.
- Fix JavaScript Heap Out of Memory in Node.js
Size the V8 heap to the container with max-old-space-size, then find what grows at peak, so Node.js stops crashing with JavaScript heap out of memory.
- Fix Socket Hang Up and ECONNRESET Behind a Load Balancer
Keep idle connections open longer on the side that should close them last, so Node.js and the load balancer stop racing to reuse a socket the other has closed.
- Next.js SSR Slow at Peak? Fix It with Caching and ISR
Turn per-request rendering into Next.js ISR caching with a cache that every replica shares, so peak traffic reads cached pages instead of rendering each one.