Use Every CPU Core with Node.js Cluster Mode

Run one Node.js worker per CPU core, as pods or node:cluster workers, so peak traffic spreads across every core instead of queuing on one event loop.

Published

Applies to Node.js 18 and later; availableParallelism() reads cgroup CPU limits from 22.12, 23.1 and 24.0, fractional ones from 24.16

A Node.js process runs your JavaScript on one thread, so it uses about one CPU core. Garbage collection and libuv’s thread pool add a little on top. Put that process in a 4-CPU container and most of the CPU you pay for sits idle while requests queue. Pick where the extra processes live first, then set them up so a crash or a deploy doesn’t drop requests.

What breaks at peak

The event loop runs one callback at a time. When requests arrive faster than one core can serve them, they wait inside the process. Event loop lag climbs first, p99 latency follows, and then clients start to time out.

Your autoscaler can miss it. A Horizontal Pod Autoscaler divides CPU usage by the pod’s CPU request, not its limit. A pod that requests 4 CPUs and runs one busy Node process reads about 25%. With a 70% target, the HPA sees a quiet pod and keeps the replica count. On your dashboards you’ll find high event loop lag next to a container that uses one core of four.

CPU-heavy endpoints gain the most from more processes: JSON serialization, server-side rendering, crypto and image work. Endpoints that wait on a database or another API gain less, because the queue sits in that dependency.

The Node.js cluster setup

Decide where the extra processes live before you write any code.

  • On Kubernetes, prefer one process per pod. Set a CPU request of 1 and let the HPA scale the pod count. The kubelet sees every crash, probes check the process that serves traffic, and the CPU percentage means what the HPA assumes. This is my default, and I only reach for cluster mode when per-pod overhead starts to hurt.
  • Use node:cluster on VMs, or when per-pod overhead makes many small pods costly. Sidecars, agents and per-pod connection pools add up. PM2 does the same job with pm2 start app.js -i 4, or pm2-runtime inside a container. Pass a number, because -i max can count the host’s cores.
  • Skip cluster under a CPU limit below 2. A pod with a 500m limit gets half a core of CPU time whatever the worker count. Extra workers add memory and context switches and no throughput.
  • Use worker threads for one CPU-heavy path. If one endpoint renders PDFs or resizes images, move that work to worker_threads. A pool such as piscina manages the threads and keeps the event loop free.

Whichever you pick, give the CPU limit some room. GC and libuv threads burst above one core for short spells, and a hard limit turns each burst into throttling. For one process per pod I set a request of 1 and a limit slightly above it, such as 1200m, then watch nr_throttled under load.

Check what your worker count reads. I run this check first on any cluster setup, because a wrong worker count undoes everything else. os.availableParallelism() reads the cgroup CPU quota from Node 22.12, 23.1 and 24.0, which ship libuv 1.49 or later. It rounds down, so a 1500m limit gives 1. A limit below one CPU reads as 1 from Node 24.16, with libuv 1.52, and older versions fall back to the host’s core count. Node 20, and os.cpus().length on any version, return every core on the host. With a request and no limit there’s no quota to read, so you get the node’s full core count. Check inside the running container:

kubectl exec deploy/api -- node -p "require('node:os').availableParallelism()"
kubectl exec deploy/api -- cat /sys/fs/cgroup/cpu.max
# on cgroup v1 nodes, read these two files instead
kubectl exec deploy/api -- cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us /sys/fs/cgroup/cpu/cpu.cfs_period_us

On cgroup v2, cpu.max prints the quota and the period, such as 400000 100000 for 4 CPUs, and max 100000 means no limit. On cgroup v1, divide cpu.cfs_quota_us by cpu.cfs_period_us, where -1 means no limit. If Node’s answer and the quota disagree, set WORKERS yourself.

The primary. cluster.js forks the workers and gives each one a stable index. It re-forks a crashed worker into the same slot with backoff, serves one /metrics for the whole pod and drains workers on SIGTERM. It needs prom-client:

// cluster.js
const cluster = require('node:cluster');
const http = require('node:http');
const os = require('node:os');
const client = require('prom-client');

const WORKERS = Number(process.env.WORKERS) || os.availableParallelism();
const GRACE_MS = Number(process.env.SHUTDOWN_GRACE_MS) || 20000;
const METRICS_PORT = Number(process.env.METRICS_PORT) || 9100;

// Create it in the primary and in every worker: workers answer the primary's
// metrics requests through the listeners it sets up.
const aggregator = new client.AggregatorRegistry();

if (cluster.isPrimary) {
  let stopping = false;
  let restartDelay = 1000;
  let lastCrash = 0;
  // Live workers and their stable index 0..WORKERS-1. cluster.workers drops a
  // worker as soon as disconnect() starts, so track them here instead.
  const slots = new Map();

  const reforks = new client.Counter({
    name: 'nodejs_cluster_worker_reforks_total',
    help: 'Workers the primary re-forked after a crash',
  });

  const fork = (index) => slots.set(cluster.fork({ WORKER_INDEX: String(index) }), index);

  console.log(`primary ${process.pid}: starting ${WORKERS} workers ` +
    `(availableParallelism=${os.availableParallelism()})`);
  for (let i = 0; i < WORKERS; i++) fork(i);

  cluster.on('exit', (worker, code, signal) => {
    const index = slots.get(worker);
    slots.delete(worker);
    if (stopping) {
      if (slots.size === 0) process.exit(0);
      return;
    }
    if (worker.exitedAfterDisconnect) return; // a planned stop, not a crash

    const now = Date.now();
    restartDelay = now - lastCrash > 60000 ? 1000 : Math.min(restartDelay * 2, 30000);
    lastCrash = now;
    reforks.inc();
    console.error(`worker ${index} (pid ${worker.process.pid}) died ` +
      `(${signal || code}), restarting in ${restartDelay} ms`);
    setTimeout(() => { if (!stopping) fork(index); }, restartDelay);
  });

  // One /metrics for the whole pod: every worker's series plus the primary's counter.
  http.createServer(async (req, res) => {
    try {
      const workers = await aggregator.clusterMetrics();
      const primary = await client.register.metrics();
      res.writeHead(200, { 'Content-Type': aggregator.contentType });
      res.end(workers + '\n' + primary);
    } catch (err) {
      res.writeHead(500);
      res.end(err.message);
    }
  }).listen(METRICS_PORT);

  const shutdown = () => {
    if (stopping) return;
    stopping = true;
    if (slots.size === 0) process.exit(0);
    for (const w of slots.keys()) w.disconnect(); // stop new connections, drain the rest
    setTimeout(() => {
      for (const w of slots.keys()) w.process.kill('SIGKILL');
      process.exit(1);
    }, GRACE_MS).unref();
  };
  process.on('SIGTERM', shutdown);
  process.on('SIGINT', shutdown);
} else {
  require('./server.js');
}

The worker. server.js runs inside each worker. It also runs alone with node server.js, and then it serves /metrics on the same port:

// server.js
const cluster = require('node:cluster');
const http = require('node:http');
const client = require('prom-client');
const app = require('./app'); // your request handler, e.g. an Express app

// Index 0..WORKERS-1 from cluster.js, or 0 when you run `node server.js` alone.
client.register.setDefaultLabels({ worker: process.env.WORKER_INDEX ?? '0' });
client.collectDefaultMetrics();

const server = http.createServer(app);
server.listen(Number(process.env.PORT) || 3000);

if (cluster.isWorker) {
  // The primary drives shutdown. Ignore signals sent to the whole process
  // group (tini -g, Ctrl-C), or they kill this worker mid-request.
  process.on('SIGTERM', () => {});
  process.on('SIGINT', () => {});
  // The primary's disconnect() stops new connections to this worker.
  // server.close() waits for in-flight requests. Close DB pools, then exit.
  cluster.worker.on('disconnect', () => server.close(() => process.exit(0)));
} else {
  // Standalone: serve /metrics on the same port the primary would use.
  const metrics = http.createServer(async (req, res) => {
    res.writeHead(200, { 'Content-Type': client.register.contentType });
    res.end(await client.register.metrics());
  }).listen(Number(process.env.METRICS_PORT) || 9100);
  process.on('SIGTERM', () => {
    metrics.close();
    server.close(() => process.exit(0));
  });
}
  • The worker label comes from WORKER_INDEX, a slot number from 0 to WORKERS - 1. A re-forked worker takes its old slot, so your series keep their names. cluster.worker.id grows on every fork and doesn’t exist when you run server.js alone.
  • AggregatorRegistry averages event loop lag across workers that share the same labels. Without the worker label, one stuck worker hides behind three idle ones.
  • A crashed worker comes back after 1 second. If crashes keep coming within a minute, the delay doubles up to 30 seconds. nodejs_cluster_worker_reforks_total counts each re-fork, because the pod’s restart counter stays flat.
  • Run node cluster.js as the container command, not npm start, because npm can swallow SIGTERM. Node as PID 1 handles SIGTERM here, but it won’t reap orphaned grandchildren. Put tini in the image as the entrypoint, or use init: true in Docker Compose.

Fit the drain inside the pod’s grace period. Kubernetes counts the preStop hook and your drain against terminationGracePeriodSeconds. Keep the preStop sleep plus SHUTDOWN_GRACE_MS below it, or the kubelet kills the pod mid-drain. The sleep gives the load balancer time to stop sending new requests. Changed lines only, in the Deployment’s pod spec:

terminationGracePeriodSeconds: 30
containers:
  - name: app
    command: ["node", "cluster.js"]
    env:
      - name: WORKERS
        value: "4"
      - name: SHUTDOWN_GRACE_MS
        value: "20000"     # 5 s preStop + 20 s drain stays under 30 s
      - name: NODE_OPTIONS
        value: "--max-old-space-size=384"
    ports:
      - name: metrics
        containerPort: 9100
    resources:
      requests:
        cpu: "4"
        memory: 2Gi
      limits:
        cpu: "4500m"       # availableParallelism() still reads 4
        memory: 2Gi
    lifecycle:
      preStop:
        exec:
          command: ["sleep", "5"]

Cap the heap per worker. Each worker is a full Node process with its own V8 heap. V8 sizes the default heap limit from the memory it sees, which is the container limit on recent Node. On the versions I checked, Node 20 to 24, that comes out near half the limit, with a cap around 2 GB. Four workers can each plan to use half the pod. Check the real figure inside your pod:

kubectl exec deploy/api -- node -p "v8.getHeapStatistics().heap_size_limit / 1048576"

Then set the old-space limit per worker:

--max-old-space-size ≈ (container memory limit − overhead) ÷ WORKERS

Overhead covers the primary, each worker’s young generation, Buffers and native memory outside the heap. Start with about a quarter of the limit and adjust from what you measure. Set it through NODE_OPTIONS, which every worker inherits. The example above fits the rule: 2 GiB, a quarter held back, four workers at 384 MB each.

Watch how connections spread. Cluster hands each new connection to a worker in turn. Every request on that connection then stays with that worker. A load balancer or an nginx upstream pool often holds a few long keepalive or HTTP/2 connections. Those few connections can pin most traffic to one or two workers. Check the per-worker lag under load. If it’s uneven, cap requests per upstream connection with nginx’s keepalive_requests, as in the nginx high-traffic recipe.

Plan for Socket.IO. Long-polling sends several HTTP requests per session, and they all need the same worker. You can set clients to the websocket transport only, which removes the need for sticky sessions. Broadcasts and rooms still span workers, so you need @socket.io/cluster-adapter either way. Keep long-polling and you also need @socket.io/sticky.

Move shared state out of the process. In-process sessions, caches and rate-limit counters live in one worker’s memory. A user’s session exists in one worker. Each worker counts its own rate limit, so the real limit multiplies by WORKERS. Move that state to Redis, and size Redis with the Redis maxmemory and eviction recipe.

Count your database connections. Each worker opens its own pool, so four workers with a pool of 10 hold 40 connections per pod. Multiply by your replica count before you deploy. If the total nears PostgreSQL’s max_connections, put PgBouncer in front of PostgreSQL.

Test it on your system

Run the same load against WORKERS=1 and then WORKERS=N on staging, with the same pod CPU and memory limits as production. Use an open-model tool that sends requests at a fixed arrival rate. Closed-loop tools such as wrk or autocannon wait for each reply before sending the next request. They slow down with the server and hide the queue you’re looking for.

This k6 script holds a fixed rate. Set rate to your peak and the thresholds to your own SLO:

import http from 'k6/http';

export const options = {
  scenarios: {
    peak: {
      executor: 'constant-arrival-rate',
      rate: 500,            // your peak requests per second
      timeUnit: '1s',
      duration: '10m',
      preAllocatedVUs: 200,
      maxVUs: 1000,
    },
  },
  thresholds: {
    http_req_failed: ['rate<0.01'],
    http_req_duration: ['p(99)<500'],
  },
};

export default function () {
  http.get(`${__ENV.TARGET}/api/products`);
}

Run it with k6 run -e TARGET=https://staging.example.com peak.js. With wrk2, wrk -t4 -c200 -d10m -R500 --latency <url> does the same job. Point the test at the endpoints your peak traffic hits, not a health check. Send it through the same load balancer or ingress as production, so the connection pattern matches.

While it runs, watch these:

  • Event loop lag per worker. The primary’s /metrics holds one series per worker. Pull it from inside the pod:

    kubectl exec deploy/api -- node -e "fetch('http://localhost:9100/metrics').then(r => r.text()).then(console.log)" | grep '^nodejs_eventloop_lag_p99'
  • CPU per worker. Run top inside the container. Every worker should carry load, and none should sit at 100% while the rest idle.

  • Throttling. cat /sys/fs/cgroup/cpu.stat shows nr_periods and nr_throttled. Compare the two before and after the run.

  • Errors and timeouts from k6 and from your app logs.

Halfway through, test the failure paths too. Kill one worker with kill -9 on its PID. The log should show the re-fork, and nodejs_cluster_worker_reforks_total should go up by one. Then roll the Deployment while k6 runs. Your error threshold should hold through both.

Write the pass bar from your SLO before the run. For example, p99 latency and error rate stay under your thresholds at peak rate. Event loop p99 stays under 100 ms on every worker, and throttled periods stay a small share of the total. Then raise the rate in 25% steps until a threshold breaks, so you know your headroom.

Keep it healthy

These rules use prom-client’s default metrics from the primary’s /metrics, plus cAdvisor and kube-state-metrics. They assume your scrape config adds namespace and pod labels and that the container is named app:

groups:
  - name: nodejs-workers
    rules:
      - alert: NodeEventLoopLagHigh
        expr: max by (namespace, pod) (nodejs_eventloop_lag_p99_seconds) > 0.1
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "Event loop p99 lag above 100 ms on {{ $labels.pod }}"
          description: "One worker or more is queuing callbacks. Check CPU per worker and throttling, then look for a slow synchronous path or uneven connection spread."
      - alert: NodeBusyOnOneCore
        expr: |
          (max by (namespace, pod) (nodejs_eventloop_lag_p99_seconds) > 0.1)
            and on (namespace, pod)
          (sum by (namespace, pod) (rate(container_cpu_usage_seconds_total{container="app"}[5m])) < 1.2)
            and on (namespace, pod)
          (sum by (namespace, pod) (kube_pod_container_resource_requests{resource="cpu", container="app"}) >= 2)
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: "{{ $labels.pod }} lags on about one core of a larger CPU request"
          description: "The pod runs one busy event loop. Check that WORKERS matches the CPU request and that availableParallelism() reads the cgroup quota."
      - alert: NodeCPUThrottled
        expr: |
          sum by (namespace, pod) (rate(container_cpu_cfs_throttled_periods_total{container="app"}[5m]))
            / sum by (namespace, pod) (rate(container_cpu_cfs_periods_total{container="app"}[5m]))
            > 0.25
        for: 15m
        labels:
          severity: warning
        annotations:
          summary: "{{ $labels.pod }} is CPU-throttled in over 25% of periods"
          description: "Workers, GC and libuv threads burst past the CPU limit. Raise the limit a little above the request, or run one worker fewer."
      - alert: NodeWorkersReforking
        expr: increase(nodejs_cluster_worker_reforks_total[15m]) > 2
        for: 1m
        labels:
          severity: critical
        annotations:
          summary: "Node workers on {{ $labels.pod }} keep crashing"
          description: "The primary re-forked workers more than twice in 15 minutes. Read the worker crash logs; check for heap OOM against --max-old-space-size."
      - alert: NodePodRestarting
        expr: increase(kube_pod_container_status_restarts_total{container="app"}[30m]) > 2
        for: 1m
        labels:
          severity: warning
        annotations:
          summary: "{{ $labels.pod }} restarted more than twice in 30 minutes"
          description: "Check the last termination reason. OOMKilled means the worker heaps add up past the memory limit; lower --max-old-space-size or WORKERS."

Set the lag and throttling thresholds from your own baseline. NodeBusyOnOneCore is the one I’d keep even after the rollout, because a later image change can drop WORKERS back to one without anyone noticing.

Roll back

Save the current Deployment first with kubectl get deploy api -o yaml > api-before.yaml. Then pick the smallest step back:

  • Set WORKERS=1. The primary runs one worker, which behaves like a single process, and you need no code change.
  • Change the start command back to node server.js to remove cluster mode. The SIGTERM handler in server.js keeps shutdown clean, and /metrics stays on the same port.
  • Run kubectl rollout undo deploy/api to return to the previous spec in one step.

If workers crash in a loop or pods hit OOMKilled, roll back now. The same goes for a feature that broke because it kept state in memory. Move that state to Redis before you try again. If you’re reverting for any other reason, wait for a quiet hour. Keep any higher CPU or memory limits until the rollback is live, then lower them.

More recipes

Further reading