Puma workers and threads decide how many Rails requests one pod serves at once. Workers are forked processes, and each runs its own pool of threads. Get the product wrong and requests wait in a queue you can’t see, or the pod runs out of memory at the first busy hour.
What breaks at peak
CRuby runs Ruby code for one thread at a time per process, under the Global VM Lock (GVL). A thread gives up the lock while it waits on the database, Redis or an HTTP call. Threads add capacity only for that waiting time. Workers add capacity for Ruby code, one CPU’s worth each.
Three failures show up at peak, and each looks different from the outside:
- Too few workers. Every thread in the pod is busy and new connections sit in Puma’s backlog. p99 climbs while the pod’s CPU stays under its limit. Nothing logs an error. Your load balancer or nginx times out first and returns 504 or 502.
- Too many threads. The GVL turns into the queue. CPU sits near the limit, latency for each request grows, and throughput stops rising. Each extra thread also wants a database connection, so the pool runs dry and Active Record raises
ActiveRecord::ConnectionTimeoutError. The Active Record connection pool recipe covers that error. - Too many workers for the memory. Each worker holds its own heap. Past the container limit, the kernel kills a worker or the whole pod with exit code 137 and Kubernetes reports
OOMKilled.
Puma logs a crashed worker in the master’s output, so watch for these lines:
[1] - Worker 2 (PID: 41) booted in 0.01s, phase: 0
[1] ! Terminating timed out worker (Worker 1 failed to check in within 60 seconds): 33
The first line repeating with new PIDs means workers keep dying and Puma keeps forking replacements. The second comes from worker_timeout, 60 seconds by default, when a worker stops checking in with the master.
The Rails Puma configuration
This config/puma.rb runs in cluster mode. It reads both counts from the environment, so one image serves pods of different sizes:
# config/puma.rb
threads_count = Integer(ENV.fetch("RAILS_MAX_THREADS", 3))
threads threads_count, threads_count
workers ENV.fetch("WEB_CONCURRENCY", 2)
port ENV.fetch("PORT", 3000)
preload_app!(true)
before_fork do
# Ruby 3.3+: compact and tidy the heap once before forking.
Process.warmup if Process.respond_to?(:warmup)
end
activate_control_app "tcp://127.0.0.1:9293", { auth_token: ENV.fetch("PUMA_CONTROL_TOKEN") }
plugin :tmp_restart
Each line does one job:
threads threads_count, threads_countsets the minimum and maximum threads per worker. Rails 7.2 and later generate 3. Puma’s own default on CRuby is 0 to 5. Setting both numbers the same stops Puma from spawning threads during a spike.workerssets the process count. Puma readsWEB_CONCURRENCYitself, and this line makes the fallback explicit. From Puma 7.2 the valueautostarts one worker per available processor. It needs theconcurrent-rubygem, which Rails already loads, reads the cgroup CPU quota and rounds down. A quota under one CPU rounds to 0 and Puma runs in single mode.preload_app!loads Rails in the master before forking, so workers share the loaded code pages through copy-on-write. Puma 7 turns this on by default in cluster mode. Keep the line for Puma 6. Preloading rules out phased restarts, so you deploy by replacing pods.Process.warmuparrived in Ruby 3.3. It runs a major GC and compacts the heap, which leaves fewer pages for a worker’s first write to copy.activate_control_appopens Puma’s stats endpoint on localhost only. You’ll read it during the load test.
Active Record drops the master’s connections after fork on its own, so the old establish_connection hook isn’t needed. Puma 7 renamed the hooks: on_worker_boot became before_worker_boot. The old names still work and print a deprecation warning.
Size workers from CPU. Set WEB_CONCURRENCY to the CPUs the pod is guaranteed. With a CPU limit, that’s the limit rounded down. With only a request, use the request, since a busy node gives you no more. On a VM, use the core count. A worker per CPU keeps Ruby code on every core without the workers fighting each other.
Size threads from I/O wait. Start at 3. If your endpoints spend most of their time waiting on the database or other services, try 5 and compare p99 under the same load. If p99 rises while throughput stays flat, the GVL is now the queue, so go back down. For a typical Rails app I stay at 5 threads per worker or fewer, and I’d add workers before I add threads.
Size memory from the forked workers. A preloaded worker shares pages with the master until it writes to them. Measure each worker’s private memory after the app has served real traffic for a while:
# inside the pod, private (unshared) memory per Puma process in kB
for p in $(pgrep -f puma); do
echo "$p $(awk '/^Private/ {s+=$2} END {print s}' /proc/$p/smaps_rollup)"
done
Then check the budget: master RSS plus workers times private memory per worker, plus headroom for spikes, must fit under the container limit. If it doesn’t fit, drop a worker and add a pod. On glibc images, MALLOC_ARENA_MAX=2 stops each thread from growing its own malloc arena. jemalloc, loaded with LD_PRELOAD, does a similar job.
Count database connections. Each thread can hold one connection. Pods times workers times threads is the most connections your web tier can open. Add Sidekiq and cron jobs on top, then compare the total with max_connections. If it doesn’t fit, put PgBouncer in front of PostgreSQL before you add pods.
On Kubernetes I run two to four workers per pod rather than one. The master restarts a crashed worker without a pod restart, and the workers share the preloaded code. The CPU request then needs to match WEB_CONCURRENCY, or the HPA reads the wrong percentage.
On Heroku, Render and other platforms, set the same two variables in the app’s config. A buildpack or base image can set its own WEB_CONCURRENCY, so read the value the running process sees with printenv before you size anything else.
Test it on your system
Load test a staging deploy with the same CPU and memory limits as production, and the same WEB_CONCURRENCY and RAILS_MAX_THREADS. Use k6 with a fixed arrival rate, so a slow server doesn’t slow the test down with it:
// puma-peak.js
import http from 'k6/http';
import { check } from 'k6';
export const options = {
scenarios: {
peak: {
executor: 'constant-arrival-rate',
rate: Number(__ENV.RATE || 50),
timeUnit: '1s',
duration: '10m',
preAllocatedVUs: 100,
maxVUs: 500,
},
},
thresholds: {
http_req_failed: ['rate<0.01'],
http_req_duration: ['p(99)<800'],
},
};
export default function () {
const res = http.get(`${__ENV.BASE_URL}/products?page=2`);
check(res, { 'status 200': (r) => r.status === 200 });
}
k6 run -e BASE_URL=https://staging.example.com -e RATE=200 puma-peak.js
Set RATE to your peak requests per second for one pod, and pick an endpoint that hits the database the way real traffic does. The 800 ms p99 is a placeholder for your own latency budget. To find the rate where the pod gives out, step the rate up as in the k6 breaking point recipe.
While it runs, read Puma’s stats every few seconds:
kubectl exec deploy/web -- sh -c \
'curl -s "http://127.0.0.1:9293/stats?token=$PUMA_CONTROL_TOKEN"' | jq '.worker_status[] | .last_status | {backlog, running, pool_capacity}'
backlog counts requests waiting for a thread. pool_capacity counts threads free to take one. Watch the pod’s CPU, container_memory_working_set_bytes against the limit and the database’s connection count too.
Run the test twice with different thread counts and compare. The setting passes when p99 and the error rate stay inside the targets you wrote down. backlog should drain back to 0 between bursts, and memory should hold under your chosen share of the limit. No worker restarts and no OOMKilled should show up. If backlog grows while CPU sits at the limit, you need more pods. If it grows while CPU has room, you need more threads or faster queries.
Keep it healthy
Export Puma’s stats with prometheus_exporter. Run its collector as a sidecar, and start the Puma instrumentation from config/puma.rb:
# config/puma.rb, changed lines only
after_worker_fork do
require "prometheus_exporter/instrumentation"
PrometheusExporter::Instrumentation::Puma.start unless PrometheusExporter::Instrumentation::Puma.started?
end
The collector prefixes metrics with ruby_ by default. Pair it with cAdvisor and kube-state-metrics for the container:
groups:
- name: puma-capacity
rules:
- alert: PumaRequestBacklog
expr: sum by (instance) (ruby_puma_request_backlog) > 0
for: 5m
labels:
severity: warning
annotations:
summary: "Puma on {{ $labels.instance }} has requests waiting for a thread"
description: "Every thread was busy for 5 minutes. Check CPU against the limit: at the limit, add pods; with room to spare, raise RAILS_MAX_THREADS or fix the slow endpoint."
- alert: PumaNoFreeThreads
expr: sum by (instance) (ruby_puma_thread_pool_capacity) == 0
for: 2m
labels:
severity: critical
annotations:
summary: "Puma on {{ $labels.instance }} has no free threads"
description: "New requests queue in the backlog and p99 climbs. Scale out the deployment now, then look for a slow dependency in your APM or the database."
- alert: PumaContainerNearMemoryLimit
expr: |
max by (namespace, pod, container) (container_memory_working_set_bytes{container="web"})
/ on (namespace, pod, container)
max by (namespace, pod, container) (kube_pod_container_resource_limits{resource="memory", container="web"})
> 0.9
for: 10m
labels:
severity: warning
annotations:
summary: "Rails pod {{ $labels.pod }} above 90% of its memory limit"
description: "Workers outgrew the budget. Check private memory per worker in /proc/<pid>/smaps_rollup, then lower WEB_CONCURRENCY or raise the limit."
- alert: PumaContainerOOMKilled
expr: |
increase(kube_pod_container_status_restarts_total{container="web"}[10m]) > 0
and on (namespace, pod, container)
kube_pod_container_status_last_terminated_reason{reason="OOMKilled", container="web"} == 1
for: 0m
labels:
severity: critical
annotations:
summary: "Rails container {{ $labels.pod }} was OOM-killed"
description: "Workers times private memory per worker passed the limit. Lower WEB_CONCURRENCY by one or raise the memory limit, then re-run the load test."
Replace web with your container name. A short backlog during a deploy is normal, which is why the warning waits 5 minutes.
Roll back
Both settings live in the environment, so save them before you change them:
kubectl get deploy web -o jsonpath='{.spec.template.spec.containers[0].env}' > puma-env-before.json
To revert, set the old values and let the rollout replace the pods:
kubectl set env deploy/web WEB_CONCURRENCY=2 RAILS_MAX_THREADS=3 # your previous values
If the new values brought OOMKilled restarts or connection pool timeouts, roll back now. Raise the pod count while you do it, so capacity holds. A rollback for any other reason can wait for the daily low, since each rollout restarts every worker and empties their warm caches.
More recipes
- Could Not Obtain a Connection From the Pool in Rails
Give each Rails process a pool that covers every thread that can query, so ActiveRecord::ConnectionTimeoutError stops at peak and the database keeps headroom.
- Improve Cache Hit Ratio to Protect Your Origin at Peak
Trim the cache key, send cache headers from the origin and serve stale while revalidating, so the CDN absorbs peak traffic instead of passing it to your origin.
- Fix ALB 502 Bad Gateway and 504 Gateway Timeout
Set each target's keep-alive above the ALB idle timeout and keep slow requests under it, so the load balancer stops answering peak traffic with 502 and 504.
- Fix Lambda TooManyRequestsException Rate Exceeded
Size Lambda concurrency from peak traffic and split it with reserved and provisioned concurrency, so critical functions stop throttling when traffic spikes.