server.tomcat.threads.max sets how many requests one Spring Boot pod works on at the same time, and it defaults to 200. On Java 21 and later you can skip the number and run each request on a virtual thread. Both choices move the limit somewhere else, so test where it lands.
What breaks at peak
Tomcat’s NIO connector splits the work in two. An acceptor takes connections, up to server.tomcat.max-connections, which defaults to 8192. A worker pool of up to server.tomcat.threads.max threads runs the requests.
When all 200 workers are busy, Tomcat parks new requests in the executor’s queue, whose capacity defaults to Integer.MAX_VALUE. Nothing logs an error. Latency climbs while the queue grows, and CPU can sit low if the workers wait on a database or an API.
A long queue hurts in three ways:
- Health checks queue too. Liveness probes wait behind slow requests, miss their timeout, and the kubelet restarts a pod that was only busy. Its traffic moves to the remaining pods, which then queue harder.
- Clients give up first. The load balancer times out and returns a 504, but Tomcat still runs the request later. Your pod spends threads on answers nobody reads.
- The connection limit bites last. Past
max-connections, new connections wait in the kernel’s backlog, sized byserver.tomcat.accept-count(default 100). Past that, connection attempts time out or get refused.
Your clients get errors from the proxy in front of the pod. Behind nginx it’s upstream timed out (110: Connection timed out) while reading response header from upstream, which the nginx upstream timeouts recipe explains.
To confirm the pool is the limit, count busy worker threads in a thread dump. Tomcat names them after the connector:
kubectl exec deploy/api -- jcmd 1 Thread.print > threads.txt
grep -c '"http-nio-8080-exec-' threads.txt
grep -A 12 '"http-nio-8080-exec-' threads.txt | grep -E '^\s+at ' | sort | uniq -c | sort -rn | head -20
The second command groups the top frames. If most workers sit in HikariPool.getConnection, the database pool is the real limit, and the HikariCP recipe comes first. If they sit in socket reads to another service, that service sets your throughput.
The fix with Tomcat threads or Spring Boot virtual threads
You have two options. Platform threads with a sized pool work on any Java version and keep a hard cap per pod. Virtual threads remove the cap and push the limit to your downstream pools.
Option one, a sized platform thread pool. Changed lines only, in application.yaml:
server:
shutdown: graceful
tomcat:
threads:
max: 200
min-spare: 20
accept-count: 100
max-connections: 8192
mbeanregistry:
enabled: true
spring:
lifecycle:
timeout-per-shutdown-phase: 30s
threads.maxcaps concurrent requests per pod. Too low and requests queue while CPU idles. Too high and each thread adds a stack, more context switches and more callers for a database pool that can’t serve them.threads.min-sparekeeps this many workers alive when idle. The default is 10. A higher floor saves thread creation during the first seconds of a spike.accept-countis the kernel backlog once Tomcat holdsmax-connections. Raise it only if your load balancer retries refused connections on another pod.max-connectionscaps open connections, idle keep-alive connections included. Keep it above your load balancer’s connection count to this pod.mbeanregistry.enabledturns on the registry Micrometer reads Tomcat’s thread metrics from. It’s off by default, and without it thetomcat_threads_*metrics don’t exist.shutdown: graceful, the default since Spring Boot 3.4, lets in-flight requests finish on a rolling deploy.timeout-per-shutdown-phasebounds that wait.
Size the pool from what one pod handles at peak. By Little’s law:
busy threads per pod = peak requests per second per pod * p95 response time in seconds
Set threads.max to that number plus headroom for bursts, about half again. Then check it against two limits. CPU-bound requests gain nothing from more threads than cores. Requests that need the database can’t run faster than the HikariCP pool, so threads beyond it wait there instead of in Tomcat.
I’d rather keep threads.max near what a pod can finish and add pods than raise it into the thousands. A queue you can see in metrics beats a thousand threads all waiting on ten connections.
Option two, Spring Boot virtual threads. One property, on Spring Boot 3.2 or later and Java 21 or later:
spring:
threads:
virtual:
enabled: true
Tomcat then runs every request on a new virtual thread. A virtual thread that blocks on I/O releases its carrier thread, so thousands of waiting requests cost little memory. Spring Boot’s docs say threads.max and threads.min-spare have no effect once this is on. The same property moves @Async tasks and the task scheduler onto virtual threads too.
Three things change when you flip it:
max-connectionsbecomes your concurrency cap. Every open connection can run a request at once, up to 8192 by default.- Downstream pools take the queue. A HikariCP pool of 20 now gets thousands of callers at peak instead of 200. They wait
connection-timeout, then fail withConnection is not available. Put aSemaphorearound calls to slow dependencies, or lowermax-connections, so the queue stays bounded. - Pinning on Java 21 to 23. A virtual thread that blocks inside a
synchronizedblock holds its carrier thread. There’s about one carrier per core, so a few pinned threads stall the whole pod. Java 24 removed this forsynchronized(JEP 491), and Java 25 is the first LTS release with that change. Native calls still pin.
Find pinning with the JDK Flight Recorder. Record under load, then print the pinned events and their stack traces:
kubectl exec deploy/api -- jcmd 1 JFR.start name=pin duration=120s filename=/tmp/pin.jfr
# wait for the recording to finish, then
kubectl exec deploy/api -- jfr print --events jdk.VirtualThreadPinned /tmp/pin.jfr
On Java 21 to 23, I keep platform threads for any service whose JDBC driver or HTTP client blocks inside synchronized. On Java 25, I default to virtual threads for I/O-heavy services, with a semaphore in front of the database.
Test it on your system
Run a staging pod with production’s CPU and memory limits, its HikariCP pool and a real downstream. Ramp a fixed arrival rate in steps, so you see where queueing starts:
import http from 'k6/http';
import { check } from 'k6';
export const options = {
scenarios: {
steps: {
executor: 'ramping-arrival-rate',
startRate: 20,
timeUnit: '1s',
preAllocatedVUs: 300,
maxVUs: 3000,
stages: [
{ target: 100, duration: '3m' },
{ target: 100, duration: '7m' },
{ target: 200, duration: '3m' },
{ target: 200, duration: '7m' },
{ target: 300, duration: '3m' },
{ target: 300, duration: '7m' },
],
},
},
thresholds: {
http_req_failed: ['rate<0.01'],
http_req_duration: ['p(99)<600'],
},
};
export default function () {
const res = http.get(`${__ENV.BASE_URL}/api/catalog/search?q=shoes`);
check(res, { 'status is 200': (r) => r.status === 200 });
}
Point it at one pod, through a port-forward or a test Service, and set the plateaus around that pod’s share of your peak. Run it once with platform threads and once with virtual threads if you’re comparing.
While each step holds, watch Tomcat and the database pool:
kubectl port-forward deploy/api 8080:8080 &
watch -n 5 "curl -s localhost:8080/actuator/prometheus | grep -E '^(tomcat_threads_(busy|config_max)|tomcat_connections_current|hikaricp_connections_(active|pending))'"
With platform threads, tomcat_threads_busy_threads reaching tomcat_threads_config_max_threads marks the start of the Tomcat queue. With virtual threads those two metrics mean nothing. Watch tomcat_connections_current_connections and hikaricp_connections_pending instead.
Set targets for each step before the run. A step passes when:
- p99 and error rate stay inside the thresholds you chose.
- Busy threads stay under the maximum with room for a burst, or pending connections return to zero on virtual threads.
- No liveness probe failures or restarts show in
kubectl get pod -w. - With virtual threads, no
jdk.VirtualThreadPinnedevents point at a hot code path.
The step where p99 breaks tells you one pod’s real capacity. Use it to set your HPA target.
Keep it healthy
Micrometer exposes Tomcat’s thread and connection metrics through Spring Boot Actuator, once mbeanregistry.enabled is on. Turn on the request histogram with management.metrics.distribution.percentiles-histogram.http.server.requests=true for the latency rule.
Spring Boot 3.3.0 reported tomcat_threads_config_max_threads as -1. Upgrade to 3.3.1 or later, or the saturation rule never fires. The > 0 guard stops a -1 from turning into a negative ratio.
groups:
- name: tomcat
rules:
- alert: TomcatThreadPoolSaturated
expr: |
(
max by (job, instance) (tomcat_threads_busy_threads)
/ max by (job, instance) (tomcat_threads_config_max_threads) > 0.9
)
and on (job, instance) max by (job, instance) (tomcat_threads_config_max_threads) > 0
for: 5m
labels:
severity: warning
annotations:
summary: "Tomcat on {{ $labels.instance }} is using over 90% of its worker threads"
description: "New requests will queue inside Tomcat. Take a thread dump with jcmd 1 Thread.print to see where workers wait, then fix that dependency or add pods."
- alert: TomcatConnectionsNearMax
expr: |
max by (job, instance) (tomcat_connections_current_connections)
/ max by (job, instance) (tomcat_connections_config_max_connections) > 0.8
for: 5m
labels:
severity: warning
annotations:
summary: "Tomcat on {{ $labels.instance }} holds over 80% of max-connections"
description: "With virtual threads this is the concurrency cap. Check downstream pools for waiting callers before you raise server.tomcat.max-connections."
- alert: HttpP99AboveBudget
expr: |
histogram_quantile(0.99,
sum by (le, job) (rate(http_server_requests_seconds_bucket{uri!~"/actuator.*"}[5m]))
) > 1
for: 10m
labels:
severity: critical
annotations:
summary: "p99 for {{ $labels.job }} is above 1 s"
description: "Requests are slow or queued. Compare busy threads with the maximum, and HikariCP pending connections, to find which pool ran out."
Roll back
Both options are configuration, so keep application.yaml in git and note the running values first. If you pass them as environment variables, save the Deployment with kubectl get deployment api -o yaml > api-before.yaml.
Virtual threads switch off without a rebuild:
kubectl set env deployment/api SPRING_THREADS_VIRTUAL_ENABLED=false
If the change brings pool timeouts, probe failures or pinned carriers, roll back now. If it only failed to help, revert it during a low-traffic hour, since every pod restarts with cold caches.
More recipes
- G1 vs ZGC and How to Catch Java GC Pause Spikes
Alert on Java GC pause latency and full GCs before users notice p99 spikes, then use the same metrics to decide between G1 and ZGC.
- Fix Java Heap Space OOM in Kubernetes Containers
Size the JVM heap from the container limit with MaxRAMPercentage and leave room for native memory, so pods stop failing with heap errors or OOMKilled.
- Fix HikariCP Connection Is Not Available, Request Timed Out
Size the HikariCP connection pool in Spring Boot from your database limit and pod count, so requests stop timing out while they wait for a connection.
- Improve Cache Hit Ratio to Protect Your Origin at Peak
Trim the cache key, send cache headers from the origin and serve stale while revalidating, so the CDN absorbs peak traffic instead of passing it to your origin.