Fix FastAPI Blocking Calls in Async Routes

Keep blocking calls off the event loop with async clients, def routes or run_in_threadpool, so one slow call stops stalling every request in the process.

Published

Applies to FastAPI 0.110 and later on Uvicorn, Python 3.10 and later; AnyIO's default limit of 40 threads applies to Starlette and FastAPI

A FastAPI blocking call inside an async def route stops the whole Uvicorn process, not one request. Every other request on that event loop waits until the call returns. At low traffic you never see it. At peak, p99 climbs while CPU stays low and nothing appears in the error log.

What breaks at peak

Uvicorn runs one event loop per process. The loop switches between requests only at an await. A call that blocks without awaiting holds the loop until it finishes:

import requests
from fastapi import FastAPI

app = FastAPI()


@app.get("/quote")
async def quote():
    # Blocks the event loop for the whole HTTP call.
    r = requests.get("https://pricing.internal/v1/quote", timeout=5)
    return r.json()

If that call takes 300 ms, every request in the process waits up to 300 ms longer, including /healthz. Ten such calls in a burst stack into seconds of delay. The usual offenders are easy to miss in review:

  • requests, urllib3 and sync SDKs such as boto3
  • psycopg2, sync SQLAlchemy sessions and the Django ORM
  • time.sleep, file reads and writes, and subprocess.run
  • CPU work: password hashing, PDF rendering, large JSON or pandas transforms

Three symptoms point to a blocked loop. Fast endpoints slow down whenever a slow one gets traffic. Kubernetes liveness probes time out, and the kubelet restarts pods that were healthy a minute earlier. Your CPU graphs stay low, because a process waiting on a socket uses no CPU.

Uvicorn won’t save you here. Its --workers supervisor pings each worker from a separate thread, so a blocked loop still answers. Under Gunicorn with a Uvicorn worker class, the heartbeat runs on the loop. A long block there ends in [CRITICAL] WORKER TIMEOUT, which the Gunicorn workers and timeouts recipe covers. Rust services stall the same way when a handler blocks a Tokio worker, and the Tokio spawn_blocking recipe covers that fix.

Find the blocking call. Python’s asyncio debug mode logs any step that holds the loop for longer than 100 ms. Turn it on in staging, with the standard asyncio loop instead of uvloop:

PYTHONASYNCIODEBUG=1 uvicorn app.main:app --loop asyncio --port 8000

Each slow step logs a warning from the asyncio logger:

Executing <Task pending name='Task-12' coro=<RequestResponseCycle.run_asgi() running at ...>> took 0.312 seconds

The task name tells you which request held the loop. Debug mode adds overhead, so keep it out of production.

Move blocking calls off the loop with run_in_threadpool

Pick the fix by what the call does. I go down this list in order.

Use an async client where one exists. It’s the only fix that adds no threads. Swap requests for httpx.AsyncClient, psycopg2 for psycopg 3 or asyncpg, and sync SQLAlchemy for AsyncSession. Create the client once in the lifespan and reuse it, so you keep connection pooling.

Declare the route with def when its body is sync. FastAPI runs def routes and def dependencies in a threadpool and awaits the result. The loop stays free:

@app.get("/quote")
def quote():
    r = requests.get("https://pricing.internal/v1/quote", timeout=5)
    return r.json()

Wrap one call with run_in_threadpool in an async route. Use this when the route awaits other things and has one sync call in the middle:

from fastapi.concurrency import run_in_threadpool


@app.post("/login")
async def login(form: LoginForm):
    user = await users.get_by_email(form.email)
    ok = await run_in_threadpool(password_hasher.verify, user.password_hash, form.password)
    if not ok:
        raise HTTPException(status_code=401)
    return await sessions.create(user.id)

run_in_threadpool comes from Starlette and calls anyio.to_thread.run_sync. asyncio.to_thread also works, but it uses the loop’s default executor, a second pool capped at min(32, CPU count + 4) threads.

Send heavy CPU work to processes. Threads still share the GIL, so a CPU-bound function in the threadpool slows every other thread. Use a ProcessPoolExecutor with loop.run_in_executor, or move the work to a queue and a separate worker.

Size the threadpool. All of the above, except processes, share one AnyIO limiter per process. It defaults to 40 threads. The 41st sync call waits for a free thread, and your p99 grows with no error. Set it at startup:

import asyncio
import os
import time
from contextlib import asynccontextmanager

import anyio.to_thread
from fastapi import FastAPI
from prometheus_client import Gauge, Histogram, make_asgi_app

LOOP_LAG = Histogram(
    "app_event_loop_lag_seconds",
    "Delay between when a timer should fire and when the event loop ran it",
    buckets=(0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5),
)
THREADS_BUSY = Gauge("app_threadpool_busy_threads", "Threads in use in the AnyIO default limiter")
THREADS_MAX = Gauge("app_threadpool_max_threads", "Size of the AnyIO default limiter")


async def watch_loop(interval: float = 0.5) -> None:
    limiter = anyio.to_thread.current_default_thread_limiter()
    while True:
        start = time.perf_counter()
        await asyncio.sleep(interval)
        LOOP_LAG.observe(max(0.0, time.perf_counter() - start - interval))
        THREADS_BUSY.set(limiter.borrowed_tokens)
        THREADS_MAX.set(limiter.total_tokens)


@asynccontextmanager
async def lifespan(app: FastAPI):
    limiter = anyio.to_thread.current_default_thread_limiter()
    limiter.total_tokens = int(os.environ.get("THREADPOOL_SIZE", "40"))
    task = asyncio.create_task(watch_loop())
    yield
    task.cancel()


app = FastAPI(lifespan=lifespan)
app.mount("/metrics", make_asgi_app())

The size depends on what the threads wait for. If most sync calls hit the database, more threads than database connections only move the queue into the connection pool. I set the threadpool to the database pool size plus a few threads for other sync work. Raise it for sync calls to slow external APIs, where threads sit idle waiting.

Uvicorn settings. These matter once the loop is free:

  • --workers starts several processes on a VM. On Kubernetes I run one process per pod and scale pods, so each pod’s CPU and the metrics above map to one loop. With several workers, prometheus_client needs its multiprocess mode.
  • --limit-concurrency returns a 503 once a process holds that many connections or tasks. A fast 503 that a client retries beats a 30-second queue. Size it near what one process serves within your latency target.
  • --timeout-keep-alive defaults to 5 seconds. If a load balancer connects straight to Uvicorn, set it above the balancer’s idle timeout, or the balancer reuses closed sockets and returns 502s.
  • --backlog defaults to 2048, and the kernel caps it at net.core.somaxconn.

On Cloud Run or App Runner, the platform’s per-instance concurrency setting caps requests per container. Keep it in line with --limit-concurrency, so the platform scales out before Uvicorn starts refusing.

Test it on your system

The test that exposes a blocked loop mixes a slow endpoint with a fast one. If the fast endpoint’s p99 rises when the slow one gets traffic, something still blocks. This k6 script runs both at fixed rates:

import http from 'k6/http';
import { check } from 'k6';

const BASE = __ENV.BASE_URL;

export const options = {
  scenarios: {
    heavy: {
      executor: 'constant-arrival-rate',
      exec: 'heavy',
      rate: Number(__ENV.HEAVY_RATE || 50),
      timeUnit: '1s',
      duration: '10m',
      preAllocatedVUs: 100,
      maxVUs: 1000,
    },
    light: {
      executor: 'constant-arrival-rate',
      exec: 'light',
      rate: Number(__ENV.LIGHT_RATE || 200),
      timeUnit: '1s',
      duration: '10m',
      preAllocatedVUs: 100,
      maxVUs: 1000,
    },
  },
  thresholds: {
    'http_req_duration{scenario:light}': ['p(99)<100'], // your target for the cheap endpoint
    'http_req_duration{scenario:heavy}': ['p(99)<1500'],
    http_req_failed: ['rate<0.001'],
  },
};

export function heavy() {
  const res = http.get(`${BASE}/quote`);
  check(res, { 'heavy 200': (r) => r.status === 200 });
}

export function light() {
  const res = http.get(`${BASE}/items/42`);
  check(res, { 'light 200': (r) => r.status === 200 });
}

Run the light scenario alone first and note its p99. Then run both together at your peak rates. Pick the endpoints from your own traffic: one that calls a slow dependency, and one that only reads from memory or a cache.

While it runs, watch these:

  • app_event_loop_lag_seconds on /metrics. Lag near zero means the loop stays free.
  • app_threadpool_busy_threads against app_threadpool_max_threads. A pool pinned at its maximum means sync calls are queueing.
  • Liveness probe failures, with kubectl get events --field-selector reason=Unhealthy.
  • Pod CPU. Low CPU with high latency points back to blocking or a full threadpool.

The run passes when the light endpoint’s p99 under the mixed load stays close to its solo run, within the margin you set. Loop lag stays under your threshold, and no pod restarts.

Keep it healthy

These rules use the metrics from the lifespan code above. If you also run prometheus-fastapi-instrumentator, its http_request_duration_seconds histogram, labelled by handler, shows which route slowed.

groups:
  - name: fastapi-event-loop
    rules:
      - alert: FastAPIEventLoopLag
        expr: |
          histogram_quantile(0.99,
            sum by (le, namespace, pod) (rate(app_event_loop_lag_seconds_bucket[5m]))
          ) > 0.1
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "Event loop p99 lag above 100 ms on {{ $labels.pod }}"
          description: "A sync call is running on the loop. Run the pod's image in staging with PYTHONASYNCIODEBUG=1 and --loop asyncio to log the slow task, then move that call to a thread or an async client."
      - alert: FastAPIEventLoopBlocked
        expr: |
          histogram_quantile(0.99,
            sum by (le, namespace, pod) (rate(app_event_loop_lag_seconds_bucket[5m]))
          ) > 0.5
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "Event loop on {{ $labels.pod }} is stalling every request"
          description: "Every request in this process waits behind a blocking call. Check recent deploys for new sync calls in async routes, and roll back if one shipped."
      - alert: FastAPIThreadpoolFull
        expr: |
          min_over_time(app_threadpool_busy_threads[5m])
            >= on (namespace, pod) app_threadpool_max_threads
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "The AnyIO threadpool on {{ $labels.pod }} has been full for 5 minutes"
          description: "Sync routes and run_in_threadpool calls are queueing. Find the slow dependency first. Raise THREADPOOL_SIZE only if the database pool has room."

Set the lag thresholds from your own baseline. A loop that serves large JSON responses shows some lag from serialization alone.

Roll back

Each fix is a code change, so Git holds the old version. Record the running image and threadpool size before you deploy:

kubectl get deploy api -o jsonpath='{.spec.template.spec.containers[0].image}' > api-image-before.txt
kubectl get deploy api -o jsonpath='{.spec.template.spec.containers[0].env}' > api-env-before.json

kubectl rollout undo deployment/api returns to the previous ReplicaSet. If you only changed THREADPOOL_SIZE, set it back with kubectl set env deployment/api THREADPOOL_SIZE=40.

If a larger threadpool starts exhausting database connections, roll back at once, because that error spreads to every route. If a route moved to def runs slower than before, revert it in your next quiet window and look for an async client instead.

More recipes

Further reading