Lambda RDS “too many connections” errors come from simple math. Each concurrent Lambda execution environment opens its own database connection, and Lambda can run far more environments than RDS has connection slots. RDS Proxy pools those connections so the database sees a small, steady number. The same fix works when dozens of pods or ECS tasks each hold their own pool.
What breaks at peak
Lambda scales by adding execution environments. Each one runs your handler for one request at a time. If the handler opens a database connection, 400 concurrent requests mean 400 connections. RDS sets max_connections from the instance’s memory by default, so a smaller instance runs out well before Lambda runs out of concurrency.
The database refuses the extra logins. MySQL and Aurora MySQL return:
ERROR 1040 (HY000): Too many connections
PostgreSQL and Aurora PostgreSQL return:
FATAL: sorry, too many clients already
The PgBouncer recipe lists the other Postgres variants, and the MySQL too many connections recipe covers the MySQL side.
Three Lambda habits make it worse:
- A connection opened inside the handler. Every invocation pays for a TCP and TLS handshake plus authentication. A crashed invocation can leave the connection open until the database times it out.
- Idle environments that keep connections. Lambda freezes an environment between requests. Its connection stays open and counts against
max_connectionswhile it does nothing. - Retries on failure. An async invocation that fails on a connection error goes back to Lambda’s queue and runs again, asking for yet another connection.
RDS Proxy fixes this by holding a pool of database connections and lending one to a client for each transaction. Your functions connect to the proxy, which can accept far more client connections than the database allows.
The fix with RDS Proxy and Lambda concurrency
Create the proxy in the same VPC as the database, with a security group that lets your functions reach it on the database port. Point your functions at the proxy endpoint instead of the instance or cluster endpoint.
Save the current pool settings first, then set your own:
aws rds describe-db-proxy-target-groups --db-proxy-name app-proxy \
> proxy-target-group-before.json
aws rds modify-db-proxy-target-group \
--db-proxy-name app-proxy \
--target-group-name default \
--connection-pool-config MaxConnectionsPercent=70,MaxIdleConnectionsPercent=35,ConnectionBorrowTimeout=10
In Terraform the same settings live on the default target group:
resource "aws_db_proxy_default_target_group" "app" {
db_proxy_name = aws_db_proxy.app.name
connection_pool_config {
max_connections_percent = 70
max_idle_connections_percent = 35
connection_borrow_timeout = 10
}
}
Each setting, and what goes wrong when it’s off:
MaxConnectionsPercentcaps the proxy’s database connections as a percentage ofmax_connections. At 100, the default, the proxy can take every slot and lock out migrations, cron jobs and your ownpsqlsession. Leave room for them. AWS also asks for at least 30% headroom above your recent peak, because the proxy shifts connection quotas between its own nodes.MaxIdleConnectionsPercentsets how many idle connections the proxy keeps warm. It defaults to half ofMaxConnectionsPercent. Too low, and a burst has to open new database connections at the worst moment. For spiky traffic I keep it close to the maximum.ConnectionBorrowTimeoutsets how long a client waits for a free database connection before it gets an error. The default is 120 seconds, which is longer than most API requests can wait. Set it below your function timeout so a full pool fails fast. MySQL clients then getERROR 9501 (HY000): Timed-out waiting to acquire database connection.IdleClientTimeoutis set on the proxy itself withaws rds modify-db-proxy. It closes client connections that sit idle, 1,800 seconds by default. Keep any client-side pool’s idle timeout below it, or your clients will reuse connections the proxy already closed.
Cap Lambda to the pool. The proxy queues clients once its pool is full. That beats refused logins, but an uncapped function can still queue thousands of requests that time out together. Set reserved concurrency on each function that uses the database:
aws lambda get-function-concurrency --function-name orders-api > concurrency-before.json
aws lambda put-function-concurrency --function-name orders-api \
--reserved-concurrent-executions 200
Size it from your own numbers. Measure the share of each invocation spent inside a transaction. Reserved concurrency times that share should fit inside the proxy’s database connections, with headroom. A function that spends a fifth of its time in transactions can run several environments per database connection. A function that spends all of it can’t.
For SQS triggers, set maximum concurrency on the event source mapping instead, so messages wait in the queue rather than failing:
aws lambda update-event-source-mapping --uuid 6d9bce8e-xmpl-4b6f-9a4b-2f2d0c3e8f71 \
--scaling-config MaximumConcurrency=50
Open the connection once per environment. Create the client outside the handler, so a warm environment reuses it. Node.js with pg:
const { Pool } = require('pg');
// one connection per execution environment, opened on first use
const pool = new Pool({
host: process.env.PROXY_ENDPOINT,
database: 'app',
user: 'app',
password: process.env.DB_PASSWORD,
ssl: { rejectUnauthorized: true },
max: 1,
idleTimeoutMillis: 120000,
});
exports.handler = async (event) => {
const { rows } = await pool.query('SELECT id, status FROM orders WHERE id = $1', [event.id]);
return rows[0] ?? null;
};
Python with PyMySQL:
import os
import pymysql
conn = None
def get_conn():
global conn
if conn is None or not conn.open:
conn = pymysql.connect(
host=os.environ["PROXY_ENDPOINT"],
user="app",
password=os.environ["DB_PASSWORD"],
database="app",
ssl={"ca": "/opt/rds-ca.pem"},
connect_timeout=5,
)
conn.ping(reconnect=True)
return conn
def handler(event, context):
with get_conn().cursor() as cur:
cur.execute("SELECT id, status FROM orders WHERE id = %s", (event["id"],))
return cur.fetchone()
max: 1 keeps one connection per environment, because each environment handles one request at a time. The PyMySQL ping reconnects if the proxy closed an idle connection while the environment was frozen. For IAM authentication, generate a token with the SDK in place of the password and turn on IAM auth on the proxy.
Avoid pinning. The proxy pins a client to one database connection when the session changes state it can’t share. A pinned connection serves one client until that client disconnects, so heavy pinning turns the proxy back into one connection per client. On PostgreSQL, a SET outside a transaction, PREPARE, temporary tables, LISTEN and session advisory locks all pin. On MySQL, SET of most variables, prepared statements, temporary tables, GET_LOCK and table locks pin. Any statement over 16 KB pins on every engine.
Move identical SET statements into the proxy’s initialization query, and use SET LOCAL inside a transaction on PostgreSQL. On MySQL, if you know your app doesn’t depend on session variables carrying over, add SessionPinningFilters=EXCLUDE_VARIABLE_SETS to the connection pool config. The proxy log records each pinned session with its reason, so check it after every release.
Pods and containers. The same pool math applies to ECS tasks and EKS pods. RDS Proxy works for them too. On EKS I often run PgBouncer instead, because it gives me pool modes and an admin console. For Lambda I go straight to RDS Proxy, since there’s nowhere to run a pooler next to a function.
Test it on your system
Point a staging copy of your function at a staging database with production-sized max_connections. Drive it at a fixed arrival rate with k6, through API Gateway or a function URL:
import http from 'k6/http';
import { check } from 'k6';
export const options = {
scenarios: {
burst: {
executor: 'ramping-arrival-rate',
startRate: 50,
timeUnit: '1s',
preAllocatedVUs: 300,
maxVUs: 2000,
stages: [
{ target: 50, duration: '2m' },
{ target: 600, duration: '1m' }, // set this to your peak
{ target: 600, duration: '10m' },
{ target: 50, duration: '2m' },
],
},
},
thresholds: {
http_req_failed: ['rate<0.001'],
http_req_duration: ['p(99)<1000'],
},
};
export default function () {
const res = http.get(`${__ENV.BASE_URL}/orders/${Math.floor(Math.random() * 100000)}`);
check(res, { 'status is 200': (r) => r.status === 200 });
}
k6 run -e BASE_URL=https://staging-api.example.com burst.js
Run it once against the instance endpoint to see the failure, then against the proxy. While it runs, watch:
- The function’s logs for
Too many connections,too many clientsand error 9501. ClientConnectionsandDatabaseConnectionson the proxy. Client connections should climb with concurrency while database connections level off.DatabaseConnectionsBorrowLatencyandDatabaseConnectionsCurrentlySessionPinned.ConcurrentExecutionsandThrottleson the function.
Write your targets before the run. It passes when no connection errors appear, DatabaseConnections stays under MaxDatabaseConnectionsAllowed, borrow latency stays inside your latency budget and the k6 thresholds hold. Throttles at the reserved concurrency are expected if your peak exceeds it. Decide whether that’s the behavior you want.
Keep it healthy
Pull the proxy metrics into Prometheus with YACE. Its AWS/RDS discovery job doesn’t map proxies to the ProxyName dimension CloudWatch uses, so use a static job per proxy:
apiVersion: v1alpha1
static:
- namespace: AWS/RDS
name: app-proxy
regions: [eu-west-1]
dimensions:
- name: ProxyName
value: app-proxy
period: 60
length: 300
metrics:
- name: DatabaseConnections
statistics: [Sum]
- name: MaxDatabaseConnectionsAllowed
statistics: [Sum]
- name: DatabaseConnectionsBorrowLatency
statistics: [Average]
- name: DatabaseConnectionsCurrentlySessionPinned
statistics: [Sum]
groups:
- name: rds-proxy
rules:
- alert: RdsProxyPoolNearMax
expr: |
aws_rds_database_connections_sum{dimension_ProxyName!=""}
/ aws_rds_max_database_connections_allowed_sum{dimension_ProxyName!=""} > 0.85
for: 5m
labels:
severity: warning
annotations:
summary: "RDS Proxy {{ $labels.dimension_ProxyName }} is using over 85% of its database connections"
description: "Check DatabaseConnectionsCurrentlySessionPinned first. If pinning is low, raise MaxConnectionsPercent or lower the reserved concurrency of the functions behind this proxy."
- alert: RdsProxyBorrowLatencyHigh
expr: aws_rds_database_connections_borrow_latency_average{dimension_ProxyName!=""} > 100000
for: 5m
labels:
severity: critical
annotations:
summary: "Clients wait over 100 ms for a connection from RDS Proxy {{ $labels.dimension_ProxyName }}"
description: "The pool is full and requests are queueing. Check the database for long transactions, then compare DatabaseConnections with MaxDatabaseConnectionsAllowed."
- alert: RdsProxyHeavyPinning
expr: |
aws_rds_database_connections_currently_session_pinned_sum{dimension_ProxyName!=""}
/ aws_rds_database_connections_sum{dimension_ProxyName!=""} > 0.5
for: 30m
labels:
severity: warning
annotations:
summary: "Over half the connections on RDS Proxy {{ $labels.dimension_ProxyName }} are pinned"
description: "Search the proxy log in CloudWatch Logs for pinned sessions and their reasons. Move SET statements into the proxy's initialization query or use SET LOCAL."
Borrow latency is in microseconds, so 100000 is 100 ms. Set it from your own latency budget. Alert on Throttles for each database function too, as the Lambda concurrency recipe shows.
Roll back
You saved proxy-target-group-before.json and concurrency-before.json before the change. To restore the pool settings, run modify-db-proxy-target-group with the values from the file. To remove a reserved concurrency limit:
aws lambda delete-function-concurrency --function-name orders-api
To take the proxy out of the path, point the function’s environment variable back at the database endpoint and publish. Before you do, check that the function’s peak concurrency fits under max_connections, or you’ll swap queueing for refused logins. If the proxy is causing errors, such as broken session state after the switch, roll back now and lower reserved concurrency to fit the database. If you’re reverting for any other reason, pick an off-peak window.
More recipes
- Fix ALB 502 Bad Gateway and 504 Gateway Timeout
Set each target's keep-alive above the ALB idle timeout and keep slow requests under it, so the load balancer stops answering peak traffic with 502 and 504.
- Fix Lambda TooManyRequestsException Rate Exceeded
Size Lambda concurrency from peak traffic and split it with reserved and provisioned concurrency, so critical functions stop throttling when traffic spikes.
- NAT Gateway ErrorPortAllocation and Port Exhaustion
Alert on ErrorPortAllocation and the signals that come before it, so you catch NAT gateway port exhaustion before outbound calls fail at peak.
- Improve Cache Hit Ratio to Protect Your Origin at Peak
Trim the cache key, send cache headers from the origin and serve stale while revalidating, so the CDN absorbs peak traffic instead of passing it to your origin.