An OCI load balancer 502 Bad Gateway at peak tends to trace back to three settings. They are your backend’s keepalive timeout, the listener’s idle timeout and the health check. Oracle fixes some of the load balancer’s own timers, so most of the fix sits on your side of the connection.
What breaks at peak
Oracle documents three connection timers on the Load Balancer service. You can change one of them.
| Timer | Value | Can you change it |
|---|---|---|
| Client to load balancer keepalive | 65 seconds idle, or 10,000 requests | No |
| Load balancer to backend keepalive | Closes backend connections idle over 300 seconds | No |
| Listener idle timeout | 60 seconds for HTTP listeners, 300 for TCP, 7200 at most | Yes |
Your backend closes pooled connections first. The load balancer reuses connections to your backends and keeps idle ones for up to 300 seconds. Oracle says the service “doesn’t honor keep-alive settings from backend servers.” Most web servers close idle connections much sooner. nginx closes them after 75 seconds by default, Node.js after 5. If the backend closes a socket as the load balancer sends a request down it, that request fails with a 502. Oracle’s guidance, “To prevent possible 502 errors,” is that backends keep idle connections for at least 310 seconds. Peak traffic makes this race more likely, because there are more pooled connections and more requests to land on a closing one.
A slow response trips the idle timeout. This timer covers the gap between two reads or two writes during a request. A report that runs 90 seconds before it sends its first byte breaks a 60-second timeout. So does an upload where the backend stays silent too long. At peak, slow queries push more requests past the limit. The timer doesn’t cover idle time between requests on a kept-alive connection; the fixed 65-second keepalive does.
Health checks fail under load. By default the check runs every 10 seconds and waits 3 seconds for an answer. Three failures in a row mark a backend unhealthy. A busy backend that answers its health endpoint in 4 seconds drops out of rotation while it still serves traffic. Its load moves to the others, they slow down and drop out next. Once every backend in the set is unhealthy, the load balancer has nowhere to send requests. The backend’s status in the console or API names the reason:
OK
INVALID_STATUS_CODE
TIMED_OUT
REGEX_MISMATCH
CONNECT_FAILED
IO_ERROR
OFFLINE
UNKNOWN
TIMED_OUT at peak means a slow health endpoint. CONNECT_FAILED means a security list or network security group blocks the health checker, or the backend stopped listening. Check one backend from the CLI:
oci lb backend-health get \
--load-balancer-id "$LB_ID" \
--backend-set-name app-backends \
--backend-name 10.0.10.15:8080
Oracle updates health status every three minutes and keeps no history, so read it while the problem is happening.
The OCI load balancer idle timeout and backend config
Keep backend connections open past 310 seconds. I set this before any load balancer change. It removes the 502s that the app logs can’t explain. These are the changed lines only:
# nginx, inside the http {} or server {} block the load balancer reaches
keepalive_timeout 320s;
// Node.js http.Server, before server.listen()
server.keepAliveTimeout = 320 * 1000;
server.headersTimeout = 321 * 1000; // keep it above keepAliveTimeout
// Go net/http
srv := &http.Server{
Addr: ":8080",
Handler: mux,
ReadTimeout: 30 * time.Second,
IdleTimeout: 320 * time.Second,
}
# Spring Boot, embedded Tomcat
server:
tomcat:
keep-alive-timeout: 320s
For Gunicorn with the gthread or an async worker, pass --keep-alive 320. For Uvicorn, pass --timeout-keep-alive 320. Each idle connection holds a file descriptor and some memory on the backend. The nginx tuning recipe shows how to size connection limits for that.
Size the listener idle timeout. Find the longest gap with no bytes sent during a real request, at peak. Look at your slowest report or export before its first byte, and at large uploads. Set the timeout a little above it. Keep it tight for API traffic: a long timeout lets hung requests hold backend workers. If one endpoint needs minutes, give it its own listener, so the API listener stays short. WebSocket and server-sent event connections need a timeout above their heartbeat interval.
This Terraform covers the listener, the backend set and its health check:
resource "oci_load_balancer_listener" "https" {
load_balancer_id = oci_load_balancer_load_balancer.app.id
name = "https"
default_backend_set_name = oci_load_balancer_backend_set.app.name
port = 443
protocol = "HTTP"
connection_configuration {
idle_timeout_in_seconds = 120
}
ssl_configuration {
certificate_ids = [var.certificate_id]
verify_peer_certificate = false
}
}
resource "oci_load_balancer_backend_set" "app" {
load_balancer_id = oci_load_balancer_load_balancer.app.id
name = "app-backends"
policy = "LEAST_CONNECTIONS"
health_checker {
protocol = "HTTP"
port = 8080
url_path = "/healthz"
return_code = 200
interval_ms = 10000
timeout_in_millis = 5000
retries = 3
}
}
Each setting explained:
idle_timeout_in_secondsis the listener idle timeout. It’s the one load balancer timer you control. Too low cuts off slow responses and uploads. Too high keeps hung requests open.policy = "LEAST_CONNECTIONS"sends new requests to the backend with the fewest open connections. If request cost varies, it stops a backend stuck on slow requests from getting more of them.url_pathshould point at a cheap endpoint that answers from memory. A health check that queries the database fails at the moment the database is busy and pulls healthy backends out with it.timeout_in_millismust sit above that endpoint’s p99 at peak. The default is 3000 ms.retriessets how many failures mark a backend unhealthy, and how many passes bring it back. With a 10-second interval and 3 retries, a dead backend takes about 30 seconds to leave rotation.
backend_max_connections caps connections from the load balancer to each backend, with a floor of 256 when set. It protects a backend from overload, but requests over the cap wait or fail at the load balancer. I leave it unset unless a backend falls over under concurrency.
On OKE, the cloud controller creates the load balancer from a Service of type LoadBalancer. Set the idle timeout with an annotation:
apiVersion: v1
kind: Service
metadata:
name: web
annotations:
oci.oraclecloud.com/load-balancer-type: "lb"
service.beta.kubernetes.io/oci-load-balancer-connection-idle-timeout: "120"
spec:
type: LoadBalancer
selector:
app: web
ports:
- port: 443
targetPort: 8080
The controller can overwrite console edits to a load balancer that OKE manages. Make the change in the Service manifest. The pod side of a clean rollout lives in the graceful shutdown recipe.
Clients in front of the load balancer. The 65-second client keepalive is fixed. A service that calls this load balancer through its own connection pool should close idle connections in under 65 seconds. Otherwise it sends requests on sockets the load balancer has closed, and gets resets. The Node.js keepalive recipe covers that socket hang up case.
Test it on your system
Point the test at a staging load balancer with the same shape, listener settings and backend count as production. Two failures need two tests.
The keepalive race needs pooled connections that go idle, then a burst onto them. A flat load keeps every connection busy and hides it. Hold a fixed arrival rate with k6:
import http from 'k6/http';
export const options = {
scenarios: {
bursts: {
executor: 'ramping-arrival-rate',
startRate: 50, timeUnit: '1s',
preAllocatedVUs: 500, maxVUs: 3000,
stages: [
{ target: 600, duration: '2m' }, // your peak rate
{ target: 600, duration: '5m' },
{ target: 10, duration: '20s' }, // let pooled connections idle
{ target: 10, duration: '2m' },
{ target: 600, duration: '20s' }, // burst onto idle connections
{ target: 600, duration: '5m' },
],
},
},
thresholds: { http_req_failed: ['rate<0.001'] },
};
export default function () {
http.get('https://staging.example.com/api/products');
}
Run it with the old backend keepalive and again with 320 seconds. Set the rate and the threshold to your own peak and error budget.
The idle timeout needs a slow request. Call your slowest real endpoint at peak, or add a staging-only route that waits a set time before its first byte. Try delays below and above the listener’s idle timeout.
While both run, open Metrics Explorer for the oci_lbaas namespace. Watch HttpResponses502, HttpResponses504, BackendTimeouts and UnHealthyBackendServers for the backend set. Run the oci lb backend-health get command above every minute, and tail the backend’s own logs.
Set your pass criteria before the run. The change passes when 502s stay inside your error budget through the burst after the quiet period. No backend should go unhealthy while it serves traffic. Requests longer than the idle timeout should fail where you expect them to, and nowhere else.
Keep it healthy
OCI publishes these metrics to its Monitoring service, under oci_lbaas. I didn’t find a maintained Prometheus exporter for that namespace, so the load balancer alarms below live in OCI Monitoring. Oracle’s metrics reference prints the names in camelCase, while Grafana’s published OCI dashboard queries use names such as HttpResponses502. Copy the exact name from Metrics Explorer in your tenancy before you apply this:
resource "oci_monitoring_alarm" "lb_502" {
compartment_id = var.compartment_id
metric_compartment_id = var.compartment_id
display_name = "OCI LB 502 responses"
namespace = "oci_lbaas"
query = "HttpResponses502[1m].sum() > 10"
pending_duration = "PT5M"
severity = "CRITICAL"
destinations = [oci_ons_notification_topic.oncall.id]
is_enabled = true
body = "502s from the load balancer. Check backend keepalive (must exceed 310 s) and backend health with oci lb backend-health get."
}
resource "oci_monitoring_alarm" "lb_unhealthy" {
compartment_id = var.compartment_id
metric_compartment_id = var.compartment_id
display_name = "OCI LB unhealthy backends"
namespace = "oci_lbaas"
query = "UnHealthyBackendServers[1m].max() > 0"
pending_duration = "PT5M"
severity = "WARNING"
destinations = [oci_ons_notification_topic.oncall.id]
is_enabled = true
body = "A backend failed its health check. Read its status (TIMED_OUT, CONNECT_FAILED) and check the health endpoint's latency under load."
}
Set the > 10 threshold from your own traffic and error budget. Add a third alarm on BackendTimeouts if your listener idle timeout is tight.
In Prometheus, probe through the load balancer from outside with blackbox_exporter. The backend’s own metrics never count a 502 that the load balancer generated, so a probe sees what users see. In blackbox.yml:
modules:
http_2xx:
prober: http
timeout: 5s
http:
valid_status_codes: [200]
preferred_ip_protocol: ip4
groups:
- name: oci-load-balancer-probe
rules:
- alert: OciLbProbeFailing
expr: avg_over_time(probe_success{job="blackbox-oci-lb"}[5m]) < 0.9
for: 5m
labels:
severity: critical
annotations:
summary: "Probes through the OCI load balancer to {{ $labels.instance }} are failing"
description: "Check probe_http_status_code. A 502 points to backend keepalive or health; run oci lb backend-health get for each backend."
- alert: OciLbProbe502
expr: max_over_time(probe_http_status_code{job="blackbox-oci-lb"}[10m]) == 502
for: 10m
labels:
severity: warning
annotations:
summary: "The OCI load balancer returned 502 to a probe of {{ $labels.instance }}"
description: "Raise backend keepalive above 310 seconds if it isn't already, then check the backend set's health status in the console."
The rules expect a scrape job named blackbox-oci-lb that probes your public URL through the load balancer. A probe every 15 to 30 seconds catches a sustained problem. It won’t catch a scatter of keepalive 502s, which is why the OCI alarm on HttpResponses502 stays.
Roll back
Save the current load balancer config first:
oci lb load-balancer get --load-balancer-id "$LB_ID" > lb-before.json
With Terraform, revert the commit and apply. On OKE, revert the Service annotation. With the CLI, oci lb listener update takes the full listener definition. Pass every setting from lb-before.json, SSL included:
oci lb listener update \
--load-balancer-id "$LB_ID" \
--listener-name https \
--default-backend-set-name app-backends \
--port 443 --protocol HTTP \
--connection-configuration-idle-timeout 60 \
--ssl-certificate-ids '["ocid1.certificate.oc1..example"]' \
--force
The backend keepalive stays at 320 seconds even if you revert the rest. Lowering it brings the 502 race back. If a longer idle timeout lets hung requests pile up on your backends, roll it back now. For any other reason, revert when traffic is lowest, so a mistake reaches fewer users.
More recipes
- Improve Cache Hit Ratio to Protect Your Origin at Peak
Trim the cache key, send cache headers from the origin and serve stale while revalidating, so the CDN absorbs peak traffic instead of passing it to your origin.
- Fix ALB 502 Bad Gateway and 504 Gateway Timeout
Set each target's keep-alive above the ALB idle timeout and keep slow requests under it, so the load balancer stops answering peak traffic with 502 and 504.
- Fix Lambda TooManyRequestsException Rate Exceeded
Size Lambda concurrency from peak traffic and split it with reserved and provisioned concurrency, so critical functions stop throttling when traffic spikes.
- NAT Gateway ErrorPortAllocation and Port Exhaustion
Alert on ErrorPortAllocation and the signals that come before it, so you catch NAT gateway port exhaustion before outbound calls fail at peak.