On GCP, the load balancer, Cloud Run and Cloud NAT each have a limit that only shows up under load. A mismatch between the backend timeout and keepalive turns into 502s, so check those first. Start with Fix GCP Load Balancer 502s and backend_timeout.
- Cloud NAT Port Exhaustion Alerts in Prometheus
Alert on Cloud NAT port usage per VM and on OUT_OF_RESOURCES dropped packets, so you see port exhaustion before outbound calls start timing out.
- Fix Cloud Run "No Available Instance" Errors
Set Cloud Run concurrency from one instance's real limit and size min and max instances for your peak, so cold starts and scaling stop dropping requests.
- Fix GCP Load Balancer 502s and backend_timeout
Size the backend service timeout to your slowest real request and hold backend keepalive at 620 seconds, so the load balancer stops answering 502.