To improve cache hit ratio, you shrink the number of distinct cache keys and keep each cached copy alive for longer. Every miss goes to your origin. At peak, a CDN with a poor hit ratio passes a share of the spike straight through to servers sized for a normal day. The fixes are the same on CloudFront, Cloudflare, Azure Front Door and Cloud CDN, though each names the settings in its own way.
What breaks at peak
Your origin handles edge requests × (1 − hit ratio) requests. At a 90% hit ratio it sees a tenth of the traffic. Drop to 70% and it sees three times as much, at the moment the edge traffic is highest. The CDN then returns your origin’s 502s and 504s to users, or its own error page when the origin stops answering.
Four things drag the hit ratio down when you need it most:
- A fragmented cache key. Each distinct key is a separate cache entry. Marketing parameters such as
utm_sourceorfbclid, a session cookie or a header likeAccept-Languagein the key give you many copies of one page. A campaign launch brings a fresh set of parameter values, so each one starts as a miss. - Responses the CDN won’t store.
Cache-Control: privateorno-storefrom a framework default, or a straySet-Cookie, makes the response uncacheable. Cloudflare won’t cache a response that carriesSet-Cookie, and Front Door won’t cacheprivate,no-cacheorno-store. - Synchronized expiry. A hot object expires, and many edge locations miss on it at once. A deploy that purges the whole cache does the same for every object. Each location has its own cache, so the misses multiply by the number of locations. The cache stampede recipe covers the same effect inside your app.
- Short or missing TTLs. Without
Cache-Control, each CDN falls back to its own default, and the defaults differ a lot. A one-minute TTL on a page that changes hourly sends sixty times the needed fills.
The fix
The fix has three parts. Send cache headers from the origin and keep the cache key to what changes the response. Then let the CDN serve a stale copy while it fetches a new one.
Cache headers from the origin. The origin knows what each response may be cached for. These two headers cover most sites; this is an nginx origin, changed lines only, inside your server block:
# fingerprinted assets, whose file name changes with the content
location /assets/ {
add_header Cache-Control "public, max-age=31536000, immutable" always;
}
# shared pages get a short browser cache and a longer CDN cache
location /products/ {
proxy_pass http://app;
proxy_hide_header Set-Cookie;
add_header Cache-Control "public, max-age=60, s-maxage=300, stale-while-revalidate=60, stale-if-error=86400" always;
}
max-age sets the browser’s cache time, and s-maxage overrides it for shared caches such as a CDN. stale-while-revalidate lets the CDN serve the old copy for that many seconds while it fetches a fresh one in the background. That turns a synchronized expiry into one background fill. stale-if-error lets it serve the old copy when the origin returns errors. proxy_hide_header Set-Cookie strips cookies the app sets on public pages. Strip them only on pages that are the same for every visitor.
Sizing comes from two rules. Your origin’s measured capacity sets the hit ratio you need:
required hit ratio ≥ 1 − (origin capacity in requests/s ÷ peak edge requests/s)
Find the origin’s capacity with the k6 breakpoint recipe, not from a guess. Then pick TTLs: without a shared upper cache, each object can miss once per TTL in every edge location. So fills ≈ hot objects × edge locations ÷ TTL. A longer TTL or a shield layer cuts that number.
Keep the cache key small. Include only the query parameters, headers and cookies that change the response. Everything else goes, after you confirm the origin ignores it. Removing something the origin uses serves one visitor’s page to another, so test this part with care. I start with the response headers of the busiest URLs, before I touch any CDN setting:
for p in / /products/ /assets/app.js; do
echo "== $p"
curl -s -o /dev/null -D - "https://www.example.com$p?utm_source=test" \
| grep -iE '^(cache-control|set-cookie|vary|age|x-cache|cf-cache-status):'
done
Shield the origin. A middle cache layer turns misses from many edge locations into one fill per object. CloudFront calls it Origin Shield, Cloudflare calls it Tiered Cache and Front Door uses its own parent caches.
The settings per provider:
| CloudFront | Cloudflare | Azure Front Door | Cloud CDN | |
|---|---|---|---|---|
| Where you set it | Cache policy on each cache behavior | Cache Rules | Route caching and Rule sets | Backend service or bucket cdnPolicy |
| What it caches by default | What your cache policy and headers allow | Static file extensions only; HTML and JSON need a rule | Nothing until you turn on caching on the route | CACHE_ALL_STATIC: static content, plus responses with valid cache headers |
TTL with no Cache-Control | The policy’s default TTL; 24 hours on CachingOptimized | Cloudflare’s default per status code, or your Edge TTL | A random duration of one to three days | 3600 seconds by default |
| Trim the key | Custom cache policy with an allowlist of query strings, headers and cookies | Ignore query string (Pay-as-you-go and Enterprise), sort query string on all plans | Query string behavior: ignore, use, or include or ignore named ones | --cache-key-query-string-whitelist, --no-cache-key-include-query-string |
| Shield layer | Origin Shield, per origin | Smart Tiered Cache, on all plans | Parent caches, no setting to manage | No setting to manage |
| Collapse misses | Request collapsing per cache key | No setting to manage | No setting to manage | Request coalescing, on by default |
| Stale serving | Honors stale-while-revalidate and stale-if-error | Serve stale while revalidating is on by default | Can serve expired content when the origin errors | Honors stale-while-revalidate; serve-while-stale defaults to 86400 seconds |
| Hit or miss header | x-cache: Hit from cloudfront | cf-cache-status: HIT | X-Cache: TCP_HIT or TCP_REMOTE_HIT | Cache result in the load balancer logs |
CloudFront. The managed CachingOptimized policy, ID 658327ea-f89d-4fab-a63d-7e88639e58f6, keeps no query strings and no cookies in the key. Its minimum TTL is 1 second, so it caches for at least that long even when the origin sends no-store. Use it for assets and pages that ignore query strings. For pages that need some parameters, create a cache policy with only those in the key. CloudFront collapses simultaneous misses only when they share a cache key, so a fragmented key also defeats collapsing. Turn on Origin Shield in the Region nearest your origin. The CacheHitRate metric is one of the additional metrics, which cost extra and need turning on:
aws cloudfront create-monitoring-subscription --distribution-id EDFDVBD6EXAMPLE \
--monitoring-subscription RealtimeMetricsSubscriptionConfig={RealtimeMetricsSubscriptionStatus=Enabled}
Cloudflare. Cloudflare caches by file extension, so HTML and JSON stay DYNAMIC until a Cache Rule marks them “Eligible for cache”. Add the rule for public paths only, and set Edge TTL to respect the origin’s headers. Turn on Smart Tiered Cache under Caching, Tiered Cache. A BYPASS status means the origin’s headers made the response uncacheable, so fix the headers there.
Azure Front Door. Turn on caching on the route and pick a query string behavior. “Ignore Specified Query Strings” lets you drop the marketing parameters. Front Door validates with Last-Modified only and doesn’t support ETag, so send Last-Modified from the origin. Rule sets override the route’s caching for paths that need it.
Cloud CDN. USE_ORIGIN_HEADERS caches only what the origin marks as cacheable, which is the safer mode for HTML. Allowlist the query parameters that change the page:
gcloud compute backend-services update web --global \
--cache-mode=USE_ORIGIN_HEADERS \
--cache-key-include-query-string \
--cache-key-query-string-whitelist=page,q
Never cache per-user responses. Keep anything that depends on Authorization or a session cookie out of the shared cache. Give it its own path or behavior with caching off. A hit ratio gain never justifies the risk of one user seeing another’s account page.
Test it on your system
Check the headers first, with the curl loop above, on your twenty busiest URLs from the CDN logs. Each public page should show a cache hit on the second request, and no response should carry both Set-Cookie and a public Cache-Control.
Then run load through the CDN to a staging origin. Read your provider’s policy on load tests first, and tell them if it asks you to. Use a fixed arrival rate and a URL list taken from real logs, query strings included, so the cache key sees real variety:
import http from 'k6/http';
import { SharedArray } from 'k6/data';
const paths = new SharedArray('paths', () => JSON.parse(open('./paths.json')));
export const options = {
discardResponseBodies: true,
scenarios: {
peak: {
executor: 'constant-arrival-rate',
rate: 2000, timeUnit: '1s', duration: '15m',
preAllocatedVUs: 1000, maxVUs: 6000,
},
},
thresholds: { http_req_failed: ['rate<0.01'], http_req_duration: ['p(99)<800'] },
};
export default function () {
const path = paths[Math.floor(Math.random() * paths.length)];
http.get(`https://staging-cdn.example.com${path}`, { tags: { name: 'mix' } });
}
Run it twice: once right after a purge, and once with a warm cache. The cold run is the one that matters, since a deploy or expiry puts you there at peak. While it runs, count requests at the origin from its access log or nginx_http_requests_total, and compare with the k6 request rate. Origin offload is 1 − origin requests ÷ edge requests.
Set the targets before you start. Origin requests should stay under the capacity you measured, during the cold run too. Offload on the warm run should meet the hit ratio your formula asked for. Origin 5xx should stay inside your error budget.
Keep CloudFront origin offload and hit ratio healthy
The CloudFront metrics come from CloudWatch through the Prometheus cloudwatch_exporter. CloudFront publishes to us-east-1 with the dimension Region=Global:
region: us-east-1
metrics:
- aws_namespace: AWS/CloudFront
aws_metric_name: CacheHitRate
aws_dimensions: [DistributionId, Region]
aws_statistics: [Average]
- aws_namespace: AWS/CloudFront
aws_metric_name: 5xxErrorRate
aws_dimensions: [DistributionId, Region]
aws_statistics: [Average]
For Cloudflare, cloudflare-exporter from lablabs exposes per-zone request counters. These rules cover both, plus the origin itself:
groups:
- name: cdn-offload
rules:
- alert: CloudFrontCacheHitRateLow
expr: aws_cloudfront_cache_hit_rate_average < 80
for: 30m
labels:
severity: warning
annotations:
summary: "CloudFront {{ $labels.distribution_id }} cache hit rate is under 80%"
description: "Check x-cache on the busiest URLs for misses, look for new query strings or cookies in the cache key, and check that a deploy didn't purge everything."
- alert: CloudFront5xxRateHigh
expr: aws_cloudfront_5xx_error_rate_average > 2
for: 15m
labels:
severity: critical
annotations:
summary: "CloudFront {{ $labels.distribution_id }} returns 5xx on over 2% of requests"
description: "The origin is failing or overloaded. Compare origin request rate with its capacity, and check the origin's own error log."
- alert: CloudflareCachedShareLow
expr: |
sum by (zone) (rate(cloudflare_zone_requests_cached[30m]))
/ sum by (zone) (rate(cloudflare_zone_requests_total[30m])) < 0.6
for: 30m
labels:
severity: warning
annotations:
summary: "Cloudflare zone {{ $labels.zone }} serves under 60% of requests from cache"
description: "Check cf-cache-status on the busiest URLs. DYNAMIC needs a Cache Rule, and BYPASS means the origin sent uncacheable headers."
- alert: OriginNearCapacity
# 1500 stands for the origin capacity your breakpoint test found
expr: sum by (instance) (rate(nginx_http_requests_total[5m])) > 0.7 * 1500
for: 10m
labels:
severity: warning
annotations:
summary: "Origin {{ $labels.instance }} is above 70% of its tested capacity"
description: "The CDN is passing more traffic through. Check the hit ratio alerts and recent cache key changes, and scale the origin if the hit ratio is normal."
CloudWatch data arrives several minutes late, and the exporter asks for data 10 minutes old by default. Keep the for times long on those rules. Cloudflare’s cached counter includes dynamic requests in the total, so its ratio runs below CloudFront’s CacheHitRate. Set every threshold from a few weeks of your own baseline.
Roll back
Save the current settings before each change:
aws cloudfront get-distribution-config --id EDFDVBD6EXAMPLE > cloudfront-before.json
gcloud compute backend-services describe web --global --format=yaml > web-before.yaml
For Cloudflare, note the Cache Rule and its position before you edit it, and disable it to roll back. For Front Door, export the route and rule set settings with az afd route show and az afd rule list.
A change might serve the wrong content, such as one user’s page to another or a stale price. Roll that back now and purge the affected paths. That’s the one CDN mistake that can’t wait. If origin load rises after a key change, the key removed something that collapsed requests before; restore it now. Any other revert can wait for a quiet hour, because a revert empties part of the cache and the misses land on your origin.
More recipes
- Fix ALB 502 Bad Gateway and 504 Gateway Timeout
Set each target's keep-alive above the ALB idle timeout and keep slow requests under it, so the load balancer stops answering peak traffic with 502 and 504.
- Fix Lambda TooManyRequestsException Rate Exceeded
Size Lambda concurrency from peak traffic and split it with reserved and provisioned concurrency, so critical functions stop throttling when traffic spikes.
- NAT Gateway ErrorPortAllocation and Port Exhaustion
Alert on ErrorPortAllocation and the signals that come before it, so you catch NAT gateway port exhaustion before outbound calls fail at peak.
- Fix Lambda RDS Too Many Connections with RDS Proxy
Put RDS Proxy between Lambda and RDS and cap concurrency to the proxy's pool, so a spike waits in a short queue instead of exhausting database connections.