Improve Cache Hit Ratio to Protect Your Origin at Peak

Trim the cache key, send cache headers from the origin and serve stale while revalidating, so the CDN absorbs peak traffic instead of passing it to your origin.

Published

Applies to CloudFront cache policies, Cloudflare Cache Rules, Azure Front Door Standard and Premium, Cloud CDN on Google Cloud load balancers

To improve cache hit ratio, you shrink the number of distinct cache keys and keep each cached copy alive for longer. Every miss goes to your origin. At peak, a CDN with a poor hit ratio passes a share of the spike straight through to servers sized for a normal day. The fixes are the same on CloudFront, Cloudflare, Azure Front Door and Cloud CDN, though each names the settings in its own way.

What breaks at peak

Your origin handles edge requests × (1 − hit ratio) requests. At a 90% hit ratio it sees a tenth of the traffic. Drop to 70% and it sees three times as much, at the moment the edge traffic is highest. The CDN then returns your origin’s 502s and 504s to users, or its own error page when the origin stops answering.

Four things drag the hit ratio down when you need it most:

  • A fragmented cache key. Each distinct key is a separate cache entry. Marketing parameters such as utm_source or fbclid, a session cookie or a header like Accept-Language in the key give you many copies of one page. A campaign launch brings a fresh set of parameter values, so each one starts as a miss.
  • Responses the CDN won’t store. Cache-Control: private or no-store from a framework default, or a stray Set-Cookie, makes the response uncacheable. Cloudflare won’t cache a response that carries Set-Cookie, and Front Door won’t cache private, no-cache or no-store.
  • Synchronized expiry. A hot object expires, and many edge locations miss on it at once. A deploy that purges the whole cache does the same for every object. Each location has its own cache, so the misses multiply by the number of locations. The cache stampede recipe covers the same effect inside your app.
  • Short or missing TTLs. Without Cache-Control, each CDN falls back to its own default, and the defaults differ a lot. A one-minute TTL on a page that changes hourly sends sixty times the needed fills.

The fix

The fix has three parts. Send cache headers from the origin and keep the cache key to what changes the response. Then let the CDN serve a stale copy while it fetches a new one.

Cache headers from the origin. The origin knows what each response may be cached for. These two headers cover most sites; this is an nginx origin, changed lines only, inside your server block:

# fingerprinted assets, whose file name changes with the content
location /assets/ {
    add_header Cache-Control "public, max-age=31536000, immutable" always;
}

# shared pages get a short browser cache and a longer CDN cache
location /products/ {
    proxy_pass http://app;
    proxy_hide_header Set-Cookie;
    add_header Cache-Control "public, max-age=60, s-maxage=300, stale-while-revalidate=60, stale-if-error=86400" always;
}

max-age sets the browser’s cache time, and s-maxage overrides it for shared caches such as a CDN. stale-while-revalidate lets the CDN serve the old copy for that many seconds while it fetches a fresh one in the background. That turns a synchronized expiry into one background fill. stale-if-error lets it serve the old copy when the origin returns errors. proxy_hide_header Set-Cookie strips cookies the app sets on public pages. Strip them only on pages that are the same for every visitor.

Sizing comes from two rules. Your origin’s measured capacity sets the hit ratio you need:

required hit ratio ≥ 1 − (origin capacity in requests/s ÷ peak edge requests/s)

Find the origin’s capacity with the k6 breakpoint recipe, not from a guess. Then pick TTLs: without a shared upper cache, each object can miss once per TTL in every edge location. So fills ≈ hot objects × edge locations ÷ TTL. A longer TTL or a shield layer cuts that number.

Keep the cache key small. Include only the query parameters, headers and cookies that change the response. Everything else goes, after you confirm the origin ignores it. Removing something the origin uses serves one visitor’s page to another, so test this part with care. I start with the response headers of the busiest URLs, before I touch any CDN setting:

for p in / /products/ /assets/app.js; do
  echo "== $p"
  curl -s -o /dev/null -D - "https://www.example.com$p?utm_source=test" \
    | grep -iE '^(cache-control|set-cookie|vary|age|x-cache|cf-cache-status):'
done

Shield the origin. A middle cache layer turns misses from many edge locations into one fill per object. CloudFront calls it Origin Shield, Cloudflare calls it Tiered Cache and Front Door uses its own parent caches.

The settings per provider:

CloudFrontCloudflareAzure Front DoorCloud CDN
Where you set itCache policy on each cache behaviorCache RulesRoute caching and Rule setsBackend service or bucket cdnPolicy
What it caches by defaultWhat your cache policy and headers allowStatic file extensions only; HTML and JSON need a ruleNothing until you turn on caching on the routeCACHE_ALL_STATIC: static content, plus responses with valid cache headers
TTL with no Cache-ControlThe policy’s default TTL; 24 hours on CachingOptimizedCloudflare’s default per status code, or your Edge TTLA random duration of one to three days3600 seconds by default
Trim the keyCustom cache policy with an allowlist of query strings, headers and cookiesIgnore query string (Pay-as-you-go and Enterprise), sort query string on all plansQuery string behavior: ignore, use, or include or ignore named ones--cache-key-query-string-whitelist, --no-cache-key-include-query-string
Shield layerOrigin Shield, per originSmart Tiered Cache, on all plansParent caches, no setting to manageNo setting to manage
Collapse missesRequest collapsing per cache keyNo setting to manageNo setting to manageRequest coalescing, on by default
Stale servingHonors stale-while-revalidate and stale-if-errorServe stale while revalidating is on by defaultCan serve expired content when the origin errorsHonors stale-while-revalidate; serve-while-stale defaults to 86400 seconds
Hit or miss headerx-cache: Hit from cloudfrontcf-cache-status: HITX-Cache: TCP_HIT or TCP_REMOTE_HITCache result in the load balancer logs

CloudFront. The managed CachingOptimized policy, ID 658327ea-f89d-4fab-a63d-7e88639e58f6, keeps no query strings and no cookies in the key. Its minimum TTL is 1 second, so it caches for at least that long even when the origin sends no-store. Use it for assets and pages that ignore query strings. For pages that need some parameters, create a cache policy with only those in the key. CloudFront collapses simultaneous misses only when they share a cache key, so a fragmented key also defeats collapsing. Turn on Origin Shield in the Region nearest your origin. The CacheHitRate metric is one of the additional metrics, which cost extra and need turning on:

aws cloudfront create-monitoring-subscription --distribution-id EDFDVBD6EXAMPLE \
  --monitoring-subscription RealtimeMetricsSubscriptionConfig={RealtimeMetricsSubscriptionStatus=Enabled}

Cloudflare. Cloudflare caches by file extension, so HTML and JSON stay DYNAMIC until a Cache Rule marks them “Eligible for cache”. Add the rule for public paths only, and set Edge TTL to respect the origin’s headers. Turn on Smart Tiered Cache under Caching, Tiered Cache. A BYPASS status means the origin’s headers made the response uncacheable, so fix the headers there.

Azure Front Door. Turn on caching on the route and pick a query string behavior. “Ignore Specified Query Strings” lets you drop the marketing parameters. Front Door validates with Last-Modified only and doesn’t support ETag, so send Last-Modified from the origin. Rule sets override the route’s caching for paths that need it.

Cloud CDN. USE_ORIGIN_HEADERS caches only what the origin marks as cacheable, which is the safer mode for HTML. Allowlist the query parameters that change the page:

gcloud compute backend-services update web --global \
  --cache-mode=USE_ORIGIN_HEADERS \
  --cache-key-include-query-string \
  --cache-key-query-string-whitelist=page,q

Never cache per-user responses. Keep anything that depends on Authorization or a session cookie out of the shared cache. Give it its own path or behavior with caching off. A hit ratio gain never justifies the risk of one user seeing another’s account page.

Test it on your system

Check the headers first, with the curl loop above, on your twenty busiest URLs from the CDN logs. Each public page should show a cache hit on the second request, and no response should carry both Set-Cookie and a public Cache-Control.

Then run load through the CDN to a staging origin. Read your provider’s policy on load tests first, and tell them if it asks you to. Use a fixed arrival rate and a URL list taken from real logs, query strings included, so the cache key sees real variety:

import http from 'k6/http';
import { SharedArray } from 'k6/data';

const paths = new SharedArray('paths', () => JSON.parse(open('./paths.json')));

export const options = {
  discardResponseBodies: true,
  scenarios: {
    peak: {
      executor: 'constant-arrival-rate',
      rate: 2000, timeUnit: '1s', duration: '15m',
      preAllocatedVUs: 1000, maxVUs: 6000,
    },
  },
  thresholds: { http_req_failed: ['rate<0.01'], http_req_duration: ['p(99)<800'] },
};

export default function () {
  const path = paths[Math.floor(Math.random() * paths.length)];
  http.get(`https://staging-cdn.example.com${path}`, { tags: { name: 'mix' } });
}

Run it twice: once right after a purge, and once with a warm cache. The cold run is the one that matters, since a deploy or expiry puts you there at peak. While it runs, count requests at the origin from its access log or nginx_http_requests_total, and compare with the k6 request rate. Origin offload is 1 − origin requests ÷ edge requests.

Set the targets before you start. Origin requests should stay under the capacity you measured, during the cold run too. Offload on the warm run should meet the hit ratio your formula asked for. Origin 5xx should stay inside your error budget.

Keep CloudFront origin offload and hit ratio healthy

The CloudFront metrics come from CloudWatch through the Prometheus cloudwatch_exporter. CloudFront publishes to us-east-1 with the dimension Region=Global:

region: us-east-1
metrics:
  - aws_namespace: AWS/CloudFront
    aws_metric_name: CacheHitRate
    aws_dimensions: [DistributionId, Region]
    aws_statistics: [Average]
  - aws_namespace: AWS/CloudFront
    aws_metric_name: 5xxErrorRate
    aws_dimensions: [DistributionId, Region]
    aws_statistics: [Average]

For Cloudflare, cloudflare-exporter from lablabs exposes per-zone request counters. These rules cover both, plus the origin itself:

groups:
  - name: cdn-offload
    rules:
      - alert: CloudFrontCacheHitRateLow
        expr: aws_cloudfront_cache_hit_rate_average < 80
        for: 30m
        labels:
          severity: warning
        annotations:
          summary: "CloudFront {{ $labels.distribution_id }} cache hit rate is under 80%"
          description: "Check x-cache on the busiest URLs for misses, look for new query strings or cookies in the cache key, and check that a deploy didn't purge everything."
      - alert: CloudFront5xxRateHigh
        expr: aws_cloudfront_5xx_error_rate_average > 2
        for: 15m
        labels:
          severity: critical
        annotations:
          summary: "CloudFront {{ $labels.distribution_id }} returns 5xx on over 2% of requests"
          description: "The origin is failing or overloaded. Compare origin request rate with its capacity, and check the origin's own error log."
      - alert: CloudflareCachedShareLow
        expr: |
          sum by (zone) (rate(cloudflare_zone_requests_cached[30m]))
            / sum by (zone) (rate(cloudflare_zone_requests_total[30m])) < 0.6
        for: 30m
        labels:
          severity: warning
        annotations:
          summary: "Cloudflare zone {{ $labels.zone }} serves under 60% of requests from cache"
          description: "Check cf-cache-status on the busiest URLs. DYNAMIC needs a Cache Rule, and BYPASS means the origin sent uncacheable headers."
      - alert: OriginNearCapacity
        # 1500 stands for the origin capacity your breakpoint test found
        expr: sum by (instance) (rate(nginx_http_requests_total[5m])) > 0.7 * 1500
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: "Origin {{ $labels.instance }} is above 70% of its tested capacity"
          description: "The CDN is passing more traffic through. Check the hit ratio alerts and recent cache key changes, and scale the origin if the hit ratio is normal."

CloudWatch data arrives several minutes late, and the exporter asks for data 10 minutes old by default. Keep the for times long on those rules. Cloudflare’s cached counter includes dynamic requests in the total, so its ratio runs below CloudFront’s CacheHitRate. Set every threshold from a few weeks of your own baseline.

Roll back

Save the current settings before each change:

aws cloudfront get-distribution-config --id EDFDVBD6EXAMPLE > cloudfront-before.json
gcloud compute backend-services describe web --global --format=yaml > web-before.yaml

For Cloudflare, note the Cache Rule and its position before you edit it, and disable it to roll back. For Front Door, export the route and rule set settings with az afd route show and az afd rule list.

A change might serve the wrong content, such as one user’s page to another or a stale price. Roll that back now and purge the affected paths. That’s the one CDN mistake that can’t wait. If origin load rises after a key change, the key removed something that collapsed requests before; restore it now. Any other revert can wait for a quiet hour, because a revert empties part of the cache and the misses land on your origin.

More recipes

Further reading