How to Configure Varnish for Magento So It Stops Caching the Wrong Thing

Varnish in front of a Magento store serves anonymous pages without waking PHP at all. The wrong Varnish in front of the same store serves one shopper's cart to everybody who arrives next, which is worse than running with no cache.

Magento generates a configuration that gets the hard parts right. Most of the work is using that file rather than one copied from a forum post, then understanding the handful of rules inside it that decide whether invalidation works.

Generate the file rather than writing one

The command renders Magento's own template with your values.

bin/magento varnish:vcl:generate \
  --export-version=6 \
  --backend-host=127.0.0.1 \
  --backend-port=8080 \
  --access-list=127.0.0.1 \
  --grace-period=300 \
  --output-file=/etc/varnish/default.vcl

Every option there has a default, and the defaults are localhost for the access list and the backend host, 8080 for the backend port, and 300 seconds for the grace period. Left without --output-file, the command prints the result to the terminal, which is the quickest way to diff a running configuration against what Magento would generate today after an upgrade.

Then point the store at Varnish and tell Magento which hosts to invalidate.

bin/magento config:set --scope=default --scope-code=0 \
  system/full_page_cache/caching_application 2
bin/magento setup:config:set --http-cache-hosts=127.0.0.1:6081

The same settings sit in the Admin under Stores > Settings > Configuration > Advanced > System > Full Page Cache, where the grace period field also defaults to 300 seconds. The cache hosts value is a comma separated list of host and port pairs, and it is stored in app/etc/env.php as http_cache_hosts.

What the generated configuration refuses to cache

Reading the generated file is worth ten minutes, because it is effectively a list of the mistakes it exists to prevent. Anything that is not a GET or a HEAD passes straight through, and so does anything under /customer or /checkout, which is what keeps carts and accounts out of the cache entirely.

# from the generated default.vcl
if (req.url ~ "/customer" || req.url ~ "/checkout") {
    return (pass);
}

The health check path passes. Requests under /media/ and /static/ pass by default, with commented lines in the template that turn caching on for stores running several locales without a CDN in front. Authenticated GraphQL requests that arrive without a cache id pass as well.

The response side is just as narrow. Only a 200 or a 404 is cached, anything marked private in Cache-Control is not, and a response that turns out to be uncacheable is remembered as such for 120 seconds so following requests skip the lookup. One further guard catches a mistake that custom code often introduces, where the configuration declines to cache a response carrying a new X-Magento-Vary cookie when the request itself did not have one.

The cookie that keeps shoppers apart

Magento sets a cookie called X-Magento-Vary that encodes the request context, and the generated hashing step adds that cookie's value to the cache key, so two shoppers in different customer groups land on different cached objects for the same URL.

Anything that strips or normalises that cookie before Varnish sees it merges those shoppers onto one cached page. A CDN rule that removes cookies for performance or an edge worker rewriting headers will both do it, and the symptom is a customer seeing content or pricing meant for a different group.

The template also lifts the hit rate in two quiet ways. It sorts query string parameters so the same page under a different parameter order is one object, and it strips a long list of marketing parameters including the utm_ family, gclid, fbclid and msclkid, so a campaign link and a clean link share one cache entry.

Invalidation is a ban, and it fails in two specific ways

When content changes, Magento sends an HTTP PURGE request carrying an X-Magento-Tags-Pattern header to every configured cache host. The generated configuration does not purge a single object, it issues a ban against the cache tags Magento attached to the objects.

if (req.method == "PURGE") {
    if (client.ip !~ purge) {
        return (synth(405, "Method not allowed"));
    }
    if (req.http.X-Magento-Tags-Pattern) {
      ban("obj.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags-Pattern);
    }
    return (synth(200, "Purged"));
}

That gives you the two failure modes precisely. A request from an address that is not in the purge access list is answered with 405, so a store whose access list does not include the machine PHP runs on will cache forever and the symptom looks like a content bug. A request carrying neither an X-Magento-Tags-Pattern nor an X-Pool header is answered with 400. Magento also splits long tag lists across several requests to stay under Varnish's request header limit, so a single content save can produce more than one PURGE.

Because it is a ban rather than a purge, watch the counters that actually move.

varnishstat -1 -f MAIN.bans -f MAIN.bans_completed \
  -f MAIN.cache_hit -f MAIN.cache_miss

You can send one by hand to prove the path works. The pattern below matches every tag in the cache, so on a busy store every following request goes to PHP until the cache refills, which is a real load spike rather than a theoretical one.

curl -X PURGE -H 'X-Magento-Tags-Pattern: .*' http://127.0.0.1:6081/

Prove a page is being served from cache

curl -sI https://www.example.com/ | grep -i 'x-magento-cache-debug'

The generated delivery step sets X-Magento-Cache-Debug to HIT, MISS or UNCACHEABLE on the way out. Adobe's verification page tells you to be in developer mode when checking headers, and to expect that header alongside X-Magento-Cache-Control and Age.

Two details will mislead you here if you do not know them. The template removes the Age header unless the backend sent X-Magento-Debug, and it removes X-Varnish, Via, Server and X-Powered-By from every response, so the absence of those headers is not evidence that Varnish is missing.

A check that does not depend on headers is that var/page_cache in the Magento root stays empty once Varnish is the caching application, because Magento is no longer writing its own full page cache. For a live view of one URL, Varnish's log takes a query.

varnishlog -g request -q 'ReqURL eq "/"'

Grace, and the health check that decides whether stale content ever ends

The generated backend definition probes /health_check.php every 5 seconds with a 2 second timeout, over a window of 10 probes and a threshold of 5, so five of the last ten must succeed for the backend to count as healthy. Cached objects are given a three day grace, and the hit path uses the probe result to decide what to do with an expired one. Within the configured grace period and with a healthy backend, the stale object is delivered while a fresh one is fetched behind it. With an unhealthy backend, it is delivered regardless of age.

That is excellent behaviour during a PHP outage and a baffling one when the probe is simply wrong. A web server that does not serve /health_check.php makes Varnish believe the backend is permanently sick, and the store then serves stale pages indefinitely while every content change appears to do nothing. Check the probe before debugging anything else about caching.

curl -sI http://127.0.0.1:8080/health_check.php | head -1
varnishstat -1 -f MAIN.cache_hit_grace

If the cache stack needs building or auditing rather than describing, that is the core of our Magento 2 speed optimization work, and the layers underneath it belong to server setup and optimization.