Google confirmed that Googlebot, Googlebot-Image, and every other crawler it operates draw against a single shared crawl capacity pool rather than separate per-crawler budgets. Search Engine Journal’s Roger Montti reported the update Monday, citing a change to Google’s Optimize Your Crawl Budget documentation. The same update adds a new recommendation: serve HTTP 304 Not Modified responses for pages that have not changed since the last crawl.
Google’s documentation now states plainly: “While each crawler has a different crawl demand, the crawl capacity limit is shared across all crawlers. This means that high demand from one crawler can reduce the capacity available for others.” That line resolves a question many technical SEOs have debated without a definitive answer: whether an image-heavy crawl session competes with the crawling of primary content pages. It does.
The practical risk is capacity contention. A large image library, a runaway faceted-navigation parameter space, or a sudden spike in Googlebot-Image activity can all draw down the same pool that would otherwise reach the URLs a site most needs crawled. Sites that have treated Googlebot and Googlebot-Image as independent budgets were operating on a false assumption. Google’s documentation now removes that ambiguity directly.
The second addition, on caching, is more actionable. Google’s updated guidance reads: “Use HTTP caching: Support 304 (Not Modified) HTTP status codes. If a page hasn’t changed since Google last crawled it, returning a 304 code tells Google to reuse the cached version, saving your server bandwidth and resources.” A 304 response tells Googlebot to reuse what it already has instead of re-downloading a page that has not changed, which frees the crawler to spend that request elsewhere.
This guidance is not for every site. Google’s crawl budget documentation targets two groups specifically: enterprise sites with a million pages or more, and medium sized sites with at least 10,000 pages that update frequently. A blog publishing a handful of posts a week has no crawl budget problem to solve. Most sites should treat this update as background knowledge, not an action item.
For teams that do qualify, the useful step is verification, not configuration. Many server and CDN setups return a 200 status with a full page body on every request, even when the client sends an If-Modified-Since or If-None-Match header signaling it already holds the current version. That behavior defeats the new recommendation before it starts. Sending a conditional GET with curl against a handful of stable URLs shows in under a minute whether the stack actually honors 304, or quietly serves full responses regardless.
Neither change alters how Googlebot crawls a page. Google is documenting a recommendation and a capacity mechanic it had not previously spelled out, not announcing new crawler behavior. Search Engine Journal noted these are the only two substantive additions to the page; the rest of the update makes existing language more precise elsewhere.
Sites large enough to have a genuine crawl budget problem should confirm 304 support this week. A shared, contested capacity pool means bandwidth spent re-serving unchanged pages is bandwidth not spent indexing the pages that matter.
Search Engine Journal’s Roger Montti first reported this update on August 3, 2026, citing Google’s Optimize Your Crawl Budget documentation.
