Google’s Gary Illyes says AI training demand is why fake Googlebot traffic has become a common headache for site owners. Asked on the Odys Podcast, hosted by Aneta Samkoff, why bot impersonation is so widespread, he answered: “I think it is AI.” Search Engine Roundtable’s Barry Schwartz reported the interview on October 30, 2026.
Impersonation means a crawler sends a Googlebot user-agent string while having no connection to Google. Illyes’s explanation is about incentive. Companies building models need content, and a scraper or a smaller AI engine that identifies itself honestly may be blocked. Claiming to be Googlebot can get it through the door.
Illyes did not say the defenses are failing outright. He said detection in firewalls and CDNs keeps getting better, but “they are not perfect,” so some traffic they were meant to stop still arrives. His practical point was that knowing who is crawling you lets you decide who gets access.
Schwartz’s summary of the video adds that Illyes has run online honeypots to attract crawlers. Some visitors labeled themselves Googlebot yet originated from networks with no link to Google. The summary also says a user-agent header can be changed in about 30 seconds, which is why the string alone proves nothing.
Illyes pointed to the proper checks: a reverse DNS lookup and a comparison of the requesting IP. Google documents both. A reverse DNS lookup on the requesting address should resolve to a Google-owned hostname, and the address should also fall inside the crawler ranges Google publishes. This is standard practice, and it takes a few lines in a log-analysis script.
The ranking angle is easy to miss. Many sites give Googlebot special treatment: exemptions from rate limits, firewall allowlists, access to content that regular visitors cannot see. Each exception based only on the user-agent is an invitation. The opposite error hurts too. A team that blocks every suspicious Googlebot claim without verifying risks cutting off the real crawler, and a blocked Googlebot cannot refresh or index your pages.
Schwartz’s summary lists several other points from the discussion. Illyes characterized the robots.txt file as a request that bad actors can ignore, so owners who need a hard block should turn to network-level controls, a WAF for example. He noted that Google-Extended and Apple’s Applebot-Extended tokens let publishers keep Search access apart from some AI training uses, and he expects finer-grained controls ahead.
Two forward-looking ideas came up. One is Web Bot Auth, an effort to sign HTTP requests cryptographically so a site can validate each request with the sender’s public key instead of trusting an IP address or a header. The other is a reputation score for crawlers. Illyes called it early-stage, and he warned that a scheme controlled by a single company risks skewed judgments.
His advice for smaller sites was restrained. Check whether bot traffic is hurting you at all before paying for protection. A CDN with bot management is a reasonable first step, and hosted platforms such as Shopify already include some safeguards.
The coverage contains no measurement of how much impersonated traffic exists or what share slips past commercial firewalls. Without that number, the scale of the problem rests on Illyes’s observations alone. The honeypot results are a useful signal, but they come from one setup.
Search teams can size their own exposure this quarter. Pull Googlebot hits from server logs, run reverse DNS and published-range checks against the source IPs, and look at the share that fails before changing any bot rules.
Reported by Search Engine Roundtable (Barry Schwartz) on October 30, 2026, from Gary Illyes’s interview on the Odys Podcast.
