A search marketer logged every bot hit on a live directory site for 41 days to see which crawlers actually follow links injected by JavaScript. The split by company was stark. Eight of the ten crawlers he tracked, including GPTBot, ClaudeBot, Bingbot, PerplexityBot, and the bots Meta and Amazon run for their AI products, never followed one of those links. Only Google’s crawler stack rendered the scripts and moved deeper, and even there the crawler that actually feeds Google’s index barely showed up.
Vinicius Stanula wrote up the test for Search Engine Land. He runs a directory site that classifies Brazilian businesses across roughly 2,400 pages. He split 21 top-level sections into two groups. Eleven kept every child link written directly into the HTML. Ten had every link, at every depth, added by a script after the page loaded, leaving nothing but an empty container in the raw source. He also switched off the sitemap, stripped breadcrumbs, and pulled internal navigation panels. Pages in the second group had no route to discovery besides a bot rendering that script.
A logging system captured every request for the full 41 days: path, agent, referrer, IP address, timestamp. Reverse DNS lookups confirmed which hits genuinely belonged to Googlebot and Bingbot rather than a spoofed identity. The number that mattered started one tier below the entry pages, since those stayed linked in ordinary HTML for both groups. A deeper crawl in the script-linked group could only happen if a bot had rendered its parent and clicked through to the link the script produced.
Across the 27-day first phase, Googlebot managed only 35 of the 748 pages in the HTML group’s deeper hierarchy, about 5 percent. It reached just 7 of the 293 comparable pages behind scripts, about 2 percent. GoogleOther, a separate Google crawler that renders scripts but does not build the index, crawled 66 percent of the HTML group’s pages. In the script-linked group, it reached 48 percent. That gap is the finding worth sitting with. GoogleOther proves Google’s stack can run a script and act on what it finds. GoogleOther is not the crawler responsible for what shows up in Search. Stanula warns that reading raw logs without separating the two Google agents can badly overstate real coverage.
Bingbot carries a documented, Edge-based rendering engine that Microsoft says has run since 2019. Yet across 20 active crawl days in the first phase, not one of its hits reached a script-produced link. It did reach three of those pages, on the crawl’s 26th and 27th days. Stanula attributes that to some other discovery route rather than rendering, since it never crawled further pages in those same sections afterward.
After 27 days, Stanula rewrote every script-injected link as static HTML and tracked the next two weeks. GPTBot came back within two days. In one pass, it crawled 250 pages in those sections that it had missed completely during the first 27 days. Bingbot added 212 pages within a week. Meta-ExternalAgent came back with its own sweep by day 41. Google reacted slowest of all. Googlebot picked up just one new page in the repaired sections through day 41, and GoogleOther added none. ClaudeBot kept returning to the site through day 41, but every visit landed on pages it already had. It never added a single freshly linked page. Amazonbot had already gone quiet by day 21, pulling out six days before that fix landed, so its coverage of the script-linked group stayed at zero throughout.
Stanula is explicit about the limits: one site, one niche, and a domain too new to have earned inbound links to the pages being tested. He expects an established site with stronger authority signals to pull more rendering effort from Googlebot. The experiment also only captured which pages crawlers could reach, not whether those pages ultimately entered the index. Search Console, not server logs, is the tool that settles whether a page actually made it into Search.
The practical read is not that AI crawlers refuse scripts outright, since Google’s stack proved otherwise. It is that the crawlers most teams now watch for AI referral traffic did not follow a single script-injected link across 41 days. Anyone auditing navigation before launch should confirm hard-coded anchor tags reach every revenue page in the raw HTML. Fixing the gap after the fact took AI search bots days to close, and it left Google still behind six weeks in.
Search Engine Land published Vinicius Stanula’s analysis on August 19, 2026.
