Pairing Google’s organic results with OpenAI’s Responses API reproduced only 26.2% of ChatGPT’s cited pages under GPT-5.5 Instant, according to a preprint from researchers at the National University of Singapore and Xi’an Jiaotong University. Measured by site, the same pairing reached 48.7%. Search Engine Journal reported the findings on October 18, and the authors posted the paper on October 11.

The result matters because many AI visibility tools rely on APIs instead of the consumer product. This study tests how faithful that shortcut is.

How the team measured ChatGPT. Researchers captured the data flow inside ChatGPT’s web app from their own signed-in sessions. That capture showed which searches ChatGPT ran, which pages came back, and which pages it picked as sources. A page counted as cited whether it appeared as its own link or inside a grouped citation.

What they asked. The 80 prompts came from Search Arena, a public collection of real user conversations with search-enabled chatbots. Half were in English and half in Chinese, covering recommendations, factual lookups, synthesis, and analysis. Each prompt went through 20 runs on GPT-5.3 Instant, and again on GPT-5.5 Instant. GPT-5.2 Instant and GPT-5.4 Thinking received fewer prompts because both were retired before collection finished.

How the stand-ins were built. For Google, the team pulled up to five pages of organic results per prompt through SerpAPI, averaging about 43 results each. For the API, they sent 20 queries per prompt to gpt-5.3-chat-latest and gpt-5.5 with web search enabled. Each stand-in’s output was gathered within three days of the matching ChatGPT run.

What matched. Together, the two stand-ins found 28.3% of GPT-5.3 Instant’s cited pages and 26.2% of GPT-5.5 Instant’s. For sites, the figures were 50.3% and 48.7%, with subdomains treated as separate sites. A page counted as cited if it appeared in at least one of the 20 runs, and the percentages are averages across the 80 prompts.

Google alone covered 9.4% of pages for GPT-5.3 Instant and 6.4% for GPT-5.5 Instant. For sites, it covered 26.1% and 22.8%. The API’s full source list did better on pages, at 22.9% and 22.5%. Pages the API cited inside its answers covered less ground but aligned more closely: 23.3% were also cited by GPT-5.5 Instant, against 16.4% for GPT-5.3 Instant.

ChatGPT rewrites prompts and shifts sources. In runs where searches were visible, GPT-5.3 Instant and GPT-5.5 Instant never searched using the prompt as originally written. GPT-5.2 Instant used the original wording in 5.5% of its runs. For Chinese prompts with visible searches, 67.0% of GPT-5.3 Instant runs and 69.6% of GPT-5.5 Instant runs searched in English only.

Run-to-run drift was also large. For GPT-5.5 Instant, 44.5% of unique pages surfaced for a prompt appeared in just one of the 20 runs, and the GPT-5.3 Instant figure was 40.8%. Overlap between runs was stronger for sites than for pages. Two collection sessions also changed models partway through, after 14 and 16 runs, so the authors recommend confirming which model produced each response.

Limits. The work sits on arXiv as a preprint, a venue that skips peer review. The authors did not test any commercial tracking tool. None of the tested configurations includes GPT-5.6 or GPT-6, so the figures describe the models current from March through July 2026.

For teams that track AI visibility through API-based tools, the practical reading is narrow. A tool built on the Responses API or a SERP feed is a reasonable signal for which sites ChatGPT draws on, but a weak one for which exact URL gets cited. Reporting a single page-level citation rate from such a tool overstates its precision, especially for prompts in languages where ChatGPT searches in English.

The authors advise tracking both pages and sites, repeating each prompt several times, and re-evaluating methods whenever the underlying model changes. Ask vendors whether their page-level numbers come from the API or from the ChatGPT app, and treat site-level trends as the more dependable metric until that answer is documented.

Search Engine Journal, “OpenAI’s API Only Partly Matched ChatGPT’s Citations, Paper Finds,” by Matt G. Southern, October 18, 2026, reporting a preprint posted October 11.