---
title: "What ChatGPT Reads on Your Page, and the Bot That Decides If You Appear"
description: "Three OpenAI user agents can reach your site and only one governs whether you show up in ChatGPT answers. A January 2026 sentinel test and logged research sessions show what each assistant can and cannot see in your markup."
author: "Team"
category: "Search & AI Visibility"
date: 2026-10-21T16:05:07.903Z
canonical: "https://contentagents.dev/blog/what-chatgpt-actually-reads-on-your-page"
---

# What ChatGPT Reads on Your Page, and the Bot That Decides If You Appear

![a book, read, literature](https://cdn.pixabay.com/photo/2020/04/22/12/28/book-5077895_1280.jpg?w=1200&q=75)

> Three OpenAI user agents can reach your site and only one governs whether you show up in ChatGPT answers. A January 2026 sentinel test and logged research sessions show what each assistant can and cannot see in your markup.

A reader opens your pricing page and sees a table. An assistant opens the same URL and may see an empty container and a script tag. Nothing in your analytics distinguishes the two visits, which is why a page can rank respectably in classic search and still never appear in an AI answer.

This is for site owners and marketers who already understand crawling and want the causal chain rather than a checklist. Three questions decide the outcome: which bot arrived, what the server handed it, and how much of that the assistant actually consumed. Each link in that chain has public evidence behind it, and each can be checked on your own domain today.

## Two gates, and both of them are silent

Gate one is permission. A named user agent either has access to the URL or it does not, and your robots.txt decides. Gate two is substance. The bytes the server returns either carry your sentences or they carry a loader. Fail either gate and the answer gets assembled without you, with no error anywhere for you to notice.

Almost every diagnosis that goes wrong confuses the two. Teams audit their content when the problem was a blanket disallow, or rewrite their robots.txt when the problem was a client-rendered template.

## One name opts you out of training; a different one hides you from answers

OpenAI's [crawler documentation](https://developers.openai.com/api/docs/bots) splits its agents by job, and the split is the whole point. GPTBot gathers material that may feed model training, and disallowing it signals that your content should stay out of foundation model training. OAI-SearchBot is the search agent, and OpenAI states that a site opted out of it will not appear in ChatGPT's search answers. ChatGPT-User covers fetches a person sets in motion, and OpenAI notes that because a human started the action, robots.txt rules may not apply to it.

So the same two-line edit produces completely different outcomes depending on which name you type. Blocking the training agent is an intellectual property position. Blocking the search agent is a withdrawal from distribution. A wildcard rule that catches everything makes both choices at once, usually by accident, and OpenAI advises allowing roughly a day for a robots.txt change to register on the search side.

If you are deciding this across several vendors rather than one, our breakdown of [which crawlers are worth allowing](/which-ai-crawlers-to-allow-openai-anthropic-perplexity-google) sets the four decisions side by side.

## Fetching and rendering are not the same verb

Googlebot fetches, queues the page, runs the scripts, then indexes what the scripts produced. The AI crawlers measured so far stop after the fetch. Research by Vercel and MERJ, published in December 2024 by Giacomo Zecchini with three co-authors, examined crawler traffic across Vercel's network and concluded that [none of the major AI crawlers currently render JavaScript](https://vercel.com/blog/the-rise-of-the-ai-crawler).

They do request script files, which misleads people reading their logs. OpenAI's crawler spent [11.50 per cent of its requests on JavaScript files and Anthropic's 23.84 per cent](https://vercel.com/blog/the-rise-of-the-ai-crawler), downloading code it never executed. The same study found heavy misdirection: roughly 34.82 per cent of OpenAI's fetches and 34.16 per cent of Anthropic's landed on missing pages, against 8.22 per cent for Googlebot.

That dataset is from late 2024. A March 2026 analysis of more than [500 million GPTBot fetches, reported second-hand](https://somethinginc.com/blog/ai-crawlers-dont-render-javascript/), reached the same conclusion and reported a similar rate of script downloads without execution. We found no public measurement that reverses the finding, though one vendor write-up claims OpenAI began rendering in late 2026 without evidence we could trace. Treat rendering as absent and verify on your own logs.

## A sentinel test showed the behaviour differs by assistant

The most useful recent experiment on this is narrower and sharper than a crawl study. In January 2026, the agency RESONEO published a [test of user-triggered fetches](https://think.resoneo.com/sentinel/geo-llm-crawler-report.html): one page, fifteen different ways of injecting content, from plain markup through Shadow DOM, iframes, delayed scripts and fetch calls, each carrying a unique marker string. Then they asked each assistant to visit the page and report which markers it found.

The results break the convenient idea that assistants behave alike. ChatGPT and Claude read server-delivered markup and saw nothing requiring script execution. Grok did execute scripts and waited around two seconds, so delayed content reached it. Bing's assistant executed scripts but allowed roughly fifty milliseconds, catching almost nothing delayed, while being the only one in the test to read open Shadow DOM and iframe content. Google's AI Mode could not fetch a URL on request at all, drawing only on what was already in Google's index.

The practical reading: markup that arrives in the first response works everywhere. Everything else works somewhere, which is the same as nowhere when you cannot choose your reader.

## What a research session actually does with your page

The retrieval layer matters as much as the parser. Peec logged Deep Research sessions across more than ten accounts and roughly two hundred runs, snapshotted in June 2026, and [published the mechanics](https://docs.peec.ai/research/how-deep-research-reads-your-site.md). The agent works with three commands: a search through Bing, an open that renders the page as numbered plaintext with the head element stripped, and a find that behaves like a text search inside the opened page.

Two details change how you write. First, the agent never clicks: Peec reports that the click command appeared zero times in every logged session, so content behind a tab, an accordion or a filter is unreachable even when the markup technically contains it. Second, retrieval is keyword-driven in a specific way. A successful find caused the agent to reopen the page positioned at the matched line in [95 per cent of sampled cases](https://docs.peec.ai/research/how-deep-research-reads-your-site.md), which means the literal phrasing in your headings decides whether the right passage gets read twice or not at all.

Peec also notes the snippet it sees before deciding whether to open a page at all is capped near [285 characters](https://docs.peec.ai/research/how-deep-research-reads-your-site.md), and that once the agent is confident it reads only the top two or three results per query.

## Chasing one broken chain end to end

Take a documentation site built as a client-rendered application. A person loads a guide and reads it fine. Starting condition: the server's response body holds a mount point and a bundle reference.

A reader asks an assistant something the guide answers directly. The agent fetches the URL and receives that shell plus a link to the bundle. It does not run the bundle. The guide's text never enters the agent's context, so the answer cites whoever did ship text. Adding keywords changes nothing, because there is no document to add them to. Server rendering or static generation for the pages that answer buyer questions is the only fix at that link in the chain.

The mirror image is just as common and cheaper to repair: a fast, fully server-rendered site whose security plugin shipped with every AI user agent denied by default. Same invisibility, opposite cause.

## Five checks you can finish before lunch

- Read robots.txt the way an agent would. Hunt for a wildcard disallow and for lines naming GPTBot, OAI-SearchBot or ChatGPT-User, then decide each on its own merits.

- Use view-source, never the element inspector. The inspector shows you the page after scripts ran, which is the one view no crawler gets.

- Pull the page without a browser and grep the response for your main heading. If it does not print, the rendering path is the problem.

- Put the exact words buyers type into your headings, because the find command searches text, not meaning.

- Count missing pages and redirect hops in your access logs. Our walkthrough on [reading server logs](/how-to-read-server-logs-to-see-whats-actually-crawling-you-0ae4) has the commands, and [verifying a bot is genuine](/fake-and-spoofed-crawlers-are-hitting-your-site-heres-how-to-tell-e6ma) matters before you act on what you find.

## Repair the permission line before you rebuild the renderer

Here is the call we will defend. When a site is missing from AI answers, fix the access rules first and leave the rendering architecture alone until you have proved it is the cause. The common advice inverts this, because rendering is the more interesting engineering problem and because vendors selling rendering infrastructure write most of the guides.

The asymmetry justifies it. A wrong robots.txt line is a one-line change, it ships today, and OpenAI's documentation says the search side picks it up within about a day. A rendering migration is a quarter of engineering work, it touches your build and your hosting, and it can only help the subset of pages whose text genuinely lives in JavaScript. Two fixes, both plausible, separated by three orders of magnitude in cost. Order them by cost, not by interest.

There is also a diagnostic reason. Permission failures are total and rendering failures are partial, so the symptom tells you where to look. A site that an agent cannot fetch produces nothing at all. A site that ships a shell often still has readable text in its blog, its documentation or its structured data, which means you see sporadic citations rather than silence. Complete absence points at the access rules almost every time.

A site whose entire content layer is client-rendered, with no text in any server response and no sporadic citations to be found, is the one case this does not describe. For that site the permission check takes four minutes and then the renderer really is the whole job. The rule holds because the cheap check is cheap enough to run even when you expect it to pass.

## Where this evidence stops

None of it explains how a model weighs your page once the text is in front of it. A URL can be fetched a thousand times and never quoted. Crawl access is necessary and nowhere near sufficient, and no public study shows which passage of a page ends up in an answer. User-triggered fetches may also behave differently from indexing ones, by OpenAI's own account. Once access and rendering are sound, the work moves to the writing, which is where our guide to [reworking existing pages for AI search](/how-to-optimize-existing-content-for-ai-search-results-pr7q) picks up.

Trust your own access logs over any article on this subject, this one included. Every figure here has a date attached for exactly that reason.

## Common questions about what assistants can see

## FAQ

### Does ChatGPT run the JavaScript on my page?

No public measurement says it does. Vercel and MERJ found in December 2024 that no major AI crawler executed JavaScript, and a March 2026 analysis of GPTBot fetches reported the same, second-hand. RESONEO's January 2026 sentinel test found ChatGPT saw only content present in the server response. Confirm against your own logs, since behaviour can change.

### Which OpenAI user agent controls whether I appear in ChatGPT search?

OAI-SearchBot. OpenAI's documentation states that a site opted out of it will not be shown in ChatGPT search answers. GPTBot is a separate agent and a separate decision.

### Is blocking GPTBot a visibility problem?

Not directly. Disallowing GPTBot signals that your content should not be used in foundation model training. It does not govern search answers, which depend on OAI-SearchBot. Many teams block both with one wildcard rule and only intended the first.

### Why would an assistant miss text that is clearly on my page?

Three usual reasons. The text arrives only after scripts run. It sits behind a tab, an accordion or a filter, which Peec's logs show Deep Research never opens. Or the agent read a different part of the page, because its in-page search matched different wording than yours.


---
Source: https://contentagents.dev/blog/what-chatgpt-actually-reads-on-your-page