What is OAI-SearchBot? What OpenAI’s search crawler fetched on our new site

OAI-SearchBot is OpenAI’s crawler for ChatGPT search; GPTBot crawls for training. On our new site OAI-SearchBot read only robots.txt for 8 days, then fetched 47 addresses on 11 November 2026, and ChatGPT first cited the site about 19 hours later. How to tell them apart and check your own logs.

On this page
  1. OAI-SearchBot vs GPTBot: what OpenAI says
  2. What OAI-SearchBot requested on our site
  3. When did ChatGPT first cite this site?
  4. How to check OAI-SearchBot in your own logs
  5. What to do
  6. Frequently asked questions
  7. Get the next Census by email
  • 8verified OAI-SearchBot requests from 3 to 10 November, all for robots.txtCensus bot log, 3 to 10 Nov 2026
  • 47verified OAI-SearchBot requests for pages, feeds and copies, in under 3 minutesCensus bot log, 05:32 to 05:35 UTC, 11 Nov 2026
  • 156verified GPTBot requests, in one 105-second crawlCensus bot log, 4 Nov 2026
  • 2 of 26ChatGPT answers to our watchlist questions that cited this site (0 of 26 a week earlier)Census AI Checks, 12 Nov 2026

Short answerOAI-SearchBot is OpenAI’s crawler for ChatGPT search; GPTBot crawls for model training. On our new site, OAI-SearchBot read only robots.txt for 8 days, then fetched 47 addresses on 11 November 2026. ChatGPT first cited the site about 19 hours later.

OAI-SearchBot vs GPTBot: what OpenAI says

FindingOpenAI’s crawler page lists 4 user agents; the 2 that crawl on their own, OAI-SearchBot for search and GPTBot for training, have separate robots.txt names, and when a site allows both, OpenAI may use 1 crawl for both jobs.

CrawlerWhat OpenAI says it is forIf you block itIP list
OAI-SearchBotSurfacing websites in ChatGPT searchNot shown in ChatGPT search answers (navigational links can still appear)searchbot.json
GPTBotCrawling content that may train OpenAI’s foundation modelsContent should not be used for traininggptbot.json
Source: OpenAI, Overview of OpenAI crawlers (checked 12 November 2026). OpenAI says a robots.txt change can take about 24 hours to affect search.

What OAI-SearchBot requested on our site

FindingOAI-SearchBot’s first 8 verified requests, over 8 days, were all for robots.txt; its first page crawl came 12 days and 19 hours after launch and took 47 addresses in 2 minutes 51 seconds.

Date (UTC)Verified requestsWhat it asked for
3 Nov (first, 06:21)2robots.txt
5 Nov2robots.txt
7, 8, 9 and 10 Nov1 eachrobots.txt
11 Nov, 05:32 to 05:3548robots.txt, then 30 HTML pages, 6 Markdown copies, llms.txt, 3 feeds and 7 WordPress API and embed addresses
12 Nov, to 15:002robots.txt
Source: AI Visibility Census bot log, 29 October 2026 (launch, 10:21 UTC) to 15:00 UTC on 12 November 2026: 58 verified requests in all. Every IP address is on OpenAI’s published OAI-SearchBot list, and every request returned 200. Free to reuse under CC BY 4.0.

The 11 November crawl came from 8 IP addresses, all using version 1.4 of the user agent. It took the home page, Editions, the methodology, the experiments page and 5 older articles, but not this page or What is ClaudeBot?, published 1 and 2 days earlier. For comparison, GPTBot took 156 addresses in 105 seconds on 4 November and came back for 23 more on 10 November (see Does GPTBot run JavaScript?), and Anthropic’s ClaudeBot arrived 49 minutes after launch (see What is ClaudeBot?).

When did ChatGPT first cite this site?

FindingBefore the 11 November crawl, 0 of 106 ChatGPT answers in our checks cited this site; on 12 November ChatGPT cited it in 6 answers, and all 6 pages it cited were in that crawl.

Watchlist questionPages ChatGPT citedAI Checks row
Which Irish professional services firms do ChatGPT and Gemini recommend?Home page and /editions/953
How visible are Irish accountants and solicitors in AI search?Home page and /editions/955
Source: AI Visibility Census AI Checks, watchlist run of 12 November 2026 (ChatGPT, Gemini and Google AI Mode, 26 questions each). Gemini and AI Mode cited the site in 0 of 26. On 5 November all three were 0 of 26.

The 6 citations came between 00:51 and 01:46 UTC on 12 November, about 19 hours after the crawl (ChatGPT’s 76 answers collected on the evening of 11 November cited it 0 times): the 2 watchlist answers above and 4 answers to test phrases that appear only on our pages. Both watchlist answers described our first Edition as due in December 2026. For “What is OAI-SearchBot and how is it different from GPTBot?” ChatGPT cited only OpenAI’s own crawler page, on 5 and 12 November. Our position: the page crawl and the first citations line up, on one site, so they fit the idea that ChatGPT search cites pages OAI-SearchBot has fetched, but they do not prove it; GPTBot had also crawled the site. We ask ChatGPT, Gemini and Google AI Mode the same 26 questions every week until the end of December and will add the date this page is first cited.

How to check OAI-SearchBot in your own logs

FindingAll 47 page-crawl requests on 11 November used user agent version 1.4; 8 of the 11 robots.txt requests used version 1.0.

  1. Filter requests whose user agent contains OAI-SearchBot.
  2. Check each IP address against openai.com/searchbot.json, or let the free crawler log check skill on our Tools page do it. Outside the list means unverified.
  3. Split robots.txt requests from page requests. A log full of robots.txt checks is not a crawl.

What to do

  1. Keep OAI-SearchBot allowed if you want ChatGPT search to show you. A block keeps you out of ChatGPT search answers (navigational links can still appear); a GPTBot block does not.
  2. Watch for the first page crawl, not the first visit. Ours came 8 days after the first robots.txt check, and ChatGPT’s first citations followed within a day.
  3. Measure citations directly. Ask ChatGPT the questions your page answers and record which sources it cites, each week.

Next: the CensusIndex™ checks whether leading sites can be crawled by AI tools; CensusRank™ and our Editions count which brands and firms AI answers name. Get each result by email.

Cite this

On a brand-new site, OpenAI’s OAI-SearchBot made 8 verified requests over 8 days, all for robots.txt, before its first page crawl (47 requests on 11 November 2026); ChatGPT first cited the site about 19 hours later, in 6 answers on 12 November 2026, including 2 of 26 watchlist answers.

Source: AI Visibility Census, “What is OAI-SearchBot?”, https://aivisibilitycensus.com/what-is-oai-searchbot/ (CC BY 4.0)

Frequently asked questions

Is OAI-SearchBot the same as GPTBot?

No. OpenAI lists them separately: OAI-SearchBot for ChatGPT search, GPTBot for model training.

How often does OAI-SearchBot crawl a new site?

On ours, it first checked robots.txt about 116 hours after launch, checked it 7 more times by 10 November, and made its first page crawl on 11 November, 12 days and 19 hours after launch. One site, so treat it as an example, not a rate.

Does blocking GPTBot block OAI-SearchBot?

No. Each has its own robots.txt name, so User-agent: GPTBot with Disallow: / leaves OAI-SearchBot free to crawl. Our robots.txt allows both, so we have not tested whether either obeys a block.

How we counted, and limits

Our server logs every crawler request that reaches WordPress and checks its IP address against the lists that Google, Microsoft, Apple, Anthropic, OpenAI and Perplexity publish. Only requests from OpenAI’s published OAI-SearchBot addresses are counted here. The ChatGPT figures come from our AI Checks: the 26 watchlist questions sent once to ChatGPT through DataForSEO on 5 and 12 November 2026, each answer’s cited links checked for an aivisibilitycensus.com address. The 106 and 6 counts cover every ChatGPT answer our AI Checks collected, through DataForSEO and Bright Data, including test phrases and listing questions. How we count is in the methodology, and every test on this site is on our experiments page.

Limits: this is one new, small site, over 14 days. Our log cannot see whether OpenAI fed GPTBot’s crawls into search, and a request is not use. Requests named ChatGPT-User or OAI-SearchBot from addresses outside OpenAI’s lists are not counted anywhere on this page.

Update log: published 10 November 2026 with data to 15:33 UTC on 9 November. Updated 12 November 2026 with the first page crawl (11 November), the second watchlist run and ChatGPT’s first citations of the site, data to 15:00 UTC on 12 November.

Data on this page is free to reuse under CC BY 4.0; please credit the AI Visibility Census with a link to this page. Spotted an error? Email hello@aivisibilitycensus.com.

Sources: OpenAI: Overview of OpenAI crawlers; OpenAI: published OAI-SearchBot IP addresses (searchbot.json); AI Visibility Census bot log, 29 October to 15:00 UTC on 12 November 2026; AI Visibility Census AI Checks, watchlist runs of 5 and 12 November 2026 (rows 953 and 955) and every ChatGPT check to 15:00 UTC on 12 November.

Published by the AI Visibility Census.

Newsletter

Get the next Census by email

Each new edition, list and experiment result, sent when it publishes. About one email a week. One click to leave.

We send a confirmation email first. If you add a website, we may tell you when your company appears in a Census result. Privacy notice