This site is also a test bed. We use it to measure how search engines and AI tools find, read and cite a brand-new website, and we publish every experiment here before the results are in. When results arrive, they go on this page too, including the ones that show nothing.
Ground rules
We push at the edges of what search and AI systems will do with a page. We do not trick them. On this site:
- Bots and people get the same content. The Markdown versions of our pages carry the same text as the HTML versions.
- There is no hidden text and no instructions aimed at AI models.
- A page’s date changes only when its content changes.
- Every test marker is visible on the page it belongs to.
What we measure
| Signal | How |
|---|---|
| Crawler visits | A server log of automated visitors. Where a company publishes its crawler IP ranges (OpenAI, Perplexity, Google, Microsoft, Apple), each visit is checked against them and marked verified or spoofed. |
| Markdown and llms.txt use | Every request for a .md page, /llms.txt or /llms-full.txt is logged, whoever makes it. |
| Search engine notification | Each IndexNow submission and its response code. |
| Indexing | Google Search Console and Bing Webmaster Tools. Not yet connected; this row will be updated when they are. |
| Citations | The same question asked in ChatGPT, Perplexity, Microsoft Copilot and Google, logged out, on days 1, 3, 7, 14 and 30 after publishing. Screenshots are kept. |
Experiment log
CT-001: how fast is a new article on a new site crawled, indexed and cited?
| Status | Running |
| Started | 29 October 2026, 11:27 UTC |
| Page | Keyword research for AI search: how to find the questions buyers ask |
| Site at start | First published the same morning. No links built to it. |
| Setup | IndexNow sent the article, its Markdown version and llms.txt one second after publishing. Listed in the sitemap and llms.txt. Article and FAQPage structured data. No manual submission to Google or Bing. |
| Test marker | The phrase “Census test marker CT-001” appears on the article and nowhere else we know of. |
Prediction, written before any results: Bingbot reaches the article before Googlebot, and the CT-001 marker turns up in Copilot or Perplexity answers before it turns up in ChatGPT or Google.
| Check | Due | Result |
|---|---|---|
| First verified crawl, by crawler | When it happens | Pending |
| Day 1 citation check | 30 October 2026 | Pending |
| Day 3 | 1 November 2026 | Pending |
| Day 7 | 5 November 2026 | Pending |
| Day 14 | 12 November 2026 | Pending |
| Day 30 | 28 November 2026 | Pending |
CS-001: does a Content Signals line change crawler behavior?
| Status | Running |
| Started | 29 October 2026 |
| Change | Added Content-Signal: search=yes, ai-input=yes, ai-train=yes to robots.txt. |
| Measure | Visits by crawler class (search, AI search, AI user fetch, AI training) per week. |
The site had no traffic history before this change, so there is no “before” to compare against. The plan is to run with all three signals set to yes for at least four weeks, then switch one signal to no for four weeks and compare. The switch will be logged here with its date.
Limits
This is one small website. A crawler arriving sooner on one page does not prove why it arrived, and AI tools change how they search from week to week. We report counts with their dates and say what a result can and cannot show. Treat our findings as signals worth testing on your own site, not as rules.
Questions or a test you want us to run: [email protected].