CensusIndex™

A monthly check of how ready the leading sites in each category are for AI search, with every check graded by the evidence behind it.

Last updated

Send us your domain
What it checks

In short: the CensusIndex™ checks, once a month, whether the leading websites in a product category can be found and read by search engines and AI tools. Every check carries a grade for the evidence behind it, and only well-evidenced checks count. The first five categories are project management, CRM, help desk, email marketing and social media management software. Their first reports are in preparation.

Monthlyone check of every site in a category
5categories at launch
3 of 5evidence grades count toward a result
Privateeach site’s own results, unless the owner asks

What it checks

Our crawler, CensusIndexBot, reads the same few public files on every site in a category, so the results can be compared month to month:

  • 01 · ACCESS

    Whether the site loads for an identified crawler, and whether the home page can be indexed

  • 02 · ROBOTS.TXT

    Whether robots.txt lets Googlebot, Bingbot and the AI search bots in

  • 03 · RENDERING

    Whether the main content is in the page before JavaScript runs

  • 04 · BASICS

    Whether the XML sitemap, structured data, title and description are in place

  • 05 · AGENT FILES

    Whether llms.txt or a markdown version is offered

  • 06 · TRAINING BOTS

    Which AI training bots are blocked. This is a business choice, so it is reported and never scored.

Why every check has a grade

A lot of advice about AI search has never been tested, so each check is graded by its evidence. Only [T], [S] and [P] checks count toward a site's result. The others are reported with their fix but never move the result. The full rules are on the methodology page.

  • TTested by the CensusCounts
  • SPublished studyCounts
  • PPlatform documentationCounts
  • VVendor claimReported, not scored
  • UUntestedReported, not scored

What gets published

Each monthly report gives category totals, for example how many of the leading sites let AI search bots crawl their home page. Each site's own results stay private unless its owner asks us to make them public. Site owners can ask for a check of their own site.

Categories

  • First report in preparation

    Project management software

  • First report in preparation

    CRM software

  • First report in preparation

    Help desk software

  • First report in preparation

    Email marketing software

  • First report in preparation

    Social media management software

  • YOUR SITE

    Check your own site

    Ask CensusIndexBot to check your site and send you the results.

    How it will work →

The CensusIndex also sets the entry bar for the Census100™: a brand can only be listed if its site passes the basic crawl checks that month.