---
title: "SeeGeo open data"
description: "Aggregate results of SeeGeo's August 2026 audit of 107 real small-business websites in seven countries: how many block AI crawlers (per bot, explicit and effective), how many carry structured data, how many state what the business does, llms.txt adoption, grade distribution and per-platform splits. Counts only — no site is named. CC BY 4.0."
canonical: https://see-geo.com/data
language: en
published: 2026-08-17
updated: 2026-09-08
publisher: SeeGeo
---

# SeeGeo open data

Aggregate results of SeeGeo's August 2026 audit of 107 real small-business websites in seven countries: how many block AI crawlers (per bot, explicit and effective), how many carry structured data, how many state what the business does, llms.txt adoption, grade distribution and per-platform splits. Counts only — no site is named. CC BY 4.0.

- Download: https://see-geo.com/data/ai-visibility-study-2026.json (application/json, 2782 bytes)
- Repository: https://github.com/Tunisian-Aaron/ai-visibility-study-2026
- License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
- Published 2026-08-17, updated 2026-09-08
- Cite as: SeeGeo, AI visibility study of 107 small-business websites, 2026.

## Fields

| Key | Meaning |
|---|---|
| `generatedAt` | ISO timestamp of the aggregation run. |
| `sampleSize` | Candidate domains collected before exclusions. |
| `audited` | Sites actually audited; the denominator for every share. |
| `exclusions` | Why candidates were dropped: unreachable, moved, or blocked the checker (never counted as blocking AI). |
| `robots.served` | Sites that returned a robots.txt. |
| `robots.blocksAnyAiExplicit` | Sites whose robots.txt names and disallows at least one major AI crawler. |
| `robots.blocksAnyAiEffective` | Same, also counting wildcard rules that happen to catch AI bots. |
| `robots.perBot` | Explicit and effective block counts per crawler (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot, OAI-SearchBot, Googlebot, Bingbot). |
| `robots.causeBreakdown` | Blocking sites split by whether CDN/WAF signals were also present. |
| `cdn` | Sites behind Cloudflare or showing WAF signals, where network-level blocking is possible regardless of robots.txt. |
| `content.whatWeDoStatement` | Homepages that plainly state what the business does near the top. |
| `content.noJsContentMissing` | Sites whose main content disappears without JavaScript. |
| `schema` | JSON-LD presence: none anywhere, on the homepage, beyond the homepage. |
| `llmsTxt.present` | Sites serving an llms.txt file. |
| `platforms` | Per-platform counts; blocksAnyAiExplicit and noSchema are null where n is too small to report (sufficient=false). |
| `cloudflareSplit` | Explicit AI blocking among Cloudflare-fronted vs other sites. |
| `grades` | Distribution of SeeGeo audit grades A–F across the audited sites. |

## Figures

SeeGeo audited 107 real small-business websites (restaurants, trades, clinics, shops, professional services) in United States, United Kingdom, Ireland, Canada, Australia, France, Germany in August 2026. Sample of 120; 8 unreachable, 3 moved and 2 that blocked the checker were excluded and never enter a denominator. Convenience sample: the figures describe these 107 sites on this date, nothing more.

## Headline figures

| Question | Answer | Count |
|---|---|---|
| What share of small-business sites explicitly block at least one major AI crawler in robots.txt? | 2.8% | 3 of 107 |
| How many block any AI crawler counting wildcard rules too? | 2.8% | 3 of 107 |
| How many have no JSON-LD structured data anywhere? | 27.1% | 29 of 107 |
| How many carry structured data on the homepage? | 70.1% | 75 of 107 |
| How many homepages plainly say what the business does? | 35.5% | 38 of 107 |
| How many never say what they do? | 64.5% | 69 of 107 |
| How many serve an llms.txt file? | 25.2% | 27 of 107 |
| How many lose their content without JavaScript? | 2.8% | 3 of 107 |
| How many sit behind Cloudflare? | 29.0% | 31 of 107 |
| How many graded C or worse on SeeGeo's audit? | 94.4% | 101 of 107 |

## Blocking per crawler

| Crawler | Explicitly blocked | Effectively blocked | Share of sites |
|---|---|---|---|
| [GPTBot](https://see-geo.com/bots/gptbot) | 3 of 107 | 3 of 107 | 2.8% |
| [OAI-SearchBot](https://see-geo.com/bots/oai-searchbot) | 0 of 107 | 0 of 107 | 0.0% |
| [ClaudeBot](https://see-geo.com/bots/claudebot) | 3 of 107 | 3 of 107 | 2.8% |
| [PerplexityBot](https://see-geo.com/bots/perplexitybot) | 1 of 107 | 1 of 107 | 0.9% |
| [Google-Extended](https://see-geo.com/bots/google-extended) | 2 of 107 | 2 of 107 | 1.9% |
| [CCBot](https://see-geo.com/bots/ccbot) | 2 of 107 | 2 of 107 | 1.9% |
| [Googlebot](https://see-geo.com/bots/googlebot) | 0 of 107 | 0 of 107 | 0.0% |
| [Bingbot](https://see-geo.com/bots/bingbot) | 0 of 107 | 0 of 107 | 0.0% |

Of the 31 Cloudflare-fronted sites, 1 also block in robots.txt; of the 76 others, 2 do.

## Grade distribution

| Grade | Sites | Share |
|---|---|---|
| A | 0 | 0.0% |
| B | 6 | 5.6% |
| C | 38 | 35.5% |
| D | 38 | 35.5% |
| F | 25 | 23.4% |

Grades are anchored to this distribution: an A is the top decile of real sites, so a B is already a good result.

## By platform

| Platform | Sites | Block any AI crawler | No structured data |
|---|---|---|---|
| wordpress | 38 | 0 | 4 |
| custom/unknown | 34 | 2 | 20 |

Only platforms with enough sites for a fair comparison are shown; smaller groups are in the JSON with null figures.

## Method and license

Each site was crawled once by SeeGeo's audit engine the way an AI crawler reads it — honest user agent, no JavaScript, homepage plus up to about eight pages, robots.txt, sitemap and an llms.txt probe — and scored deterministically. "Blocks bot X" means robots.txt names X and disallows the site root; wildcard-only rules are counted separately as "effective". Individual businesses are never named. Aggregate generated 2026-08-17.

- Data (JSON): https://see-geo.com/data/ai-visibility-study-2026.json
- Dataset page with field dictionary: https://see-geo.com/data
- Repository with methodology and findings: https://github.com/Tunisian-Aaron/ai-visibility-study-2026
- Full write-up: https://see-geo.com/blog/how-many-websites-block-ai-crawlers
- License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
- Cite as: SeeGeo, AI visibility study of 107 small-business websites, 2026.