research

How AI-visible is your website platform by default? What the docs, the research, and the live files actually say

No major website platform blocks AI crawlers by default — and the 'block AI' toggles don't do what owners think. Docs + live-file checks, cited.

Quick answer: none of the six major small-business website platforms — Shopify, Wix, Squarespace, WordPress (.com and self-hosted), Webflow, GoDaddy Website Builder — blocks a single AI crawler by default. We verified that two ways: by reading every platform's own documentation, and by checking the robots.txt of 45 live sites across four of the platforms (zero blocked GPTBot). The differences that actually matter sit elsewhere: two platforms now generate an llms.txt for you without asking (Shopify and Wix), the "block AI" toggles that Squarespace and WordPress.com offer quietly exempt the crawlers that power AI search (so flipping them doesn't remove you from ChatGPT or Perplexity answers), Webflow won't let you edit robots.txt at all without a paid plan, and GoDaddy documents no AI controls whatsoever. This is a research synthesis with light verification, not a lab test — every claim below is cited to a primary source with its retrieval date, and where the evidence is practitioner-grade rather than official, we say so. As far as we can tell, nobody has published a controlled head-to-head platform comparison; we name that gap honestly at the end.

Jump to your platform: Shopify · WordPress · Wix · Squarespace · Webflow · GoDaddy

First, the finding that reframes the question

The scary story — "your website platform is silently blocking AI" — is not what the evidence shows. Large-scale measurement finds AI-crawler blocking concentrated among big publishers: a UC San Diego study presented at the Internet Measurement Conference 2025 found 12–14% of the web's top 5,000 sites fully disallow at least one AI crawler (Liu et al., IMC 2025), and the blocking wave concentrates in publishing — the Reuters Institute counted 48% of top news sites blocking OpenAI's crawlers by the end of 2023 (Fletcher, 2024). Small-business sites behave differently: our own 107-site audit found only 2.8% blocking any major AI crawler.

The platform layer is consistent with that. All six platforms ship permissive robots.txt files, per their own documentation — and our spot-check of 45 live platform sites found zero blocking GPTBot.

The documented technical risk for AI visibility is different: rendering. Vercel and MERJ measured how AI crawlers actually behave and found that none of the major ones execute JavaScript — OpenAI's, Anthropic's, Meta's, and Perplexity's crawlers all fetch JavaScript files but never run them; Google's Gemini (riding Googlebot's infrastructure) is the main exception (Vercel, Dec 2024). A site whose content only exists after JavaScript runs is invisible to most AI systems.

Here's the twist for platform users: that risk mostly spares you. Wix states outright that its infrastructure is server-side rendered (Wix SEO features); Webflow publishes compiled static HTML (practitioner-documented — Webflow's own docs don't state the rendering model); Squarespace serves core content server-side (practitioner-observed, same caveat); and the GoDaddy sites we fetched carry their full text in the raw HTML. The JavaScript-invisibility problem concentrates in custom-built sites, not platform sites. If you're on a major platform, your AI-visibility problems are almost certainly content and clarity problems, not plumbing.

The comparison at a glance

Platform AI bots blocked by default "Block AI" setting llms.txt robots.txt editable
Shopify None No toggle — theme template edit only Auto-generated (since May 2026) Via robots.txt.liquid (unsupported customization)
WordPress.com None Opt-out toggle, off by default — doesn't block OAI-SearchBot No No (platform-managed)
WordPress self-hosted None Nothing in core; SEO plugins offer manual tools Via plugins (AIOSEO: on by default; Yoast/Rank Math: opt-in) Yes
Wix None No toggle — manual robots.txt edit Auto-generated (premium + custom domain) Yes
Squarespace None Toggle, off by default — doesn't block PerplexityBot, OAI-SearchBot, or ChatGPT-User Manual, disabled by default No (platform-managed)
Webflow None No toggle — and robots.txt editing requires a paid plan Manual upload, custom domains only Paid plans only
GoDaddy Website Builder None (observed) None documented None documented Not documented

Sources for every cell are linked in the platform sections below. Retrieved 2026-08-18; platforms change defaults silently — see the update policy at the end.

The toggle finding: training opt-outs dressed as visibility opt-outs

Two platforms offer a "block AI" checkbox. Both lists have the same hole, and it's the most consequential detail in this report.

Squarespace's "Block known artificial intelligence crawlers" setting (off by default) writes robots.txt rules for 26 bots — GPTBot, ClaudeBot, Google-Extended, CCBot, Bytespider, and more. Not on the list: PerplexityBot, OAI-SearchBot, and ChatGPT-User (Squarespace help, retrieved 2026-08-18).

WordPress.com's "Prevent third-party sharing" setting (also off by default) appends blocks for thirteen bots including GPTBot, ClaudeBot, and PerplexityBot — but not OAI-SearchBot (WordPress.com support, confirmed against a live opted-out site's robots.txt).

Why that matters: OpenAI and Perplexity split their crawlers by purpose — GPTBot gathers training data, while OAI-SearchBot and ChatGPT-User power live search and citations. (Most platform guides miss this distinction entirely.) So an owner who flips either toggle has opted out of model training — but their site still appears in ChatGPT search results and Perplexity answers. If you flipped the toggle to stay visible while limiting training use, that's arguably exactly what you wanted — but nothing in either interface tells you that's the deal you're getting.

llms.txt became a platform decision while nobody was looking

In our 107-site study, 25.2% of small-business sites had an llms.txt file — and almost none of the owners put it there. The platform docs now confirm the mechanism:

  • Shopify serves /llms.txt (mirroring a primary /agents.md) on every store, customizable via theme templates (Shopify changelog, May 2026).
  • Wix "automatically generates and maintains" llms.txt for premium sites with custom domains, with an opt-out toggle (Wix help). Our spot-check found 6 of 7 live Wix sites serving a ~3KB llms.txt.
  • All in One SEO, one of WordPress's biggest SEO plugins, generates llms.txt by default once installed (AIOSEO docs); Yoast and Rank Math offer it opt-in.
  • Squarespace (manual, disabled by default) and Webflow (manual upload, which candidly calls the format "experimental") leave it to the owner. GoDaddy doesn't mention it.

Whether llms.txt matters is a separate question — no major AI company has committed to reading it — but adoption statistics now measure platform product decisions, not site-owner behavior. Any study (including ours) that counts llms.txt files is partly counting Shopify and Wix market share.

Shopify

Everything is allowed by default and bot management happens at Shopify's network layer — their docs say stores are readable by "search engines and large language models" with no action needed (crawling docs). There is no admin toggle for AI crawlers — posts describing one are talking about merchants' own Cloudflare accounts. Control means editing a theme template (robots.txt.liquid), which Shopify calls an unsupported customization. The subtle one: robots.txt only guards one of two doors. Shopify Catalog syndicates eligible product data to AI shopping surfaces (ChatGPT, Microsoft Copilot) by default, and blocking crawlers doesn't stop it — that has its own per-platform opt-outs. A practitioner failure mode worth checking on older stores: a 2023-era "block all AI bots" snippet copy-pasted into robots.txt.liquid and forgotten.

WordPress (.com and self-hosted)

Core WordPress ships a two-line robots.txt that blocks nothing but the admin area (developer reference), and none of the three big SEO plugins (Yoast, AIOSEO, Rank Math) blocks any AI bot by default. WordPress.com adds the opt-out toggle described above — off by default, and porous to OpenAI's search crawler when on. Self-hosted owners hold full control and full responsibility: the same 2023 blocklist snippet circulates widely in WordPress tutorials (the canonical version dates to August 2023), and a copied-then-forgotten robots.txt is the likeliest way a small business is accidentally invisible today.

Wix

Permissive by default with a real robots.txt editor, no AI toggle (blocking is a documented manual edit), first-party SSR — and the auto-generated llms.txt above. Of the six, Wix currently ships the most AI-visibility infrastructure without being asked.

Squarespace

Platform-managed robots.txt you can't edit directly, with the off-by-default toggle — and one detail our live-file check surfaced: Squarespace's generated robots.txt names the AI-bot roster explicitly on every site we checked (15/15), with path-level rules that don't block content. The scaffolding is pre-built; the toggle just flips it. Remember what the toggle doesn't cover (Perplexity, OpenAI search) before treating it as an off switch.

Webflow

The default robots.txt contains only a sitemap line — nothing blocked. The catch is access: adding robots.txt rules requires a paid Site plan, making Webflow the only platform here where blocking is a premium feature. Webflow also documents a Content-Signal HTTP header for granular AI preferences (train/search/input) — new, and its real-world effect depends entirely on crawler cooperation.

GoDaddy Website Builder

The least documented surface of the six: no robots.txt documentation, no AI setting, no llms.txt for Websites + Marketing sites (their SEO help article mentions none of it). Live sites we fetched serve a near-empty robots.txt and server-rendered HTML — fine defaults, zero control. Watch item: GoDaddy announced a Cloudflare partnership (April 2026) to integrate AI Crawl Control into its hosting platform; whether it reaches Website Builder, and with what defaults, is unannounced.

What we checked ourselves, and the gap we're naming

The documentation claims above were spot-verified against reality: we fetched robots.txt and llms.txt (public files only, honest user-agent, rate-limited, aggregate reporting) from 45 live sites fingerprinted as Shopify (8), WordPress (15), Wix (7), and Squarespace (15). Results: zero sites blocked GPTBot; every Squarespace robots.txt named the AI-bot roster; 8/8 Shopify and 6/7 Wix sites served llms.txt versus 1/15 WordPress and 0/15 Squarespace — the platform-decision finding, visible in the wild. (Webflow and GoDaddy verification is pending a large enough site sample; their sections above rest on documentation and small-n observation.)

What nobody has published — us included — is a controlled head-to-head test: the same business, same content, built on each platform, audited identically. Everything above is synthesis of documentation plus observation of sites that differ in a thousand ways besides their platform. We checked for such a study and couldn't find one; if it exists, we'll link it here. If it continues not to exist, running it is on our roadmap.

One more honesty note, borrowed from a practitioner who re-tested his own AI-crawler findings six months later and found 11 of 17 had changed: this field rots fast. Cloudflare alone changed AI-crawler defaults for new domains in July 2025 and again for September 2026. Every claim in this report carries its retrieval date (2026-08-18), and we re-verify quarterly — the changes become their own report.

What to actually do, whatever your platform

Your platform's defaults are almost certainly not your problem — which means the fixes are yours to make, not theirs. Check the three things that actually separated visible from invisible sites in our audit data: whether your homepage plainly says what you do and where, whether you have any structured data, and whether a stale blocklist snippet is sitting in your robots.txt. Or let the free SeeGeo audit check all of it — including your actual, live robots.txt as the crawlers see it — in about 30 seconds.

Frequently asked questions

Does Shopify block ChatGPT or other AI crawlers? No. Shopify's default robots.txt blocks no AI crawlers, and Shopify states stores are readable by LLMs with no merchant action. There's no admin toggle either way — blocking requires editing a theme template. Separately, Shopify Catalog shares eligible product data with AI shopping surfaces by default, governed by its own opt-outs, not robots.txt.

If I turn on Squarespace's or WordPress.com's "block AI" setting, will I disappear from ChatGPT? Mostly no. Both toggles block training crawlers (GPTBot, ClaudeBot, CCBot, and others) but leave out the crawlers that power live AI search — OAI-SearchBot and ChatGPT-User on both platforms, plus PerplexityBot on Squarespace. Your content can still be retrieved and cited in AI answers with the toggle on.

Which website builder is best for AI visibility? By defaults alone, none of them blocks AI, so "best" comes down to what's built for you versus locked from you: Wix and Shopify generate llms.txt automatically and serve crawler-readable HTML; Webflow and Squarespace are solid but leave AI files manual (and Webflow paywalls robots.txt control); GoDaddy offers no documented AI controls. No platform choice substitutes for clear content — that's where small-business sites actually fail.

Do I need an llms.txt file? It's cheap insurance, not a ranking lever — no major AI company has committed to reading it. If your platform generates one (Shopify, Wix, or WordPress with AIOSEO), check what's in it rather than adding one. The interesting fact is that adoption statistics now mostly measure platform defaults, not owner intent.

How current is this report? All sources and live-file checks are dated 2026-08-18, and each claim links its source. Platforms change defaults silently — we re-verify quarterly and publish what changed.

Want to know where you stand? SeeGeo audits your site for search and AI visibility, then tells you what to fix — in plain language.

Run a free audit