--- title: "What is Google-Extended?" description: "What Google-Extended (Google) does with your content, whether to block it, and the exact robots.txt lines — in plain language." canonical: https://see-geo.com/bots/google-extended language: en published: 2026-09-08 updated: 2026-09-08 publisher: SeeGeo alternates: fr: https://see-geo.com/fr/bots/google-extended de: https://see-geo.com/de/bots/google-extended --- # What is Google-Extended? Google-Extended is Google's AI training crawler. Controls whether Google may train Gemini on your content. Blocking it does NOT affect Google Search or AI Overviews. ## What does Google-Extended do with your content? Controls whether Google may train Gemini on your content. Blocking it does NOT affect Google Search or AI Overviews. Like most AI crawlers, Google-Extended reads your pages as plain HTML — it does not run JavaScript — so content that only appears after scripts execute is invisible to it. ## Should you block Google-Extended? Blocking Google-Extended only opts your content out of Gemini model training. It does NOT remove you from Google Search or from AI Overviews — both of those run on Googlebot. If your goal is staying visible in Google while opting out of model training, blocking Google-Extended is a reasonable, low-risk choice. ## How do you allow or block Google-Extended in robots.txt? Add one of these snippets to the robots.txt file at the root of your site. The first explicitly allows Google-Extended everywhere; the second blocks it completely. ``` # Allow Google-Extended User-agent: Google-Extended Allow: / # Block Google-Extended User-agent: Google-Extended Disallow: / ``` ## Is robots.txt enough? Not always. CDNs and firewalls (Cloudflare in particular) can block crawlers at the network level regardless of what robots.txt says — Cloudflare blocks AI crawlers by default for many accounts. If you want Google-Extended to reach your site, check your CDN's bot settings too. SeeGeo's free audit checks both. ## How do I check whether Google-Extended can read my site right now? Two checks, both needed: read your robots.txt for a group naming Google-Extended (or the * group it falls back to), then request your homepage with Google-Extended's user agent and confirm you get a 200 with your real HTML rather than a 403 or a challenge page. SeeGeo's free crawler check runs both in a few seconds. [Run the free AI crawler check](https://see-geo.com/ai-crawler-check) ## How does Google-Extended compare to similar crawlers? Google-Extended is one of 16 crawlers the SeeGeo audit checks individually. The closest comparisons: | Crawler | Operator | Role | Runs JavaScript? | |---|---|---|---| | Google-Extended | Google | AI training crawler | No | | Googlebot | Google | search engine crawler | Delayed rendering | | GPTBot | OpenAI | AI training crawler | No | | ClaudeBot | Anthropic | AI training crawler | No | | CCBot | Common Crawl | AI training crawler | No | [All 16 crawlers, compared](https://see-geo.com/bots) · [What is an AI crawler?](https://see-geo.com/glossary/ai-crawler) ## Why does crawler access matter? The numbers Crawler access is where AI visibility starts or ends: a blocked crawler can't read you, and an AI that can't read you can't recommend you. - Visitors arriving from AI assistants convert at roughly 4.4x the rate of traditional organic search on average (Semrush, cross-industry, 2026). - AI-assistant referrals are still only about 1% of total web traffic — small, but the fastest-growing acquisition channel measured (multiple 2026 studies). - Adobe Digital Insights (Q1 2026) measured AI-assistant visitors converting 42% better than non-AI traffic — a full reversal from the year before. - Cloudflare blocks AI crawlers by default for many accounts, so sites are often invisible to AI without anyone having decided to be. - Content edits alone — statistics, citations, quotable structure — can raise AI visibility on the order of 30–40% (Princeton GEO study, KDD 2024). ## Frequently asked questions ### What is Google-Extended? Google-Extended is Google's AI training crawler. Controls whether Google may train Gemini on your content. Blocking it does NOT affect Google Search or AI Overviews. ### Should I block Google-Extended? Blocking Google-Extended only opts your content out of Gemini model training. It does NOT remove you from Google Search or from AI Overviews — both of those run on Googlebot. ### Does Google-Extended run JavaScript? No. Google-Extended, like most AI crawlers, reads raw HTML without executing JavaScript. If your content only renders client-side, it is effectively invisible to it. ### Can Google-Extended read my website? Only if two things are true: your robots.txt does not disallow Google-Extended, and your server or CDN actually serves the page when Google-Extended asks. Firewalls can block it regardless of robots.txt, so the reliable way to know is to fetch your homepage as Google-Extended — the free check on see-geo.com/ai-crawler-check does exactly that. ### Does blocking Google-Extended affect Google Search rankings? No. Google-Extended only controls Gemini model training. Google Search and AI Overviews use Googlebot, so blocking Google-Extended has no effect on either. ## Related pages - [What is Googlebot?](https://see-geo.com/bots/googlebot) - [What is GPTBot?](https://see-geo.com/bots/gptbot) - [What is ClaudeBot?](https://see-geo.com/bots/claudebot)