--- title: "How do I unblock AI crawlers on Imperva?" description: "Imperva (Incapsula) challenges suspected bots and blocks unknown ones, hiding you from AI assistants. The settings to change, and how to verify." canonical: https://see-geo.com/unblock/imperva language: en published: 2026-09-02 updated: 2026-09-02 publisher: SeeGeo --- # How do I unblock AI crawlers on Imperva? On Imperva Cloud WAF (the product many still know as Incapsula), AI crawlers are blocked when Bot Access Control blocks their known-bot category, when "challenge suspected bots" is on and the crawler isn't a recognized good bot, or when a security rule matches its user-agent — and the fix is to allow the search-engine and AI-crawler categories, add an allow rule for anything unrecognized, and verify with a crawler-user-agent request. Changes propagate within minutes. ## Why is Imperva blocking AI crawlers on my site? Imperva classifies clients as humans, good bots (a known-bots list with categories), and bad or suspected bots, and its default policy blocks bad bots and challenges suspected ones with a JavaScript test no crawler can pass. An AI crawler that isn't on the good-bots list, or whose category is set to block, receives Imperva's "Request unsuccessful. Incapsula incident ID" page — and the response carries the x-iinfo header, which is how SeeGeo's audit recognizes an Imperva wall. Imperva added AI and LLM crawler categories to its known-bots handling as those crawlers became common, so most accounts can now make one category-level decision; before that, each crawler needed its own exception. ## Where does the block live in the Imperva Cloud Security Console? Paths are current as of mid-2026; Imperva has reorganized the console more than once, so search the settings for these terms if the menus differ. - Websites → your site → Security → Bot Access Control: the known-bots categories (search engines, monitoring, AI / LLM crawlers…) with Allow or Block per category, plus the "Block bad bots" and "Challenge suspected bots" switches. - Security → Security Rules (or Policies → Custom Rules): conditions on client type, user-agent, URL or country with an Allow, Block or Challenge action — where blanket "Crawler → Block" rules live. - Security → Allowlist / Exceptions: IP, URL or user-agent exceptions to the policies above. - Advanced Bot Protection (if licensed): its own policies per path, which sit on top of Bot Access Control. ## How do I allow AI crawlers on Imperva, step by step? Save each change; Imperva applies them to its edge within a few minutes with no separate activation step. - 1. Bot Access Control → known bots: set Search Engines and the AI / LLM crawler category to Allow. If your console shows separate assistant and scraper categories, allow assistants and decide scrapers on policy. - 2. If "Challenge suspected bots" is on, keep it — but add an exception so recognized crawlers aren't treated as suspected: Security Rules → new rule → client type is Crawler and user-agent contains OAI-SearchBot (repeat or combine for ChatGPT-User, ClaudeBot, PerplexityBot) → action Allow. - 3. Security Rules: read any rule with action Block or Challenge that matches client type Crawler or a broad user-agent pattern, and exempt the crawlers above. - 4. If Advanced Bot Protection is enabled, check its per-path policies for the homepage and key pages — they can override the site-level allow. - 5. Verify with the crawler-user-agent requests below: a 200 with your HTML and no x-iinfo header means the crawler is through. - 6. Re-run the audit and confirm the crawler table. ## Which AI crawlers should I allow, and which can I keep blocking? The distinction that matters is search versus training. Search crawlers (OAI-SearchBot, ChatGPT-User, PerplexityBot, ClaudeBot, Googlebot) are what makes an assistant able to find and cite you; blocking them makes you invisible in AI answers. Training crawlers (GPTBot, Google-Extended, CCBot) feed model training and blocking them costs you nothing in visibility today. Most "Imperva blocks AI bots" settings treat both groups as one, which is exactly why owners who only meant to opt out of training end up invisible. | Crawler | What it feeds | Allow? | |---|---|---| | OAI-SearchBot | ChatGPT search answers and citations | Yes — this is the one that recommends you | | ChatGPT-User | Live fetches when a user asks ChatGPT about a page | Yes | | GPTBot | OpenAI model training | Your call — no effect on being cited today | | ClaudeBot / Claude-User | Anthropic's index and live fetches for Claude | Yes | | PerplexityBot / Perplexity-User | Perplexity answers and citations | Yes | | Googlebot | Google Search AND AI Overviews / AI Mode | Yes — blocking it removes you from Google entirely | | Google-Extended | Gemini training (not Search) | Your call | | Bytespider, CCBot | Third-party scrapers and training sets | Block if you like — no visibility cost | [Every crawler, one page each: what it is and how to control it](https://see-geo.com/bots) · [The platform toggles that block the wrong crawlers](https://see-geo.com/blog/website-platform-ai-visibility-defaults) ## How do I verify the wall is actually open? Test from outside, as a crawler would — not from your browser, which is exactly the client the wall was built to admit. Run these from any terminal (or an online HTTP tester) and compare the responses: A healthy answer is a 200 status with your real HTML. A 403, a challenge page, or a response with the vendor's mitigation header means the crawler is still blocked. If your wall verifies bots by IP range rather than user-agent, a spoofed user-agent from your laptop may still be challenged even though the real crawler gets through — in that case the definitive test is the vendor's own bot analytics, or simply re-running the audit and checking the crawler table. ``` # As ChatGPT's search crawler: curl -sI -A "OAI-SearchBot/1.0" https://yoursite.com/ | head -5 # As Claude's crawler: curl -sI -A "ClaudeBot/1.0" https://yoursite.com/ | head -5 # As a plain browser, for comparison: curl -sI -A "Mozilla/5.0" https://yoursite.com/ | head -5 # SeeGeo's own crawler, if you want the audit itself to get through: curl -sI -A "SeeGeoAudit/1.0" https://yoursite.com/ | head -5 ``` [Re-run the free audit — the crawler table is the receipt](https://see-geo.com/) ## Frequently asked questions ### What is the x-iinfo header? It's the response header Imperva's edge attaches to requests it handled, and it appears prominently on its block and challenge pages ("Request unsuccessful. Incapsula incident ID…"). A crawler-user-agent request that comes back with that page was stopped at Imperva's edge and never reached your site. ### Is Imperva Cloud WAF the same thing as Incapsula? Yes. Incapsula was acquired and rebranded as Imperva Cloud WAF; the challenge pages, incident IDs and some console labels still carry the Incapsula name, which is why SeeGeo's audit reports the wall as "Imperva/Incapsula". ### Can I allow AI crawlers on some pages only? Yes. Security rules and Advanced Bot Protection policies can be scoped by URL, so you can allow crawlers on your public content pages while keeping stricter handling on login, checkout or API paths — which is usually the right shape: the pages you want cited are exactly the public ones. ## Related pages - [How do I unblock AI crawlers on Cloudflare?](https://see-geo.com/unblock/cloudflare) - [How do I unblock AI crawlers on Akamai?](https://see-geo.com/unblock/akamai) - [How do I unblock AI crawlers on DataDome?](https://see-geo.com/unblock/datadome)