AI crawlers

What is Perplexity-User?

Perplexity-User is Perplexity's on-demand AI fetcher. Fetches your page live when a Perplexity user asks.

What does Perplexity-User do with your content?

Fetches your page live when a Perplexity user asks. Like most AI crawlers, Perplexity-User reads your pages as plain HTML — it does not run JavaScript — so content that only appears after scripts execute is invisible to it.

Should you block Perplexity-User?

Blocking Perplexity-User makes your business invisible to Perplexity's AI when customers ask it for recommendations. If you want to be found, allow it; block it only if you have a specific reason to keep Perplexity's AI away from your content.

How do you allow or block Perplexity-User in robots.txt?

Add one of these snippets to the robots.txt file at the root of your site. The first explicitly allows Perplexity-User everywhere; the second blocks it completely.

# Allow Perplexity-User
User-agent: Perplexity-User
Allow: /

# Block Perplexity-User
User-agent: Perplexity-User
Disallow: /

Is robots.txt enough?

Not always. CDNs and firewalls (Cloudflare in particular) can block crawlers at the network level regardless of what robots.txt says — Cloudflare blocks AI crawlers by default for many accounts. If you want Perplexity-User to reach your site, check your CDN's bot settings too. SeeGeo's free audit checks both.

How do I check whether Perplexity-User can read my site right now?

Two checks, both needed: read your robots.txt for a group naming Perplexity-User (or the * group it falls back to), then request your homepage with Perplexity-User's user agent and confirm you get a 200 with your real HTML rather than a 403 or a challenge page. SeeGeo's free crawler check runs both in a few seconds.

Run the free AI crawler check

How does Perplexity-User compare to similar crawlers?

Perplexity-User is one of 16 crawlers the SeeGeo audit checks individually. The closest comparisons:

CrawlerOperatorRoleRuns JavaScript?
Perplexity-UserPerplexityon-demand AI fetcherNo
PerplexityBotPerplexityAI search index crawlerNo
ChatGPT-UserOpenAIon-demand AI fetcherNo
Claude-UserAnthropicon-demand AI fetcherNo

All 16 crawlers, compared · What is an AI crawler?

Why does crawler access matter? The numbers

Crawler access is where AI visibility starts or ends: a blocked crawler can't read you, and an AI that can't read you can't recommend you.

  • Visitors arriving from AI assistants convert at roughly 4.4x the rate of traditional organic search on average (Semrush, cross-industry, 2026).
  • AI-assistant referrals are still only about 1% of total web traffic — small, but the fastest-growing acquisition channel measured (multiple 2026 studies).
  • Adobe Digital Insights (Q1 2026) measured AI-assistant visitors converting 42% better than non-AI traffic — a full reversal from the year before.
  • Cloudflare blocks AI crawlers by default for many accounts, so sites are often invisible to AI without anyone having decided to be.
  • Content edits alone — statistics, citations, quotable structure — can raise AI visibility on the order of 30–40% (Princeton GEO study, KDD 2024).

Frequently asked questions

What is Perplexity-User?

Perplexity-User is Perplexity's on-demand AI fetcher. Fetches your page live when a Perplexity user asks.

Should I block Perplexity-User?

Blocking Perplexity-User makes your business invisible to Perplexity's AI when customers ask it for recommendations. If you want to be found, allow it; block it only if you have a specific reason to keep Perplexity's AI away from your content..

Does Perplexity-User run JavaScript?

No. Perplexity-User, like most AI crawlers, reads raw HTML without executing JavaScript. If your content only renders client-side, it is effectively invisible to it.

Can Perplexity-User read my website?

Only if two things are true: your robots.txt does not disallow Perplexity-User, and your server or CDN actually serves the page when Perplexity-User asks. Firewalls can block it regardless of robots.txt, so the reliable way to know is to fetch your homepage as Perplexity-User — the free check on see-geo.com/ai-crawler-check does exactly that.

Is your site readable to AI right now? SeeGeo's free audit checks crawler access (all 16 bots), structure, and content extractability in under half a minute.

Run a free audit

Related