How to track AI crawler and referral traffic
See which AI bots crawl your site and how much traffic AI engines send back. A guide to AI traffic analytics in Orchly.
AI engines touch your site in two ways. Their crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and others) read your pages so they can answer questions about you. Their products also send real people to your site as referral visits.
AI traffic analytics shows you both, split from your human traffic, so you can see how much AI is actually driving.
You’ll find it under Organic & Social Analytics → AI Traffic Analytics, on the Analytics tab.
Why AI traffic matters
For most sites, AI is a fast-growing and almost invisible channel. Standard analytics tools miss it, because AI crawlers rarely run JavaScript and AI referrals get lumped in with “direct” or “other.”
Watching it matters for two reasons:
- Crawls are the on-ramp to AI answers. Before ChatGPT or Perplexity can recommend you, its crawler has to read your pages. Rising bot visits to a page are an early signal you’re on the path to being cited.
- AI referrals are real buyers. Someone who clicks through from an AI answer has already been pre-sold by a trusted source. That visit is worth understanding and growing.
AI traffic data comes from your site’s request logs, so you need to connect a source before anything shows up. See connect a CMS to track AI traffic.
Bot vs human, per platform
The cards across the top split traffic into Bot and Human for each AI platform: ChatGPT, Perplexity, Claude, and Gemini, plus an All Platforms total. Each number carries its change over the period, so you can tell whether a given engine is crawling you more or sending more people.
- Bot visits are the AI crawlers reading your content.
- Human visits are real people arriving from an AI product, like someone clicking through from a ChatGPT answer.
The chart below plots both over time, and you can toggle Bot and Human on or off to focus.
The AI bots you’ll see
Different bots visit for different reasons. Knowing which is which tells you whether AI is training on you, answering with you, or just indexing you:
| Bot | Who it belongs to | Why it visits |
|---|---|---|
| GPTBot | OpenAI | Crawls pages to improve its models |
| ChatGPT-User | OpenAI | Fetches a page live to answer a user right now |
| ClaudeBot | Anthropic | Crawls pages for Claude |
| PerplexityBot | Perplexity | Crawls for Perplexity’s answer engine |
| Google-Extended | Controls whether your content is used for Gemini and AI Overviews | |
| Googlebot / Bingbot | Google / Microsoft | Traditional search indexing |
A live, per-request view of exactly which bot hit which page is in the Traffic log. See how to read AI traffic analytics and logs.
Break it down
Under the chart, three tabs slice the data:
- Platforms, which bots and engines hit you most.
- Pages, which of your URLs get the most AI attention.
- Countries, where that traffic comes from.
Each has an AI / Human toggle and an Export button.
A jump in bot visits to a page often comes before that page starts getting cited in AI answers. Cross-reference your most-crawled pages with your AI citations to see what’s converting from crawl to mention.
What to do with the data
A few ways to act on what you find:
- Double down on what AI reads. If the engines crawl a handful of pages far more than the rest, those are your strongest assets. Keep them fresh and link to your weaker pages from them.
- Spot pages AI ignores. Important pages with little bot traffic may have crawl or content issues. Check them in Pages.
- Track the referral trend. Growing human visits from AI products means your visibility work is paying off in real clicks.
- Catch sudden changes. A drop in bot visits can mean a block, a broken page, or a robots rule that needs a look.
Frequently asked questions
Why doesn’t Google Analytics show AI bot traffic?
Most AI crawlers don’t run JavaScript, so the GA tracking script never fires for them. Orchly reads your server or edge request logs instead, so it captures the bots GA can’t see.
Should I block AI bots?
Usually not, if you want to appear in AI answers. Blocking crawlers like GPTBot or ClaudeBot in your robots.txt can remove you from those engines. Block only the bots you have a specific reason to keep out.
What’s the difference between AI crawler traffic and AI referral traffic?
Crawlers are bots reading your pages so an engine can learn or answer. Referrals are humans arriving on your site after clicking a link in an AI answer. Orchly separates the two so you can read each clearly.
How do I get more AI referral traffic?
Earn more citations in AI answers. The more often engines recommend and link to you, the more people click through. See how to track AI visibility and AI citations.