- AI crawlers: which AI bots visit your website, from your Cloudflare zone. Below it, the robots.txt check.
- AI referrals: how many visits reach your website from AI assistants such as ChatGPT, Perplexity and Gemini, from Google Analytics 4.
Connect Cloudflare
Palagus reads how often known AI bots requested pages of your website. It needs a Cloudflare API token that can read the analytics of one zone, nothing more.The token reads. It cannot change your zone, your DNS or your settings. Palagus needs no write access of any kind, and installs nothing in your Cloudflare account.
- In the Cloudflare dashboard, go to My Profile → API Tokens and select Create Token.
- Give the token a name, for example
Palagus AI Traffic. - Under Policies, open Start from scratch and select Start with a Zone policy. Not a template: each template grants more than Palagus needs.
- In the zone policy, keep Include, select Specific zone, and select your website’s zone. Not All zones.
- In the list of permission groups, open Analytics & Logs and select Read next to Analytics. Nothing else.
- Select Continue to summary, check the summary, and select Create Token.
- Copy the token. Cloudflare shows it only once.
- Copy your Zone ID: go to Domains, select your domain, and scroll to the API section at the bottom of its Overview page.
Connect Google Analytics
Palagus reads how many sessions of your Google Analytics 4 property came from an AI assistant.- Open AI Traffic and go to the AI referrals tab.
- Select Connect Google Analytics.
- Sign in with a Google account that can view your GA4 property, and allow read access to Google Analytics.
- Back in Palagus, pick your property and select Connect property.
When the numbers come
Palagus fetches new numbers once a day. Right after you connect, it also fetches the last 30 days, which takes about twenty minutes, and the tab says so until they are in. After that, it shows when the numbers were last updated. If a fetch finds nothing, the tab says so: the connection works, there were simply no visits in that period.What the numbers mean
AI crawlers counts requests from known AI bots, per bot and day. Each bot shows its vendor and its type:
AI referrals counts sessions in Google Analytics whose source is an AI assistant, per assistant and day. The days are the days of your GA4 property’s time zone.
Each total is compared with the period before it, of the same number of days.
The periods are whole days, up to yesterday: from the last 7 days to all time, no hours. Today is still running, so it is not counted yet. In the chart, a day without visits shows as 0. A day Palagus did not fetch, for example one more than 30 days before you connected, has no point, and the line breaks there.
Where the numbers are incomplete
- AI referrals is a lower bound. Google Analytics knows where a visit came from only when the app passes that on. Many apps, and many links copied out of a chat, do not. Those visits count as direct traffic in Google Analytics, and Palagus cannot see them.
- AI crawlers counts what passes through Cloudflare. Requests to parts of your site that Cloudflare does not proxy are not counted.
- A bot is recognised by the name it sends. A bot that does not send its own name is not counted, and new bots appear only once Palagus knows them.
Check robots.txt
Below the bot numbers on the AI crawlers tab, the Access for AI bots card reads the publicrobots.txt of your project’s website. It shows, for each AI bot:
- Allowed: the bot may read every page.
- Partially blocked: some pages are closed to it.
- Blocked: the whole site is closed to it.
*), or no rule at all. This check needs no connection.
If your website has no robots.txt, every bot may read every page. If the file cannot be read, the page says so. If the project has no website, or its website is an IP address rather than a domain, there is nothing to check.

