GPTBot, ClaudeBot, PerplexityBot and 175+ more — who visits, what they fetch, and whether your robots.txt covers them. No Cloudflare account, no guessing.
free forever core · no account needed · nothing leaves your site unless you opt in
How it works
Install like any plugin. CrawlTally starts counting immediately — no configuration, no account.
From your WordPress admin, Plugins → Add New. The fingerprint library ships inside the plugin — it works offline.
“GPTBot made 1,204 requests this week (+38%)” — trends, first-seen dates and top crawled paths, in plain English.
Audit your robots.txt per AI crawler, then generate a new one in one click: block training bots, keep search bots citable.
Features
Built for WordPress long-tail sites — readable answers, not a security dashboard.
Every known AI crawler and agent with vendor, purpose (training / search / agent) and official docs link. Updated weekly from two open upstreams — ships inside the plugin.
Per-bot verdicts and one-click generation with sane presets — “block training, allow search” recommended.
“This week: 6 AI bots, 3,200 requests. ClaudeBot visited for the first time.” Opt-in only, aggregated counters, no visitor IPs.
Page cache or CDN eating your numbers? The never-cached census endpoint keeps “which bots visit” measurable even behind full-page caches.
Behind aggressive caching? Import your Nginx / Apache access logs for cache-immune, byte-accurate history. Runs on your server via WP-CLI — the data never leaves it.
User-agent-level detection (and we say so), lower-bound counts behind caches (we label it), and we won’t sell you a llms.txt generator — independent measurements show it barely matters. You get the truth, not snake oil.
Pricing
Start free. Upgrade only if you want it. No subscriptions unless you choose one.
Secure checkout by Creem (Merchant of Record) · VAT handled · License key arrives by email in minutes.
FAQ
Requests served from cache never reach PHP, so request counts are a lower bound on cached sites — CrawlTally detects your cache and says so in the dashboard. Which bots visit and first-seen dates stay reliable, and the census beacon plus Pro log import keep detection complete.
Cloudflare sees every request at the edge; a WordPress plugin sees what reaches your server. If Cloudflare caches your HTML, the difference is exactly your cache hit rate. Note that Cloudflare's default does not cache HTML — most sites see near-identical totals.
No. Independent measurements (500 million AI-bot visits, 408 llms.txt reads) show it has almost no effect on AI visibility. We check whether you have one and tell you the truth instead of selling snake oil.
Nothing, by default. If you opt in to cloud reports, only aggregated counters do: bot ID, path section, hit counts. No visitor IPs, no full URLs, no content.
Yes — identification is user-agent based, and a spoofed UA counts as that bot. For most site owners this precision is exactly right; IP-range verification is on the roadmap.
Free forever core, one-minute setup, honest numbers.
Download free from WordPress.orgworks with any cache plugin · any CDN · any host