How to Choose an AI Traffic Analytics Tool: 12 Questions to Ask Any Vendor

A year ago the first question about any analytics tool was whether it could see AI traffic at all. That question has mostly been answered. Google Analytics 4 now has an AI Assistant channel in its default channel group, PostHog has an AI channel type, and Plausible groups known assistants under AI Assistants. Seeing that a visit came from ChatGPT is quickly becoming table stakes.

So the buying question has moved from "does it see ChatGPT" to "what does it do with the visit once it has seen it": how sure it is, what the visit earned, what the crawlers did weeks earlier, and whether you can get the numbers out. Here are twelve questions that separate tools on those points. Each has an answer you can check in the documentation or a one-week trial, and the useful answers are always specific.

1. Which assistants does it name, and from which signals? There are three honest signals for a human visit from an assistant: the referrer header, UTM tags such as utm_source=chatgpt.com, and the user agent. Ask for the list of assistants and ask how a new one gets added. An assistant that launched last month is a good test.

2. Does every visit carry a confidence and a method? A label with no confidence treats a clean referrer and a weak hint as the same thing. You want to be able to filter to the visits the tool is sure about, and to see whether a visit was named by its referrer or by its UTM tag when two reports disagree.

3. What happens to visits that arrive with no referrer? Many assistant clicks reach you with nothing attached (the mechanics are in the referrer is gone). Ask the vendor directly what they do with those visits and what their answer rests on. A tool that leaves them in Direct is being honest about the evidence. A tool that claims to recover them should be able to tell you how it checked that claim.

4. Which payment providers connect natively, and what do you have to build? "Supports Stripe" can mean a native connection that reads your payments, or an event you send from your own code. Both work; they differ in who maintains it.

5. How are refunds handled, and on which date? A refund can rewrite the original day or land on the day it happened, and the choice changes every historical chart. We argue for the second in refunds belong on the day they happen. Whatever the vendor chooses, they should be able to state it in one sentence.

6. Which currency do you report in, and at which day's rate? If you sell in more than one currency, ask whether each order is converted at its own date's rate and whether a multi-day total is converted day by day. Converting a month's total at today's rate reprices your history every time you refresh.

7. Does it log AI crawlers at all, and from where? A crawler does not run your tracking script, so a browser-only tool cannot see a crawl. Crawler logs come from your server, your CDN or an edge integration. Some tools deliberately filter crawlers out of their reports: Plausible, for example, says known AI crawlers are filtered so referral traffic is not mixed with automated requests. That is a reasonable design for a traffic report; it just means crawler analysis lives somewhere else.

8. How is a crawler verified? A user agent is a claim that anyone can type. Google documents verification by reverse DNS and by its published IP ranges, and OpenAI publishes IP range files for its crawlers. Ask which of those checks the tool runs, what share of your crawls it could verify, and what it does with a request that claims to be GPTBot and fails the check. The runbook covers where each method silently fails.

9. For AI visibility, which engines, through which surface, how many runs? Asking a model through its API is not the same as reading the consumer app (the difference matters), and one run per prompt is not a measurement (why it is a range). A good vendor names the engine, the surface and the number of repetitions behind every rate.

10. How long is the data kept, and for which reports? Year-over-year comparison needs more than twelve months. GA4, for example, lets a property keep user-level data for 2 or 14 months, and that setting affects explorations and funnel reports but not the standard aggregated reports. Retention is often per report type, so ask per report.

11. Can you reach it from code and from an assistant? A REST API, an MCP server for assistants such as Claude, a CLI and outbound webhooks are four different ways out. If an assistant can change settings through the tool, ask what stops it doing so without asking you first.

12. What is the billing unit, and what happens at the limit? Pageviews, events, sessions and visitors produce very different bills for the same site. Then ask what the limit does. DataFast, to take one documented example, includes 100,000 accepted crawler requests per account, resets that allowance with the billing cycle, and pauses bot traffic ingestion when it is reached while normal analytics keep working. Paused, degraded and billed are three different contracts, and a crawl quota measures something specific.

Three tests in a trial answer these better than any documentation.

Click your own link. Ask ChatGPT and Perplexity a question that surfaces your site, click through, and find that visit. Check which assistant it was given to, how confident the tool was, and how long it took to appear.

Refund a test order. Buy something in test mode, refund half of it the next day, and look at both days. You will learn the refund date rule and whether counts and revenue moved separately.

Read your crawler log for one page. Pick a page you know is crawled and see whether the tool shows the crawler, whether it verified it, and whether the count matches your server log.

For our own answers, side by side with the tools teams most often run today, see the comparison pages.