Six Numbers That Tell You If AI Search Is Working

Most AI search reporting is one number: sessions from AI sources, plotted over time. It goes up, someone says the strategy is working. It goes down, and nobody can say why, because a single number at the end of a five-stage chain cannot tell you which stage failed.

Six numbers can. They are ordered, each one is the precondition for the next, and together they turn "AI traffic is down" from an anxiety into a diagnosis. Five of the six you can compute today from data your own systems already produce or could produce this week.

The chain

1. Crawl coverage

Definition. The share of your commercially important URLs fetched by a search-index crawler within the last 30 days.

Why it is first. A page that has never been fetched by a retrieval crawler cannot be cited. Not unlikely to be. Cannot be. Everything else on this list is conditional on this number.

How to compute it. Take a fixed list of priority URLs: the pages that answer buying questions, the docs pages people land on, your top converting content. Fifty is plenty. From server-side crawl records, count how many were fetched by OAI-SearchBot, PerplexityBot or another search-index agent in the window. Exclude training crawlers: a GPTBot hit tells you nothing about being citable.

What good looks like. Close to 100% for your priority list: you chose these pages precisely because they matter, so a miss is always worth explaining. There is no published industry threshold here; as a working rule of thumb, a priority list where a meaningful fraction is going uncrawled points at something structural rather than at crawl budget: blocked paths, orphaned pages, a robots.txt rule, or content only reachable through JavaScript navigation.

The failure mode it catches. Site migrations. A replatform that breaks crawler access shows up here the same day and in revenue two months later.

2. Crawl freshness

Definition. Median days since the last search-index crawl of your priority URLs.

Why it is separate from coverage. Coverage is binary and forgiving: a page crawled 29 days ago counts the same as one crawled yesterday. Freshness is the recency distribution, and it is what actually determines whether an assistant is working from your current pricing or last quarter's.

How to compute it. Per priority URL, days since the most recent search-index crawl. Report the median and the 90th percentile. The tail is the interesting half: a good median hides a set of important pages nobody has looked at in months.

What good looks like. Depends entirely on your update cadence. A pricing page recrawled every few days is healthy; a reference page recrawled monthly is fine. What matters is whether freshness is shorter than the interval at which the content materially changes.

The failure mode it catches. Slow decay. Freshness degrades gradually as a site grows and crawl budget spreads thinner, and it never triggers an alarm because nothing ever breaks.

3. Citation rate

Definition. The share of a fixed prompt set where your site appears in the assistant's answer.

Why it is the hardest one. No assistant reports impressions to publishers. There is no Search Console for AI answers. The only way to know whether you are being cited is to ask the questions and look.

How to compute it. Build a set of 20–50 questions a real buyer would ask, not brand queries, which you will win trivially and which measure nothing. Run them on a schedule, same day each week, and record for each: did your site appear, where in the answer, and who else appeared. Note the assistant and, if it is shown, the model version.

What good looks like. Direction over level. The absolute rate depends entirely on how competitive your category is; the trend against a fixed prompt set is the signal.

The discipline that makes it valid. The prompt set must not change. Every time you add or reword a prompt you break the series, so keep the original set frozen and start a second set if you need new questions.

4. AI-referred sessions

Definition. Visits your detection attributes to an AI source, with a confidence score attached.

Why the confidence matters more than the count. A majority of AI-referred visits carry no referrer, so a large share of this number is inference rather than observation. A tool reporting AI sessions as a bare integer is hiding that. One reporting "1,240 sessions, 68% high-confidence" is telling you something you can actually reason about, and only the second kind survives contact with a sceptical finance team.

How to compute it. Referrer where present, and detection over the remaining signals where it is not. Whatever the method, record which method produced each classification so a disputed number can be traced rather than defended.

What to watch. The split between assistants, and the share arriving with no referrer at all. Both move when an assistant changes its link handling, which happens without announcement.

The trap. Comparing this number to last year's. Detection improves, definitions change, and a jump in "AI traffic" is frequently a jump in what you can see rather than in what is happening.

5. AI-attributed revenue

Definition. Revenue from sessions attributed to an AI source, joined through your payment system.

Why it is the number that ends arguments. Sessions are a proxy. Revenue is the thing the business is trying to produce, and until AI traffic is expressed in it, AI search stays a curiosity in the marketing report rather than a line anyone funds.

How to compute it. Join sessions to payments (Stripe, Shopify, your own orders table) with the ordering and idempotency care that join demands. Double-counting is the standard failure and it always inflates. The pipeline design has real traps in it.

What to report alongside it. The conversion rate versus your site baseline, and the confidence split. AI referral traffic is widely reported to convert well; the published multipliers range from 1.3x to 23x, which should tell you how much weight any single figure deserves. Your own ratio against your own baseline is the only version worth planning with.

6. Crawl-to-refer ratio

Definition. Pages crawled per human visitor sent back, per operator.

Why it belongs on the scoreboard. The first five numbers measure what you are getting. This one measures what you are giving. Cloudflare's published figures for some AI operators run into the tens of thousands of pages crawled per referral, and computing your own version per operator turns the abstract "should we block AI crawlers" debate into a table with numbers in it.

How to compute it. Search-index crawl count divided by referred session count, per operator, per month. Both halves need server-side crawl data; there is no way to get the numerator from browser analytics.

How to read it. A high ratio is not automatically bad: it is the price of eligibility, and eligibility is what produces stages 3 through 5. It becomes a problem when it is high and the referral side stays near zero over several months, which is the case where you are paying bandwidth for a relationship that returns nothing. That is the moment the blocking question deserves a real answer rather than a reflex.

Reading them together

The point of six numbers rather than one is that the pattern names the problem.

The third row is the one that saves the most wasted effort. A meaningful share of AI traffic swings are measurement artefacts, and a team without the upstream numbers will happily spend a quarter rewriting content to fix a classifier change.

The vanity metrics to leave out

Total AI bot requests. Dominated by training crawlers, which have no traffic implication. Big number, no decision attached.

Brand-name prompts. You will appear in answers about your own product. It measures nothing except that the assistant can read your homepage.

"AI traffic share of total." Moves when your other channels move, which makes it a ratio of two things that have nothing to do with each other.

Single-day citation checks. Assistant answers vary run to run. One check is an anecdote; a fixed prompt set polled weekly is a series.

Cadence

One closing constraint that governs the whole scoreboard: none of these numbers can be backfilled. Crawl records that were never collected do not exist, and the first date you can answer any of these questions is weeks after the date you start collecting. That is the entire argument for starting with number 1 this week, even if numbers 3 through 6 are months away.