Every number in AI visibility eventually meets somebody who does not believe it. Usually the person whose budget depends on the answer, usually in a meeting, usually with the words "how do you actually know that."
There is exactly one reply that ends the conversation well, and it is not a methodology page. It is the text of the answer, with the sentence that names you highlighted, next to the model identifier that produced it and the timestamp it happened. Everything else (the rate, the interval, the trend) is a summary of that, and a summary you cannot expand back into evidence is a summary that gets discarded the first time it is inconvenient.
The unfalsifiability problem
Most metrics in this category are structurally unfalsifiable, and it is worth being precise about why, because it is not a matter of vendors being careless.
A presence rate is produced by a pipeline: prompts are sent to engines, answers come back, something decides whether each answer counts as a mention, and the counts are divided. If only the final division is stored, the customer receives a number whose entire derivation has been discarded. There is no way to check it, no way to disagree with a specific part of it, and no way to distinguish "we were named eight times" from "our matcher fired eight times."
Those two are not the same claim. A matcher can fire on a competitor's product that shares a word with your brand, on a coincidental capitalised phrase, on your name inside a URL that belongs to someone else. Without the text, none of that is visible, and it does not show up as an obviously wrong number, because it is a plausible number produced by a slightly wrong process.
Show me the eight answers.
The point of storing answers is not archival completeness. It is that the number stops being a claim and becomes a count of things a person can look at.
What a stored run has to carry
Six fields, and each one is there because its absence makes a specific question unanswerable.
The verbatim answer text. The primary evidence. Everything else is metadata about it.
Every source the engine cited, and which of those resolve to a domain you own. Without the ownership resolution, a source list is a wall of URLs; with it, the citation rate becomes checkable row by row.
The model identifier. Model versions change without a changelog. A drop in a rate has two candidate causes (your visibility moved, or the model did), and this is the only field that separates them.
The tool configuration. Whether web search was enabled and how it was set up. Two runs with different retrieval configurations are not comparable, and the difference is invisible in the output.
The locale. Answers vary by market. A run without one cannot be compared to a run with a different one, which you will not know you are doing.
The timestamp. Answers change under you with no announcement. An undated observation is unusable within weeks, because you cannot say what it was evidence of.
How a number decomposes
Each level is derived from the one below it. A product that stores only the top level has no way to answer a challenge except by restating the number.
The trended score
A rate over a scan window, with its confidence interval. What most reporting shows.
The per-prompt counts
Named in 3 of 25 answered runs, on these prompts, against these engines.
The individual runs
One prompt, one engine, one moment. Carries the model id, tool config, locale and timestamp.
The answer text
The verbatim words, with each mention highlighted and every cited source resolved. The bottom of the stack.
Every mention says how it was found
A mention is never stored as a bare yes. It carries a confidence between 0 and 1 and the method that produced it:
alias_exact: the canonical brand name, matched on word boundaries.alias_variant: a configured alias (a product name, a common misspelling, a social handle).domain_reference: a domain you own named in the answer's prose rather than in its source list.
These are not equally strong evidence and should not be collapsed into one verdict. A word-boundary match on an unambiguous brand name is close to certain. A match on a common-word product name is a judgement call. A domain named in prose is a different phenomenon from a brand named in prose, and rolling them together loses that.
The reason to keep the method rather than just the outcome is the same reason the detection engine has always recorded its detection method: a disputed classification has to be traceable to something specific. "We were named eight times" is an assertion. "Six exact matches on the brand name, two on the product name, and here are the eight sentences" is a claim somebody can disagree with in a useful way: they can tell you that two of the eight are a different company, and you can fix the alias list.
That is what auditability actually buys. Not the ability to prove you were right, but the ability to be told exactly where you were wrong.
The cap on the stored span, and why it exists
Here is a small constraint that carries more weight than it looks like it should.
Alongside each mention, the matched span of text is stored: the verbatim words that produced the hit. That span has a hard length ceiling.
Without a ceiling, an extraction bug that mis-computed a span boundary would copy the entire answer into the mentions table. That sounds like a storage problem and is actually a data-governance one, because the mentions table and the raw-answers table have different retention horizons. An answer that was supposed to be deleted after its window would survive, indefinitely, inside a table that outlives it, pasted there by an off-by-one.
A retention rule that only governs one table is a retention rule with a hole in it. The membership rule has to constrain a table's contents, not just its lifetime, and a length cap is how that is enforced mechanically rather than by intention.
The general shape of this
Any time two stores hold overlapping data under different retention rules, the shorter-lived one's content must be prevented from leaking into the longer-lived one by construction. Policy will not do it. A bound that makes the leak impossible will.
Two memories, stated out loud
Stored answers are large. Rollups are small. So answers are retained for less time than the rates computed from them, which is an ordinary cost decision with an interesting consequence.
Select a date range that reaches past the answer horizon and the rates are still there while the evidence behind part of them is not.
There are two ways to handle that and only one of them is honest. You can return whatever answers survive and present them as the evidence for the range, which quietly implies the sample is complete. Or you can say the range reached past what is retained, name the shorter range actually searched, and let the reader weigh it.
Partial evidence silently labelled complete is worse than no evidence, precisely because it is convincing. Somebody reads four answers, concludes the pattern is clear, and does not know that the four are a truncated tail of forty. It is the same failure as reading a blank cell as a zero, and it comes from the same instinct: the interface preferring to look complete over being accurate.
Why most tools do not do this
Two reasons, and the second is more interesting.
Storage is not free. Answers are long, runs are many, and a year of raw text across a customer base is a real line item. Keeping it is a deliberate cost, not an accident of engineering.
Evidence constrains you. A product that stores every answer cannot quietly improve its numbers. If the matcher gets more generous, the stored text says so. If a metric definition shifts, the old runs are right there disagreeing with the new number. Every methodology change becomes visible to anyone who looks.
That constraint is the feature. A vendor whose numbers cannot be checked is a vendor whose numbers can drift in whichever direction is commercially convenient, and neither of you would ever know. The stored answer is the thing that makes the relationship falsifiable in both directions, and falsifiable is the only kind of measurement worth building a budget on.
6 fields
Text, sources, model identifier, tool configuration, locale, timestamp. Any one missing makes some movement unattributable
impossible to reconstruct later
Method + confidence
On every mention, never a bare boolean. A disputed hit resolves to a span of text and a matching rule
so you can be told where you are wrong
Capped span
The stored matched text is length-bounded, so an extraction bug cannot leak a whole answer into a longer-lived table
enforced, not intended
What to do with the answers
Reading them is not an audit chore. It is where the qualitative findings live, and they are findings no aggregate will ever produce.
Read the losses. Take the prompts you consistently lose and read the answers in full. You will find out what the model thinks the category is, who it considers the default, and (often) a specific factual claim about your product that is out of date and being repeated.
Read the near-misses. Answers where you were cited and not named tell you which passage got lifted, which is the most direct possible instruction about how to write the next one.
Read after every change. When a rate moves, the first question is whether the model identifier changed. The second is what the answers look like now compared to before. Both are two minutes of work and they resolve most movements without a single hypothesis being invented.
And keep the habit for the boring reason: at some point somebody will not believe a number, and the difference between a programme that survives that and one that does not is whether you can put the text on the screen.
Frequently asked
Because a rate computed from discarded evidence cannot be checked, and an unverifiable number gets thrown out the first time it is challenged. Storing the verbatim answer means a disputed figure resolves to specific text a person can read, rather than to a restatement of the number. It also makes matcher errors visible: eight mentions and eight matcher firings are different claims, and only the stored text distinguishes them.
Six things. The verbatim answer text, every source it cited and which of those resolve to domains you own, the model identifier, the tool configuration, the locale, and the timestamp. Each covers a specific question that becomes unanswerable without it, most importantly the model identifier, since model versions change with no changelog and a rate movement otherwise cannot be attributed to your visibility rather than to the model.
Ask the tool to show you the answers behind it. A presence rate of 8 in 40 should decompose into eight readable answers, each with the mention highlighted, the matching method that established it, and the model and timestamp that produced it. If the product can only restate the number or point at a methodology page, the number is unfalsifiable and should be treated as an estimate of unknown quality.
Because raw answers are large and rollups are small, so keeping the text as long as the aggregates would be substantially more expensive. The consequence is that a long date range can reach past the answer horizon while the rates for that range still exist. The honest behaviour then is to say the range exceeded what is retained and name the shorter range actually searched, rather than presenting a truncated set of answers as if it were the whole sample.
To stop an extraction bug from copying an entire answer into a table that outlives the answer's own retention window. The mentions table and the raw answer table are kept for different lengths of time, so unbounded matched text would let short-lived content survive indefinitely in a long-lived store. A retention rule that governs only a table's lifetime and not its contents has a hole in it, and a length cap closes that hole mechanically rather than by policy.
Sources & further reading
- 01Web search tool, Anthropic
- 02Sonar API reference, Perplexity
- 03Art. 5 GDPR: Principles relating to processing of personal data, EUR-Lex
- 04GEO: Generative Engine Optimization, Aggarwal et al., KDD 2024 (arXiv:2311.09735)