Store the Answer, Not Just the Score

Every number in AI visibility eventually meets somebody who does not believe it. Usually the person whose budget depends on the answer, usually in a meeting, usually with the words "how do you actually know that."

There is exactly one reply that ends the conversation well, and it is not a methodology page. It is the text of the answer, with the sentence that names you highlighted, next to the model identifier that produced it and the timestamp it happened. Everything else (the rate, the interval, the trend) is a summary of that, and a summary you cannot expand back into evidence is a summary that gets discarded the first time it is inconvenient.

Most metrics in this category are structurally unfalsifiable, and it is worth being precise about why, because it is not a matter of vendors being careless.

A presence rate is produced by a pipeline: prompts are sent to engines, answers come back, something decides whether each answer counts as a mention, and the counts are divided. If only the final division is stored, the customer receives a number whose entire derivation has been discarded. There is no way to check it, no way to disagree with a specific part of it, and no way to distinguish "we were named eight times" from "our matcher fired eight times."

Those two are not the same claim. A matcher can fire on a competitor's product that shares a word with your brand, on a coincidental capitalised phrase, on your name inside a URL that belongs to someone else. Without the text, none of that is visible, and it does not show up as an obviously wrong number, because it is a plausible number produced by a slightly wrong process.

The point of storing answers is not archival completeness. It is that the number stops being a claim and becomes a count of things a person can look at.

Six fields, and each one is there because its absence makes a specific question unanswerable.

The verbatim answer text. The primary evidence. Everything else is metadata about it.

Every source the engine cited, and which of those resolve to a domain you own. Without the ownership resolution, a source list is a wall of URLs; with it, the citation rate becomes checkable row by row.

The model identifier. Model versions change without a changelog. A drop in a rate has two candidate causes (your visibility moved, or the model did), and this is the only field that separates them.

The tool configuration. Whether web search was enabled and how it was set up. Two runs with different retrieval configurations are not comparable, and the difference is invisible in the output.

The locale. Answers vary by market. A run without one cannot be compared to a run with a different one, which you will not know you are doing.

The timestamp. Answers change under you with no announcement. An undated observation is unusable within weeks, because you cannot say what it was evidence of.

A mention is never stored as a bare yes. It carries a confidence between 0 and 1 and the method that produced it:

• alias_exact: the canonical brand name, matched on word boundaries.
• alias_variant: a configured alias (a product name, a common misspelling, a social handle).
• domain_reference: a domain you own named in the answer's prose rather than in its source list.

These are not equally strong evidence and should not be collapsed into one verdict. A word-boundary match on an unambiguous brand name is close to certain. A match on a common-word product name is a judgement call. A domain named in prose is a different phenomenon from a brand named in prose, and rolling them together loses that.

The reason to keep the method rather than just the outcome is the same reason the detection engine has always recorded its detection method: a disputed classification has to be traceable to something specific. "We were named eight times" is an assertion. "Six exact matches on the brand name, two on the product name, and here are the eight sentences" is a claim somebody can disagree with in a useful way: they can tell you that two of the eight are a different company, and you can fix the alias list.

That is what auditability actually buys. Not the ability to prove you were right, but the ability to be told exactly where you were wrong.

Here is a small constraint that carries more weight than it looks like it should.

Alongside each mention, the matched span of text is stored: the verbatim words that produced the hit. That span has a hard length ceiling.

Without a ceiling, an extraction bug that mis-computed a span boundary would copy the entire answer into the mentions table. That sounds like a storage problem and is actually a data-governance one, because the mentions table and the raw-answers table have different retention horizons. An answer that was supposed to be deleted after its window would survive, indefinitely, inside a table that outlives it, pasted there by an off-by-one.

A retention rule that only governs one table is a retention rule with a hole in it. The membership rule has to constrain a table's contents, not just its lifetime, and a length cap is how that is enforced mechanically rather than by intention.

Stored answers are large. Rollups are small. So answers are retained for less time than the rates computed from them, which is an ordinary cost decision with an interesting consequence.

Select a date range that reaches past the answer horizon and the rates are still there while the evidence behind part of them is not.

There are two ways to handle that and only one of them is honest. You can return whatever answers survive and present them as the evidence for the range, which quietly implies the sample is complete. Or you can say the range reached past what is retained, name the shorter range actually searched, and let the reader weigh it.

Partial evidence silently labelled complete is worse than no evidence, precisely because it is convincing. Somebody reads four answers, concludes the pattern is clear, and does not know that the four are a truncated tail of forty. It is the same failure as reading a blank cell as a zero, and it comes from the same instinct: the interface preferring to look complete over being accurate.

Two reasons, and the second is more interesting.

Storage is not free. Answers are long, runs are many, and a year of raw text across a customer base is a real line item. Keeping it is a deliberate cost, not an accident of engineering.

Evidence constrains you. A visibility product that stores every answer cannot quietly improve its numbers. If the matcher gets more generous, the stored text says so. If a metric definition shifts, the old runs are right there disagreeing with the new number. Every methodology change becomes visible to anyone who looks.

That constraint is the feature. A vendor whose numbers cannot be checked is a vendor whose numbers can drift in whichever direction is commercially convenient, and neither of you would ever know. The stored answer is the thing that makes the relationship falsifiable in both directions, and falsifiable is the only kind of measurement worth building a budget on.

Reading them is not an audit chore. It is where the qualitative findings live, and they are findings no aggregate will ever produce.

Read the losses. Take the prompts you consistently lose and read the answers in full. You will find out what the model thinks the category is, who it considers the default, and (often) a specific factual claim about your product that is out of date and being repeated.

Read the near-misses. Answers where you were cited and not named tell you which passage got lifted, which is the most direct possible instruction about how to write the next one.

Read after every change. When a rate moves, the first question is whether the model identifier changed. The second is what the answers look like now compared to before. Both are two minutes of work and they resolve most movements without a single hypothesis being invented.

And keep the habit for the boring reason: at some point somebody will not believe a number, and the difference between a programme that survives that and one that does not is whether you can put the text on the screen.