The same prompt, on the same model, produces different answers on different runs. Generation samples from a probability distribution, and a sampling setting called temperature adds deliberate randomness at each step, so one answer is one draw. Every metric in this chapter is built around that fact, and every measurement habit imported from SEO that ignores it, checking once, checking daily, reading one answer as a ranking, produces numbers that look precise and mean little.
The metric family
Seven metrics cover the channel. Define each precisely and keep them separate, because each answers a different question and they move independently.
- Mention rate: the share of sampled answers in which your brand appears at all, in any role.
- Recommendation rate: the share in which the answer actually recommends you, rather than mentioning you in passing or unfavorably.
- Citation rate: the share in which your site is linked or named as a source.
- Share of voice: your mentions as a fraction of all brand mentions across the sampled answers, the competitive version of mention rate.
- Prominence: where you appear when you appear: first pick, list position, or a trailing aside.
- Sentiment: how the answer characterizes you when it names you.
- Accuracy: whether what the answer says about you is true and current.
The separations carry real information. A brand can be mentioned constantly and recommended never. It can be cited as a source in answers that recommend a competitor. And a model can recommend a brand it never read a page from: in OppAlerts' study, 53.2% of recommended brands had no page fetched from their own site during the answer.The cited-versus-recommended analysis in the OppAlerts ranking-factors study: 403,000 prompts, 100 industries, 13 models. Method and tables in The Research Behind This Guide, part of the AI Search Visibility research. A dashboard that collapses these into one "visibility score" is discarding the information you need for diagnosis.
Sampling: never measure once
A prompt tested once tells you almost nothing. The measurement literature has reached the same conclusion from the empirical side: answers vary across runs, prompt phrasings, and time, so visibility is a distribution, and only repeated measurement characterizes it.Don't Measure Once: Measuring Visibility in AI Search, the paper making the empirical case that one-off observations are unreliable and visibility should be reported as a distribution.
The protocol I use, through the API rather than the chat interface, so every response is logged and the configuration is explicit:
- Runs: each prompt at least 10 times per model per mode. After 5 runs a trend is usually visible; 10 gives a rate you can compare month over month.
- Modes: the same prompt with search disabled and with search enabled. The memory answer and the retrieval answer are different measurements, per Model Memory and Live Retrieval, and averaging them hides both.
- Paraphrases: 3 to 5 phrasings per intent, because models can answer "best X" and "top X for beginners" from different distributions, and real users phrase things many ways. Build these from Prompt and Topic Research.
- Personas: where your category serves distinct buyers, prefix prompts with who is asking. Answers shift by persona, and aggregate-only measurement misses collapses inside segments.
- Models: every platform your audience uses, not the one that is easiest to query. Model-to-model disagreement is large enough that one model is not a proxy for the channel.
- Versions: pin a model version for a stable time series, or use the latest alias to track what the product actually serves. Either is defensible; record which you chose.
Report each metric as a rate across runs: recommended in 7 of 10, cited in 2 of 10. Rates are comparable across months and survive the underlying randomness; single observations do neither.
The confounders
Several things change answers without anything changing about you. Logged-in and logged-out sessions of the same product can serve different configurations, and assistant personalization and persistent memory mean the same prompt can produce different answers per user account, so a clean panel measures the model, not an account's accumulated context. Model updates move answers in bulk: when a provider ships a new version, your numbers can jump with no action on your part, which is why the version log matters. And an answer that quietly ran a search and one that did not are different measurements even inside the same chat window; the API's explicit search toggle is the only clean control I know of.
Reading the split in your data
Once mention and citation rates are measured with search on and off, the pattern reads directly. Strong memory-mode mentions with weak searched-mode presence means the model knows you but retrieval is replacing you with competitors who win the underlying rankings. Weak memory with strong searched presence means retrieval is carrying you, and your standing depends on pages and rankings that can change in weeks. Citation metrics reflect retrieval wins; unprompted memory mentions reflect training-data fame. The gap between modes is large on average, 44.1% of brands in a searched answer are absent from the same model's memory answer, so measuring only one mode misses roughly half the picture.The memory-versus-search analysis: 5,440 paired answers across 100 industries. Detail in The Research Behind This Guide. What to do about each pattern is the subject of Auditing and Diagnosing Visibility Problems.
Reading vendor dashboards without being fooled
Much of the commercial tracking market queries logged-out, web-based chat sessions through proxies, then reports daily movement. Every part of that deserves scrutiny. Logged-out sessions are a minority slice of real usage served by an unspecified configuration. Daily deltas on small run counts are mostly temperature noise dressed as trend: the model behind the product changes on a cadence of weeks or months, so a daily reading of it mostly measures the randomness, and a weekly, well-sampled reading catches everything a daily one does. Retrieval-side movement runs faster, but tracking it daily is tracking the underlying search engines, a thing SEO already knows how to watch.
So put four questions to any dashboard, yours or a vendor's: which model and version answered, was search on, how many runs produced this number, and is it reported as a rate or a single observation. A tool that cannot answer them is reporting noise with confidence intervals missing; the build-versus-buy decision this feeds is covered in Building a Visibility Tracking System.