If you report on AI visibility, you need to distinguish a recurring pattern from an answer that changed on a single run. Ask an AI assistant the same question twice and you can get two different answers naming two different sets of companies. That is normal behavior and not a fault, and it limits what any tracking report can honestly claim.
Why the answer moves
Generation is not deterministic. These systems pick each next piece of text from a set of candidates with probabilities attached. They do not replay one stored answer. Two runs take different paths through that selection and arrive at different wording, different ordering, sometimes different examples.
Retrieval changes. Where an assistant searches the web before answering, it is working from whatever came back at that moment. New pages, changed pages and a different result order all produce a different answer.
The question gets interpreted. A prompt like "best project management tools for small teams" never defines small, and nothing fixes how the assistant settles that. Two runs can end up answering two slightly different questions.
The service changes underneath you. Models get updated, retrieval behavior gets adjusted, formats change. A shift in your series can be a change to the assistant and nothing to do with your market position.
Things you can no longer claim
Any claim that a brand holds a position in an answer. There is usually no ordered list at all, and where one appears the next run may reorder it or name different options entirely.
Any conclusion from a single run. One observation of a variable system is an anecdote. Worth reading. Not a measurement.
Any precise attribution of a single change. You appear this week and not last week. That could be your work, a competitor's work, a retrieval difference, or exactly the variation you would have seen if nobody had done anything at all.
Three readings that survive it
Variability is noise around a signal, not the absence of one. Three readings survive it.
Consistent presence and consistent absence. Named in most runs of a prompt across several weeks means you are established for that question. Named in none of them means you are absent. Both conclusions are safe and both are worth acting on.
Direction over time. Two runs out of ten in March and seven out of ten in July is a move, and no single-run explanation covers it. The trend outlives the noise. Recurring collection beats occasional checking for that reason.
Relative standing. A competitor named consistently across the same prompts where you are named rarely holds up as a finding, because both brands sat through the same variation.
Designing a prompt set for answers that move
- Fix the wording and leave it alone. The prompt is your measurement instrument. Change it and the series restarts, because you are now asking a different question. Record the date whenever you deliberately change one.
- Track enough prompts. Ten prompts checked regularly tell you more than one prompt checked obsessively. On a single question, variation and a real change in your standing look identical.
- Read across runs. How often a brand appears over a month, not what happened on Tuesday.
- Do not react to one observation. Absent in a single run is not a problem. Absent across several weeks is.
The question to ask any tracking product
The sampling method decides how much evidence a report carries. Ask how many times each prompt runs per collection, and whether the number you are shown is one answer or an aggregate.
In OppAlerts each prompt runs once per provider per scheduled collection and the report stores that single answer. Read every row as one observation in a longer series. No competitor pricing page read for this site states a sampling method, so ask them directly and get it in writing.
Before treating an answer change as a trend, inspect the repeated collections. Use the series to decide what deserves investigation and what you can fairly report.
See how OppAlerts collects tracked prompts, how to select a prompt set, or whether AI rank tracking is useful.