Part VII · Chapter 29 of 42

Building a Visibility Tracking System

From spreadsheet to tracking system

A disciplined spreadsheet beats an undisciplined platform. This chapter takes the metrics and sampling rules from Measuring AI Search Visibility and turns them into a running system: a prompt panel, a schedule, an archive, scoring, and change detection. The manual version comes first, deliberately, because it is sufficient for smaller programs and because building it teaches you exactly what to demand from a tool if you later buy one.

Design the prompt panel

The panel is the fixed set of prompts you measure, drawn from the inventory built in Prompt and Topic Research. It needs four dimensions: the intents that matter commercially (best-of, comparison, alternative-to, problem-first), 3 to 5 paraphrases per intent, the personas your category actually splits on, and prompts naming your main competitors, because share of voice needs their numbers too.

Keep the panel stable and versioned. Adding a prompt is fine; silently rewording one breaks the time series it belonged to. A workable starting size is 20 to 50 prompts; multiplied by 10 runs, two modes, and several models, that is already thousands of answers per cycle, which is the argument for starting narrow and expanding once the habit holds.

Run it manually first

The manual loop: run each panel prompt on each target platform, record whether you were mentioned, recommended, and cited, note position and sentiment, and file the full answer text. On a spreadsheet this is one row per run, with columns for date, platform, model version, mode, prompt, and the scored metrics. Published how-to methods for tracking visibility this way describe essentially the same loop, and it holds up as the entry point.TechRadar, How to track your brand's visibility in AI search results, which also draws the useful three-way split: being mentioned in answers, getting clicks from AI products, and being crawlable by AI bots are different questions needing different methods.

Two disciplines decide whether the sheet stays useful for a year.

First, archive the raw answers, not just the scores. The full text of every answer, filed by date, is the only way to answer questions you have not thought of yet: when a competitor starts appearing, six months of archived answers shows when and in what wording. Screenshots work; saved API responses are better.

Second, log the model version on every row. Answers shift in bulk when providers ship new models, and without the version column you cannot distinguish a model update from the effect of your own work. When a jump coincides with a version change, attribute it to the version change until proven otherwise.

Change detection and baselines

Score each cycle as rates: recommended in 7 of 10 runs, cited in 2 of 10. A rate that moves within its normal range is noise; with 10 runs, a single prompt drifting from 7 to 6 means nothing, while 7 to 2 across the panel is a real event. Watch for three patterns worth acting on: a broad drop across prompts on one platform (usually a model or configuration change), a drop concentrated in searched mode (a retrieval problem, see Auditing and Diagnosing Visibility Problems), and a competitor's rate rising across your panel. Weekly cycles are enough; the reasoning is in Measuring AI Search Visibility.

Build versus buy

The commercial market is crowded and young: dozens of platforms, most doing the same basic thing, running prompts against the major models and reporting whether you were mentioned, with pricing commonly in the tens to hundreds of dollars or euros per month.Lumentir's tested comparison of AI visibility tools, itself a vendor in the category it reviews, but a usable map of who covers which platforms at what price. Free checkers give a one-off snapshot of whether major platforms mention you, useful as a first look and structurally unable to show trend.For example, Ahrefs' free AI visibility checker, which queries several platforms with search-backed prompts for a brand name. Dedicated trackers add the pieces the free tier lacks: scheduled runs, prompt panels, citation feeds, and competitor share.Citations.io is a representative example: tracked prompt sets across several engines with a citation feed and share-of-answer reporting.

The build case: the raw work is API calls and a database, the marginal cost of a run is a few tokens, and you control every variable that matters, model version, search mode, run counts, and the archive. The buy case: coverage of platforms without public APIs, the operational work you do not have to own, and interfaces your stakeholders can read. Judge any tool with the four questions from Measuring AI Search Visibility: which model and version, was search on, how many runs, rates or single observations. A vendor that cannot answer them is selling the noise; the wider tool market is mapped in Tools, Templates, and Checklists.

The dashboard that matters

However the data is collected, the output view is the same: share of answers over time, by model, by prompt class, by mode, with model-version changes marked on the time axis. That one chart answers the questions leadership actually asks, are we in the answers, is it improving, and did that jump come from us or from a model update. Connecting it to revenue is the next chapter, Attribution and Business Impact.