A visibility tracking system needs a fixed prompt set, response records, scoring rules, and a schedule. A spreadsheet can support a small program if those records are maintained consistently.
Design the prompt panel
Select prompts from real customer questions. Include the decision types, wording variants, and buyer requirements that matter. Track competitor mentions in the same answers where possible.
Version the prompt set. A change in wording should be visible in the data. For planning, 20 prompts run ten times in two conditions produce 400 responses per model. Estimate the full request and review cost before expanding.
Run it manually first
Use one record per attempted run. Store the prompt version, date, product, model information, search condition, completion status, full answer, and scores. Keep errors separate from completed answers with no brand mention.TechRadar, How to track your brand's visibility in AI search results, which also draws the useful three-way split: being mentioned in answers, getting clicks from AI products, and being crawlable by AI bots are different questions needing different methods.
Two records are especially useful for later review:
First, save the full answer and available source or tool data. This lets you inspect a disputed score or answer a new question later. A screenshot can preserve the displayed result; a structured response can preserve additional fields.
Second, record model and configuration changes where known. A coincident change is a possible explanation for a visibility change, not proof of causation.
Change detection and baselines
Compare repeated batches using the same denominator. A change from 7 of 10 to 6 of 10 is weak evidence by itself. Review the sample size, variation, and relevant updates before treating a difference as a trend.
Build versus buy
Compare tools by product coverage, exportable evidence, documented scoring, supported conditions, and operating cost. A one-time checker can show a result; it cannot establish a time trend.
Building a tracker requires collection, error handling, storage, scoring, and maintenance. Buying can reduce some of that work. Check whether a vendor measures the API, the consumer interface, or both. Inspect an actual export before accepting a combined visibility score.
Reporting the results
Show mention, recommendation, citation, and accuracy results by product, prompt group, and search condition. Include sample counts and known configuration changes. Keep business outcomes in a separate view linked to the measurement period.
Preserve failed attempts and scoring changes
Give every attempted request an ID. A timeout is a failed attempt, not an answer in which the brand was absent. If you retry it, create a new attempt and link the two records. Keep successful answers that contain no recommendation.
Store the full answer separately from the score. If a scoring rule changes, preserve the earlier rule and rescore comparable batches. Otherwise a scoring change can look like a visibility change.
For automated scoring, manually review a sample that includes positive, negative, and ambiguous cases. Check shortened brand names, parent companies, and citations to third-party pages. Save disagreements and the final rule used to resolve them.
A small recurring report
Report the prompt versions, products, date range, attempted and completed counts, brand scores, factual errors, and known configuration changes. Include links to the underlying answers so another person can inspect a result.
The completed audit example shows an initial batch and follow-up. The worksheets provide the record structure. Start with those records before adding automated collection.