Begin an audit with the exact problem: missing mentions, inaccurate facts, missing citations, or a change in frequency. Then check the evidence available for access, retrieval, returned content, attribution, and answers without search.
These checks organize an investigation. They are not a universal sequence inside every AI product. Some intermediate operations may be unavailable to an outside observer.
Repeat the relevant prompts before diagnosing a persistent issue. Keep failed requests separate from successful responses and preserve the answers used as evidence.Don't Measure Once: Measuring Visibility in AI Search, on why one-off observations mislead. The sampling protocol is in Measuring AI Search Visibility.
Check access
Check robots.txt, status codes, redirects, firewall rules, and returned content for important pages. Use verified crawler logs where available. For Google AI supporting links, check indexing and snippet eligibility. A local curl test identifies some access problems but does not reproduce the provider’s full retrieval system.Google, Optimizing for AI features on Google Search: AI features are grounded in the Search index via RAG and query fan-out, and standard indexing and snippet controls govern eligibility.
Check retrieval
Inspect search or tool logs when available. Record returned URLs separately from citations. Ordinary search checks can provide context, but they may not reproduce the AI product’s queries or source selection.
Check returned content
Read the content returned by an available fetch or extraction tool. Check for missing prices, ambiguous subjects, broken tables, or qualifications separated from claims. A fetch followed by no citation does not prove that a reranker rejected the page.
Check citations
Verify that citations support the statements attached to them. If your page is absent, review whether it provides original, relevant evidence. Without a source log, you cannot establish that uncited material shaped the answer.Research sources and limitations. Identifies the available report data and the analysis still needed to support earlier draft figures.
Check answers without search
Test brand facts without retrieval where supported. Record omissions, outdated claims, and confusion with another business. Correct conflicting public information and rerun comparable tests. These steps do not directly edit learned model parameters.
Symptoms and checks
| Observation | Next check |
|---|---|
| Absent from the tested answers | Prompt relevance, sample size, source coverage, and access |
| Present without search, absent with search | Available source logs and repeated results in both conditions |
| Verified fetch, no mention | Returned content and answer relevance; selection remains unknown |
| Similar claim without citation | Evidence of source use; similarity alone is insufficient |
| Outdated facts | Authoritative pages and the cited sources |
| Facts from another company | Name and domain ambiguity |
| Change on one product | Known model, configuration, and source changes |
Initial and recurring reviews
Start with several representative questions and the pages needed to answer them. A small initial test identifies possible issues for follow-up. It is not enough to establish their prevalence across all customer experiences.
Repeat checks after relevant changes and on a schedule suited to the project. Report each finding with evidence, uncertainty, a proposed action, and a verification method. Prioritize confirmed access failures and material factual errors.
A completed audit example
The business and results in this example are invented. Example Cooling serves central Phoenix. The company offers weekend appointments and charges a $99 diagnostic fee. Its current service page states those facts, but an older FAQ still lists $79.
The test question is: “Which HVAC companies serve central Phoenix and offer weekend appointments? Compare service area, weekend availability, and any published diagnostic fee.” The prompt does not name Example Cooling. It tests whether the business appears when the stated requirements fit.
Collect the initial evidence
Plan 20 runs in one product and search condition. Record all attempts. In this example, 18 complete and 2 fail. Example Cooling appears in 9 completed answers, is recommended in 6, and receives a link to its site in 3. Four answers repeat the old $79 fee.
| Observation | Evidence | Conclusion |
|---|---|---|
| Old fee in four answers | Saved answers contain $79; the old FAQ also states $79 | A published contradiction exists. The source of each answer still needs checking. |
| Old FAQ cited in two answers | The citation URL opens the FAQ containing $79 | Those citations support investigating the old FAQ. They do not explain every uncited answer. |
| Service page available | The tested request returns 200 and the current $99 fee | This request obtained the page. Universal provider access is not established. |
| Weekend service sometimes omitted | The fact appears only in an image on one landing page | Add a visible text explanation so the information is easier to find and verify. |
Assign a correction that can be checked
The service manager confirms the current fee and weekend conditions. The editor corrects the FAQ and adds the conditions in visible text. The developer checks that supported markup uses the same values. The analyst saves the old and new page content with the publication date.
The completion check is specific: every maintained page and supported data representation agrees on the fee and appointment conditions. “Improve AI visibility” would not tell the team whether this correction was completed.
Read the follow-up results
In a later example batch, 19 of 20 attempts complete. The business appears in 12 answers, is recommended in 9, and receives a site citation in 7. One answer contains the old fee.
| Measure | Initial batch | Follow-up batch |
|---|---|---|
| Completed / attempted | 18 / 20 | 19 / 20 |
| Mention rate | 9 / 18 = 50.0% | 12 / 19 = 63.2% |
| Recommendation rate | 6 / 18 = 33.3% | 9 / 19 = 47.4% |
| Site citation rate | 3 / 18 = 16.7% | 7 / 19 = 36.8% |
| Answers with the old fee | 4 | 1 |
The pages were corrected, and the later sample contains fewer fee errors. The mention rate increased by about 13.2 percentage points. These small batches do not establish that the page edits caused the increase. Record other changes and repeat comparable batches before making that claim.
The next action is to inspect the remaining stale answer and its sources. That is a concrete follow-up. Repeating the same edits without examining the evidence would not explain the error.
Use the measurement and audit worksheets to keep the question, observations, correction, and follow-up together.