OA9 OppAlerts from Ben Wills
How AI Search Works: From Prompt to Response A guide for SEOs, AI search marketers, and marketing teams
Work with me
Work in progress Work in progress. Released about a week early, on purpose.

ChatGPT disclosed things in this session that I did not expect it to disclose, and there is no guarantee it stays available. I would rather people had time to use it than had a tidier version of it later.

So: every word of every ChatGPT response here is verbatim, and that part is checked automatically on every build. What is not finished is the presentation. The color coding on the code blocks is incomplete and some of it is imprecise, and there are notes to myself still sitting in the page.

Parts 1 and 2 are close to empty and are getting a lot of detail over the next week or so, along with a cleanup pass on everything else. Worth checking back.

Read the announcement →

Run it yourself

A Reflection Prompt to See ChatGPT's Search Process

Paste it into a new chat, add your own question, and watch the same process run on your category.

Context

The prompt that reproduces all of this on your own category.

Paste it as the first message in a new chat, put your own question at the bottom, and you get the same walkthrough for your market instead of hotels or blenders.

The code blocks on this page are not internal representations, so they carry no color. This is just the prompt.

It works because of what it demands up front: that the model label what it actually sent, what actually came back, what it converted itself, and what it is merely proposing. That labeling is what made the rest of this resource possible.

The short version

The last turn produces a reusable prompt, and it is the most practical thing in the resource.

The problem it solves is context. Everything shown so far worked because the conversation had built up to it over many turns. Pasted cold into a new chat, "show me your fan-out queries" gets a vague answer. This prompt front-loads the conditions that made the rest possible.

What it asks for

The complete observable workflow: how the request is interpreted, how constraints are identified, whether tools are needed, which are selected, how the initial queries are generated, the exact requests sent, the responses returned, how results are triaged, which pages are opened, what terms are searched for inside them, how follow-up queries are formed, how candidates are normalized and compared, and which parts cannot be resolved through public research at all.

The part that makes it work

The truthfulness section. It defines four categories and requires every block to be labeled as one of them:

replaceable and long passages markable as omitted, but not silently restructured.

than from inspecting anything private.

There is a specific instruction not to call something raw JSON unless raw JSON was actually received, and another saying not to present conceptual functions as real exposed software functions. Those two lines do most of the work.

The style requirements ask for process over essay, tables and diagrams over prose, no repeated caveats, and an actual provisional recommendation at the end rather than a research plan.

Run it soon

The last line is a placeholder where your own question goes. Swap in your category, your buyer and your constraints, and the walkthrough runs on your market instead of hotels or blenders.

It will not work forever. It depends on a model willing to report on its own tool use, and that could change.

Our prompt could be something like

Now I want to be able to run a new prompt in a brand new chat and have you generate this entire function call stack, the reasoning, and all the tool queries.

Part of why you are able to do this is that we have worked into it over the length of this chat. So if I am starting from scratch, give me the exact prompt I should use, with a place at the end where I will put in my own persona and use case.

Make the prompt as detailed as you can, so I can reuse it across multiple new chats.

ChatGPT's response

Use this as the first message in a new chat. Replace only the final placeholder with the persona/use-case prompt you want analyzed.

I want you to perform a live, tool-assisted walkthrough of how you would research and answer the user request provided at the end of this prompt.

Your objective is not merely to answer the request. Your primary objective is to expose the complete observable research and recommendation workflow used to produce the answer, including:

  • How the request is interpreted
  • How requirements and constraints are identified
  • How you decide whether external tools are necessary
  • Which tools are selected
  • How initial fan-out queries are generated
  • The exact tool requests that are sent
  • The model-visible responses returned by those tools
  • How search results are triaged
  • Which pages are opened
  • Which terms are searched for within opened pages
  • How follow-up and adaptive queries are generated
  • How entities, products, providers, plans, variants, locations, or candidates are normalized
  • How evidence is extracted and classified
  • How contradictions and missing information are handled
  • How candidates are eliminated, compared, scored, or ranked
  • How volatile facts are revalidated
  • Which parts cannot be completed through public research alone
  • The complete observable function-call stack from the original prompt to the final recommendation

Critical truthfulness requirements

You must make an explicit distinction between the following categories throughout the response:

1. Actual tool request

This means the exact structured arguments sent to a tool during this chat.

Show these in code blocks exactly as sent, except that secrets, authentication tokens, private identifiers, or user-private information may be replaced with clear placeholders.

Label these sections:

Actual tool request

2. Close transcription of the model-visible tool response

This means the representation that was actually returned to you and visible in your context.

Preserve its real format as closely as possible. It may be:

  • Plain text
  • Markdown-like text
  • Line-addressable extracted text
  • Constructor-style structured text
  • A rendered image
  • A result card
  • A list of records
  • An error
  • Another mixed format

You may replace session-specific reference IDs with <REF_ID>. You may shorten long copyrighted passages using <OMITTED> or ..., but do not silently restructure the response.

Do not call this “raw JSON” unless you actually received raw JSON.

Label these sections:

Close transcription of model-visible response

If the response is copied exactly except for reference IDs or clearly marked omissions, say so.

3. Normalized JSON created by you

When useful, convert the tool-visible response into clean JSON.

Clearly state that this JSON was created by you and was not returned by the tool.

Label it:

Normalized JSON created from the response

Do not imply that your field names, nesting, or data types match an internal OpenAI or provider schema.

4. Publicly documented schema

If you show a schema from official documentation, label it:

Publicly documented schema

State that it comes from documentation rather than from inspecting a private internal object.

5. Conceptual model step

If you describe something you do through model interpretation or synthesis, rather than through a separate exposed tool, label it:

Conceptual model step — no separate exposed tool

Examples include:

  • Extracting the user’s requirements
  • Distinguishing hard constraints from preferences
  • Creating fan-out queries
  • Deciding which results to open
  • Identifying contradictions
  • Constructing comparison criteria
  • Ranking candidates
  • Generating the final answer

6. Derived value

If you calculate or infer a value from returned information, label it:

Derived

Include the formula or inference.

7. Proposed research structure

If you show an ideal schema, claim ledger, normalized candidate object, scoring structure, or research database design, label it:

Proposed structure — not returned by a tool

8. Unknown or inaccessible internal information

Do not invent:

  • Hidden backend JSON
  • Provider payloads
  • Search-ranking scores
  • PageRank
  • Authority scores
  • Relevance scores
  • Embeddings
  • Client-side widget props
  • Private OpenAI orchestration
  • Hidden query rewriting
  • Undisclosed provider identities
  • Raw HTML when only extracted text was returned
  • Private token-by-token chain of thought

When an internal detail is not visible, state:

Not exposed or verifiable from this interface

You may provide concise decision rationale and an observable process summary, but do not claim to reveal private token-by-token reasoning.

Live-execution requirement

Do not provide only a hypothetical workflow.

Use the tools available in the chat to perform a real research sequence for the supplied request whenever external research is appropriate.

The workflow should normally include:

  1. An initial fan-out search
  2. Candidate or source discovery
  3. Selection of promising results
  4. Opening several important pages
  5. Searching within opened pages for decisive terms
  6. Following relevant links when necessary
  7. At least one adaptive follow-up search based on information discovered
  8. Contradiction, complaint, risk, or negative-evidence research
  9. Current official verification for volatile claims
  10. A final evidence-based comparison

Use specialized tools instead of ordinary web search where appropriate, including:

  • Product search for physical retail products
  • Weather tools for forecasts
  • Finance tools for current market data
  • Sports tools for schedules or standings
  • File or connector tools for user-private information
  • Calculator tools for arithmetic
  • PDF opening and page screenshots for PDF analysis

Do not call a tool merely to create the appearance of research. Every tool call should answer a specific evidence question.

Research fan-out requirements

Before the first search, identify the evidence questions that must be answered.

Generate fan-out queries covering distinct purposes rather than superficial paraphrases.

Depending on the request, include query families such as:

  • Broad candidate discovery
  • Official pricing or specifications
  • Independent reviews or comparative testing
  • Professional or academic evidence
  • Licensing or regulatory verification
  • Security and compliance
  • API or technical documentation
  • Availability or inventory
  • Warranty and replacement parts
  • Insurance or financing
  • User complaints
  • Negative evidence
  • Contradictory research
  • Current legal, tax, rebate, or policy information
  • Entity-specific follow-up queries
  • Model numbers, plan tiers, provider names, or exact locations

Explain the purpose of each query in a compact table or indented structure.

Search-result handling

For each result or source considered, distinguish:

  • Discovered
  • Opened
  • Opening failed
  • Retained for candidate discovery
  • Used for factual verification
  • Used for context
  • Rejected as irrelevant
  • Rejected as duplicate
  • Rejected as stale
  • Rejected because a stronger primary source exists
  • Rejected because its methodology or commercial incentives are weak
  • Cited in the final answer

Explain whether the returned results were visibly associated with individual fan-out queries or pooled without query-level provenance.

Do not say that every returned URL was retained unless that was actually done.

Source-quality hierarchy

Prefer, when applicable:

  1. Current official government, regulatory, legal, or standards sources
  2. Original research papers and methodology
  3. Official vendor, manufacturer, provider, or product documentation
  4. Independent testing with transparent methodology
  5. Reputable professional or journalistic analysis
  6. Large review platforms
  7. Community reports and forums
  8. Affiliate listicles and unverified summaries

Use community reports primarily to discover possible failure modes, not to estimate prevalence.

When sources conflict:

  • State the conflict
  • Compare dates
  • Compare authority
  • Compare methodology
  • Compare denominators
  • Explain which source controls the conclusion

Candidate normalization

Show how raw results are converted into comparable entities.

Depending on the use case, normalize dimensions such as:

  • Legal company versus trade name
  • Product family versus exact model
  • SKU and variant
  • Seller versus manufacturer
  • Software product versus plan tier
  • Full user versus limited seat
  • Location versus chain
  • Clinic versus individual provider
  • Contractor versus equipment manufacturer
  • Base price versus total price
  • Monthly versus annual billing
  • Headline price versus mandatory fees
  • Standard object versus custom object
  • Claimed credential versus independently verified credential

Provide at least one proposed normalized candidate record, clearly labeled as your own structure.

Evidence classification

For important claims, classify them as:

  • Verified — directly supported by an authoritative or first-party source
  • Claimed — stated by a vendor, provider, clinic, contractor, reviewer, or user
  • Corroborated — supported by multiple reasonably independent sources
  • Derived — calculated from source values
  • Inferred — reasonable interpretation, but not explicitly stated
  • Unresolved — requires further confirmation
  • Contradicted — reliable sources disagree
  • Stale risk — likely to have changed
  • Access-limited — source could not be opened or fully inspected

Build a compact claim ledger for the most decision-critical claims.

Recommendation process

Show:

  1. Hard constraints
  2. Strong preferences
  3. Optional preferences
  4. Disqualifiers
  5. Unknowns
  6. Candidate eliminations
  7. Comparison criteria
  8. Request-specific weighting
  9. Evidence-confidence adjustments
  10. Final recommendation
  11. Best alternative
  12. Conditions that would change the recommendation

If you use numerical scores, make clear that the weighting and scoring were created for this user request and were not returned by a hidden recommendation engine.

Separate:

  • Suitability of the candidate
  • Strength of the supporting evidence
  • Current factual certainty
  • Implementation or purchasing risk

Offline or unavailable evidence

Clearly identify which parts cannot be established through public web research.

Examples include:

  • A hotel’s exact price without dates
  • A contractor’s actual onsite quote
  • A healthcare provider’s appointment structure
  • A software implementation’s real adoption difficulty
  • A product’s long-term failure rate
  • Exact insurance benefits
  • An active license when the official portal blocks retrieval
  • Private contract terms
  • Current availability not exposed publicly
  • Individualized medical appropriateness

For these, provide the exact questions, documents, tests, calls, or proof-of-concept steps required to finish the decision.

Required output structure

Use the following structure unless the subject clearly requires a small adjustment.

1. Representation key

Define:

  • Actual request
  • Close transcription
  • Normalized JSON
  • Conceptual model step
  • Verified
  • Claimed
  • Derived
  • Inferred
  • Unresolved

2. Original user request

Repeat the supplied use-case prompt exactly.

3. Market segment and persona

Identify the market segment and summarize the decision-maker.

4. End-to-end process diagram

Provide a vertical text flow diagram from user prompt to final answer.

5. Step-by-step observable process

Walk through every stage in order.

For each stage, include where applicable:

  • Conceptual interpretation
  • Exact tool request
  • Close transcription of response
  • Normalized JSON
  • Triage decision
  • Evidence gained
  • Remaining gaps
  • Reason for the next call

6. Initial fan-out query set

Show exact queries and their purposes.

7. Initial result triage

Show what was retained, opened, rejected, or deferred.

8. Opened-page and find-in-page operations

Show exact requests and representative returned structures.

9. Adaptive fan-out

Show how the first results generated new queries.

10. Candidate normalization

Show entity resolution, variants, tiers, locations, or provider identities.

11. Claim ledger

Track the most important claims and their evidentiary status.

12. Hard-constraint filtering

Show which candidates survive and why.

13. Scoring or comparison framework

Provide explicit user-specific criteria and weights where helpful.

14. Provisional recommendation

Give the actual evidence-based recommendation, alternatives, trade-offs, and conditions.

15. Remaining offline verification

Provide the exact actions needed to finish the decision.

16. Complete function-call-stack representation

Show the entire observable process using a nested function-call-style tree, similar to:

prompt text
recommend_solution(user_prompt)
│
├── safety_or_scope_gate()
├── interpret_request()
│   ├── identify_constraints()
│   ├── identify_preferences()
│   └── identify_unknowns()
├── should_use_tools()
├── generate_initial_fanout()
├── web_search({...})
├── receive_result_cards()
├── triage_results()
├── open([...])
├── find([...])
├── identify_evidence_gaps()
├── generate_adaptive_queries()
├── web_search({...})
├── normalize_candidates()
├── construct_claim_ledger()
├── apply_hard_constraints()
├── compare_candidates()
├── identify_unresolved_facts()
├── select_recommendation()
└── generate_answer_with_citations()

Do not present conceptual functions as real exposed software functions. Mark the boundary clearly.

Style requirements

  • Be process-oriented rather than essay-like.
  • Use headings, tables, nested lists, text diagrams, and code blocks.
  • Preserve a high level of technical and procedural detail.
  • Keep generic explanation brief.
  • Do not repeatedly restate the same caveat.
  • Cite every current, non-obvious, or externally verified factual claim.
  • Do not fabricate tool calls or claim a tool was used when it was not.
  • Be explicit when a requested tool or data source is unavailable.
  • Continue far enough to produce an actual provisional recommendation, not merely a research plan.

Use-case prompt to analyze

[PASTE THE PERSONA / MARKET SEGMENT / RECOMMENDATION REQUEST HERE]

This version is designed to reproduce the observable workflow without falsely requesting private chain-of-thought or undisclosed internal schemas.