OA9 OppAlerts from Ben Wills
How AI Search Works: From Prompt to Response A guide for SEOs, AI search marketers, and marketing teams
Work with me
Work in progress Work in progress. Released about a week early, on purpose.

ChatGPT disclosed things in this session that I did not expect it to disclose, and there is no guarantee it stays available. I would rather people had time to use it than had a tidier version of it later.

So: every word of every ChatGPT response here is verbatim, and that part is checked automatically on every build. What is not finished is the presentation. The color coding on the code blocks is incomplete and some of it is imprecise, and there are notes to myself still sitting in the page.

Parts 1 and 2 are close to empty and are getting a lot of detail over the next week or so, along with a cleanup pass on everything else. Worth checking back.

Read the announcement →

Recommendation walkthrough

How ChatGPT Recommends a Hotel

Thirteen steps from one traveler's sentence to a recommendation, with the actual tool calls.

Context

The same hotel question, run for real, step by step, with the actual tool calls.

Thirteen steps from prompt to answer, ending in a function call stack. Watch what happens at step 7, where it searches inside pages it already opened for the words "Destination Fee" and "noise." That is a machine checking whether the marketing copy is true.

Our prompt could be something like

A lot of what you have described so far is high level. Help me understand exactly what your internal process is.

Are you taking my prompt and sending it to an internal system to generate the fan-out queries? Is there a tool for generating fan-out queries, or for turning results into a recommendation?

You put the decision specification in section one into JSON, and I understand that is an illustration rather than the real thing. Do you have an instruction that converts my prompt into those structured requirements, or is that a tool it goes to and comes back from? Is there then another tool that generates the fan-out queries, and do you decide which ones to use?

Or do you just search those fan-out queries and get the responses back? And then scrape each URL? Or are you looking at the result card? Is there an internal tool that scrapes a webpage and returns the card? If so, are you sending the initial prompt to it, so it can determine what is most relevant? Or do you get the entire HTML response and have to parse it yourself?

It also looks like you might run a set of fan-out queries, get results, and then decide on the next set. It seems like a process with many steps.

Take me through, concretely and concisely, exactly what those steps are for the prompt you used above: find me the best hotel in Chicago, I am traveling for a conference at McCormick Place, want to spend under $300 a night, need a quiet room and reliable Wi-Fi, and would prefer a decent gym.

Start there and walk me through the exact chain of what happens. A flow diagram would be great. A bulleted list with sub-bullets that reads like a function call stack would also work.

Show me how the fan-out queries are generated and then queried. I want every part of your interaction with that. When you send something to a tool, what is the request and response format? I want that in excruciating detail.

Keep the explanatory material to a minimum. This whole conversation has been explanatory. Make this one process oriented.

ChatGPT's response

The exact boundary

From my perspective in this chat:

  • There is no separately exposed “fan-out query generator” tool.
  • There is no separately exposed “prompt-to-requirements” parser.
  • There is no separately exposed hotel recommendation engine.
  • There is no automatic scraper that opens every returned URL.
  • I formulate search queries and tool arguments directly from the conversation.
  • Search, open, find, and click are separate tool operations.
  • The final comparison and recommendation are synthesized by the model, not returned by a ranking tool.

There may be additional OpenAI product-level orchestration outside my visible interface. I cannot inspect that. OpenAI publicly says ChatGPT Search may rewrite a prompt into multiple queries, inspect results, and send narrower follow-up queries, but the exact proprietary implementation is not disclosed. (OpenAI Help Center)

I also cannot provide hidden token-by-token reasoning. What follows is the observable orchestration and tool interaction.


End-to-end flow

conceptual, drawn by ChatGPT
USER PROMPT
	│
	▼
MODEL CONTEXT
	│
	├── Current user message
	├── Prior conversation
	├── System/developer instructions
	├── Available tool descriptions and schemas
	└── Potentially relevant memory/context
	│
	▼
IN-MODEL TASK INTERPRETATION
	│
	├── Determine requested outcome
	├── Identify hard constraints
	├── Identify preferences
	├── Identify missing information
	├── Determine whether facts are current/unstable
	└── Determine whether tools are required
	│
	▼
TOOL SELECTION
	│
	├── No tool → answer from model knowledge/reasoning
	└── Tool needed
		│
		├── web search
		├── product search
		├── weather
		├── finance
		├── sports
		├── files/connectors
		└── other available tool
	│
	▼
INITIAL TOOL CALL
	│
	├── Model creates query strings
	├── Model fills the tool's argument schema
	└── Structured function call is emitted
	│
	▼
TOOL EXECUTION
	│
	├── Search/index/provider processes queries
	└── Tool returns model-visible result representations
	│
	▼
RESULT TRIAGE BY MODEL
	│
	├── Relevant?
	├── Correct geography/entity?
	├── Primary or secondary source?
	├── Current enough?
	├── Answers which constraint?
	├── Duplicate?
	├── Contradictory?
	└── Worth opening?
	│
	▼
FOLLOW-UP TOOL CALLS
	│
	├── open selected page
	├── find exact term within page
	├── click a page link
	├── run narrower entity-specific searches
	└── search for contradictions or missing evidence
	│
	▼
EVIDENCE SYNTHESIS
	│
	├── Combine official facts
	├── Compare independent assessments
	├── identify conflicting evidence
	├── Eliminate hard-constraint failures
	├── Compare remaining candidates
	└── Assess uncertainty
	│
	▼
FINAL ANSWER
	│
	├── Recommendation
	├── Alternatives/trade-offs
	├── Qualifications
	└── Citations

Concrete example

User message

unclassified, needs review
Find me the best hotel in Chicago. I’m traveling for a conference at
McCormick Place, want to spend under $300 a night, need a quiet room and
reliable Wi-Fi, and would prefer a decent gym.

Step 1: The prompt enters my context

There is no visible call such as:

unclassified, needs review
{
	"tool": "parse_hotel_request",
	"prompt": "..."
}

No parser result comes back.

I interpret the prompt directly.

Conceptually, I identify:

conceptual
Task:
	Recommend a hotel

Location:
	Chicago

Destination anchor:
	McCormick Place

Hard or near-hard constraints:
	Under $300/night
	Reliable Wi-Fi
	Quiet enough to sleep/work

Preferences:
	Decent gym
	Close to conference

Missing:
	Travel dates
	Number of guests
	Whether $300 includes taxes/fees
	Refundability requirement
	Parking requirement

This conceptual structure does not necessarily exist as JSON internally. It is a convenient description of what I extract from the language.


Step 2: Decide whether search is required

For this request, search is required because these facts are unstable:

unclassified, needs review
Hotel availability
Nightly prices
Mandatory fees
Renovation status
Recent guest complaints
Amenities
Hotel operating status
Current review consensus

Without search, I could name plausible hotels from model knowledge, but I could not responsibly call one “best” now.

The missing dates are especially important. Without them, exact price and availability cannot be completed.

The workflow can either:

unclassified, needs review
Branch A:
	Ask for exact travel dates before searching prices

Branch B:
	Discover and evaluate likely candidates now
	Mark price and availability as unresolved

For the live demonstration, I used Branch B.


Step 3: Generate the initial fan-out queries

No separate tool returned these queries to me.

I generated them as the arguments of the search-tool call.

Actual request sent in this chat

sent exactly as shown
{
	"system1_search_query": [
		{
			"q": "best hotels near McCormick Place Chicago quiet rooms reliable Wi-Fi gym"
		},
		{
			"q": "site:hyatt.com McCormick Place Chicago hotel fitness center Wi-Fi"
		},
		{
			"q": "site:marriott.com McCormick Place Chicago hotel fitness center Wi-Fi"
		},
		{
			"q": "McCormick Place Chicago hotel noise reviews quiet rooms"
		}
	],
	"response_length": "long"
}

Why those four

unclassified, needs review
Query 1:
	Candidate discovery
	Attempts to cover all user requirements

Query 2:
	Official Hyatt evidence
	Verifies amenities for a likely adjacent hotel

Query 3:
	Official Marriott evidence
	Verifies another likely adjacent hotel

Query 4:
	Subjective quietness evidence
	Targets recent reviews and noise complaints

The original user prompt was not copied wholesale into the tool call. The visible function arguments contained the four query strings and response-length setting.

Any further query rewriting or provider-side processing is outside my visibility.


Step 4: Search returns candidate cards

The response was not raw provider JSON.

It returned model-visible search-result representations resembling:

returned exactly as shown
HYATT REGENCY MCCORMICK PLACE CHICAGO - 2026 Prices & Reviews (IL)
(https://www.tripadvisor.com/Hotel_Review-...)

<REF_ID> [wordlim: 200] Crawled: today;

Very quiet and clean...
...
Noise level in my room was extreme...
returned exactly as shown
The 10 best hotels near McCormick Place in Chicago, United States
(https://www.booking.com/landmark/us/mccormick-place.html)

<REF_ID> [wordlim: 200] Crawled: 5 days ago;

Hyatt Regency McCormick Place...
Marriott Marquis Chicago...
Hampton Inn Chicago McCormick Place...
Hilton Garden Inn Chicago McCormick Place...
returned exactly as shown
Hyatt Regency McCormick Place Chicago | Book Your Stay Now
(https://www.hyatt.com/hyatt-regency/...)

<REF_ID> [wordlim: 200] Crawled: last month;

Fitness Center
Keep active during your stay...

These cards established several candidates and supplied initial text. The result set also included unrelated academic and news results caused by broad matches involving “Chicago,” “room,” “quiet,” and “Wi-Fi.” (Tripadvisor)


Step 5: Triage the result cards

This is an in-model decision, not a separate tool call.

Triage performed

ResultDecisionReason
Hyatt official pageOpenPrimary source for location, Wi-Fi, gym and fees
TripAdvisor Hyatt pageOpenRecent quietness/noise evidence
Booking candidate listRetainCandidate discovery and rough price context
Marriott/Hilton candidatesRetain for follow-upPlausible alternatives
Old unrelated Chicago hotel articlesRejectWrong location or stale
Wi-Fi academic papersRejectKeyword match, irrelevant to hotel selection
Room-recognition paperRejectKeyword match, not lodging evidence

Nothing automatically opened every result.

The search cards themselves already contained indexed extracts. They were enough to triage many URLs without fetching the complete page.


Step 6: Open selected pages

Actual request

sent exactly as shown
{
	"open": [
		{
			"ref_id": "<HYATT_OFFICIAL_RESULT_REF>"
		},
		{
			"ref_id": "<BOOKING_HYATT_RESULT_REF>"
		},
		{
			"ref_id": "<TRIPADVISOR_HYATT_RESULT_REF>"
		}
	],
	"response_length": "long"
}

Hyatt response header

This is a close transcription; only the reference ID is replaced:

returned exactly as shown
Hyatt Regency McCormick Place Chicago | Book Your Stay Now
(https://www.hyatt.com/hyatt-regency/en-US/chimc-hyatt-regency-mccormick-place-chicago)

<REF_ID> [wordlim: 200]
Content type: text/html;
Source: open({"ref_id":"<HYATT_OFFICIAL_RESULT_REF>","lineno":null});
Total lines: 268

L0: Site Menu
L1:
L2: Hyatt Regency McCormick Place Chicago
...
L40: ## Discover Chicago From Our South Loop Hotel, Steps From Soldier Field
L42: Located in Chicago’s bustling South Loop neighborhood...

The tool did not return the raw HTML. It returned extracted, line-addressable content.

Important extracted facts included:

returned exactly as shown
L42:
	Connected to McCormick Place

L124–125:
	24-hour fitness center
	Cardio and strength equipment
	Two Peloton bikes
	Free weights
	Yoga mats

L216–218:
	Fitness Center
	Free Internet Access

L229–232:
	Destination fee
	$25 daily for bookings on or after May 15, 2026

(Hyatt)

Booking response

returned exactly as shown
www.booking.com
(https://www.booking.com/hotel/us/hyatt-regency-mccormick-place.html)

<REF_ID> [wordlim: 200]
Content type: text/html;
Source: open({"ref_id":"<BOOKING_RESULT_REF>","lineno":null});
Redirected to URL: https://www.booking.com/hotel/us/hyatt-regency-mccormick-place.html;
Total lines: 3

L0: # JavaScript is disabled
L1:
L2: In order to continue, we need to verify that you're not a robot.

The open failed to provide usable booking inventory. The search card remained available, but the page could not be independently inspected through this operation. (Booking)

TripAdvisor response

returned exactly as shown
HYATT REGENCY MCCORMICK PLACE CHICAGO - Updated 2026 Prices & Hotel Reviews
(https://www.tripadvisor.com/Hotel_Review-...)

<REF_ID> [wordlim: 200]
Content type: text/html;
Source: open({"ref_id":"<TRIPADVISOR_RESULT_REF>","lineno":null});
Total lines: 1006

L25: # Hyatt Regency Mccormick Place Chicago
...
L33: 4.2
L37: ( 7,326 reviews )
...
L132: Sleep Quality
L134: 4.3

Further down, it exposed reviews and review metadata. (Tripadvisor)


Step 7: Search within opened pages

No new web-wide query is needed when the page is already open and the required term is known.

Actual request

sent exactly as shown
{
	"find": [
		{
			"ref_id": "<OPENED_HYATT_PAGE_REF>",
			"pattern": "Free Internet Access"
		},
		{
			"ref_id": "<OPENED_HYATT_PAGE_REF>",
			"pattern": "Destination Fee"
		},
		{
			"ref_id": "<OPENED_TRIPADVISOR_PAGE_REF>",
			"pattern": "noise"
		},
		{
			"ref_id": "<OPENED_TRIPADVISOR_PAGE_REF>",
			"pattern": "quiet"
		}
	],
	"response_length": "long"
}

Exact response pattern

returned exactly as shown
Hyatt Regency McCormick Place Chicago | Book Your Stay Now
(<URL>)

<REF_ID> [wordlim: 200]
Content type: text/html;
Source: find({
	"ref_id":"<OPENED_HYATT_PAGE_REF>",
	"pattern":"Destination Fee"
});
Total lines: 268

L216:   * Fitness Center
L218:   * Free Internet Access
...
L229: ### Destination Fee
L231: A daily destination fee...
L232: Future Changes: ... $25.00...

The search for noise returned relevant review passages. The exact search for quiet returned:

returned exactly as shown
No matching text found for "quiet"

That does not mean no guest considered it quiet. It means that exact pattern was not found in the opened extracted page representation. (Hyatt)


Step 8: Evaluate contradictions and gaps

At this point, the context contained:

unclassified, needs review
Official Hyatt claims:
	Connected to McCormick Place
	Free Internet Access
	Large 24-hour gym
	$25 daily destination fee

Review evidence:
	Strong overall review volume
	Sleep Quality 4.3/5
	Some guests report quiet rooms
	Some guests report thin walls and severe noise
	Some renovation-related noise concerns

The official page can verify an amenity exists. It cannot establish whether guest-room Wi-Fi is consistently reliable or whether rooms are quiet.

The model therefore identifies unresolved questions:

unclassified, needs review
How widespread are Hyatt noise complaints?
Do nearby Hilton properties have fewer complaints?
What are the fees at each candidate?
Are rates under $300 for the actual dates?

No separate contradiction-detector tool returns that list. It is part of model synthesis.


Step 9: Generate adaptive fan-out queries

I then made a second live search call.

Actual request

sent exactly as shown
{
	"system2_search_query": [
		{
			"q": "Marriott Marquis Chicago McCormick Place official fitness center Wi-Fi destination fee"
		},
		{
			"q": "Hampton Inn Chicago McCormick Place official fitness center Wi-Fi"
		},
		{
			"q": "Hilton Garden Inn Chicago McCormick Place official fitness center Wi-Fi"
		},
		{
			"q": "Hyatt Regency McCormick Place Chicago recent noise reviews 2026"
		}
	],
	"response_length": "long"
}

This is the adaptive stage:

sent exactly as shown
Initial search:
	Find candidate universe

Adaptive search:
	Verify individual candidates
	Resolve weaknesses and contradictions

Representative returned results

returned exactly as shown
Hilton Garden Inn Chicago McCormick Place Hotel
(<OFFICIAL HILTON URL>)

<REF_ID> [wordlim: 200] Crawled: last month;

Fitness center...
directly connected to McCormick Place...
returned exactly as shown
Hotels Around McCormick Place | Hiltons by McCormick Place
(<OFFICIAL URL>)

<REF_ID> [wordlim: 200] Crawled: 2 weeks ago;

Hampton Inn...
free Wi-Fi...
indoor pool...
fitness center...
returned exactly as shown
HYATT REGENCY MCCORMICK PLACE CHICAGO - 2026 Prices & Reviews
(<TRIPADVISOR URL>)

<REF_ID> [wordlim: 200] Crawled: today;

Noise level in my room was extreme...

It also returned duplicate regional TripAdvisor pages, Reddit discussions, unrelated Wi-Fi academic papers and other noise. Those require another triage pass. (Hilton Hotels)


Step 10: Create the recommendation comparison

There is no hotel-ranking tool call such as:

unclassified, needs review
{
	"tool": "rank_hotels",
	"candidates": [...]
}

The comparison occurs in the model using the evidence currently in context.

No explicit internal JSON object is guaranteed to be created.

The functional comparison is approximately:

unclassified, needs review
For each candidate:

	1. Does it satisfy location?
	2. Does official evidence verify Wi-Fi?
	3. Does official evidence verify a gym?
	4. What mandatory fees exist?
	5. What does recent review evidence say about noise?
	6. What is the review volume and recency?
	7. Is exact price available?
	8. Are there unresolved contradictions?

A possible explicit table generated during the answer might be:

CandidateLocationWi-FiGymQuietnessFeesPrice status
Hyatt RegencyDirectly connectedOfficially freeStrongMixed$25 daily feeDates required
Hampton InnConnected/adjacentOfficially freeYesRequires deeper reviewNeeds verificationDates required
Hilton Garden InnConnected/adjacentNeeds page verificationYesRequires deeper reviewNeeds verificationDates required
Marriott MarquisDirectly connectedNeeds plan/fee verification24-hour gymRequires deeper reviewNeeds verificationDates required

That table would be output I create, not a response from a tool.


Step 11: Apply hard constraints

The user stated a maximum of $300 per night.

But the prompt omitted dates and did not specify whether $300 includes:

unclassified, needs review
Base rate
Taxes
Destination fee
Parking
Booking fees

Therefore, no candidate can yet be definitively marked as satisfying the budget.

The proper internal decision is:

unclassified, needs review
Price constraint status:
	UNRESOLVED

Reason:
	No dates
	No occupancy
	No total-price definition

Allowed output:
	Provisional candidate recommendation

Not allowed:
	Definitive claim that the hotel will cost under $300

Step 12: Decide whether more searches are needed

Possible next calls:

unclassified, needs review
If exact dates are supplied:
	Search official and booking inventory for those dates

If noise remains decisive:
	Search each finalist plus:
		"noise"
		"thin walls"
		"quiet room"
		"sleep quality"

If fees remain unclear:
	Open official policy/pricing pages

If a page contains a pricing link:
	Use click on the page-local link ID

If a page is very long:
	Use find for:
		"destination fee"
		"parking"
		"Wi-Fi"
		"fitness"
		"renovation"

The loop ends when:

unclassified, needs review
All hard constraints are verified
OR
Remaining uncertainty is clearly disclosed
OR
Additional searches mostly produce duplicates

Step 13: Generate the answer

No final-answer tool is invoked.

The model generates the answer using:

unclassified, needs review
Original request
Conversation context
Search result cards
Opened-page extracts
Find results
Source quality judgments
Constraint comparison
Uncertainty
Citation references

The source references returned by tools are used to attach citations to claims.


Function-call-stack representation

conceptual, drawn by ChatGPT
respond_to_user(prompt)
│
├── interpret_request(prompt)
│	├── identify_task("hotel recommendation")
│	├── identify_location("McCormick Place, Chicago")
│	├── identify_budget("$300/night")
│	├── identify_requirements(["quiet", "reliable Wi-Fi"])
│	├── identify_preferences(["decent gym"])
│	└── identify_missing(["dates", "occupancy", "fee treatment"])
│
├── should_use_tools()
│	└── true
│
├── create_initial_queries()
│	└── performed by model; no separate exposed tool
│
├── web_search({
│		system1_search_query: [...]
│	})
│
├── receive_model_visible_cards()
│
├── triage_cards()
│	├── retain official hotel pages
│	├── retain recent review pages
│	├── retain candidate-list pages
│	└── reject irrelevant results
│
├── open([
│		hyatt_official,
│		booking_hyatt,
│		tripadvisor_hyatt
│	])
│
├── receive_line_addressable_pages()
│
├── find([
│		"Free Internet Access",
│		"Destination Fee",
│		"noise",
│		"quiet"
│	])
│
├── identify_evidence_gaps()
│
├── create_adaptive_queries()
│	└── performed by model; no separate exposed tool
│
├── web_search({
│		system2_search_query: [...]
│	})
│
├── normalize_candidates_conceptually()
│	└── no guaranteed explicit JSON object
│
├── compare_constraints_and_evidence()
│	└── performed by model; no separate exposed recommender
│
├── determine_completion_status()
│	├── location: resolved
│	├── Wi-Fi/gym: partially resolved
│	├── quietness: mixed evidence
│	└── exact price: unresolved because dates missing
│
└── generate_answer_with_citations()

Direct answers to your architecture questions

Is the prompt sent to a separate requirements tool?

Not from my visible perspective. I interpret it directly.

Is a structured requirements JSON returned?

No. Any such JSON shown previously was illustrative.

Is there a separate fan-out query-generation tool?

Not exposed here. I generate the query strings while constructing the search call.

Does the search tool receive the whole original prompt?

The visible call contains only the arguments shown—primarily query strings and filters. I do not see an automatic full-prompt field.

Does the tool scrape every result URL?

No. Search returns cards or indexed extracts. I selectively call open on useful URLs.

Does open return the complete HTML?

No. It returns an extracted, line-addressable representation, plus metadata such as content type, total lines and operation provenance.

Does the initial prompt get sent to each opened page?

No visible evidence of that. The open request identifies the result or URL. The prompt remains in my model context and determines how I interpret the page.

Is there a separate recommendation tool?

Not for this hotel example. The model synthesizes and compares the evidence.

Are subsequent searches automatic?

They are generated when the model determines that evidence is missing, contradictory or insufficient. In standard search this can happen without a user-visible intermediate step; deep research adds a proposed plan, progress view and source history. (OpenAI Help Center)

Is there a fixed scoring formula?

No exposed universal formula. A score can be constructed explicitly, but ordinary recommendation synthesis is not returned by a fixed hotel-ranking function.