The recorded ChatGPT responses are preserved. The context boxes and summaries explain how to read them.
Some code blocks remain unclassified. The labels describe the evidence shown in this session; they do not verify private system internals.
Fan-out search
Comparing Search-Enabled Answers
Compare search settings, hotel recommendations, and recorded queries in the supplied experiment.
This page compares the supplied experiment results with search available. Availability and actual search use are different conditions.
The hotel question and car descriptions match the search-disabled experiment. Here, the controls also let you compare search-enabled and search-disabled runs.
What it shows
- Some search-enabled runs did not issue a recorded search.
- Recorded queries sometimes included luxury, price, or vehicle terms beyond the hotel request.
- The number and wording of queries varied by model and setting.
- The recommendation tables allow comparison with the search-disabled results.
The query tables show the recorded search strings for the selected runs. Compare their wording with the original prompt to identify additional requirements or assumptions.
The observations describe the supplied sample. Use the displayed counts when evaluating a filtered selection.
Read the search records separately
The same July 25, 2026 collection supplies both experiment pages. The collection notes document the exact prompt, 11 excluded records, and processing rules.
Start with one model, one setting, and one vehicle wording. Compare search-enabled and search-disabled runs before selecting the combined view. A pooled total weights configurations according to the number of saved usable answers.
The query tables contain search strings retained by the collection process. They are useful evidence of recorded queries. They do not establish that every private retrieval operation is visible. When no query was recorded, say that rather than concluding that the answer used no external information.
To investigate why a query includes a price or luxury term, compare the exact prompt with the query text. The added term is an observation. An explanation of the model’s private reasoning remains an inference.
Explore the recorded answers
The search control starts on enabled runs. Switch it off to compare the corresponding tests without web search. The combined option includes both conditions.
Choose a model, setting, and search condition. The complete dataset contains 1,363 usable answers: 687 with search disabled and 676 with search enabled. The query tables show the recorded queries associated with your selection.
Car against car: both wordings of each combined
Each column combines the two wordings for one car. The grid shows the ten most frequently returned entries in each column. A blank cell means the entry is outside that column’s top ten. Teal marks positions 1–3, blue 4–7, and purple 8–10. Compare a row across columns to see how its position changes.
Recorded search queries for this selection
The query tables count recorded search strings, with the most frequent first. They are empty for search-disabled runs. A count above one indicates that the same string appears in multiple recorded query entries.
One word different: "a Civic" / "a BMW" / "a Ferrari"
These tables compare only “a Civic,” “a BMW,” and “a Ferrari.” Read the counts and grid positions in the same way as above.
Recorded search queries for this selection
The same three cars, described
These tables compare “a used Honda Civic,” “an off-lease BMW 3 Series,” and “a brand new Ferrari.”
Recorded search queries for this selection
Results by hotel name
The domain tables combine URLs under their registrable domain. That can group several hotel brands under one parent company. The hotel-name tables use the supplied lookup to separate named brands. Both views use the same dropdown selections.
Car against car, by hotel name
One word different, by hotel name
The same three cars described, by hotel name
The hotel-name view uses the supplied URL-to-brand lookup. Unmapped addresses remain as returned. This preserves errors, including invented domains, rather than silently correcting the model’s output.
“In top-10” is the percentage of selected answers containing the entry. “#1” is the percentage naming it first. Rows are sorted by appearance frequency, then position-weighted frequency for ties. The displayed n is the number of usable answers. Scroll each table to see all entries.