OA9 OppAlerts from Ben Wills
How AI Search Works: From Prompt to Response A guide for SEOs, AI search marketers, and marketing teams
Work with me
Work in progress Transcript and examples under review.

The recorded ChatGPT responses are preserved. The context boxes and summaries explain how to read them.

Some code blocks remain unclassified. The labels describe the evidence shown in this session; they do not verify private system internals.

Read the announcement →

Evidence and sourcing

Asking Where the Example Data Came From

ChatGPT distinguishes visible information from JSON structures it created afterward.

Context

We ask whether the sample structures came from actual tool calls. ChatGPT acknowledges that it constructed the JSON and supplied illustrative values.

This reply explains why a plausible example must be checked against its source.

The short version

Information and format

ChatGPT distinguishes information that a tool can display from the JSON it wrote to organize that information. Titles, URLs, dates, prices, and scores can be visible without revealing the original property names or nesting.

For example, a displayed sports score identifies the teams and result. That display alone cannot establish whether an underlying object uses homeTeam, competitors, or another field name.

Invented fields

A field shown as null may have been added to illustrate missing information. It does not prove that the underlying service has that field or stores a null value.

How to check the examples

Compare documented claims with the linked documentation. Compare reported requests and results with tool logs where available. The next pages ask ChatGPT to run tools and then distinguish the returned information from its own formatting.

Our prompt could be something like

I want to confirm what you mean by observable.

Do you mean this is the shape of an actual result you received in this chat session, or something close to it? That it is not speculation about how this works internally when you are dealing with fan-out queries and additional research?

Is that correct? Not a guess, but an actual representation of what you use, and reasonably close to real sample data?

ChatGPT's response

Partly correct, with an important qualification.

When I labeled a field “observed,” I meant that the underlying information is something the tool interface can expose to me in actual results—for example:

  • A result reference such as turn1search0
  • Page title
  • URL
  • Extracted text or snippet
  • A displayed publication or crawl date
  • Finance values such as ticker, price, change, volume, and market cap
  • Sports teams, dates, scores, and event status
  • An image preview and its associated source page

Those are not guesses about what a search system might theoretically contain.

However, the complete JSON objects I showed were not raw internal JSON payloads received verbatim by me. They were normalized representations I constructed from the fields that are exposed in the rendered tool output.

The accurate relationship is:

unclassified, needs review
Actual tool result exposed to me
	↓
I identify the visible fields
	↓
I reorganize them into a clean JSON example

It is not:

unclassified, needs review
Internal search system returns this exact JSON
	↓
I copied it verbatim

Three levels of concreteness

1. Directly observed value

An actual result may expose something equivalent to:

unclassified, needs review
Advanced Micro Devices Inc.
AMD
$123.45
+2.10%
Volume: 27,200,730

Those individual facts are concrete tool output.

2. Normalized representation of observed values

I might reorganize that into:

ChatGPT JSON, invented values
{
	"instrument": {
		"name": "Advanced Micro Devices Inc.",
		"ticker": "AMD"
	},
	"quote": {
		"price": 123.45,
		"change_percent": 2.1,
		"volume": 27200730
	}
}

That JSON structure is my representation. The values and field concepts may be observed, but the nesting and property names are not guaranteed to match the internal system.

3. Proposed or inferred fields

Fields such as these were included to show what a comprehensive normalized record might contain:

ChatGPT JSON, invented values
{
	"provider_result_id": null,
	"authority_score": null,
	"canonical_url": null,
	"license": null,
	"syndication": {
		"is_syndicated": null
	}
}

Those are not evidence that the internal result has those properties set to null. They mean:

This would be a useful field in a comprehensive schema, but I cannot observe or verify it.

Was the sample data from an actual result in this chat?

Not every example value should be treated as a verbatim capture of a result from this specific chat session.

Some examples were:

  • Representative sample values
  • Values modeled on the kinds of fields exposed by the tool
  • Normalized illustrations of a rendered result
  • Derived values added to demonstrate the data type

Therefore, statements such as:

“This is the exact shape of an actual result received in this chat”

would be too strong.

The more accurate statement is:

The examples are reasonably close semantic representations of the information the tools expose, but they are not verified copies of the tools’ raw internal JSON.

What I can claim confidently by type

Result typeConcrete observable informationExact backend JSON known?
General webReference ID, title, URL, extracted content/snippet, sometimes dates and word limitNo
NewsReference ID, headline, URL, publisher/date information, summary or snippetNo
ImageReference ID, rendered image, source-page information, associated textNo
FinanceInstrument identity and structured quote/fundamental valuesNo
Sports scheduleLeague/team context, event dates, status, teams and scoresNo
Sports standingsTeams, divisions/conferences, wins and lossesNo
WeatherLocation, conditions, temperatures and forecast valuesNo
TimeUTC offset and current date/timeNo

“Observable” does not necessarily mean a literal field

There is another subtle distinction.

Suppose a result is displayed as:

conceptual
Yankees 8
Pirates 5
Final
Monday, July 20 at 4:05 PM PDT

I can concretely observe:

  • Yankees
  • Pirates
  • Scores 8 and 5
  • Final status
  • Date and time
  • Time zone

But I do not know whether the underlying payload looked like:

returned exactly as shown
{
	"homeTeam": "Yankees",
	"homeScore": 8
}

or:

unclassified, needs review
{
	"competitors": [
		{
			"homeAway": "home",
			"displayName": "New York Yankees",
			"score": "8"
		}
	]
}

or some third format that was transformed before reaching me.

Thus:

  • The information is observable.
  • Its original key names and nesting are not.

What was too strong in my previous answer

Phrases such as:

“The actual finance result exposed these values”

should be interpreted as:

“These are values of the kind exposed by the rendered finance result.”

It should not be interpreted as:

“I inspected a raw JSON payload with exactly these keys and copied it.”

Similarly, the Observable normalized … shape sections were intended to mean:

A JSON representation containing the maximum fields that can be supported from visible tool output.

They were not literal serialization schemas.

The most precise terminology

I should have used these labels:

Better labelMeaning
Directly exposedThe value or information appears in tool output
Normalized from exposed dataI converted exposed information into my own JSON structure
DerivedI calculated or inferred it from exposed values
DocumentedOfficial public documentation specifies the field
IllustrativeSample data used to demonstrate type and structure
ProposedRecommended field for a research system
Unknown internallyThe backend may contain it, but I cannot verify that

So the final confirmation is:

The field concepts labeled observable were grounded in information the tools can actually expose. The JSON shapes themselves were clean reconstructions, not verbatim internal payloads. They are reasonably close as semantic sample data, but they should not be used as authoritative OpenAI backend schemas or exact integration contracts.