Tool Requests and Results Reported in the Session

I’ll run one live example for each tool/result family I can actually invoke here, capture the returned structure, and separate exact exposed fields from UI-only or undocumented backend fields.

...

What the live tests establish

I ran actual calls in this conversation for the available result families.

The central finding is:

The request sent to the tool is JSON, but most responses are not exposed to me as raw JSON.

Instead, I receive a model-facing representation that may be:

• Structured prose
• Markdown-like text
• Line-addressable extracted content
• A rendered image
• A Python-like nested object
• A typed reference such as a web, news, image, finance, sports, forecast, or view reference

I do not receive:

• The HTTP response body from the underlying provider
• The provider’s original JSON
• A complete OpenAI-internal transport object
• Private ranking metadata
• The UI widget’s client-side data payload
• A raw serialized object that I can inspect key by key

I will therefore distinguish:

1. Exact request JSON — the actual arguments I sent.
2. Actual model-visible response — substantially the format returned to me.
3. Lossless normalized JSON — my JSON encoding of the information that was exposed.
4. Unknown internal data — fields that may exist upstream but were not shown.

I have replaced session-specific reference IDs with <REF_ID>. Those IDs exist, but they are citation handles rather than meaningful content fields.

Result families actually observed

There are presentation objects such as weather widgets, image carousels, product carousels, finance charts, schedules and standings tables, but I do not see their raw browser-side payloads.

1. Publicly documented web-search JSON

This is the one place where there is an official JSON contract rather than an inferred internal shape.

OpenAI publicly documents a web-search result item resembling:

{
	"id": "ws_abc123",
	"type": "web_search_call",
	"status": "completed",
	"action": {
		"type": "search",
		"queries": [
			"consumer uses of large language models",
			"generative AI use case survey"
		],
		"sources": [
			{
				"type": "url",
				"url": "https://example.com/report"
			},
			{
				"type": "url",
				"url": "https://example.org/methodology"
			}
		]
	}
}

The action union can also represent opening a page:

{
	"id": "ws_abc124",
	"type": "web_search_call",
	"status": "completed",
	"action": {
		"type": "open_page",
		"url": "https://example.com/report"
	}
}

Or finding text within a page:

{
	"id": "ws_abc125",
	"type": "web_search_call",
	"status": "completed",
	"action": {
		"type": "find_in_page",
		"url": "https://example.com/report",
		"pattern": "sample size"
	}
}

The public schema identifies the tool call, status, action, queries and source URLs. It does not document snippets, authors, publication dates, PageRank, JSON-LD or relevance scores as fields of sources. (OpenAI Platform)

A citation in generated output has a separate documented shape:

{
	"type": "url_citation",
	"title": "Consumer Uses of Generative AI",
	"url": "https://example.com/report",
	"start_index": 42,
	"end_index": 97
}

The title, URL and character offsets attach the citation to the generated answer. (OpenAI Platform)

That documented API object is different from the richer model-facing results shown below.

2. General web-search result

Actual request

{
	"system2_search_query": [
		{
			"q": "site:platform.openai.com/docs web search Responses API sources citations"
		}
	],
	"response_length": "long"
}

Actual model-visible format

A returned result looked substantially like this:

Streaming events | OpenAI API Reference
(https://platform.openai.com/docs/api-reference/responses-streaming/response/file_search_call/completed?lang=javascript)

<REF_ID> [wordlim: 200] Crawled: 5 months ago;
The URL of the source. ... A citation for a web resource used to generate a model response. ...

This is not a JSON object. It is a structured rendered block with:

• Title
• URL
• Reference
• Word limit
• Crawl-age display
• Extracted or summarized text

(OpenAI Platform)

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "search",
	"title": "Streaming events | OpenAI API Reference",
	"url": "https://platform.openai.com/docs/api-reference/responses-streaming/response/file_search_call/completed?lang=javascript",
	"word_limit": 200,
	"published_display": null,
	"crawled_display": "5 months ago",
	"text": "The URL of the source. ... A citation for a web resource used to generate a model response. ..."
}

The key names and nesting above are mine. The underlying values are exposed.

Fields concretely visible

{
	"title": "string",
	"url": "string",
	"reference_id": "session reference",
	"word_limit": "integer, sometimes",
	"published_display": "string or absent",
	"crawled_display": "string or absent",
	"text": "string"
}

Not visible

{
	"rank_score": null,
	"pagerank": null,
	"authority_score": null,
	"retrieval_score": null,
	"embedding_similarity": null,
	"raw_html": null,
	"json_ld": null,
	"open_graph": null,
	"canonical_tag": null,
	"author_object": null
}

Those null values mean “not exposed,” not that the internal system necessarily lacks them.

3. News result

Actual request

{
	"system2_search_query": [
		{
			"q": "latest artificial intelligence news July 2026"
		}
	],
	"response_length": "long"
}

Actual model-visible format

Amazon cuts jobs in its artificial general intelligence group
(https://www.reuters.com/business/world-at-work/amazon-cuts-jobs-its-artificial-general-intelligence-group-2026-07-22/)

<REF_ID> [wordlim: 100]
Published: 3 days ago;
Date: Wed Jul 22 14:31:11 2026 UTC

Amazon cut jobs in its artificial general intelligence group...

(Reuters)

The result exposed both:

• A relative publication display
• An absolute date/time string

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "news",
	"title": "Amazon cuts jobs in its artificial general intelligence group",
	"url": "https://www.reuters.com/business/world-at-work/amazon-cuts-jobs-its-artificial-general-intelligence-group-2026-07-22/",
	"word_limit": 100,
	"published_display": "3 days ago",
	"date_display": "Wed Jul 22 14:31:11 2026 UTC",
	"text": "Amazon cut jobs in its artificial general intelligence group..."
}

Important observation

The initial news card did not give me a normalized author field. After opening and searching the article, the extracted page exposed:

By Greg Bensinger
July 22, 2026 2:31 PM UTC
Updated July 22, 2026

It also exposed:

• The photo caption
• Photographer credit
• Dateline
• Reporting and editing credits
• Author-profile link
• Suggested topics
• Related stories

(Reuters)

That demonstrates the difference between the card and the opened page:

{
	"card": {
		"author": null,
		"published": "relative and absolute date strings"
	},
	"opened_page": {
		"visible_byline": "Greg Bensinger",
		"visible_published_at": "July 22, 2026 2:31 PM UTC",
		"visible_updated_at": "July 22, 2026",
		"photo_credit": "REUTERS/Stephane Mahe",
		"reporting_credit": "Greg Bensinger",
		"editing_credits": [
			"Chizu Nomiyama",
			"Kirsten Donovan"
		]
	}
}

Again, those opened-page fields were extracted from line-addressable text, not handed to me as that JSON object.

4. Academic result

The same broad search produced a specialized academic result.

Actual model-visible format

Artificial Intelligence Index Report 2024
(https://arxiv.org/abs/2405.19522)

<REF_ID> [wordlim: 200]
Published: 2.2 years ago;

Title: Artificial Intelligence Index Report 2024
Authors: Nestor Maslej, Loredana Fattorini, Raymond Perrault, ...
Date: Wed May 29 20:59:57 2024

The 2024 Index tracks and analyzes developments in artificial intelligence...

(arXiv)

This is richer than a general web result. It concretely exposes:

• A recognized academic-result classification
• Title
• Authors
• Exact submission/publication date
• Abstract or abstract-like content
• Relative age
• URL
• Word limit

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "academia",
	"url": "https://arxiv.org/abs/2405.19522",
	"word_limit": 200,
	"published_display": "2.2 years ago",
	"paper": {
		"title": "Artificial Intelligence Index Report 2024",
		"authors": [
			"Nestor Maslej",
			"Loredana Fattorini",
			"Raymond Perrault"
		],
		"authors_truncated_for_display": true,
		"date_display": "Wed May 29 20:59:57 2024",
		"abstract_text": "The 2024 Index tracks and analyzes developments in artificial intelligence..."
	}
}

The actual result had the complete author list; I shortened it above for readability.

Not observed

{
	"doi": null,
	"journal": null,
	"volume": null,
	"issue": null,
	"pages": null,
	"citation_count": null,
	"references": null,
	"subjects": null,
	"license": null,
	"pdf_url_as_separate_field": null,
	"arxiv_categories": null
}

Some of those may exist on arXiv or in an upstream academic index, but they were not exposed in this result.

5. Reddit result

A general search can return a specialized Reddit representation.

Actual model-visible format

Weekly AI Timeline Estimates For RSI, AGI, ASI, LEV, UBI and Home Multipurpose Robots
(https://www.reddit.com/r/accelerate/comments/...)

<REF_ID> [wordlim: 200]
Published: 4 days ago;
[Wednesday July 22 2026]
[+28 votes]

... post text ...

(Reddit)

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "reddit",
	"title": "Weekly AI Timeline Estimates For RSI, AGI, ASI, LEV, UBI and Home Multipurpose Robots",
	"url": "https://www.reddit.com/r/accelerate/comments/...",
	"word_limit": 200,
	"published_display": "4 days ago",
	"date_display": "Wednesday July 22 2026",
	"score_display": "+28 votes",
	"text": "... post text ..."
}

Fields apparently parsed or formatted

{
	"subreddit": "inferable from URL",
	"post_title": "visible",
	"post_score": "visible as display text",
	"post_date": "visible",
	"post_body": "visible or partially visible"
}

Fields not exposed in this result

{
	"reddit_post_id": null,
	"author_username": null,
	"upvote_ratio": null,
	"comment_count": null,
	"award_count": null,
	"flair": null,
	"edited": null,
	"permalink_as_separate_field": null,
	"crosspost_parent": null
}

The URL contains a Reddit post ID, but there was no separately labeled ID field.

6. Image-search result

Actual request

{
	"image_query": [
		{
			"q": "Grand Canyon sunrise National Park Service"
		}
	]
}

Actual model-visible format

An image result appeared as:

Grand Canyon National Park HDR - Mather Point Sunrise | Flickr
(https://www.flickr.com/photos/tombricker/13046251014)

<REF_ID>
Published: 7.8 years ago;

<RENDERED IMAGE>

Grand Canyon National Park HDR - Mather Point Sunrise

Mather Point is a popular sunrise location...
Uploaded on March 9, 2014
Taken on February 23, 2014
Tom B. By: Tom B.
24,713 views
149 faves
23 comments

This is much richer than a conventional thumbnail-only SERP card. The model-visible result included:

• Rendered image content
• Source-page title and URL
• Relative publication age
• Source-page text
• Creator/byline text
• Upload and capture dates
• View/favorite/comment counts
• Rights text

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "image",
	"title": "Grand Canyon National Park HDR - Mather Point Sunrise | Flickr",
	"source_page_url": "https://www.flickr.com/photos/tombricker/13046251014",
	"published_display": "7.8 years ago",
	"rendered_image": {
		"available_to_model": true,
		"raw_image_url_exposed": false,
		"binary_payload_exposed": false
	},
	"associated_page_text": {
		"image_title": "Grand Canyon National Park HDR - Mather Point Sunrise",
		"creator_display": "Tom B.",
		"uploaded_display": "March 9, 2014",
		"taken_display": "February 23, 2014",
		"views": 24713,
		"favorites": 149,
		"comments": 23,
		"rights_display": "All rights reserved"
	}
}

Important caveat

I cannot determine from the result whether every piece of text was derived from:

• Page HTML
• Image metadata
• Alt text
• Open Graph
• Flickr’s structured API data
• Search-index extraction
• A generated visual caption

The result does not provide field-level provenance.

Not exposed

{
	"original_image_url": null,
	"thumbnail_url": null,
	"width": null,
	"height": null,
	"mime_type": null,
	"file_size": null,
	"exif": null,
	"iptc": null,
	"xmp": null,
	"visual_embedding": null,
	"perceptual_hash": null,
	"image_quality_score": null,
	"safe_search_score": null
}

7. Product result

Actual request: exploratory search

{
	"product_query": {
		"search": [
			"wireless mouse"
		]
	}
}

Actual request: exact lookup

{
	"product_query": {
		"lookup": [
			"Logitech M330 Silent Plus 910-004905"
		]
	}
}

Both returned the same product family.

Actual model-visible format

Logitech M330 Silent Plus Wireless Optical USB Mouse, Black
(https://www.staples.com/.../product_2406183)

<REF_ID>

Desc:
Logitech M330 Silent Plus advanced wireless USB mouse...

Rating:
4.7/5 (1033 reviews)

Merchants:
- $22.99 (in_stock): Staples

Specs:
- Brand: Logitech
- Sale Price: $18.99
- Color: Black
- Size: Each
- Material: Rubber
- Product Category: Computers & Accessories > ... > Computer Mice

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "product",
	"title": "Logitech M330 Silent Plus Wireless Optical USB Mouse, Black",
	"url": "https://www.staples.com/logitech-m330-silent-plus-wireless-optical-usb-mouse-black-910-004905/product_2406183",
	"description": "Logitech M330 Silent Plus advanced wireless USB mouse...",
	"rating": {
		"value": 4.7,
		"scale": 5,
		"review_count": 1033
	},
	"merchants": [
		{
			"name": "Staples",
			"price": 22.99,
			"currency": "USD",
			"availability": "in_stock"
		}
	],
	"specs": {
		"Brand": "Logitech",
		"Sale Price": "$18.99",
		"Color": "Black",
		"Size": "Each",
		"Material": "Rubber",
		"Product Category": "Computers & Accessories > Computer Accessories & Peripherals > Keyboards & Mice > Computer Mice"
	}
}

Interesting inconsistency

The result contained both:

{
	"merchant_price": 22.99,
	"sale_price_spec": 18.99
}

There is no exposed provenance or explanation for the difference. That means downstream code should not assume that all price-like fields describe the same offer.

Potentially dynamic spec keys

The Specs section is not necessarily a fixed schema. Different products returned fields such as:

• Brand
• Color
• Size
• Material
• Height
• Length
• Depth
• Weight
• Sale Price
• Product Category

A faithful representation is therefore probably closer to:

{
	"specs": {
		"<provider-defined-label>": "<provider-defined-value>"
	}
}

rather than a fixed universal product schema.

8. Finance result

Actual request

{
	"finance": [
		{
			"ticker": "AMD",
			"type": "equity",
			"market": "USA"
		}
	],
	"response_length": "short"
}

Exact model-visible response structure

The response was a single structured prose paragraph:

<REF_ID> Advanced Micro Devices Inc. (AMD) is a equity in the USA market.
The price is 521.95 USD currently with a change of -17.66000 (-0.03272%)
from the previous close. The intraday high is 551.01 USD and the intraday
low is 518.5 USD. The latest open price was 547.0 USD and the intraday
volume is 27200730. The market cap is 861,217,500,000. The PE ratio is
171.13114754098362. The EPS ratio is 3.05. The latest trade time is
Friday, July 24, 23:59:18 UTC.

This is important: I did not receive a visible JSON object. The result was already verbalized into a sentence.

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "finance",
	"instrument": {
		"name": "Advanced Micro Devices Inc.",
		"ticker": "AMD",
		"type": "equity",
		"market": "USA"
	},
	"quote": {
		"price": 521.95,
		"currency": "USD",
		"change": -17.66,
		"change_percent_raw": -0.03272,
		"intraday_high": 551.01,
		"intraday_low": 518.5,
		"open": 547.0,
		"volume": 27200730,
		"latest_trade_time_display": "Friday, July 24, 23:59:18 UTC"
	},
	"fundamentals": {
		"market_cap": 861217500000,
		"pe_ratio": 171.13114754098362,
		"eps": 3.05
	}
}

Percentage ambiguity

The response said:

-17.66000 (-0.03272%)

Mathematically, a change of -17.66 from approximately 539.61 is about:

{
	"fractional_change": -0.03272,
	"percentage_change": -3.272
}

The rendered output attached a % sign to -0.03272. This suggests either:

• The upstream value was a decimal fraction and the renderer added % without multiplying by 100, or
• The value’s intended semantics differ from the conventional interpretation.

That is a concrete example of why seeing only rendered prose can lose type semantics.

Not visible

{
	"exchange": null,
	"market_status": null,
	"bid": null,
	"ask": null,
	"quote_delay": null,
	"provider": null,
	"historical_series": null,
	"premarket": null,
	"after_hours": null,
	"isin": null,
	"figi": null
}

A chart widget may receive historical points through another channel, but that time-series object was not shown to me in the text result.

9. Weather result

Actual request

{
	"weather": [
		{
			"location": "Phoenix, Arizona",
			"duration": 3
		}
	],
	"response_length": "long"
}

Actual model-visible format

The beginning of the result was:

<REF_ID> Weather for Phoenix, AZ, United States:

Current Conditions:
Sunny, 93°F (34°C)

Daily Forecast:
Sunday, July 26:
Partly sunny and toasty conditions; a thunderstorm in the area late this
afternoon; heat will be dangerous, minimize outdoor activity,
High: 112°F (44°C), Low: 93°F (34°C)

Monday, July 27:
Partly sunny and toasty conditions; a thunderstorm in spots late in the
afternoon; heat will be dangerous, minimize outdoor activity,
High: 112°F (44°C), Low: 94°F (34°C)

Tuesday, July 28:
Hot with intervals of clouds and sun; heat will be dangerous,
High: 111°F (44°C), Low: 91°F (33°C)

Severe weather alerts:
[SevereWeather(
	start_timestamp=1784826000,
	end_timestamp=1785207600,
	utc_offset_sec=-25200.0,
	name='Buckeye/Avondale',
	summary='Extreme Heat Warning in effect until Monday, 8:00 PM MST...',
	details='...'
), ...]

This response is particularly informative because it combines:

1. Human-readable forecast prose
2. A Python-representation-like list of structured alert objects

It is not valid JSON because it uses:

• SevereWeather(...)
• Single-quoted strings
• Named constructor arguments
• Python float representations
• An object-list display rather than JSON dictionaries

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "forecast",
	"location": {
		"display": "Phoenix, AZ, United States"
	},
	"current_conditions": {
		"condition": "Sunny",
		"temperature_f": 93,
		"temperature_c": 34
	},
	"daily_forecast": [
		{
			"date_display": "Sunday, July 26",
			"description": "Partly sunny and toasty conditions; a thunderstorm in the area late this afternoon; heat will be dangerous, minimize outdoor activity",
			"high_f": 112,
			"high_c": 44,
			"low_f": 93,
			"low_c": 34
		},
		{
			"date_display": "Monday, July 27",
			"description": "Partly sunny and toasty conditions; a thunderstorm in spots late in the afternoon; heat will be dangerous, minimize outdoor activity",
			"high_f": 112,
			"high_c": 44,
			"low_f": 94,
			"low_c": 34
		},
		{
			"date_display": "Tuesday, July 28",
			"description": "Hot with intervals of clouds and sun; heat will be dangerous, minimize outdoor activity",
			"high_f": 111,
			"high_c": 44,
			"low_f": 91,
			"low_c": 33
		}
	],
	"severe_weather_alerts": [
		{
			"start_timestamp": 1784826000,
			"end_timestamp": 1785207600,
			"utc_offset_sec": -25200.0,
			"name": "Buckeye/Avondale",
			"summary": "Extreme Heat Warning in effect until Monday, 8:00 PM MST. Source: U.S. National Weather Service",
			"details": "<FULL ALERT TEXT>"
		}
	]
}

Alert fields concretely observed

{
	"start_timestamp": "integer",
	"end_timestamp": "integer",
	"utc_offset_sec": "floating-point number",
	"name": "string",
	"summary": "string",
	"details": "string"
}

This is the clearest example from the tests of an underlying typed object leaking through into the model-visible representation.

10. Sports schedule

Actual request

{
	"sports": [
		{
			"tool": "sports",
			"fn": "schedule",
			"league": "mlb",
			"team": "NYY",
			"num_games": 5
		}
	],
	"response_length": "medium"
}

Exact model-visible format

<REF_ID>
Yankees @ Phillies on Sunday, Jul 26, 2026 at 04:20 PM PDT
Yankees @ White Sox on Monday, Jul 27, 2026 at 04:40 PM PDT
Yankees @ White Sox on Tuesday, Jul 28, 2026 at 04:40 PM PDT
Yankees @ White Sox on Wednesday, Jul 29, 2026 at 04:40 PM PDT
Yankees @ White Sox on Thursday, Jul 30, 2026 at 11:10 AM PDT

No event objects were shown. The result was a line-oriented schedule.

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "sports",
	"result_subtype": "schedule",
	"events": [
		{
			"away_team_display": "Yankees",
			"home_team_display": "Phillies",
			"date_display": "Sunday, Jul 26, 2026",
			"time_display": "04:20 PM",
			"timezone_display": "PDT"
		},
		{
			"away_team_display": "Yankees",
			"home_team_display": "White Sox",
			"date_display": "Monday, Jul 27, 2026",
			"time_display": "04:40 PM",
			"timezone_display": "PDT"
		}
	]
}

Not exposed

{
	"event_id": null,
	"league_id": null,
	"home_team_id": null,
	"away_team_id": null,
	"start_timestamp": null,
	"venue": null,
	"broadcasts": null,
	"probable_pitchers": null,
	"ticket_url": null,
	"event_status": null,
	"provider": null
}

The UI schedule widget may have more information, but the model-visible text did not.

11. Sports standings

Actual request

{
	"sports": [
		{
			"tool": "sports",
			"fn": "standings",
			"league": "mlb"
		}
	],
	"response_length": "long"
}

Actual model-visible format

<REF_ID>

# Conference: American League

## Division: East
Tampa Bay Rays 61-43
New York Yankees 59-45
Boston Red Sox 53-50
Baltimore Orioles 51-54
Toronto Blue Jays 48-57

## Division: Central
Chicago White Sox 54-49
Cleveland Guardians 54-52
...

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "sports",
	"result_subtype": "standings",
	"conferences": [
		{
			"name": "American League",
			"divisions": [
				{
					"name": "East",
					"teams": [
						{
							"name": "Tampa Bay Rays",
							"wins": 61,
							"losses": 43
						},
						{
							"name": "New York Yankees",
							"wins": 59,
							"losses": 45
						}
					]
				}
			]
		}
	]
}

Values that are derived rather than directly returned

From ordering and records, I could derive:

{
	"rank": 1,
	"games_played": 104,
	"win_percentage": 0.5865384615
}

But none of those were explicitly displayed. A rigorous extraction should preserve provenance:

{
	"rank": {
		"value": 1,
		"provenance": "derived_from_display_order"
	},
	"win_percentage": {
		"value": 0.5865384615,
		"provenance": "calculated_from_wins_and_losses"
	}
}

12. Calculator result

Actual request

{
	"calculator": [
		{
			"expression": "275000000*0.4",
			"prefix": "",
			"suffix": " tokens"
		}
	]
}

Exact model-visible format

<REF_ID> 275000000*0.4 = 110000000.0 tokens

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "calculator",
	"expression": "275000000*0.4",
	"result": 110000000.0,
	"prefix": "",
	"suffix": " tokens",
	"rendered": "275000000*0.4 = 110000000.0 tokens"
}

The numeric type was rendered with a decimal, even though the mathematical result is an integer quantity.

Not exposed:

{
	"parser_ast": null,
	"numeric_precision": null,
	"calculation_engine": null,
	"rounding_mode": null
}

13. Time result

Actual request

{
	"time": [
		{
			"utc_offset": "-07:00"
		}
	]
}

Exact model-visible format

<REF_ID> The time in UTC-07:00 is Jul 26, 2026, 7:24:10 AM

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "time",
	"utc_offset": "-07:00",
	"date_display": "Jul 26, 2026",
	"time_display": "7:24:10 AM",
	"rendered": "The time in UTC-07:00 is Jul 26, 2026, 7:24:10 AM"
}

Not exposed:

{
	"unix_timestamp": null,
	"iana_timezone": null,
	"dst_status": null,
	"locale": null,
	"source_clock": null
}

Because the request specifies only an offset, the result cannot inherently distinguish Phoenix from Los Angeles or another location sharing that offset at the moment.

14. Opened HTML page

Actual request

{
	"open": [
		{
			"ref_id": "<SEARCH_RESULT_REF_ID>"
		}
	],
	"response_length": "long"
}

Actual model-visible header

Create a model response | OpenAI API Reference
(https://platform.openai.com/docs/api-reference/responses-streaming/response/file_search_call/completed?lang=javascript)

<REF_ID> [wordlim: 200]
Content type: text/html;
Source: open({"ref_id":"<SEARCH_RESULT_REF_ID>","lineno":null});
Redirected to URL: https://developers.openai.com/api/reference/resources/responses/methods/create;
Total lines: 9628

It was followed by line-addressed content:

L1529: ResponseFunctionWebSearch = object { id, action, status, type }
L1530:
L1531: The results of a web search tool call.
L1532:
L1533: id: string
...

(OpenAI Platform)

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "view",
	"title": "Create a model response | OpenAI API Reference",
	"url": "https://platform.openai.com/docs/api-reference/responses-streaming/response/file_search_call/completed?lang=javascript",
	"word_limit": 200,
	"content_type": "text/html",
	"source_operation": {
		"type": "open",
		"ref_id": "<SEARCH_RESULT_REF_ID>",
		"lineno": null
	},
	"redirected_url": "https://developers.openai.com/api/reference/resources/responses/methods/create",
	"total_lines": 9628,
	"lines": [
		{
			"line": 1529,
			"text": "ResponseFunctionWebSearch = object { id, action, status, type }"
		},
		{
			"line": 1531,
			"text": "The results of a web search tool call."
		}
	]
}

Rich features exposed only after opening

• Content type
• Redirect destination
• Total extracted line count
• Stable line addresses
• Link IDs embedded in lines
• Headings
• Some page structure
• Source operation provenance

Still not exposed:

• Raw HTML
• DOM nodes
• CSS
• JavaScript
• HTTP headers
• Cookies
• Network requests
• Raw JSON-LD
• Browser console
• Redirect chain before the final URL

15. Find-in-page result

Actual request

{
	"find": [
		{
			"ref_id": "<OPENED_PAGE_REF_ID>",
			"pattern": "ResponseFunctionWebSearch"
		}
	]
}

Actual model-visible header

Create a model response | OpenAI API Reference
(https://platform.openai.com/docs/api-reference/responses-streaming/response/file_search_call/completed?lang=javascript)

<REF_ID> [wordlim: 200]
Content type: text/html;
Source: find({
	"ref_id":"<OPENED_PAGE_REF_ID>",
	"pattern":"ResponseFunctionWebSearch"
});
Redirected to URL: https://developers.openai.com/api/reference/resources/responses/methods/create;
Total lines: 9628

The returned content was centered around occurrences of the requested pattern. (OpenAI Platform)

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "view",
	"view_subtype": "find_in_page",
	"title": "Create a model response | OpenAI API Reference",
	"url": "https://platform.openai.com/docs/api-reference/responses-streaming/response/file_search_call/completed?lang=javascript",
	"content_type": "text/html",
	"source_operation": {
		"type": "find",
		"ref_id": "<OPENED_PAGE_REF_ID>",
		"pattern": "ResponseFunctionWebSearch"
	},
	"redirected_url": "https://developers.openai.com/api/reference/resources/responses/methods/create",
	"total_lines": 9628,
	"matching_regions": [
		{
			"start_line": 1529,
			"end_line": 1615,
			"lines": [
				{
					"line": 1529,
					"text": "ResponseFunctionWebSearch = object { id, action, status, type }"
				}
			]
		}
	]
}

The actual output did not wrap matches into matching_regions; that is my natural JSON representation of the returned passages.

16. Clicked-page result

Actual request

{
	"click": [
		{
			"ref_id": "<OPENED_REUTERS_ARTICLE_REF_ID>",
			"id": 111
		}
	],
	"response_length": "short"
}

The link ID 111 corresponded to the author link in the opened article.

Actual model-visible header

Greg Bensinger | Reuters
(https://www.reuters.com/authors/greg-bensinger/)

<REF_ID> [wordlim: 100]
Content type: text/html;
Source: click({
	"ref_id":"<OPENED_REUTERS_ARTICLE_REF_ID>",
	"id":111
});
Total lines: 376

It was followed by line-addressed page content, including:

L151: Image
L152: # Greg Bensinger
L154: Thomson Reuters
L157: Greg Bensinger joined Reuters as a technology correspondent...
L161: 158 results

(Reuters)

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "view",
	"view_subtype": "clicked_page",
	"title": "Greg Bensinger | Reuters",
	"url": "https://www.reuters.com/authors/greg-bensinger/",
	"word_limit": 100,
	"content_type": "text/html",
	"source_operation": {
		"type": "click",
		"parent_ref_id": "<OPENED_REUTERS_ARTICLE_REF_ID>",
		"link_id": 111
	},
	"total_lines": 376,
	"lines": [
		{
			"line": 152,
			"text": "# Greg Bensinger"
		},
		{
			"line": 154,
			"text": "Thomson Reuters"
		},
		{
			"line": 161,
			"text": "158 results"
		}
	]
}

The link ID is page-extraction-local. It is not necessarily an HTML element ID, database ID or stable URL identifier.

17. Opened PDF

Actual request

{
	"open": [
		{
			"ref_id": "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf"
		}
	],
	"response_length": "short"
}

Exact model-visible format

(https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf)

<REF_ID> [wordlim: 200]
Content type: application/pdf;
Number of pages: 1;
Source: open({
	"ref_id":"https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
	"lineno":null
});
Total lines: 1

L0@P0: Dummy PDF file

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "view",
	"view_subtype": "pdf",
	"url": "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
	"word_limit": 200,
	"content_type": "application/pdf",
	"number_of_pages": 1,
	"source_operation": {
		"type": "open",
		"ref_id": "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
		"lineno": null
	},
	"total_lines": 1,
	"lines": [
		{
			"line": 0,
			"page_index": 0,
			"text": "Dummy PDF file"
		}
	]
}

The line notation contains both dimensions:

L0@P0
│  └── PDF page index 0
└───── Extracted text line 0

Not exposed

{
	"pdf_version": null,
	"document_author": null,
	"document_title_metadata": null,
	"creator_application": null,
	"creation_date": null,
	"modification_date": null,
	"xmp": null,
	"embedded_files": null,
	"fonts": null,
	"page_dimensions": null,
	"encryption": null,
	"digital_signatures": null
}

18. PDF screenshot

Actual request

{
	"screenshot": [
		{
			"ref_id": "<OPENED_PDF_REF_ID>",
			"pageno": 0
		}
	],
	"response_length": "short"
}

Model-visible result

The result contained:

(https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf)

<REF_ID>

<RENDERED PAGE IMAGE>

The rendered image showed the first PDF page.

Lossless normalized representation

{
	"ref_id": "<REF_ID>",
	"reference_family": "view",
	"view_subtype": "pdf_screenshot",
	"source_pdf_url": "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
	"page_index": 0,
	"rendered_image": {
		"available_to_model": true,
		"raw_image_url_exposed": false,
		"binary_payload_exposed": false,
		"width": null,
		"height": null,
		"mime_type": null
	}
}

The screenshot is genuinely provided to my visual input. I am not shown its binary encoding or filesystem path.

19. Failed fetch or click

One attempted PDF click produced a concrete error representation.

Actual model-visible format

Internal Error ()

<REF_ID> [wordlim: 200]
Source: click({
	"ref_id":"<OPENED_PAGE_REF_ID>",
	"id":45
});
Total lines: 1

L0: Failed to fetch https://hai.stanford.edu/assets/files/hai_ai-index-report-2024-smaller2.pdf:
(400) Content length is too large: 24167379

Lossless normalized JSON

{
	"ref_id": "<REF_ID>",
	"reference_family": "view",
	"view_subtype": "error",
	"title": "Internal Error",
	"source_operation": {
		"type": "click",
		"parent_ref_id": "<OPENED_PAGE_REF_ID>",
		"link_id": 45
	},
	"total_lines": 1,
	"error": {
		"operation": "fetch",
		"url": "https://hai.stanford.edu/assets/files/hai_ai-index-report-2024-smaller2.pdf",
		"status_or_internal_code": 400,
		"message": "Content length is too large",
		"reported_content_length": 24167379
	}
}

Whether 24167379 represents exact bytes is not explicitly labeled in the error text, although that is the natural interpretation.

A malformed sports request produced a different tool-level error:

Found no tool response. This likely means the arguments you provided were not valid.

That error did not produce a source reference.

20. Rich presentation objects

These are different from the underlying result references.

Examples

• Finance price chart
• Weather forecast widget
• Sports schedule widget
• Sports standings table
• Product carousel
• Image carousel
• News-source link group

A presentation instruction might reference a result:

finance chart → finance result reference
weather widget → forecast result reference
schedule widget → sports schedule reference

But I do not receive a visible object like:

{
	"widget": {
		"chart_type": "line",
		"x_axis": {},
		"series": [],
		"interaction_config": {}
	}
}

The client application may receive something of that kind, but it is not placed in my inspectable context.

That means I can determine:

{
	"finance_result_can_render_chart": true,
	"weather_result_can_render_forecast_widget": true
}

But I cannot inspect:

{
	"chart_points": "unknown",
	"widget_configuration": "unknown",
	"client_component_props": "unknown"
}

21. Map and local results

I do not have a dedicated map/local-business function in the tool interface used for these calls.

A conventional web query can find local pages, and some ChatGPT user interfaces can render maps or local entity results. However, I cannot run a dedicated map-result operation here and inspect its response.

Therefore, I cannot honestly provide an observed local-result JSON shape from this session.

The status is:

{
	"result_family": "map_or_local",
	"invokable_in_current_tool_interface": false,
	"raw_schema_visible": false,
	"known_to_exist_in_some_product_interfaces": true
}

22. What “the exact representation I receive” really means

The closest representation of the system is:

JSON request arguments
	↓
Tool execution
	↓
Provider or search-system data
	↓
Unknown internal transformation
	↓
Model-visible rendered result

For example, finance may begin upstream as something resembling:

{
	"symbol": "AMD",
	"regularMarketPrice": 521.95,
	"regularMarketVolume": 27200730
}

But what I actually see is:

Advanced Micro Devices Inc. (AMD) is a equity...
The price is 521.95 USD...

I cannot know whether the speculative upstream JSON above is remotely accurate, because the transformation occurs before the result enters my visible context.

Weather is slightly different because a typed object representation survived:

SevereWeather(
	start_timestamp=...,
	end_timestamp=...,
	utc_offset_sec=...,
	name=...,
	summary=...,
	details=...
)

That is stronger evidence of an underlying object’s field names, although it is still not proof of the original provider’s JSON schema. It may be an intermediate OpenAI-side data class.

Most defensible conclusions

Directly established

• Search requests are passed as structured JSON arguments.
• The responses I receive are generally rendered representations, not raw JSON.
• Result references have distinct families such as search, news, academia, reddit, image, product, finance, forecast, sports, calculator, time and view.
• Opened pages expose content type, operation provenance, line counts, line addresses and link identifiers.
• PDFs add MIME type, page count and page-indexed text.
• Weather alerts expose identifiable constructor-style field names.
• Finance, sports, calculator and time results are rendered primarily as prose or line-oriented text.
• Product results expose labeled subsections and dynamic specification fields.
• Image results can include an actual visual input plus extensive source-page text.

Not established

• Exact internal backend JSON
• Exact upstream provider schemas
• Search ranking scores
• PageRank or authority values
• Client widget props
• Raw image or PDF byte representations
• Raw structured webpage metadata
• Whether every visible metadata value comes from HTML, JSON-LD, an API, a search index or generated extraction
• Whether field names in a normalized JSON object match internal field names

The accurate final formulation is:

The examples above are based on actual calls and actual model-visible results from this conversation. The textual formats are concrete. The JSON versions are lossless normalizations I created from those concrete results; they are not copies of a hidden backend serialization.