ChatGPT disclosed things in this session that I did not expect it to disclose, and there is no guarantee it stays available. I would rather people had time to use it than had a tidier version of it later.
So: every word of every ChatGPT response here is verbatim, and that part is checked automatically on every build. What is not finished is the presentation. The color coding on the code blocks is incomplete and some of it is imprecise, and there are notes to myself still sitting in the page.
Parts 1 and 2 are close to empty and are getting a lot of detail over the next week or so, along with a cleanup pass on everything else. Worth checking back.
Search tool calls
20 Live ChatGPT Search Tool Calls and What They Return
Requests go out as JSON. Responses come back as plain text. Twenty result families, run live.
So we tell it to stop describing and go run the calls. It does.
Twenty result types, live, in the session. From here on the values are real: an actual stock price, an actual baseball score, an actual Reuters article with its byline, an actual heat warning for Phoenix.
The headline finding is at the top of the reply. Requests go out as JSON. Responses come back as plain text. Everything a marketer cares about follows from that.
The longest page here, and the most important one.
This is the turn that makes the rest of the resource worth publishing. It stops describing and runs the calls.
Twenty result families, live, in the session, with the request that went out and the text that came back. The headline finding is stated before any of them: the request sent to the tool is JSON, but the responses are not. What comes back is a rendered representation. Structured prose, markdown-like text, line-addressed page content, an image, or something resembling a Python object.
What you can actually control
The requests are real, and they show the levers. Two search routes,
system1_search_query and system2_search_query. A query can carry a recency
window in days and a domains allowlist. There is a response_length of short,
medium or long. Beyond search there are separate calls for products, images,
finance, weather, sports, calculation and time, plus open, find, click and
screenshot for working with a fetched page.
What comes back
A web result arrives as a title, a URL in parentheses, a reference ID, a marker
like [wordlim: 200], a relative age such as "Crawled: 5 months ago," and an
extract. News results carry both a relative age and an absolute timestamp.
Academic results add an author list and an abstract. Reddit results add a vote
count. Products arrive as labeled sections for description, rating, merchants and
specs, where the spec keys vary by retailer.
The finance bug
Finance is the one to read closely. The whole quote comes back as a sentence: name, ticker, price, change, intraday high and low, volume, market cap, P/E and EPS, written out in prose.
And it contains a real bug. The response reads -17.66000 (-0.03272%) when the
actual change is -3.272%. Someone attached a percent sign to a decimal fraction
without multiplying by 100.
That single line proves the text is a lossy rendering of something structured underneath.
Weather, the exception
Its severe-weather alerts arrive as constructor-style text, SevereWeather(...),
with field names intact: start timestamp, end timestamp, UTC offset, name,
summary, details.
It is the one place real internal field names are visible.
Failure is a result
Opened pages return a header with content type, the operation that produced them, any redirect, and a total line count, followed by numbered lines. PDFs use a combined line-and-page notation.
Failures come back as structured text too, which matters more than it sounds. A page that cannot be fetched is a result, not a silence.
I would like to see those internal representations.
Actually perform fan-out queries or internal searches of all the different types you listed above. I think there were 15 or 20 of them.
I understand if you cannot show me the exact data. If you can, show me the exact JSON of what is received. That is my ideal.
Go ahead and do that, and show me exactly what those responses look like.
I’ll run one live example for each tool/result family I can actually invoke here, capture the returned structure, and separate exact exposed fields from UI-only or undocumented backend fields.
...
What the live tests establish
I ran actual calls in this conversation for the available result families.
The central finding is:
The request sent to the tool is JSON, but most responses are not exposed to me as raw JSON.
Instead, I receive a model-facing representation that may be:
- Structured prose
- Markdown-like text
- Line-addressable extracted content
- A rendered image
- A Python-like nested object
- A typed reference such as a web, news, image, finance, sports, forecast, or view reference
I do not receive:
- The HTTP response body from the underlying provider
- The provider’s original JSON
- A complete OpenAI-internal transport object
- Private ranking metadata
- The UI widget’s client-side data payload
- A raw serialized object that I can inspect key by key
I will therefore distinguish:
- Exact request JSON — the actual arguments I sent.
- Actual model-visible response — substantially the format returned to me.
- Lossless normalized JSON — my JSON encoding of the information that was exposed.
- Unknown internal data — fields that may exist upstream but were not shown.
I have replaced session-specific reference IDs with <REF_ID>. Those IDs exist, but they are citation handles rather than meaningful content fields.
Result families actually observed
| Family | Reference category | Actual return format |
|---|---|---|
| General web result | search | Title, URL, metadata and extracted text |
| News result | news | Headline, URL, publication metadata and snippet |
| Academic result | academia | Title, URL, authors, date and abstract-like text |
| Reddit result | reddit | Title, URL, date, votes and post text |
| Image result | image | Rendered image, source URL and associated text |
| Product result | product | Product, description, rating, merchants and specs |
| Finance result | finance | Structured values rendered as prose |
| Weather result | forecast | Forecast prose plus nested alert objects |
| Sports schedule | sports | One text line per event |
| Sports standings | sports | Conference/division hierarchy and team records |
| Calculator result | calculator | Expression and result |
| Time result | time | Offset and current time |
| Opened HTML page | view | Content type, source operation, URL, lines and links |
| Find-in-page result | view | Same page representation centered on matches |
| Clicked-page result | view | Opened linked page with line-addressable content |
| Opened PDF | view | MIME type, page count and page-addressed text |
| PDF screenshot | view | Rendered page image |
| Failed open/click | view | Error description and failed-operation context |
There are presentation objects such as weather widgets, image carousels, product carousels, finance charts, schedules and standings tables, but I do not see their raw browser-side payloads.
1. Publicly documented web-search JSON
This is the one place where there is an official JSON contract rather than an inferred internal shape.
OpenAI publicly documents a web-search result item resembling:
{
"id": "ws_abc123",
"type": "web_search_call",
"status": "completed",
"action": {
"type": "search",
"queries": [
"consumer uses of large language models",
"generative AI use case survey"
],
"sources": [
{
"type": "url",
"url": "https://example.com/report"
},
{
"type": "url",
"url": "https://example.org/methodology"
}
]
}
}The action union can also represent opening a page:
{
"id": "ws_abc124",
"type": "web_search_call",
"status": "completed",
"action": {
"type": "open_page",
"url": "https://example.com/report"
}
}Or finding text within a page:
{
"id": "ws_abc125",
"type": "web_search_call",
"status": "completed",
"action": {
"type": "find_in_page",
"url": "https://example.com/report",
"pattern": "sample size"
}
}The public schema identifies the tool call, status, action, queries and source URLs. It does not document snippets, authors, publication dates, PageRank, JSON-LD or relevance scores as fields of sources. (OpenAI Platform)
A citation in generated output has a separate documented shape:
{
"type": "url_citation",
"title": "Consumer Uses of Generative AI",
"url": "https://example.com/report",
"start_index": 42,
"end_index": 97
}The title, URL and character offsets attach the citation to the generated answer. (OpenAI Platform)
That documented API object is different from the richer model-facing results shown below.
2. General web-search result
Actual request
{
"system2_search_query": [
{
"q": "site:platform.openai.com/docs web search Responses API sources citations"
}
],
"response_length": "long"
}Actual model-visible format
A returned result looked substantially like this:
Streaming events | OpenAI API Reference
(https://platform.openai.com/docs/api-reference/responses-streaming/response/file_search_call/completed?lang=javascript)
<REF_ID> [wordlim: 200] Crawled: 5 months ago;
The URL of the source. ... A citation for a web resource used to generate a model response. ...This is not a JSON object. It is a structured rendered block with:
- Title
- URL
- Reference
- Word limit
- Crawl-age display
- Extracted or summarized text
Lossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "search",
"title": "Streaming events | OpenAI API Reference",
"url": "https://platform.openai.com/docs/api-reference/responses-streaming/response/file_search_call/completed?lang=javascript",
"word_limit": 200,
"published_display": null,
"crawled_display": "5 months ago",
"text": "The URL of the source. ... A citation for a web resource used to generate a model response. ..."
}The key names and nesting above are mine. The underlying values are exposed.
Fields concretely visible
{
"title": "string",
"url": "string",
"reference_id": "session reference",
"word_limit": "integer, sometimes",
"published_display": "string or absent",
"crawled_display": "string or absent",
"text": "string"
}Not visible
{
"rank_score": null,
"pagerank": null,
"authority_score": null,
"retrieval_score": null,
"embedding_similarity": null,
"raw_html": null,
"json_ld": null,
"open_graph": null,
"canonical_tag": null,
"author_object": null
}Those null values mean “not exposed,” not that the internal system necessarily lacks them.
3. News result
Actual request
{
"system2_search_query": [
{
"q": "latest artificial intelligence news July 2026"
}
],
"response_length": "long"
}Actual model-visible format
Amazon cuts jobs in its artificial general intelligence group
(https://www.reuters.com/business/world-at-work/amazon-cuts-jobs-its-artificial-general-intelligence-group-2026-07-22/)
<REF_ID> [wordlim: 100]
Published: 3 days ago;
Date: Wed Jul 22 14:31:11 2026 UTC
Amazon cut jobs in its artificial general intelligence group...(Reuters)
The result exposed both:
- A relative publication display
- An absolute date/time string
Lossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "news",
"title": "Amazon cuts jobs in its artificial general intelligence group",
"url": "https://www.reuters.com/business/world-at-work/amazon-cuts-jobs-its-artificial-general-intelligence-group-2026-07-22/",
"word_limit": 100,
"published_display": "3 days ago",
"date_display": "Wed Jul 22 14:31:11 2026 UTC",
"text": "Amazon cut jobs in its artificial general intelligence group..."
}Important observation
The initial news card did not give me a normalized author field. After opening and searching the article, the extracted page exposed:
By Greg Bensinger
July 22, 2026 2:31 PM UTC
Updated July 22, 2026It also exposed:
- The photo caption
- Photographer credit
- Dateline
- Reporting and editing credits
- Author-profile link
- Suggested topics
- Related stories
(Reuters)
That demonstrates the difference between the card and the opened page:
{
"card": {
"author": null,
"published": "relative and absolute date strings"
},
"opened_page": {
"visible_byline": "Greg Bensinger",
"visible_published_at": "July 22, 2026 2:31 PM UTC",
"visible_updated_at": "July 22, 2026",
"photo_credit": "REUTERS/Stephane Mahe",
"reporting_credit": "Greg Bensinger",
"editing_credits": [
"Chizu Nomiyama",
"Kirsten Donovan"
]
}
}Again, those opened-page fields were extracted from line-addressable text, not handed to me as that JSON object.
4. Academic result
The same broad search produced a specialized academic result.
Actual model-visible format
Artificial Intelligence Index Report 2024
(https://arxiv.org/abs/2405.19522)
<REF_ID> [wordlim: 200]
Published: 2.2 years ago;
Title: Artificial Intelligence Index Report 2024
Authors: Nestor Maslej, Loredana Fattorini, Raymond Perrault, ...
Date: Wed May 29 20:59:57 2024
The 2024 Index tracks and analyzes developments in artificial intelligence...(arXiv)
This is richer than a general web result. It concretely exposes:
- A recognized academic-result classification
- Title
- Authors
- Exact submission/publication date
- Abstract or abstract-like content
- Relative age
- URL
- Word limit
Lossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "academia",
"url": "https://arxiv.org/abs/2405.19522",
"word_limit": 200,
"published_display": "2.2 years ago",
"paper": {
"title": "Artificial Intelligence Index Report 2024",
"authors": [
"Nestor Maslej",
"Loredana Fattorini",
"Raymond Perrault"
],
"authors_truncated_for_display": true,
"date_display": "Wed May 29 20:59:57 2024",
"abstract_text": "The 2024 Index tracks and analyzes developments in artificial intelligence..."
}
}The actual result had the complete author list; I shortened it above for readability.
Not observed
{
"doi": null,
"journal": null,
"volume": null,
"issue": null,
"pages": null,
"citation_count": null,
"references": null,
"subjects": null,
"license": null,
"pdf_url_as_separate_field": null,
"arxiv_categories": null
}Some of those may exist on arXiv or in an upstream academic index, but they were not exposed in this result.
5. Reddit result
A general search can return a specialized Reddit representation.
Actual model-visible format
Weekly AI Timeline Estimates For RSI, AGI, ASI, LEV, UBI and Home Multipurpose Robots
(https://www.reddit.com/r/accelerate/comments/...)
<REF_ID> [wordlim: 200]
Published: 4 days ago;
[Wednesday July 22 2026]
[+28 votes]
... post text ...(Reddit)
Lossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "reddit",
"title": "Weekly AI Timeline Estimates For RSI, AGI, ASI, LEV, UBI and Home Multipurpose Robots",
"url": "https://www.reddit.com/r/accelerate/comments/...",
"word_limit": 200,
"published_display": "4 days ago",
"date_display": "Wednesday July 22 2026",
"score_display": "+28 votes",
"text": "... post text ..."
}Fields apparently parsed or formatted
{
"subreddit": "inferable from URL",
"post_title": "visible",
"post_score": "visible as display text",
"post_date": "visible",
"post_body": "visible or partially visible"
}Fields not exposed in this result
{
"reddit_post_id": null,
"author_username": null,
"upvote_ratio": null,
"comment_count": null,
"award_count": null,
"flair": null,
"edited": null,
"permalink_as_separate_field": null,
"crosspost_parent": null
}The URL contains a Reddit post ID, but there was no separately labeled ID field.
6. Image-search result
Actual request
{
"image_query": [
{
"q": "Grand Canyon sunrise National Park Service"
}
]
}Actual model-visible format
An image result appeared as:
Grand Canyon National Park HDR - Mather Point Sunrise | Flickr
(https://www.flickr.com/photos/tombricker/13046251014)
<REF_ID>
Published: 7.8 years ago;
<RENDERED IMAGE>
Grand Canyon National Park HDR - Mather Point Sunrise
Mather Point is a popular sunrise location...
Uploaded on March 9, 2014
Taken on February 23, 2014
Tom B. By: Tom B.
24,713 views
149 faves
23 commentsThis is much richer than a conventional thumbnail-only SERP card. The model-visible result included:
- Rendered image content
- Source-page title and URL
- Relative publication age
- Source-page text
- Creator/byline text
- Upload and capture dates
- View/favorite/comment counts
- Rights text
Lossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "image",
"title": "Grand Canyon National Park HDR - Mather Point Sunrise | Flickr",
"source_page_url": "https://www.flickr.com/photos/tombricker/13046251014",
"published_display": "7.8 years ago",
"rendered_image": {
"available_to_model": true,
"raw_image_url_exposed": false,
"binary_payload_exposed": false
},
"associated_page_text": {
"image_title": "Grand Canyon National Park HDR - Mather Point Sunrise",
"creator_display": "Tom B.",
"uploaded_display": "March 9, 2014",
"taken_display": "February 23, 2014",
"views": 24713,
"favorites": 149,
"comments": 23,
"rights_display": "All rights reserved"
}
}Important caveat
I cannot determine from the result whether every piece of text was derived from:
- Page HTML
- Image metadata
- Alt text
- Open Graph
- Flickr’s structured API data
- Search-index extraction
- A generated visual caption
The result does not provide field-level provenance.
Not exposed
{
"original_image_url": null,
"thumbnail_url": null,
"width": null,
"height": null,
"mime_type": null,
"file_size": null,
"exif": null,
"iptc": null,
"xmp": null,
"visual_embedding": null,
"perceptual_hash": null,
"image_quality_score": null,
"safe_search_score": null
}7. Product result
Actual request: exploratory search
{
"product_query": {
"search": [
"wireless mouse"
]
}
}Actual request: exact lookup
{
"product_query": {
"lookup": [
"Logitech M330 Silent Plus 910-004905"
]
}
}Both returned the same product family.
Actual model-visible format
Logitech M330 Silent Plus Wireless Optical USB Mouse, Black
(https://www.staples.com/.../product_2406183)
<REF_ID>
Desc:
Logitech M330 Silent Plus advanced wireless USB mouse...
Rating:
4.7/5 (1033 reviews)
Merchants:
- $22.99 (in_stock): Staples
Specs:
- Brand: Logitech
- Sale Price: $18.99
- Color: Black
- Size: Each
- Material: Rubber
- Product Category: Computers & Accessories > ... > Computer MiceLossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "product",
"title": "Logitech M330 Silent Plus Wireless Optical USB Mouse, Black",
"url": "https://www.staples.com/logitech-m330-silent-plus-wireless-optical-usb-mouse-black-910-004905/product_2406183",
"description": "Logitech M330 Silent Plus advanced wireless USB mouse...",
"rating": {
"value": 4.7,
"scale": 5,
"review_count": 1033
},
"merchants": [
{
"name": "Staples",
"price": 22.99,
"currency": "USD",
"availability": "in_stock"
}
],
"specs": {
"Brand": "Logitech",
"Sale Price": "$18.99",
"Color": "Black",
"Size": "Each",
"Material": "Rubber",
"Product Category": "Computers & Accessories > Computer Accessories & Peripherals > Keyboards & Mice > Computer Mice"
}
}Interesting inconsistency
The result contained both:
{
"merchant_price": 22.99,
"sale_price_spec": 18.99
}There is no exposed provenance or explanation for the difference. That means downstream code should not assume that all price-like fields describe the same offer.
Potentially dynamic spec keys
The Specs section is not necessarily a fixed schema. Different products returned fields such as:
- Brand
- Color
- Size
- Material
- Height
- Length
- Depth
- Weight
- Sale Price
- Product Category
A faithful representation is therefore probably closer to:
{
"specs": {
"<provider-defined-label>": "<provider-defined-value>"
}
}rather than a fixed universal product schema.
8. Finance result
Actual request
{
"finance": [
{
"ticker": "AMD",
"type": "equity",
"market": "USA"
}
],
"response_length": "short"
}Exact model-visible response structure
The response was a single structured prose paragraph:
<REF_ID> Advanced Micro Devices Inc. (AMD) is a equity in the USA market.
The price is 521.95 USD currently with a change of -17.66000 (-0.03272%)
from the previous close. The intraday high is 551.01 USD and the intraday
low is 518.5 USD. The latest open price was 547.0 USD and the intraday
volume is 27200730. The market cap is 861,217,500,000. The PE ratio is
171.13114754098362. The EPS ratio is 3.05. The latest trade time is
Friday, July 24, 23:59:18 UTC.This is important: I did not receive a visible JSON object. The result was already verbalized into a sentence.
Lossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "finance",
"instrument": {
"name": "Advanced Micro Devices Inc.",
"ticker": "AMD",
"type": "equity",
"market": "USA"
},
"quote": {
"price": 521.95,
"currency": "USD",
"change": -17.66,
"change_percent_raw": -0.03272,
"intraday_high": 551.01,
"intraday_low": 518.5,
"open": 547.0,
"volume": 27200730,
"latest_trade_time_display": "Friday, July 24, 23:59:18 UTC"
},
"fundamentals": {
"market_cap": 861217500000,
"pe_ratio": 171.13114754098362,
"eps": 3.05
}
}Percentage ambiguity
The response said:
-17.66000 (-0.03272%)Mathematically, a change of -17.66 from approximately 539.61 is about:
{
"fractional_change": -0.03272,
"percentage_change": -3.272
}The rendered output attached a % sign to -0.03272. This suggests either:
- The upstream value was a decimal fraction and the renderer added
%without multiplying by 100, or - The value’s intended semantics differ from the conventional interpretation.
That is a concrete example of why seeing only rendered prose can lose type semantics.
Not visible
{
"exchange": null,
"market_status": null,
"bid": null,
"ask": null,
"quote_delay": null,
"provider": null,
"historical_series": null,
"premarket": null,
"after_hours": null,
"isin": null,
"figi": null
}A chart widget may receive historical points through another channel, but that time-series object was not shown to me in the text result.
9. Weather result
Actual request
{
"weather": [
{
"location": "Phoenix, Arizona",
"duration": 3
}
],
"response_length": "long"
}Actual model-visible format
The beginning of the result was:
<REF_ID> Weather for Phoenix, AZ, United States:
Current Conditions:
Sunny, 93°F (34°C)
Daily Forecast:
Sunday, July 26:
Partly sunny and toasty conditions; a thunderstorm in the area late this
afternoon; heat will be dangerous, minimize outdoor activity,
High: 112°F (44°C), Low: 93°F (34°C)
Monday, July 27:
Partly sunny and toasty conditions; a thunderstorm in spots late in the
afternoon; heat will be dangerous, minimize outdoor activity,
High: 112°F (44°C), Low: 94°F (34°C)
Tuesday, July 28:
Hot with intervals of clouds and sun; heat will be dangerous,
High: 111°F (44°C), Low: 91°F (33°C)
Severe weather alerts:
[SevereWeather(
start_timestamp=1784826000,
end_timestamp=1785207600,
utc_offset_sec=-25200.0,
name='Buckeye/Avondale',
summary='Extreme Heat Warning in effect until Monday, 8:00 PM MST...',
details='...'
), ...]This response is particularly informative because it combines:
- Human-readable forecast prose
- A Python-representation-like list of structured alert objects
It is not valid JSON because it uses:
SevereWeather(...)- Single-quoted strings
- Named constructor arguments
- Python
floatrepresentations - An object-list display rather than JSON dictionaries
Lossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "forecast",
"location": {
"display": "Phoenix, AZ, United States"
},
"current_conditions": {
"condition": "Sunny",
"temperature_f": 93,
"temperature_c": 34
},
"daily_forecast": [
{
"date_display": "Sunday, July 26",
"description": "Partly sunny and toasty conditions; a thunderstorm in the area late this afternoon; heat will be dangerous, minimize outdoor activity",
"high_f": 112,
"high_c": 44,
"low_f": 93,
"low_c": 34
},
{
"date_display": "Monday, July 27",
"description": "Partly sunny and toasty conditions; a thunderstorm in spots late in the afternoon; heat will be dangerous, minimize outdoor activity",
"high_f": 112,
"high_c": 44,
"low_f": 94,
"low_c": 34
},
{
"date_display": "Tuesday, July 28",
"description": "Hot with intervals of clouds and sun; heat will be dangerous, minimize outdoor activity",
"high_f": 111,
"high_c": 44,
"low_f": 91,
"low_c": 33
}
],
"severe_weather_alerts": [
{
"start_timestamp": 1784826000,
"end_timestamp": 1785207600,
"utc_offset_sec": -25200.0,
"name": "Buckeye/Avondale",
"summary": "Extreme Heat Warning in effect until Monday, 8:00 PM MST. Source: U.S. National Weather Service",
"details": "<FULL ALERT TEXT>"
}
]
}Alert fields concretely observed
{
"start_timestamp": "integer",
"end_timestamp": "integer",
"utc_offset_sec": "floating-point number",
"name": "string",
"summary": "string",
"details": "string"
}This is the clearest example from the tests of an underlying typed object leaking through into the model-visible representation.
10. Sports schedule
Actual request
{
"sports": [
{
"tool": "sports",
"fn": "schedule",
"league": "mlb",
"team": "NYY",
"num_games": 5
}
],
"response_length": "medium"
}Exact model-visible format
<REF_ID>
Yankees @ Phillies on Sunday, Jul 26, 2026 at 04:20 PM PDT
Yankees @ White Sox on Monday, Jul 27, 2026 at 04:40 PM PDT
Yankees @ White Sox on Tuesday, Jul 28, 2026 at 04:40 PM PDT
Yankees @ White Sox on Wednesday, Jul 29, 2026 at 04:40 PM PDT
Yankees @ White Sox on Thursday, Jul 30, 2026 at 11:10 AM PDTNo event objects were shown. The result was a line-oriented schedule.
Lossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "sports",
"result_subtype": "schedule",
"events": [
{
"away_team_display": "Yankees",
"home_team_display": "Phillies",
"date_display": "Sunday, Jul 26, 2026",
"time_display": "04:20 PM",
"timezone_display": "PDT"
},
{
"away_team_display": "Yankees",
"home_team_display": "White Sox",
"date_display": "Monday, Jul 27, 2026",
"time_display": "04:40 PM",
"timezone_display": "PDT"
}
]
}Not exposed
{
"event_id": null,
"league_id": null,
"home_team_id": null,
"away_team_id": null,
"start_timestamp": null,
"venue": null,
"broadcasts": null,
"probable_pitchers": null,
"ticket_url": null,
"event_status": null,
"provider": null
}The UI schedule widget may have more information, but the model-visible text did not.
11. Sports standings
Actual request
{
"sports": [
{
"tool": "sports",
"fn": "standings",
"league": "mlb"
}
],
"response_length": "long"
}Actual model-visible format
<REF_ID>
# Conference: American League
## Division: East
Tampa Bay Rays 61-43
New York Yankees 59-45
Boston Red Sox 53-50
Baltimore Orioles 51-54
Toronto Blue Jays 48-57
## Division: Central
Chicago White Sox 54-49
Cleveland Guardians 54-52
...Lossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "sports",
"result_subtype": "standings",
"conferences": [
{
"name": "American League",
"divisions": [
{
"name": "East",
"teams": [
{
"name": "Tampa Bay Rays",
"wins": 61,
"losses": 43
},
{
"name": "New York Yankees",
"wins": 59,
"losses": 45
}
]
}
]
}
]
}Values that are derived rather than directly returned
From ordering and records, I could derive:
{
"rank": 1,
"games_played": 104,
"win_percentage": 0.5865384615
}But none of those were explicitly displayed. A rigorous extraction should preserve provenance:
{
"rank": {
"value": 1,
"provenance": "derived_from_display_order"
},
"win_percentage": {
"value": 0.5865384615,
"provenance": "calculated_from_wins_and_losses"
}
}12. Calculator result
Actual request
{
"calculator": [
{
"expression": "275000000*0.4",
"prefix": "",
"suffix": " tokens"
}
]
}Exact model-visible format
<REF_ID> 275000000*0.4 = 110000000.0 tokensLossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "calculator",
"expression": "275000000*0.4",
"result": 110000000.0,
"prefix": "",
"suffix": " tokens",
"rendered": "275000000*0.4 = 110000000.0 tokens"
}The numeric type was rendered with a decimal, even though the mathematical result is an integer quantity.
Not exposed:
{
"parser_ast": null,
"numeric_precision": null,
"calculation_engine": null,
"rounding_mode": null
}13. Time result
Actual request
{
"time": [
{
"utc_offset": "-07:00"
}
]
}Exact model-visible format
<REF_ID> The time in UTC-07:00 is Jul 26, 2026, 7:24:10 AMLossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "time",
"utc_offset": "-07:00",
"date_display": "Jul 26, 2026",
"time_display": "7:24:10 AM",
"rendered": "The time in UTC-07:00 is Jul 26, 2026, 7:24:10 AM"
}Not exposed:
{
"unix_timestamp": null,
"iana_timezone": null,
"dst_status": null,
"locale": null,
"source_clock": null
}Because the request specifies only an offset, the result cannot inherently distinguish Phoenix from Los Angeles or another location sharing that offset at the moment.
14. Opened HTML page
Actual request
{
"open": [
{
"ref_id": "<SEARCH_RESULT_REF_ID>"
}
],
"response_length": "long"
}Actual model-visible header
Create a model response | OpenAI API Reference
(https://platform.openai.com/docs/api-reference/responses-streaming/response/file_search_call/completed?lang=javascript)
<REF_ID> [wordlim: 200]
Content type: text/html;
Source: open({"ref_id":"<SEARCH_RESULT_REF_ID>","lineno":null});
Redirected to URL: https://developers.openai.com/api/reference/resources/responses/methods/create;
Total lines: 9628It was followed by line-addressed content:
L1529: ResponseFunctionWebSearch = object { id, action, status, type }
L1530:
L1531: The results of a web search tool call.
L1532:
L1533: id: string
...Lossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "view",
"title": "Create a model response | OpenAI API Reference",
"url": "https://platform.openai.com/docs/api-reference/responses-streaming/response/file_search_call/completed?lang=javascript",
"word_limit": 200,
"content_type": "text/html",
"source_operation": {
"type": "open",
"ref_id": "<SEARCH_RESULT_REF_ID>",
"lineno": null
},
"redirected_url": "https://developers.openai.com/api/reference/resources/responses/methods/create",
"total_lines": 9628,
"lines": [
{
"line": 1529,
"text": "ResponseFunctionWebSearch = object { id, action, status, type }"
},
{
"line": 1531,
"text": "The results of a web search tool call."
}
]
}Rich features exposed only after opening
- Content type
- Redirect destination
- Total extracted line count
- Stable line addresses
- Link IDs embedded in lines
- Headings
- Some page structure
- Source operation provenance
Still not exposed:
- Raw HTML
- DOM nodes
- CSS
- JavaScript
- HTTP headers
- Cookies
- Network requests
- Raw JSON-LD
- Browser console
- Redirect chain before the final URL
15. Find-in-page result
Actual request
{
"find": [
{
"ref_id": "<OPENED_PAGE_REF_ID>",
"pattern": "ResponseFunctionWebSearch"
}
]
}Actual model-visible header
Create a model response | OpenAI API Reference
(https://platform.openai.com/docs/api-reference/responses-streaming/response/file_search_call/completed?lang=javascript)
<REF_ID> [wordlim: 200]
Content type: text/html;
Source: find({
"ref_id":"<OPENED_PAGE_REF_ID>",
"pattern":"ResponseFunctionWebSearch"
});
Redirected to URL: https://developers.openai.com/api/reference/resources/responses/methods/create;
Total lines: 9628The returned content was centered around occurrences of the requested pattern. (OpenAI Platform)
Lossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "view",
"view_subtype": "find_in_page",
"title": "Create a model response | OpenAI API Reference",
"url": "https://platform.openai.com/docs/api-reference/responses-streaming/response/file_search_call/completed?lang=javascript",
"content_type": "text/html",
"source_operation": {
"type": "find",
"ref_id": "<OPENED_PAGE_REF_ID>",
"pattern": "ResponseFunctionWebSearch"
},
"redirected_url": "https://developers.openai.com/api/reference/resources/responses/methods/create",
"total_lines": 9628,
"matching_regions": [
{
"start_line": 1529,
"end_line": 1615,
"lines": [
{
"line": 1529,
"text": "ResponseFunctionWebSearch = object { id, action, status, type }"
}
]
}
]
}The actual output did not wrap matches into matching_regions; that is my natural JSON representation of the returned passages.
16. Clicked-page result
Actual request
{
"click": [
{
"ref_id": "<OPENED_REUTERS_ARTICLE_REF_ID>",
"id": 111
}
],
"response_length": "short"
}The link ID 111 corresponded to the author link in the opened article.
Actual model-visible header
Greg Bensinger | Reuters
(https://www.reuters.com/authors/greg-bensinger/)
<REF_ID> [wordlim: 100]
Content type: text/html;
Source: click({
"ref_id":"<OPENED_REUTERS_ARTICLE_REF_ID>",
"id":111
});
Total lines: 376It was followed by line-addressed page content, including:
L151: Image
L152: # Greg Bensinger
L154: Thomson Reuters
L157: Greg Bensinger joined Reuters as a technology correspondent...
L161: 158 results(Reuters)
Lossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "view",
"view_subtype": "clicked_page",
"title": "Greg Bensinger | Reuters",
"url": "https://www.reuters.com/authors/greg-bensinger/",
"word_limit": 100,
"content_type": "text/html",
"source_operation": {
"type": "click",
"parent_ref_id": "<OPENED_REUTERS_ARTICLE_REF_ID>",
"link_id": 111
},
"total_lines": 376,
"lines": [
{
"line": 152,
"text": "# Greg Bensinger"
},
{
"line": 154,
"text": "Thomson Reuters"
},
{
"line": 161,
"text": "158 results"
}
]
}The link ID is page-extraction-local. It is not necessarily an HTML element ID, database ID or stable URL identifier.
17. Opened PDF
Actual request
{
"open": [
{
"ref_id": "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf"
}
],
"response_length": "short"
}Exact model-visible format
(https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf)
<REF_ID> [wordlim: 200]
Content type: application/pdf;
Number of pages: 1;
Source: open({
"ref_id":"https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
"lineno":null
});
Total lines: 1
L0@P0: Dummy PDF fileLossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "view",
"view_subtype": "pdf",
"url": "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
"word_limit": 200,
"content_type": "application/pdf",
"number_of_pages": 1,
"source_operation": {
"type": "open",
"ref_id": "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
"lineno": null
},
"total_lines": 1,
"lines": [
{
"line": 0,
"page_index": 0,
"text": "Dummy PDF file"
}
]
}The line notation contains both dimensions:
L0@P0
│ └── PDF page index 0
└───── Extracted text line 0Not exposed
{
"pdf_version": null,
"document_author": null,
"document_title_metadata": null,
"creator_application": null,
"creation_date": null,
"modification_date": null,
"xmp": null,
"embedded_files": null,
"fonts": null,
"page_dimensions": null,
"encryption": null,
"digital_signatures": null
}18. PDF screenshot
Actual request
{
"screenshot": [
{
"ref_id": "<OPENED_PDF_REF_ID>",
"pageno": 0
}
],
"response_length": "short"
}Model-visible result
The result contained:
(https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf)
<REF_ID>
<RENDERED PAGE IMAGE>The rendered image showed the first PDF page.
Lossless normalized representation
{
"ref_id": "<REF_ID>",
"reference_family": "view",
"view_subtype": "pdf_screenshot",
"source_pdf_url": "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
"page_index": 0,
"rendered_image": {
"available_to_model": true,
"raw_image_url_exposed": false,
"binary_payload_exposed": false,
"width": null,
"height": null,
"mime_type": null
}
}The screenshot is genuinely provided to my visual input. I am not shown its binary encoding or filesystem path.
19. Failed fetch or click
One attempted PDF click produced a concrete error representation.
Actual model-visible format
Internal Error ()
<REF_ID> [wordlim: 200]
Source: click({
"ref_id":"<OPENED_PAGE_REF_ID>",
"id":45
});
Total lines: 1
L0: Failed to fetch https://hai.stanford.edu/assets/files/hai_ai-index-report-2024-smaller2.pdf:
(400) Content length is too large: 24167379Lossless normalized JSON
{
"ref_id": "<REF_ID>",
"reference_family": "view",
"view_subtype": "error",
"title": "Internal Error",
"source_operation": {
"type": "click",
"parent_ref_id": "<OPENED_PAGE_REF_ID>",
"link_id": 45
},
"total_lines": 1,
"error": {
"operation": "fetch",
"url": "https://hai.stanford.edu/assets/files/hai_ai-index-report-2024-smaller2.pdf",
"status_or_internal_code": 400,
"message": "Content length is too large",
"reported_content_length": 24167379
}
}Whether 24167379 represents exact bytes is not explicitly labeled in the error text, although that is the natural interpretation.
A malformed sports request produced a different tool-level error:
Found no tool response. This likely means the arguments you provided were not valid.That error did not produce a source reference.
20. Rich presentation objects
These are different from the underlying result references.
Examples
- Finance price chart
- Weather forecast widget
- Sports schedule widget
- Sports standings table
- Product carousel
- Image carousel
- News-source link group
A presentation instruction might reference a result:
finance chart → finance result reference
weather widget → forecast result reference
schedule widget → sports schedule referenceBut I do not receive a visible object like:
{
"widget": {
"chart_type": "line",
"x_axis": {},
"series": [],
"interaction_config": {}
}
}The client application may receive something of that kind, but it is not placed in my inspectable context.
That means I can determine:
{
"finance_result_can_render_chart": true,
"weather_result_can_render_forecast_widget": true
}But I cannot inspect:
{
"chart_points": "unknown",
"widget_configuration": "unknown",
"client_component_props": "unknown"
}21. Map and local results
I do not have a dedicated map/local-business function in the tool interface used for these calls.
A conventional web query can find local pages, and some ChatGPT user interfaces can render maps or local entity results. However, I cannot run a dedicated map-result operation here and inspect its response.
Therefore, I cannot honestly provide an observed local-result JSON shape from this session.
The status is:
{
"result_family": "map_or_local",
"invokable_in_current_tool_interface": false,
"raw_schema_visible": false,
"known_to_exist_in_some_product_interfaces": true
}22. What “the exact representation I receive” really means
The closest representation of the system is:
JSON request arguments
↓
Tool execution
↓
Provider or search-system data
↓
Unknown internal transformation
↓
Model-visible rendered resultFor example, finance may begin upstream as something resembling:
{
"symbol": "AMD",
"regularMarketPrice": 521.95,
"regularMarketVolume": 27200730
}But what I actually see is:
Advanced Micro Devices Inc. (AMD) is a equity...
The price is 521.95 USD...I cannot know whether the speculative upstream JSON above is remotely accurate, because the transformation occurs before the result enters my visible context.
Weather is slightly different because a typed object representation survived:
SevereWeather(
start_timestamp=...,
end_timestamp=...,
utc_offset_sec=...,
name=...,
summary=...,
details=...
)That is stronger evidence of an underlying object’s field names, although it is still not proof of the original provider’s JSON schema. It may be an intermediate OpenAI-side data class.
Most defensible conclusions
Directly established
- Search requests are passed as structured JSON arguments.
- The responses I receive are generally rendered representations, not raw JSON.
- Result references have distinct families such as
search,news,academia,reddit,image,product,finance,forecast,sports,calculator,timeandview. - Opened pages expose content type, operation provenance, line counts, line addresses and link identifiers.
- PDFs add MIME type, page count and page-indexed text.
- Weather alerts expose identifiable constructor-style field names.
- Finance, sports, calculator and time results are rendered primarily as prose or line-oriented text.
- Product results expose labeled subsections and dynamic specification fields.
- Image results can include an actual visual input plus extensive source-page text.
Not established
- Exact internal backend JSON
- Exact upstream provider schemas
- Search ranking scores
- PageRank or authority values
- Client widget props
- Raw image or PDF byte representations
- Raw structured webpage metadata
- Whether every visible metadata value comes from HTML, JSON-LD, an API, a search index or generated extraction
- Whether field names in a normalized JSON object match internal field names
The accurate final formulation is:
The examples above are based on actual calls and actual model-visible results from this conversation. The textual formats are concrete. The JSON versions are lossless normalizations I created from those concrete results; they are not copies of a hidden backend serialization.