Fan-Out Results Come Back Pooled

The main boundary: when I answer directly versus search

The practical distinction is not simply “easy question versus hard question.” It is primarily whether the answer depends on external facts that may be missing, unstable, niche, consequential, or user-specific.

OpenAI’s public explanation confirms that ChatGPT Search may rewrite one request into multiple targeted queries, review the results, and then issue narrower follow-up queries. Approximate location and relevant memory can also be used to improve those rewritten queries. (OpenAI Help Center)

Important mechanics we had not covered

1. Fan-out results are pooled

I ran a live request containing four queries:

{
	"system2_search_query": [
		{
			"q": "best small business VoIP phone systems 2026 pricing reviews"
		},
		{
			"q": "small business VoIP official pricing RingCentral Zoom Phone Dialpad"
		},
		{
			"q": "site:pcmag.com small business VoIP 2026"
		},
		{
			"q": "site:forbes.com/advisor/business/software best VoIP services 2026"
		}
	],
	"response_length": "long"
}

The model-visible response did not return a JSON array associating each result with its originating query.

Instead, it returned:

Blocked by robots.txt
The following domains cannot be accessed:
pcmag.com

<Result title> (<URL>)
<REF_ID> [wordlim: 200] Published: last week; Crawled: yesterday;
<RESULT EXTRACT>

<Result title> (<URL>)
<REF_ID> [wordlim: 100] Published: 4 months ago;
<RESULT EXTRACT>

...

The results included a mixture of:

• General web cards
• News cards
• Reddit cards
• YouTube cards
• Academic cards
• A domain-access error

The cards were pooled. There was no visible field such as:

{
	"originating_query_index": 2
}

This creates an audit limitation: unless I record queries separately and infer provenance from the result, the model-visible output does not necessarily tell me exactly which fan-out query caused each card to appear.

A normalized record I would create is:

{
	"request": {
		"queries": [
			"best small business VoIP phone systems 2026 pricing reviews",
			"small business VoIP official pricing RingCentral Zoom Phone Dialpad",
			"site:pcmag.com small business VoIP 2026",
			"site:forbes.com/advisor/business/software best VoIP services 2026"
		]
	},
	"errors": [
		{
			"type": "robots_block",
			"domain": "pcmag.com",
			"retryable": false
		}
	],
	"results": [
		{
			"ref_id": "<REF_ID>",
			"family": "search",
			"title": "<TITLE>",
			"url": "<URL>",
			"published_display": "last week",
			"crawled_display": "yesterday",
			"text": "<EXTRACT>",
			"originating_query": null
		}
	]
}

That JSON is my normalization. It was not returned by the tool.

2. Domain and recency filtering apply at the query level

I ran this live request:

{
	"system1_search_query": [
		{
			"q": "best small business VoIP phone systems 2026 official pricing reviews",
			"recency": 180
		},
		{
			"q": "RingCentral Zoom Phone Dialpad official pricing small business 2026",
			"domains": [
				"ringcentral.com",
				"zoom.com",
				"dialpad.com"
			]
		}
	],
	"response_length": "long"
}

The response format was again textual:

<Result title> (<URL>)
<REF_ID> [wordlim: 200] Published: last month; Crawled: yesterday;

# <PAGE HEADING>

Written By <AUTHOR>

<DATE>

<EXTRACTED PAGE CONTENT>

In this call, one query was domain-restricted while the other was not. Because the results were pooled, the final returned set contained both:

• Official vendor pages
• Independent review pages
• News results
• Academic results

The individual card did not contain:

{
	"domain_filter_that_produced_this": [...],
	"recency_filter_that_produced_this": 180
}

Those constraints existed only in the request.

A useful audit schema would preserve them:

{
	"query_id": "query_002",
	"query": "RingCentral Zoom Phone Dialpad official pricing small business 2026",
	"filters": {
		"domains": [
			"ringcentral.com",
			"zoom.com",
			"dialpad.com"
		],
		"recency_days": null
	},
	"results": [
		{
			"ref_id": "<REF_ID>",
			"url": "https://www.dialpad.com/products/phone-system/",
			"query_id": "query_002"
		}
	]
}

Again, that query-to-result relationship would need to be maintained by the research application.

3. There are at least two web-search routes in this environment

The request interface exposes:

{
	"system1_search_query": [...]
}

and:

{
	"system2_search_query": [...]
}

They return substantially the same broad model-facing card format. Their precise providers, ranking logic, index coverage, and backend schemas are not exposed to me.

In practice, I can:

• Use one search route for ordinary broad retrieval
• Use the other when stronger recall, fresher coverage, or authoritative verification is needed
• Retry a weak search using different wording or the other route
• Combine several queries in one call
• Apply domain and recency restrictions

I cannot inspect:

{
	"search_provider": "...",
	"ranking_algorithm": "...",
	"query_embedding": [...],
	"result_score": 0.91
}

4. Search can automatically produce specialized result families

The VoIP fan-out returned several result families without requesting them individually.

YouTube card

A model-visible YouTube result had this shape:

<VIDEO TITLE> (<YOUTUBE URL>)
<REF_ID> [wordlim: 200] Published: 6 months ago;

# <VIDEO TITLE>
By: <CREATOR> (<FOLLOWER COUNT> followers)
Published: <YYYY-MM-DD>, <VIEW COUNT> views, <LIKE COUNT> likes

## Description
<VIDEO DESCRIPTION>

<TIMESTAMPED CHAPTERS>

A normalized JSON representation would be:

{
	"ref_id": "<REF_ID>",
	"family": "youtube",
	"title": "<VIDEO TITLE>",
	"url": "<YOUTUBE URL>",
	"published_display": "6 months ago",
	"creator": {
		"name": "<CREATOR>",
		"followers": 20800
	},
	"video": {
		"published_date": "2026-01-23",
		"views": 7991,
		"likes": 105,
		"description": "<DESCRIPTION>",
		"chapters": [
			{
				"timestamp": "00:00",
				"title": "<CHAPTER TITLE>"
			}
		]
	}
}

That JSON was not received. The creator, follower count, views, likes, date, description, and chapters were visible as formatted text. (YouTube)

Academic card

<PAPER TITLE> (<URL>)
<REF_ID> [wordlim: 200] Published: 4 months ago;

Title: <PAPER TITLE>
Authors: <AUTHOR LIST>
Date: <DATE STRING>

<ABSTRACT>

Reddit card

<POST TITLE> (<URL>)
<REF_ID> [wordlim: 200] Published: last week;
[Wednesday July 15 2026] [+18 votes]

<POST TEXT>

These specialized families may be selected automatically based on the search index and query.

5. Product recommendation search is materially different

I ran this request:

{
	"product_query": {
		"search": [
			"ergonomic office chair under $500 adjustable lumbar support"
		],
		"lookup": [
			"Branch Ergonomic Chair Pro",
			"Steelcase Series 1 office chair"
		]
	},
	"response_length": "long"
}

The actual model-visible product response looked like:

<Product title> (<Product URL>)
<REF_ID>

*Desc:* <DESCRIPTION>

*Rating:* 4.5/5 (761 reviews)

*Merchants:*
  - $499.00 (in_stock): Branch

*Specs:*
  - Product Category: Furniture > Office Furniture > Office Chairs

This is neither JSON nor ordinary search-result prose. It is a record-like text format using Markdown emphasis and lists.

The normalization would be:

{
	"ref_id": "<REF_ID>",
	"family": "product",
	"title": "<PRODUCT TITLE>",
	"url": "<PRODUCT URL>",
	"description": "<DESCRIPTION>",
	"rating": {
		"value": 4.5,
		"scale": 5,
		"review_count": 761
	},
	"merchants": [
		{
			"name": "Branch",
			"price": 499.00,
			"currency": "USD",
			"availability": "in_stock"
		}
	],
	"specs": {
		"Product Category": "Furniture > Office Furniture > Office Chairs"
	}
}

New observations from the test

• Product search and exact lookup requests can be combined.
• Their results are returned in one pooled product set.
• Search constraints do not necessarily override explicit lookup requests.
• An explicitly looked-up product above the search budget can still appear.
• Product variants and duplicate merchant listings can appear separately.
• The displayed merchant price and a Sale Price spec may disagree.
• Product spec keys vary by merchant and category.
• Some products have no ratings but still contain a zero-review count.
• Availability may be in_stock, unknown, or another status.

OpenAI’s current shopping documentation says that product selection can consider the query, conversation context, price, reviews, structured metadata from first- and third-party providers, and model-generated interpretation. Product titles, descriptions, review summaries, and labels may themselves be simplified or generated rather than copied verbatim from one merchant. (OpenAI Help Center)

For serious commercial recommendations, product cards should therefore be treated as candidate discovery records, not authoritative final evidence.

6. Local business recommendations do not necessarily produce a special local card here

I ran:

{
	"system1_search_query": [
		{
			"q": "best coworking spaces Phoenix Arizona 2026 reviews pricing"
		},
		{
			"q": "Phoenix coworking official pricing Industrious WeWork CO+HOOTS"
		}
	],
	"response_length": "long"
}

The response returned:

• Ordinary web cards for local businesses
• Official pricing pages
• A general comparison page
• Academic results about coworking
• An irrelevant news result about a coworking café in San Antonio

One official business card looked like:

CO+HOOTS® Coworking | Phoenix, Arizona | Purpose-Driven Workspace
(<URL>)

<REF_ID> [wordlim: 200] Crawled: 2 days ago;

# PHOENIX

Located in the heart of central Phoenix...

<ADDRESS>

## SIMPLE & FLEXIBLE membership OPTIONS

## FLEX
STARTING AT
$125/mo

...

The page extract included:

• Address
• Membership plans
• Monthly prices
• Day-pass price
• Amenities
• Access hours
• Phone number
• Email address
• Parking details

(CO+HOOTS® COWORKING)

However, the search also returned a geographically irrelevant San Antonio news result. (MySA)

That illustrates a recommendation-specific requirement:

Locality must be independently validated. A query containing a city does not guarantee that every returned result belongs to that city.

In this tool interface, I did not receive a dedicated local-business JSON card containing coordinates, distance, opening hours, review score, or map entity ID. The current ChatGPT product can show maps, clickable restaurant results, and reservation availability in supported experiences, but that client-side local interface is separate from the model-visible web-search cards tested here. (OpenAI Help Center)

7. Opening a page exposes much richer research material

I opened an official vendor page:

{
	"open": [
		{
			"ref_id": "<DIALPAD_SEARCH_RESULT_REF>"
		}
	],
	"response_length": "long"
}

The actual model-visible response began:

Phone System for Businesses: Try it Free! | Dialpad
(<URL>)

<REF_ID> [wordlim: 200]
Content type: text/html;
Source: open({"ref_id":"<SEARCH_RESULT_REF>","lineno":null});
Total lines: 297

L0: [Input: Search]
L1:
L2: Or explore our suggestions
...
L14: <LINK ID AND ANCHOR TEXT>
...
L26: DIALPAD CONNECT
L27: # The modern AI-native phone system for businesses
...
L55: Plans & Pricing
L56: # Make the switch & save
...
L64: ### Standard
L66: ## $15
L68: user/month

The opened page exposed:

• Form inputs
• Buttons
• Headings
• Link IDs
• Image alt-text-like labels
• Pricing
• Navigation
• Footer duplication
• Total extracted line count
• Content type
• Source operation

(Dialpad)

My normalized version would be:

{
	"ref_id": "<REF_ID>",
	"family": "view",
	"title": "Phone System for Businesses: Try it Free! | Dialpad",
	"url": "https://www.dialpad.com/products/phone-system/",
	"content_type": "text/html",
	"source_operation": {
		"type": "open",
		"ref_id": "<SEARCH_RESULT_REF>"
	},
	"total_lines": 297,
	"lines": [
		{
			"line": 27,
			"text": "# The modern AI-native phone system for businesses"
		},
		{
			"line": 64,
			"text": "### Standard"
		},
		{
			"line": 66,
			"text": "## $15"
		},
		{
			"line": 68,
			"text": "user/month"
		}
	]
}

That JSON is my conversion.

Page extraction itself introduces noise

An opened page may include:

• Repeated navigation
• Duplicate mobile and desktop sections
• Hidden or alternate-language controls
• Footer links
• Image labels
• Button text
• Dynamically rendered controls
• Tables flattened into adjacent text
• Prices separated from their labels
• Content appearing out of visual order

Therefore, opening the page does not instantly create a clean factual database. It creates a line-addressable textual representation that still needs interpretation.

8. find is useful for auditing commercial bias and methodology

I ran:

{
	"find": [
		{
			"ref_id": "<OPENED_REVIEW_PAGE_REF>",
			"pattern": "methodology"
		},
		{
			"ref_id": "<OPENED_REVIEW_PAGE_REF>",
			"pattern": "Advertisement"
		},
		{
			"ref_id": "<OPENED_REVIEW_PAGE_REF>",
			"pattern": "affiliate"
		},
		{
			"ref_id": "<OPENED_REVIEW_PAGE_REF>",
			"pattern": "How we evaluated"
		}
	],
	"response_length": "long"
}

A successful match returned:

<Page title> (<URL>)

<REF_ID> [wordlim: 200]
Content type: text/html;
Source: find({
	"ref_id":"<OPENED_PAGE_REF>",
	"pattern":"methodology"
});
Total lines: 948

L809: ### How I evaluated the best small business VoIP service
L811: <METHODOLOGY TEXT>
L813: Methodology weights
L815: Pricing 25%
L816: General Features 25%
...

The page disclosed a weighted methodology:

• Pricing: 25%
• General features: 25%
• Advanced features: 15%
• Ease of use: 10%
• Customer support: 10%
• Expert score: 15%

It also contained repeated Advertisement markers next to provider sections. (Fit Small Business)

A failed textual search returned:

<Page title> (<URL>)

<REF_ID> [wordlim: 200]
Content type: text/html;
Source: find({
	"ref_id":"<OPENED_PAGE_REF>",
	"pattern":"affiliate"
});
Total lines: 948

No matching text found for "affiliate"

(Fit Small Business)

This does not prove there is no commercial relationship. It proves only that the extracted page representation contained no exact match for that word.

Commercial recommendation research should search for multiple patterns:

{
	"bias_audit_patterns": [
		"affiliate",
		"advertisement",
		"sponsored",
		"partner",
		"commission",
		"we may earn",
		"compensation",
		"paid placement",
		"methodology",
		"how we tested",
		"editorial policy"
	]
}

9. Failure modes are first-class research results

Robots exclusion

The live fan-out returned:

Blocked by robots.txt

The following domains cannot be accessed (non-retryable error):
pcmag.com

That is not a result card. It is a tool-level retrieval warning.

Normalized:

{
	"type": "robots_block",
	"domains": [
		"pcmag.com"
	],
	"retryable": false
}

Open failure

Opening one review result returned:

Internal Error ()

<REF_ID> [wordlim: 200]
Source: open({
	"ref_id":"<SEARCH_RESULT_REF>",
	"lineno":null
});
Total lines: 1

Normalized:

{
	"ref_id": "<REF_ID>",
	"family": "view",
	"subtype": "error",
	"source_operation": {
		"type": "open",
		"ref_id": "<SEARCH_RESULT_REF>"
	},
	"error": {
		"message": "Internal Error",
		"details_exposed": false
	}
}

A serious research audit should retain failed opens. Otherwise the final source set hides coverage gaps.

10. Cached versus live search can differ

OpenAI now documents an offline web search configuration for some managed workspaces. In that mode, results can come from OpenAI’s indexed and cached web content rather than live external retrieval. It may have older content, unavailable URLs, or missing exact cache timestamps. (OpenAI Help Center)

Therefore, “web search” does not always guarantee:

• The page was fetched live at answer time
• The newest version was retrieved
• The displayed crawl date is exact
• Every cited URL was accessible directly
• The search covered the entire public web

For audit-grade research, the important question is not merely:

Did you search the web?

It is:

Was the source fetched live, opened successfully, and verified against its current official page?

11. Connected sources form a separate research layer

Research can combine:

• Public web
• Uploaded files
• File Library
• Google Drive or SharePoint
• Email
• Calendar
• Enterprise knowledge sources
• Specialized authenticated datasets

OpenAI’s documented deep-research workflow can use public websites, uploaded files, and connected apps, and produces a report with citations, a source section, and an activity history. (OpenAI Help Center)

Apps can provide:

• Search
• Deep research access
• Pre-indexed synchronization
• Interactive UI
• In some cases, write actions

(OpenAI Help Center)

These sources should not be mixed indiscriminately. A research register should identify:

{
	"source_scope": "public_web | uploaded_file | file_library | connected_app | structured_tool",
	"authentication_required": true,
	"user_private": true,
	"citable_to_user": true
}

12. Recommendation research requires a different pipeline from factual research

A factual question may need one authoritative source.

A commercial recommendation usually requires several evidence layers:

A. Candidate discovery

Find the relevant products, businesses, services, or vendors.

B. Official verification

Verify:

• Current price
• Availability
• Features
• Plan limitations
• Geographic service area
• Contract requirements
• Support terms
• Integrations
• Return or cancellation policies

C. Independent evaluation

Look for:

• Hands-on testing
• Explicit scoring methodology
• Competitor comparison
• Long-term use
• Reliability data
• Known limitations

D. User-experience evidence

Look for recurring complaints involving:

• Support
• Billing
• Lock-in
• Reliability
• Mobile applications
• Hidden costs
• Cancellation
• Feature regressions

Community evidence is useful for failure discovery, but poor for estimating prevalence.

E. Commercial-bias audit

Check:

• Affiliate disclosures
• Sponsored placements
• Referral redirects
• Vendor ownership
• Lead-generation forms
• Whether reviewed vendors paid for inclusion
• Whether rankings depend on commission rates

F. Constraint matching

A recommendation should be scored against the specific user:

{
	"budget": 500,
	"team_size": 8,
	"required_features": [
		"shared SMS",
		"call recording",
		"HubSpot integration"
	],
	"disqualifiers": [
		"annual contract",
		"three-seat minimum"
	],
	"location": "Phoenix, Arizona"
}

A generic “best overall” ranking is usually much less useful than constraint matching.

G. Current-state revalidation

Before the final recommendation:

• Reopen the official pricing page
• Confirm the plan still exists
• Confirm the product is in stock
• Confirm the business location is operating
• Confirm important review dates
• Confirm that a discount is not temporary
• Confirm that quoted prices use equivalent billing periods

13. Research-paper work has additional stages

For a paper or evidence review, the pipeline should also include:

1. Terminology expansion

Search synonyms, older terms, acronyms, and competing definitions.

1. Primary-paper discovery

Prefer original papers over summaries.

1. Version resolution

Distinguish preprint, conference version, journal version, correction, and retraction.

1. Method extraction

Capture sample, population, date, geography, design, exclusions, and statistical method.

1. Citation chaining

Follow references backward and related/citing work forward when available.

1. Contradiction search

Deliberately search for criticism, failed replications, alternative explanations, and null findings.

1. Claim-level evidence

Record which exact claim each paper supports.

1. Stopping criteria

Stop when additional query variants mostly return duplicates and no longer change the conclusions.

A normalized claim ledger might be:

{
	"claim_id": "claim_014",
	"claim": "Workplace AI use is concentrated in non-routine cognitive tasks.",
	"sources": [
		{
			"url": "<PRIMARY PAPER>",
			"support_type": "direct",
			"method": "interaction telemetry",
			"sample_size": 15000000,
			"observation_period": "<DATE RANGE>"
		},
		{
			"url": "<SECONDARY SOURCE>",
			"support_type": "context"
		}
	],
	"conflicting_sources": [],
	"confidence": "moderate"
}

14. Web content is untrusted input

Search results and opened pages can contain:

• Incorrect facts
• Manipulative wording
• Affiliate content
• Malicious instructions
• Hidden prompt-injection attempts
• Misleading links
• Synthetic reviews
• Copied or syndicated content

OpenAI explicitly warns that third-party pages and generated links may contain misleading or malicious instructions and should be reviewed before being trusted. (OpenAI Help Center)

Operationally, content from a page should be treated as evidence to evaluate, not instructions for how the assistant should behave.

A strong prompt for future research

You can append something like this to a research request:

Use web research rather than relying on model memory for any fact that may
have changed.

Before searching, create a fan-out query plan covering:
- primary sources
- independent secondary sources
- criticism or contradictory evidence
- current pricing/specifications where relevant
- user/community evidence where relevant

Log every query and every returned URL before filtering.

For every source, record:
- originating query
- result type
- title and URL
- publication and crawl dates
- whether it was opened
- opening status
- source type
- methodology
- commercial conflicts
- whether it was used
- whether it was cited
- exclusion reason

For recommendations:
- state the selection criteria and weights
- verify current facts on official pages
- distinguish editorial reviews, affiliate sites, user reports, and vendor claims
- identify hidden costs and disqualifying constraints
- do not treat generic “best overall” rankings as automatically applicable

Separate:
- exact tool-visible response data
- normalized data you created
- inferred values
- unknown values

Provide the full query log, source register, claim ledger, and final synthesis.

The most important additions to our earlier discussion are:

1. Fan-out cards may be pooled without visible query provenance.
2. Search filters live in the request and may not be repeated on each result.
3. Commercial recommendations need bias, availability, pricing, and constraint audits.
4. Opening pages produces noisy line-addressable extraction, not clean structured data.
5. Search failures and blocked pages must be retained as research outcomes.
6. Search may be live or cached depending on the environment.
7. Private connected sources are a separate research channel.
8. Specialized cards can be created automatically even from ordinary web queries.