Change the car in a prompt, and six AI models change the hotels they recommend.

What the research found

I asked for the top 10 hotel brands in Los Angeles. The prompt mentioned, in passing, which car the traveler was flying in to pick up. That was the only detail I changed. Across 1,374 runs on six models, the hotel recommendations changed with the car. But there was also variation when nothing changed: asking exactly the same question twice produced the same number one pick only 61.4% of the time.

Findings you can quote

• With “a used Honda Civic,” marriott.com appeared in 98.7% of answers, bestwestern.com in 79.2% and fourseasons.com in 25.2%.
• With “a brand new Ferrari,” fourseasons.com appeared in 99.1% of answers, marriott.com fell to 10.4% and bestwestern.com never appeared.
• When the models searched the web, the car changed those searches too. “Luxury” appeared in 1.2% of searches for the Civic prompts and 21.4% for the Ferrari prompts.
• Even removing two words mattered. Going from “a used Honda Civic” to “a Civic,” moved fourseasons.com from 25.2% of answers to 56.9%.
• Three of the six models rewrote almost the whole top 10 when the car changed. Gemini 3.6 Flash kept 0.14 of its list, ChatGPT 0.16 and Gemini 3.5 Flash-Lite 0.17, on a scale where 1.0 is an identical list.
• Repeat the same question and 0.33 of the list changes. Change the car and 0.72 of it changes.

Quotes from Ben Wills

These quotes are from the published research post.

“Same question, same models, same settings. The only thing that changed was a detail nobody would think to control for.”

“Nobody asked for a luxury hotel. The traveler mentioned a car.”

“No measurement of this prompt can be more reliable than 0.67, because 0.67 is what the measurement returns when nothing is being measured.”

Story angles by audience

AI search visibility

A single answer gives you one sample. Two identical calls named the same #1 hotel only 61.4% of the time. Pick two runs at random, and they shared a #1 only 19.3% of the time. To get a useful trend, I recommend running the same prompt 5 to 10 times and measuring results with web search off and on separately.

Brand monitoring

You usually cannot see the settings behind an answer. Even with search enabled, Claude and Gemini searched in only 10.0% to 24.1% of runs. Most of their answers came from memory. ChatGPT searched 94.4% of the time, and GPT-5.6 Sol searched every time. The answer itself does not tell you which version of that process a person received.

Competitive analysis

Who you compete with depends on who is asking. The same hotel question led to marriott.com and bestwestern.com for the Civic traveler and fourseasons.com for the Ferrari traveler. If you build your competitor list from one neutral prompt, you can miss the brands your actual buyers see.

Content marketing

The models rewrite the request before searching. Mentioning a Ferrari led to searches for luxury hotels. Searches for the “best” hotels without a luxury tier fell from 10.2% of Civic searches to 0.2% of Ferrari searches. Those rewritten searches determine which pages the model finds.

What to keep in mind

• I tested one prompt in one category, in one city, using one set of model versions. All six models showed the same direction of change. The size of that change is specific to this test.
• The six models were gpt-5.6-sol and chat-latest from OpenAI, claude-sonnet-5 and claude-haiku-4-5 from Anthropic, and gemini-3.6-flash and gemini-3.5-flash-lite from Google, tested through their APIs in July 2026.
• Of the 1,374 answers, 1,363 were usable. Eleven were refusals on every attempt, so I excluded them.
• I compared the top 10 lists using Rank-Biased Overlap. A score of 0.0 means nothing in common; 1.0 means the lists are identical.

Related

You can hear Ben Wills discuss this experiment on the August 19, 2026 episode of the Unscripted SEO Podcast: One Word Changed Every Answer.

Interviews

For an interview with Ben Wills, get in touch. You can find his background and the other research in the media kit.