aceModerate evidence
More
ChatGPT didn't beat Booking.com for picking a hotel in a 165-person experiment
Study title: Artificial intelligence in digital tourism: evidence from an experimental study of consumer decision-making
- Evidence type
- Controlled or validated comparison
- Sample
- n = 165
- Sources
- 1 sourcePeer-reviewed
- Published
- 2 min read
- Truvace record
- TRV-2026-2662cite or check it
Study title: Artificial intelligence in digital tourism: evidence from an experimental study of consumer decision-making
A controlled experiment run in April 2026 with 165 participants from Croatia, Serbia and Bosnia and Herzegovina compared three ways of choosing accommodation: Booking.com, ChatGPT and general internet search. Booking users scored significantly higher than web-search users on pragmatic user experience (1.84 vs 1.03), booking likelihood (5.52 vs 4.80) and perceived simplicity (6.40 vs 5.28).
Read the study abstract
Introduction This study examines how different digital environments influence user experience and consumer behavior when selecting accommodation in tourism, comparing Booking, ChatGPT, and general internet search. Methods A controlled experiment was conducted with 165 participants from Croatia, Serbia, and Bosnia and Herzegovina. User experience was assessed through pragmatic and hedonic dimensions, while decision-making outcomes included booking likelihood and perceived simplicity. Results Significant differenc…
The problem
In a controlled experiment with 165 people choosing a hotel in Graz, Booking.com users rated their experience and booking likelihood significantly higher than web-search users, while ChatGPT users did not differ significantly from either group and did not surpass Booking.
In brief
ChatGPT users landed in between on all three measures (1.62, 5.15 and 5.90) and did not differ significantly from either of the other groups. The authors read this as AI being a useful complement to booking platforms rather than a replacement.
Price and decision time were similar in every group. Chosen stays averaged €203.70 with Booking, €214.42 with ChatGPT and €217.68 with web search, and decisions took roughly 211 to 223 seconds. The authors say the effects were small to moderate and that the results reflect the technology available when the experiment was run.
Main points
- Participants were split into three groups: Booking.com (67 people), ChatGPT (48) and general internet search (50). Each person used only one tool.
- The task was to pick lodging in Graz, Austria, for two nights with a budget of €300 for two people, within about 3 km of the city centre, in at most 15 minutes.
- Pragmatic user experience differed significantly across groups (F = 5.90, ω² = 0.056); only the Booking vs web-search gap was significant.
- Hedonic (enjoyment) experience did not differ significantly, although ChatGPT had the highest mean (1.22 vs 0.76 for Booking and 0.77 for web search).
- Perceived simplicity showed the largest gap (F = 12.91, ω² = 0.126): Booking 6.40, ChatGPT 5.90, web search 5.28.
- Self-rated likelihood of booking the chosen stay was 5.52 for Booking, 5.15 for ChatGPT and 4.80 for web search; only Booking vs web search was significant.
- In an exploratory analysis, older participants rated ChatGPT as more pragmatic, and the link from ease of use to booking likelihood via pragmatic quality held for women only with ChatGPT.
The full story
Participants averaged 24.72 years old (range 19 to 54); 83 were men and 82 were women. They came from Croatia (68), Serbia (55) and Bosnia and Herzegovina (40). Experimenters supervised sessions and read standardized instructions. Participants were told to imagine a short business trip, and ChatGPT users could not open external links the tool suggested.
User experience was measured with the short User Experience Questionnaire, which covers pragmatic quality (efficiency and usability) and hedonic quality (enjoyment). Group differences were tested with one-way ANOVA and post hoc comparisons. Booking beat web search on pragmatic quality, simplicity and booking likelihood, and no other pairwise comparison reached significance.
Objective outcomes barely moved. Price (F = 1.00) and decision time (F = 0.08) did not differ between tools, so the differences were in how easy and trustworthy the process felt, not in what people chose or how fast.
The authors say ChatGPT was the weaker contender partly because, unlike Booking.com, it offered no live availability, real-time prices, verified reservation data or booking function during the experiment. They suggest AI conversation features may work best inside established booking platforms.
Sources
1 on file- Frontiers in Sustainable Tourismdoi:10.3389/frsut.2026.1924194
Talk it over
Who carries this cost, and what would have to change to prevent it?
ChatGPT didn't beat Booking.com for picking a hotel in a 165-person experimenthttps://truvace.com/item/artificial-intelligence-in-digital-tourism-evidence-from-an-experimental-study-o
Who carries this cost, and what would have to change to prevent it?
The problem In a controlled experiment with 165 people choosing a hotel in Graz, Booking.com users rated their experience and booking likelihood significantly higher than web-search users, while ChatGPT users did not differ significantly from either group and did not surpass Booking.
Answer in the threadHow do we know? What can this kind of evidence show, and what can’t it?
The evidence This rests on a controlled comparison with a sample of 165, from Frontiers in Sustainable Tourism.
Answer in the threadWhat does it leave open? What else would you want to know before acting on it?
The record’s own caveat ChatGPT was compared with Booking.com without live availability, verified reservations or booking functions, and the sample was mostly young adults, so the comparison is not between equivalent services and may not generalise.
Answer in the threadHave you seen AI used this way where you study, work or live? Did it go the same way?
Answer in the thread
The questions are the same on every record. The details under them are taken from this record as written.
What readers say