Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1440
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-15T10:42:59.236776Z
status: published
lens: trace
sector: health
headline: Reliability, Readability, and Safety of Generative AI Chatbots for Osteoporosis Health Counseling: A Multidimensional Cross-Sectional Comparative Study
dek: Background and aims: Generative artificial intelligence chatbots are increasingly used for health information, but their reliability, readability, and safety in osteoporosis counseling remain uncertain. This study compared four widely available chatbots under standardized conditions.
Methods: Twenty-five standardized osteoporosis questions were submitted once to ChatGPT-5, Gemini 2.5 Pro, Copilot, and Perplexity Pro, generating 100 responses. Two model-blinded specialists evaluated the final responses using DISC…
gain_title: Perplexity Pro and Copilot earned higher DISCERN information-quality medians (55.0 and 54.0) than Gemini 2.5 Pro (44.0) and ChatGPT-5 (42.0) on osteoporosis questions.
problem_title: All four chatbots wrote above the sixth-grade readability benchmark, and the safety review found clinically relevant omissions and an unsupported claim in the answers.
trace_subject: Generative AI chatbot responses to osteoporosis health counseling questions
gain_reading: Perplexity Pro and Copilot earned higher DISCERN information-quality medians (55.0 and 54.0) than Gemini 2.5 Pro (44.0) and ChatGPT-5 (42.0) on osteoporosis questions.
gain_evidence: Median scores were 55.0 for Perplexity Pro, 54.0 for Copilot, 44.0 for Gemini 2.5 Pro, and 42.0 for ChatGPT-5.
problem_reading: All four chatbots wrote above the sixth-grade readability benchmark, and the safety review found clinically relevant omissions and an unsupported claim in the answers.
problem_evidence: all models exceeded the sixth-grade benchmark | The qualitative safety assessment also identified clinically relevant omissions and an unsupported claim.
quick_read: Researchers put the same 25 standardized osteoporosis questions to ChatGPT-5, Gemini 2.5 Pro, Copilot and Perplexity Pro, producing 100 responses. Two specialists who did not know which model wrote each answer scored them with several quality and safety instruments, and automated indices measured readability.

The results suggest that the chatbots people use for health information differ in answer quality, and that the better-scoring answers were also harder for ordinary readers to understand. Safety reviewers still found omissions and an unsupported claim, so the findings do not show that any of these tools is safe to rely on for patient counseling.
limitation: The findings apply only to the tested models, prompts and evaluation period, and the authors say they do not establish a universal effect of retrieval augmentation.
tag: Dual reading
key_points: DISCERN scores differed significantly across the four models (p < 0.001; Kendall's W = 0.82). | JAMA benchmark scores did not differ significantly between models (p = 0.63). | Perplexity Pro had the highest median Flesch-Kincaid Grade Level (16.18); Copilot had the lowest (9.49). | All six readability indices differed across models, and every model exceeded the sixth-grade benchmark.
rundown: Each of the 25 questions was submitted once to each of the four chatbots. Two specialists blinded to the model scored the final responses with DISCERN, EQIP, the Global Quality Scale, JAMA benchmarks and the NOHARM safety framework.

Because every question was answered by all four models, the authors compared outcomes with Friedman tests, followed by paired Wilcoxon signed-rank tests with Holm adjustment. The study was published in 2026 in Health Science Reports.

The authors link the higher information-quality scores to retrieval-enabled systems, which also produced more difficult text. They say this pattern is specific to the tested set and does not establish a general effect of retrieval augmentation.
sources:
- peer_reviewed | Health Science Reports | https://doi.org/10.1002/hsr2.73381 | 2026-10-10
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

c35f8398aba6539e1481560298029c192175a19a259e90f9bff00faee30ede14

previous

0000000000000000000000000000000000000000000000000000000000000000