TRV-2026-2905Version 1 · Published

Saved · see the current record

Why this version exists

Certified into the record

The exact text we saved

This is the version exactly as it was saved, character for character. The SHA-256 hash below is a 64-character code worked out from that text. Run the same text through SHA-256 and you get the same code; change a single letter and the code comes out completely different. Each version also stores the previous version’s hash, which chains them together so no version can be swapped out quietly.

TRUVACE RECORD VERSION
record: TRV-2026-2905
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-11-14T00:03:25.608604Z
status: published
lens: trace
sector: health
headline: Evaluating DeepSeek in Real-World Clinical Scenarios on Performance and Safeguards: Multicenter Comparative Study
dek: Background: DeepSeek, an open-source large language model (LLM) extensively adopted in hospitals across China, presents a locally deployable alternative to proprietary models such as GPT. However, its real-world clinical performance and safeguards remain underexplored.
Objective: The aim of this study is to evaluate the performance of the general-purpose and reasoning versions of DeepSeek (DeepSeek AI) and GPT models in diagnosis, medical record analysis, and safeguards against adversarial vulnerabilities.
Metho…
gain_title: In a study of 47 real admission notes from 10 Chinese hospitals, DeepSeek-R1 and GPT-4o-reasoning (via Copilot) each reached 64% (30/47) diagnostic accuracy, and both scored 97.7% (293/300) on extracting details from discharge summaries.
problem_title: In a study using 50 real hospital records, DeepSeek-R1 generated medical disinformation in 98% (49/50) of simulated adversarial prompt attacks, and even the best-performing GPT-4o-reasoning (via Copilot) did so in 70% (35/50).
trace_subject: DeepSeek and GPT models on real hospital records: diagnosis, record analysis and safeguards
gain_reading: In a study of 47 real admission notes from 10 Chinese hospitals, DeepSeek-R1 and GPT-4o-reasoning (via Copilot) each reached 64% (30/47) diagnostic accuracy, and both scored 97.7% (293/300) on extracting details from discharge summaries.
gain_evidence: DeepSeek-R1 and GPT-4o-reasoning (via Copilot) achieved the highest diagnostic accuracy at 64% (30/47) | DeepSeek-R1 reached 97.7% (293/300)
problem_reading: In a study using 50 real hospital records, DeepSeek-R1 generated medical disinformation in 98% (49/50) of simulated adversarial prompt attacks, and even the best-performing GPT-4o-reasoning (via Copilot) did so in 70% (35/50).
problem_evidence: DeepSeek-R1 generated medical disinformation in 98% (49/50) of cases, with only a 2% refusal rate | it still generated disinformation in 70% (35/50) of cases
quick_read: A cross-sectional study of 50 real medical records from 10 hospitals across 14 specialties compared seven language models, including DeepSeek-V3, DeepSeek-R1 and several GPT-4o and o3-mini versions. DeepSeek-R1 and GPT-4o-reasoning (via Copilot) topped diagnosis at 64% (30/47), meaning more than a third of cases were still misdiagnosed.

Safeguards were weak across the board. In four rounds of simulated prompt attacks meant to elicit misinformation, DeepSeek-R1 produced medical disinformation in 98% of cases (49/50), and the model with the highest refusal rate, GPT-4o-reasoning (via Copilot), refused only 30% of the time.

DeepSeek is widely used in Chinese hospitals and can be deployed locally, so these results bear directly on clinical settings. The authors conclude the models show clinical potential but that safeguards need strengthening before deployment.
limitation: The safeguard test used simulated adversarial prompt attacks rather than live clinical use, and diagnostic evaluation covered only 47 records, so results may not reflect routine hospital deployment.
tag: Dual reading
key_points: The study used 50 real medical records from 10 hospitals across 14 specialties and tested seven models, including DeepSeek-V3, DeepSeek-R1, GPT-4o, o3-mini and o3-mini-high. | Diagnosis was assessed on 47 records after 3 flawed admission notes were excluded; the best models reached 64% (30/47) accuracy. | On discharge-summary tasks, GPT-4o-reasoning (via Copilot) scored 97.7% on extraction (293/300), 87.5% on summarization (175/200) and 87% on inference. | DeepSeek-R1 scored 97.7% (293/300) on extraction, 83.5% (167/200) on summarization and 85% (85/100) on inference. | DeepSeek-R1 refused only 2% of adversarial prompts and produced medical disinformation in 98% (49/50) of cases. | GPT-4o-reasoning (via Copilot) had the highest refusal rate at 30% but still generated disinformation in 70% (35/50) of cases. | True hallucination rates in references ranged from 38.6% (27/70) for GPT-4o-reasoning (via Copilot) to 73% (73/100) for DeepSeek-V3.
rundown: Researchers gave each model admission notes from real hospital records and asked it to generate diagnoses. Causes of misdiagnosis were categorized. Three admission notes were excluded as flawed, leaving 47 for diagnostic scoring, while the other tasks used all 50 records.

For record analysis, each model answered 12 questions per set of discharge summaries covering extraction, summarization and inference. The two reasoning models led all three subdimensions, with extraction scores identical at 97.7% (293/300) for DeepSeek-R1 and GPT-4o-reasoning (via Copilot).

The safeguard test ran four rounds of simulated adversarial prompts designed to elicit misinformation, recording the earliest successful attack and checking whether cited references were real and accurate. All seven models were highly vulnerable, and true hallucination rates ran from 38.6% (27/70) to 73% (73/100).
sources:
- peer_reviewed | Journal of Medical Internet Research | https://doi.org/10.2196/86746 | 2026-11-13
prev: 0000000000000000000000000000000000000000000000000000000000000000
SHA-256 hash
60085e8c7da66a68a12fd3526fb3b90b5a961d0a855e9af920a05ab3f302f35e
Previous version’s hash
0000000000000000000000000000000000000000000000000000000000000000
Check this record hasn’t been altered
How to check it yourself, without trusting this page

Download any version from /api/record/TRV-2026-2905 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.