Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-2340
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-11-03T00:01:52.694210Z
status: published
lens: p_space
sector: health
headline: Model-Dependent Prompt-Component Ablation for Medical LLM Decision Support: Application to ENT Biologics Screening
dek: Large language models (LLMs) show promise for clinical decision support, but privacy constraints and unreliable free-text decisions limit deployment. We present Medotor, a locally deployed system for screening patients with chronic rhinosinusitis with nasal polyps (CRSwNP) for biologic therapy, and evaluate prompt design on 163 retrospective cases (132 eligible and 31 ineligible). Five models were tested on the same patients under five variants: a minimal prompt (P0), the full role/safety/structured nine-point p…
gain_title: (none)
problem_title: In screening 163 sinus-disease cases, the best balanced accuracy was only 0.720, and a longer full prompt sharply degraded one local model, producing 84 undecidable outputs.
trace_subject: (none)
gain_reading: (none)
gain_evidence: (none)
problem_reading: In screening 163 sinus-disease cases, the best balanced accuracy was only 0.720, and a longer full prompt sharply degraded one local model, producing 84 undecidable outputs.
problem_evidence: the full prompt sharply degraded Gemma 3n and produced 84 undecidable outputs | a longer or retrieval-augmented prompt was not universally safer or more accurate
quick_read: Researchers built Medotor, a locally run system that uses language models to screen patients with chronic rhinosinusitis with nasal polyps for biologic therapy. They tested five models on the same 163 retrospective cases under five prompt variants, from a minimal prompt to a full structured prompt with added knowledge retrieval.

The results show that prompt design cannot be applied uniformly across models: the same change helped one model and badly hurt another. That matters for hospitals considering locally deployed AI under privacy limits, though the study measured classification and output reliability only, not hallucination reduction or real clinical outcomes.
limitation: The knowledge block was neither personalized nor updated between patients, no standard-RAG arm was run, and hallucination labels were qualitative, so the study cannot show hallucination reduction or MemoRAG superiority.
tag: Evidence-backed problem
key_points: The test used 163 retrospective cases, 132 eligible and 31 ineligible for biologic therapy. | Best balanced accuracies ranged from 0.720 for DeepSeek V4 Flash to 0.549 for local Gemma 3n. | Removing the nine-point schema improved Qwen 3.5 compared with the full prompt (Holm-corrected p=0.015). | Adding a fixed MemoRAG-derived knowledge block did not significantly improve any model after correction.
rundown: Five models were compared on identical patients under five prompt variants: minimal, full role/safety/structured nine-point prompt, no role/safety block, no nine-point schema, and the full prompt plus a shared knowledge block. Every arm kept all 163 attempts, and undecidable or malformed final decisions counted as errors.

Balanced accuracy was the primary metric, analyzed with patient-level bootstrap intervals, exact paired McNemar tests and Holm correction. The top results were 0.713 for Qwen 3.5 397B and 0.704 for GPT-OSS 120B, while local Qwen 2.5 reached 0.657.

The authors conclude that prompt effects are strongly model-dependent. The full prompt sharply degraded the small local Gemma 3n model, which produced 84 undecidable outputs.
sources:
- peer_reviewed | IEEE Journal of Biomedical and Health Informatics | https://doi.org/10.1109/jbhi.2026.3749729 | 2026-11-02
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

d1ef2c852046f56b1d3c38492d9949236ac54261aafbfb74e9a699cbf5e656b8

previous

0000000000000000000000000000000000000000000000000000000000000000