Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1783
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-18T06:53:34.955778Z
status: published
lens: g_space
sector: health
headline: Responses of large language models to common questions on premature ovarian insufficiency: A comparative analysis
dek: Objective: Large language models are widely used in clinical care. Their reliability in specialized gynecological diseases such as premature ovarian insufficiency (POI) has not been fully verified. This study aimed to systematically evaluate and compare the accuracy and completeness of three mainstream large language models when answering standardized clinical questions about POI.
Methods: We developed 34 standardized questions covering five core domains of POI according to international clinical guidelines. Two…
gain_title: Two gynecologists rated GPT-5.5, Claude Opus-4.7 and DeepSeek-R1 as accurate and complete on 34 guideline-based POI questions, supporting supervised use for information retrieval and patient education.
problem_title: (none)
trace_subject: (none)
gain_reading: Two gynecologists rated GPT-5.5, Claude Opus-4.7 and DeepSeek-R1 as accurate and complete on 34 guideline-based POI questions, supporting supervised use for information retrieval and patient education.
gain_evidence: These models can serve as reliable auxiliary tools for clinical information retrieval and patient education under professional supervision
problem_reading: (none)
problem_evidence: (none)
quick_read: Researchers built 34 standardized questions across five core domains of premature ovarian insufficiency, drawn from international clinical guidelines. They put the questions to three large language models and had two senior obstetricians and gynecologists independently score the answers on a 5-point scale for accuracy and completeness.

The results suggest these chatbots can handle a specialized gynecological topic reasonably well, which matters as such tools are increasingly used in clinical care. The authors still require qualified clinicians to check all AI-generated content before clinical use, and the test covers only a fixed question set rather than real patient interactions.
limitation: Some individual-item comparisons between models exceeded the ±0.5 range, and the authors say all AI-generated content must be checked by qualified clinicians before clinical use.
tag: Evidence-backed gain
key_points: Accuracy scores across the three models ranged from 4.59 to 4.84, and completeness scores from 4.43 to 4.65. | Exact agreement between the two raters was 91.18% to 97.06%, with Cohen's kappa between 0.675 and 0.915. | GPT-5.5 did better on clinical manifestations and treatment questions, while DeepSeek-R1 was more stable on etiology and counseling questions. | Most item-level differences between models fell within ±0.5 points, with no obvious systematic bias.
rundown: The study tested GPT-5.5, Claude Opus-4.7 and DeepSeek-R1 on 34 standardized questions covering five core POI domains. Inter-rater reliability was assessed with Cohen's kappa, the intraclass correlation coefficient and mean absolute error, and the agreement was described as good to excellent.

Pairwise score differences between models were computed and shown in item-level heatmaps. Overall mean score differences fell within ±0.5, though some individual-item comparisons exceeded that range. The authors describe overall performance as similar across the models despite deviations on several questions.

The authors conclude the models could serve as auxiliary tools for clinical information retrieval and patient education, but only under professional supervision, with qualified clinicians checking all generated content before clinical use.
sources:
- peer_reviewed | International Journal of Gynecology & Obstetrics | https://doi.org/10.1002/ijgo.71422 | 2026-10-17
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

61d9c7fa41cc0cf4c0d282466d42956a3348120ea611dcb3029f94099c141dc1

previous

0000000000000000000000000000000000000000000000000000000000000000