Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1862
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-18T22:02:01.919274Z
status: published
lens: g_space
sector: health
headline: Benchmarking commercial large language models for gene-disease-phenotype extraction from full-text human genetics literature
dek: Manual curation of gene-disease-phenotype relationships from the human genetics literature is a persistent bottleneck for maintaining its bioinformatics databases. Whereas large language models (LLMs) offer a promising alternative, there is currently no systematic benchmark that evaluates whether state-of-the-art commercial LLMs can perform this task reliably on the full-text articles. To address this gap, we introduce a standardized benchmark comprising 406 full-text articles covering 180 congenital heart disea…
gain_title: Commercial LLMs extracted structured gene, disease, and phenotype fields from full-text genetics papers with high accuracy, which could ease the manual curation bottleneck.
problem_title: (none)
trace_subject: (none)
gain_reading: Commercial LLMs extracted structured gene, disease, and phenotype fields from full-text genetics papers with high accuracy, which could ease the manual curation bottleneck.
gain_evidence: The top-performing model, Grok-4, achieved 97.6% overall accuracy | Manual curation of gene-disease-phenotype relationships from the human genetics literature is a persistent bottleneck
problem_reading: (none)
problem_evidence: (none)
quick_read: Researchers built a benchmark of 406 full-text articles covering 180 genes linked to congenital heart disease. They used it to test seven commercial language models on pulling out structured gene, disease, and phenotype information, comparing outputs against a curated database.

The result matters because curation of genetics databases is slow and manual, and the tested models performed well on this task. The abstract does not describe real-world deployment, and the evaluation covers one disease area, so how the results carry over to other fields remains unclear.
limitation: The benchmark covers only congenital heart disease-associated genes, so performance on other disease areas or literature is not established by this source.
tag: Evidence-backed gain
key_points: The benchmark includes 406 full-text articles covering 180 congenital heart disease-associated genes. | Seven models were tested: GPT-4o, Claude-Opus-4, DeepSeek-R1, Grok-4, Qwen-3.5, Gemini-2.5 (Pro), and GPT-5. | Grok-4 scored highest at 97.6% overall accuracy, while the lowest-performing model reached roughly 88%. | The evaluation framework uses fuzzy matching to account for synonyms and partial matches. | The dataset, framework, and model outputs are released online for reproducible follow-up studies.
rundown: The study responds to the absence of a systematic benchmark for whether commercial LLMs can reliably handle gene-disease-phenotype extraction from full-text articles rather than abstracts alone.

Models were asked to extract structured gene, disease, and phenotype fields, and results were scored with a multi-dimensional framework that tolerates synonyms and partial matches.

Even the weakest model, at about 88%, outperformed many earlier zero-shot or n-shot prompting benchmarks in biomedical relation extraction, according to the authors. The paper has two components: a human genetics field benchmark against a curated database and the evaluation framework itself.
sources:
- peer_reviewed | Quantitative Biology | https://doi.org/10.1002/qub2.70050 | 2026-08-17
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

29c2e02e97abe6da17e85679cc775234b330df319e5cc1aadce64355640457ae

previous

0000000000000000000000000000000000000000000000000000000000000000