Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1665
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-17T06:54:11.109161Z
status: published
lens: p_space
sector: health
headline: Multilingual Disparities in Large Language Model-Based Symptom Detection for Global Disease Surveillance: Evaluation Study
dek: Background Symptom detection is essential in global disease surveillance to detect potential outbreaks, as symptoms are the first observable signs of infection. To reflect real-time ground truth conditions during pandemics, social media has emerged as a valuable data source. Moreover, effective digital disease surveillance systems must operate across diverse linguistic settings, and large language models (LLMs) have been shown to perform inconsistently across languages, tending to have lower performance in low-r…
gain_title: (none)
problem_title: GPT-5 detected symptoms less accurately in Southeast Asian and low-resource languages than in others, which could make social media based disease surveillance uneven across regions.
trace_subject: (none)
gain_reading: (none)
gain_evidence: (none)
problem_reading: GPT-5 detected symptoms less accurately in Southeast Asian and low-resource languages than in others, which could make social media based disease surveillance uneven across regions.
problem_evidence: Southeast Asian (SEA) languages generally achieved lower scores than non-SEA languages | low-resource languages obtained the lowest overall scores
quick_read: Researchers tested GPT-5 as a symptom detector on the MedWeb dataset, a collection of pseudo-social media texts in 12 languages. They looked for eight symptoms and conditions, including fever, cough, hay fever and influenza, and scored results with macro precision, recall and F1.

Performance differed by language, by language resource group and by symptom. This matters because outbreak surveillance that relies on social media text needs to work in every language community, and uneven accuracy could leave some regions under-monitored. The results come from a single model and a pseudo-social media dataset, so real-world performance remains untested.
limitation: The evaluation used a pseudo-social media dataset and a single model, GPT-5, so the findings may not reflect real social media posts or other systems.
tag: Evidence-backed problem
key_points: Japanese had the highest F1-score at 0.812, while Lao had the lowest at 0.716. | High-resource languages performed most consistently; low-resource languages scored lowest overall. | Diarrhea, headache, cough and fever were detected stably across languages. | Hay fever, runny nose, influenza and cold varied more, with hay fever forming two performance clusters tied to language resource and region. | Error analysis found four patterns, with cross-lingual variation most frequent and explicitly mentioned symptoms having the largest potential impact.
rundown: The study used the MedWeb dataset, which covers 12 languages across diverse regions and language resource classifications. Symptoms evaluated were fever, headache, runny nose, cough, diarrhea, hay fever, influenza and cold. GPT-5 served as the detection system, chosen as a strong model for health-related tasks.

Southeast Asia was framed as a test case because it combines many languages with the potential for emerging infectious disease outbreaks. Southeast Asian languages generally scored below the others.

The error analysis identified four misclassification patterns: explicitly mentioned symptoms, cross-lingual variation, symptom overgeneralization and context misinterpretation. The authors conclude that more reliable and equitable detection would benefit from broader training data representation for low-resource languages, better cultural and linguistic sensitivity, and stronger contextual understanding.
sources:
- peer_reviewed | JMIR AI | https://doi.org/10.2196/103321 | 2026-10-09
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

f9b50cf4434a6aa035dc730654a5ffbe2b248cd6feb6a8513c389e51bd92c32d

previous

0000000000000000000000000000000000000000000000000000000000000000