aceConcept-Aware Learning Framework for Detecting Common Symptoms in Social Media Text: Model Development and Evaluation Study
Background: Social media offers valuable information for public health surveillance, but detecting valid symptom expressions in noisy and informal text remains challenging. Fine-tuned language models can achieve strong test accuracy while relying on spurious patterns, limiting their reliability in real-world applications. Objective: This study aimed to develop a more robust symptom detection model by incorporating surveillance-relevant attributes into the training objective and introducing a counterfactual-based…
A concept-aware training framework raised sensitivity to surveillance-relevant attributes (flip accuracy 0.830 vs 0.342) and out-of-distribution flu-or-fever detection (0.856 vs 0.797) without losing standard test accuracy.
Standard fine-tuned symptom detectors scored well on the held-out test (m-F1 0.871) yet were largely insensitive to recovery status, timing, and uncertainty, relying on spurious patterns.
The source notes that results depended on the augmentation model and that prompting GPT-4.1 improved robustness metrics but lowered standard test performance, so no single approach was best across all measures.
Evidence
- Peer-reviewedJMIR Medical Informatics2026-10-15
How should this claim be treated?
Truvace Impact Record TRV-2026-1536, v1: “Concept-Aware Learning Framework for Detecting Common Symptoms in Social Media Text: Model Development and Evaluation Study.” Truvace, 2026-10-16. /record/TRV-2026-1536 (accessed at citation time). sha256 70fbd5688e916ce0…
Calibration history
Every change to this record since certification, in the open. None yet — the reading has held since it entered the record.
Certified into the record
How to verify without trusting this page
Fetch the canonical text of any version from /api/record/TRV-2026-1536 and hash it yourself — for example shasum -a 256 on the saved canonical field. The result must equal content_hash, and each version’s text ends with prev:followed by the prior version’s hash (version 1 chains to 64 zeros). If a single character of any version had been altered since certification, the chain would not reproduce.