TRV-2026-2935Version 1 · Published

Saved · see the current record

Why this version exists

Certified into the record

The exact text we saved

This is the version exactly as it was saved, character for character. The SHA-256 hash below is a 64-character code worked out from that text. Run the same text through SHA-256 and you get the same code; change a single letter and the code comes out completely different. Each version also stores the previous version’s hash, which chains them together so no version can be swapped out quietly.

TRUVACE RECORD VERSION
record: TRV-2026-2935
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-11-15T12:01:58.957704Z
status: published
lens: p_space
sector: health
headline: Benchmarking Large Language Models for Sentiment Analysis in Pregnancy-Related Weight Discussions on Reddit: Human Performance-Aligned Comparative Evaluation Study
dek: Background Pregnancy-related weight gain is both physiologically complex and socially charged, and online discussions often combine clinical information with implicit emotional distress. Although automated sentiment analysis could support research on pregnancy-related discourse, model performance in this context remains inadequately characterized relative to expert human judgment. Objective This study aimed to benchmark contemporary sentiment classification models on pregnancy-related weight discussions, with em…
gain_title: (none)
problem_title: In a JMIR Formative Research benchmark of 10 models on 150 Reddit pregnancy-weight posts, the best (Gemini-3-Flash) reached a weighted F1 of 0.662 against expert consensus, and the authors say such tools cannot stand alone as a measure of patient experience.
trace_subject: (none)
gain_reading: (none)
gain_evidence: (none)
problem_reading: In a JMIR Formative Research benchmark of 10 models on 150 Reddit pregnancy-weight posts, the best (Gemini-3-Flash) reached a weighted F1 of 0.662 against expert consensus, and the authors say such tools cannot stand alone as a measure of patient experience.
problem_evidence: Gemini-3-Flash had the numerically highest weighted F 1 -score of 0.662 (κ=0.478) | broad polarity classification can support corpus prioritization and hypothesis generation but is insufficient as a stand-alone measure of patient experience
quick_read: Researchers tested how well 10 sentiment models classify the tone of Reddit discussions about weight gain in pregnancy, using 150 held-out posts labelled by expert consensus. Gemini-3-Flash scored highest with a weighted F1 of 0.662, followed by GPT-OSS at 0.645, and confidence intervals across the leading models overlapped.

The human experts themselves found the task hard: the two primary experts agreed on only 59.0% of posts before adjudication (Cohen κ=0.344). Among Gemini-3-Flash's errors, 55.8% (29 of 52) fell on posts where those experts had disagreed, so much of the model error reflects real ambiguity in the text.

The authors conclude that broad polarity classification can help prioritise posts and generate hypotheses but is not enough by itself to measure how pregnant people experience their care or bodies. Agreement between the two AI models was high on posts (92.3%) but low on comments (46.3%).
limitation: The models were tested on only 150 held-out posts, the two primary experts agreed on just 59.0% of posts before adjudication, and confidence intervals overlapped across the leading models, so rankings are uncertain.
tag: Evidence-backed problem
key_points: The team built a corpus of 15,619 Reddit posts and comments through LLM-guided keyword expansion and set up an adjudicated reference set of 200 posts. | On the 150-post held-out test, Gemini-3-Flash scored a weighted F1 of 0.662 (κ=0.478) and GPT-OSS 0.645 (κ=0.436); BERTweet scored 0.623 and RoBERTa 0.605. | With fine-tuning in 5-fold cross-validation, RoBERTa's mean F1 rose to 0.708 (SD 0.087) while BERTweet's was 0.616 (SD 0.079). | In a separate 106-post extension, expert agreement was 64.2% (κ=0.367); Gemini-3-Flash and GPT-OSS scored 0.633 and 0.680, and both identified 56 of 58 expert-negative posts. | Agreement between Gemini-3-Flash and GPT-OSS was 92.3% on 400 posts (κ=0.797) but only 46.3% on 1,500 comments (κ=0.102). | All 714 post IDs retrieved through the API could be independently recovered through Arctic Shift.
rundown: The study started with a pool of 15,619 Reddit posts and comments about pregnancy-related weight, gathered with a keyword strategy expanded by a large language model. Two primary experts labelled sentiment and adjudicated a 200-post reference set, of which 150 posts formed a fixed test set. Models were scored with weighted metrics and 1,000-resample bootstrap confidence intervals.

No architecture came out uniformly ahead. The two general-purpose LLMs led under direct use, but a fine-tuned RoBERTa reached a mean F1 of 0.708 in cross-validation. The authors say performance depended on how the model was adapted, how ambiguous the labels were, and whether the text was a post or a comment.

When applied to unlabeled data, Gemini-3-Flash and GPT-OSS agreed closely on 400 posts but diverged on 1,500 comments, suggesting short replies are much harder to classify. In an exploratory analysis, both models caught 56 of 58 posts the experts rated negative, which the authors frame as a way to prioritise posts for human review.
sources:
- peer_reviewed | JMIR Formative Research | https://doi.org/10.2196/92821 | 2026-11-13
prev: 0000000000000000000000000000000000000000000000000000000000000000
SHA-256 hash
c9cb8b6c9c98221b1cce8e9d977847317aac3f06c3fb4d81238f461e8f3f15b9
Previous version’s hash
0000000000000000000000000000000000000000000000000000000000000000
Check this record hasn’t been altered
How to check it yourself, without trusting this page

Download any version from /api/record/TRV-2026-2935 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.