Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1669
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-17T06:54:33.574905Z
status: published
lens: trace
sector: health
headline: Reproducible but Incoherent: A Reference Standard-Free Reliability Benchmark of LLM-Assisted Nutritional Estimation Across 20 Model Configurations
dek: Nutritional estimation pipelines built on large language models are entering biomarker research, but free-text meal descriptions have no ground truth, so outputs cannot be validated against a reference standard. What can be established without one is whether the same input returns the same number. We submitted 191 de-identified Spanish meal descriptions, byte-identically, three times each, at temperature 0, to 20 model configurations across two inference platforms (12,119 calls), and measured test–retest reliabi…
gain_title: All 20 LLM configurations tested gave more repeatable energy estimates for the same meal description than the audited cascade comparator, with the best reaching a test-retest ICC of 1.0000.
problem_title: High reproducibility did not ensure internal coherence: the most reproducible model departed from its own macronutrients by more than 10% in 65.58% of responses, versus 3.80% for the cascade.
trace_subject: Reliability and coherence of LLM-assisted nutritional estimation from meal descriptions
gain_reading: All 20 LLM configurations tested gave more repeatable energy estimates for the same meal description than the audited cascade comparator, with the best reaching a test-retest ICC of 1.0000.
gain_evidence: All 20 models exceeded the audited cascade’s energy ICC under both comparators
problem_reading: High reproducibility did not ensure internal coherence: the most reproducible model departed from its own macronutrients by more than 10% in 65.58% of responses, versus 3.80% for the cascade.
problem_evidence: the most reproducible model (ICC 1.0000) departed from its own macronutrients by more than 10% under the simplified 4/4/9 Atwater approximation in 65.58% of responses, against 3.80% for the cascade
quick_read: Researchers sent 191 de-identified Spanish meal descriptions, each three times at temperature 0, to 20 LLM configurations on two inference platforms, totalling 12,119 calls. They measured whether the same input produced the same number, how models agreed with one another, and whether reported energy matched reported macronutrients.

The work matters because LLM nutrition pipelines are entering biomarker research where no ground truth exists for free-text meals. The authors conclude that repeatability alone cannot justify use and that a reference-standard-free coherence check is also needed, though their coherence measure is only a ranked screening dimension rather than a validated criterion.
limitation: The coherence measure is not a validated criterion, and the binary acceptance gate is sensitive to the chosen bound, so it cannot establish the accuracy of nutritional estimates.
tag: Dual reading
key_points: The study used 191 Spanish meal descriptions submitted three times each to 20 model configurations across two platforms. | Every model exceeded the audited cascade on energy test-retest reliability under both comparators. | The most reliable model also ranked first of 20 in incoherence at every bound from 5% to 20%. | A pass/fail coherence gate was unstable, with 3 to 17 of 20 configurations passing depending on the bound. | The ranking of models by coherence was stable, with Kendall tau of at least 0.74 between any two bounds.
rundown: The design deliberately avoided a reference standard because free-text meal descriptions have no ground truth. Instead it tested measurable properties: test-retest reliability by ICC(1,1), cross-model agreement, Atwater coherence between energy and macronutrients, and the reliability gradient across input specificity.

The central finding is a dissociation between two nearly orthogonal axes. One configuration scored an ICC of 1.0000 yet had the largest share of incoherent responses under the 4/4/9 Atwater approximation, at 65.58%, compared with 3.80% for the audited cascade.

Because a binary acceptance threshold changed outcomes sharply across bounds, the authors report coherence as a ranked screening dimension with a decision curve. They recommend at least one reference-standard-free coherence check alongside reliability before authorising an LLM-assisted nutritional pipeline.
sources:
- peer_reviewed | Machine Learning and Knowledge Extraction | https://doi.org/10.3390/make8100331 | 2026-10-15
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

ad7e406144ac84103e1efe8447997d015610725e78d74f104de2acc2678cf2fa

previous

0000000000000000000000000000000000000000000000000000000000000000