The exact text we saved

This is the version exactly as it was saved, character for character. The SHA-256 hash below is a 64-character code worked out from that text. Run the same text through SHA-256 and you get the same code; change a single letter and the code comes out completely different. Each version also stores the previous version’s hash, which chains them together so no version can be swapped out quietly.

TRUVACE RECORD VERSION
record: TRV-2026-2490
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-11-12T07:42:05.133884Z
status: published
lens: p_space
sector: climate
headline: Evaluation criteria for AI-assisted product carbon footprinting systems: the cases of mapping and supply chain modeling
dek: Abstract Large language models (LLMs) are increasingly used to automate components of product carbon footprinting (PCF). However, the carbon footprinting literature lacks criteria for evaluating the quality, documentation, uncertainty indication, and reviewability of AI-assisted systems for specific PCF use cases. We developed a set of evaluation criteria for AI-assisted carbon footprinting (AI-CF) through standards review, structured stakeholder elicitation with corporate practitioners and verification speciali…
gain_title: (none)
problem_title: In a test by the developer's own staff, an AI carbon-footprint modeler had a 33% median error against 269 product declarations, and its materials mapper matched experts on only 59.7% of vague inputs.
trace_subject: (none)
gain_reading: (none)
gain_evidence: (none)
problem_reading: In a test by the developer's own staff, an AI carbon-footprint modeler had a 33% median error against 269 product declarations, and its materials mapper matched experts on only 59.7% of vague inputs.
problem_evidence: yielded a 33% median relative error
quick_read: Authors from Watershed Technology, a company that sells AI carbon-accounting tools, built evaluation criteria for AI-assisted product carbon footprinting and tested two of their own systems. Against 269 Environmental Product Declarations (EPDs), the modeling system had a median relative error of 33%, with an 80% interval of 6% to 98%.

The companion materials mapper, which links procurement text such as '304 SS sheet' to life-cycle datasets, matched expert-accepted answers on 91.2% of well-specified inputs across 1,198 test cases. For vague inputs such as internal product codes or generic descriptors like 'plastic part', that fell to 59.7%.

The gap matters because companies want footprints for thousands of products and suppliers, and checking each one by hand is impractical. The authors say the errors show where human review or better data are needed. They also say the system is not meant as proof that it meets the proposed criteria, and the figures come from the vendor's own evaluations.
limitation: The evaluation covers only two proprietary systems from the authors' own company, with labels from small annotator pools and no formal inter-annotator agreement; component-level checks could only be tested against product totals.
tag: Evidence-backed problem
key_points: The mapper scored 91.2% expert-accepted accuracy on 1,039 non-vague test cases versus 59.7% on 159 vague ones, out of 1,198 total. | On non-vague inputs, accuracy was 98.9% on the public Amazon Parakeet benchmark and 85.9% on production data; vague production inputs reached only 55.4%. | The modeling system's median relative error against 269 EPDs was 33%, its median signed error was -2%, and 86% of cases fell within 80% of the benchmark. | Thirteen runs overshot by more than 200% (maximum +4494%), pushing the mean signed error to +48%, well above the median; the largest over-predictions came from cashmere allocation, specialty chemicals and recycled alloys. | Run ten times, 'plastic part' varied between polyethylene and polypropylene (normalized entropy 0.57), while exact database names mapped identically; flagging any disagreement sent 39% of items to review and caught 79% of incorrect mappings. | Component-level checks on the 269 cases passed 75-99% of the time depending on the check, but the authors say the evidence does not yet support using them to waive human review. | Criteria were built from a standards review, a practitioner group of eight enterprises and more than sixty participants, two verification firms, and pilot testing.
rundown: The authors developed the criteria in three phases: a review of ISO 14040/44, the GHG Protocol Product Standard and the WBCSD PACT framework; structured workshops with practitioners from eight enterprises across seven sectors; and iterative pilot tests on real procurement data. Two verification firms, in Europe and North America, were consulted separately. The criteria rest on transparent logic, benchmarking against expert judgment and uncertainty characterization.

The mapper was tested on 1,198 cases drawn from an external benchmark and production data. Labels came from six in-house sustainability scientists, three of them with doctorates in life-cycle assessment or adjacent fields. Each item was reviewed by two to six annotators. Accuracy counted any mapping on an expert-accepted list, and 24.8% of items had two accepted mappings. Its quality reviewer put 0.3% of mappings in the highest-confidence tier as erroneous, and one of 97 errors in the evaluation set landed there.

For the modeling system, the authors point out that EPDs for identical products can differ by more than 50% under equally compliant assumptions, and that six expert teams modeling the same cotton data diverged by 44%. They say the 33% median error sits within that range of practitioner disagreement. The systems used Gemini 2.5 Flash for mapping, Claude Haiku 4.5 for quality review and Claude Opus 4.5 for modeling as of January 2026.

The authors recommend that reviewers always examine the highest-materiality outputs and any flagged items. They warn that companies could cherry-pick favorable AI outputs or suppress low-confidence flags, and propose documentation, review trails and random sampling to make that detectable.
sources:
- peer_reviewed | Journal of Industrial Ecology | https://doi.org/10.1007/s44498-026-00196-9 | 2026-11-05
prev: 0000000000000000000000000000000000000000000000000000000000000000

SHA-256 hash

a5d312b769f85298900e6b19706d857fd4f4c66561364c6412e348393a261e8b

Previous version’s hash

0000000000000000000000000000000000000000000000000000000000000000