Carbon footprint AI had 33% median error against 269 product declarations, company test finds

The companion materials mapper, which links procurement text such as '304 SS sheet' to life-cycle datasets, matched expert-accepted answers on 91.2% of well-specified inputs across 1,198 test cases. For vague inputs such as internal product codes or generic descriptors like 'plastic part', that fell to 59.7%.

The gap matters because companies want footprints for thousands of products and suppliers, and checking each one by hand is impractical. The authors say the errors show where human review or better data are needed. They also say the system is not meant as proof that it meets the proposed criteria, and the figures come from the vendor's own evaluations.

The authors developed the criteria in three phases: a review of ISO 14040/44, the GHG Protocol Product Standard and the WBCSD PACT framework; structured workshops with practitioners from eight enterprises across seven sectors; and iterative pilot tests on real procurement data. Two verification firms, in Europe and North America, were consulted separately. The criteria rest on transparent logic, benchmarking against expert judgment and uncertainty characterization.

The mapper was tested on 1,198 cases drawn from an external benchmark and production data. Labels came from six in-house sustainability scientists, three of them with doctorates in life-cycle assessment or adjacent fields. Each item was reviewed by two to six annotators. Accuracy counted any mapping on an expert-accepted list, and 24.8% of items had two accepted mappings. Its quality reviewer put 0.3% of mappings in the highest-confidence tier as erroneous, and one of 97 errors in the evaluation set landed there.

For the modeling system, the authors point out that EPDs for identical products can differ by more than 50% under equally compliant assumptions, and that six expert teams modeling the same cotton data diverged by 44%. They say the 33% median error sits within that range of practitioner disagreement. The systems used Gemini 2.5 Flash for mapping, Claude Haiku 4.5 for quality review and Claude Opus 4.5 for modeling as of January 2026.

The authors recommend that reviewers always examine the highest-materiality outputs and any flagged items. They warn that companies could cherry-pick favorable AI outputs or suppress low-confidence flags, and propose documentation, review trails and random sampling to make that detectable.