52/100

Moderate evidence

Trace: a gain and a problemPublished recordScienceThe Trace

LLMs in the Laboratory: Benchmarking Conversational AI for Multivariate Analysis of NMR Spectroscopic Data

Truvace record
TRV-2026-3025raw data (JSON)
Version
v1Published
First published
Evidence type
Benchmark or lab evaluation
Sources
1 sourcePeer-reviewed
sha256
a0d5b891c5c2…check it

This study presents a comparative evaluation of five state-of-the-art large language models (LLMs) (Claude Opus 4.6, Gemini 3.1 Pro, Grok-4, ChatGPT (GPT-5.4 Thinking), and Qwen 3.5) for their ability to perform key chemometric tasks through natural language interaction. Using a low-field nuclear magnetic resonance (LF-NMR) dataset of edible vegetable oils, the models were prompted to execute a complete multivariate analysis workflow, including data exploration, unsupervised and supervised classification, model…

49Gain side

How the two sides scoreProblem scores higher

54Problem side

The gain

In a benchmark of five chatbots on low-field NMR data from edible vegetable oils, Claude Opus 4.6 ran the whole analysis pipeline on its own and correctly predicted all nine unlabeled samples.

The problem

In a benchmark of five chatbots on low-field NMR data from edible vegetable oils, the open-source Qwen 3.5 failed unrecoverably at the supervised classification stage and could not finish the workflow, and Grok-4 showed limits in data handling autonomy.

A fixed formula sets both scores from the evidence and its sources. Reader votes don’t move them.

What the scores mean

G is the gain side, out of 100. It goes up when the gain is specific, when it measurably helped people and when the sources behind it are strong.

P is the problem side, out of 100. It goes up when the problem is specific, when it measurably hurt people and when the sources behind it are strong.

What this doesn’t fix

The evaluation used a single low-field NMR dataset of edible vegetable oils, and the authors note that general-purpose benchmarks may have limitations for domain-specific chemometric performance.

Sources

1 on file
  1. Analytical Chemistrydoi:10.1021/acs.analchem.6c02701
    Peer-reviewed2026-11-17

Cite this record

v1

Truvace Impact Record TRV-2026-3025, v1: “LLMs in the Laboratory: Benchmarking Conversational AI for Multivariate Analysis of NMR Spectroscopic Data.” Truvace, 2026-11-18. /record/TRV-2026-3025 (accessed on the day you copy it). sha256 a0d5b891c5c2aa9c…

Change history

1 version

Every change made to this record since it was first published, and why. Nothing has changed since it went up.

  1. Publishedv1a0d5b891c5c2…

    Certified into the record

Check this record hasn’t been altered

Check this record hasn’t been altered
How to check it yourself, without trusting this page

Download any version from /api/record/TRV-2026-3025 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.