Truace
Subscribe
55/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

AI energy benchmark of 60 systems finds easy efficiency gains are running out

Study title: MLPerf Power: Addressing the Artificial Intelligence Power Wall

Evidence type
Benchmark or lab evaluation
Sample
n = 1,841
Sources
1 sourcePeer-reviewed
Published
1 min read
Truvace record
TRV-2026-2619cite or check it

Study title: MLPerf Power: Addressing the Artificial Intelligence Power Wall

The MLPerf Power methodology measures energy efficiency for machine learning systems ranging from milliwatt edge devices to multi-megawatt training clusters, a span of nine orders of magnitude. It reports shared metrics such as samples per joule and tokens per joule, and has produced 1,841 reproducible submissions from 60 systems.

Read the study abstract

Power has become the defining constraint for modern machine learning, from milliwatt edge devices to multi-megawatt training clusters. Across this nine-order-of-magnitude span, the field has lacked a consistent way to measure and compare energy efficiency. We present MLPerf Power, a cross-scale energy benchmarking methodology that adapts instrumentation while preserving common measurement principles and reports shared metrics such as samples per joule and tokens per joule. The methodology has produced 1,841 repr…

The problem

A benchmark of 1,841 submissions from 60 systems found that easy AI energy-efficiency gains are plateauing and that energy use scales nonlinearly with accelerator count.

In brief

Analysis of that corpus shows that easy efficiency gains are plateauing and that energy scales nonlinearly with accelerator count. It also shows that low-precision innovation has substantially reduced the cost of high-accuracy inference.

For large language model inference, the authors report that sparse mixture-of-experts models pay a memory-bandwidth penalty that contradicts training intuitions. They say the public corpus already informs industry roadmaps, infrastructure planning and verified carbon accounting.

Main points

  1. The methodology adapts instrumentation across scales while keeping common measurement principles.
  2. It yielded 1,841 reproducible submissions from 60 systems.
  3. Metrics include samples per joule and tokens per joule.
  4. Energy scales nonlinearly with accelerator count.
  5. Low-precision innovation has substantially reduced the cost of high-accuracy inference.
  6. Sparse mixture-of-experts LLMs pay a memory-bandwidth penalty during inference that contradicts training intuitions.

The full story

The paper describes MLPerf Power as a cross-scale energy benchmarking methodology. It covers systems from milliwatt edge devices to multi-megawatt training clusters, where the field previously lacked a consistent way to measure and compare energy efficiency.

The submission corpus shows that easy efficiency gains are plateauing, and that adding accelerators increases energy use nonlinearly. Low-precision techniques have substantially lowered the cost of running high-accuracy inference.

Extending the benchmark to LLM inference, the authors show that sparse mixture-of-experts models pay a memory-bandwidth penalty. The publicly accessible corpus is said to inform industry roadmaps, infrastructure planning, and energy-aware optimization and verified carbon accounting.

Sources

1 on file
  1. IEEE Microdoi:10.1109/mm.2026.3700126
    Peer-reviewed2026-06-04

Talk it over

Four questions to start onWho carries this cost, and what would have to change to prevent it?

Start with a question

AI energy benchmark of 60 systems finds easy efficiency gains are running outhttps://truvace.com/item/mlperf-power-addressing-the-artificial-intelligence-power-wall

  1. Who carries this cost, and what would have to change to prevent it?

    The problem A benchmark of 1,841 submissions from 60 systems found that easy AI energy-efficiency gains are plateauing and that energy use scales nonlinearly with accelerator count.

    Answer in the thread

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a benchmark or lab test with a sample of 1,841, from IEEE Micro.

    Answer in the thread

  3. What does it leave open? What else would you want to know before acting on it?

    Answer in the thread

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

    Answer in the thread

The questions are the same on every record. The details under them are taken from this record as written.

What readers say

More from Climate