Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1741
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-17T22:02:05.257184Z
status: published
lens: g_space
sector: climate
headline: Does Lower Precision Always Save Energy? Benchmarking Energy–Performance Trade-Offs in Weight-Quantized Large Language Model Inference on a GPU
dek: Large language model (LLM) inference is an increasing component of AI energy demand, while the energy effects of weight quantization depend on workload phase and implementation. We benchmarked four LLM families—Gemma 4 E4B IT, Qwen3.5 4B, Nemotron 3 Nano 4B, and Ministral 3 3B Instruct—across Q4_K_M, Q6_K, Q8_0, and BF16 weight representations on a single NVIDIA GeForce RTX 4090 using llama.cpp/CUDA. Five workloads separated prefill from autoregressive generation, and GPU energy was estimated from NVML telemetry…
gain_title: On one RTX 4090 using llama.cpp/CUDA, 4-bit Q4_K_M weights cut dynamic energy per token during generation by 63.6–85.7% versus BF16 and raised throughput across all four tested model families.
problem_title: (none)
trace_subject: (none)
gain_reading: On one RTX 4090 using llama.cpp/CUDA, 4-bit Q4_K_M weights cut dynamic energy per token during generation by 63.6–85.7% versus BF16 and raised throughput across all four tested model families.
gain_evidence: Q4_K_M reduced dynamic energy per token by 63.6–85.7% | increased generation throughput by 85.2–160.2% across all four model families
problem_reading: (none)
problem_evidence: (none)
quick_read: Researchers benchmarked four small language model families at four weight precisions (Q4_K_M, Q6_K, Q8_0, BF16) on a single consumer GPU. They ran five workloads that separated prompt prefill from token-by-token generation, and estimated GPU energy from NVML telemetry.

The results suggest lower precision does not automatically save energy in every phase, but it clearly helps during generation, where inference energy tends to accumulate. The findings come from a narrow setup, and output quality was not evaluated, so the efficiency gains cannot yet be weighed against any accuracy loss.
limitation: Findings are limited to single-request execution on one RTX 4090 with llama.cpp/CUDA, output quality was not assessed, and the 50 ms NVML sampling limits interpretation of short prefill measurements.
tag: Evidence-backed gain
key_points: Generation scenarios showed significant effects of model family, precision, and their interaction on the primary endpoints. | In the 512-token prefill workload, no statistically significant differences were detected under the measurement protocol. | In the 1024-token prefill workload, precision affected total energy and mean GPU power. | Ministral 3 3B Instruct–Q4_K_M ranked first in the TOPSIS multi-criteria ranking. | Nemotron 3 Nano 4B–Q4_K_M required less GPU memory than the other configurations.
rundown: The study tested Gemma 4 E4B IT, Qwen3.5 4B, Nemotron 3 Nano 4B and Ministral 3 3B Instruct on one NVIDIA GeForce RTX 4090 with llama.cpp and CUDA. Energy was estimated from NVML telemetry across five workloads.

Prefill results were mixed: the shorter 512-token case showed no significant differences, while the 1024-token case showed precision effects on total energy and mean GPU power. This is why the authors question whether lower precision always saves energy.

For generation, the 4-bit configuration was the most efficient against BF16 in every family, and a TOPSIS ranking placed Ministral 3 3B Instruct–Q4_K_M first, with Nemotron 3 Nano 4B–Q4_K_M using the least GPU memory.
sources:
- peer_reviewed | Energies | https://doi.org/10.3390/en19204893 | 2026-10-16
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

82bef67fd20c5f7f18b97403edc5842b10e04145e9420708699e0eac41936f30

previous

0000000000000000000000000000000000000000000000000000000000000000