aceModerate evidence
More
Simple spam filters came close to the top AI model, which emitted 88 times more CO₂, test finds
Study title: Beyond accuracy: a multi-dimensional green AI framework for sustainable machine learning—energy, carbon, and performance trade-offs in SMS spam detection
- Evidence type
- Benchmark or lab evaluation
- Sample
- n = 5,169
- Sources
- 1 sourcePeer-reviewed
- Published
- 3 min read
- Truvace record
- TRV-2026-2613cite or check it
Study title: Beyond accuracy: a multi-dimensional green AI framework for sustainable machine learning—energy, carbon, and performance trade-offs in SMS spam detection
A benchmark of 10 machine-learning models on the SMS Spam Collection (5,169 unique messages) found a sharp trade-off between accuracy and environmental cost. DistilBERT scored best, at 99.13% accuracy and an MCC of 0.9603, but it emitted 88 times more CO₂ per million predictions than Logistic Regression.
Read the study abstract
Abstract Until recently, machine learning research has primarily focused on prediction accuracy, often neglecting computational efficiency and environmental impact. In this study, 10 models encompassing classical machine learning, ensemble learning, and deep learning methods were evaluated on the SMS Spam Collection dataset (5,169 messages) using the proposed Multidimensional Green AI Framework. This framework considers models in three dimensions: (i) classification performance (MCC and F1-score), (ii) operation…
54Gain side
How the two sides scoreProblem scores higher
61Problem side
The gain
In a test of 10 models on 5,169 text messages, Logistic Regression and Naive Bayes came close to DistilBERT's accuracy (MCC 0.9219 and 0.9250 against 0.9603) while using far less energy and carbon.
The problem
The same test found DistilBERT's edge over Logistic Regression came with 88 times more CO₂ per million predictions, and its training emissions ran about 1,000 times higher than the most efficient models.
A fixed formula sets both scores from the evidence and its sources. Reader votes don’t move them.
What the scores mean
G is the gain side, out of 100. It goes up when the gain is specific, when it measurably helped people and when the sources behind it are strong.
P is the problem side, out of 100. It goes up when the problem is specific, when it measurably hurt people and when the sources behind it are strong.
In brief
Logistic Regression reached an MCC of 0.9219 and Naive Bayes 0.9250, with p95 latencies under 0.5 ms and models under 2 MB. DistilBERT's model was 255.5 MB with an 86.26 ms p95 latency, and its total pipeline took 963 seconds against 0.56 seconds for Naive Bayes, about 1,720 times longer.
Spam filtering runs continuously on billions of messages, so small per-prediction differences add up. In the author's example of 1 million predictions a day, Logistic Regression would emit about 1.17 kg of CO₂ a year and DistilBERT about 102.86 kg. The results come from one dataset and one laptop-class machine, so absolute figures will differ elsewhere.
Main points
- DistilBERT ranked first on accuracy (99.13%), MCC (0.9603) and F1 (0.9644), and it flagged no genuine messages as spam (100% precision).
- Naive Bayes ranked fourth by MCC (0.9250) and Logistic Regression fifth (0.9219), ahead of SVM, XGBoost, CNN, LightGBM and Random Forest.
- Total CO₂ for DistilBERT was 0.002240 kg against 0.000006 kg for Naive Bayes, a 373-fold gap. Training emissions were 974 times higher than Naive Bayes.
- Energy per 1,000 predictions was 0.0069 Wh for Logistic Regression and 0.6060 Wh for DistilBERT; CO₂ per million predictions was 0.0032 kg against 0.2816 kg.
- Deep learning models needed 97 to 291 MB of RAM during inference; classical and ensemble models used a negligible amount.
- XGBoost had the smallest model at 0.13 MB and a p95 latency of 0.81 ms, which the paper presents as a balance of accuracy and efficiency. LightGBM was the slowest at inference, with a p95 of 204.18 ms.
- The paper recommends classical models for mobile and IoT devices, ensemble models for edge computing, and deep learning for cloud systems where resource use is acceptable.
- Emissions were measured with CodeCarbon on Turkey's grid (0.465 kg CO₂/kWh); the paper estimates a low-carbon grid such as Sweden's would cut absolute CO₂ by about 97% without changing energy use.
The full story
The study compared three Logistic Regression, Naive Bayes and SVM models, three ensemble models (Random Forest, XGBoost, LightGBM) and four deep-learning models (LSTM, GRU, CNN, DistilBERT). After removing 403 duplicates from 5,572 messages, 4,516 were legitimate and 653 (12.63%) were spam. All models used the same stratified 80/20 split, 5-fold cross-validation and a decision threshold tuned to maximise MCC.
Operational costs separated the models more sharply than accuracy did. Accuracy ranged only from 97.78% (Random Forest) to 99.13% (DistilBERT). DistilBERT's cross-validation alone took 721 seconds and its training 171 seconds, against 0.18 and 0.003 seconds for Naive Bayes. Classical models averaged 8.10 seconds of total development time, ensembles 80.70 seconds and deep-learning models 358.54 seconds.
Scaled to higher volumes, the gap grows. In a system handling 10 million SMS a day, the paper calculates Naive Bayes would emit 0.036 kg of CO₂ a day and DistilBERT 2.82 kg. The paper says a gain of under 1% in accuracy carried 79 times more carbon cost. All 10 models had train-test accuracy gaps under 0.02, and deep-learning models showed no clear generalisation advantage over classical ones.
All experiments ran on one Intel Core i7-10750H laptop with an NVIDIA RTX 2060 GPU and 16 GB of RAM. The author says the aim was to propose a multidimensional evaluation framework covering performance, efficiency and sustainability, not to solve spam detection, and proposes compression, pruning and multilingual testing as next steps.
What this doesn’t fix
Only one English-language dataset was analysed, no tests against adversarial spam were run, and the CodeCarbon energy figures reflect the single hardware setup used.
Sources
1 on file- Computingdoi:10.1007/s00607-026-01706-0
Talk it over
This record found a gain and a problem in the same place. Which matters more to you, and who might answer differently?
Simple spam filters came close to the top AI model, which emitted 88 times more CO₂, test findshttps://truvace.com/item/beyond-accuracy-a-multi-dimensional-green-ai-framework-for-sustainable-machine-l
This record found a gain and a problem in the same place. Which matters more to you, and who might answer differently?
The gain In a test of 10 models on 5,169 text messages, Logistic Regression and Naive Bayes came close to DistilBERT's accuracy (MCC 0.9219 and 0.9250 against 0.9603) while using far less energy and carbon.
The problem The same test found DistilBERT's edge over Logistic Regression came with 88 times more CO₂ per million predictions, and its training emissions ran about 1,000 times higher than the most efficient models.
Answer in the threadHow do we know? What can this kind of evidence show, and what can’t it?
The evidence This rests on a benchmark or lab test with a sample of 5,169, from Computing.
Answer in the threadWhat does it leave open? What else would you want to know before acting on it?
The record’s own caveat Only one English-language dataset was analysed, no tests against adversarial spam were run, and the CodeCarbon energy figures reflect the single hardware setup used.
Answer in the threadHave you seen AI used this way where you study, work or live? Did it go the same way?
Answer in the thread
The questions are the same on every record. The details under them are taken from this record as written.
What readers say