68/100

Moderate evidence

Cite
More
EmailLinkedInX

Study finds AI trained on common pipe records guesses lead pipes no better than chance

Study title: Effectiveness of Different Ground Truths at Detecting Lead Service Pipes Using Machine Learning

Evidence type
Controlled or validated comparison
Sources
1 sourcePeer-reviewed
Published
2 min read

Study title: Effectiveness of Different Ground Truths at Detecting Lead Service Pipes Using Machine Learning

A study in the Journal of Water Resources Planning and Management tested whether machine learning can find lead service pipes using different training labels. Models trained on lead concentration sampling or on asset records did no better than random chance at telling whether a property has a lead pipe, using a large real-world dataset from a UK water company.

Read the study abstract

Replacing all existing lead service pipes (SPs) is an important strategic goal for the water industry because lead SPs represent a clear health risk. For example, the UK water industry has been instructed by the Drinking Water Inspectorate to be lead-free by 2050, and the USEPA has assigned $3 billion to tackle lead. The first key challenge in achieving this goal is determining whether a property’s service pipes are made of lead. Previous research into the application of machine learning (ML) to this problem has…

Effectiveness of Different Ground Truths at Detecting Lead Service Pipes Using Machine Learning
"Laser Optics - Idaho National Laboratory" by Idaho National Laboratory, CC BY 2.0.

The problem

A study using a large real-world UK water company dataset found that machine learning models trained on lead concentration sampling or asset records performed no better than random chance at identifying properties with lead service pipes.

In brief

The finding matters because water companies face deadlines to remove lead pipes: the UK water industry has been told by the Drinking Water Inspectorate to be lead-free by 2050, and the USEPA has assigned $3 billion to tackle lead. Finding which properties have lead pipes is the first hurdle.

The authors say models need training data from excavations and trial holes to work effectively. They argue this could cut the number of false positives that crews have to visit, though the work draws on one company's data.

Main points

  1. The study compared three ways of labelling training data for machine learning models that predict lead service pipes.
  2. Two of the labels were proxies widely used in earlier research: lead concentration sampling and asset records.
  3. The third label came from excavations and trial holes, treated as the real-world ground truth.
  4. Models trained on the two proxies performed no better than random chance on the real-world dataset.
  5. The authors conclude effective models need data collected from excavations and trial holes.
  6. Better models could reduce false positives, meaning fewer properties visited needlessly during replacement work.

The full story

Replacing lead service pipes is a strategic goal for the water industry because lead pipes are a clear health risk. The first challenge is knowing which properties have them. Earlier machine learning work often tackled stand-in problems rather than directly identifying lead pipes.

This paper trained models on lead concentration sampling and on asset records as target classifications, and compared them with a model trained on excavations and trial holes. The data came from a large set held by a UK water company.

The results challenge assumptions about the proxies. Models trained on them performed no better than random chance, so the authors say excavation and trial-hole data must be collected before an effective model can be trained, which should guide future research and make replacement efforts more efficient.

Sources

1 on file
  1. Peer-reviewed2027-01-01

Talk it over

Three questions to start on, two more if you have time
Models trained on lead concentration sampling or asset records did no better than random chance. What would a water company lose by relying on those labels to pick properties to visit?

Start with a question

Study finds AI trained on common pipe records guesses lead pipes no better than chancehttps://truvace.com/item/effectiveness-of-different-ground-truths-at-detecting-lead-service-pipes-using-m

  1. Models trained on lead concentration sampling or asset records did no better than random chance. What would a water company lose by relying on those labels to pick properties to visit?

    Answer in the thread
  2. All the data came from one UK water company. What differences between that company's network and another could change how well these models work?

    Answer in the thread
  3. Excavations and trial holes gave the real-world ground truth, but digging costs money. Who should decide how much digging is worth doing before a model is trusted?

    Answer in the thread
If you have more time
  • The Drinking Water Inspectorate has set a lead-free target of 2050. What would you ask your own water supplier about how it finds lead pipes?

    Answer in the thread
  • Which result from another water company would make you more or less confident that excavation data fixes the problem?

    Answer in the thread

What readers say