Moderate evidence
More
Review of 9 studies finds AI chatbots can match experts on rehab plans, but they hallucinate
Study title: The Role of Artificial Intelligence in Treatment Planning in Physical and Rehabilitation Medicine: A Systematic Review
- Evidence type
- Systematic review or meta-analysis
- Sample
- n = 9
- Sources
- 1 sourcePeer-reviewed
- Published
- 2 min read
- Truvace record
- TRV-2026-2436cite or check it
Study title: The Role of Artificial Intelligence in Treatment Planning in Physical and Rehabilitation Medicine: A Systematic Review
A systematic review of nine studies assessed how accurately large language models, the AI chatbots behind tools like ChatGPT, produce rehabilitation treatment plans. Agreement with expert opinion varied across studies and reached 98.75% once an expert advisory board had reviewed the AI-generated statements.
Read the study abstract
Objective Individualized rehabilitation plans are crucial for maximizing clinical outcomes. Generative artificial intelligence (AI), particularly large language models (LLMs), which can generate new content based on large knowledge bases, could improve personalized rehabilitation. This systematic review aimed to assess accuracy and application of LLMs in rehabilitation. Design A systematic search was conducted in PubMed, Web of Science, and Scopus until February 2026. The PICO included LLMs (P), their use in reh…
58Gain side
How the two sides scoreClose call
59Problem side
The gain
A systematic review of nine studies found AI chatbot rehabilitation plans reached 98.75% concordance with expert opinion after an expert advisory board reviewed the AI-generated statements.
The problem
The same review of nine studies found the included studies reported hallucinations from AI chatbots, so their rehabilitation plans needed clinician supervision.
A fixed formula sets both scores from the evidence and its sources. Reader votes don’t move them.
What the scores mean
G is the gain side, out of 100. It goes up when the gain is specific, when it measurably helped people and when the sources behind it are strong.
P is the problem side, out of 100. It goes up when the problem is specific, when it measurably hurt people and when the sources behind it are strong.
In brief
In musculoskeletal and cardiovascular rehabilitation, the chatbots produced plans aligned with clinical guidelines. The included studies also reported hallucinations, meaning the models generated incorrect content, which the authors say makes clinician supervision necessary.
The reviewers searched PubMed, Web of Science and Scopus up to February 2026. They conclude the tools show promise for adaptable, personalised planning, but say higher-quality studies are needed to test real-world effectiveness and define how to implement them.
Main points
- Nine studies met the inclusion criteria after searches of PubMed, Web of Science and Scopus through February 2026.
- Concordance with expert opinion varied, with the highest figure at 98.75% after an expert advisory board reviewed the AI statements.
- In musculoskeletal and cardiovascular rehabilitation, the chatbots showed alignment with clinical guidelines.
- The included studies reported hallucinations, so the authors stress supervision of AI-generated plans.
- The comparison was against expert opinion or other AI systems such as machine learning, deep learning or clinical decision support systems.
- Methodological quality was assessed with JBI tools, and two reviewers extracted data with a third resolving conflicts.
The full story
The review asked whether large language models can generate appropriate rehabilitation plans that comply with expert clinicians or clinical practice guidelines. It compared them with expert opinion or other AI systems, and excluded studies of non-generative AI.
Across nine included studies, agreement with experts varied. The 98.75% figure came after an expert advisory board reviewed the AI-generated statements, not from unreviewed output alone.
Alignment with guidelines was reported in musculoskeletal and cardiovascular rehabilitation. Hallucinations appeared in the included studies, which the authors say makes supervision necessary.
The authors call the capabilities promising and say they may offer adaptability and personalisation. They add that high-quality studies are needed to validate real-world effectiveness and define implementation strategies.
What this doesn’t fix
The review included only nine studies, and the authors say high-quality studies are still needed to validate real-world effectiveness and define implementation strategies.
Talk it overFour questions for a class, a team or the dinner table
This record found a gain and a problem in the same place. Which matters more to you, and who might answer differently?
The gain A systematic review of nine studies found AI chatbot rehabilitation plans reached 98.75% concordance with expert opinion after an expert advisory board reviewed the AI-generated statements.
The problem The same review of nine studies found the included studies reported hallucinations from AI chatbots, so their rehabilitation plans needed clinician supervision.
How do we know? What can this kind of evidence show, and what can’t it?
The evidence This rests on a systematic review or meta-analysis with a sample of 9, from American Journal of Physical Medicine & Rehabilitation.
What does it leave open? What else would you want to know before acting on it?
The record’s own caveat The review included only nine studies, and the authors say high-quality studies are still needed to validate real-world effectiveness and define implementation strategies.
Have you seen AI used this way where you study, work or live? Did it go the same way?
The questions are the same on every record. The details under them are taken from this record as written.
Sources
1 on file- American Journal of Physical Medicine & Rehabilitationdoi:10.1097/phm.0000000000003193
ace
The debate