acePerformance of AI Chatbots in Adult Reconstruction Education: A Comparative Evaluation Using AAOS ResStudy Questions
Background: Artificial intelligence (AI) chatbots are increasingly incorporated into orthopedic education, yet their ability to handle complex, clinically oriented questions remains unclear. This study evaluates the accuracy, response time, and citation reliability of contemporary AI chatbots when answering adult reconstruction questions from the AAOS ResStudy question bank. Methods: Fifty-five multiple-choice adult reconstruction questions were extracted and categorized by anatomic region (hip vs. knee), concep…
GPT-4o answered 80% of adult reconstruction questions correctly, above the 75% score of AAOS members, and Co-Pilot's citations were all real and relevant.
Overall chatbot accuracy was only 67.5%, falling to 45.4% on difficult questions and 63.1% on prognosis, so chatbots struggle with applied reasoning.
The study is a level IV cross-sectional comparison using only 55 questions from one question bank, so it is a limited test of chatbot performance.
Evidence
- Peer-reviewedRhode Island medical journal (2013)2026-11-02
How should this claim be treated?
Truvace Impact Record TRV-2026-2294, v1: “Performance of AI Chatbots in Adult Reconstruction Education: A Comparative Evaluation Using AAOS ResStudy Questions.” Truvace, 2026-10-28. /record/TRV-2026-2294 (accessed at citation time). sha256 59c26e12bc4f289d…
Calibration history
Every change to this record since certification, in the open. None yet — the reading has held since it entered the record.
Certified into the record
How to verify without trusting this page
Fetch the canonical text of any version from /api/record/TRV-2026-2294 and hash it yourself — for example shasum -a 256 on the saved canonical field. The result must equal content_hash, and each version’s text ends with prev:followed by the prior version’s hash (version 1 chains to 64 zeros). If a single character of any version had been altered since certification, the chain would not reproduce.