aceDevelopment and Pilot Evaluation of a Standardized and Accessible Rubric (OMNI) for Assessing Generative AI in Health Care Contexts: Four-Stage Mixed Methods Study
Background: The rapid adoption of generative AI in the community and health care necessitates reliable, human-led evaluation. Objective: This study aimed to develop a standardized rubric to support health care professionals, administrators, and researchers in evaluating generative AI outputs in health contexts. Methods: A rubric was developed through a 4-stage mixed methods study: literature synthesis, expert refinement, pilot application, and post-pilot feedback. The rubric was informed by 69 publications and r…
When 15 clinicians rated generative AI outputs across four health scenarios, recurring weaknesses appeared in referencing, recognizing uncertainty, completeness, empathy and accuracy, and image and video outputs fared worst.
Further work is needed to assess whether OMNI applies to purpose-built health AI tools, agentic systems and organizational evaluation settings, and rater agreement on the 4-point scales was modest.
Evidence
- Peer-reviewedJournal of Medical Internet Research2026-10-27
How should this claim be treated?
Truvace Impact Record TRV-2026-2290, v1: “Development and Pilot Evaluation of a Standardized and Accessible Rubric (OMNI) for Assessing Generative AI in Health Care Contexts: Four-Stage Mixed Methods Study.” Truvace, 2026-10-27. /record/TRV-2026-2290 (accessed at citation time). sha256 93facf8a57d794b7…
Calibration history
Every change to this record since certification, in the open. None yet — the reading has held since it entered the record.
Certified into the record
How to verify without trusting this page
Fetch the canonical text of any version from /api/record/TRV-2026-2290 and hash it yourself — for example shasum -a 256 on the saved canonical field. The result must equal content_hash, and each version’s text ends with prev:followed by the prior version’s hash (version 1 chains to 64 zeros). If a single character of any version had been altered since certification, the chain would not reproduce.