A randomized controlled trial on evaluating clinician-supervised generative AI for decision support.
International Journal of Medical Informatics · 2024-11-29 · Randomised trial
Abstract
BACKGROUND: The integration of generative artificial intelligence (AI) as clinical decision support systems (CDSS) into telemedicine presents a significant opportunity to enhance clinical outcomes, yet its application remains underexplored. OBJECTIVE: This study investigates the efficacy of one of the most common generative AI tools, ChatGPT, for providing clinical guidance during cardiac arrest scenarios. METHODS: We examined the performance, cognitive load, and trust associated with traditional methods (paper guide), autonomous ChatGPT, and clinician-supervised ChatGPT, where a clinician supervised the AI recommendations. Fifty-four subjects without medical backgrounds participated in randomized controlled trials, each assigned to one of three intervention groups: paper guide, ChatGPT, or supervised ChatGPT. Participants completed a standardized CPR scenario using an Augmented Reality (AR) headset, and performance, physiological, and self-reported metrics were recorded. MAIN FINDINGS: Results indicate that the Supervised-ChatGPT group showed significantly higher decision accuracy compared to the paper guide and ChatGPT groups, although the scenario completion time was longer. Physiological data showed a reduced LF/HF ratio in the Supervised-ChatGPT group, suggesting potentially lower cognitive load. Trust in AI was also highest in the supervised condition. In one instance, ChatGPT suggested a risky option, highlighting the need for clinician supervision. CONCLUSION: Our findings highlight the potential of supervised generative AI to enhance decision-making accuracy and user trust in emergency healthcare settings, despite trade-offs with response time. The study underscores the importance of clinician oversight and the need for further refinement of AI systems to improve safety. Future research should explore strategies to optimize AI supervision and assess the implementation of these systems in real-world clinical settings.
Abstract © the authors or publisher, via PubMed. Reuse and removal
Tags
Bystanders · Decision support tools · Teamwork & human factors · Telemedicine
Related articles
- Tele-Instruction Tool for Multiple Lay Responders Providing Cardiopulmonary Resuscitation in Telehealth Emergency Dispatch Services: Mixed Methods Study. (J Med Internet Res, 2023)
- Video-Assisted Versus Audio-Assisted Dispatcher Cardiopulmonary Resuscitation for Optimizing Compression Quality in Simulated Scenarios: A Network Meta-Analysis of Randomized Controlled Trials. (Prehosp Emerg Care, 2026)
- Design and rationale of the TANGO2 trial: A randomized controlled non-inferiority trial of bystander compression-only CPR versus standard CPR in out-of-hospital cardiac arrest. (Am Heart J, 2026)
- Enhanced community first responder support system: protocol of a randomized trial to improve bystander cardiopulmonary resuscitation quality for out-of-hospital cardiac arrest patients. (Resusc Plus, 2026)
- Drone-Facilitated Real-Time Video-Guided Feedback Helps to Improve the Quality of Lay Bystander Basic Life Support. A Randomized Controlled Simulation Trial. (Prehosp Emerg Care, 2025)
- Feasibility of video-assisted cardiopulmonary resuscitation when the lay responder is alone with the victim: a randomized controlled crossover pilot study. (Sci Rep, 2025)