0
0
A recent study published in JAMA reveals that while AI models have shown potential in enhancing diagnostic decisions for certain demographics, they can exacerbate decision-making for clinicians, especially when these models have absorbed biased medical data.
Researchers from the University of Michigan emphasized that despite efforts to ensure the safety and reliability of AI models in healthcare, clinicians can be misled by biased AI models, even when provided with explanations regarding the AI’s decision-making process.
Sarah Jabbour, a doctoral candidate in computer science and engineering at the University of Michigan, highlighted the challenge where clinicians must comprehend both the explanation provided by AI models and the explanation itself.
The study specifically focused on AI models and explanations used in cases of acute respiratory failure among patients.
Michael Sjoding, Associate Professor of internal medicine at the University of Michigan’s Medical School, explained the diagnostic difficulty in cases of respiratory failure and the potential for AI models to improve diagnostic accuracy by aiding clinicians in synthesizing patient history, lab tests, and imaging results.
The study evaluated 457 hospitalist physicians, nurse practitioners, and physician assistants to assess diagnostic accuracy with and without assistance from AI models. The clinicians were presented with clinical cases of patients with respiratory failure and asked to make treatment recommendations based on their diagnoses. Some were provided with explanations from the AI model, while others received only the AI’s decision without any explanation.
While the study found that AI models without explanations slightly improved clinicians’ diagnostic accuracy, those presented with explanations witnessed a higher accuracy increase. However, when clinicians were exposed to intentionally biased AI models, their accuracy plummeted significantly, regardless of the provided explanations highlighting the model’s biases.
This study underscores the complexity of integrating AI models into clinical decision-making processes and the critical need for ensuring these models are devoid of biases and provide transparent and understandable explanations to clinicians.
