Risk-based assessment of AI in medicine
A recent article by Prof. Dr. Martin Haimerl and Prof. Dr. Christoph Reich of Furtwangen University shows that machine learning (ML) in medicine is often evaluated without a comprehensive risk assessment. The authors investigated the extent to which current scientific papers include risk-based metrics in the evaluation of AI models for medical devices.
The result: most publications use classic metrics such as accuracy or sensitivity without sufficiently considering the clinical risks of misclassification.
Lack of risk assessment - a critical problem
Haimerl and Reich emphasize that different types of errors in the use of ML models − in particular false-negative (overlooked diseases) and false-positive (misdiagnosis) results − can have serious and sometimes very different consequences. While a false-negative result can lead to omitted treatment, a false-positive result may cause unnecessary interventions. A risk-based assessment could help to improve the safety and effectiveness of AI-supported diagnostic models.
Regulatory requirements and necessary adjustments
The authors refer to the EU Medical Device Regulation (MDR) and the risk management standard for medical devices ISO 14971, which prescribe a systematic risk assessment for medical devices. Nevertheless, many ML models do not take these regulatory requirements sufficiently into account. The authors show that a stronger focus on risk-based performance metrics could significantly improve patient safety and this approach should be considered an essential building block for the approval of AI-based medical devices.
The article was written in collaboration with the KISS Project (Artificial Intelligence Services and Systems) at Furtwangen University and was funded by the Federal Ministry of Education and Research (MBMF) and the Ministry of Science, Research and the Arts of the State of Baden-Württemberg.
The article was published in BMC Medical Informatics and Decision Making, a Springer Nature journal.
