The need for the human-centred explanation for ML-based clinical decision support systems

TY - GEN

T1 - The need for the human-centred explanation for ML-based clinical decision support systems

AU - Jia, Yan

AU - McDermid, John

AU - Hughes, Nathan

AU - Sujan, Mark

AU - Lawton, Tom

AU - Habli, Ibrahim

N1 - Funding Information: ACKNOWLEDGEMENT This project is funded by Lloyd’s Register Foundation and University of York through Assuring Autonomy International Programme (Project ref 06/22/04) and by the Engineering and Physical Sciences Research Council through the Assuring Responsibility for Trustworthy Autonomous Systems project (EP/W011239/1). Publisher Copyright: © 2023 IEEE.

PY - 2023

Y1 - 2023

N2 - Machine learning has shown great promise in a variety of applications, but the deployment of these systems is hindered by the "opaque" nature of machine learning algorithms. This has led to the development of explainable AI methods, which aim to provide insights into complex algorithms through explanations that are comprehensible to humans. However, many of the explanations currently available are technically focused and reflect what machine learning researchers believe constitutes a good explanation, rather than what users actually want. This paper highlights the need to develop human-centred explanations for machine learning-based clinical decision support systems, as clinicians who typically have limited knowledge of machine learning techniques are the users of these systems. The authors define the requirements for human-centred explanations, then briefly discuss the current state of available explainable AI methods, and finally analyse the gaps between human-centred explanations and current explainable AI methods. A clinical use case is presented to demonstrate the vision for human-centred explanations.

AB - Machine learning has shown great promise in a variety of applications, but the deployment of these systems is hindered by the "opaque" nature of machine learning algorithms. This has led to the development of explainable AI methods, which aim to provide insights into complex algorithms through explanations that are comprehensible to humans. However, many of the explanations currently available are technically focused and reflect what machine learning researchers believe constitutes a good explanation, rather than what users actually want. This paper highlights the need to develop human-centred explanations for machine learning-based clinical decision support systems, as clinicians who typically have limited knowledge of machine learning techniques are the users of these systems. The authors define the requirements for human-centred explanations, then briefly discuss the current state of available explainable AI methods, and finally analyse the gaps between human-centred explanations and current explainable AI methods. A clinical use case is presented to demonstrate the vision for human-centred explanations.

KW - CDSS

KW - Explainable AI

KW - Human-centred XAI

UR - http://www.scopus.com/inward/record.url?scp=85181579288&partnerID=8YFLogxK

U2 - 10.1109/ICHI57859.2023.00064

DO - 10.1109/ICHI57859.2023.00064

M3 - Conference contribution

AN - SCOPUS:85181579288

T3 - Proceedings - 2023 IEEE 11th International Conference on Healthcare Informatics, ICHI 2023

SP - 446

EP - 452

BT - Proceedings - 2023 IEEE 11th International Conference on Healthcare Informatics, ICHI 2023

PB - IEEE

T2 - 11th IEEE International Conference on Healthcare Informatics, ICHI 2023

Y2 - 26 June 2023 through 29 June 2023

ER -