Abstract
This overview investigates the evolution and current landscape of eXplainable Artificial Intelligence (XAI) in healthcare, highlighting its implications for researchers, technology developers, and policymakers. Following the PRISMA protocol, we analysed 89 publications from January 2000 to June 2024, spanning 19 medical domains, with a focus on Neurology and Cancer as the most studied areas. Various data types are reviewed, including tabular data, medical imaging, and clinical text, offering a comprehensive perspective on XAI applications. Key findings identify significant gaps, such as the limited availability of public datasets, suboptimal data preprocessing techniques, insufficient feature selection and engineering, and the limited utilisation of multiple XAI methods. Additionally, the lack of standardised XAI evaluation metrics and practical obstacles in integrating XAI systems into clinical workflows are emphasised. We provide actionable recommendations, including the design of explainability-centric models, the application of diverse and multiple XAI methods, and the fostering of interdisciplinary collaboration. These strategies aim to guide researchers in building robust AI models, assist technology developers in creating intuitive and user-friendly AI tools, and inform policymakers in establishing effective regulations. Addressing these gaps will promote the development of transparent, reliable, and user-centred AI systems in healthcare, ultimately improving decision-making and patient outcomes.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This publication has emanated from research conducted with the support of the Irish Research Council under award nos. GOIPG/2022/660 & GOIPG/2021/1354 and grant number 18/CRT/6183 from SFI Centre for Research Training in Machine Learning (ML-Labs) at Technological University Dublin with the financial support of Science Foundation Ireland under grant no. 13/RC/2106\_P2 at the ADAPT SFI Research Centre at Technological University Dublin.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
The study summarises various datasets from other research and details information such as variables (features), missing values, availability status, size of the dataset, sampling strategy, and strategy to overcome missing values. It is a desk study and does not identify or engage with any human subject. Details of datasets are described in Tables 8 and 9, along with references to the original research. We did not release any new datasets.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Footnotes
↵5 Student Author.
↵6 Faculty/Supervisor Author.
We have significantly enhanced the manuscript based on reviewer feedback. Specifically: We expanded the literature base, increasing the number of reviewed studies from 68 to 89, including newer publications up to June 2024. We introduced a broader and clearer scope by refining the article's title to reflect both current trends and future directions in Explainable AI (XAI) in healthcare. The structure of the article has been reorganized and clarified, with the addition of a new figure outlining the relationships between sections. To improve readability and data interpretation, we incorporated five new visualisations (pie charts, bar charts, and stacked bar charts) across Sections 3 to 6, each enhancing the explanation of dataset characteristics, predictive models, XAI methods, and multi-method studies. The methodological section has been expanded, providing a clearer account of the PRISMA process and strengthening the reliability of the review by detailing reviewer calibration and consensus mechanisms. We refined and expanded the "Applications and Datasets" and "Predictive Models" sections by providing more granular breakdowns of healthcare domains and models used. The updated article now includes actionable recommendations aimed at researchers, developers, and policymakers to improve the future implementation of XAI systems in clinical practice. These revisions collectively improve the depth, clarity, and visual appeal of the article while directly addressing reviewer concerns about presentation and completeness.
Data Availability
This study did not generate any new data. All data discussed and analyzed are sourced from previously published research studies. The datasets referenced in the study have been fully described in Tables 8 and 9, including information on their variables, missing values, and the methods used in their original collection. The research remains a secondary analysis, utilizing existing datasets without the involvement of new data collection efforts
