Is artificial intelligence for medical professionals serving the patients? Protocol for a mixed method systematic review on patient-relevant benefits and harms of algorithmic decision-making

Study 1, Systematic review

1. Are there studies that compare patient-relevant effectiveness of AI-related ADM for medical professionals compared to medical professionals without AI-related ADM?

2. Do these studies show adequate methodological quality and are their findings generalisable?

3. Can AI-related ADM systems help medical professionals to make better decisions in terms of benefits and harms for patients?

Study 2, expert interviews

1. How do experts assess the overall benefit-to-harm ratio of identified AI-related ADM systems?

2. In which medical area of competences (diagnosis, treatment, prognosis, prevention) would patients in Western healthcare systems particularly benefit from AI-related ADM systems?

3. What are the consequences (e.g., side effects) of implementing AI-related ADM systems in healthcare?

4. What are the main challenges that hinder the implementation of AI-related ADM systems in clinical practice and what improvements could be suggested?

5. What are the regulatory requirements for implementation of AI-related ADM systems in clinical practice?

Study 1: Systematic review

We will follow the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (Page et al. 2021) and the Methodological Expectations of Cochrane Intervention Reviews (MECIR) standards (Higgins et al. 2021).

Searches

We will search systematically using English free text terms in title/abstract, Medical Subject Headings (MeSH) terms and Embase Subject Headings (Emtree) fields for various forms of keywords related to ‘artificial intelligence’ and relevant subcategories of computer generated and processed decision-making algorithms, ‘medical professionals’ and keywords describing effectiveness parameters and outcomes as well as preferred study types. Based on the block building approach, keywords and terms are combined using the Boolean operators AND and OR and progressively checked for relevant hits.

Databases to be used for searches

MEDLINE and PubMed (via PubMed), EMBASE (via Elsevier), Institute of Electrical and Electronics Engineers (IEEE) Xplore, will be searched for peer-reviewed articles as well as ClinicalTrials.gov and ICTRP (via CENTRAL) for ongoing trials and protocols.

To reduce potential publication bias, additional studies will be identified by contacting authors of included studies, contacting experts in the field, and through reference lists of relevant studies. Grey literature searches will be conducted in Google Scholar. For this purpose, the keywords used in the systematic search will be used in different combinations, as well as their German equivalents. Google Scholar will be searched up to the 10th hit page. The detailed search strategy for each database will be reported under the PROSPERO record once the searches have been conducted.

Search strategy

We developed our search strategy using the PICOS scheme (Table 1).

While doing preliminary searches for basic literature in Medline and PubMed (via PubMed), we noticed that study conductors from different scientific fields (e.g., computer scientists) used different terms for the intervention outcomes we were looking for. In addition, some studies were not indexed appropriately in PubMed, which complicated our initial search strategy. To carry out the search strategy, we have created and tested the blocks consecutively to gather the best results from each block, expanding and narrowing the search strategy. To assess the right direction of the search strategy, we have used fundamental literature, such as Choudhury & Asan 2020, Park et al. 2020 and Nagendran et al. 2020 as test sets, making sure the results of our search had common ground with these studies.

The resulting search string for Medline and PubMed in the individual blocks can be found in table 2 and describes the basis for other databases.

Types of studies to be included

For the systematic search peer reviewed interventional and observational studies published in German or English, 10 years retrospectively from the date of the search will be considered. For the search of grey literature, scientific reports published in German or English 10 years retrospectively from the date of the search will be considered. To extract potentially relevant studies from (systematic) reviews and meta-analyses, secondary studies will be gathered and screened. However, secondary studies will not be included in the synthesis.

In contrast to studies of effectiveness and safety, pure efficacy studies (e.g., focusing on algorithms accuracy) will be excluded as these outcomes are not directly relevant for patients. Patient-relevant outcomes will be defined according to the IQWiG method paper (2022). In addition, studies that used AI systems beyond our scopes, such as robotics (systems that support the implementation of decisions) will be excluded. Editorials, commentaries, letters, and other informal publication types will be excluded as well.

We will provide a list of all references screened in full text including exclusion reasons in the appendix of the final study.

Participants

Our study is focusing on human patients without restriction of age or sex. Therefore, the input data for the algorithms must include real human data gathered either during routine care and saved for use in research or generated specifically for the individual study.

Intervention

Out study is focusing on medical professionals utilizing an AI-related ADM system to address a clinical problem.

In our working definition, a medical professional is a qualified individual who has the authority to perform necessary medical procedures within their professional scope of practice. Their goal is to improve, maintain, or restore the health of individuals by examining, diagnosing, prognosticating, and/or treating clinical problems. This may include medical doctors, registered nurses, and other medical professionals. Clinical problems can encompass illnesses, injuries, and physical or mental disorders, among other conditions.

In our working definition, an AI-related ADM system is a clinical decision support system that either apply AI (e.g., relying on ML models) or that has been developed with the help of AI. Clinical decision support models without any involvement of AI will be excluded.

Control

Medical professionals, as described in the working definition, are addressing a clinical problem without the support of an AI-related ADM system (standard care).

Outcomes

Patient-relevant benefits and harms, according to the IQWiG method paper (2022), are gathered. These may include, for example, mortality, morbidity (with regard to complaints and complications), length of hospital stay, readmission, time to intervention and health-related quality of life.

Study types

We will collect both interventional and observational studies, which may encompass randomised controlled trials, cohort studies, case-control studies, randomised surveys, retrospective and prospective studies, phase studies, as well as non-inferiority or diagnostic studies.

Data extraction

Records arising from the literature search will be stored in the citation manager Citavi 6 (c) Swiss Academic Software. After removing duplicates, two reviewers will independently review all titles and abstracts via the browser application Rayyan (Ouzzani et al. 2016). Studies potentially meeting the inclusion criteria will then be screened in full text independently by two reviewers. Disagreements over eligibility of studies will be discussed and, if necessary, resolved by a third reviewer. Authors of the included studies will be contacted if clarification of their data or study methods is required. The PRISMA 2020 flow diagram (Page et al. 2021) will be used to keep the study selection process transparent.

Using a standardised data collection form, two reviewers will extract data independently from the included studies and will compare them for discrepancies. Missing data will be requested from study authors. Extracted data will include the following (Table 3).

Risk of bias and quality assessment

Risk of bias will be assessed by using the revised Cochrane risk-of-bias tool for randomized trials (RoB 2) (Sterne et al. 2019) and the risk-of-bias in non-randomized studies for interventions (ROBINS-I) tool (Sterne et al. 2017). Disagreements between the authors over the risk of bias in the included studies will be resolved by discussion, or with involvement of a third author if necessary. Transparent reporting of the included studies will be assessed trough the Consolidated Standards of Reporting Trials interventions involving Artificial Intelligence (CONSORT-AI) extension by Liu et al. (2020). The CONSORT-AI extension includes 14 new items that were considered sufficiently important for AI interventions to be routinely reported in addition to the core CONSORT items by Schulz et al. (2010). CONSORT-AI aims to improve the transparency and completeness in reporting clinical trials for AI interventions. It will assist to understand, interpret, and critically appraise the quality of clinical trial design and risk of bias in the reported outcomes.

Data synthesis

Given the expected likelihood of heterogeneity between studies in the different medical specialties in terms of outcome measures, study designs and interventions, we do not know if performing a meta-analysis will be possible. However, a systematic narrative synthesis will be provided of the results with an overview of the relevant effects for the outcomes, with information presented in the text and tables to summarise and explain the characteristics and findings of the included studies. The measures will be presented in absolute risks and risk differences. Studies of unclear or high risk of bias will not be excluded, as the authors assume that most potentially relevant studies will be of low methodological quality. This is also reported in recent reviews that examine the methodological quality of machine learning systems in the clinical setting (e.g., Nagendran et al. 2020). Nevertheless, the influence of the potential biases on the results will be discussed.

In addition to the synthesis trough Study 1, the results of this review will be discussed with experts (Study 2) and presented in the text.

Study 2: Expert interviews

In order to meet the study objectives, and considering the findings of Study 1, we elicit experts’ experiences, knowledge, and their perceived benefits and risks of implementing AI-related ADM systems. By doing so, we hope to gain a practice-related understanding of the findings of Study 1 and identify barriers and needs regarding future research questions.

Based on the research questions and the findings of Study 1, the study and interview guide will be developed a priori. The study design and the interview guide will be submitted to the ethics committee of the University of Potsdam and pre-registered within the PROSPERO protocol for the systematic review. If necessary, post-registration changes to the protocol will be detailed and justified in the PROSPERO protocol. To ensure transparency of the qualitative research part of our review, we will follow the Standards for Reporting Qualitative Research (SRQR) (O’Brien et al. 2014).

Study design

We will conduct semi-structured interviews. To analyse the interviews, we will use qualitative content analysis according to Mayring (2020). This systematic and rule-based approach will ensure that the analysis is understandable, comprehensible, and verifiable by others.

Data collection

As our interview study aims to understand the practical implications of the findings of Study 1, we will use a purposive sampling strategy to select at least 3 interviewees according to their expertise through their function, profession, practice, and research in relation to the research questions (Mey & Mruck 2020: 322, Schreier 2020: 24). Therefore, we will combine experts of the following types: Auditing bodies responsible for auditing AI systems for safety (e.g., TÜV NORD IT Secure Communications GmbH & Co. KG), auditing bodies that support implementation procedures under scientific and regulatory conditions in the healthcare sector (e.g., VDI/VDE Innovation + Technik GmbH), and institutions that combine practical healthcare-related work and research with AI systems (e.g., Berlin Institute of Health at Charité Berlin). The experts will be recruited on the basis of publications, via professional networks, and by sending inquiries to the relevant organisations and institutions. Once relevant experts will have been identified, they will be contacted via e-mail or phone, informing about the specific research questions, and showing some exemplary findings from Study 1, to ask for participation and to set the scope of the interview. There is no reimbursement planned for the experts who participate in the interview.

Written informed consent will be obtained prior to each interview and participants will be fully informed in writing and verbally about the purpose, risks, and scope of the study.

Approximately one week prior to the interview, participants will be presented with the key findings of Study 1 and the interview guide, which they will be asked about in the interview. In addition to the questions on research interest, descriptive data will be collected on field of activity, specific expertise, organisation, length of involvement in the topic, as well as age and gender of the respective interviewee. Interviews should not exceed 45 minutes. All interviews will be audio recorded and automatically transcribed verbatim using Adobe Premiere Pro 2023 (version 23.1.0). The automated transcripts will be corrected manually by the interviewer.

Throughout the data collection process, the interviewer will keep a research diary to record initial impressions immediately after each interview and to encourage reflexivity about how their role in the researcher-participant relationship might influence the data collection process (Mayring 2020: 12).

Data analysis and extraction

The transcripts and notes from the research diary will be transferred to the relevant memos of each transcript in MAXQDA Plus 2022 (version 22.2.0). All data analysis will be carried out by two independent researchers to ensure rigour and trustworthiness. In the event of disagreement between the two researchers on coding, we will revisit the data to develop a shared understanding of the consistent interpretations. The interviews will be analysed using qualitative content analysis according to Mayring (2020). The main categories will initially be developed deductively from the interview guide, which will be developed from the findings of Study 1. Additional major categories will be developed inductively from the data. The final code system, including illustrative interview quotes, will be part of the scientific publication. For this purpose, extracts from the transcript will be translated into English after coding, if necessary.

Data management, data storage and data privacy

After being contacted by the interviewer via phone or e-mail, the experts will receive all relevant study documents via mail or e-mail. Verbal consent to receive the study documents will be obtained by phone or e-mail, and written consent will be obtained in the consent form before the interview takes place. All participants will be assured of confidentiality, anonymity of data, and the right to withdraw from the study or skip questions without negative consequences. Individual contact details will be stored on the interviewer’s password-protected computer until the time of the interview. Contact details will be deleted immediately after the interview. The audio recording is used only for quality assurance and will be deleted immediately after it is transcribed and compared. Transcripts will be pseudonymised by changing personal characteristics (such as names and addresses) and personal data, i.e., data that make indirect identification possible (e.g., places or institutions) to pseudonyms.