Article Text

Download PDFPDF

Strategies to evaluate healthcare provider trainings in shared decision-making (SDM): a systematic review of evaluation studies
  1. Evamaria Müller1,
  2. Alena Strukava1,
  3. Isabelle Scholl1,
  4. Martin Härter1,
  5. Ndeye Thiab Diouf2,
  6. France Légaré2,
  7. Angela Buchholz1
  1. 1 Department of Medical Psychology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
  2. 2 Department of Family Medicine and Emergency Medicine, Laval University, Quebec, Canada
  1. Correspondence to Evamaria Müller; e.mueller{at}uke.de

Abstract

Design and objectives We performed a systematic review of studies evaluating healthcare provider (HCP) trainings in shared decision-making (SDM) to analyse their evaluation strategies.

Setting and participants HCP trainings in SDM from all healthcare settings.

Methods We searched scientific databases (Medline, PsycInfo, CINAHL), performed reference and citation tracking, contacted experts in the field and scanned the Canadian inventory of SDM training programmes for healthcare professionals. We included articles reporting data of summative evaluations of HCP trainings in SDM. Two reviewers screened records, assessed full-text articles, performed data extraction and assessed study quality with the integrated quality criteria for review of multiple study designs (ICROMS) tool. Analysis of evaluation strategies included data source use, use of unpublished or published measures and coverage of Kirkpatrick’s evaluation levels. An evaluation framework based on Kirkpatrick’s evaluation levels and the Quadruple Aim framework was used to categorise identified evaluation outcomes.

Results Out of 7234 records, we included 41 articles reporting on 30 studies: cluster-randomised (n=8) and randomised (n=9) controlled trials, controlled (n=1) and non-controlled (n=7) before-after studies, mixed-methods (n=1), qualitative (n=1) and post-test (n=3) studies. Most studies were conducted in the USA (n=9), Germany (n=8) or Canada (n=7) and evaluated physician trainings (n=25). Eleven articles met ICROMS quality criteria. Almost all studies (n=27) employed HCP-reported outcomes for training evaluation and most (n=19) additionally used patient-reported (n=12), observer-rated (n=10), standardised patient-reported (n=2) outcomes or training process and healthcare data (n=10). Most studies employed a mix of unpublished and published measures (n=17) and covered two (n=12) or three (n=10) Kirkpatrick’s levels. Identified evaluation outcomes covered all categories of the proposed framework.

Conclusions Strategies to evaluate HCP trainings in SDM varied largely. The proposed evaluation framework maybe useful to structure future evaluation studies, but international agreement on a core set of outcomes is needed to improve evidence.

PROSPERO registration number CRD42016041623.

  • decision making
  • review [publication type]
  • education [subheading]
  • professional-patient relations
  • outcome assessment (health care)

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • A strength of this study is the fact that we sought all types of evaluation strategies for healthcare provider trainings in shared decision-making and included all types of study designs from post-test studies to qualitative and cluster-randomised controlled studies.

  • A strength of this study is the fact that we developed an evaluation framework, which may be a first step towards agreement on a core set of evaluation outcomes and measures to improve evidence on healthcare provider trainings in shared decision-making.

  • A limitation of this study is the fact that we did not analyse which measures are useful to evaluate healthcare provider trainings in shared decision-making.

  • A limitation of this study is the fact that we focused on the analysis of evaluation outcomes, but did not analyse evaluation strategies with regard to appropriate measurement points or the match between training contents and use of evaluation outcomes.

  • A limitation of the proposed evaluation framework is that it focuses on evaluation outcomes, but does not take into account aspects like appropriate study designs.

Introduction

Healthcare policies, clinical guidelines and a growing body of research strongly advocate for the implementation of shared decision-making (SDM) as a central element of patient-centred care.1 Policy makers are interested in SDM, because it tackles overuse, underuse and misuse of healthcare interventions all at the same time.2 In SDM, the patient and at least one clinician share information and values, deliberate the next step and arrive at a jointly made decision.3 Patients who experienced SDM reported less decisional conflict and improved satisfaction,4 but evidence regarding health-related outcomes is limited.4–6 To date, the most conclusive argument for SDM is ethical. Patients have the right to learn about available treatment options and their implications, and to participate in decision-making regarding their health.4 7 Despite multiple implementation initiatives8 and widespread support, SDM is not yet implemented in routine care.7 9

Interventions to foster the implementation of SDM usually target healthcare providers (HCPs), patients or both.10 They may include the distribution of written educational material or patient decision aids, patient coaching, audit and feedback for HCPs or HCP trainings in SDM.11 HCP trainings in SDM are group or online courses that address HCP SDM attitudes, knowledge or skills. They include the use of lectures, case studies, role play, group discussion or didactic materials.12 HCP trainings in SDM are considered key to implement SDM in healthcare, but it is unclear what kind of trainings are most effective and which outcomes they affect.10–13 The lack of consensus on an evaluation framework for HCP trainings in SDM partly accounts for this lack of evidence.14

Evaluation frameworks support practitioners and researchers in the design of coherent evaluation strategies.15 Kirkpatrick’s four-level training evaluation model16 is the most established and feasible model for training evaluation and can be applied to the context of HCP professional development.17 Kirkpatrick’s four levels are: 1) reaction, 2) learning, 3) behaviour and 4) results. The reaction level includes participant reactions to the training and can be assessed with attendance levels or subjective training impressions. The learning level covers participant changes in attitudes, knowledge or skills after the training. The behaviour level covers changes in participant behaviours or transference of training content to the workplace. The results level describes more tangible trainings results, for example, system effects or patient health outcomes.4 17 18

Elwyn et al 9 argue that SDM research has neglected investigation of diverse long-term consequences on the results level. They postulate that widespread implementation of SDM leads to safer and more cost-effective decisions, to reduce utilisation rates and to improve patient health outcomes, but evidence is lacking.9 The influential Quadruple Aim framework aims to improve the experience of care, the health of populations, the per capita cost of healthcare and the work life of HCPs,19 and may be useful to structure evaluation of HCP trainings in SDM on the results level.

In this review, we aimed to analyse how the diversity of evaluation strategies and the quality of published evaluations contributes to the current lack of evidence on HCP trainings in SDM. Thus, we aimed to investigate the quality of published evaluations of HCP trainings in SDM, and to analyse their evaluation strategies. We aimed to analyse evaluation strategies regarding 1) use of data sources, 2) use of unpublished or self-developed and published or psychometrically tested measures and 3) coverage of Kirkpatrick’s four levels. We aimed to categorise identified outcomes in an evaluation framework for HCP trainings in SDM based on Kirkpatrick’s four-level evaluation model16 and the Quadruple Aim framework19 to guide future research and to initiate discussion about a core set of evaluation outcomes for this purpose.

Methods

Registration and search strategy

This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines for systematic reviews20 in most parts (see online supplementary file S1). We made the following changes to the protocol: we adapted the PICOS (P: patient, problem or population, I: intervention, C: comparison, control or comparator, O: outcomes, S: study type) criteria to meet our research purpose, we did not remove duplicates in the secondary search, we did not assess risk of bias across studies and we did not use any summary measures or additional analyses as we limited our work to qualitative synthesis only. We performed an electronic database search employing Medline, CINAHL and PsycInfo databases (via OVID) on 26 June 2016. For this purpose, we developed a detailed search strategy for each database. We adapted the PICOS criteria20 and considered a combination of the following aspects appropriate: population AND intervention AND construct AND outcome OR study design. Terms and keywords were adapted for each database and searches in Medline and PsycInfo were limited to publications concerning humans. We updated the electronic database search on 30 January 2019. Full insight in the electronic database search strategy is attainable in online supplementary file S2. Moreover, we performed a secondary search including reference and citation tracking of included full-text articles, consultation of experts in the field of research via a shared decision-making facebook group and a screening of the Canadian inventory of SDM training programmes for healthcare professionals (http://www.decision.chaire.fmed.ulaval.ca/en/list-of-sdm-programs). Additionally, we screened references of two reviews on SDM interventions for HCPs.11 12 We registered details of the protocol for this systematic review on PROSPERO website accessible via www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42016041623.

Supplementary file 1

Supplementary file 2

Article selection

We aimed to include articles reporting on summative evaluations (outcome or study design) of HCP (population) trainings (intervention) in SDM (construct) and developed inclusion and exclusion criteria, accordingly. We aimed to exclude articles reporting on formative evaluations or interventions that do not have the main aim to teach SDM skills to HCPs (Table 1). Following the first database search (26 June 2016), two reviewers independently screened titles and abstracts of a random sample of 300 (>5%) records identified in the electronic database to ensure sufficient inter-rater reliability. We discussed any differences until we reached consensus. Records identified in the electronic database search were then split in half to be assessed for possible inclusion in the study by one of two reviewers. Following the update of the database search (30January 2019), two reviewers independently screened all identified records and discussed any differences until consensus was reached. Two reviewers independently assessed full-text articles for eligibility by applying inclusion and exclusion criteria (box 1). We resolved differences by discussion until we reached consensus. If consensus could not be reached, the final decision was made by discussion with two other reviewers.

Table 1

Descriptive data of included study articles

Box 1

Inclusion and exclusion criteria

Inclusion criteria*

  1. Article reports data on a health care provider training in SDM

  2. Article reports data on summative evaluation of a SDM training

  3. Article has the aim to evaluate a SDM training

Exclusion criteria

  1. Article is a study protocol or clinical trial report

  2. Article reports on a study in which participants were medical or health care students

  3. Article only reports on formative evaluation of a SDM training

  4. Article reports on an intervention that is limited to or has the main aim to teach the use of a decision-making tool

  5. Article reports on an intervention that does not have the main aim to teach SDM communication skills

  6. Article reports on a complex intervention, in which health care provider training is only a component and no separate summative evaluation data on the training are reported

  7. Article is written in a language other than English, German, French or Dutch (languages spoken by teams members)

  8. Full text of the article is not available

  • *Articles were excluded if inclusion criteria were not met.

Data extraction, quality assessment and analysis of evaluation strategies

We used data extraction sheets to collect descriptive data of included articles, for example, country of origin of the study, study design, characteristics of HCP and patient samples. Furthermore, we extracted data on evaluation outcomes reported in included articles and all data relevant to assess study quality of included articles. Data extraction sheets were pilot-tested and adjusted accordingly. We assessed study quality of included articles with the integrated quality criteria for review of multiple study designs (ICROMS) tool.21 Two reviewers independently performed data extraction and quality evaluation and discussed any differences until consensus was reached. One reviewer performed analysis of evaluation strategies in discussion with the team. As study results are repeatedly published in more than one article, we will present results on two levels: study and article, if applicable.

Quality assessment with the ICROMS tool

The ICROMS tool appraises the quality of multiple study designs and stems from an iterative process over 2 years that included review of existing quality criteria, pilot testing and expert consensus. It aims to establish criteria critically appraising the quality of multiple study designs, in order to broaden the database for systematic reviews and to inspire rigorous research.21 The ICROMS tool comprises 7 dimensions and defines 33 specific criteria for these dimensions applicable only to some study designs. ICROMS dimensions are 1) clear aims and justification, 2) managing bias in sampling between groups, 3) managing bias in outcome measurements and blinding, 4) managing bias in follow-up, 5) managing bias in other study aspects, 6) analytical rigour, 7) managing bias in reporting/ethical considerations. The ICROMS tool is applicable for cluster-randomised and randomised controlled trials, controlled and non-controlled before-after studies, controlled and non-controlled interrupted times series, cohort studies and qualitative studies. As the ICROMS tool is not applicable to post-test studies, we did not assess study quality for articles reporting on this study type. ICROMS-specific criteria are answerable with yes (2 points), no (0 points) or unclear (1 point). The ICROMS tool defines mandatory criteria and minimum scores for different study types to distinguish if studies are fit for inclusion in a systematic review. Minimum scores vary per study type and range from 16 for qualitative studies over 18 for controlled before-after studies to 22 for non-controlled before-after studies or cluster-randomised and randomised controlled trials. Detailed information on the ICROMS tool is attainable in the original publication.21 We analysed quality assessment results on article level.

Analysis of evaluation strategies

One reviewer analysed evaluation strategies regarding use of data sources (HCPs, patients, standardised patients, observers, training process and healthcare data), use of unpublished or self-developed and published or psychometrically tested measures and coverage of Kirkpatrick’s four levels of reaction, learning, behaviour and results.16 One reviewer categorised identified evaluation outcomes in the proposed evaluation framework for HCP trainings in SDM (Figure 1) that is based on the Kirkpatrick’s four-level evaluation model16 and the Quadruple Aim framework.19 One reviewer developed comprehensive subcategories of evaluation outcomes based on the measures identified in the review and categorised evaluation outcomes accordingly. The study team supervised this process and provided feedback in team discussions. As study results are repeatedly published in more than one article, we will present results on two levels: study and article, if applicable.

Figure 1

Evaluation framework for healthcare provider trainings in shared decision-making (SDM).

Patient and public involvement

We did not involve patients in the conduction of this study.

Results

Literature search and article selection

The electronic database search on 26 June 2016 identified 5317 records. After removal of duplicates, 4543 records remained. We found an additional number of 1636 records through the secondary search. The electronic database search on 30 January 2019 identified additional 1222 records. After removal of duplicates, 1055 records remained. We finally screened 7234 records, of which some are likely to be unidentified duplicates due to our complex search strategy. We excluded 7137 records based on title and abstract screening and assessed 97 full-text articles for eligibility. Of the remaining 97 full-text articles, we excluded 56 full-text articles by applying inclusion and exclusion criteria (Table 1). We excluded the majority of full-text articles because they did not meet the first inclusion criterion and did not report data on an SDM training for HCPs. We included 41 articles in this review. Figure 2 shows the process of article selection.

Figure 2

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow chart. *Due to multiple search strategies, duplicates were not removed.

Descriptive data of included studies and articles

Identified articles (n=41) report on studies (n=30) conducted in a limited number of countries (n=10). Most studies were conducted in the USA (n=9), Germany (n=8) and Canada (n=7). Eleven articles report on studies from Germany,22–32 nine articles report on studies from Canada33–41 and eight articles report on studies from the USA.42–49 Six articles depict one study from the UK50–53 and four articles present studies conducted either multinationally,54 in Australia,55 56 the Netherlands57 58 or Korea.59 The majority of included articles (n=27) report on cluster-randomised25–27 29–31 33–36 44 48 50 52 57 58 and randomised22 23 30 31 37 38 42 45 46 50 54 55 controlled trials. Further articles report on one controlled59 and seven non-controlled24 32 39 41 43 56 60 before-after studies, three qualitative40 51 53 and three post-test28 47 49 studies. Most articles (n=34) report on the evaluation of physician trainings,22–25 28–33 35 36 39–46 48–58 60 two articles report on trainings for nurses38 59 and five articles report on  trainings for diverse HCPs.26 27 34 37 47 Overall, identified articles report on HCP samples ranging from 6 to 306, and n=25 articles22 23 25 27 29–36 40 42 44 48 50–52 54 55 57 58 report on the use of patient samples ranging from 38 to 1120. Table 1 illustrates descriptive data of included studies and articles.

Quality results of the ICROMS tool

Assessment of the quality of included articles with the ICROMS tool was applicable to 38 of the included articles (Table 2). Three articles were post-test studies,28 47 49 which could not be assessed with the ICROMS tool. Of the 22 articles that met the minimum score, 7 reported on randomised controlled trials,22 30 31 37 38 42 55 12 on cluster-randomised controlled trials25 29 33 35 36 44 48 50 52 57 58 and 3 reported on qualitative studies.40 51 53 Looking in detail at the 16 articles23 24 26 27 32 34 39 41 43 45 46 50 54 56 59 60 that did not meet the minimum score, most of them failed to meet criterion 3E (blinded assessment of primary outcome), 3F (reliable primary outcome measures) and 7D (free of other bias). For detailed results regarding ICROMS criteria, see online supplementary file S3.

Supplementary file 3

Table 2

Quality results of the ICROMS tool

Most of the included studies (n=30) and articles (n=41) report use of more than one type of data source to evaluate training effects. Of the studies employing HCP-reported data (n=27), eight studies24 28 45–47 49 56 59 relied solely on HCPs for training evaluation. The remaining 19 studies additionally employed other types of outcomes, for example, patient-reported,22 23 27 31–36 40 44 48 51 54 55 observer-rated,31 32 37–40 42 43 52 60 standardised patient-reported outcomes44 48 or training process and healthcare data.32 33 36–39 41 44 60 The three studies not relying on HCPs as data source25 29 30 57 58 combined patient-reported data with observer-rated measures25 57 or training process and healthcare data.29 57 Table 3 presents an overview of the data sources used for training evaluation in identified studies and articles.

Table 3

Analysis of data source use

All but one study40 employed quantitative evaluation strategies. Of the 29 remaining studies, articles of 17 studies22 23 26 27 29 32–39 44–46 48 52 55 57–59 reported use of a combination of unpublished or self-developed and published or psychometrically tested measures. Nine studies24 28 41–43 47 49 56 60 employed only unpublished or self-developed measures and three studies25 30 31 54 applied only published or psychometrically tested outcomes.

Looking at studies’ evaluation strategies with regard to coverage of Kirkpatrick’s four levels, most studies covered two (n=12) or three (n=10) levels. One study54 measured only on the results level and seven studies covered all four evaluation levels (table 4). All but one32 of the studies measuring on the reaction level also used outcomes on the learning level.24 26 28 33 37–39 41 45–48 53 55 56 60 Articles of five studies33 36 49 54 55 59 indicate measurement on the results level without covering the evaluation level of behaviour.

Table 4

Coverage of Kirkpatrick’s evaluation levels

Evaluation outcomes identified in the articles of this review were categorised in comprehensive subcategories of the proposed evaluation framework (table 5). The first level of HCPs' reactions to the training includes provider-reported appraisal of the training and objective acceptability and feasibility data. The second level of HCPs’ learning includes provider-reported learning like subjective knowledge gain, attitudes and intentions to engage in SDM, confidence in SDM communication skills and medical competencies. Objective learning measures are knowledge tests like multiple-choice or open questions on contents covered in the training. The third level of HCPs’ behaviour includes provider-reported or (standardised) patient-reported and observer-rated SDM performance and assessment of the patient-provider interaction. The fourth-level HCP training in SDM results reflects the Quadruple Aim framework including the work life of HCPs, patient population health, patient experience of care and healthcare system costs. Table 5 presents the subcategories of evaluation outcomes and how frequently they were used in included articles. Detailed information on the outcomes used in respective articles can be found in the online supplementary file S4.

Supplementary file 4

Table 5

Categories of evaluation outcomes integrated in the evaluation framework

Discussion

Our review aimed to investigate how the diversity of evaluation strategies and the quality of published evaluations contributes to the current lack of evidence on HCP trainings in SDM. Thus, we analysed the quality of published articles on HCP trainings in SDM, and analysed their evaluation strategies regarding 1) use of data sources, 2) use of unpublished or self-developed and published or psychometrically tested measures and 3) coverage of Kirkpatrick’s four levels. We found 41 articles reporting on 30 studies that met our inclusion criteria. Most of these studies were cluster-randomised and randomised controlled trials that evaluated SDM trainings for physicians and were conducted in high-income countries like Canada, the USA, the UK or Germany. Sample sizes varied largely. Of the 38 articles eligible for assessment with the ICROMS tool, only 11 articles met ICROMS quality criteria. Diverse strategies were used to evaluate HCP trainings in SDM, but most studies relied on provider-reported outcomes, covered two or three of Kirkpatrick’s levels and combined published and unpublished measures. The proposed evaluation framework based on Kirkpatrick’s four-level evaluation model16 and the Quadruple Aim framework19 appears useful for the design or analysis of strategies to evaluate HCP trainings in SDM.

The poor quality of identified publications indicates that researchers should aim to design more methodologically sound studies to evaluate HCP trainings in SDM. The ICROMS tool is a decision matrix to evaluate the robustness of studies for inclusion in a review21 and present results could inspire researchers to be more rigorous in their study. Since measurement bias was a common problem of many included studies, it would be good to use more objective training acceptability and feasibility data, more objective learning and observer-rated measures and healthcare data for evaluation. However, assessment of specific learning objectives may require application of self-developed measures. Combined with psychometrically sound primary outcomes, this may be the ideal evaluation approach.

Although HCPs were the main data source in included studies, reaction to the training was the least studied evaluation level. Training participants’ favourable reactions are substantial for the training to be effective as participants’ positive appraisal determines their motivations to learn from the training.16 Following the reaction level, researchers should assess HCP learning using objective learning measures for knowledge gain. Provider-reported learning measures are useful to establish training effects on HCP attitudes, intentions and confidence regarding SDM-related behaviour, which are the predecessors of actual behaviour.17 According to the theory of planned behaviour, a positive attitude, acquirement of relevant knowledge and improvement of skills determine HCP behavioural intentions, and thus behaviour change.17

Measurement of behaviour change is central, as change of SDM-related behaviours is usually the main aim of HCP trainings in SDM. Since there is no gold standard for measuring SDM61 and measurement from different viewpoints is inconsistent,62 multiperspective assessment from the viewpoints of HCPs (standardised), patients and observers appears the best approach.5 Ideally, validated measures should be used to ensure quality and comparability of results, but a lack of psychometrically tested SDM measures61 poses a problem. It is also difficult to assess behaviour change in clinical practice, because it is unclear when changes manifest themselves.16 However, it is critical to establish behaviour change, before measuring training effects on the results level4 16 to avoid the risk of interpreting random effects independent from the training.

To establish training effects on the results level relevant to multiple stakeholders,9 we recommend reference to the Quadruple Aim framework.19 Beneficial training effects on the work life of HCPs may increase their motivation to implement SDM in practice. Currently, HCPs often experience SDM as another burden and demand on their time, and are therefore often reluctant to implement SDM in routine practice.63 Although effects of SDM on affective-cognitive aspects of patient experience of care are well established, evidence regarding patient population health is sparse.4 If studies showed beneficial SDM training effects on healthcare system costs, policy makers could be encouraged to initiate system changes to foster the implementation of SDM.9

In sum, the poor study quality and the multitude of evaluation strategies used in identified studies limit conclusive evidence on HCP trainings in SDM. The heterogeneous use of SDM and other outcome measures compromises the interpretation and integration of research results.4 10 11 64 65 To achieve solid empirical evidence, we need consensus on a core set of evaluation outcomes and validated measures on all levels of the proposed framework for HCP trainings in SDM. In the design of evaluation studies, researchers should aim to cover all four levels of the framework and include outcomes on the results level that relate to the Quadruple Aim framework. Researchers should aim to use outcomes that are valued by multiple stakeholders like patients, HCPs as well as healthcare managers, executives and policy makers. They should also aim to use validated observer-rated measures and objective data to limit bias, whenever feasible. If researchers applied these recommendations, evaluation studies could have more impact and better support the implementation of SDM in routine practice.9 65

This review has some limitations. First, our primary search included only three databases and inclusion criteria were limited to studies aiming to evaluate HCP trainings in SDM. Consequently, we may have missed some studies, but we assume that our broad secondary search strategy made up for this limitation. Second, we did not analyse evaluation strategies regarding a match of training contents and evaluation outcomes. Additionally, we did not analyse which evaluation outcomes previously showed SDM training effects, which could be valuable information for the design of an evaluation study. However, previous studies investigated the relation between SDM and patient outcomes4 5 65 and interested researchers may obtain valuable information there. Third, our quality assessment with the ICROMS tool can be seen as a limitation as well as a strength of this review. On the one hand, the ICROMS tool is not applicable to post-test studies and considers patient-reported and provider-reported outcomes as unreliable, which introduces a negative bias to our quality results. On the other hand, we provided an overview of the quality of studies in the field, demonstrating a lack of robust evaluation studies. This review has further strengths. First, this review comprises multiple study designs from post-test studies to qualitative and cluster-randomised controlled studies, which reflect the diversity of studies in the field. Second, this review provides an analysis of current strategies to evaluate HCP trainings in SDM and how their diversity functions as a barrier to conclusive evidence. Third, this review proposes an evaluation framework for HCP trainings in SDM that is based on the well-established Kirkpatrick’s evaluation model and the Quadruple Aim framework. The framework may provide guidance in the design of coherence evaluation strategies for HCP trainings in SDM. Fourth, the proposed framework may initiate discussion and hopefully agreement on a core set of validated outcome measures useful for the purpose and meaningful to stakeholders.

Acknowledgments

The authors would like to thank Janka Nölle (JN) for performing title and abstract screening, data extraction and quality assessments for the update of the review. The authors would like to thank Professor Dr Sigrid Harendza and Professor Dr Corinna Bergelt for their expert advice on the conduction of this study. The authors would also like to thank their student assistant Alice Diesing.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.
  38. 38.
  39. 39.
  40. 40.
  41. 41.
  42. 42.
  43. 43.
  44. 44.
  45. 45.
  46. 46.
  47. 47.
  48. 48.
  49. 49.
  50. 50.
  51. 51.
  52. 52.
  53. 53.
  54. 54.
  55. 55.
  56. 56.
  57. 57.
  58. 58.
  59. 59.
  60. 60.
  61. 61.
  62. 62.
  63. 63.
  64. 64.
  65. 65.

Footnotes

  • Contributors EM, IS, MH and AB conceived and planned the study. EM developed and conducted the electronic database search strategy. EM and NTD performed the secondary search strategy. EM and AS screened records. EM, AS and NTD screened full-text articles. EM and AS performed data extraction and quality assessment of included articles. EM, IS, MH, FL, NTD and AB contributed to the analysis and interpretation of study results and to the development of the evaluation framework for healthcare provider trainings in SDM. EM, AS, IS, MH, FL, NTD and AB contributed to the writing of the manuscript and approved submission.

  • Funding This work was partly funded by Mundipharma GmbH, a pharmaceutical company.

  • Disclaimer Mundipharma GmbH had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

  • Competing interests EM and AS report grants from Mundipharma GmbH during the conduct of the study. EM, AS, IS, MH and AB conducted SDM communication skills trainings in a project funded by Mundipharma GmbH. NTD and FL have nothing to disclose.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement Extracted data can be obtained from the corresponding author.

  • Patient consent for publication None.