Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

HOW, WHY AND UNDER WHAT CIRCUMSTANCES DOES A QUALITY IMPROVEMENT COLLABORATIVE BUILD KNOWLEDGE AND SKILLS IN CLINICIANS WORKING WITH PEOPLE WITH DEMENTIA? A REALIST INFORMED PROCESS EVALUATION

View ORCID ProfileLenore de la Perrelle, View ORCID ProfileMonica Cations, View ORCID ProfileGaery Barbery, Gorjana Radisic, View ORCID ProfileBillingsley Kaambwa, View ORCID ProfileMaria Crotty, View ORCID ProfileJanna Anneke Fitzgerald, View ORCID ProfileSusan Kurrle, View ORCID ProfileIan D Cameron, View ORCID ProfileCraig Whitehead, Jane Thompson, View ORCID ProfileKate Laver
doi: https://doi.org/10.1101/2020.08.05.20169185
Lenore de la Perrelle
1College of Medicine and Public Health, Flinders University, Adelaide, Australia
2NHMRC Cognitive Decline Partnership Centre, Sydney, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Lenore de la Perrelle
  • For correspondence: Lenore.delaperrelle@flinders.edu.au
Monica Cations
3College of Education, Psychology and Social Work, Flinders University, Adelaide, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Monica Cations
Gaery Barbery
4Health Services Management, School of Medicine, Griffith University, South Bank, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Gaery Barbery
Gorjana Radisic
1College of Medicine and Public Health, Flinders University, Adelaide, Australia
2NHMRC Cognitive Decline Partnership Centre, Sydney, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Billingsley Kaambwa
1College of Medicine and Public Health, Flinders University, Adelaide, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Billingsley Kaambwa
Maria Crotty
1College of Medicine and Public Health, Flinders University, Adelaide, Australia
2NHMRC Cognitive Decline Partnership Centre, Sydney, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Maria Crotty
Janna Anneke Fitzgerald
2NHMRC Cognitive Decline Partnership Centre, Sydney, Australia
4Health Services Management, School of Medicine, Griffith University, South Bank, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Janna Anneke Fitzgerald
Susan Kurrle
2NHMRC Cognitive Decline Partnership Centre, Sydney, Australia
5Faculty of Medicine and Health, University of Sydney, Sydney, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Susan Kurrle
Ian D Cameron
2NHMRC Cognitive Decline Partnership Centre, Sydney, Australia
6John Walsh Centre for Rehabilitation Research, Faculty of Medicine and Health, The University of Sydney, St Leonards, New South Wales, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Ian D Cameron
Craig Whitehead
1College of Medicine and Public Health, Flinders University, Adelaide, Australia
2NHMRC Cognitive Decline Partnership Centre, Sydney, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Craig Whitehead
Jane Thompson
2NHMRC Cognitive Decline Partnership Centre, Sydney, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kate Laver
1College of Medicine and Public Health, Flinders University, Adelaide, Australia
2NHMRC Cognitive Decline Partnership Centre, Sydney, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Kate Laver
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

Abstract

In increasingly constrained health and aged care services, strategies are needed to improve quality and translate evidence into practice. In dementia care, recent failures in quality and safety have led the World Health Organisation to prioritise the translation of known evidence into practice. While quality improvement collaboratives have been widely used in healthcare, there are few examples in dementia care.

We describe a recent quality improvement collaborative to improve dementia care across Australia and assess the implementation outcomes of acceptability and feasibility of this strategy to translate known evidence into practice. A realist-informed process evaluation was used to analyse how, why and under what circumstances a quality improvement collaborative built knowledge and skills in clinicians working in dementia care.

This realist-informed process evaluation developed, tested, and refined the program theory of a quality improvement collaborative. Data were collected pre-and post-intervention using surveys and interviews with participants (n=24). A combined inductive and deductive data analysis process integrated three frameworks to examine the context and mechanisms of knowledge and skill building in participant clinicians.

A refined program theory showed how and why clinicians built knowledge and skills in quality improvement in dementia care. Seven mechanisms were identified: motivation, accountability, identity, collective learning, credibility, and reflective practice. Each of these mechanisms operated differently according to context.

A quality improvement collaborative designed for clinicians in different contexts and roles was acceptable and feasible in building knowledge and skills of clinicians to improve dementia care. A supportive setting and a credible, flexible, and collaborative process optimises quality improvement knowledge and skills in clinicians working with people with dementia.

Trial registration Australian and New Zealand Clinical Trials Registry 21 February 2018 (ACTRN 12618000268246)

Background

The challenge of implementing evidence-based guidelines into clinical practice continues to be of concern in healthcare (1). In dementia care recent OECD reports show poor care and low training persists in member countries (2). In Australia serious failures in dementia care (3) have prompted inquiries into safety and quality (4). In this context, the World Health Organisation Global action plan on the public health response to dementia 2017-2025 (5) identified as a priority, the need to translate what is already known about dementia care into action. In the complex field of dementia care, understanding what strategies work (6) is key to improve the quality of care for people living with dementia.

One approach widely used to implement evidence-based practices is the quality improvement collaborative (QIC) (7). This approach developed by the Institute for Healthcare Improvement (8) involves bringing together health professionals to learn and share methods to improve care. Key elements include a focus on a specific topic of healthcare, participants from multiple sites, clinical and quality improvement experts to provide advice and guidance to participants, structured activities to identify and try out improvements over time, and monitoring of progress against the aims of the improvement (9, 10). Despite their appeal in improving healthcare, high set up costs and varied success limit confidence in their use (7, 11, 12). There are few examples of QIC applications in dementia care (13–15).

Recent studies of QICs have described components (9, 16), reported on evaluations (17, 18), effectiveness (7) and cost-effectiveness (12), and identified factors influencing outcomes (19–21). Researchers identified the need to open the ‘black box’ of QICs to understand how components contribute to success (10, 22, 23). A theory-based understanding of the QIC process is advocated to better understand the influence of context on outcomes (24, 25). Understanding how and why QICs work under different circumstances is critical to assess suitability and justify the approach. Complex interventions such as QICs are multicomponent processes that interact with each other and the external and organisational contexts in which they operate (26). Linking a theoretical approach to an evaluation framework helps better understand how to design implementation interventions and evaluate how they work. Realist approaches have been used to understand how QICs work (25) and several studies have reported realist evaluations of process and outcomes (27–29). Few studies have used a realist approach (30) or explored the use of QICs to improve quality of dementia care (31). Realist evaluation (32) provides a method to understand how clinicians build knowledge and skills to improve dementia care in diverse settings (33).

Methods

Aim

This realist-informed process evaluation aims to improve our understanding of the ways in which QICs work to build knowledge and skills in quality improvement and implementation of evidence-based guidelines into practice in dementia care. Research questions are:

  1. How, why and under what circumstances do QICs build knowledge and skills in clinicians to improve quality and practice?

  2. Was the QIC approach acceptable and feasible to clinicians?

The process evaluation was embedded within a translational research trial (referred to as ‘Agents of Change’) which examined whether QICs could improve adherence to several recommendations in the Australian Clinical Practice Guidelines and Principles of Care for People with Dementia (referred to as the guidelines hereafter) (34). Full methods for the trial have been published in a protocol paper (34). The effect of the QIC on the outcome of guideline adherence was measured using an interrupted time series design and results will be reported separately. Ethical approval for the study was granted by the Southern Adelaide Clinical Human Research Ethics Committee (HREC/17/SAC/88). Clinicians responded to advertisements for the collaborative and self-selected to join a sub-group within the collaborative related to exercise, carer support or occupational therapy recommendations. A light touch, low cost intervention was trialled. This included online learning modules, teleconference meetings and email communication to reduce time and costs of participation. Local adaptation of the guideline recommendations was encouraged.

Patient and Public Involvement

Experts by experience of dementia (people with dementia and care-partners) were involved throughout the Agents of Change trial. The roles included membership of the Investigator and management group (JT), identification of priorities for the trial, review of training modules for collaborative topics, presentations to participants, and feedback on implementation plans. They were not involved in the design of this study. JT as co-author reviewed the manuscript and offered comments and wording for acknowledgement. An evaluation of the impact of experts by experience of dementia will be reported separately. Their contribution is acknowledged at the end of this article. Supplementary file 1 summarises the components in the QIC.

Study Design

This evaluation followed available guidance on process evaluation (35, 36) and realist evaluation (32) in knowledge translation interventions (37). It addressed implementation outcomes of acceptability and feasibility of the QIC approach in building skills and knowledge of participating clinicians (27). Outcomes of fidelity, penetration, and uptake of the clinical guidelines for dementia care as described in the protocol paper are addressed elsewhere (34).

The study was completed in four phases:

Phase 1: Development of the initial program logic and program theory to be tested This involved: 1) describing the strategy and logic of the program, 2) considering the context (C) of the intervention, 3) identifying the underlying mechanisms (M) and 4) reporting on the implementation outcomes (O) achieved (35). This is denoted as Context (C), Mechanism (M) and Outcome (O) configurations (4, 16) to enable understanding of the relationship between these program aspects.

The initial program theory was developed through searches of grey literature for theory components and academic database searches (26). The multiple components of the QIC method (38) were explored, then theory components were identified (39). This was confirmed with stakeholders using ‘If…then’ statements (Figure 1-part A), to be tested with clinicians.

Phase 2: Pre-and post-intervention data collection of surveys and interviews (quant +QUAL)

A concurrent mixed methods approach (40) was used to develop an understanding of the clinicians’ experience in the QIC. The survey tool QIKAT-R (41) identified changes in their level of knowledge of quality improvement over two time points. The NoMAD survey (42) was used to identify clinicians’ understanding of processes to normalise changes in practice on commencement and at completion of the program. These measures were compared with the mechanisms of change identified by clinicians in interviews, to examine patterns hypothesised in the initial program theory (39).

Phase 3: Data analysis, outcome patterns, and hypothesis testing

Survey and interview data identified change in knowledge and skills, and clinicians’ experience of the collaborative. Exit interviews with clinicians who withdrew from the collaborative provided an understanding of their decision. The reasons for how and why the collaborative process worked for clinicians was configured as context (C) + mechanism (M) = outcome (O) for each component. Summaries of the configurations were presented for three major settings in which clinicians worked. The interviews described acceptability and feasibility of the QIC and how learning generated change (43).

Phase 4: Refinement of the initial program theory

Survey data and identified configurations were tested against the initial program theory to confirm, refute, or revise the theory of how and why the QIC worked and in what circumstances (39). Where patterns matched hypotheses the program theory was confirmed. Where data did not match, the hypothesis was refuted and where additional conditions were identified, the program theory was modified. A revised program theory was developed to understand how QICs build knowledge and skills for clinicians (37)

Data Collection

Surveys

Quantitative data were collected in two surveys, using QIKAT-R (41) and NoMAD (42) administered pre-and post-participation in the collaborative. QIKAT-R is designed to assess clinicians’ ability to write an aim, a measure and change for a quality improvement scenario (41). NoMAD (42) assessed the degree of agreement of clinicians with statements based on the four Normalization Process Theory (NPT) (44) constructs of normalising a change to practice.

Clinicians consented to participate in the evaluation and undertook the surveys online in the introductory and final learning modules. Data were extracted for analysis in the evaluation. Supplementary file 2 provides an outline of the interview questions and Supplementary files 3 and 4 provide an example of the NoMAD and QIKAT-R surveys used in the online learning modules. On completion of the program clinicians were asked to comment online on their degree of success in implementing change.

Interviews

Clinicians were invited to participate in interviews and were introduced to the evaluator via an email from the project coordinator (MCa). The first author (LdlP) undertook the evaluation as a PhD student with experience as a clinician in aged care and sought consent via the approved ethics process.

Semi-structured private telephone interviews, up to an hour, were conducted with clinicians, on commencement and completion of the program. The same questions were asked of each person to describe their motivations, experiences, setting and role. A realist interviewing approach was used to understand their reasoning and responses (45). Interviews were recorded and transcribed with consent, checked for accuracy, and sent to clinicians for comment or addition. Field notes made by LdlP during the interviews added information for accuracy, emphasis, or requests to stop recording of parts of the interview.

Data Analysis

Surveys

Responses were extracted from the online surveys, de-identified for each clinician and compared to identify change in knowledge and skills of quality improvement (QIKAT-R), and engagement in processes of normalising implementation (NoMAD). Results were scored (by LdlP and GR) for QIKAT-R (41) using the rubric provided. The principal researcher (KL) resolved any discrepancies. The NoMAD (42) survey responses were converted to a five-point Likert scale (46) (by LdlP and checked by GR). Descriptive statistics were used to present the spread of responses from clinicians. Small sample sizes and lack of controls limited further statistical analysis.

Interviews

Interview data were transcribed verbatim, de-identified and entered into NVivo-12 software (47), for analysis using a combined inductive and deductive (48) framework analysis approach (49). Three frameworks were used to identify: issues related to the context (CFIR) (50), the social processes involved in normalising the change (NPT) (44), and the mechanisms at work within the collaboratives and the broader context (RE) (32). These frameworks aligned to focus on context (C), mechanism (M), and outcomes (O) to understand how, why and in what circumstances the collaboratives may work. Table 1. shows the alignment of these frameworks.

View this table:
  • View inline
  • View popup
Table 1.

Alignment of frameworks for analysis of qualitative data.

Coding categories were developed from the frameworks and interviews were coded deductively (LdlP) with 30% checked for consistency (GR). Any differences were resolved by discussion or consultation with the principal researcher (KL). Elements of context, mechanism and outcomes patterns were searched for in the data through a deliberate inductive process (27). Quotes from interviews were extracted and presented in the results. This adapted framework analysis was used to refine, confirm, or refute initial program theory (54).

Integration of results

Data from interviews and surveys were integrated at both the pre-and post-intervention stages through description and joint display (55) to identify where they confirmed, refuted, or modified the initial program theory. A revised program theory was developed to explain how and why the collaborative built knowledge and skills in quality improvement.

Results

Participants

Of the 45 clinicians in the Agents of Change trial, 28 (62%) were involved in the process evaluation. The QIKAT-R was completed by 26 (58%) clinicians at pre-intervention and 18 (38%) at post-intervention. The NoMAD survey was completed by 13 (29%) clinicians at pre-intervention and 15 (33%) post-intervention. Table 2 presents the characteristics of clinicians, showing the range of professions, settings, locations, type of organisation as well as the subgroup chosen for the collaborative.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 2.

Characteristics of participant clinicians by collaborative sub-group in the process evaluation

Pre-intervention survey results

Results for surveys are presented in Supplementary file 5 (NoMAD) and Supplementary file 6 (QIKAT-R) in comparison to post-intervention results. Most clinicians (80%) scored poorly on the QIKAT-R prior to the intervention, demonstrating limited knowledge about quality improvement. This finding validated the need for learning.

In the NoMAD survey, over 70% of clinicians saw the need for change, and how the guidelines differed from their current practice. Over 70% valued the proposed changes and 90% were optimistic about the support they would have from managers. Over 50% thought resources and training would be sufficient but only 40% were confident in their co-worker abilities to implement the changes.

Pre-intervention interview results

Interviews were conducted with 24 (53%) clinicians. They reported feeling highly motivated to undertake the process and participate in the collaborative sub-groups. Over 85% reported having no experience of leading quality improvement processes and were unsure of their knowledge or how the implementation process would unfold in their setting.

Context:

Most clinicians identified external policy and funding constraints on their organisations which could impact on their practice. This was reflected in changes to their roles, restructuring of the organisation and time constraints.

“We’re going through a major…change with the new CEO…challenge for me is that because staff are unhappy, we are having a high turnover1’ (participant S11)

In government funded hospital settings, multidisciplinary teams and formalised quality improvement structures were identified as being supportive of the proposed changes. In aged care settings, however, most participants identified role boundaries and workloads as barriers to quality improvement.

“No doubt there’ll be a bit of resistance from … staff, … ‘why should I do it your way, I’ve done it this

way my… entire life?’” (participant E11)

For clinicians working part time, any quality improvement activity had to be done in their own time. Sole providers expected to work in their own time to improve practice but sought support from their networks and peers.

“it will take a little bit of time …and work outside of those hours of working face to face with them - I’m

prepared to do that” (participant C05)

Wait and see approach:

Most clinicians were unaware of the existence of the guidelines before commencement and were uncertain of how recommendations could be adapted to their practice. While most understood how the QIC would work, they were cautious about what would be required in their setting.

“I need to have a little bit more understanding of what will be required from me… to see how my work practice … fits in with what everybody’s looking for” (participant O09)

Implementation processes:

Clinicians in all settings understood the intent to adapt implementation of the guidelines to suit their setting and expressed confidence that this approach was acceptable and feasible.

“I feel fairly confident that we will be able to get things off the ground and make some changes”

(participant S15)

Those working in hospital settings were more likely to have experience of quality improvement and had begun to identify who they needed to involve in the change process.

“.needs to go through my director and the …reference group …so any reporting back on any changes in process or procedure …would have to be verified … and approved’ (participant C06)

Most clinicians understood that implementing changes would involve communicating with others and engaging them in new practices.

Mechanisms:

The mechanisms identified in interviews were similar but described differently by clinicians in each of the three main settings. Table 3 presents mechanisms identified across three settings for participants.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 3.

Mechanisms identified across three key settings, with example quotations

Post-intervention survey results

Results for the post-intervention surveys are presented in Supplementary files 5 (NoMAD) and 6 (QIKAT-R) to compare with pre-intervention results. Clinician scores on the QIKAT-R rose modestly, with 80% showing improvement. This suggested that they had built quality improvement skills. The results from the NoMAD survey showed that most clinicians (85%) found the process made sense, and all highly valued the changes. The training and support through the QIC were considered sufficient for 75% of clinicians to develop implementation plans and make changes to their practice. While confidence in co-workers’ abilities increased, the level of support in their workplaces decreased 15% from initial expectations. All clinicians agreed that they were able to modify practice and deliver changes.

Post-intervention interview results

On completion of the QIC, interviews were conducted with 16 (36%) clinicians. Most reported their acceptance of and satisfaction with the process. They identified significant skills and understanding gained from the process.

“A solid methodology and a solid quality improvement plan have been really critical in getting us to a point where it’s working and sustainable” (participant S13)

They reported how the process enabled them to review their own practice.

“.quite a bit of reading and reflection that was involved in the project, especially when you’re going

through that PDSA cycle” (participant C05)

However, they reported that engaging others in their workplace was demanding. For those who were successful in making change, the support of managers and involvement of others were key to implementing the guidelines. Clinicians in aged care and public health settings reflected on the team effort. “.it was really a team effort at the end of the day. “ (participant S13)

“……identifying your local heroes and putting responsibility on other people …, ‘this isn’t just me doing this. This is us doing this’” (participant E11)

Others were able to align the improvements with organisational strategies and structures and gain support from others. In public health settings with quality improvement structures, this alignment made the process feasible and provided both accountability and recognition.

“. it crosses over many of the domains from the organisational point of view and accountability…. It’s been great to have that recognised” (participant C06)

External and internal context changes led six clinicians (13%) to withdraw from the program. Two had personal family circumstances that led them to leave their work and participation in the program. Others were related directly to organisational changes.

Funding changes at a national level resulted in significant organisational and role changes and stress for two clinicians in aged care settings.

“…the sector is facing quite dramatic reform…, - our focus upon managing dementia in the community, may not be a priority going forward” (participant S06)

That led to changes in the level of support available from their managers.

“…the support from management is very limited because their energy is all being focused on the (organisational) change itself.” (participant S02)

In public healthcare, time constraints impacted on the level of inter-disciplinary team support, with one clinician withdrawing due to tensions in the team.

“The dynamics were more difficult than I had anticipated, and making any change was going to alienate me” (participant O08)

Those who left the collaboratives were disappointed in not completing the program. They valued the learning modules, access to peers, and research team support. Feedback to the project team was positive with suggestions to reduce the lead-in time and the preference of having another colleague working with them for support.

Mechanisms at work within the collaborative:

The initial program theory was shared with clinicians in the interviews to consider and reflect on their experiences. The ’if…then’ propositions were presented to clinicians to assess if and how each applied to them. The mechanisms identified on commencement were generally supported and some were modified on reflection. The structured process of the collaborative provided confidence while a sense of accountability to complete the program drove commitment to the changes. The collaborative provided a sense of community and confidence in the process. The credibility of the evidence base and the team of experts and researchers engendered trust. An additional mechanism was identified of how achievements were recognised through reflection. Table 4 summarises the mechanisms and reasoning identified.

View this table:
  • View inline
  • View popup
Table 4.

Summary of mechanisms identified by clinicians at the conclusion of the program, with example quotations

Integration of results

Results from the post-intervention interviews and surveys were integrated and compared to the preintervention results to identify where they confirmed, refuted, or modified the program theory.

While results from the QIKAT-R survey showed modest improvement in knowledge of quality improvement methods, data from interviews provided examples from clinicians across settings that they gained knowledge and skills in quality improvement.

The results from the NOMAD survey confirmed that clinicians were engaged with the changes and made efforts to involve others in implementing changes. All clinicians agreed that feedback on implementation plans helped them to modify practice and deliver changes. The interview data confirm the value of reflective practice to clinicians to consider gaps and to monitor progress in changes.

Table 5 presents a summary of how the qualitative and quantitative results aligned to confirm, refute, or modify aspects of the program theory.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 5.

Integration of main findings and alignment with program theory

A refined program theory was developed and is presented in Figure 1-part B. Support through the QIC built confidence (mechanism) for most clinicians to make changes (outcome) despite time constraints and changing funding (context). When support was lacking in their setting, those same contextual constraints led some to withdraw or only partially complete the implementation. The credibility of the experts (context) encouraged trust in the process (mechanism) by clinicians to commit to implementation (outcome). Review of the program (context) enabled reflection and recognition of efforts (mechanism) in improving dementia care (outcome).

Discussion

A realist informed approach provided insights into how, why and under what circumstances a QIC built knowledge and skills in clinicians working in dementia care. The QIC attracted clinicians with a passion to improve dementia care in a context of resource constraints. It provided resources and opportunities for clinicians that were not usually available in their setting and met their needs for support, coaching, practice reflection and a flexible structure. They valued the credibility of the program, the flexible approach which suited their work needs, being part of a dementia specific collaborative and the process of trying out changes before adopting a new practice. When their personal motivation aligned with organisational structures and resources, clinicians successfully built the knowledge and skills to implement significant systems improvements and were recognised for their achievements. Others were able to change their practice for the selected recommendations of the guidelines and reported improvements for their clients. Many faced contextual barriers through time and resource constraints, manager or team resistance, major organisational restructures, and policy changes. While some clinicians withdrew due to contextual barriers, most gained knowledge, skills, and the confidence to engage in quality improvement which improved practice in their setting. There was a sense of empowerment for many clinicians in overcoming barriers to change. Seven mechanisms in the QIC were identified: motivation, accountability, identity, collective learning, credibility, and reflective practice. The relationships between context, mechanisms and outcomes showed how components of the QIC worked to build confidence, knowledge, and skills for most clinicians. The flexible, on-line delivery, and guidance of the QIC program made the process acceptable and feasible for most clinicians.

While QICs have been studied extensively, implementation has differed and outcomes have been inconsistent (7, 56). Few studies have used a realist approach (30) or explored the use of collaboratives to improve quality in dementia care (31). Applying a theory-based evaluation to understand how and why a QIC built knowledge and skills in clinicians, is key to capacity building (57) and identifying strategies for knowledge translation efforts internationally.

This study advances the understanding of how components of QICs contribute to success and why they matter to clinicians. It offers an understanding of a key component of collaboratives: how, why and in what circumstances clinicians build knowledge and skills in quality improvement. The findings offer insights to inform the spread of improved dementia care.

Evaluation strengths and limitations

The use of realist-informed process evaluation was a key strength. A theory-led framework analysis offered perspectives of context, implementation process and mechanisms at work within the collaboratives. The mixed methods design offered the opportunity to gather rich qualitative and quantitative data to examine how QICs work.

A limitation of this evaluation was the use of the QIKAT-R survey to measure knowledge about quality improvement. The survey was presented in a way that led to participants focussing on clinical responses rather than a process improvement approach, resulting in low scores. The interview data provided stronger evidence of improved knowledge and skills. Small numbers of participants in the evaluation limited statistical analysis but still offered a rich exploration of the mechanisms and contextual factors affecting their learning.

Conclusion

This study addresses a strategy to improve dementia care. A QIC designed to suit geographically dispersed clinicians in different settings and roles was acceptable and feasible in building knowledge and skills to improve dementia care. The motivations of clinicians and the credibility of the collaborative process built commitment and trust to learn implementation processes and empower clinicians to improve dementia care. This offers insight in how collaborative improvement processes work for designing QICs in complex and resource constrained settings.

Data Availability

Consent for making individual data available was not sought. aggregated and de-identified data may be available on reasonable request.

Competing Interests

MCa has been employed in the last 5 years to assist with data collection for Alzheimer’s disease drug trials funded by Janssen and Merck. All other authors declare they have no competing interests

Funding

This work was supported by the National Health and Medical Research Council (NHMRC) Partnership Centre on Dealing with Cognitive and Related Functional Decline in Older People (grant no. GNT9100000).

Ian Cameron is supported by an Australian Health and Medical Research Council Senior Practitioner Fellowship. Kate Laver is supported by an Australian Health and Medical Research Council Dementia Research Development Fellowship.

Acknowledgements

We gratefully acknowledge the involvement of the following people living with dementia and care-partners (Experts by Experience of Dementia) in the Agents of Change Quality Improvement

Collaborative project: Jane Thompson, Ian Gladstone, John Quinn, Glenys Petrie, Gary Collins, Mae Collins and Nadine Hedger.

REFERENCES

  1. 1.↵
    Chan Wiley V, Pearson Thomas A, et al. ACC/AHA Special Report: Clinical Practice Guideline Implementation Strategies: A Summary of Systematic Reviews by the NHLBI Implementation Science Work Group: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Journal of the American College of Cardiology. 2017;69(8):1076.
    OpenUrlFREE Full Text
  2. 2.↵
    OECD. Care needed: Improving the Lives of People with Dementia,. Paris: OECD; 2018 12 June 2018.
  3. 3.↵
    Groves A TD, McKellar D and Proctor N,. The Oakden Report. Adelaide, South Australia: SA Health, Department for Health and Ageing, Health S; 2017 2017.
  4. 4.↵
    Royal Commission of Inquiry into Aged Care Quality and Safety. Interim Report. Adelaide; 2019.
  5. 5.↵
    World Health Organisation. Global action plan on the public health response to dementia 2017 – 2025. Geneva: World health Organisation; 2017 2017.
  6. 6.↵
    Morley JE, Caplan G, Cesari M, et al. International Survey of Nursing Home Research Priorities. Journal of the American Medical Directors Association. 2014;15(5):309–12.
    OpenUrlCrossRefPubMed
  7. 7.↵
    Wells S, Tamir O, Gray J, et al. Are quality improvement collaboratives effective? A systematic review. BMJ Quality & Safety. 2018;27(3):226.
    OpenUrlAbstract/FREE Full Text
  8. 8.↵
    Institute for Healthcare Improvement. The Breakthrough Series : IHI’s Model for Achieving Breakthrough Improvement. Cambridge Massacheusetts: Institute for Healthcare Improvement; 2003 2003.
  9. 9.↵
    Nadeem E, Olin SS, Hill LC, et al. Understanding the Components of Quality Improvement Collaboratives: A Systematic Literature Review. Milbank Quarterly. 2013;91(2):354–94.
    OpenUrlCrossRefPubMedWeb of Science
  10. 10.↵
    Shaw E, K., Chase S, M., Howard J, et al. More black box to explore: how quality improvement collaboratives shape practice change. Journal of the American Board of Family Medicine : JABFM. 2012;25(2):149.
    OpenUrl
  11. 11.↵
    Schouten L, Hulscher M, van Everdingen J, et al. Evidence for the impact of quality improvement collaboratives: systematic review. Br Med J. 2008;336(7659):1491-+.
    OpenUrlAbstract/FREE Full Text
  12. 12.↵
    de la Perrelle L, Radisic G, Cations M, et al. Costs and economic evaluations of Quality Improvement Collaboratives in healthcare: a systematic review. BMC Health Services Research. 2020;20(1).
  13. 13.↵
    Devi R, Meyer J, Banerjee J, et al. Quality improvement collaborative aiming for Proactive HEAlthcare of Older People in Care Homes (PEACH): a realist evaluation protocol. BMJ Open. 2018;8(11):e023287.
    OpenUrlAbstract/FREE Full Text
  14. 14.
    Zubair M, Chadborn N, Gladman J, et al. Using comprehensive geriatric assessment for quality improvements in healthcare of older people in UK care homes: protocol for realist review within Proactive Healthcare of Older People in Care Homes (PEACH) study. BMJ Open. 2017;7(10).
  15. 15.↵
    Mittman BS. Improving the Quality of Dementia Care: The Role of Education. Clinical Gerontologist. 2006;29(2):61–9.
    OpenUrlCrossRef
  16. 16.↵
    Wilson T, Berwick DM, Cleary PD. What Do Collaborative Improvement Projects Do? Experience from Seven Countries. The Joint Commission Journal on Quality and Safety. 2003;29(2):85–93.
    OpenUrl
  17. 17.↵
    Mittman BS. Creating the Evidence Base for Quality Improvement Collaboratives. Annals of Internal Medicine. 2004;140(11):897–901.
    OpenUrlCrossRefPubMedWeb of Science
  18. 18.↵
    Schouten LM, Hulscher ME, Van Everdingen JJ, et al. Short- and long-term effects of a quality improvement collaborative on diabetes management. Implementation Science. 2010;5(1):94.
    OpenUrl
  19. 19.↵
    Hulscher ME, Schouten LM, Grol RP, et al. Determinants of success of quality improvement collaboratives: what does the literature show? BMJ Qual Saf. 2013;22(1):19–31.
    OpenUrlAbstract/FREE Full Text
  20. 20.
    Brown V, Fuller J, Ford D, et al. The enablers and barriers for the uptake, use and spread of primary health care Collaboratives in Australia. Herston QLD: APHCRI Centre of research Excellence in Primary Health Care Microsystems; 2014 2014.
  21. 21.↵
    Gustafson D, Quanbeck A, Robinson J, et al. Which elements of improvement collaboratives are most effective? A cluster-randomized trial. Addiction. 2013;108(6):1145–57.
    OpenUrl
  22. 22.↵
    Dückers MLA, Spreeuwenberg P, Wagner C, et al. Exploring the black box of quality improvement collaboratives: modelling relations between conditions, applied changes and outcomes. Implementation Science : IS. 2009;4:74-.
    OpenUrl
  23. 23.↵
    Algurén B, Nordin A, Andersson-Gäre B, et al. In-depth comparison of two quality improvement collaboratives from different healthcare areas based on registry data—possible factors contributing to sustained improvement in outcomes beyond the project time. Implementation Science. 2019;14(1).
  24. 24.↵
    Dixon-Woods M, Bosk CL, Aveling EL, et al. Explaining Michigan: Developing an Ex Post Theory of a Quality Improvement Program. 2011;89(2):167–205.
  25. 25.↵
    Zamboni K, Baker U, Tyagi M, et al. How and under what circumstances do quality improvement collaboratives lead to better outcomes? A systematic review. Implementation Science. 2020;15(1).
  26. 26.↵
    Shearn K, Allmark P, Piercy H, et al. Building Realist Program Theory for Large Complex and Messy Interventions. International Journal of Qualitative Methods. 2017;16(1):160940691774179.
    OpenUrl
  27. 27.↵
    Rycroft-Malone J, Seers K, Eldh AC, et al. A realist process evaluation within the Facilitating Implementation of Research Evidence (FIRE) cluster randomised controlled international trial: an exemplar. Implementation Science. 2018;13(138).
  28. 28.
    Willis CE, Reid S, Elliott C, et al. A realist evaluation of a physical activity participation intervention for children and youth with disabilities: what works, for whom, in what circumstances, and how? BMC Pediatrics. 2018;18(1).
  29. 29.↵
    Brand SL, Quinn C, Pearson M, et al. Building programme theory to develop more adaptable and scalable complex interventions: Realist formative process evaluation prior to full trial. Evaluation. 2018;25(2):149–70.
    OpenUrl
  30. 30.↵
    Handley M, Bunn F, Goodman C. Dementia-friendly interventions to improve the care of people living with dementia admitted to hospitals: a realist review. BMJ Open. 2017;7(7):e015257.
    OpenUrlAbstract/FREE Full Text
  31. 31.↵
    Dixon N, Wellsteed L. Effects of team-based quality improvement learning on two teams providing dementia care. BMJ Open Quality. 2019;8(2):e000500.
    OpenUrl
  32. 32.↵
    Pawson R. Realistic Evaluation. London: Sage Publications; 1997. 256 p.
  33. 33.↵
    Ogrinc G, Batalden P, Moore S. Realist Evaluation as a Framework for the Assessment of Teaching About the Improvement of Care/Commentary on “Realist Evaluation as a Framework for the Assessment of Teaching About the Improvement of Care”. Journal of Nursing Education. 2009;48(12):661–7.
    OpenUrlCrossRefPubMedWeb of Science
  34. 34.↵
    Cations M, Crotty M, Fitzgerald JA, et al. Agents of change: establishing quality improvement collaboratives to improve adherence to Australian clinical guidelines for dementia care. Implementation Science. 2018;13(1).
  35. 35.↵
    Moore GF, Audrey S, Barker M, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ : British Medical Journal. 2015;350.
  36. 36.↵
    Scott SD, Rotter T, Flynn R, et al. Systematic review of the use of process evaluations in knowledge translation research. Systematic Reviews. 2019;8(1).
  37. 37.↵
    Salter K, Kothari A. Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review. Implement Sci. 2014;9(1).
  38. 38.↵
    Berwick D, M. The Science of Improvement. JAMA. 2008;299(10):1182.
    OpenUrlCrossRefPubMedWeb of Science
  39. 39.↵
    Wong G, Greenhalgh T, Westhorp G, et al. Realist methods in medical education research: what are they and what can they contribute? Medical Education. 2012;46(1):89–96.
    OpenUrlCrossRefPubMed
  40. 40.↵
    Creswell JW. Designing A Mixed Methods Study In Primary Care. The Annals of Family Medicine. 2004;2(1):7–12.
    OpenUrlAbstract/FREE Full Text
  41. 41.↵
    Singh MK, Ogrinc G, Cox KR, et al. The Quality Improvement Knowledge Application Tool Revised (QIKAT-R). Academic medicine : journal of the Association of American Medical Colleges. 2014;89(10):1386–91.
    OpenUrl
  42. 42.↵
    Finch T L, Girling M, May C R, et al. Normalization Process Theory On-line Users’ Manual, Toolkit and NoMAD instrument [Measurement instrument]. UK: NPT 2020; 2015 [NoMAD toolkit]. Available from: http://www.normalizationprocess.org.
  43. 43.↵
    Pawson R, Manzano-Santaella A. A realist diagnostic workshop. 2012;18:176–91.
  44. 44.↵
    May C, Finch T. Implementing, Embedding, and Integrating Practices: An Outline of Normalization Process Theory. Sociology. 2009;43(3):535–54.
    OpenUrlCrossRefWeb of Science
  45. 45.↵
    Manzano A. The craft of interviewing in realist evaluation. Evaluation. 2016;22(3):342–60.
    OpenUrlCrossRef
  46. 46.↵
    Sullivan GM, Artino AR. Analyzing and Interpreting Data From Likert-Type Scales. Journal of Graduate Medical Education. 2013;5(4):541–2.
    OpenUrlCrossRefPubMed
  47. 47.↵
    NVivo 12 [Internet]. QSR Intrenational Pty Ltd. 2020. Available from: https://www.qsrinternational.com/.
  48. 48.↵
    Fletcher Amber J. Applying Critical Realism in Qualitative Research: Methodology Meets Method. International Journal of Social Research Methodology. 2017;20(2):181–94.
    OpenUrl
  49. 49.↵
    Furber C. Framework analysis: a method for analysing qualitative data. African Journal of Midwifery and Women’s Health. 2010;4(2):97–100.
    OpenUrl
  50. 50.↵
    Damschroder L, J., Aron D, C., Keith R, E., et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science. 2009;4(1):50.
    OpenUrl
  51. 51.
    Dalkin SM, Greenhalgh J, Jones D, et al. What’s in a mechanism? Development of a key concept in realist evaluation. Implementation Science. 2015;10(1).
  52. 52.
    Emmel N, Greenhalgh J, Manzano A, et al. Doing realist research: London : Sage; 2018.
  53. 53.
    Lacouture A, Breton E, Guichard A, et al. The concept of mechanism from a realist approach: a scoping review to facilitate its operationalization in public health program evaluation. Implementation Science. 2015;10(1).
  54. 54.↵
    Bergeron DA, Gaboury I. Challenges related to the analytical process in realist evaluation and latest developments on the use of NVivo from a realist perspective. International Journal of Social Research Methodology. 2020;23(3):355–65.
    OpenUrl
  55. 55.↵
    Guetterman TC, Fetters MD, Creswell JW. Integrating Quantitative and Qualitative Results in Health Science Mixed Methods Research Through Joint Displays. The Annals of Family Medicine. 2015;13(6):554–61.
    OpenUrlAbstract/FREE Full Text
  56. 56.↵
    Robert G, Sarre S, Maben J, et al. Exploring the sustainability of quality improvement interventions in healthcare organisations: a multiple methods study of the 10-year impact of the ‘Productive Ward: Releasing Time to Care’ programme in English acute hospitals. BMJ Quality & Safety. 2020;29(1):31–40.
    OpenUrlAbstract/FREE Full Text
  57. 57.↵
    Knight AW, Dhillon M, Smith C, et al. A quality improvement collaborative to build improvement capacity in regional primary care support organisations. BMJ Open Quality. 2019;8(3):e000684.
    OpenUrl
Back to top
PreviousNext
Posted August 07, 2020.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
HOW, WHY AND UNDER WHAT CIRCUMSTANCES DOES A QUALITY IMPROVEMENT COLLABORATIVE BUILD KNOWLEDGE AND SKILLS IN CLINICIANS WORKING WITH PEOPLE WITH DEMENTIA? A REALIST INFORMED PROCESS EVALUATION
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
HOW, WHY AND UNDER WHAT CIRCUMSTANCES DOES A QUALITY IMPROVEMENT COLLABORATIVE BUILD KNOWLEDGE AND SKILLS IN CLINICIANS WORKING WITH PEOPLE WITH DEMENTIA? A REALIST INFORMED PROCESS EVALUATION
Lenore de la Perrelle, Monica Cations, Gaery Barbery, Gorjana Radisic, Billingsley Kaambwa, Maria Crotty, Janna Anneke Fitzgerald, Susan Kurrle, Ian D Cameron, Craig Whitehead, Jane Thompson, Kate Laver
medRxiv 2020.08.05.20169185; doi: https://doi.org/10.1101/2020.08.05.20169185
Reddit logo Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
HOW, WHY AND UNDER WHAT CIRCUMSTANCES DOES A QUALITY IMPROVEMENT COLLABORATIVE BUILD KNOWLEDGE AND SKILLS IN CLINICIANS WORKING WITH PEOPLE WITH DEMENTIA? A REALIST INFORMED PROCESS EVALUATION
Lenore de la Perrelle, Monica Cations, Gaery Barbery, Gorjana Radisic, Billingsley Kaambwa, Maria Crotty, Janna Anneke Fitzgerald, Susan Kurrle, Ian D Cameron, Craig Whitehead, Jane Thompson, Kate Laver
medRxiv 2020.08.05.20169185; doi: https://doi.org/10.1101/2020.08.05.20169185

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Health Systems and Quality Improvement
Subject Areas
All Articles
  • Addiction Medicine (228)
  • Allergy and Immunology (504)
  • Anesthesia (110)
  • Cardiovascular Medicine (1240)
  • Dentistry and Oral Medicine (206)
  • Dermatology (147)
  • Emergency Medicine (282)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (531)
  • Epidemiology (10023)
  • Forensic Medicine (5)
  • Gastroenterology (499)
  • Genetic and Genomic Medicine (2453)
  • Geriatric Medicine (238)
  • Health Economics (479)
  • Health Informatics (1643)
  • Health Policy (753)
  • Health Systems and Quality Improvement (636)
  • Hematology (248)
  • HIV/AIDS (533)
  • Infectious Diseases (except HIV/AIDS) (11864)
  • Intensive Care and Critical Care Medicine (626)
  • Medical Education (252)
  • Medical Ethics (75)
  • Nephrology (268)
  • Neurology (2281)
  • Nursing (139)
  • Nutrition (352)
  • Obstetrics and Gynecology (454)
  • Occupational and Environmental Health (537)
  • Oncology (1245)
  • Ophthalmology (377)
  • Orthopedics (134)
  • Otolaryngology (226)
  • Pain Medicine (158)
  • Palliative Medicine (50)
  • Pathology (324)
  • Pediatrics (730)
  • Pharmacology and Therapeutics (313)
  • Primary Care Research (282)
  • Psychiatry and Clinical Psychology (2281)
  • Public and Global Health (4834)
  • Radiology and Imaging (837)
  • Rehabilitation Medicine and Physical Therapy (492)
  • Respiratory Medicine (651)
  • Rheumatology (285)
  • Sexual and Reproductive Health (238)
  • Sports Medicine (227)
  • Surgery (267)
  • Toxicology (44)
  • Transplantation (125)
  • Urology (99)