1932

Abstract

There is strong interest in implementation science to address the gap between research and practice in public health. Research on the sustainability of evidence-based interventions has been growing rapidly. Sustainability has been defined as the continued use of program components at sufficient intensity for the sustained achievement of desirable program goals and population outcomes. This understudied area has been identified as one of the most significant translational research problems. Adding to this challenge is uncertainty regarding the extent to which intervention adaptation and evolution are necessary to address the needs of populations that differ from those in which interventions were originally tested or implemented. This review critically examines and discusses conceptual and methodological issues in studying sustainability, summarizes the multilevel factors that have been found to influence the sustainability of interventions in a range of public health and health care settings, and highlights key areas for future research.

Loading

Article metrics loading...

/content/journals/10.1146/annurev-publhealth-040617-014731
2018-04-01
2024-03-28
Loading full text...

Full text loading...

/deliver/fulltext/publhealth/39/1/annurev-publhealth-040617-014731.html?itemId=/content/journals/10.1146/annurev-publhealth-040617-014731&mimeType=html&fmt=ahah

Literature Cited

  1. Aarons GA, Green AE, Trott E, Willging CE, Torres EM. 1.  et al. 2016. The roles of system and organizational leadership in system-wide evidence-based intervention sustainment: a mixed-method study. Adm. Policy Mental Health Mental Health Serv. Res. 43:991–1008 [Google Scholar]
  2. Aarons GA, Hurlburt M, Horwitz SM. 2.  2011. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm. Policy Mental Health Mental Health Serv. Res. 38:4–23 [Google Scholar]
  3. Allen JD, Shelton RC, Emmons KM, Linnan L. 3. 2017 Fidelity and its relationship to implementation effectiveness, adaptation, and dissemination. See Ref. 17 267–84
  4. Amazigo U, Okeibunor J, Matovu V, Zoure H, Bump J, Seketeli A. 4.  2007. Performance of predictors: evaluating sustainability in community-directed treatment projects of the African programme for onchocerciasis control. Soc. Sci. Med. 64:2070–82 [Google Scholar]
  5. Ament S, Gillissen F, Moser A, Maessen J, Dirksen CD. 5.  et al. 2017. Factors associated with sustainability of 2 quality improvement programs after achieving early implementation success: a qualitative case study. J. Eval. Clin. Pract. 23:61135–43 [Google Scholar]
  6. Ammerman A, Smith TW, Calancie L. 6.  2014. Practice-based evidence in public health: improving reach, relevance, and results. Annu. Rev. Public Health 35:47–63 [Google Scholar]
  7. Barrera M, Berkel C, Castro FG. 7.  2017. Directions for the advancement of culturally adapted preventive interventions: local adaptations, engagement, and sustainability. Prev. Sci. 18:6640–48 [Google Scholar]
  8. Baumann A, Cabassa L, Wiltsey Stirman S. 8. 2017 Adaptation in dissemination and implementation science. See Ref. 17 285–300
  9. Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS. 9.  et al. 2004. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium. Health Psychol 23:443–51 [Google Scholar]
  10. Berendsen BA, Kremers SP, Savelberg HH, Schaper NC, Hendriks MR. 10.  2015. The implementation and sustainability of a combined lifestyle intervention in primary care: mixed method process evaluation. BMC Fam. Pract. 16:37 [Google Scholar]
  11. Bernal G, Sáez-Santiago E. 11.  2006. Culturally centered psychosocial interventions. J. Community Psychol. 34:121–32 [Google Scholar]
  12. Blanchet K, Palmer J, Palanchowke R, Boggs D, Jama A, Girois S. 12.  2014. Advancing the application of systems thinking in health: analysing the contextual and social network factors influencing the use of sustainability indicators in a health system—a comparative study in Nepal and Somaliland. Health Res. Policy Syst. 12:46 [Google Scholar]
  13. Bond GR, Drake RE, McHugo GJ, Peterson AE, Jones AM, Williams J. 13.  2014. Long-term sustainability of evidence-based practices in community mental health agencies. Adm. Policy Mental Health Mental Health Serv. Res. 41:228–36 [Google Scholar]
  14. Bopp M, Saunders RP, Lattimore D. 14.  2013. The tug-of-war: fidelity versus adaptation throughout the health promotion program life cycle. J. Prim. Prev. 34:193–207 [Google Scholar]
  15. Bossert TJ. 15.  1990. Can they get along without us? Sustainability of donor-supported health projects in Central America and Africa. Soc. Sci. Med. 30:1015–23 [Google Scholar]
  16. Brown CH, Curran G, Palinkas LA, Aarons GA, Wells KB. 16.  et al. 2017. An overview of research and evaluation designs for dissemination and implementation. Annu. Rev. Public Health 38:1–22 [Google Scholar]
  17. Brownson RC, Colditz GA, Proctor EK. 17. , eds. 2017 Dissemination and Implementation Research in Health: Translating Science to Practice New York: Oxford Univ. Press, 2nd ed..
  18. Buller DB, Walkosz BJ, Andersen PA, Scott MD, Cutter GR. 18.  2015. Sustained use of an occupational sun safety program in a recreation industry: follow-up to a randomized trial on dissemination strategies. Transl. Behav. Med. 5:361–71 [Google Scholar]
  19. Calhoun A, Mainor A, Moreland-Russell S, Maier RC, Brossart L, Luke DA. 19.  2014. Using the Program Sustainability Assessment Tool to assess and plan for sustainability. Prev. Chronic Dis. 11:130185 [Google Scholar]
  20. Campbell S, Pieters K, Mullen K-A, Reece R, Reid RD. 20.  2011. Examining sustainability in a hospital setting: case of smoking cessation. Implement. Sci. 6:108 [Google Scholar]
  21. Carvalho ML, Honeycutt S, Escoffery C, Glanz K, Sabbs D, Kegler MC. 21.  2013. Balancing fidelity and adaptation: implementing evidence-based chronic disease prevention programs. J. Public Health Manag. Pract. 19:348–56 [Google Scholar]
  22. Castro FG, Barrera M, Martinez CR Jr. 22.  2004. The cultural adaptation of prevention interventions: resolving tensions between fidelity and fit. Prev. Sci. 5:41–45 [Google Scholar]
  23. Chambers DA, Glasgow RE, Stange KC. 23.  2013. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement. Sci. 8:117 [Google Scholar]
  24. Chambers DA, Norton WE. 24.  2016. The adaptome: advancing the science of intervention adaptation. Am. J. Prev. Med. 51:S124–31 [Google Scholar]
  25. Chaudoir SR, Dugan AG, Barr CH. 25.  2013. Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures. Implement. Sci. 8:22 [Google Scholar]
  26. Cook JM, Dinnen S, Thompson R, Simiola V, Schnurr PP. 26.  2014. Changes in implementation of two evidence‐based psychotherapies for PTSD in VA residential treatment programs: a national investigation. J. Trauma. Stress 27:137–43 [Google Scholar]
  27. Cooper BR, Bumbarger BK, Moore JE. 27.  2015. Sustaining evidence-based prevention programs: correlates in a large-scale dissemination initiative. Prev. Sci. 16:145–57 [Google Scholar]
  28. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. 28.  2012. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med. Care 50:217 [Google Scholar]
  29. Curry SJ, Mermelstein RJ, Sporer AK. 29.  2016. Sustainability of community-based youth smoking cessation programs: results from a 3-year follow-up. Health Promot. Pract. 17:6845–52 [Google Scholar]
  30. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. 30.  2009. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement. Sci. 4:50 [Google Scholar]
  31. Dearing JW. 31.  2009. Applying diffusion of innovation theory to intervention development. Res. Soc. Work Pract. 19:5503–18 [Google Scholar]
  32. Dearing JW, Kreuter MW. 32.  2010. Designing for diffusion: How can we increase uptake of cancer communication innovations?. Patient Educ. Couns. 81:S100–10 [Google Scholar]
  33. Doyle C, Howe C, Woodcock T, Myron R, Phekoo K. 33.  et al. 2013. Making change last: applying the NHS institute for innovation and improvement sustainability model to healthcare improvement. Implement. Sci. 8:127 [Google Scholar]
  34. Durlak JA, DuPre EP. 34.  2008. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am. J. Community Psychol. 41:327–50 [Google Scholar]
  35. Estabrooks PA, Smith-Ray RL, Dzewaltowski DA, Dowdy D, Lattimore D. 35.  et al. 2011. Sustainability of evidence-based community-based physical activity programs for older adults: lessons from Active for Life. Transl. Behav. Med. 1:208–15 [Google Scholar]
  36. Feinberg ME, Bontempo DE, Greenberg MT. 36.  2008. Predictors and level of sustainability of community prevention coalitions. Am. J. Prev. Med. 34:495–501 [Google Scholar]
  37. Fleiszer AR, Semenic SE, Ritchie JA, Richer MC, Denis JL. 37.  2015. An organizational perspective on the long-term sustainability of a nursing best practice guidelines program: a case study. BMC Health Serv. Res. 15:535 [Google Scholar]
  38. Fleiszer AR, Semenic SE, Ritchie JA, Richer MC, Denis JL. 38.  2016. Nursing unit leaders' influence on the long‐term sustainability of evidence‐based practice improvements. J. Nurs. Manag. 24:3309–18 [Google Scholar]
  39. Fleiszer AR, Semenic SE, Ritchie JA, Richer MC, Denis JL. 39.  2015. The sustainability of healthcare innovations: a concept analysis. J. Adv. Nurs. 71:1484–98 [Google Scholar]
  40. Ford JH II, Krahn D, Wise M, Oliver KA. 40.  2011. Measuring sustainability within the Veterans Administration Mental Health System Redesign Initiative. Q. Manag. Health Care 20:263–79 [Google Scholar]
  41. Forehand R, Dorsey S, Jones DJ, Long N, McMahon RJ. 41.  2010. Adherence and flexibility: They can (and do) coexist!. Clin. Psychol. Sci. Pract. 17:258–64 [Google Scholar]
  42. Francis L, Dunt D, Cadilhac DA. 42.  2016. How is the sustainability of chronic disease health programmes empirically measured in hospital and related healthcare services? A scoping review. BMJ Open 6:e010944 [Google Scholar]
  43. Galbraith JS, Stanton B, Boekeloo B, King W, Desmond S. 43.  et al. 2009. Exploring implementation and fidelity of evidence-based behavioral interventions for HIV prevention: lessons learned from the Focus on Kids diffusion case study. Health Educ. Behav. 36:532–49 [Google Scholar]
  44. Glanz K, Escoffery C, Elliott T, Nehl EJ. 44.  2015. Randomized trial of two dissemination strategies for a skin cancer prevention program in aquatic settings. Am. J. Public Health 105:1415–23 [Google Scholar]
  45. Glasgow RE, Chambers D. 45.  2012. Developing robust, sustainable, implementation systems using rigorous, rapid and relevant science. Clin. Transl. Sci. 5:48–55 [Google Scholar]
  46. Glasgow RE, Green LW, Klesges LM, Abrams DB, Fisher EB. 46.  et al. 2006. External validity: We need to do more. Ann. Behav. Med. 31:105–8 [Google Scholar]
  47. Glasgow RE, Klesges LM, Dzewaltowski DA, Bull SS, Estabrooks P. 47.  2004. The future of health behavior change research: What is needed to improve translation of research into health promotion practice?. Ann. Behav. Med. 27:3 [Google Scholar]
  48. Glasgow RE, Lichtenstein E, Marcus AC. 48.  2003. Why don't we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. Am. J. Public Health 93:1261–67 [Google Scholar]
  49. Glasgow RE, Vinson C, Chambers D, Khoury MJ, Kaplan RM, Hunter C. 49.  2012. National Institutes of Health approaches to dissemination and implementation science: current and future directions. Am. J. Public Health 102:1274–81 [Google Scholar]
  50. Glasgow RE, Vogt TM, Boles SM. 50.  1999. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am. J. Public Health 89:1322–27 [Google Scholar]
  51. Gloppen KM, Brown EC, Wagenaar BH, Hawkins JD, Rhew IC, Oesterle S. 51.  2016. Sustaining adoption of science‐based prevention through Communities That Care. J. Community Psychol. 44:78–89 [Google Scholar]
  52. Goodman RM, Steckler A. 52.  1989. A model for the institutionalization of health promotion programs. Fam. Community Health 11:63–78 [Google Scholar]
  53. Green LW. 53.  2008. Making research relevant: If it is an evidence-based practice, where's the practice-based evidence?. Fam. Pract. 25:i20–24 [Google Scholar]
  54. Green LW, Glasgow RE. 54.  2006. Evaluating the relevance, generalization, and applicability of research: issues in external validation and translation methodology. Eval. Health Prof. 29:126–53 [Google Scholar]
  55. Green LW, Kreuter MW. 55.  2005. Health Program Planning: An Educational and Ecological Approach. New York: McGraw-Hill Co.
  56. Greenberg MT, Feinberg ME, Johnson LE, Perkins DF, Welsh JA, Spoth RL. 56.  2015. Factors that predict financial sustainability of community coalitions: five years of findings from the PROSPER partnership project. Prev. Sci. 16:158–67 [Google Scholar]
  57. Greenhalgh T, Macfarlane F, Barton‐Sweeney C, Woodard F. 57.  2012. “If we build it, will it stay?” A case study of the sustainability of whole‐system change in London. Milbank Q 90:516–47 [Google Scholar]
  58. Gruen RL, Elliott JH, Nolan ML, Lawton PD, Parkhill A. 58.  et al. 2008. Sustainability science: an integrated approach for health-programme planning. Lancet 372:1579–89 [Google Scholar]
  59. Hansen WB, Hill LG, Owens RW. 59.  2013. Component analysis of adherence in a family intervention. Health Educ 113:264–80 [Google Scholar]
  60. Hodge LM, Turner KM. 60.  2016. Sustained implementation of evidence‐based programs in disadvantaged communities: a conceptual framework of supporting factors. Am. J. Community Psychol. 58:192–210 [Google Scholar]
  61. Hodge LM, Turner KM, Sanders MR, Filus A. 61.  2017. Sustained implementation support scale: validation of a measure of program characteristics and workplace functioning for sustained program implementation. J. Behav. Health Serv. Res. 44:3442–64 [Google Scholar]
  62. Hunter SB, Han B, Slaughter ME, Godley SH, Garner BR. 62.  2015. Associations between implementation characteristics and evidence-based practice sustainment: a study of the Adolescent Community Reinforcement Approach. Implement. Sci. 10:173 [Google Scholar]
  63. Iwelunmor J, Blackstone S, Veira D, Nwaozuru U, Airhihenbuwa C. 63.  et al. 2016. Toward the sustainability of health interventions implemented in sub-Saharan Africa: a systematic review and conceptual framework. Implement. Sci. 11:43 [Google Scholar]
  64. Johnson K, Collins D, Shamblen S, Kenworthy T, Wandersman A. 64.  2017. Long-term sustainability of evidence-based prevention interventions and community coalitions survival: a five and one-half year follow-up study. Prev. Sci. 18:5610–21 [Google Scholar]
  65. Johnson K, Collins D, Wandersman A. 65.  2013. Sustaining innovations in community prevention systems: a data‐informed sustainability strategy. J. Community Psychol. 41:322–40 [Google Scholar]
  66. Johnson K, Hays C, Center H, Daley C. 66.  2004. Building capacity and sustainable prevention innovations: a sustainability planning model. Eval. Program Plan. 27:135–49 [Google Scholar]
  67. Knapp H, Hagedorn H, Anaya HD. 67.  2014. A five-year self-sustainability analysis of nurse-administered HIV rapid testing in Veterans Affairs primary care. Int. J. STD AIDS 25:837–43 [Google Scholar]
  68. Kumpfer KL, Alvarado R, Smith P, Bellamy N. 68.  2002. Cultural sensitivity and adaptation in family-based prevention interventions. Prev. Sci. 3:241–46 [Google Scholar]
  69. Kumpfer KL, Pinyuchon M, de Melo AT, Whiteside HO. 69.  2008. Cultural adaptation process for international dissemination of the Strengthening Families Program. Eval. Health Prof. 31:226–39 [Google Scholar]
  70. Leadbeater BJ, Gladstone EJ, Sukhawathanakul P. 70.  2015. Planning for sustainability of an evidence‐based mental health promotion program in Canadian elementary schools. Am. J. Community Psychol. 56:120–33 [Google Scholar]
  71. Lean M, Leavey G, Killaspy H, Green N, Harrison I. 71.  et al. 2015. Barriers to the sustainability of an intervention designed to improve patient engagement within NHS mental health rehabilitation units: a qualitative study nested within a randomised controlled trial. BMC Psychiatry 15:209 [Google Scholar]
  72. Leeman J, Calancie L, Hartman MA, Escoffery CT, Herrmann AK. 72.  et al. 2015. What strategies are used to build practitioners’ capacity to implement community-based interventions and are they effective? A systematic review. Implement. Sci. 10:80 [Google Scholar]
  73. Lewis CC, Scott K, Marti CN, Marriott BR, Kroenke K. 73.  et al. 2015. Implementing measurement-based care (iMBC) for depression in community mental health: a dynamic cluster randomized trial study protocol. Implement. Sci. 10:127 [Google Scholar]
  74. Lewis CC, Stanick CF, Martinez RG, Weiner BJ, Kim M. 74.  et al. 2015. The society for implementation research collaboration instrument review project: a methodology to promote rigorous evaluation. Implement. Sci. 10:2 [Google Scholar]
  75. Lewis CC, Weiner BJ, Stanick C, Fischer SM. 75.  2015. Advancing implementation science through measure development and evaluation: a study protocol. Implement. Sci. 10:102 [Google Scholar]
  76. Luke DA, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. 76.  2014. The Program Sustainability Assessment Tool: a new instrument for public health programs. Prev. Chronic Dis. 11:130184 [Google Scholar]
  77. Lyon AR, Koerner K. 77.  2016. User‐centered design for psychosocial intervention development and implementation. Clin. Psychol. Sci. Pract. 23:180–200 [Google Scholar]
  78. Maher L, Gustafson DH, Evans A. 78.  2010. NHS sustainability model NHS Inst. Innov. Improv., London. http://www.institute.nhs.uk/sustainability
  79. Maher L, Gustafson D, Evans A. 79.  2010. Sustainability Model and Guide London: NHS Inst. Innov. Improv http://www.institute.nhs.uk/index.php?option=com_joomcart&main_page=document_product_info&products_id=291&cPath=88 (login required)
  80. Mancini JA, Marek LI. 80.  2004. Sustaining community‐based programs for families: conceptualization and measurement. Fam. Relat. 53:339–47 [Google Scholar]
  81. Manning MA, Bollig-Fischer A, Bobovski LB, Lichtenberg P, Chapman R, Albrecht TL. 81.  2014. Modeling the sustainability of community health networks: novel approaches for analyzing collaborative organization partnerships across time. Transl. Behav. Med. 4:46–59 [Google Scholar]
  82. Marek LI, Brock D-JP, Sullivan R. 82.  2006. Cultural adaptations to a family life skills program: implementation in rural Appalachia. J. Prim. Prev. 27:113–33 [Google Scholar]
  83. Martinez RG, Lewis CC, Weiner BJ. 83.  2014. Instrumentation issues in implementation science. Implement. Sci. 9:118 [Google Scholar]
  84. McIntosh K, Mercer SH, Nese RN, Strickland-Cohen MK, Hoselton R. 84.  2016. Predictors of sustained implementation of school-wide positive behavioral interventions and supports. J. Posit. Behav. Interv. 18:209–18 [Google Scholar]
  85. Minkler M, Wallerstein N. 85.  2011. Community-Based Participatory Research for Health: From Process to Outcomes. New York: Wiley
  86. Moore JE, Bumbarger BK, Cooper BR. 86.  2013. Examining adaptations of evidence-based programs in natural contexts. J. Prim. Prev. 34:147–61 [Google Scholar]
  87. Moore JE, Grouchy M, Graham ID, Shandling M, Doyle W, Straus SE. 87.  2016. The Council of Academic Hospitals of Ontario (CAHO) Adopting Research to Improve Care (ARTIC) program: reach, sustainability, spread and lessons learned from an implementation funding model. Healthc. Policy 11:27 [Google Scholar]
  88. Moore JE, Mascarenhas A, Bain J, Straus SE. 88.  2017. Developing a comprehensive definition of sustainability. Implement. Sci. 12:1110 [Google Scholar]
  89. Najafizada SAM, Labonté R, Bourgeault IL. 89.  2017. Stakeholder's perspective: sustainability of a community health worker program in Afghanistan. Eval. Program Plan. 60:123–29 [Google Scholar]
  90. Nigg C, Geller K, Adams P, Hamada M, Hwang P, Chung R. 90.  2012. Successful dissemination of Fun 5—a physical activity and nutrition program for children. Transl. Behav. Med. 2:276–85 [Google Scholar]
  91. Niven DJ, Mrklas KJ, Holodinsky JK, Straus SE, Hemmelgarn BR. 91.  et al. 2015. Towards understanding the de-adoption of low-value clinical practices: a scoping review. BMC Med 13:255 [Google Scholar]
  92. O'Loughlin J, Renaud L, Richard L, Gomez LS, Paradis G. 92.  1998. Correlates of the sustainability of community-based heart health promotion interventions. Prev. Med. 27:702–12 [Google Scholar]
  93. Pagoto S, Waring ME, May CN, Ding EY, Kunz WH. 93.  et al. 2016. Adapting behavioral interventions for social media delivery. J. Med. Internet Res. 18:e24 [Google Scholar]
  94. Palinkas L, Cooper B. 94. 2017 Mixed methods evaluation in dissemination and implementation science. See Ref. 17 335–54
  95. Palinkas LA, Chavarin CV, Rafful CM, Um MY, Mendoza DV. 95.  et al. 2015. Sustainability of evidence-based practices for HIV prevention among female sex workers in Mexico. PLOS ONE 10:e0141508 [Google Scholar]
  96. Perkins DF, Feinberg ME, Greenberg MT, Johnson LE, Chilenski SM. 96.  et al. 2011. Team factors that predict to sustainability indicators for community-based prevention teams. Eval. Program Plan. 34:283–91 [Google Scholar]
  97. Peters D, Tran N, Adam T. 97.  2013. Implementation Research in Health: A Practical Guide Geneva: WHO Press
  98. Peterson AE, Bond GR, Drake RE, McHugo GJ, Jones AM, Williams JR. 98.  2014. Predicting the long-term sustainability of evidence-based practices in mental health care: an 8-year longitudinal analysis. J. Behav. Health Serv. Res. 41:337–46 [Google Scholar]
  99. Pluye P, Potvin L, Denis J-L. 99.  2004. Making public health programs last: conceptualizing sustainability. Eval. Program Plan. 27:121–33 [Google Scholar]
  100. Pluye P, Potvin L, Denis J-L, Pelletier J. 100.  2004. Program sustainability: focus on organizational routines. Health Promot. Int. 19:489–500 [Google Scholar]
  101. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC. 101.  et al. 2017. Methods to improve the selection and tailoring of implementation strategies. J. Behav. Health Serv. Res. 44:2177–94 [Google Scholar]
  102. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL. 102.  et al. 2015. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement. Sci. 10:21 [Google Scholar]
  103. Proctor E, Luke D, Calhoun A, McMillen C, Brownson R. 103.  et al. 2015. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement. Sci. 10:88 [Google Scholar]
  104. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G. 104.  et al. 2011. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. Policy Mental Health Mental Health Serv. Res. 38:65–76 [Google Scholar]
  105. Rabin BA, Brownson RC, Haire-Joshu D, Kreuter MW, Weaver NL. 105.  2008. A glossary for dissemination and implementation research in health. J. Public Health Manag. Pract. 14:117–23 [Google Scholar]
  106. Rabin BA, Lewis CC, Norton WE, Neta G, Chambers D. 106.  et al. 2016. Measurement resources for dissemination and implementation research in health. Implement. Sci. 11:42 [Google Scholar]
  107. Racine DP. 107.  2006. Reliable effectiveness: a theory on sustaining and replicating worthwhile innovations. Adm. Policy Mental Health Mental Health Serv. Res. 33:356–87 [Google Scholar]
  108. Rhoades BL, Bumbarger BK, Moore JE. 108.  2012. The role of a state-level prevention support system in promoting high-quality implementation and sustainability of evidence-based programs. Am. J. Community Psychol. 50:386–401 [Google Scholar]
  109. Richardson WC, Berwick DM, Bisgard J, Bristow L, Buck C, Cassel C. 109.  2001. Crossing the Quality Chasm: A New Health System for the 21st Century Washington, DC: Natl. Acad. Press
  110. Rogers EM. 110.  1995. Diffusion of Innovations New York: Free Press
  111. Sarriot E, Morrow M, Langston A, Weiss J, Landegger J, Tsuma L. 111.  2015. A causal loop analysis of the sustainability of integrated community case management in Rwanda. Soc. Sci. Med. 131:147–55 [Google Scholar]
  112. Sarriot EG, Winch PJ, Ryan LJ, Bowie J, Kouletio M. 112.  et al. 2004. A methodological approach and framework for sustainability assessment in NGO‐implemented primary health care programs. Int. J. Health Plan. Manag. 19:23–41 [Google Scholar]
  113. Saunders RP, Pate RR, Dowda M, Ward DS, Epping J, Dishman RK. 113.  2012. Assessing sustainability of Lifestyle Education for Activity Program (LEAP). Health Educ. Res. 27:319–30 [Google Scholar]
  114. Savaya R, Spiro S, Elran-Barak R. 114.  2008. Sustainability of social programs: a comparative case study analysis. Am. J. Eval. 29:478–93 [Google Scholar]
  115. Scheirer MA. 115.  1990. The life cycle of an innovation: adoption versus discontinuation of the fluoride mouth rinse program in schools. J. Health Soc. Behav. 31:2203–15 [Google Scholar]
  116. Scheirer MA. 116.  2005. Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am. J. Eval. 26:320–47 [Google Scholar]
  117. Scheirer MA. 117.  2013. Linking sustainability research to intervention types. Am. J. Public Health 103:e73–80 [Google Scholar]
  118. Scheirer MA, Dearing JW. 118.  2011. An agenda for research on the sustainability of public health programs. Am. J. Public Health 101:2059–67 [Google Scholar]
  119. Scheirer MA, Hartling G, Hagerman D. 119.  2008. Defining sustainability outcomes of health programs: illustrations from an on-line survey. Eval. Program Plan. 31:335–46 [Google Scholar]
  120. Scheirer MA, Santos SLZ, Tagai EK, Bowie J, Slade J. 120.  et al. 2017. Dimensions of sustainability for a health communication intervention in African American churches: a multi-methods study. Implement. Sci. 12:43 [Google Scholar]
  121. Schell SF, Luke DA, Schooley MW, Elliott MB, Herbers SH. 121.  et al. 2013. Public health program capacity for sustainability: a new framework. Implement. Sci. 8:15 [Google Scholar]
  122. Shediac-Rizkallah MC, Bone LR. 122.  1998. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ. Res. 13:87–108 [Google Scholar]
  123. Shelton RC, Charles T-A, Dunston SK, Jandorf L, Erwin DO. 123.  2017. Advancing understanding of the sustainability of lay health advisor (LHA) programs for African-American women in community settings. Transl. Behav. Med. 7:3415–26 [Google Scholar]
  124. Shelton RC, Dunston SK, Leoce N, Jandorf L, Thompson HS. 124.  et al. 2016. Predictors of activity level and retention among African American lay health advisors (LHAs) from the National Witness Project: implications for the implementation and sustainability of community-based LHA programs from a longitudinal study. Implement. Sci. 11:41 [Google Scholar]
  125. Singh M, Gmyrek A, Hernandez A, Damon D, Hayashi S. 125.  2017. Sustaining Screening, Brief Intervention and Referral to Treatment (SBIRT) services in health‐care settings. Addiction 112:92–100 [Google Scholar]
  126. Slaghuis SS, Strating MM, Bal RA, Nieboer AP. 126.  2011. A framework and a measurement instrument for sustainability of work practices in long-term care. BMC Health Serv. Res. 11:314 [Google Scholar]
  127. Spoth R, Guyll M, Redmond C, Greenberg M, Feinberg M. 127.  2011. Six‐year sustainability of evidence‐based intervention implementation quality by community‐university partnerships: the PROSPER study. Am. J. Community Psychol. 48:412–25 [Google Scholar]
  128. Stanton B, Guo J, Cottrell L, Galbraith J, Li X. 128.  et al. 2005. The complex business of adapting effective interventions to new populations: an urban to rural transfer. J. Adolesc. Health 37:163 [Google Scholar]
  129. Steckler A, Goodman RM. 129.  1989. How to institutionalize health promotion programs. Am. J. Health Promot. 3:34–43 [Google Scholar]
  130. Steckler A, Goodman RM, McLeroy KR, Davis S, Koch G. 130.  1992. Measuring the diffusion of innovative health promotion programs. Am. J. Health Promot. 6:214–24 [Google Scholar]
  131. Stetler CB, Ritchie JA, Rycroft-Malone J, Schultz AA, Charns MP. 131.  2009. Institutionalizing evidence-based practice: an organizational case study using a model of strategic change. Implement. Sci. 4:78 [Google Scholar]
  132. Stirman SW, Dearing JW. 132. 2018 Sustainability of cancer interventions. Optimizing the Cancer Control Continuum: Advancing Implementation Research D Chambers, C Vinson, W Norton New York: Oxford Univ. Press. In press [Google Scholar]
  133. Stirman SW, Finley EP, Shields N, Cook J, Haine-Schlagel R. 133.  et al. 2017. Improving and sustaining delivery of CPT for PTSD in mental health systems: a cluster randomized trial. Implement. Sci. 12:32 [Google Scholar]
  134. Stirman SW, Gutner C, Crits-Christoph P, Edmunds J, Evans AC, Beidas RS. 134.  2015. Relationships between clinician-level attributes and fidelity-consistent and fidelity-inconsistent modifications to an evidence-based psychotherapy. Implement. Sci. 10:115 [Google Scholar]
  135. Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. 135.  2012. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement. Sci. 7:17 [Google Scholar]
  136. Stirman SW, Matza A, Gamarra J, Toder K, Xhezo R. 136.  et al. 2015. System-level influences on the sustainability of a cognitive therapy program in a community behavioral health network. Psychiatr. Serv. 66:734–42 [Google Scholar]
  137. Stirman SW, Miller CJ, Toder K, Calloway A. 137.  2013. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implement. Sci. 8:65 [Google Scholar]
  138. Stoll S, Janevic M, Lara M, Ramos-Valencia G, Stephens TB. 138.  et al. 2015. Peer reviewed: a mixed-method application of the Program Sustainability Assessment Tool to evaluate the sustainability of 4 pediatric asthma care coordination programs. Prev. Chronic Dis. 12:E214 [Google Scholar]
  139. Swain K, Whitley R, McHugo GJ, Drake RE. 139.  2010. The sustainability of evidence-based practices in routine mental health agencies. Community Mental Health J 46:119–29 [Google Scholar]
  140. Swerissen H, Crisp BR. 140.  2004. The sustainability of health promotion interventions for different levels of social organization. Health Promot. Int. 19:123–30 [Google Scholar]
  141. Tabak RG, Duggan K, Smith C, Aisaka K, Moreland-Russell S, Brownson RC. 141.  2016. Assessing capacity for sustainability of effective programs and policies in local health departments. J. Public Health Manag. Pract. 22:2129–37 [Google Scholar]
  142. Tricco AC, Cogo E, Ashoor H, Perrier L, McKibbon KA. 142.  et al. 2013. Sustainability of knowledge translation interventions in healthcare decision-making: protocol for a scoping review. BMJ Open 3:e002970 [Google Scholar]
  143. Vamos S, Mumbi M, Cook R, Chitalu N, Weiss SM, Jones DL. 143.  2014. Translation and sustainability of an HIV prevention intervention in Lusaka, Zambia. Transl. Behav. Med. 4:141–48 [Google Scholar]
  144. Van Acker R, De Bourdeaudhuij I, De Cocker K, Klesges LM, Willem A, Cardon G. 144.  2012. Sustainability of the whole-community project “10,000 Steps”: a longitudinal study. BMC Public Health 12:155 [Google Scholar]
  145. Welsh JA, Chilenski SM, Johnson L, Greenberg MT, Spoth RL. 145.  2016. Pathways to sustainability: 8-year follow-up from the PROSPER project. J. Prim. Prev. 37:263–86 [Google Scholar]
  146. Woodbridge MW, Sumi WC, Yu J, Rouspil K, Javitz HS. 146.  et al. 2014. Implementation and sustainability of an evidence-based program: lessons learned from the PRISM applied to First Step to Success. J. Emot. Behav. Disord. 22:95–106 [Google Scholar]
  147. Wright DB. 147.  2009. Care in the country: a historical case study of long-term sustainability in 4 rural health centers. Am. J. Public Health 99:1612–18 [Google Scholar]
  148. Yancey AK, Ortega AN, Kumanyika SK. 148.  2006. Effective recruitment and retention of minority research participants. Annu. Rev. Public Health 27:1–28 [Google Scholar]
  149. Yin RK. 149.  1981. Life histories of innovations: how new practices become routinized. Public Adm. Rev. 41:121–28 [Google Scholar]
  150. Zakumumpa H, Bennett S, Ssengooba F. 150.  2017. Modifications to ART service delivery models by health facilities in Uganda in promotion of intervention sustainability: a mixed methods study. Implement. Sci. 12:45 [Google Scholar]
/content/journals/10.1146/annurev-publhealth-040617-014731
Loading
/content/journals/10.1146/annurev-publhealth-040617-014731
Loading

Data & Media loading...

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error