Abstract
Background Remote interventions known as telemental health care increased in use due to the COVID-19 pandemic when social distancing requirements were in place. Whilst there is some evidence regarding the cost-effectiveness of telemental health prior to the pandemic, there is a need for further evaluation due to the increase in remote care.
Aims To systematically review the literature to explore whether remote mental health care interventions are cost-effective in terms of incremental cost per quality adjusted life year and in relation to condition specific outcomes compared to usual care or an alternative intervention.
Method A multilayer search strategy was conducted to build on the searches of a previous systematic review, as well as including grey literature and economic models. Six databases (PubMed, EMBASE, Cochrane Central, PsychINFO, CINAHL, and EconLit) were searched for literature relating to the cost effectiveness of telemental health. Quality appraisal was conducted for all included studies, and findings were synthesised using narrative synthesis.
Results 7386 studies were identified of which 59 met our inclusion criteria and were included in the synthesis of findings. 45 studies were rated as very good or excellent quality. Of the 59 included studies, 40 indicated that the telemental health intervention was cost-effective, whilst a further 16 suggested the intervention had potential to be cost-effective, but there was some uncertainty in the findings. Three studies reported that the intervention was not cost-effective.
Conclusions This evidence will be used to inform practice in the UK as we respond to and recover from the COVID-19 pandemic.
Introduction
Remote interventions in mental health care, or “tele-mental health” have been implemented for many years and evaluations often include assessment of cost-effectiveness. Telemental health care is defined as any interventions or modes of working where remote technology (e.g. telephone/video call/instant messaging) is used to facilitate direct communication between staff and service users, between service users and peers, or between mental health professionals in different locations.
Telemental health has the potential to result in benefits for both service users and clinicians. For example, service users have identified increased flexibility as a key advantage of remote care (1), and a recent rapid realist review (2) found that telemental health care can improve access to mental health support for people who may struggle to travel to face-to-face appointments due to disability, anxiety about travel, or caring or work responsibilities.
Reported benefits for staff included an opportunity for more flexible working, less time spent travelling, and communicating with other clinicians at different sites (2, 3). However, it is important to note that telemental health is not suitable for everyone; not all service users may wish for care to be remote and inequalities and digital exclusion should be taken into consideration during the decision to offer mental health care remotely (1, 2). Clinicians have also reported finding telemental health less suitable than face-to-face care when treating trauma, or for service users in crisis or those with psychotic symptoms or severe anxiety (3).
The uptake of telemental health care increased rapidly during the COVID-19 pandemic as services moved from largely face-to-face models of care due to social distancing requirements (4, 5). However, a systematic review conducted during the early stages of the pandemic identified a lack of evidence on the cost-effectiveness of telemental health interventions (4).
Pre-pandemic evidence regarding the cost-effectiveness of telemental health was also inconclusive. An umbrella review found that in studies conducted prior to the COVID-19 pandemic, evidence regarding the cost-effectiveness of telemental health was mixed (6).
Some studies identified lower costs of telemental health care due to savings on travel time for service users, or not needing to take time off work, whereas higher costs in others were attributed to the expense of videoconferencing equipment (6). A further systematic review by Naslund et al (7) identified 26 economic evaluations of telepsychiatry programmes, also prior to the COVID-19 pandemic. They found that 60% reported telepsychiatry programmes to be less expensive and 32% reported they were more expensive than usual care, again primarily due to costs of technology and equipment. This review was limited to peer-reviewed studies that used primary data collection and therefore economic models were excluded.
This proposed review therefore aims to build on Naslund et al (7) and bring together further evidence on economic evaluations of remote mental healthcare. We originally set out to answer the following secondary review questions: (i) Does cost-effectiveness of remote mental health care interventions differ by subgroup? (ii) Does cost-effectiveness of remote mental health care interventions differ by model type? (iii) What is the impact of remote technology for staff communication on costs for the health and social care system? (iv) What is the impact of remote technology for mental healthcare interventions on costs for the health and social care system? (v) How is the amount of contact time between service users and mental health care professionals affected by remote interventions? However, given the data reported in the selected studies only research questions ii and iv will be addressed in this paper.
Methods
The review was registered on International prospective register of Systematic Reviews (PROSPERO) ID: CRD42020216755. We developed a multilayer search strategy to capture both new evidence published since the completion of the Naslund et al. (7) searches on 16 March 2018 and to cover additional sources of information excluded by these authors, namely grey literature and simulation models. We used the same search criteria as outlined in Naslund et al. (7) to identify any relevant peer reviewed literature published since 16 March 2018. The following six databases were searched on 8-10 December 2020: PubMed (MEDLINE), EMBASE, Cochrane Central, PsychINFO (EBSCOhost), CINAHL, and EconLit1.
An example of the search terms used, as per Naslund et al. (7), is reported in Table S12 of the Supplementary Material. To identify economic models, which were excluded by Naslund et al. (7), the same strategy was used but with a different date range (January 2000 – October 2020) and two additional search terms added to Table S1: “model OR simulation”. Grey literature databases were searched from 1 January 2000 to 31 October 2020 for terms related to “tele” “mental health” and “costs” as outlined in Table S2. MedNAR and Google were also searched using the following terms to improve relevance of results: “cost effective” AND (tele* OR telehealth OR telepsychiatry OR remote OR technology OR digital) AND mental. Records identified in the search were uploaded to Rayyan QCRI, a web-based reference manager system for collaborative systematic review, for de-duplication and blinded screening and study selection.
Study selection criteria
After de-duplication, the results were pre-screened by three student research assistants (MT, SS, and JP). A 25 per cent sample was selected at random and checked for accuracy by the second author (EK). The titles and abstracts of the remaining studies were screened in duplicate by three reviewers (AC, EK and JB) against the eligibility criteria shown in Table 1. Disagreements and records marked as undecided were resolved by discussion or a consensus involving a third author.
In the process of abstract and title screening, we identified 21 peer-reviewed cost effectiveness studies published prior to March 2018 which appeared eligible according to the study selection criteria outlined in Naslund et al. (7) but were not included in their review. Given that these studies were identified in grey literature databases not considered by Naslund et al. (7), we decided to assess them for inclusion along with the other studies retained for full text screening.
Where full text was not available, the corresponding authors were contacted. If an answer was not obtained following a reminder and a 3-month waiting period, the study was excluded. Reference list searching was undertaken by the second author (EK). The search and study selection process has been documented in Figure 1 using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses PRISMA guidelines (8).
PRISMA flow chart.
Data extraction and analysis
Partial data extraction was undertaken by three student research assistants (SS, HA and FB) and completed by two of the authors (EG and AC). For 25 per cent of studies a third team member extracted the data in duplicate. Following data extraction, the studies were sorted by intervention type, categorised into study design type (i.e., model/cost utility analysis or cost-effectiveness analysis), and we considered whether the evidence favoured the intervention or not. Given the heterogeneity of the retained studies meta-analysis was not undertaken. As with most reviews of economic evaluations, the results are presented in the form of a narrative synthesis (9).
Quality appraisal
We used the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) (10) checklist to assess different aspects of the retained studies’ quality and reporting. The checklist contains 24 items and accompanying recommendations (see Table S3). The overall quality of evidence was assessed using GRADE certainty ratings (11). Final quality was rated high, moderate, low, or very low. The quality assessment was completed by EK. A 25 per cent sample was selected at random and assessed in duplicate by AC. Where results differed, consensus was reached through discussion or the involvement of a third assessor (PM).
Results
The initial search identified 7386 papers. Duplicates were then removed and after screening by three research assistants (MT, SS, and JP) a further 2462 records were excluded. The random sample of 25% used for checking revealed no conflicts. After the duplicate screening of remaining papers, disagreements (n=26) and records marked as undecided (n=77) were resolved. A total of 59 unique studies met the inclusion criteria (Figure 1). Eight full text papers could not be obtained and reference lists revealed no extra studies. Search 1 identified four grey literature reports, search 2 identified 41 unique studies from the updated Naslund et al (7) search, and search 3 found a further 14 studies of economic models. The non-modelling studies enrolled 5-1514 participants. The USA was the setting for most studies (16), followed by Sweden (10) and the UK (8). For more detailed study characteristics see Table 3.
Summary of GRADE certainty ratings.
Study characteristics.
The majority (n=40) of included studies found that the telemental health intervention was cost-effective. A further 16 studies indicated that there was some uncertainty around whether the intervention was more cost-effective than the comparator, or that the intervention had potential to be cost-effective but there were some biases in how the study was conducted. Only three studies found that the intervention was less cost-effective than the comparator.
Table 3 shows that the most common interventions were internet-based therapy and videoconferencing. Twenty-one studies evaluated some form of internet-based/delivered therapy (usually cognitive behavioural therapy). QALYs were used in 13 of these studies, DALYs in two studies, productivity losses in one study, and clinically specific measures in the remaining studies. In eight studies the costs for the intervention were lower than the comparator and outcomes were better, indicating a situation of dominance (12–19). In five studies where the intervention had higher costs and produced better outcomes, the ICER was relatively low and indicated cost-effectiveness (20–24). Two studies indicated that therapy was not cost-effective (25, 26).
Videoconferencing and video support to clinicians and patients was evaluated in eight studies. Four of these did not use an outcome measure in the economic analysis and all showed cost savings (27–30). Two other studies reported similar outcomes and lower costs (31, 32). One study, evaluating video coaching for carers of children with behavioural problems, found it to cost saving and outcome improving (27). Another study, found videoconferencing to result in costs per improvement in depression of $222 and per improvement in competence of $437 (33).
Studies that evaluated other remote working interventions usually demonstrated costs savings, dominance (costs savings and better outcomes), or higher costs and better outcomes that justified the extra expense. The study by Pot-Kolder et al (34) found the cost per QALY of virtual reality-based CBT for people with psychosis and paranoid symptoms to be nearly €50,000. However, this was deemed to be cost-effective based on a threshold of €80,000 per QALY for conditions such as this.
The number of studies taking a societal perspective was 27 with 22 taking a healthcare perspective. QALYs were reported in 31 studies and DALYs in four. The reporting quality of each included article was assessed using the 24-item CHEERS checklist: the proportion of studies that met the criteria for each item is shown in Figure 2. 28 studies were of excellent quality, meaning that they addressed satisfactorily at least 90% of applicable checklist items (12, 13, 15–17, 20–22, 24–27, 31, 34–47). 17 studies (14, 19, 23, 48–61) were of very good quality (70-<90%), and further five studies (18, 62–65); had an acceptable quality of reporting (> 60%). The remaining nine studies failed to address or addressed only partially ten or more relevant checklist items. The least reported items were study perspective (item # 6), currency, price date, and conversion (item # 14), the effects of uncertainty (item # 20), and potential conflicts of interest (item # 24). About 40 % of the studies failed to describe fully the characteristics of the base-case population (item # 4), and the sources and methods used to establish clinical effectiveness (item # 11). Similarly, only 80% of the studies reported in sufficient detail the approaches and data sources used to estimate costs (item # 13) and the analytical methods which supported their evaluation (item # 17). Data for subgroup analyses was not available in most cases (item # 21). Finally, approximately half of the studies either did not name its funding source or failed to disclose the funder’s role in the identification, design, conduct, and reporting of the analysis. The detailed performance of each included study for the CHEERS checklist (10) is shown in Table S4 of the Supplement. The overall quality of evidence was assessed using GRADE certainty ratings (11). The assigned ratings shown in Table 2 were based on the quality of the economic evidence.
Proportion of included studies that complied with applicable items of the Consolidated Health Economic Evaluation Reporting Standard (CHEERS) checklist.
Overall, there was an over reliance on single-study based estimates. The certainty of evidence was downgraded due to several factors, including imprecision (effect estimates coming from one study with a small sample), risk of bias (narrow perspective, cost omissions related to comorbidity, healthcare utilisation and other relevant aspects, losses to follow-up, short time horizon and limited sensitivity analysis), indirectness (cost estimates based on secondary sources rather than actual resource use, and high percentage of missing data imputed) and potential sponsor bias. The main reasons for downgrading for each study rated as moderate or below are summarised in Table 2.
Discussion
This systematic review identified 59 studies to build on the findings from Naslund et al (7) regarding the impact of remote technology for mental healthcare interventions in terms of costs and cost-effectiveness. The review by Naslund et al (7) included 26 studies and found good evidence for cost savings associated with telepsychiatry but somewhat limited evidence of cost-effectiveness. We included a broader range of studies and again these suggest that remote working can result in reduced costs but we also identified a reasonable number of full economic evaluations where remote working was either dominant (cost saving and outcome improving) or to result in incremental cost-effectiveness ratios that are below acceptable thresholds. However, as with Naslund et al (7), the studies were markedly heterogeneous with very different designs and perspectives.
The greatest number of studies evaluated some form of internet-delivered therapy (usually cognitive behavioural therapy). These tended to be cost effective as were the various forms of videoconferencing (although many of the latter were limited to cost comparisons).
The perspective taken in the studies was fairly evenly split between a societal perspective (generally meaning that lost work time was included) and a healthcare perspective. This is likely to reflect both the expected benefits of remote working but also the different ways in which healthcare decisions are made in each country. While the latter consideration is important, from an economic point of view we might expect remote working to save patient time and this may include time in work. As such, a societal perspective seems appropriate in this area.
Decision makers often base recommendations on the relationship between costs and QALYs and the studies reviewed here reflect this with 31 using QALYs as the main economic outcome measure. This is interesting especially given some concerns over the use of QALYs in mental health research (66, 67). Most of these concerns though are with the sue of QALYs in studies of interventions for schizophrenia and other severe mental illnesses. As stated above, internet-based cognitive behavioural therapy was most commonly evaluated, and this usually provided for people with depression or anxiety. QALYs do tend to work reasonably well for these conditions.
Implications
Evaluations of remote working interventions for mental health problems have been criticised for being methodologically limited and this may have held back developments in this area (68). It is from this review that the amount of cost-effectiveness research has increased substantially, and the evidence base is getting stronger. However, methodological issues persist.
The COVID-19 pandemic has clearly led to remote working being given more prominence. While services will to some extent revert back to usual ways of working, some aspects of care delivery that have emerged since early 2020 will most likely remain. In the UK, the charity Mind has identified challenges including the extent to which people have good access to engage with digital approaches, the quality of care delivered in this way not always being of a high standard, and potential breaches of confidentiality (69). As demands on health services continue to increase, it is likely that innovations such as those reviewed here will be needed more and more and it is thus encouraging that on balance they do appear to represent reasonable value for money.
Limitations
We based our search terms on those used previously by Naslund et al (7). This was appropriate in that we were wanting to update that review but also to include studies from a broader range of sources. However, the strategy may have been too specific and might have missed economic models published prior to 2018. Some terms were excluded that could have been relevant, particularly those relating to the use of social media and platforms such as Zoom or Microsoft Teams.
Another limitation is that some interventions/services only included remote working as a component of a wider package of care or support. It was appropriate to include such studies in order to obtain a broad overview of remote working but identifying the specific impact of remote working is challenging in such studies.
As with other reviews of economic evaluations, included studies were very heterogeneous. This meant that we could only provide a narrative review rather than a more formal synthesis of findings.
Conclusion
This review has found numerous studies assessing the costs and cost-effectiveness of remote working interventions in mental health. Such approaches appear to be cost-effective although methodological quality of studies needs to be enhanced.
Data Availability
This is a systematic review and no primary data have been colelcted.
Data Availability
Data availability is not applicable to this article as no new data were created or analysed in this study.
Author Contributions
The study was developed by AC and PM. Papers were identified and data extracted by AC, EK, EG, and PM. The manuscript was written by AC, RA, EK, EG and PM.
Funding
This paper presents independent research commissioned and funded by the National Institute for Health Research (NIHR) Policy Research Programme, conducted by the NIHR Policy Research Unit (PRU) in Mental Health. AC was funded through an NIHR research fellowship. The views expressed are those of the authors and not necessarily those of the NIHR, the Department of Health and Social Care or its arm’s length bodies, or other government departments.
Declaration of interest
None
Supplemental material
Naslund et al. (2020) search strategy used in Medline
Grey literature search strategy
Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist
CHEERS checklist: Results
Acknowledgements
Joseph Botham assisted with the initial search and data extraction. UCL post-graduate placement students provided invaluable contribution to this study. Initial title and abstract screening was completed by Magdalena Tomoskova, Jiping Mo and Sypros Spyridonids. Data extraction support was completed by Hana Afra, Spyros Spyridonidis and Fatima Bashir.
Footnotes
↵1 The following databases used in Naslund et al. (2020) were not included in this review due to organisational access and the number of duplicates already identified: Web of Science, Health Economic Evaluations Database, Cost-Effectiveness Analysis Registry, Research Papers in Economics (RcPEc), and European Network of Health Economic Evaluation Database (EURONHEED).
↵2 In CINAHL we were unable to apply MeSH terms, and in EconLit the simplified grey literature strategy was used (see Table S2).