Abstract
This systematic review aims to identify the Key Performance Indicators (KPIs) which are the most frequently used and important in the Performance Evaluation (PE) of Health Care Organizations (HCOs). Also, it aimed to analyze the resulted Balanced Scorecard (BSC) dimensions during the Coronavirus Disease 2019 (COVID-19) era. This systematic review was prepared according to PRISMA guidelines. PubMed, Embase, Cochrane, and Google Scholar databases, as well as Google search engine, were inspected to find all implementations of BSC at HCOs until 20 September 2020. The Risk of Bias (RoB) was assessed for each included article. The eligible studies were 33, in which 36 BSC implementations were identified. The categorization and re-grouping of the 797 KPIs resulted in 13 major-dimensions: The financial, efficiency and effectiveness, availability and quality of supplies and services, managerial tasks, Health Care Workers (HCWs) scientific development error-free and safety, the time, HCW-centeredness, patient-centeredness, technology and information system, community care and reputation, HCO building, and the communication. Under them, 45 sub-dimension were identified. The high RoB in most implementations was due to the selection bias. None of the 36 BSC implementations was utilized during the COVID-19 pandemic. This review solves the dilemma of categorization differences among BSC implementations, which in turn will improve the data sharing and comparability among studies. A modification for the BSC design to include external and managerial perspectives is required. However, a lack of a comprehensive PE for HCOs during the pandemic was observed, so the resulted dimensions and KPIs are recommended for planning, monitoring, evaluation, and continuous improvement of HCOs. Many dimensions were negatively influenced by the pandemic. However, some dimensions still need further investigation. BSC dimensions have an essential role in tackling the COVID-19 pandemic and will help the health care managers to mitigate its consequences on HCOs.
1. Introduction
Evaluating the health care sector is quite challenging and complex. The unsatisfactory performance can result from long Waiting Time (WT), in-efficiency, dissatisfied patients, and the burnout of Health Care Workers (HCWs) [1,2]. Coronavirus Disease 2019 (COVID-19) imposed further burdens on the health care system worldwide due to the limited capacity of hospital beds, and the increased psychological stress of HCWs during the COVID-19 pandemic [3,4]. There is still a lack of information that would help health care managers and policymakers in the era of COVID-19 to improve the delivery of health care quality and to learn for the future [5]. Higher pandemic burdens such as HCWs’ burnout and stress are expected when the Health Care Organizations (HCOs) lack plans and preparedness to strengthen their surge capacity and HCWs’ resilience [6,7].
1.1 Key Performance Indicators (KPIs) before COVID-19 pandemic
HCOs employed different tools for the Performance Evaluation (PE) with a variety of KPIs. The most utilized PE tools were the International Organization for Standardization (ISO standards), Malcolm Baldrige National excellence model (MBNQA), European Foundation for Quality Management (EFQM) excellence model, Singapore Quality Award (SQA), Six Sigma, Data Envelopment Analysis (DEA), Pabon Lasso model, and Balanced Scorecard (BSC) [8–12].
Performance Assessment Tool for quality improvement in Hospital (PATH), was a project initiated by World Health Organizations (HCOs) in 2003. It aimed at developing a framework for the assessment of hospitals’ performance. The resulted dimensions from this project were: clinical effectiveness, efficiency, HCWs’ orientation, responsive governance, safety, patient-centeredness. However, studies showed that there are still some gaps in this model, and issues concerning the dimensions investigated [13,14]. Also, Organization for Economic Co-operation and Development (OECD) launched Health Care Quality Indicator (HCQI) project in 2006, it aimed to develop KPIs to compare quality in health care at the international level and to achieve international benchmarking. This project concluded that health care has to be safe, effective, patient-centered, timely, efficient, equitable, acceptable, and accessible [15,16].
Most of the above-mentioned managerial tools mainly focused on the KPIs related to quality, efficiency, productivity, and timeliness dimensions [8–12,17]. Each of them can be considered as a dimension at the internal perspective of the BSC, which consists of 4 perspectives: the internal process, customer, innovation and growth perspectives, and financial perspectives [18]. Dimensions are collections of homogeneous or related KPIs which were also referred to as Diagnostic Related Groups (DRG) [19]. Dimensions allowed a comparison of performance across hospitals and had a positive impact on improving efficiency [19].
The use of KPIs in the health care system before the pandemic has been beneficial for many reasons. First, increasing the satisfaction rates of patients and HCWs. Second, they lead to better efficiency, effectiveness, and financial performance, as well as adapting to new technologies and ideas. Third, they lead to higher productivity and profitability [20–22]. In the pandemic, it is also important for the HCO to track the performance of KPIs, which could draw faster attention to the areas that require rapid responses and strengthening [6].
BSC was considered different from the other managerial tools for 2 reasons. First, it offers a holistic approach to PE, since it allows the managers to highlight both the financial and the non-financial metrics. The first generation of the BSC, unveiled by Kaplan and Norton in 1992, involved 4 perspectives: the financial, customer, internal process, and innovation and growth perspectives, steered by the organizational vision and strategy [18]. Second, the BSC is not only planning or a PE tool. It is also a strategic managerial tool, as it assigns KPIs which are compatible with the HCO strategy [23,24]. However, other PE tools such as Total Quality Management (TQM) tools lack these thorough properties [25]. Later on, the BSC second generation was developed to include strategic maps, in which cause-effect cascades between perspectives or KPIs were inspected. In the third generation of BSC, a destination statement was incorporated, which evokes where does the organization plan to go within a time horizon, and what are the action plans to achieve each targeted objective [26]. In health care, BSC was initially implemented in 1997 at Duke Children’s Hospital in the United States of America (USA), and as a result of this use, the hospital was able to convert USD 11M of losses into USD 4M of profits after 4 years of implementation [27]. Since then, BSC has gained increasing attention, and many HCOs in high-income countries, as well as low- and middle-income countries, utilized BSC to strategically develop their organizations [28–32].
Most of the available PE models mainly focus on the internal perspective but lacks covering the other dimensions or perspectives that are also important according to the health care managers. Reviewing the previous BSC reviews, we noticed that they only focus on the general narration for which perspectives or KPIs were used [33–43]. However, they lack being systematic in the categorization of KPIs.
1.2 KPIs during COVID-19 pandemic
One of the first studies to assess the impact of COVID-19 on KPIs [44], analyzed the impact of COVID-19 on pancreato-biliary endoscopy-related KPIs. However, limited research was found to assess the HCOs -related KPIs at COVID-19 in general. Health policy experts stated that the current approach to quality measurement in the COVID-19 era is challenged by the insufficient standardization of data for sharing purposes, so the comparison between the performance of the health care systems is disrupted [5]. It was considered that this is especially important in cases where the optimal performance is not fully understood as in pandemics, and a comparison with other health systems would be informative and necessary [5]. So, addressing the lack of data standardization was suggested to be established by quickly defining measures, which could allow health systems, at least in the short term, to use standardized methods to better understand their performance [5].
In correspondence with these requirements, this research aims at a) finding all the perspectives, dimensions, and KPIs which were employed in BSC implementations and re-categorizing them, b) ranking dimensions according to their frequency of use by HCOs across the world, and c) ranking dimensions according to their importance from the health care perspective d) analyzing the impact of COVID-19 on BSC’s perspectives, dimensions, and sub-dimensions.
2. Materials and methods
This systematic review was conducted according to the 27-point checklist of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist [45], see Appendix (S1).
2.1. Eligibility Criteria
The inclusion and exclusion criteria were set as shown in Table (1).
Table (1) to be placed here.
2.2. Data Sources, Search Strategy, and Study Selection
First, a search strategy was developed for the PubMed database, see Table (1). Then, the strategy was adapted for Embase, Cochrane CENTRAL, and Google Scholar databases. Furthermore, an additional search in the Google engine was performed to find grey literature or unpublished papers; including theses and conference abstracts. Additionally, the reference lists of all the eligible articles were checked by the 1st author. The databases were searched until September 20, 2020. The search strategy was developed by the 1st, 2nd, and 4th authors, the first 2 authors are experts in health care management and BSC, while the 4th author is an expert in systematic reviews and meta-analysis. To widen the search frame and get more results, the search strategy was developed to include both Medical Subject Headings (MeSH) terms and keywords. Then appropriate truncation and relevant indexing terms were used. See Appendix (S2) which shows all search strategies in all databases. Afterward, the 1st author conducted an electronic database search and removed the duplicates by using the EndNote X9.2 program.
The selection of eligible studies was done by the 1st and the 2nd authors independently in all the steps. Disagreements were resolved by discussion after each step or, if necessary, through arbitration by the 4th author. Firstly, the articles’ titles and abstracts were examined to eliminate irrelevant papers. Then, full texts were carefully inspected to decide the final papers’ inclusion list. If different KPIs were used in more than 1 implementation in the same study, each was counted as a different implementation. Whereas, implementations using the same KPIs in different locations or times in the same study were considered as 1 implementation. Authors of studies with no available full texts or with partially reported results were contacted for missing data.
2.3 Data Collection Process
Data extraction was done by the 1st and 2nd authors independently, then compared to discuss differences. The following data were extracted from the eligible studies: 1) author/s, 2) year of publication, 3) country of origin, 4) data collection duration, 5) data collection tool, 6) psychometric properties, 7) the number of perspectives, 8) the number of KPIs, 9) availability of weights for perspectives or KPIs, and 10) outcome which is represented in the KPIs that have been used and their weights. For that, the frequency of use for each KPI at each implementation was plotted on Microsoft Excel and the sum was calculated. In addition, weights assigned for each KPI at each implementation were reported on a scale of 100%. While in the case of studies that did not assign weights explicitly, each KPI weight was calculated by dividing 1 at the number of used KPIs in that study, to assign an equal weight for each KPI there. Consequently, an average of the weights assigned for each KPI has been computed by the 1st and 2nd authors independently. Next, re-grouping and coding for the KPIs were performed by the 1st author, to find the frequency of use and the assigned weights percentages for each dimension. Then the resulted major and sub-dimensions were listed and described.
2.4 The Risk of Bias (ROB)
The 1st and the 2nd authors evaluated the resulted 36 implementations separately to assess: the selection bias, performance bias, attrition bias, detection bias, and reporting bias for each implementation. Each Risk of Bias (RoB) was categorized into 3 categories according to the Cochrane Handbook for Systematic Reviews: high risk, low risk, and unclear risk [46]. Then the total risk was categorized for each study as high risk, medium risk, or low risk. The assessments were compared; in case of disagreement, the 4th author was consulted. Summary of RoB was prepared using RevMan 5 [47].
3. Results/ Discussion
3.1 Study Selection
A total of 4028 records have resulted from running the search strategy in the 4 databases. In addition, another 3 records were identified through Google search. So, a total of 4031 records have resulted. Duplicates were removed (n=1046) using the EndNote program, then the remaining articles were screened based on their titles and abstracts (n=2985). Irrelevant papers were excluded (n=2794). Consequently, the remaining 191 studies were examined by reading the full texts. Among these papers, 22 papers were written in non-English languages including Spanish, German, French, Chinese, and Persian, for which a full-text translation was performed for each paper to decide whether to include or exclude any of them, see Appendix (S3). As a result of reading the full texts, 158 studies were excluded, and only 33 studies were eligible for this review in which 36 full implementations of different BSC designs were actually applied. Table (2) shows a summary of the 36 implementations. Details of the study selection process are shown in the PRISMA flow-chart [45]. See Fig. (1).
3.2 Study Characteristics
3.2.1 Implementation
After analysis of implementations, 2 studies [48,49] were found to include 3 and 2 implementations respectively with different KPIs per each. Thus, the 33 resulted studies contained 36 different implementations. No BSC implementation in the COVID-19 era was found.
3.2.2 Settings differences
21 of the implementations were performed in hospitals (2ry and 3ry HCOs) [41,49–67], and 15 in medical centers or health facilities (1ry HCOs) [30–32,48,68–76].
3.2.3 Study Design
All studies had a cross-sectional design, except 2 were retrospective longitudinal studies [65,70].
3.2.4 Language and Location
From the resulted 36 implementations, 1 was in Spanish, 1 was in Persian, and the rest were in English language. The 36 implementations were done in various countries; 19 in Asia [30–32,48,50–52,56–58,65,66,68,70,74,75,77], 7 in North America [55,60–62,64,67,71], 6 in Europe [49,53,54,59,63], 3 in Africa [69,72,76], and 1 without location information [73].
3.3 Perspectives Frequency of Use and Importance
A total of 797 KPIs were extracted from the resulted implementations. These KPIs were categorized in the studies under 15 perspectives. The average number of perspectives used per study was 4.5, and for the KPIs was 22. The top frequently used perspectives were: the internal, financial, patient, learning and growth, HCWs, managerial, community, and stakeholder perspectives. The total use frequency of these perspectives at the implementations were 29.6%, 17%, 12.6%, 12.6%, 9.4%, 6.3%, 5%, and 3.1% respectively. On the other hand, the topmost important perspectives from the health managers’ view of point were: the internal, financial, learning and growth, patient, HCWs, community, managerial, and stakeholder perspectives with a total weight of 37.9%, 15.4%, 12%, 11.3%, 7.8%, 7.7%, 3.6%, and 2.8% respectively, see Appendix (S4).
After re-grouping the 797 indicators, 45 sub-dimensions resulted. This figure shows the frequency and the weight for each sub-dimension independently.
Reassembling the 45 sub-dimensions resulted in 13 major-dimensions. This figure shows the frequency and the weight for each major dimension independently.
3.4 Categorization and Re-grouping of KPIs
In the categorization process, the 797 extracted KPIs were plotted according to their frequencies and weights. After re-grouping of these KPIs into homogenous major dimensions and sub-dimensions, 13 major-dimensions resulted, with 45 sub-dimensions. The resulted major- and sub-dimensions were listed and described in Table (3).
3.5 Dimensions/Sub-dimensions Frequency of Use and Importance
The frequencies in which the KPIs were used and the weights that have been assigned to each of them at these implementations were plotted separately in Appendix (S4). Re-grouping and re-categorization of KPIs resulted in Fig. (2 & 3), which respectively show 13 major dimensions and the 45 sub-dimensions based on their frequency of use and importance.
3.6 Risk of Bias (ROB)
The RoB were found to be medium in 9 implementations at 9 studies [30–32,59,61,74,76,78,79], low in 6 implementation at 4 studies [48,69,72,77], while it was high for the rest. The reporting bias was low in all implementations. The selection bias was high in almost 70% of the implementations. For more details, see Appendix (S5).
3.7 Implications of the resulted dimensions in the COVID-19 era
Although this systematic review was updated to include 10 months after the initiation of the COVID-19 pandemic, no research for BSC utilization in COVID-19 was found. Since this topic was not yet analyzed in the literature, the authors pursued a further analysis based on separate researches per each resulted dimension during the COVID-19 era.
3.7.1 The Financial dimension in COVID-19
Due to COVID-19 hospitalizations at the beginning of the pandemic, health policy experts suggested that HCOs in some regions will have more significant revenue as well as greater costs related to additional HCW and resources. In contrast, other hospitals will experience mostly sharp reductions in elective and outpatient revenues, which will create unprecedented financial challenges for HCOs [80]. However, in addition to the greater costs of HCWs and resources, researchers found higher costs of treatment occurred as a result of extra diagnostic tests and isolation costs [81].
In the United Kingdom (UK), the total expenditure on National Health System (NHS) has increased significantly during the pandemic [82]. The NHS made funding upgrades to expand waiting areas and treatment cubicles [83]. Some researches focused on the cost-effectiveness calculations. A study in South Africa indicated that purchasing Intensive Care Unit (ICU) capacity from the private sector during COVID-19 surges may not be a cost-effective investment [84]. Up to now, there is still a lack of studies that handled the financial dimension or developed cost-saving strategies at the health organization level in COVID-19.
3.7.2 The error-free and safety dimension in COVID-19
This dimension includes monitoring, analysis, and comparability of mortality rates, as well as investigation of its determinants in HCOs. Although mortality may not be directly related to errors, higher mortality rates than the average of HCOs can reveal an underlying error. A cohort study in Mexico City [85] found that the mortality rates at the hospital’s ICU and non-ICU departments were similar. The reason behind this finding was the ICU bed’s unavailability. Around 45% of the patients who did not survive did not get an ICU bed, which raised the mortality rate in the non-ICU admitted patients. However, this study revealed that the main cause of the non-ICU admitted patients was the Acute Respiratory Distress
Syndrome (ARDS). Whereas, the main cause of mortality for admitted patients was due to septic shock, followed by ARDS and multi-organ failure. The WHO has provided clear guidelines for Infection Control (IC) during health care when COVID-19 is suspected or confirmed [86]. Patient safety was investigated in a systematic review for Indian-related researches [87]. It found that patient safety was negatively impacted during the COVID-19 pandemic due to the inadequate preparation of the health care system, such as infrastructure, human and material resources. Also, researchers categorized diagnostic errors that could occur in the COVID-19 pandemic into 8 types and suggested how to reduce them [88].
However, many studies showed improvements in this dimension during the pandemic. A study in the UK [89] found a significant increase in the Safety Attitude Questionnaire (SAQ) scores of doctors and other clinical HCW, and no change in the nursing group. It also showed a significant decrease in error reporting after the onset of the COVID-19 pandemic. Another study in Iran [90] found that the Health-Associated Infections (HAIs) during the pandemic were reduced, which could be referred to the proper implementation of IC protocols. This is supported by a study in Ghana [91] which found that the HCW compliance to IC measures was high during the pandemic.
Health Waste (HW) management sub-dimension was intensively investigated due to the tremendous increase of HW volume during the pandemic [92]. A study in Iran [93] indicated that the infectious waste increased by 121 % compared with earlier the pandemic. Direct exposure of HCWs to virus-contaminated waste with inadequate safety measures and mismanagement of the HW may lead to their infection and can facilitate the transmission of the COVID-19 [92,94]. The WHO has provided clear guidelines for managing health care waste during the pandemic [95]. Despite that, many studies all around the world [94,96,97] illustrated the existence of gaps and a defective system for handling HW during the pandemic.
A mini-review [92] for HW during the pandemic showed that disinfecting the waste, followed by the proper segregation and on-site treatment of the waste, can also provide better and healthier HW management. It also revealed that surplus HW accommodation, mobile treatment, and temporary storage strategies might aid the sustainable management of health care waste without further spreading the virus. Another study in Brazil [96] proposed a model for the proper management of HW. It did not focus only on the operational management KPIs of the HW, but also on environmental management such as sustainable practices. Moreover, it highlighted the importance of employee training on HW guidelines since the HW management was not considered as an essential competence or a priority for all HCOs.
3.7.3 The efficiency and effectiveness dimension in COVID-19
Analyzing the number of patients’ visits and admissions in the USA [98] revealed a decrease in ER visits, with an increase in hospital admissions. However, another study in Alberta [99] perceived decreased admissions and ER visits to the hospital, despite the low volume of COVID-19 hospital admissions.
Many studies were performed to analyze the efficiency, utilization, and productivity of HCOs during the pandemic. A study [100] indicated that hospitals that were efficient under normal conditions, lost their efficiency during the outbreak of COVID 19 and have to adapt to the new criteria. A systematic review [101] showed that health care utilization decreased by about a third during the pandemic, with greater reductions among people with less severe illness.
A study at an isolation hospital in Egypt [102] utilized the DEA tool to improve efficiency. It confirmed that the number of nurses and the number of beds impacted the operational efficiency of COVID-19, while the number of physicians had no significant effect on the efficiency. These results are compatible with a study in Mauritius [19] which found that nurses and beds are the most important factors in hospital production; that is, a 1% increase in the number of beds and nurses, resulted in an increase in hospital outputs by 0.73 and 0.51%, respectively.
3.7.4 The availability and quality of supplies and services dimension in COVID-19
This dimension includes evaluating the availability and the quality of COVID-related medications, masks, personal protective equipment (PPE), detergents, medical services, supportive services, etc. Supplies and logistics management dimension was considered an important KPI in the tackling of COVID-19 [6]. Also, the availability of both clinical services and supportive services at hospitals was viewed as essential in responding to the COVID-19 pandemic and flow of COVID-positive patients [103]. The spectrum of supportive services to a hospital encompasses linen and laundry, dietary, Central Sterile Supply Department (CSSD), transport, consumables in large quantities at hospital stores, mortuary, and engineering services [103]. Some of the most crucial items were the filtering face-piece respirators or N95 respirators and the PPE kits’ availability [103]. The challenge during this pandemic in terms of inadequate availability of PPE in HCOs throughout the world highlighted the important role of the CSSD, a method of decontamination and re-use of filtering face respirators that has been suggested by the Centers for Disease Control and Prevention (CDC) to overcome the shortage of these respirators is their extended use or re-use [103].
However, researchers referred to the lack of studies on the quality of supplies and services at HCOs in COVID-19 [5]. This can be referred to data lag in pandemics which is the time between the provision of care and the quality measurements reporting [5]. To solve this, policymakers suggested that measures should be less reliant on claims data, which by nature have a time lag, and focus on measures that can be generated from the electronic health record (EHR) [5].
3.7.5 The time dimension in COVID-19
An “extra layer of processes” was added due to the donning and doffing protocols and cleaning requirements, which slowed all the operational processes down, and increased the time required to accomplish serving the medical care to patients [82]. The patients’ WT was also influenced. In the UK, WT reached high records with a special impact on elective surgery. The number of patients who waited for more than a year to get NHS treatment in July 2020 was 81-fold more than the previous year’s number [83].
Moreover, patients’ Length of Stay (LoS) increased also for another 2–3 days. A reason for that was the delays in COVID-19 testing results [104]. LoS in the USA was found to be 2 days more than that in Italy and 5 days less than that in Germany [105]. A systematic review for patients LoS in COVID-19 [106] concluded that LoS in China was longer than any other country, which was referred to differences in criteria for admission and discharge, and different timing within the pandemic. In another study [105], a negative association between the LoS and the case fatality rate was found. Therefore, the estimation of the LoS can be introduced as a KPI to scale the success of the countries fighting the ongoing pandemic. Moreover, LoS provides insights as to when hospitals will reach capacity, as well as predicting associated HCWs or equipment requirements [106]. Discharge status should be taken into consideration when analyzing LoS, since the patients who were discharged alive have longer LoS than those who died during their admission [106]. Hospitals reported that health insurance plans resisted paying for patients’ additional days in the hospital while awaiting COVID-19 test results [104].
However, complying with the CDC guidance on testing and disposition of patients was suggested to reduce the patients’ LoS, so freeing up hospital beds for incoming COVID-19 patients [104]. Another study in the UK [104] suggested that due to the complexity and partiality of different data sources and the rapidly evolving nature of the COVID-19 pandemic, it is most recommended to use multiple analysis method approaches of LoS on multiple datasets.
A combination of an Accelerated Failure Time (AFT) survival model and a Truncation Corrected (TC) method with the Multi-State (MS) survival model was found to be useful in epidemic planning and management. Finally, the finding of a cohort study [107] concluded that a Multi-Mechanism Approach (MMA) was effective in decreasing the average LoS in ICU by 5.4 days, and up to 9 days in older patients, which suggests that implementation of this treatment protocol could allow a health care system to manage 60% more COVID-19 patients with the same number of ICU beds.
3.7.6 The HCO building dimension in COVID-19
Design and infrastructure preparation was considered an important dimension in the PE of some HCOs at the pandemic [6]. Health care systems made adaptations in HCO buildings after the COVID-19 pandemic. Such as expanding the waiting areas, increasing ICU capacity, establishing isolation areas, and building new hospitals [83]. In the UK, the NHS temporarily used private hospitals to provide public care, leading to an increase in the number of beds, ventilators, and all HCW categories. Moreover, non-hospital sites were temporarily turned into hospitals [108]. However, the ease of access to HCOs in the pandemic was not sufficiently investigated in the research.
3.7.7 The responsiveness and communication dimension in COVID-19
The main goal of HCOs was considered to provide high-quality care to patients and meet their needs and expectations during an outbreak like COVID 19 [100]. Moreover, the importance of dialogue and listening to health demands in COVID-19-suspected patients was highlighted as the main step in the flows of care and guidance [109].
Communication among HCWs was also highlighted. A study [4] considered that HCWs’ reception of family support, colleagues support, clear communication, and COVID-19 information as some of the most useful resources in the pandemic. It found that lower levels of HCW’s psychological distress symptoms, burnout, and intentions to quit were perceived when these communication resources were more available. Another study [110] indicated that gratitude in communication could reduce depression in HCWs by promoting social support and hope, respectively.
Besides, the communication between HCWs and patients was also investigated during the pandemic. A study in Jordan [111] found that Physician-Patient Communication (PPC) positively affected the patients’ psychological status in COVID-19. It recommended avoiding communication errors using jargon, not being available to patients, not showing empathy in communication. Also, it emphasized the benefit of the physicians to be a good listener to patients. However, HCWs-patients’ communication faced few obstacles during the pandemic. The protective equipment used by the HCWs in the pandemic could have imposed a barrier on having effective communication or eye contact with them [112]. Some pediatricians reported a noticeable difficulty in communicating with families and following up with patients especially the newly discharged neonates and infants using the telephone [113].
However, more research is still needed to improve and evaluate the patient education programs, patient guidelines, counseling and consultation services, and the communication skills between HCWs and patients in the pandemic.
3.7.8 The patient-centeredness dimension in COVID-19
Many studies were conducted to evaluate patients’ satisfaction. A study [114] indicated no difference in patients’ satisfaction of the period spent in the emergency room before and during the pandemic. Another study [115] showed positive patient experience and satisfaction rates in one of Saudi Arabia’s largest institutions during the pandemic. Moreover, many studies focused on the psychological assessment of COVID-19 impact on the population in general. However, few studies focused on assessing the psychological effects on patients in specific. For example, a study [79] found that COVID-19 patients with low education levels and females who have undergone divorce or bereavement tended to have a high prevalence of adverse psychological events. Another study [116] found that the psychological consequences of the pandemic were better handled by cancer patients of 65 years of age or older, while younger cancer patients were more psychologically affected. Early psychological status identification and intervention should be conducted to avoid extreme events such as self-mutilating or suicidal impulsivity for patients [79]. The patient complaints and loyalty assessment during the pandemic as well as the psychological impact of COVID-19 on non-COVID-19 patients still need more investigation.
3.7.9 The HCWs-centeredness dimension
Physicians referred to the importance of reliable acknowledgment and motivation both emotionally and financially, considering the sacrifices they provide every day [113]. Parallel to this, staffing and recruitment of an adequate number of medical and non-medical HCWs were considered an important KPI for the PE of HCOs at COVID-19 [6]. In the UK, the NHS employed some strategies during the pandemic to facilitate the staffing process due to the shortage in HCWs. Firstly, deployment of newly qualified/final year medicine and nursing students. Secondly, the return of former HCWs was made [108].
HCW satisfaction rate and burnout were evaluated in many studies at the pandemic. A study [113] showed that the prevalence of burnout among physicians was found to be (57.7%) in the pandemic, which is considered so high. HCWs who lack PPE reported lower occupational satisfaction than those who did not [113,117]. To increase the HCWs’ satisfaction rates, their accomplishments at the pandemic were found to be positively associated with higher occupational satisfaction rates [117]. Moreover, as mentioned earlier, the better the performance of communication dimension, including the psychological support, will raise the HCWs’ job satisfaction and lower the rates of burnout and stress [110,117]. Some HCWs felt stress and fear mainly due to the possibility of transmitting the virus to their family member and the elderly living in their house [113]. To reduce HCWs’ stress, a study in Canada [4] showed that HCWs’ training and counseling services were perceived as a useful tool, however, they were underutilized in HCOs.
On the other hand, despite that most nurses had to increase their workload as a result of staff shortages, a study [117] found that the elevation of the workload was not associated with lower occupational satisfaction. Also, another study in Singapore [118] found that HCWs’ burnout was similar to the pre-pandemic rates. Nevertheless, HCWs’ vaccination, engagement, motivation, teamwork, and loyalty sub-dimensions as well as their impact are still not well investigated during the pandemic.
3.7.10 The HCWs’ scientific development dimension
Due to its importance, many studies aimed at evaluating the HCWs’ Knowledge, Attitudes, and Practices (KAP) at the beginning of the pandemic [119]. HCWs’ adherence to IC measures is affected by their KAP towards COVID-19 [120]. Some studies referred to the insufficient knowledge about COVID-19 among nurses [121]. Surgeons were worried about losing their skills after months of lockdown due to the paused practice [113]. However, HCWs were obliged to learn digital health skills and effectively utilize them in communication with patients during the pandemic crisis [113].
A study [122] found that both COVID-19 and non-COVID-19 publication productivity correlates with epidemiologic factors, health care system-related factors, and pre-COVID publication expertise. So, countries with a stable scientific infrastructure appear to maintain non-COVID-19 publication productivity nearly at the pre-year level. More incentives have to be drawn by HCOs towards their HCWs, to encourage research and the scientific productivity related to COVID-19.
3.7.11 The technology and information system dimension
Experts emphasized the role of Technology and Information (TI) in tackling the COVID-19 as inevitable due to its importance in the response, prevention, preparedness, and recovery phases, [123,124]. TI systems application varies from allowing HCOs to maintain and share records between each other, to being able to produce different reports and to follow-up with pandemic analysis. Telehealth is another example that proved to be useful in the pandemic, as it allowed HCWs to provide care for patients without direct physical contact, especially to patients at quarantine while keeping safe [125].
Researchers summarized the emerging technologies used to mitigate the threats of COVID-19 in the following categories: artificial intelligence/ deep learning, big data analytics, High-Performance Computing (HPC) infrastructures, robots, 3D printing technology, digital contact tracing technology, blockchain [113], bioinformatics systems, telemedicine, mobile phone, decision support system, the IC system in HCO, online interactive dashboard/Geographic Information System (GIS), Internet of Things (IoT), Virtual Reality (VR), surveillance systems, internet search queries [123,124]
Governments, health care systems, and HCO need to keep updated with the emerging technologies in this field, allocate resources to invest in them, and develop the required skills in HCWs to utilize them properly.
3.7.12 The community care and reputation (the external) dimension
Social Sustainability Indicators (SSI) for health care facilities facing a crisis can be ambiguous to define and apply, so SSI has been organized under the broad categorical concerns of well-being, values, agency, and inequality [126]. Despite the doctor-patient confidentiality clause and the protection law for the patients’ data privacy, the Department of Health and Social Care for England has relaxed the rules on sharing patients’ confidential data. It required from HCOs and the NHS to provide patient information to each other to help fight covid-19 [127]. Moreover, a leak of COVID-19 patients’ data to society and the breaches of their privacy occurred in some countries [113,128], which may have led to the stigmatization of those patients [113].
As mentioned earlier, a study [96] emphasized the importance of sustainable environmental practices for better HW management. The political situation was also considered as an external influence during the pandemic. It was highlighted in a study [129] at the Palestinian territories, which referred to the COVID-19 situation in the presence of the Israeli military occupation to have a double epidemic effect, and which eventually impacted the performance of the Palestinian health system and HCOs during the pandemic.
However, the community role, such as exemptions offered by HCOs for poor patients and social responsibility, as well as patients’ privacy concerns and HCOs market shares in COVID-19, are still poorly investigated.
3.7.13 The managerial tasks and assessment dimension
Standard policies, procedures, the availability of written standardized guidelines, and its delivery in full and on time were considered important in tackling COVID-19 [6]. Lack of standardization capability and conflicting or irrational managerial decisions were considered as dissatisfactory factors to HCWs in the pandemic [113].
Planning and preparedness are also important managerial task. The CDC developed a checklist to help hospitals assess and improve their preparedness for responding to COVID-19 [130]. Hospitals utilized a collection for some of the previously explained KPIs and dimensions to perform planning and internal assessment of their performance [126,131].
Centralized governance impact on HCOs during the pandemic was examined in few studies [132,133]. It was found to have a positive effect on the reactive strategies while learning from the past pandemics positively influences proactive and reactive strategies. However, the role of PE internally such as the utilization of BSC or MBNQA tools, or the external assessment such as Joint Commission International (JCI) accreditations, ISO certification, auditing, or peer-review on HCOs during the pandemic still require more investigation.
3.8 Discussion of the main results
In fulfillment of the research aims, all the perspectives, dimensions and KPIs employed in BSC implementations were collected. Categorization and re-grouping of the KPIs into major- and sub-dimensions were performed. Then the dimensions were ranked according to their frequency of use, as well as their importance. Finally, the resulted BSC dimensions were analyzed separately during the COVID-19 pandemic. The BSC tool can offer comprehensive planning, monitoring, evaluation, and improvement of the HCOs’ KPIs. Hence, improving their performance in the short term as well as in the long term.
In general, the high risk of selection bias at most implementations can be referred to as why HCOs were chosen purposively instead of randomly in most of them, besides due to the non-randomly choice of HCWs and patients. On the other hand, the low risk of reporting bias for all resulted implementations can be referred to the inclusion and exclusion criteria that we set at the beginning of this review, which has a complete reporting of indicators’ evaluation measures.
3.9 Overall completeness and applicability of evidence
Analyzing the results shows that BSC implementations had a typical utilization of the basic 4 perspectives; financial, customer, internal, and innovation and knowledge. However, the frequent employment of other perspectives shows the need for slight modifications of BSC design. For example, adding the sustainability, external, environmental, or community perspective that focused on the needs, perceptions, and reputation of the HCO among the community. This corresponds with a study [134] that referred to the sustainability perspective of the BSC as the fifth pillar. Also, there was a need for adding the managerial perspective, which did not only include the merge of the strategy with other perspectives but also included an evaluation for the managerial tasks and tools utilization at the HCO.
The variation among BSC implementations in the categorization of the same KPIs reflects the need for data standardization. HCWs’ training-related KPIs, for example, were categorized under the learning and growth perspective in almost half of the resulted studies [30–32,48–50,56–59,64–70,74–77]. Meanwhile the rest of studies categorized them under the perspectives of HCWs [69,76], quality [71], capacity for service, provision/service capacity [30,31,50,68,69,75], and health care facility functionality [32]. These results are consistent with a study [5] that referred to the lack of defining measures and the lack of data standardization. The differences in categorization prove our assumptions of calculation imprecision in the use frequency or the importance of the perspectives and KPIs at the previous reviews. Our systematic review has solved this calculation bias by uni-forming the 797 KPIs categorization. Re-grouping similar or semi-similar KPIs under the same category resulted in more precise results. Unification of dimensions can guide uniformed future implementations of PE or BSC at HCOs, allowing data sharing and comparability. This can be the reason why these findings are different from another systematic review [135] that did not consider unifying the classification of KPIs. It found that the average LoS, HAIs, patient satisfaction, bed occupancy, and bed turnover rate were the most useful KPIs according to the HCOs’ management.
Analyzing the results also shows a lack of BSC utilization in HCOs during the pandemic. Also, there has been a lack of studies comprehensively examining the impact of COVID-19 on the KPIs. Our analysis reflects that most KPIs were negatively affected during the pandemic, except the IC and safety measures which improved in some cases. A comprehensive PE of HCOs during the COVID-19 pandemic all over the world is required. Some dimensions which are essential for PE are still poorly investigated. So, future researchers are recommended to perform a comprehensive PE for HCOs during COVID-19 using the resulted dimensions. This will provide a better understanding dimensions the causal relationships between them. It will also allow comparability of the interventions’ outcomes, which will boost the performance and mitigate the consequences of the pandemic on HCOs. Moreover, researchers are encouraged to perform systematic reviews for each dimension, especially those dimensions that are already well investigated, as well as the investigation of dimensions that are still poorly investigated but essential for PE.
3.10 Strengths and weaknesses
We believe that this paper has several strengths. First, this systematic review includes all types of studies with BSC implementations, such as books, theses, conference papers, letters to the editor, etc. Second, this review includes all implementations despite the country, language, or HCO administrative type. This gives an advantage of being able to generalize the results on HCOs worldwide. Third, this review not only calculates the frequency of use of perspectives but also calculates the weights (importance) assigned to them. Fourth, it is the first review that has uniformed the KPIs in homogenous major dimensions and sub-dimensions despite the categorization differences among implementations, which yields more precise results. Finally, this study is the first to analyze the implication of BSC in HCOs during the pandemic. The resulted KPIs and dimensions at this review can be generalized or replicable to other HCOs and hospitals. However, an initial assessment needs to be done by top management to evaluate the importance of each according to the health organizations’ strategy.
On the other hand, this systematic review has some limitations. First, unlike previous reviews, it excludes some HCOs such as laboratories, pharmacies, radiology departments, and biobanks as specified in the inclusion/exclusion criteria. However, this will allow better concentration on KPIs directly related to HCOs offering 1ry, 2ry, and 3ry medical services. It is recommended for future reviews to study the other excluded types separately. Second, it includes only articles that report the complete implementation of BSC while excluding articles that display only the BSC design without reporting full implementation results. Third, the KPIs were extracted from all resulted implementations despite their RoB. Future systematic reviews are encouraged to re-performed when the number of studies with low and medium RoB is higher.
4. Conclusion
In conclusion, our review shows that the top frequently used perspectives in BSC papers were: the internal, the financial, the patient, the learning and growth, the HCW, the managerial, the community, and the stakeholder, respectively. While the perspectives which had the highest importance were: the internal, the financial, the learning and growth, the patient, the HCW, the community, the managerial, and the stakeholder, respectively.
Moreover, this review solves the dilemma of the KPIs categorization difference between BSC implementations. The 13 major dimensions that resulted after re-grouping the KPIs are the financial, efficiency and effectiveness, availability and the quality of supplies and services, managerial tasks, HCWs’ scientific development, error-free and safety, time, HCW-centeredness, patient-centeredness, technology and information system, community care and reputation, HCO building, and communication. The proper utilization of the 13 major dimensions and the 45 sub-dimensions will serve as a planning, monitoring, evaluation, and continuous improvement tool for HCOs, resulting in performance augmentation.
This research showed a lack of BSC utilization and any holistic PE approach in HCOs during the COVID-19 pandemic. Also, some dimensions which are essential for PE are still poorly investigated. Future research for comprehensive PE of HCOs during COVID-19 is required. This will lead to performance enhancement and mitigate the consequences of the pandemic on HCOs.
Data Availability
The data underlying this article are available in the Appendices upon the request of the journal.
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
This systematic review was planned and designed by all authors. Search and data extraction was performed by F.A & S.H. Interpretation of data analysis was made by F.A, S.H & H.K. Paper was written by F.A. All authors participated in revising the manuscript. Systematic review was supervised by S.L., I.B & D.E. All authors approved the final manuscript and agreed both to be personally accountable for the author’s contributions and to ensure that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Availability of data and materials
The data underlying this article are available in the article, and its online supplementary material are going to be provided upon journal request.
Funding
The authors received no specific funding for this work.
Competing interests
The authors have declared that no competing interests exist.
6. Appendix
Appendix (S1): PRISMA checklist.
Appendix (S2): Search strategies in PubMed, Embase, Cochrane, Google scholar.
Appendix (S3): Screened articles by title-abstract and full texts, articles represent partial results, not available articles.
Appendix (S4): Extracted, classified and re-grouped perspectives and indicators.
Appendix (S5): Quality assessment and Risk of Bias (RoB).
Acknowledgements
We would like to thank Dr. Vahideh Moghaddam, Diêgo Andrade, and Wang Zhe for their help in translating and data extraction of Persian, Spanish, and Chinese articles respectively. Also, we would like to thank Dr. Duha Shellah for contributing to revising the manuscript.
Abbreviations
- AACI
- American Accreditation Commission International
- AFT
- Accelerated Failure Time
- ARDS
- Acute Respiratory Distress Syndrome
- BSC
- Balanced Scorecard
- CDC
- Centers for Disease Control and Prevention
- COVID-19
- Coronavirus Disease 2019
- CSR
- Community Social Responsibility
- CSSD
- Central Sterile Supply Department
- DEA
- Data Envelopment Analysis
- DRG
- Diagnostic Related Groups
- EFQM
- European Foundation for Quality Management
- ER
- Emergency Room
- GIS
- Geographic Information System
- HAIs
- Health-Associated Infections
- HCOs
- Health Care Organizations
- HCQI
- Health Care Quality Indicator
- HCWs
- Health Care Workers
- HMIS
- Health Management Information System
- HPC
- High-Performance Computing
- HW
- Health Waste
- IC
- Infection Control
- ICU
- Intensive Care Unit
- IoT
- Internet of Things
- ISO
- International Organization for Standardization
- JCI
- Joint Commission International
- KAP
- Knowledge, Attitude, and Practices
- KPIs
- Key Performance Indicators
- LoS
- Length of Stay
- MBNQA
- Malcolm Baldrige National excellence model
- MBO
- Management by Objectives
- MeSH
- Medical Subject Headings
- MMA
- multi-mechanism approach
- MS
- multi-state
- NHS
- National Health System
- OECD
- Organization for Economic Co-operation and Development
- PATH
- Performance Assessment Tool for quality improvement in Hospital
- PE
- Performance Evaluation
- PPC
- Physician-Patient Communication
- PPE
- personal protective equipment
- PRISMA
- Preferred Reporting Items for Systematic Reviews and Meta-Analyses
- QOPI
- Quality Oncology Practice Initiative
- ROA
- Return on Assets
- RoB
- The Risk of Bias
- ROI
- Return On Investment
- SAQ
- Safety Attitude Questionnaire
- SQA
- Singapore Quality Award
- SSI
- Social Sustainability Indicators
- TC
- Truncation Corrected
- TI
- Technology and information
- TOPSIS
- Technique for Order of Preference by Similarity to Ideal Solution
- TQIP
- Trauma Quality Improvement Program
- TQM
- Total Quality Management
- UK
- United Kingdom
- USA
- United States of America
- USD
- United States Dollar
- VR
- Virtual Reality
- WT
- Waiting Time
5. References
- 1.↵
- 2.↵
- 3.↵
- 4.↵
- 5.↵
- 6.↵
- 7.↵
- 8.↵
- 9.
- 10.
- 11.
- 12.↵
- 13.↵
- 14.↵
- 15.↵
- 16.↵
- 17.↵
- 18.↵
- 19.↵
- 20.↵
- 21.
- 22.↵
- 23.↵
- 24.↵
- 25.↵
- 26.↵
- 27.↵
- 28.↵
- 29.
- 30.↵
- 31.↵
- 32.↵
- 33.↵
- 34.
- 35.
- 36.
- 37.
- 38.
- 39.
- 40.
- 41.↵
- 42.
- 43.↵
- 44.↵
- 45.↵
- 46.↵
- 47.↵
- 48.↵
- 49.↵
- 50.↵
- 51.
- 52.↵
- 53.↵
- 54.↵
- 55.↵
- 56.↵
- 57.
- 58.↵
- 59.↵
- 60.↵
- 61.↵
- 62.↵
- 63.↵
- 64.↵
- 65.↵
- 66.↵
- 67.↵
- 68.↵
- 69.↵
- 70.↵
- 71.↵
- 72.↵
- 73.↵
- 74.↵
- 75.↵
- 76.↵
- 77.↵
- 78.↵
- 79.↵
- 80.↵
- 81.↵
- 82.↵
- 83.↵
- 84.↵
- 85.↵
- 86.↵
- 87.↵
- 88.↵
- 89.↵
- 90.↵
- 91.↵
- 92.↵
- 93.↵
- 94.↵
- 95.↵
- 96.↵
- 97.↵
- 98.↵
- 99.↵
- 100.↵
- 101.↵
- 102.↵
- 103.↵
- 104.↵
- 105.↵
- 106.↵
- 107.↵
- 108.↵
- 109.↵
- 110.↵
- 111.↵
- 112.↵
- 113.↵
- 114.↵
- 115.↵
- 116.↵
- 117.↵
- 118.↵
- 119.↵
- 120.↵
- 121.↵
- 122.↵
- 123.↵
- 124.↵
- 125.↵
- 126.↵
- 127.↵
- 128.↵
- 129.↵
- 130.↵
- 131.↵
- 132.↵
- 133.↵
- 134.↵
- 135.↵
