Interoperability Assessment of Emergency Department 2 Processes Based on Multicriteria Decision-Making 3 Methods

It is noticeable that, because of the amount and quality of information exchanged and the


Introduction
alternatives, an attribute is already delivering sufficient performance or needs exchange information with each other and use this information" [10]. Of late, this concept 149 has received a broader meaning, so that enterprise interoperability may be understood as 150 the capability of inter and intraorganizational systems to work together with mutual 151 objectives [11]. [21] highlight that systems should be understood from a generic 152 perspective, wherein a system could be a stakeholder, an enterprise, a department, an 153 information system, or a resource. This understanding enables one to comprehend the HC . 164 Following this, [23] proposed another model, which divides interoperability into human, 165 non-human, and heterogeneous systems, and then evaluates overall enterprise 166 interoperability. Thus, these authors highlighted human systems for the academic 167 community inclined toward interoperability. 168 Further, [24], in a literature review, identified that none of the existing models with 169 maturity levels at par with those of interoperability could verify the existing weaknesses 170 that impair interoperability. Therefore, they proposed other concerns that would best fit 171 human-dependent contexts: (i) business (strategic aspects such as culture, vision, values, 172 and strategy policies); (ii) process management (related to working methods); (iii) human . CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 22, 2022. ; https://doi.org/10.1101/2022.02.21.22271273 doi: medRxiv preprint resources (skills, competencies, roles, collaboration, and social culture); (iv) policies and 174 procedures (normative aspects such as standards and guidelines); (v) information 175 technologies (interconnecting applications, data, and communication); and (vi) semantics 176 (the understanding of different terminologies). 177 Healthcare interoperability frameworks in the literature are termed as E-Health, 178 Health Information Systems (HIS), and Personal Health Systems (PHS) [25]. However, 179 these preexistingframeworks only focus on health records and hospitals strategy, not The MCDA (multicriteria decision analysis) is a discipline that evaluates multiple 185 conflicting criteria in decision-making, facilitating the choice of a solution in scenarios 186 with several qualitative and quantitative variables, useful for benefit-risk analysis. As the 187 volume and complexity of health data, information, and knowledge is growing, the 188 MCDA has become invested with more potential in optimizing decision-making at any 189 level of HC [26][27][28]. Its popularity in HC is explained by the large variety of attributes 190 that exist in such a context [29]. The bibliometric analysis conducted by [29], which 191 reviewed 66 papers on the MCDA in the HC field, revealed that 39% of them used the 192 MCDA aiming for the "diagnosis and treatment" subarea, and the second and the third 193 subareas more prevalent were "priority setting" and "health technology assessment," both 194 representing 12% each. [30] reasoned that MCDA could be applied to any field of 195 emergency medicine. Regarding public health, [28] affirm that, recently, MCDAs have 196 been more utilized because, conducting a systematic scientific empirical assessment . CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The AHP allows to model some problem in a hierarchical structure, deploying it 216 downward from a goal, subsequently branching it into criteria and subcriteria and, in the is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint for , as illustrated in Table 2.

249
= (( á -) ∕ ( -1)) / ) (4) is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint   The mathematical foundations are summarized as follows [19].  is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The final ranking is obtained by using Equation 13. Systems are intrinsically linked to the enterprise interoperability concept. In sum, a 306 system is a set in which a limit can be established, and the elements within it are the 307 attributes that characterize and differentiate it from others. Thus, attributes are the 308 elements that define the structure of a class that allow the qualification and relation of 309 confidence and commitment between interoperable systems [25]. Following this 310 rationale, it is necessary to identify the attributes that define critical HC processes. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 22, 2022. ; https://doi.org/10.1101/2022.02.21.22271273 doi: medRxiv preprint by [23] and later reinforced by [24]. The PHS is divided into two structures: execution 314 and technical aspects; legal and organizational aspects [25]. Based on this distinction, HC 315 attributes could be divided into organizational aspects related to legal and organizational 316 matters and the clinical aspects referring to treatment execution and techniques. The 317 attributes existing in a HC process directly interfere with its clinical outcomes as well as 318 face interference by enterprise interoperability.

319
In this sense, this research proposes a framework capable of diagnosing the 320 interoperation capacity and provide guidance on how to improve it (i.e., a set of actions 321 suggested to improve interoperability to optimize clinical outcomes). The framework is 322 subsequently based on two stages: (S1) diagnosis of the current scenario of   The framework and its stages could be performed after defining these key elements.

333
Further, both stages employed MCDA methods. The first one, (S1), used AHP, 334 which performed the EIA, pointing out a level of interoperability corresponding to the 335 "as is." In sequence, the second stage, (S2), utilized PROMETHEE II, based on S1, to 336 weight the criteria, and its outcome was a ranking of interoperability concerns, which . CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 22, 2022. ; https://doi.org/10.1101/2022.02.21.22271273 doi: medRxiv preprint prioritized the attributes that led to a more advanced level of interoperability. Thus, S2 338 helped decide which attributes must be prioritized in terms of efforts for improvement.   is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint Technology and information systems existing in automated processes Table 6. Interoperability levels and their description.

Interoperability Levels Description Basic Level
The institution has no or insufficient capacity to address clinical emergencies, being unable to support the whole patient care. There is a variation in its processes and practices, and it depends on analogical systems.

Intermediate Level
The institution admits and manages clinical emergencies, but is unable to dispense the best existing options of treatment in the whole patient care. It seeks reducing variability of its processes and increasing collaboration and integration and has complex systems and processes governed by a central body.

Advanced Level
An institution at this level works with extremely developed policies and resources. Moreover, it is constantly innovating and seeking partnerships. Its processes are effective and concrete, designed with the purpose of achieving satisfactory results in the treatment of patients admitted in emergencies.

391
The AHP's goal is represented by the EIA itself; the criteria are the

399
The pairwise comparisons between elements within the same AHP layer were 400 made in the upper part of the matrices, following the Saaty Scale (Table 1)  is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint know-how to judge OAs from the interoperability concern helped with both judgements 409 (Fig 3-B and 3-C).

414
The second matrix had the OAs in its rows and CAs in its columns; thus, it gave 415 way to pairwise judgements between CAs (Fig 4-A) and relationship judgements between 416 clinical and organizational attributes (Fig 4-B).

420
When incorporating the elements into the AHP hierarchical structure, only "very 421 strong" (weight 9) relationships were considered. Tables 8 and 9 list these correlations. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint In sequence, the design of the AHP allows for the EIA to proceed. It is necessary 430 that an HC professional, directly involved in the assessment process, evaluates the CAs.

431
To facilitate this assessment by professionals who are not used to interoperability 432 semantics, a questionnaire is employed, whose answers must be translated into the 433 pairwise judgements of the AHP (46). In this way, the questionnaire respondent must 434 evaluate their satisfaction regarding the performance of the CAs and answers should be 435 translated into interoperability levels to best suit the AHP design in such a way that 436 attributes with "unsatisfactory," "partially satisfactory," and "very satisfactory" 437 performances fit, respectively, into Basic, Intermediate, and Advanced levels of is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint  is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint For each concern, two indices must be calculated: the positive flow + and the 471 negative flow -. The symbol + expresses how much a concern needs to overcome the 472 others by, while the symbolindicates how much its performance is already sufficient 473 by to deliver the desired level of interoperability [47]. A sensitivity analysis function 474 enables an understanding of how to make the improvements, since it relates the concerns' 475 changing behavior to changes in OA prioritization. As a result, a sensitivity analysis is 476 relevant to indicate which OA should be prioritized. 478 The framework proposed by this paper was applied in a UH. This UH is one of 479 the largest hospitals in the south of Brazil and has a high volume of patients admitted in 480 the ED, including some with ischemic stroke symptoms. Even though the UH has a 481 residency program in neurology and neurosurgery, it has acknowledged that its 482 performance in stroke patient management can be improved further.

483
The most efficient therapeutic procedure to avoid increasing the infarcted brain is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

505
Facing a complex multi-criteria decision analysis, the AHP points out the part of  is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint suggests performing a sensitivity analysis to understand how and when the current 518 interoperability level could be superimposed by another. Analyzing sensitivity in 519 decision-making methods targets the evaluation of the behavior of "alternatives" 520 whenever there is a change in the "criteria" input data. Thus, in this case, it evidenced the 521 influence that each CA had on the interoperability level.

530
The darker curves represent the advanced level of enterprise interoperability, the medium-colored ones 531 represent the intermediate level, and the lighter ones, the basic level.

532
Thus, for example, if the prioritization of Clinical Attribute 1 (CA 1 -Fig 8) is 533 maximized, the attribute will leverage the global level of EIA (because the line that 534 represents the advanced level will be in its upper half), but if its prioritization is 535 minimized, the current overall EIA level (intermediate) will be maintained.

538
On the other hand, if the Clinical Attribute 3 (CA 3 -Fig 9) receives the maximum 539 or minimum of investment and efforts, even though more investments will tend to an . CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 22, 2022. ; https://doi.org/10.1101/2022.02.21.22271273 doi: medRxiv preprint advanced level, its EIA level will vary insignificantly. All the CAs can be analyzed 541 following this rationale.

542
After the AHP sensitivity analysis was performed, it is notable that some CAs 543 leveraged the global level of enterprise interoperability. These attributes are those that, 544 the more prioritized they were, the more they tended to rise to the advanced level, and, at (following Eq. 14), shown in Table 10. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint for each concern in order to obtain the results exhibited in Table 11. 562 Since + expresses how much a given concern needs to be prioritized to overcome 564 the others and -indicates by how much its performance is already sufficient to achieve 565 the desired level of interoperability, it is possible to infer that those requiring more is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 22, 2022. ; https://doi.org/10.1101/2022.02.21.22271273 doi: medRxiv preprint at a satisfactory level of interoperation and efforts for improvement must be invested in 580 the other concerns. Fig 11 represents the sensitivity analysis that evaluated OA 2, OA 12, 581 and OA 13.

584
In these three sensitivity analyses, the prioritization values were changed until a 585 "turning point" was found, i.e., a percentage of prioritization of the OA that resulted in a 586 significant change in the concerns. The first attribute analyzed was OA 2 -Electronic 587 Medical Record, whose priority was changed from 0 to 25%, the "turning point." This 588 change in OA 2 impacted the Process Management (PM) concern, turning it into the new 589 strategic focus.

590
The second attribute analyzed was OA 12 -Compliance between practice and 591 standards. Its weighting was increased from 2% to 47%, with the others decreasing is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint assisting patients with stroke symptoms and striving to decrease its process variability 608 levels and increase the integration among its sectors. As per the proposed classification, 609 its complex systems and processes are centralized by a single entity (information system),  The S2 pointed out that the interoperability concerns that needed more 620 prioritization were Business (B) and Information Technology (IT). As previously defined, is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint   Aggregating this analysis is suggested as future research since the MCDA approach 651 identifies the cause-effect chain between elements.

652
It is also important to note that the design of the proposed framework can be 653 adapted (in relation to CAs) and applied to other processes in the HC domain. Thus, for . CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 22, 2022. ; https://doi.org/10.1101/2022.02.21.22271273 doi: medRxiv preprint future works, it is also suggested that such assessments are carried out in other hospital is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint  is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 22, 2022. ; https://doi.org/10.1101/2022.02.21.22271273 doi: medRxiv preprint The ATHENA Interoperability Framework. In: Enterprise Interoperability II. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 22, 2022. ; https://doi.org/10.1101/2022.02.21.22271273 doi: medRxiv preprint