An Endpoint Adjudication Committee for the Assessment of Computed Tomography Scans in Fracture Healing

The use of endpoint adjudication committees (EACs) has the potential to reduce subjectivity and potential bias in clinical research trials and contribute to a higher quality of research. In a recent randomized control trial (RCT), we used serial computed tomography (CT) imaging to visualize fracture healing of the scaphoid as a primary outcome. The scaphoid bone poses a challenge in the diagnosing of fractures and non-unions due to its complicated shape. An EAC was created to increase the quality of the data and the validity of our findings. While an adjudication process has long been proposed and described for X-rays, this study outlines a rational approach to CT scan adjudication for bone fracture healing. A total of 364 scans were acquired in the RCT and of these, 101 were adjudicated for a binary endpoint of union vs. non-union. The application of EACs such as described in this paper is a useful tool in orthopaedic research requiring the adjudication of fracture healing as a study outcome.


32
A randomized controlled trial is only as good as its primary outcome. In studies regarding bone 33 healing, fracture union is often a desired trial outcome, albeit a historically unreliable one. 34 Fracture healing determined by radiographs or CT scans lends itself to a large degree of 35 subjectivity. Standard definitions of union, delayed union, non-union and partial union remain 36 elusive. Dozens of studies have highlighted the poor intra-and inter-observer reliability of 37 assessing radiographs for fracture union [1][2][3][4][5][6][7][8]. Vannabouathong et al. [1] observed this 38 knowledge gap in the assessment of long bone fractures. They described and advocated for an 39 endpoint adjudication committee (EAC) to be used when determining fracture union as a . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 5, 2023. ; https://doi.org/10.1101/2023.06.04.23290949 doi: medRxiv preprint 3 40 primary outcome in research. The interpretation of fracture union is subject to the individual 41 opinions of professionals and must be cooperatively decided upon by a committee. Bhandari et 42 al. [2] found that in the case of tibial shaft fractures, surgeons' definitions of delayed union 43 ranged from one to eight months. This disparity is crucial in determining the primary outcomes 44 of clinical trials. An adjudication committee is highly recommended to determine subjective 45 assessments, such as bone healing [3]. 46 Adjudication is the process by which a debatable topic is deliberated by a panel of experts. The 47 main goal of an EAC is to ultimately derive the best possible answer and remove uncertainty,  . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted June 5, 2023.   Patients were randomized to either 'active' or 'sham' units. Union status was determined with . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.  hypothesized that subjects randomized to a functional LIPUS device have a shorter time to 100 union than subjects randomized to a placebo LIPUS device. The patients were instructed to use 101 the LIPUS device once daily for 20 minutes until either union or non-union was declared. Serial 102 CT scans were collected at 8 +/-2 weeks, 12 +/-2 weeks, and then every 4-6 weeks thereafter 103 until either union was reached or a recurrent non-union was declared. All subjects were asked 104 to continue to attend follow-up appointments until two years after the index procedure.
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 5, 2023. Over the course of the trial, 364 CT scans from 142 participants were accumulated (average 106 2.56/patient). These scans consisted of coronal and lateral two-dimensional serial views. Exact 107 CT scan acquisition and reconstruction parameters used can be found in S2 Appendix. For each 108 CT scan, both the treating fellowship-trained hand surgeon and MSK radiologist reported union 109 status based on the following six classifications: 0%, 1-24%, 25-49%, 50-74%, 75-99% and 100%.

110
The scaphoid was considered united once a threshold of 50% union was met as outlined by  3. The treating surgeon and MSK radiologist disagreed on the union status. This was 120 declared as a "major discrepancy" and the CT scans in question were subsequently 121 flagged for adjudication. A major discrepancy was defined as any time the surgeon and 122 radiologist disagreed by more than one quartile (i.e., 25-49% vs. 75-99%) or more than 123 one quartile away from the extremes 0% and 100% (i.e., 0% vs. 25-49%) or a single 124 quartile that crossed the critical threshold of 50% union. Note that in the SNAPU trail, . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

128
To address these discrepancies between professionals, the investigators implemented an EAC, 129 using previously established guidelines regarding adjudication of radiographic imaging [2, 3].

130
The investigators designed an efficient adjudication protocol that could be effectively adapted 131 to CT images in the SNAPU trial.  Step 2: Charter of Adjudication (See S3 Appendix for more details) 154 The EAC was created by the selected chair. The charter created for the SNAPU trial EAC outlines 155 the process, and rules, of the adjudication process. The rules for the committee included the

163
-Time from surgery to adjudicated CT scan was known to the committee . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 5, 2023.  (Table 1).

176
-Project manager would provide spreadsheets with discrepant cases to review 177 -The reviewer would access cases through IMPAX and AYCAN using existing credentials.   were collected across 5 different study sites. Seventy-five (74.3%) of these CT scans were . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 5, 2023. ; https://doi.org/10.1101/2023.06.04.23290949 doi: medRxiv preprint 11 204 defined as major discrepancies (Table 2) and the remaining 26 CT scans were adjudicated due 205 to missing data or as a random selection to validate the other groups (S4 Appendix). Of these 206 75 major discrepant cases, the final reading of the CT images by the EAC matched the initial 207 reading of the MSK radiologist in 42 cases (56.0%) and the treating hand surgeon in 25 cases 208 (33.3%). The remaining 8 CT images that were adjudicated to agree with both (6) or neither (2) 209 the surgeon and/or MSK radiologist (10.7%). The EAC resulted in a final diagnosis of union in 41 210 (40.6%) cases and in non-union in 60 cases (59.4%) (Fig 1). . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 5, 2023. The primary outcome of the SNAPU trial was time to union as defined by 50% osseous bridging.  The process of adjudication based on quartiles and both "minor" and "major" discriminators . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 5, 2023. ; https://doi.org/10.1101/2023.06.04.23290949 doi: medRxiv preprint Limitations 244 We recognize as pioneers of adjudication committees for CT images, that the design presented 245 here is not without faults. One limitation is that review of the CT scans was limited to a 246 maximum of 10 minutes each to provide efficacy in this process. This could possibly influence 247 the decisions made by the clinicians as the diagnosis may have been different if they had been 248 given more time to read and analyze the images. Images were adjudicated for an average of 7 249 minutes, but more time, such as a 15-minute limit, for example, would ensure that the reading 250 would not be rushed. It was necessary to balance accuracy with feasibility and efficiency.

251
Another weakness is that not every CT scan obtained from the trial was adjudicated. Only major 252 discrepancies were reassessed (union vs. non-union or two or more grades difference) by 253 adjudication. This was for reasons of efficiency as we deemed it was not practical to ask 254 professionals to adjudicate every image. The multiple study sites in the SNAPU trial created a 255 third limitation. There were multiple CT scanners and CT scanning protocols used allowing for 256 varying imaging quality and reformatting methods. This limitation was present because not 257 every site was able to use the exact same CT parameters for all imaging. However, this is 258 reflective of practice in the real world and of the fact that surgeons and radiologists are 259 required to interpret varying CT images in their day to day practice. To reduce discrepancies 260 caused by this challenge, using the same CT scanner parameters as well as radiologist personnel 261 would help with consistency in image acquisition.

Conclusion
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The analysis in this paper confirms that the adjudication process had an impact on the 265 assessment of scaphoid fracture union on CT scan during the SNAPU trial. The process allowed 266 for imaging to be reviewed subjectively by a team of experts to define union in cases where 267 discrepancies were present, allowing for a more valid analysis of the LIPUS device in scaphoid 268 fracture healing. The adjudication process in the analysis of serial CT scans in fracture healing 269 trials is a novel and pivotal step to the assessment, treatment, and outcome of fracture healing.

270
EACs should be used routinely in orthopaedic trials where the endpoint is radiographic union.
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted June 5, 2023. ; https://doi.org/10.1101/2023.06.04.23290949 doi: medRxiv preprint