Registration, Publication, and Outcome Reporting among Pivotal Clinical Trials that Supported FDA Approval of High-Risk Medical Devices Before and After FDAAA

Background: Selective registration, publication, and outcome reporting of clinical trials distorts the primary clinical evidence that is available to patients and clinicians regarding the safety and efficacy of FDA-approved medical devices. The purpose of this study is to compare registration, publication, and outcome reporting among pivotal clinical trials that supported FDA approval of high-risk (Class III) medical devices before and after the U.S. Food and Drug Administration (FDA) Amendment Act (FDAAA) was enacted in 2007. Methods: Using publicly available data from ClinicalTrials.gov, FDA summaries, and PubMed, we determined registration, publication, and reporting of findings for all pivotal clinical studies supporting FDA approval of new high-risk cardiovascular devices between 2005 and 2020, before and after FDAAA. For published studies, we compared both the primary efficacy outcome with the PMA primary efficacy outcome and the published interpretation of findings with the FDA reviewer's interpretation (positive, equivocal, or negative). Results: Between 2005 and 2020, the FDA approved 156 high-risk cardiovascular devices on the basis of 165 pivotal trials, 48 (29%) of which were categorized as pre-FDAAA and 117 (71%) as post-FDAAA. Post-FDAAA, pivotal clinical trials were more likely to be registered (115 of 117 (98%) vs 24 of 48 (50%); p < 0.001), to report results (98 of 115 (85%) vs 7 of 24 (29%); p < 0.001) on ClinicalTrials.gov, and to be published (100 or 117 (85%) vs 28 of 48 (58%); p < 0.001) in peer-reviewed literature when compared to pre-FDAAA. Among published trials, rates of concordant primary efficacy outcome reporting were not significantly different between pre-FDAAA trials and post-FDAAA trials (24 of 28 (86%) vs 96 of 100 (96%); p = 0.07), nor were rates of concordant trial interpretation (27 of 28 (96%) vs 93 of 100 (93%); p = 0.44). Conclusions: FDAAA was associated with increased registration, results reporting, and publication for trials supporting FDA approval of high-risk medical devices. Among published trials, rates of accurate primary efficacy outcome reporting and trial interpretation were high and no different post-FDAAA. Keywords: Clinical trials, Publications, Device approval, United States Food and Drug Administration

CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted March 2, 2021. ; https://doi.org/10.1101/2021.02.28.21252619 doi: medRxiv preprint 6 We excluded AEDs because FDA published a final order on January 29, 2015, stating that AED 109 clinical study information can be leveraged from both published studies and clinical data 110 previously submitted under the 510(k) process instead of requiring the conduct of a pivotal trial 111 to support FDA approval. 11 Otherwise, devices were selected if they met the following 112 parameters: "Cardiovascular" under advisory committee and "Originals Only" under supplement 113 type. All devices were characterized by the following using publicly available information on the 114 FDA website: FDA review type (priority/standard), implantable designation (yes/no), life-115 sustaining designation (yes/no), and combination product (yes/no). Sponsor company 116 management (public/private) was also determined by Google searching the sponsor company 117 name along with "publicly traded," "stock price," "IPO," or "privately held." 118 119

Characterization of pivotal clinical trials 120
For each device identified, we then identified the pivotal clinical studies that supported device 121 approval from the "Summary of Safety and Effectiveness" documents. Pivotal clinical studies 122 supporting approvals were categorized as pre-FDAAA if the clinical trial primary completion 123 date was before December 26, 2007 (the date the policy took effect), in a manner described 124 previously; 7 all other studies were categorized as post-FDAAA. Also, we categorized pivotal 125 trials by specific design characteristics: use of randomization (yes/no), use of blinded allocation 126 (yes/no), primary efficacy endpoint (surrogate marker/clinical outcome or scale), and study 127 center and patient enrollment numbers. Study characteristics and data were abstracted from the 128 FDA summaries by one author (MJS); 10% of characterized trials were randomly selected and 129 validated by a second author (JLJ). Any differences were reconciled by consensus. 130 131 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) multiple studies, such as reviews and meta-analyses, were also excluded unless the results of 144 each study were analyzed and discussed individually at the level of detail as one would expect 145 from a full-length publication. 146 147

Comparison to corresponding publications 148
First, for each pivotal trial for which a publication was identified, we compared the primary 149 effectiveness endpoint specified in the FDA documents with the effectiveness endpoint specified 150 as primary in the publication. If there was more than one primary effectiveness endpoint reported 151 in the FDA documents, we verified the one that matched the primary endpoint specified in the 152 publication. If none of the specified primary endpoints matched, we categorized the primary 153 effectiveness outcomes reported as discordant. If one matched, we determined whether the 154 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted March 2, 2021. ; https://doi.org/10.1101/2021.02.28.21252619 doi: medRxiv preprint 8 primary effectiveness endpoint result reported in the FDA documents was the same as the result 155 reported in the publication. The outcomes reported were categorized as concordant if they were 156 an exact numerical match or if there was a relative difference of less than 5% to the FDA PMA, 157 and discordant if they were not. Second, for each pivotal trial for which a publication in the peer-158 reviewed literature was identified, we compared the overall study interpretation between the two 159 sources. The overall interpretation was categorized as positive, equivocal, or negative based on 160 the FDA officer's language in the "Effectiveness Conclusions" and "Overall Conclusions" 161 subsections of the "Summary of Safety and Effectiveness" document and the author's language 162 in the conclusion of the publication; the FDA and publication interpretation were categorized as 163 concordant or discordant. 164 165

Statistical analysis 166
We determined the rate of ClinicalTrials.gov registration, ClinicalTrials.gov results reporting, 167 and PubMed publication for all identified pivotal trials, overall and stratified by device and 168 design characteristics. We then determined the overall rate of concordant primary outcome 169 reporting between the FDA PMA summaries and corresponding publications, as well as the 170 overall rate of concordance between the FDA PMA reviewer's interpretations and the trial 171 publication's interpretations. Summary statistics were calculated for each comparison, presented 172 as numbers, percentages, means, standard deviations, and ranges, as appropriate. Chi-square and 173 two-tailed Fisher exact tests were used to compare rates pre-and post-FDAAA of registration, 174 results reporting, publication, concordant outcome reporting, and concordant interpretation, as 175 appropriate. All statistical tests were two-tailed and used the Bonferroni method to adjust P 176 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted March 2, 2021. Studies in Epidemiology (STROBE) reporting guideline for cross-sectional studies. 13 The study 181 did not require institutional review board approval or patient informed consent because it was 182 based on publicly available information and involved no patient records. 183

Concordant outcome reporting and trial interpretation 213
A primary effectiveness outcome and main result were reported in the FDA documents for all 214 pivotal trials, pre-and post-FDAAA. Overall, 37 of 165 (22%) trials were not published, 215 precluding primary effectiveness outcome comparison. Among 120 of 128 (94%) published 216 trials, the primary endpoint specified in the FDA documents was the same endpoint specified as 217 primary in the publication; among these, the primary endpoint result reported in the FDA 218 documents was the same or within 5% of the result reported in the publication for 120 (100%) 219 trials. Rates of concordant primary effectiveness outcome reporting were not significantly 220 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted March 2, 2021. ; https://doi.org/10.1101/2021.02.28.21252619 doi: medRxiv preprint Our study demonstrates that 98% of post-FDAAA trials were registered on ClinicalTrials.gov, 244 85% reported their results on ClinicalTrials.gov, and 85% were published in the peer-reviewed 245 literature. These rates are high but lag behind those reported for new drugs. 7,8 Impressive clinical 246 trial registration and results reporting after FDAAA enactment was expected given the explicit 247 requirement to require trial registration among all trials investigating FDA-regulated products. 248 However, there is still room for improvement because 2% of post-FDAAA trials remain 249 unregistered on ClinicalTrials.gov, 15% have not posted their results on ClinicalTrials.gov, and 250 15% remain unpublished in the peer-reviewed literature. As required by law, all of these rates 251 should be 100%. Follow-up studies will be needed to see whether these rates persist or improve. 252 253 Among post-FDAAA trials, publication rate in the peer-reviewed literature was higher when 254 compared to the rate of 80% observed among studies supporting FDA approval of novel, high-255 risk cardiovascular devices between January 2011 and December 2013. 14 Although it continues 256 to greatly exceed the rate of 49% observed for trials supporting FDA-approved, high-risk 257 cardiovascular devices between January 2000 and December 2010, 5 given that these pivotal trials 258 represent the best evidence of medical device safety and effectiveness, there is no reason that the 259 clinical and research community should not expect a publication rate of 100%. Also, 96% of 260 published studies reported primary effectiveness outcomes in a manner concordant with FDA 261 reviews, which is nearly identical to a prior study of medical device publication and results 262 reporting. 14 Our rate may be more representative of concordant results reporting post-FDAAA 263 because we analyzed 15 years of FDA approvals post-FDAAA while the aforementioned study 264 analyzed three years post-FDAAA. Nonetheless, our results showed higher rates than the initial 265 study of selective reporting for medical devices that reported a rate of 69% for both identical and 266 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Fellowship. This funding was solely used for living expenses and was not used in the design of 333 the study or collection, analysis, and interpretation of data or in writing the manuscript. 334 335 AUTHORS' CONTRIBUTIONS 336 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted March 2, 2021. ;https://doi.org/10.1101https://doi.org/10. /2021    is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)