RT Journal Article
SR Electronic
T1 A Crowdsourcing Approach to Develop Machine Learning Models to Quantify Radiographic Joint Damage in Rheumatoid Arthritis
JF medRxiv
FD Cold Spring Harbor Laboratory Press
SP 2021.10.25.21265495
DO 10.1101/2021.10.25.21265495
A1 Dongmei Sun
A1 Thanh M. Nguyen
A1 Robert J. Allaway
A1 Jelai Wang
A1 Verena Chung
A1 Thomas V Yu
A1 Michael Mason
A1 Isaac Dimitrovsky
A1 Lars Ericson
A1 Hongyang Li
A1 Yuanfang Guan
A1 Ariel Israel
A1 Alex Olar
A1 Balint Armin Pataki
A1 RA2 DREAM Challenge Community
A1 Gustavo Stolovitzky
A1 Justin Guinney
A1 Percio S. Gulko
A1 Mason B. Frazier
A1 James C. Costello
A1 Jake Y. Chen
A1 S. Louis Bridges, Jr.
YR 2021
UL http://medrxiv.org/content/early/2021/10/26/2021.10.25.21265495.abstract
AB To develop machine learning methods to quantify joint damage in patients with rheumatoid arthritis (RA), we developed the RA2 DREAM Challenge, a crowdsourced competition that utilized existing radiographic images and “gold-standard” scores on 674 sets of films from 562 patients. Training and leaderboard sets were provided to participants to develop methods to quantify joint space narrowing and erosions. In the final round, participants submitted containerized codes on a test set; algorithms were evaluated using weighted root mean square error (RMSE). In the leaderboard round, there were 173 submissions from 26 teams in 7 countries. Of the 13 submissions in the final round, four top-performing teams were identified. Robustness of results was assessed using Bayes factor and validated using an independent set of radiographs. The top-performing algorithms, which consisted of different styles of deep learning models, provided accurate and robust quantification of joint damage in RA. Ultimately, these methods lay the groundwork to accelerate research and help clinicians to optimize treatments to minimize joint damage.Competing Interest StatementThe following authors have financial interests in these companies: YG, Genentech, F. Hoffmann-La Roche AG, Lilly, Merck KGaA, Ann Arbor Algorithms Inc; MM, Bristol-Myers Squibb; PG, Oryn Therapeutics, CannBioRx, Gilead; JCC, PrecisionProfile. All other authors have no competing interests to disclose.Funding StatementWe would like to acknowledge funding support from Bristol Myers Squibb (BMS) and the CTSA Program National Center for Data to Health (CD2H) is supported by the National Center for Advancing Translational Sciences (NCATS) at the National Institutes of Health (Grant U24TR002306).Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:the UAB Institutional Review Board for Human Use gave ethical approval for this workI confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesAll radiogrpahic images used in this study are available upon reasonable request to the authors. Method write-ups and Docker containers for all submitted methods are available from the Challenge workspace (www.doi.org/10.7303/syn20545111). https://www.synapse.org/#!Synapse:syn20545111/wiki/594083