Abstract
Background Machine learning may offer a superior alternative to traditional prediction tools when used to model complicated, nonlinear interactions between variables. While modern machine learning methods are tagged as “black boxes”, the random forest (RF) classifier can be interrogated to understand the contribution of input variables (feature importance), thereby improving the interpretability of its predictions. We hypothesized that a random forest (RF) classifier would have equivalent, if not superior, performance to the 4-variable Kidney Failure Risk Equation (KFRE) in predicting progression to end stage kidney disease (ESKD) in a chronic kidney disease (CKD) population and explored the impact of serum creatinine and primary renal disease on prediction accuracy.
Methods A 2-year risk of ESKD was calculated using the 4-variable KFRE and compared to a RF model using the same four variables (age, gender, eGFR and urine albumin creatinine ratio). Four more RF models were developed using a combination of these as well as serum creatinine and primary renal disease. Performance of the KFRE and RF models was assessed by area under a receiver operating (AUC ROC) curve and feature importance was evaluated for each RF model.
Results Of 1365 patients with CKD from two renal units included in the analysis, 208 progressed to ESKD in the 2-year follow-up period. The AUC ROC for KFRE was 0.95 (95% confidence interval, 0.93 – 0.96) and for the RF model using the same 4 variables 0.97. The remaining four RF models had similar performance (AUC ROC 0.97 – 0.98). In the RF models, eGFR and serum creatinine had the largest effect on risk prediction while gender had the smallest.
Conclusions Our findings suggest that RF models provide a potential tool to predict CKD progression with competing accuracy and interpretability to the current benchmark equation. They therefore warrant validation in larger and more diverse populations
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
MW declared funding from a Research and Training Scholarship from the University of Queensland.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Approval for the use of the SESLHD patient data was granted by Human Research Ethics Committee of the South Eastern Sydney Local Health District (HREC Ref 11/STG/212). The MNHHS and QLD registry study was approved by the Queensland Health Human Research Ethics Committee (HREC approval: HREC/10/QHC/41 in November 2010.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
All data produced in the present work are contained in the manuscript
