Classification of glomerular pathological findings using deep learning and nephrologist–AI collective intelligence approach
Graphical abstract
Conclusion: AI models for classifying 7 major findings of glomeruli were developed, which may improve clinicians' diagnostic accuracy.
Introduction
Renal pathology is important for the diagnosis and management of patients with kidney disease. The renal survival rate tends to be better with histologic evaluation by renal biopsy than without renal biopsy [1], thus, accurate and robust diagnosis is essential for the proper management of patients with kidney disease. On the other hands, making an accurate diagnosis is a time-consuming process even for experienced pathologists. It has been expected that automated processing to support this procedure will improve the efficiency of renal pathology and contribute to a more objective and standardized diagnosis [2], especially in hospitals, areas, or countries where there are an insufficient number of nephropathologists. A field called digital pathology, which aims to diagnose and quantify disease based on image data obtained by scanning pathological tissue specimens, has rapidly been developed. With the use of current state-of-the-art techniques of deep learning (DL), the artificial intelligence (AI) approach has made a significant progress in medical image analysis of retinal fundus images [3], skin images [4], and pathology mainly on cancer [5]. Currently, the implementation of these technologies in the clinical process and their effect on healthcare workers are of great interest [6].
There are some studies trying to apply DL to renal pathology. While some studies have validated DL models analyzing the structures other than the glomeruli, such as the tubules, blood vessels, and interstitium [[7], [8], [9], [10]], many studies have focused on the glomeruli, which present various histological findings essential for diagnosis. As a first step in the automation of this diagnostic procedure, detection of a glomerulus in a whole slide image (WSI) of renal tissue specimens has been recently attempted in many studies with the use of methods to define various features [[11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24]] or using convolutional neural networks (CNNs) [25], such as InceptionV3 [26], AlexNet [27], U-Net [28], R-CNN [29,30], or DeepLab V2 ResNet [31].
On the other hands, studies trying to classify pathologic findings from the glomerular images are still very few, and the pathological findings analyzed in these studies are quite limited. Barros et al. [32] constructed a model to classify proliferative lesions. Sheehan et al. [24] quantified mesangial matrix proliferation, numbers of nuclei, and capillary openness. Ginley et al. [31] also quantified nuclei, luminal space, and periodic acid-Schiff-positive component. Kannan et al. [26] and Marsh et al. [33] reported models to distinguish between sclerotic and nonsclerotic glomeruli. The pathological findings analyzed in these studies are quite limited, and do not cover the pathological findings necessary for accurate diagnosis, and there has been no study which enables the comprehensive evaluation of the essential pathological findings necessary for the diagnosis.
In this study, we focused on seven major pathological findings required for pathological diagnosis: global sclerosis, segmental sclerosis, endocapillary proliferation, mesangial matrix accumulation, mesangial cell proliferation, crescent, and basement membrane structural changes, and developed AI models to classify these findings. In addition, we examined whether our AI model can cooperate with nephrologists and improve their diagnostic performance. Although many studies have compared the performance between AI and the specialists [3,4], validation of the effect of the collaboration between AI and clinicians on the diagnostic judgment is also important and clinically relevant. Assuming a situation in which a majority decision of diagnosis is taken among specialists at a case conference, we demonstrated that the diagnostic performance was improved by adding AI model as one of the specialists.
Section snippets
Data preparation
We used WSIs of 283 renal biopsy cases that were agreed to be used for research at the Kyoto University Hospital between 2012 and 2017. The renal biopsy samples, including the transplanted allografts, were obtained by needle biopsy. Specimens that were stained by periodic acid-Schiff (PAS) and periodic acid methenamine silver (PAM) were used. Details of the staining and scanning of the slides are provided in the Supplementary Material.
Patients provided written informed consent for the use of
Patients and annotation of images
The train/validation and test datasets included 218 and 65 cases, respectively. The demographics and pathological diagnoses of these cases are shown in Table 1. The median numbers of annotated images by one nephrologist were 1625 (532–1698 [minimum, maximum]). In the train/validation dataset, the cropped glomerular images comprised of 5571 images on PAS staining and 5876 images on PAM staining. After removing the images labeled as artifact (examples in Supplementary Figure S2) by at least one
Discussion
We constructed AI models to classify several pathological findings of glomerular images. To the best of our knowledge, this is the first study to verify classification models that comprehensively included as many as seven findings essential for renal pathological diagnosis. In the classification of global sclerosis, our model showed high performance, AUC greater than 0.98, and that was also close to the performances of the nephrologists. Marsh et al. [33] reported a model to distinguish between
Author statement
All persons who meet authorship criteria are listed as authors, and all authors certify that they have participated sufficiently in the work to take public responsibility for the content, including participation in the concept, design, analysis, writing, or revision of the manuscript. Furthermore, each author certifies that this material or similar material has not been and will not be submitted to or published in any other publication before its appearance in International Journal of Medical
Transparency document
Summary Table
What was already known
- -
Deep learning models for analyzing renal pathology have been developed to establish an efficient and objective diagnosis in renal pathology, mainly for detecting glomeruli in whole slide images.
- -
Previous studies for the classification of glomerular pathological findings have focused on a limited number of findings, and there has been no study which enables the comprehensive evaluation of findings necessary for the diagnosis.
- -
It also remains unknown whether these
Declaration of Competing Interest
E. Uchino and Y. Tamada were given a budget for a joint research project with Fujitsu Ltd. M. Yanagita received research grants from Astellas, Chugai, Daiichi Sankyo, Kyowa Hakko Kirin, Mitsubishi Tanabe Pharma Corporation, MSD, Baxter, Takeda Pharmaceutical, KISSEI PHARMACEUTICAL, Dainippon Sumitomo Pharma, TAISHO TOYAMA PHARM, and Torii. The other authors declare no conflicts of interest.
Acknowledgements
For annotation to the dataset and discussion, we thank the following nephrologists in Department of Nephrology, Graduate School of Medicine, Kyoto University:
Yuki Sato, MD, PhD; Akira Ishii, MD, PhD; Keita P. Mori, MD, PhD; Naohiro Toda, MD, PhD; Keisuke Osaki, MD; Sayaka Sugioka, MD; Shinya Yamamoto, MD; Keiichi Kaneko, MD; Shunsuke Kawamura, MD; Youngna Kang, MD; Takahisa Yoshikawa, MD; Yukiko Kato, MD, PhD; Makiko Kondo, MD; Shigenori Yamamoto, MD; Yuichiro Kitai, MD; Akiko Oguchi, MD;
References (43)
- et al.
Outcome study of renal biopsy patients in Okinawa, Japan
Kidney Int.
(2004) - et al.
Computational pathology: challenges and promises for tissue analysis
Comput. Med. Imaging Graph.
(2011) - et al.
Deep learning for digital pathology image analysis: a comprehensive tutorial with selected use cases
J. Pathol. Inform.
(2016) - et al.
An innovative neural network framework to classify blood vessels and tubules based on Haralick features evaluated in histological images of kidney biopsy
Neurocomputing
(2017) - et al.
Association of pathological fibrosis with renal survival using deep neural networks
Kidney Int. Rep.
(2018) - et al.
Segmenting renal whole slide images virtually without training data
Comput. Biol. Med.
(2017) - et al.
Segmentation of glomeruli within trichrome images using deep learning
Kidney Int. Rep.
(2019) The future of Radiology augmented with Artificial Intelligence: a strategy for success
Eur. J. Radiol.
(2018)- et al.
Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs
JAMA
(2016) - et al.
Dermatologist-level classification of skin cancer with deep neural networks
Nature
(2017)
Unintended consequences of machine learning in medicine
JAMA
A fully automated system using a convolutional neural network to predict renal allograft rejection: extra-validation with giga-pixel immunostained slides
Sci. Rep.
Deep learning–Based histopathologic assessment of kidney tissue
J. Am. Soc. Nephrol.
Glomerulus extraction by optimizing the fitting curve
2008 International Symposium on Computational Intelligence and Design
Glomerulus extraction by using genetic algorithm for Edge patching
2009 IEEE Congress on Evolutionary Computation
Automated quantitative image analysis of glomerular desmin immunostaining as a sensitive injury marker in spontaneously diabetic torii rats
Journal of Biomedical Image Processing.
Segmental HOG: new descriptor for glomerulus detection in kidney microscopy image
BMC Bioinformatics
Do we need large annotated training data for detection applications in biomedical imaging? A case study in renal glomeruli detection
Mach. Learn. Med. Imaging
An approach for detection of glomeruli in multisite digital pathology
2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI)
Automatic glomerulus extraction in whole slide images towards computer aided diagnosis
2016 IEEE 12th International Conference on E-Science (E-Science)
Extraction of glomeruli in whole slide imaging of kidney biopsy specimens
Medical Imaging 2017: Digital Pathology, Proc. of SPIE
Cited by (58)
Global attention based GNN with Bayesian collaborative learning for glomerular lesion recognition
2024, Computers in Biology and MedicineMonkeypox diagnosis based on Dynamic Recursive Gray wolf (DRGW) optimization
2024, Biomedical Signal Processing and ControlPerformance and limitations of a supervised deep learning approach for the histopathological Oxford Classification of glomeruli with IgA nephropathy
2023, Computer Methods and Programs in BiomedicineDiagnosis of diabetic kidney disease in whole slide images via AI-driven quantification of pathological indicators
2023, Computers in Biology and MedicineA scale and region-enhanced decoding network for nuclei classification in histology image
2023, Biomedical Signal Processing and ControlGenerative adversarial feature learning for glomerulopathy histological classification
2023, Biomedical Signal Processing and Control
- 1
These authors contributed equally to this work.