Development and Validation of a Deep Learning Model for Automated View Classification of Pediatric Focused Assessment with Sonography for Trauma (FAST)

INTRODUCTION

Focused Assessment with Sonography for Trauma (FAST) is a point-of-care ultrasound diagnostic study, in which a sequence of thoracic and abdominal ultrasonographic views performed by the treating provider is used to rapidly identify free fluid (hemorrhage) secondary to blunt abdominal trauma. The FAST is beneficial because it avoids the use of ionizing radiation, decreases computed tomography (CT) usage, and improves outcomes in injured adults.(1) However, FAST has not been shown to improve outcomes in injured children, mainly because of variable diagnostic test characteristics.(2) The varying diagnostic test characteristics of FAST in children arise from patient characteristics, but also inconsistent provider experience in technical reliability in image acquisition and interpretation, leading to variable diagnostic accuracy and provider confidence.(3-5) Expertise in FAST improves the overall accuracy of the study, and therefore, the utility in potentially reducing unnecessary CT use.(6) However, barriers in education, training, skill maintenance, and quality assurance have perpetuated a lack of global FAST expertise.(3, 7, 8) This lack of expertise is often cited as the primary reason for FAST not being accurately implemented in clinical practice.(9-11) Therefore, it would be advantageous to develop novel methods of disseminating widespread expertise in pediatric FAST, as a potentially life-saving and radiation-sparing diagnostic tool for use in the treatment of injured children.

Deep learning has been shown to be useful in many aspects of clinical image processing including automatic classification of diagnostic ultrasound, but not yet in pediatric FAST.(12) View classification models are often the first step in creating robust and interpretable deep learning models for successful diagnostic ultrasound applications.(13, 14) View classification allows researchers insight into the feasibility of a deep learning approach, while also contributing to the understanding of a multistage model.(15, 16) Previous diagnostic ultrasound deep learning classification models, including those for echocardiography and radiology-performed diagnostic ultrasound, have utilized imaging data acquired from operators often with advanced training and accreditation in image acquisition.(13, 14, 17) Accredited operators are apt to derive standard sequences and higher quality image capture.(18, 19) In contrast, pediatric FAST represents a distinct challenge because image acquisition and interpretation are performed by treating providers with varying ultrasound training and skill levels. As such, provider-performed FAST studies have variability in image acquisition,(20) creating a layer of complexity when building a classification model. Therefore, we sought to evaluate the ability of a deep learning model to recognize point-of-care ultrasound features for view classification, in an effort toward eliminating barriers to clinical implementation of FAST in the treatment of injured children.

MATERIALS AND METHODS

Imaging Dataset and Preprocessing

An imaging dataset of the pediatric FAST studies from children with blunt abdominal trauma was created. A FAST study was defined as one or more video clips from a single patient. Each video clip represents one of the possible FAST views (Figure 1). FAST views include standard abdominal and thoracic sonographic views, including the cardiac views, thoracic views, abdominal upper quadrant views, and suprapubic views. The ultrasound systems capture still frames at a rate of 30 still frames per second. Video clips for review were then generated at a rate of 60 still frames per second from labeling. These FAST still frames were all then resized to squared portable network graphics (PNG) of 299 × 299 pixels and normalized to appear with a standardized intensity.

• Download figure
• Open in new tab

FIGURE 1.

A FAST study is made up of a sequence of (a) FAST Views. Each FAST View is archived as a (b) FAST Video Clip(s). Each FAST Video Clip is made up of multiple (c) FAST Still frames.

RESULTS

We deidentified 701 FAST studies with recorded video clips. Two studies were excluded for missing linked patient-level data. 699 FAST studies were included in our analysis. There were a total of 4,925 video clips representing 1,062,612 still frames. The video clips were comprised of abdominal upper quadrant (45.8%), suprapubic (24.8%), cardiac (18%), and thoracic (12%). The majority of the FAST studies came from children less than 11 years of age, and one-third of the total studies came from children less than 6 years of age (Table 1). Sixty percent of the cohort were males. The majority of FASTs were obtained at the level 1 pediatric trauma center and were performed by 30 attending providers. There was no single attending provider that performed more than 11% of the FAST studies.

The 699 FAST studies were divided into the training (70%), validation (20%), and test (10%) datasets (Table 1). Age, gender, source, and expert labeling were evenly distributed between the different datasets. The test dataset was made up of 70 FAST studies, 459 video clips, and 104,220 still frames, which had a slightly higher proportion of studies from children 6-10 years of age, females, and suprapubic video clips/still frames.

FAST View Classifier

After training on 757,583 frames, we applied the model to the test dataset. The standard clock time needed to predict the view in 1000 still frames in this set was approximately 40 seconds. The overall model classification accuracy was 97.8% (96.0 - 99.0%) for video clips, and 93.4% (93.3 – 93.6%) for still frames (Table 2). The F-scores were 0.98 and 0.94 for video clips and frames, respectively. The model had view classification accuracies of video clips of 98.7%, 99.8%, 98.3%, and 98.9% for the cardiac, thoracic, abdominal upper quadrant, and suprapubic views, respectively. The model had view classification accuracies of still frames of 96.0%, 99.8%, 95.2%, 95.9% for the cardiac, thoracic, abdominal upper quadrant, and suprapubic views, respectively. The clustering analyses showed that the model could sort FAST still frames into groups according to FAST view (Figure 2). The model performed with similar accuracy on the augmented test dataset (Table 3). Based on two independent study experts, Grad-CAM indicated that the model received prediction contributions from relevant view classification sites (Figure 3).

View this table:

• View inline
• View popup
• Download powerpoint

TABLE 2.

Pediatric FAST view classification accuracy by the deep learning model for (a) video clips and (b) still frames.

View this table:

• View inline
• View popup
• Download powerpoint

TABLE 3.

Pediatric FAST view classification accuracy (95% confidence interval) by the deep learning model for still frames after augmentation techniques.

• Download figure
• Open in new tab

FIGURE 2.

T-distributed stochastic neighbor embedding (t-SNE) plot. Clustering analysis of pediatric FAST views: Cardiac views (red), Thoracic views (peach), Abdominal upper quadrant views (light blue), suprapubic views (dark blue).

• Download figure
• Open in new tab

FIGURE 3.

Gradient Class Activation Mapping (Grad-CAM). A. Thoracic view with highest contribution from costal-pleural interface. B. Cardiac view with highest contribution from left atrium. C. Upper Quadrant view with highest contribution from spleen-renal-diaphragm interface. D. Suprapubic view with highest contribution from posterior bladder wall.

We also evaluated a model with the view classification accuracy with the additional discernment of the abdominal right and left upper quadrant views. Forty-nine (24%) video clips of the upper abdominal views received a secondary expert label for right or left upper quadrant. 151 (76%) were given a definitive label, of which 69 (46%) right upper quadrant and 82 (54%) left upper quadrant views. The overall model view classification accuracies were 88.5% (85.0 – 91.5%) for video clips and 83.5% (83.3 – 83.8%) for still frames (Supplement Table A). The F-scores were 0.88 and 0.83 for video clips and still frames, respectively. In this five-class model, view classification accuracy was lower in the abdominal upper quadrant views than the other anatomic views.

DISCUSSION

We found that a deep learning model can be trained to accurately perform view classification from a diverse set of real-world clinician-performed pediatric FAST studies. The model’s high accuracy implies the ability to parse out identifying features within an ultrasound image. Feature identification is the essential first step toward developing an accurate and interpretable deep learning multistage model to automate pediatric FAST. Notably, the model is accurate in separating abdominal FAST views, including the suprapubic view, which is the most sensitive abdominal view for intraabdominal hemorrhage detection in pediatric patients.(26)

The inclusion of real-world clinician-performed FAST studies strengthens our model’s generalizability by avoiding an idealized dataset. Further, this study’s strength is that our pediatric FAST dataset is a relatively large expert-labeled set of pediatric FAST video clips and still frames from children with suspicion for abdominal hemorrhage. Similarly, data were curated from a large group of clinician-performed FAST studies from 30 providers with varying levels of FAST training and expertise. This dataset is also unique given the relatively young patient population, with greater than 30% of children less than six years of age.

Our model’s pediatric FAST view classification accuracy suggests that a deep learning model can identify pertinent features to achieve class predication accuracy nearly as well as an expert. There is no clear association of the number of FAST studies one needs to perform for expertise gained.(27) Furthermore, the FAST was accepted by the trauma community in the 1990s (28, 29), but barriers to training and implementation still exist, suggesting a need for alternative methods to develop its use.(20) Pediatric FAST is a technically complex study because of a child’s compact anatomy. Previous reports of FAST show excellent reliability for view classification between expert FAST providers, but only moderate agreement for less experienced users.(30) Our model performed with similar accuracy to that of an expert, suggesting deep learning may offer a possible solution for variability between providers with inconsistent expertise in FAST.

Deep learning classifiers have been trained and tested in different diagnostic ultrasound applications, including echocardiography view classification and lesion detection within the liver and breast.(12, 13, 15) However, FAST represents a complex problem because of the variability in the expertise of FAST operators and inconsistency in the quality of images obtained. In contrast to other deep learning classifiers, FAST is performed by the treating provider instead of a certified ultrasonography technician, and real-time clinical decisions are made from the results. Furthermore, these studies are also conducted at a fast pace within a high stakes environment that is prone to error.(31, 32) The inclusion of a heterogeneous group of FAST operators adds natural variation to the dataset and increases the complexity of the task.

Our findings build on previous work, demonstrating that machine learning models can identify pertinent features from FAST. Specifically, Sjogren et al. demonstrated feasibility in using basic machine learning techniques, including support-vector machine models, in interpreting adult FAST.(33) The model achieved 100% (95% CI, 69 −100%) sensitivity and 90% (95% CI, 56-100%) specificity for free fluid detection using only the abdominal right upper quadrant view from 10 positive and 10 negative adult FAST studies. However, this study utilized a labor-intensive annotation method, a basic machine learning approach, a small number of FAST studies, and may not be adaptable to pediatric FAST, where post-traumatic abdominal pathology displays wide variability. In contrast, our study demonstrates that less labor-intensive expert labeling, along with an advanced deep learning technique, can also perform with a high degree of accuracy. Our view classification deep learning model also suggests that these techniques are well-suited for additional development of a multistage free-fluid classifier to interpret the presence or absence of free fluid representing intraabdominal hemorrhage for the pediatric FAST (1, 34, 35).

One of this study’s goals was to develop automated processes and methods to reduce the need for human input. Human input in dataset curation is laborious, subject to human error, resource-intensive, and is a rate limiting step in high-quality, in-depth learning model development.(12) Our automated image preprocessing pipeline and view classification model can accurately expedite the development of larger and more robust datasets for the future multistage FAST models.(36) Our imaging preprocessing pipeline masked extraneous data outside the ultrasound viewing area to avoid training from non-clinical relevant features.

Additionally, our preprocessing pipeline included data augmentation techniques to simulate image variation outside the standard differences in image quality to avoid overfitting the model. Similar to our image preprocessing, we developed a system optimized for discordant frame evaluation in which experts could quickly review images (Supplement Figure A). This discordant image review process allows for a rapid review and understanding of model inaccuracy.

We recognize there are limitations to our study design. First, this study’s primary outcome was to develop an accurate deep learning model for view classification of pediatric FAST instead of pathology identification. However, recognizing relevant features within pediatric FAST still frames ensures our preprocessing pipeline is working and that the model distinguishes relevant features. Second, in our secondary analysis, the accuracy declined once the model was tasked to discriminate between right and left abdominal upper quadrant views. These two views are similar in sonographic appearance, which may be why the model has difficulty in classification, but could also represent the difficulty in the expert’s interpretation of right from left. Similar to previous studies of FAST view classification, the abdominal upper quadrant view includes both right and left because of the similarity of features and location of free fluid detection, which may be difficult for clinicians to define.(30)

Third, our model’s generalizability is limited by using a small group of FAST experts as the reference, and the majority of images came from a single institution and one ultrasound manufacturer. Previous studies have shown a strong agreement between experts for FAST study view classification.(30) Our team created a series of augmentations to simulate a more robust set of scenarios to account for homogenized institutional and manufacturer data. These simulations and reviewable frameworks will be necessary for future models and will allow our team to expand to include more extensive, more diverse populations. These augmentation methods will be important for future work as they require less effort without compromising accuracy.(37)

CONCLUSION

In conclusion, we demonstrate that a deep learning model can be developed to provide an accurate view classification for a heterogeneous set of pediatric FAST video clips and still frames. This model suggests that our in-depth learning approach recognizes discerning imaging features within the pediatric FAST. Accurate view classification is the important first step to support developing a consistent and accurate multi-stage deep learning model for pediatric FAST interpretation.