Screening for Chagas disease from the electrocardiogram using a deep neural network

Introduction

Worldwide it is estimated that Chagas disease (ChD) infects more than 6 million people, with thousands of deaths each year [1]. Caused by the protozoan parasite Try-panosoma cruzi (T. cruzi), the disease is endemic to countries in continental Latin America, but migration has carried ChD to new regions, including Europe and the United States [2]. The most critical consequence of ChD is chronic Chagas cardiomyopathy (CCC), which occurs in 20-40% of the infected individuals [3]. CCC comprises a wide range of manifestations, including heart failure, arrhythmias, heart blocks, sudden death, thromboembolism, and stroke [1], [3].

ChD is often a lifelong infection in which most chronically infected patients remain asymptomatic but at risk of progression to cardiac damage [4], [5]. The incidence of cardiomyopathy in those in this asymptomatic (indeterminate) form of ChD varies from 0.9 to 7% new cases annually [1] and is related to the parasite burden [5], [6]. There is no single gold-standard laboratory test for diagnosing chronic Chagas disease. Instead, at least two serological tests with different methods for detecting antibodies to T. cruzi and complementary sensitivity and specificity are needed to confirm infection [1], [3]. Treatment with antitrypanosomal drugs such as benznidazole can prevent progression to the cardiac form [7], [8], but it does not seem to prevent death and cardiac complications in those with advanced cardiomyopathy [9]. Thus, the early recognition of chronic ChD patients is a necessary step for treatment in the early phases, when treatment success rates are higher and can prevent severe organ damage from occur [10].

Even if the newly diagnosed patient has established cardiomyopathy, an early diagnosis will allow the initiation of guideline-directed medical therapy for clinical conditions, such as heart failure and atrial fibrillation, to halt disease progression and eventually prevent death [10]. ChD patients generally have low socio-economical levels and limited access to health services, and they frequently do not realize that they are infected. The awareness of ChD among healthcare providers is also low, and there is a lack of knowledge on who to screen as well as a lack of clarity on the appropriate tests and clinical management [11], [12].

In many countries, there are detection rates below 10%, even more frequently, below 1%. The low detection rates create a barrier to the health care system, preventing patients from receiving adequate treatment [13]. The under-appreciation of early diagnosis and treatment, especially at the primary healthcare level, represents a missed opportunity for modifying the natural history of the disease [10]. For this reason, the theme of World Chagas Disease Day 2022 was “finding and reporting every case to defeat Chagas disease” [13].

Here we study the possibility of using the electrocardiogram (ECG) to screen for ChD. The ECG is a widely available, low-cost exam, often provided in primary care settings in endemic countries [14]. The automated analysis of ECG is a successful technology and has already improved the analysis of this exam over the past decades [15].

The field of artificial intelligence, in particular deep learning [16], has demonstrated promising performance for automated analysis. Besides the success of classifying common ECG diagnoses with high-performance [17], [18], the technology has presented successes in predicting and screening for diseases and diagnoses which traditionally were not directly possible only from the ECG. These include detection of myocardial infarction without ST-elevation [19], predicting the future development of atrial fibrillation from sinus rhythm exams [20], [21] and the ability to screen for cardiac contractile dysfunction [22]. Indeed, there is evidence that deep learning reading of ECGs detects more than traditional features, as is indicated by studies showing good prediction of age and even the risk of death [23]–[25].

In this study, we investigate whether a deep neural network can detect ChD and CCC from ECG tracings. Being able to evaluate ChD from this exam can help to detect cases in an early stage and enables early and more effective treatment.

Methods

Model

Data preprocessing

The ECG signals have been resampled such that all ECGs have the same sampling frequency of 400 Hz. Each input ECG has 4, 096 time samples for each of the 12 standard ECG leads. Original signals of a shorter time span have been extended through zero-padding. The output data comprises binary scalar variables corresponding to positive or negative diagnose. We combine positive cases with and without CCC in our model in order to focus on the class of positive ChD cases in general.

Architecture

The deep learning model consists of a residual neural network (ResNet) adapted to uni-dimensional signals, and includes convolutional layers both before and within the residual blocks. Our network architecture is visualised in Figure 1. We make use of the same network architecture as [17], where the CODE data set was utilised to classify multiple ECG abnormalities; we refer to that work for further details and note that we have modified the final output layer in adaptation to our binary classification. The model is implemented in PyTorch [38], building upon code used in related work [39], [40].

• Download figure
• Open in new tab

Figure 1: Network architecture.

The figure was originally illustrated in [17].

Parameter tuning

The learnable parameters of the neural network are chosen through minimisation of the binary cross-entropy loss function. For increased computational efficiency, we split the training data into mini-batches of size 32.

We use both the CODE and SaMi-Trop data sets during the training phase. This way, we utilise the size of the CODE data set — with many examples of negative diagnoses — as well as the high-quality (mainly positive) entries of SaMi-Trop. Both data sets contribute with 50% of the data that the model experience in each mini-batch. The validation data is an independent mix of 30% of the SaMi-Trop entries and twice as many entries from CODE.

The dropout rate is 0.5, and we use a weight decay of 0.001 to reduce the risk of overfitting. The learning rate is initially set to 0.001 and is decreased in a step-wise manner by a factor 10 when the validation loss has not decreased for ten subsequent epochs (counted with respect to SaMi-Trop) — we terminate the optimisation if the learning rate drops below 10⁻⁷. We apply early stopping by using the network parameter values associated with the lowest validation loss for testing.

To reduce the sensitiveness of the weight initialisation, we use an ensemble approach by running the optimisation 15 times with different random seeds, and then averaging the outputs of the final models. The progression of the losses evaluated on the training and validation data sets are displayed in Figure 2.

• Download figure
• Open in new tab

Figure 2: Loss function evaluation.

The shaded regions correspond to the maximum and minimum values of 15 separate learning processes with different weight initialisations. The solid lines are the averages.

Threshold selection

The model output is a value between 0 and 1 and can loosely be interpreted as the predicted probability of ChD being present in the exam analysed. The Chagas diagnose is predicted as positive when the model output is above a given classification threshold. We consider two different approaches to selecting the threshold. The first one is by maximising the F1 score (i.e. the harmonic mean of precision and recall) on the validation data. This threshold is suitable for balanced or moderately imbalanced data sets where the main interest is to diagnose the patients under consideration.

The second approach is to choose the threshold by requiring a certain specificity on the validation data. The higher the specificity, the more likely is the model to correctly diagnose a negative patient. As a high specificity typically is desired for screening purposes, this approach for threshold selection is motivated on highly imbalanced data sets (which reflects the Chagas prevalence in the population as a whole).

The first approach is used on the REDS-II test set since this data set is only moderately imbalanced (55.8% ChD and 23.6% CCC ECGs). On the ELSA-Brasil test set the threshold is selected according to the second approach since this data set is more imbalanced (2.0% ChD and 0.3% CCC ECGs). We select the threshold by requiring a 90% specificity on the validation data.

Results

We evaluated the model performance on the validation data and the external test data sets. The ROC curve performance is displayed in Figure 3. The model attains AUC-ROC values of 0.80 (CI 95% 0.79-0.82) for the validation data set, 0.68 (CI 95% 0.63-0.71) for REDS-II and 0.59 (CI 95% 0.56-0.63) for ELSA-Brasil. The confidence intervals have been formed by bootstrapping the output of the ensemble model. Table 2 lists all performance metrics evaluated on the validation data for two different thresholds selected through the aforementioned approaches. The same metrics evaluated on the test data sets are listed in Table 3. Additionally, we also analysed the precision-recall curve and the empirical probabilities predicted by the model. These results are displayed in the Supplementary Material Figures S.1-S.4. The metrics for subgroups stratified by age and sex are displayed in Figure 5.

• Download figure
• Open in new tab

Figure 3: ROC curves on validation and test data: ChD vs normal.

Receiver operating characteristics (ROC) computed on the validation data set (SaMi-Trop + CODE) (a) and the test data sets, REDS-II (b) and ELSA-Brasil (c). The shaded regions encapsulate the maximum and minimum values corresponding to 15 different weight initialisations. The outputs of the 15 trained models are averaged to produce the output of the ensemble model, the result of which is shown by the solid lines. The 95% CI is obtained by bootstrapping the ensemble model. The dotted blue lines correspond to completely random assignment of class probabilities.

• Download figure
• Open in new tab

Figure 4: ROC curves on test data: CCC vs all.

Receiver operating characteristics computed in REDS-II and ELSA-Brasil for predicting Chagas Cardiomiopathy. The shaded regions encapsulate the maximum and minimum values corresponding to 15 different weight initialisations – the outputs of these models are averaged to produce the output of the ensemble model, the result of which is given by the solid lines. The dotted blue lines correspond to completely random assignment of class probabilities.

• Download figure
• Open in new tab

Figure 5: Results stratified by subgroup.

Box plots of the model performance on the REDS-II (top row) and ELSA-Brasil (bottom row) test sets stratified by age (left column) and sex (right column). The box plots give the performance on 1000 bootstrapped samples.

We also evaluated the model for considering only patients with CCC as positive. In this case, the model attains an AUC-ROC of 0.82 (CI 95% 0.77-0.86) for REDS-II and 0.77 (CI 95% 0.68-0.85) for ELSA-Brasil. All metrics for this configuration are included in Table 3.

In the Supplementary Material Figure S.5 and Table S.1, we show the additional results for another test set configuration. Namely where the patients with CCC have been excluded; the remaining patients where ChD was detected are here constituting the positive cases (this configuration is indicated “no CCC”). In the Supplementary Material, we also show the result of a model trained to detect CCC (with all others being considered negative). Figure S.6 shows the training curve, Figure S.7 shows the ROC curves, precision-recall curves and empirical distribution of the probabilities, and finally, Table S.2-S.3 give the performance metrics in this case.

The Grad-CAM analysis is presented in Figure 6, which shows three representative leads of a patient with CCC from the ELSA-Brasil data set. The shaded regions illustrate what parts of the signals the model considers to be of particular importance for the prediction. In the Supplementary Material Figure S.8 we include the equivalent plots for another three patients with positive Chagas diagnose, one with and two without CCC.

• Download figure
• Open in new tab

Figure 6: Grad-CAM analysis.

Grad-CAM plot for a patient with CCC from the ELSA-Brasil data set, correctly classified by the model as Chagas positive. This plot includes three representative leads (top to bottom: aVL, V1 and V6). The shading indicates regions that the model assigns particular importance for its prediction.

Discussion

Deep neural network-enabled analysis of the ECG is a topic of intense research [19]–[25]. Such methods have shown promising potential in detecting diverse conditions that are not traditionally diagnosed from the ECG, such as contractile disfunction [22] or non-STEMI myocardial infarction [19]. ChD is the parasitic disease with the most impact in South America [43] and it affects the lives of millions of individuals worldwide. Early detection of this disease can therefore have a huge impact. Antiparasitic drugs are most effective in the early stage of the disease, however, most patients only become aware that they are infected much later when the patient is already in the later stage of the disease and presents other manifestations. Providing early treatment and the usage of advanced artificial intelligence or machine learning methods for the detection of this disease presents itself as a promising alternative. To the best of the authors’ knowledge, this is the first study to present such an application.

The development of data-driven methods for automatic diagnosis of neglected diseases presents a challenge of its own. These diseases usually affect areas where the population is underprivileged and have little access to the health-care system. The data might not come in well-organised databases or might not even be stored in electronic format. In this sense, the CODE, SaMi-Trop, ELSA-Brasil and REDS-II cohorts are extremely valuable: they are medium or large-size and well-kept data sets that can be used for developing and testing such tools.

The results we present are promising and indicate that the model is capable of detecting patients with CCC from the ECG tracings with high discrimination. For patients without CCC, the discrimination is lower.

In light of the results, it is natural to ask if we can further improve the performance with respect to patients with CCC. Therefore, we restrict the positive diagnoses to patients with CCC during the training phase and consider all patients without CCC as negatives (this implies that ChD positive patients without CCC are considered negative in this scenario). The result of this approach is given in the Supplementary Material. All metrics considered, except for the recall, are indeed improved. Thus, this model might be the preferable choice for CCC detection.

Chagas cardiomyopathy is characterised by a group of typical ECG abnormalities, frequently combining conduction disturbances, especially right bundle branch block with left anterior hemiblock, associated with rhythm disorders, such as ventricular ectopic beats and atrial fibrillation [44]– [46]. Thus, it is unsurprising that our Grad-CAM analysis depicts exactly the late portion of the QRS in cases with a bundle branch block. It is interesting that the Grad-CAM map also depicts the QRS complex when recognising the ChD patients with CCC, maybe related to the presence of high frequency, low amplitude abnormalities typical of fibrosis, which can occur early in the natural history of ChD [47]. However, this type of analysis has clear limitations [48], [49] since heatmaps can provide information on where the critical area for the neural network model is to make a decision but not inform if the abnormality is related to changes in voltage, duration or morphology modification of the ECG tracing. Moreover, recurrent features, like the RR interval, are not shown in this kind of heatmaps. Our analysis here is also limited to a small set of correct model predictions and does not represent a statistical analysis. Hence, we cannot deduct general rules for the diagnosis of ChD but we can identify from the unsurprising areas where the model focuses on that it does not use some unrelated proxy information to make its predictions.

Comparing the two test data sets, we obtain similar performance for discrimination in terms of AUC-ROC, but very different precision. This indicates that our model predicts many false positives for the ELSA-Brasil data set. Given the vast difference in prevalence for ChD patients in ELSA-Brasil (2.0%) and REDS-II (55.1%), it is reasonable that for ELSA-Brasil our model will by default have lower precision. We can also observe the large portion of false positive cases in Figure S.3c when choosing a threshold of 0.60 (based on F1 score) or even 0.71 (based on 90% specificity). We believe the performance could be improved with the addition of epidemiological questions, and that our model can be a useful tool in helping pre-selecting patients for further testing in order to determine the infection with ChD.

As previously mentioned, the ChD status in the CODE data set is based on self-reporting by the patients, and the labels are thus suffering from notable uncertainty. Thus, testing on these labels might be uninformative and we have used more reliable databases such as ELSA-Brasil and REDS-II to get a better estimate of our model performance. Nonetheless, the labels in CODE still contain a sufficient amount of information to learn about CCC patients and the data set was indeed useful in developing a better-performing model. Methods designed to reduce the impact of label noise (see e.g. [50], [51]) could potentially be employed for more efficient use of the CODE data.

Our model could be even more insightful if we could test it on other openly available data sets. However, data sets about neglected diseases are scarce and both ELSA-Brasil as well as REDS-II are valuable but also medium to large-scale sources to rigorously test the model. Furthermore, a comparison with other models or software for Chagas detection would be useful, but unfortunately, it is not possible — to the best of our knowledge, this is the first work that tackles automatic diagnosis of Chagas directly from the ECG. Therefore, this study serves as a first baseline that opens a new line of work for further improvements.

Our findings are particularly valuable under the scantiness of validated strategies to detect ChD patients in endemic regions. Current recommendations for screening include all patients who were born in or have lived for an extended period in ChD endemic zones [44], which can be challenging, especially in endemic countries, since it can encompass the whole population of a region. A risk score was developed specifically to answer the question, “Does my patient have chronic Chagas disease?” but it seems to have limited practical value since it includes 13 variables obtained from clinical and epidemiological history and from a conventionally analysed 12-lead ECG [52]. It implies that the best approach would merge conventional and non-conventional methods [53], including the use of rapid point-of-care serological tests [54].

A clinical study would be particularly valuable, as the performance of the model could be evaluated directly by clinicians and patients. At this stage, we foresee our model as a pre-selection method of patients for further screening of the serological status. It is important to underline that more available data will enable improvements of the model that can be adapted into its daily clinical practice. We hope that a future study will evaluate the clinical relevance of our model to improve the early diagnosis of ChD.

Contributors

AHR, CJ, ALPR were responsible for the study design. ALPR, TBS, AHR conceived the project and acted as the project leader. CJ, DG, AHR choose the neural network architecture, implemented, and tuned the deep neural network. CJ, DG did all the statistical tests. ALPR, CSC, CLO interpreted the results and provided clinical interpretation. CJ, AHR, DG were responsible for preprocessing the training data. ECS, CSC, CLO, AMF were responsible for cohort design and management, data acquisition, follow-up, and ECG exams in SaMi-Trop and REDS-II cohorts. LG and SMB were responsible for cohort design and management, data acquisition, follow-up, and ECG exams in ELSA cohort. ALPR and AHR were responsible for management of the CODE cohort. AHR, CJ, ALPR, DG, TBS contributed to the writing and all authors revised it critically for important intellectual content. All authors read and approved the submitted manuscript.

Competing interests

There are no competing interests.

Data sharing

SaMi-Trop cohort was made openly available (https://doi.org/10.5281/zenodo.4905618). The CODE-15% cohort was also made openly available (https://doi.org/10.5281/zenodo.4916206). The data sets contain information about Chagas condition mortality, age, sex, the ECG tracings, and the flag indicating whether the ECG tracing is normal. The DNN model parameters that give the results presented in this paper are also available (https://doi.org/10.5281/zenodo.7371623). This should allow the reader to partially reproduce the results presented in the paper. Restrictions apply to additional clinical information on the CODE-15% and SaMi-Trop cohorts; to the full CODE cohort; to the REDS-II dataset and, to the ELSA-Brasil cohort. Researchers affiliated to educational or research institutions might make requests to access the data sets. Requests should be made to the corresponding author of this paper. They will be forwarded and considered on an individual basis by the Telehealth Network of Minas Gerais, SaMi-Trop and ELSA-Brasil Steering Committee. An estimate for the time needed for data access requests to be evaluated is three months. If approved, any data use will be restricted to non-commercial research purposes. The data will only be made available on the execution of appropriate data use agreements.

Code availability

The code for the model training, evaluation and statistical analysis is available at the GitHub repository https://github.com/carji475/ecg-chagas.

Supplementary material

• Download figure
• Open in new tab

Figure S.1: Precision-recall curves on validation and test data: ChD+CCC vs normal.

The shaded regions encapsulate the maximum and minimum values corresponding to 15 different weight initialisations — the outputs of these models are averaged to produce the output of the ensemble model, the result of which is given by the solid lines.

• Download figure
• Open in new tab

Figure S.2: Precision-recall curves on test data: CCC vs rest.

We consider only CCC as positive and ChD as well as normal as negative here.

• Download figure
• Open in new tab

Figure S.3: Output histograms: ChD+CCC vs normal.

Histograms are computed on the validation data and the test data. Note the logarithmic scale of the y-axis. We can see the number of false positive/negatives when applying the selected thresholds on the x-axis: 0.60 for REDS-II and 0.71 for ELSA-Brasil.

• Download figure
• Open in new tab

Figure S.4: Output histograms: CCC vs rest.

Histograms are computed on the test data where we only consider CCC as positive and ChD as well as normal as negative. Note the logarithmic scale of the y-axis.

• Download figure
• Open in new tab

Figure S.5: Results on test data: no-CCC configuration.

Receiver operating characteristics (left), precision-recall curves (middle) and output histograms (right) computed on the test data for the no-CCC configuration (see the text for details). This set removed the CCC cases and shows ChD vs normal.

• Download figure
• Open in new tab

Figure S.6: Loss function evaluation: CCC-specific training.

Equivalent to Figure 2 when the training is adapted to specifically target patients with CCC.

• Download figure
• Open in new tab

Figure S.7: Results on test data: CCC-specific training.

Receiver operating characteristics (left), precision-recall curves (middle) and output histograms (right) computed on the validation data and the test data when the training is adapted to specifically target patients with CCC.

• Download figure
• Open in new tab

• Download figure
• Open in new tab

Figure S.8: Grad-CAM analysis: additional patients.

Complementing Figure 6 with another three Grad-CAM plots for patients from the ELSA-Brasil data set, correctly classified by the model as Chagas positive. We here include one patient with chronic Chagas cardiomyopathy (a), and two without (b-c). The plots include three representative leads (top to bottom: aVL, V1 and V6). The shading indicates regions that the model assigns particular importance.