A One-Step open RT-qPCR for SARS-CoV-2 detection

Ariel Cerda, Maira Rivera, Grace Armijo, Catalina Ibarra-Henriquez, Javiera Reyes, Paula Blázquez-Sánchez, Javiera Avilés, Aníbal Arce, Aldo Seguel, Alexander J. Brown, Yesseny Vásquez, Marcelo Cortez-San Martín, Francisco Cubillos, Patricia García, Marcela Ferres, César A. Ramírez-Sarmiento, Fernán Federici, Rodrigo A. Gutiérrez
doi: https://doi.org/10.1101/2021.11.29.21267000

Abstract

The Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), etiological agent of the coronavirus disease 2019 (COVID-19), is currently detected by reverse transcription followed by quantitative polymerase chain reaction (RT-qPCR) of its viral RNA genome. Within the available alternatives, One-Step procedures are preferred since they are fast and significantly decrease preanalytical errors, minimizing the risk of diagnostic errors.

Increasing the testing capacity and tracing contacts are essential steps to control the pandemic. However, high-cost commercial reagents subject to shortage and poor scalability have hindered the use of these technologies and their adoption for a wide population-scale testing, being even more critical in developing countries. In the current context, open-source initiatives have promoted global collaboration to promote accessible solutions for rapid local deployment. As a result, open protocols are being developed for the local production of SARS-CoV-2 diagnostics.

This work aimed to produce an open-source system for SARS-CoV-2 diagnostic tests in RNA clinical samples. We provide guidelines for standardizing an open One-Step RT-qPCR master mix using recombinant M-MLV reverse transcriptase together with either Pfu-Sso7d or Taq DNA polymerase. Both were tested on synthetic RNA and clinical samples, observing a good correlation when compared to commercial RT-qPCR kits. Nevertheless, the best results were obtained using M-MLV RT combined with Taq DNA polymerase in a probe-based RT-qPCR assay, allowing successful discrimination between positive and negative samples with accuracies comparable to a CDC-recommended commercial kit.

Here, we demonstrate that these open RT-qPCR systems can be successfully used to identify SARS-CoV-2 in clinical samples and potentially be implemented in any molecular diagnostic laboratory.

Introduction

The Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) is a positive-sense single-stranded RNA virus, member of the Coronaviridae family, and the etiological agent of the coronavirus disease 2019 (COVID-19)[1]. The symptoms expressed by COVID-19 patients vary between fever (89%), cough (68%), fatigue (38%), sputum production (34%), and shortness of breath (19%). However, these symptoms are not necessarily specific to COVID-19 and cannot be used for an accurate diagnosis[2]. Therefore, molecular techniques are more suitable to identify specific pathogens, playing an essential role in SARS-CoV-2 detection[3].

Although lockdown, social distance, mask usage, and vaccination measures have helped in containing the spread of the virus, measures such as testing and tracing for the rapid and efficient identification of positive COVID-19 individuals are still essential for containment of the pandemic[3]. In this regard, the significant challenges in molecular diagnosis are detecting small amounts of viral RNA, discriminating a positive signal among different pathogens, and being able to test large numbers of patients in a time-effective manner[4]. Laboratory diagnosis of SARS-COV-2 is mainly based on nucleic acid amplification tests (NAAT)[5], such as the One-Step RT-qPCR recommended by sanitary authorities[6,7]. One-Step procedures are preferred as they avoid potential preanalytical errors occurring during sample preparation, including pipetting, manual sample preparation, aliquoting, cross-contamination, sample mismatch, among others[8]. Reducing the steps involved in laboratory testing speeds up the test and minimizes the risk of diagnostic errors, thus improving the accuracy of molecular diagnostics.

Despite the high supply of RT-qPCR reagents and kits, these have not been enough to meet the high demand. Healthcare providers worldwide have been frequently left without supplies to efficiently test samples because of unprecedented volume testing needs and supply-chain shortages caused by the COVID-19 pandemic[9]. These obstacles hinder the control of the pandemic, as increasing testing capacity and contact tracing are essential in surveillance tests to quickly and accurately identify infected individuals[10]. In addition, the use of high-cost commercial reagents subject to shortage and poor scalability has limited the use of these technologies for population-scale testing[11]. This has also posed challenges for low-middle income countries (LMIC), particularly in the Southern Hemisphere due to their dependency on the import of reagents. This is worsened by regional factors such as social inequalities, deficient public health systems, lower living standards, and a lack of services in densely populated urban areas or widely dispersed rural communities[12-14].

To overcome this problem, molecular supplies that can be efficiently produced at low cost and deployed locally are necessary. Unfortunately, enzymes used in diagnosis protocols, such as reverse transcriptases (RT) and DNA polymerases (DNA pol) essential for RT-qPCR, are usually subjected to centralized production models and intellectual property restrictions[15]. Thus, the access, production, and transfer of these components between researchers are highly restricted, hampering the rapid response of LMIC to testing and tracing emergent pathogens.

Open-source technology has promoted global collaboration, offering more accessible solutions and distributed responses[16,17]. Open protocols and accessible resources are being developed for the local production of SARS-CoV-2 diagnostics. For instance, Graham et al (2021) evaluated low-cost and more straightforward protocols for RNA extraction and RT-qPCR. This study provides an open-source master mix for RT-qPCR that can be used in nasopharyngeal swab samples with results similar to commercial kits and a limit of detection of ∼50 RNA copies[18]. Other studies provide procedures to prepare reagents and a ‘blueprint’ that can be followed to replicate RT-qPCR kits in academic laboratories to overcome test shortages[19,20]. In addition, a barcoding system has also been proposed to reduce manipulation and scale the number of samples per sequencing run[21]. Using this open-source multiplexed platform together with homemade enzymes and non-protected buffers considerably reduces the cost per sample. Finally, a different dye-based RT-qPCR assay approach employing a patented thermostable enzyme (RTX) with both RT and DNA pol activity[22] led to similar One-Step one-enzyme developments using Taq DNA pol, a public domain enzyme with intrinsic RT activity, by optimizing the buffer conditions to promote its intrinsic activity[23]. Most of these working groups provide detailed protocols for the purification of RT-qPCR reagents and distribute them freely[18,20,22,23]. Besides, there are also international networks such as the Reagent Collaboration Network or ReClone (https://reclone.org), whose approach is gathering and sharing open resources for diagnostics. This initiative uses off-patent enzymes, open access protocols, and material transfer under an openMTA[24], which guarantees open and expedited resources access.

Despite the potential of open-source methods for SARS-CoV-2 molecular diagnosis, these would be irrelevant if they are not assessed in a real-world setting such as clinical samples. This is a critical step for promoting a wider utilization of the open-source methods since the detection of viral RNA from clinical specimens is a decisive factor for optimal assay accuracy[25]. Furthermore, the nature and quality variability of clinical samples (quantity and type of sample, presence of possible inhibitors, viral load, among others) make it necessary to standardize this system, avoiding potential preanalytical and analytical errors vulnerabilities[8]. Furthermore, the quality of RT-qPCR testing provides accurate and interpretable results, irrespective of whether the tests are carried out using conventional or open-source molecular resources[26,27].

In this work, framed within the ReClone collaboration network, we aimed to produce an open-source system consisting of recombinant enzymes purified in-house and a standardized RT-qPCR protocol for diagnostic tests to detect SARS-CoV-2 RNA in clinical samples. We provide a straightforward protocol for standardizing a One-Step RT-qPCR master mix using Moloney Murine Leukemia Virus RT (M-MLV) and Pyrococcus furiosus DNA pol fused to the Saccharolobus solfataricus DNA-binding protein 7d (Pfu-Sso7d)[28] or Taq DNA polymerase (Taq DNA pol)[18]. Both systems were tested on synthetic RNA and clinical samples and no significant differences were observed with commercial RT-qPCR kits. However, the best results were obtained using M-MLV RT and Taq DNA pol, since they allowed a better distinction between positive and negative samples.

Results

Enzyme Purification and One-Step RT-PCR standardization

M-MLV RT[29], Pfu-Sso7d polymerase[28], and Taq DNA polymerase[18] in-house enzymes were produced by recombinant expression in Escherichia coli, as described in the Material & Methods section. Briefly, protein overexpression was induced by IPTG; then cells were collected, resuspended in lysozyme-supplemented lysis buffer, and disrupted by sonication. For the thermostables Pfu-Sso7d and Taq DNA pol, the cell lysates were also subjected to heat-treatment to unfold and precipitate most of E. coli proteins. Next, the collected soluble fractions were subjected to a two-step protein purification process of immobilized metal (IMAC) and heparin affinity chromatography, and the protein-containing fractions were pooled and analyzed by SDS-PAGE (Supplemental Figure 1).

Subsequently, the performance of purified enzymes was assessed. First, Pfu-Sso7d polymerase activity was evaluated, amplifying different DNA fragment sizes from a cDNA sample (Supplemental Figure 2A). Afterwards, Pfu-Sso7d was used together with M-MLV RT to test their performance in a Two-Step RT-PCR context using a known RNA sample, different commercial reaction buffers, and different reducing reagents (Supplemental Figure 2B). Amplification of a DNA fragment of expected size was obtained in all commercial buffers, with best results when dithiothreitol (DTT) was used as a reducing agent in the reaction mix (Supplemental Figure 2B).). Once the pol and RT activities were tested, the activity of both enzymes (Pfu-Sso7d and M-MLV RT) were confirmed in a One-Step RT-PCR (Supplemental Figure 2C) using a homemade buffer (HM-Buffer) whose composition is described in the Material & Methods section.

With the aim of obtaining a probe-based RT-qPCR, a One-Step RT-PCR using Taq DNA pol instead of Pfu-Sso7d was evaluated (Supplemental Figure 2D) since the latter lacks the necessary 5’-exonuclease activity required for the degradation of qPCR probes. The expected amplification from a known RNA sample was obtained with both Pfu-SSo7d and Taq DNA pol (Supplemental Figures 2C and 2D), allowing us to use them in One-Step reactions based on dyes or probes, respectively.

Dye- and Probe-based One-Step RT-qPCR using synthetic RNA from SARS-CoV-2

To set up and validate our open system for diagnostic tests, we evaluated its sensitivity on in vitro-transcribed N gene RNA samples. One-Step RT-qPCR was performed, using M-MLV RT plus Pfu-SSo7d (M-MLV/Pfu) in dye-based assays and M-MLV RT plus Taq DNA pol (M-MLV/Taq) in probe-based assays.

To perform a dye-based RT-qPCR, EvaGreen DNA-binding agent and specific Center for Disease Control and Prevention(CDC)-recommended primers that specifically amplify the N1 and N2 targets of the SARS-CoV-2 nucleocapsid (N) protein were used (Table 6, Material & Methods section). The reaction mix was prepared for each pair of primers, as shown in Table 1, and all reactions were carried out following the RT-qPCR cycling conditions described in Table 2.

View this table:
Table 1. Dye-based RT-qPCR reaction mix using M-MLV RT and Pfu-Sso7d.
View this table:
Table 2. Dye-based One-Step RT-qPCR cycling conditions.

Because the fluorescent signal generated by intercalating dye molecules is not sequence-specific (i.e. EvaGreen can bind to any double-stranded DNA product), a melting curve analysis was added to corroborate that a specific product was amplified. Representative results of dye-based SARS-CoV-2 RT-qPCR tests using synthetic RNA templates and employing M-MLV RT and Pfu-Sso7d are depicted in Figure 1.

Figure 1:
Figure 1: CDC SARS-CoV-2 N1 and N2 One-Step RT-qPCR assays performed with synthetic RNA using M-MLV RT and Pfu-Sso7d.

(A-B) Representative amplification curves using EvaGreen as DNA intercalating dye. Each curve represents a specific dilution of SARS-CoV-2 synthetic N RNA used as template: 1.4 × 107 copies approximately (red line), 1.4 × 106 (yellow line), 1.4 × 105 (green line), 1.4 × 104 (light blue line), 1.4 × 103 (blue line), 1.4 × 102 (purple line) and no template control (NTC, gray line). Characteristic Cq values are indicated on the upper left side of each panel in order to evaluate if amplification curves correspond to single amplicons (peaks); N1 and N2 melting curve analyses are shown in panels C and D, respectively.

As seen in N1 and N2 amplification curves of Figure 1A and B, respectively, dye-based RT-qPCR assay allowed the detection of amounts of synthetic RNA close to 100 and 10 copies of the SARS-CoV-2 N gene per reaction. However, nonspecific amplification was obtained in non-template controls (NTC), which could be due to primer dimers or cross-contamination with synthetic RNA. In melting curves of reactions with the N1 primer pair (Figure 1C), a single peak is observed between 80°C and 85°C, even in the NTC. Meanwhile, in reactions with the N2 primer pair (Figure 1D), a single peak is observed up to approximately 1000 copies. From 100 copies and below, another product at lower temperature is amplified, which is the dominant product without a synthetic RNA template.

Alternatively, the “gold standard” TaqMan RT-qPCR methods are highly preferred due to their specificity and high sensitivity. Therefore, a probe-based RT-qPCR using the pairs of primers N1 and N2 used previously, but in the presence of CDC-approved double-quenched probes was also performed (N1 and N2 5’ FAM/ ZEN™ / 3’ lowa Black™ FQ propes, Table 6, Material & Methods section).

A probe-based reaction mix was prepared as indicated in Table 3, and the thermal cycling protocol is described in Table 4. Representative results of our in-house One-Step RT-qPCR assay using M-MLV RT and Taq DNA pol are shown in Figure 2.

View this table:
Table 3. Probe-based RT-qPCR reaction mix using M-MLV RT and Taq DNA pol.
View this table:
Table 4. Probe-based One-Step RT-qPCR cycling conditions.
Figure 2:
Figure 2: CDC SARS-CoV-2 N1 and N2 probe-based One-Step RT-qPCR assays performed with synthetic RNA using M-MLV RT and Taq DNA pol.

(A-B) Representative amplification curves using N1 and N2 CDC-approved double-quenched probes. Each curve represents a specific dilution of SARS-CoV-2 synthetic N RNA used as template: 1.4 × 107 copies approximately (red line), 1.4 × 106 (yellow line), 1.4 × 105 (green line), 1.4 × 104 (light blue line), 1.4 × 103 (blue line), 1.4 × 102 (purple line), 1.4 × 101 (light brown line) and no template control (NTC, gray line). Characteristic Cq values are indicated on the upper left side of each panel. N.d.: not determined (no Cq reported).

As seen in Figure 2, probe-based RT-qPCR assay improved the sensitivity, detecting up to 10 copies of synthetic N gene, and no signal was obtained in the NTCs (Figure 2A and B).

Altogether, these results on synthetic SARS-CoV-2 RNA demonstrate that previously purified enzymes can be used in One-Step RT-qPCR either using intercalating dyes, such as EvaGreen, or Taqman probes as those indicated by the CDC. More specifically, our results demonstrate that a probe-based essay seems to be a more suitable option for SARS-CoV-2 synthetic RNA detection, mainly because it could potentially allow better and easier discrimination between samples.

Validation of a Taqman probe-based One-Step RT-qPCR in clinical samples

To validate the results obtained with the synthetic N gene of SARS-CoV-2, clinical samples that had been previously analyzed with commercial RT-qPCR kits were evaluated.

As previously described, CDC-recommended primers and probes that specifically amplify the N1 and N2 targets of the SARS-CoV-2 nucleocapsid (N) protein were used. In addition, the human RNAse P gene target was also amplified as an endogenous internal control.

We analyzed 40 clinical nasopharyngeal samples, of which 20 had been classified as positive and 20 as negative, according to RT-qPCR guidelines recommended by CDC[6]. These known samples were evaluated using two kits, a commercial one (TaqPath ™ 1-Step RT-qPCR Master Mix) and our probe-based RT-qPCR reaction mix, using M-MLV RT and Taq DNA pol. The analysis was performed in parallel to avoid a sensitivity decrease of the homemade kit due to sample degradation compared to the commercial one.

The Cq values obtained for both kits in each of the primers/probe sets (N1, N2, and RNAse P) are shown in Table 5. Our results were consistent with the commercial kit, and all the samples previously reported as negative did not show evidence of signal or amplification (20 out of 20). Similarly, those samples previously reported as positive were also found as positive in the current assay, except for a single sample that was not successfully detected as positive using either the commercial or the homemade kit (19 out of 20). Regardless of whether the target was viral (N1 and N2) or human (RNAse P), no significant differences were observed between the results obtained with the commercial kit and the homemade probe-based One-Step Open RT-qPCR (Figure 3). All samples showed amplification for RNAse P primers/probe mix, either for the commercial kit (mean Cq value of 27.82) or the homemade kit (mean Cq value of 27.99).

View this table:
Table 5. Comparative Cq data between a commercial RT-qPCR kit and an Open RT-qPCR method based on M-MLV RT and Taq DNA pol.
Figure 3:
Figure 3: Probe-based Open One-Step RT-qPCR reaction mix (M-MLV RT and Taq DNA pol) provides comparable results to commercial kits in SARS-CoV-2 clinical samples.

Bars represent the mean + /- standard deviation Cq values for each RT-qPCR target gene N1, N2, and RNase P. Each dot represents one sample. White bars show results obtained with the commercial kit (TaqPath ™ 1-Step RT-qPCR Master Mix); gray bars show results obtained with the open RT-qPCR system for diagnostic method. Pairwise comparisons of mean Cq values for each target gene were made using a two-tailed paired Student’s t-test, with a confidence IeveI of 95%. ‘ns’ meansno statistically significant differences.

Despite the unfavorable results of our dye-based One-Step RT-qPCR reaction mix (M-MLV RT/Pfu) with synthetic RNA samples, due to NTC nonspecific amplification, we decided to test this mix in 33 clinical nasopharyngeal samples to evaluate a correlation with the TaqPath™ Commercial kit. The Cq values obtained in each of the primers/probe sets (N1, N2, and RNAse P) are shown in Supplemental Table 1. Of the total samples analyzed, 24 corresponded to samples previously reported as positive and did not change their report for either of the two kits (homemade and commercial). However, 9 samples, previously reported as negative, changed their report when the dye-based reaction mix was used; 3 were positive, and the rest were undetermined.

On the other hand, the commercial kit correctly reported all the negative samples. Even though our kit could not report the clinical samples correctly, there were no significant differences between the Cq values in the positive samples for all the sets of primers/probes used (Supplemental Figure 3), mainly due to the correlation between the Cq values obtained by both kits. Furthermore, the commercial and homemade kits showed amplification for the RNAse P control, with a mean of 28.2 and 28.04, respectively. In this way, we showed that the mix dye-based reaction mix (M-MLV/Pfu) correlates with a commercial kit currently used in clinical diagnostic laboratories. Therefore, we believe it could be used as a starting point for adjustments and optimizations that improve the differentiation of positive and negative samples.

Overall, these results demonstrate the lack of specificity of the dye-based kit, and suggest a greater potential utilization in clinical samples of the One-Step RT-qPCR using Taq pol. In this sense, our results demonstrate that our probe-based One-Step RT-qPCR reaction mix using M-MLV RT and Taq DNA pol can detect the SARS-CoV-2 RNA virus in clinical samples. Furthermore, this reaction mix also reported Cq values similar to those of the TaqPath reaction mix, allowing effective discrimination between positive and negative samples.

Discussion

In this work, we set up and standardized a probe-based One-Step open system for RT-qPCR detection of SARS-CoV-2. We demonstrate that this reaction mix can be locally produced using standard laboratory equipment, deployed and used in RNA clinical samples obtained from nasopharyngeal swabs, with results comparable to those obtained using commercial kits.

Laboratory diagnosis of SARS-COV-2 is based on One-Step RT-qPCR counseled by sanitary authorities[6,7]. Therefore, increasing testing capacity with RT-qPCR assays is essential to control the pandemic[10]. However, RT-qPCR tests are limited due to the high demand and global supply chain disruptions, which is undermining the COVID-19 health and public policy response[30]. In response to this situation, open-source initiatives have promoted global collaboration, offering more accessible solutions for communities during the pandemic. For instance, 3D printing for personal protective equipment (PPE) and critical care devices have become an essential source during the SARS-CoV-2 outbreak[16,17]. In the context of diagnosis, low-cost devices for NAAT with 3D-printed pieces and accessible electronic parts have been developed in the last years[31-33]. Furthermore, a recent report for an open-source liquid-handling robot for RT-qPCR SARS-CoV-2 diagnosis comes to complement the outreach of low-cost equipment[34]. Nevertheless, these devices are still dependent on commercial kits for molecular diagnostics.

In response to the limited access to reagents, many groups have developed homebrew NAATs[29,35-39] and the Reagents Collaboration Network, ReClone, has been created to host crowd-sourced open access protocols and distribute off-patent enzymes under openMTA[24] and Reclone[40]. Contributing to these collaborative efforts, we have developed open-source master mixes for One-Step RT-qPCR based on homebrew M-MLV RT and either Pfu-Sso7d or Taq DNA pol that could be used in nasopharyngeal swab samples with results comparable to commercial kits. We also aimed to provide protocols to purify these enzymes using equipment and reagents often found in most university-level molecular biology and biochemistry laboratories, thus avoiding the use of costly liquid chromatography instruments and minimizing the use of expensive compounds such as reducing agents and detergents. Cost reduction to facilitate production and local deployment was also considered when setting up these One-Step protocols using non-proprietary buffers.

The dye-based (M-MLV/Pfu) and probe-based (M-MLV/Taq) approaches were tested based on the purified enzymes. Both methods were used on synthetic RNA and clinical samples; however, the best results in our hands were obtained using M-MLV RT and Taq DNA pol, since they allowed a better distinction and correlation between positive and negative clinical samples. Dye-based RT-qPCR assay allowed the detection of synthetic RNA copies of the SARS-CoV-2 N gen; however, the nonspecific amplification obtained in no-template controls (NTC) and low template samples interfered with the results. In qPCR assays based on DNA-binding dyes, the amplification of a nonspecific product frequently occurs and can readily be confused with a positive reaction without proper controls. Therefore, it is recommended to check the identity of the amplified product with melting curves[41]. N1 and N2 melting curve analysis suggests that the results could be due to cross-contamination with synthetic RNA or primer-dimers. For the N2 primer set, we think that the amplification artifacts obtained correspond to primer dimers because these tend to be smaller fragments with peaks of lower melting temperature[41]. Furthermore, a peak similar to synthetic RNA dilution samples is observed in the NTC of both sets of primers, likely due to reaction components contamination or samples cross-contamination with synthetic RNA. Controversially, we use clean working practices to avoid template contamination and separate working areas for PCR mix preparation, template addition, and PCR reactions. This reinforced the idea of creating a probe-based assay, as the dye-based strategy could be harder to implement in laboratories that do not routinely perform these types of assays. Even though the results obtained with M-MLV RT/Pfu-SSo7d in dye-based assays were not optimal compared to the M-MLV RT/Taq DNA pol in probe-based assays, they were consistent with positive samples previously analyzed in clinical laboratories. Therefore, we think it is possible to optimize the components of our dye-based reaction master mix, which could eventually be utilized for SARS-CoV-2 detection in clinical samples. A straightforward strategy could be to use a Uracil-DNA Glycosylase (UDG) or AmpErase Uracil N-Glycosylase (UNG) step before PCR to control carry-over contamination[42,43].

On the other hand, our validation of a Taqman probe-based One-Step RT-qPCR in clinical samples was made using M-MLV RT and Taq DNA pol, since Pfu-SS 7 k 5′-3′ exonuclease activity necessary to degrade Taqman probes. Probe-based RT-qPCR assay was more specific and sensitive, detecting up to 10 copies of synthetic N gene, and no signal was obtained in the NTCs. It has already been described that probe-based real-time PCR presents low background fluorescence and high sensitivity to detect a single transcript copy[44]. Nevertheless, previous studies comparing the analytical performance of primer and probe sets for SARS-CoV-2 showed a high similarity in the sensitivities for SARS-CoV-2 detection, indicating that different assays’ outcomes are comparable[45]. However, in our experience and laboratory conditions, the probe-based test would be a more suitable option for diagnostic tests for SARS-CoV-2 due to better and easier discrimination between positive and negative clinical samples.

Clinical samples were re-tested using commercial and homemade kits to override the samples’ transport and constant freeze/thaw cycles effects. The probe-based One-Step RT-qPCR assay corroborated almost all sample reports compared to those provided by the clinical laboratory. Given this, the sample that changed its report did it in both kits (commercial and homemade), so the most probable explanation is sample degradation due to mishandling. Similar causes have been reported to explain differences or changes in the Cq values of clinical samples[18]. Another reason for re-testing was that the clinical samples’ Cq values had been obtained with primers different from those of N1 and N2 targets.

Finally, it should be considered that this protocol tries to be a first step in elaborating a clinical diagnostic kit. Several factors still need improvement, such as buffer stability over time. This is important to consider because all master mixes were freshly prepared. In addition, modifications to the buffer reaction composition could improve sensitivity and reduce the false positives obtained in the dye-based approach. Nevertheless, the procedure described here is an available alternative to commercial kits to test for SARS-CoV-2.

Open solutions to enzyme production and distribution can mitigate the SARS-CoV2 impact in commercial kit shortages and low resource settings worldwide. In accordance with that, our results support an open One-Step RT-qPCR set up compatible with Taqman probes used in clinical diagnostic laboratories. Furthermore, our results demonstrate high sensitivity and discrimination capacity comparable to commercial kits. Certainly some parameters could still be optimized, but this collaborative effort and the protocols here described represent a valuable resource in COVID-19 pandemia.

Materials & Methods

Biological Samples

For validation of the TaqMan probe-based One-Step RT-qPCR, nasopharyngeal swabs from 40 patients from Metropolitan Region of Chile were obtained from the active SARS-CoV-2 screening program and processed in the Laboratory of Molecular Virology of the Universidad de Santiago of Chile. RNA was extracted using E.Z.N.A. Total RNA kit I (Omega Biotek) according to the manufacturer’s instructions. The preliminary SARS-CoV-2 detection using GenomeCoV19 Detection Kit (Applied Biological Materials Inc.) was performed in Aria Mx Thermocycler (Agilent) with a thermal cycle suggested by Applied Biological Materials (ABM). The interpretation of the data was performed according to the manufacturer’s instructions, considering samples with both amplifications in N and S genes as positive. All methods and procedures were performed following the relevant guidelines and regulations approved by the Ethics Committee of the Universidad de Santiago of Chile. Informed consent was obtained from all participants and/or their legal guardians.

For validation of the dye-based One-Step RT-qPCR, 33 nasopharyngeal swabs samples in Universal Transport Medium (UTM) derived from symptomatic cases of COVID-19 consultants of the UC-Christus Health Network in Chile were used. RNA was automatically extracted using the Mag-Bind RNA Extraction Kit (Maccura Biotechnology CO., LTD) according to the manufacturer’s instructions. Samples were preliminary tested using the RT-qPCR commercial kit LightMix® SARS-CoV-2 RdRp-gene EAV PSR & Ctrl (TIB MOLBIOL) and the reactions were run in a LightCycler® 480 real time-PCR system (Roche). The interpretation of the data was performed by analyzing the 2nd Derivative Maximum Method, obtaining the quantification cycle (Cq) value for each sample. Samples were processed in the Clinical Laboratory of Medicine School. Approval for the use of these biological samples for diagnostic tests was obtained from the Ethical Review Board of the Faculty of Medicine, Pontificia Universidad Católica de Chile (code: 210105007). Informed consent was obtained from all participants and/or their legal guardians.

Enzyme cloning, expression and purification

The Pfu-sso7d construct was obtained from Dr. Alexander Klenov at York University. The Taq DNA Polymerase (pET-28a_6H-TAQ_E602D) was obtained from Dr. Robert Tjian (Addgene plasmid 166944). In parallel, the M-MLV construct was synthesized as a Gblock from IDT, bearing an additional 8 AA N-terminal leader segment. This M-MLV contains point mutations D200N, L603W, T330P, L139P, and E607K shown to increase thermostability and processivity[46]. In addition, NdeI and BamHI were used to clone it in a pET-19 vector with N-terminal 10x His-tag. Finally, sequence constructs were verified by Sanger sequencing services (Eton Bio).

A complete description of our protein expression and purification protocols can be found elsewhere[47-49]. Briefly, the three DNA constructs were independently transformed into E. coli T7 expression strains BL21(DE3), C41(DE3), and C41(DE3), for MMLV, Pfu-Sso7d and Taq DNA pol, respectively. First, single colonies were picked into liquid LB media and allowed to grow overnight at 37 °C with constant agitation at 200-250 rpm. The saturated cultures were then used to inoculate one liter of LB media and grow with constant agitation at 160-200 rpm at 37 °C until OD600 = 0.8. At this point, protein overexpression was induced by adding IPTG to a final concentration of 0.5 mM and incubated by 16 hr at 160 rpm and 18 °C for Pfu-Sso7d and MMLV, while incubation was only 2 hr at 37 °C for Taq DNA pol[47-49].

Cells were then collected by centrifugation and resuspended in a lysis buffer, which contained lysozyme at ∼0.2 mg/mL. Subsequently, cells were incubated at room temperature for 20-30 min with constant agitation of 80 rpm, and the resuspended cells were then disrupted by sonication. For the thermostables Pfu-Sso7d and Taq DNA pol, this step was followed by a heat shock for 30 min at 70-75 °C. Then, the homogenate was centrifuged, and the soluble fraction was collected for further steps[47-49].

Proteins were purified from the clarified lysate by two subsequent purification using affinity chromatography. First, a nickel affinity purification was performed, obtaining the target protein purity of >80%. Then, a heparin affinity chromatography was performed for a second purification step to simultaneously eliminate protein and nucleic impurities, recover the proteins in a smaller volume and perform a buffer exchange onto appropriate storage buffers. The eluted protein fractions were pooled, quantified using the Bradford method, supplemented with 50% glycerol and other appropriate reagents, and stored at -20 °C until further use[47-49]. The enrichment of purified enzymes in the eluted fractions was analyzed by SDS-PAGE.

Assessment of enzymatic activity

DNA polymerase activity was evaluated by amplifying different DNA fragment sizes of 122, 427, and 1067 base pairs from a plant organism (A. thaliana) unrelated to the SARS-CoV-2 virus. For this, 0.3 µg of homemade Pfu-Sso7d were compared to 0.4 U of Phusion High-Fidelity DNA Polymerase (Thermo Scientific), the latter used as a positive control. The 5X Phusion HF Buffer (Thermo Scientific) was used for both enzymes. Subsequently, a Two-Step RT-PCR reaction was carried out to evaluate the reverse transcriptase activity. For this reaction, 3 ng of homemade M-MLV-RT and 0.3 µg of homemade Pfu-Sso7d were used for cDNA synthesis and DNA amplification, respectively. A known sample of plant RNA was used and ImProm-II Retrotranscriptase (Promega) was used as a positive control of the reverse transcriptase activity. Commercial reaction buffers ImProm-II 5X (Promega) and 5X Phusion HF Buffer (Thermo Scientific) were used, and dithiothreitol (DTT) and β-mercaptoethanol (BME) were tested as reducing agents.

Once the polymerase and reverse transcriptase activities were confirmed, the activity of both enzymes (M-MLV RT and Pfu-Sso7d) were evaluated in a One-Step RT-PCR using a homemade buffer composed by 150 mM Tris-HCl pH 8.4, 50 mM KCl, 50 mM NH4OAc, 15 mM MgSO4, 50 mM DTT, 0.5 % Triton X-100 and 0.5 mg/mL BSA. In parallel, a One-Step RT-PCR using 0.4 µg of homemade Taq DNA polymerase instead of Pfu-Sso7d was evaluated in a One-Step RT-PCR using the same conditions described above. In both cases, 1 and 0.5 µg of plant RNA from a known sample were used as a template.

In vitro transcription of N gene RNA from SARS-CoV-2

Synthetic viral RNA was used for preliminary evaluation of M-MLV open reverse transcriptase and Pfu-Sso7d or Taq DNA polymerase. The positive control of the N gene from SARS-CoV-2 was purchased from IDT (Cat # 10006625), from which an overhang PCR was performed to generate an RNAPT7 transcriptional unit of the full-length N gene, using the following primers: NT_Fw (5”-CGAAATTAATACGACTCACTATAGGGGCAACGC GATGACGATGGATAG -3”) and T7_Nter_Rv (5”-ACTGATCAAAAAACCCCTCAAGACCCG TTTAGAGGCCCCAAGGGGTTATGCTAGTTAGGCCTGAGTTGAGTCAG -3”)

The product was purified from agarose gel electrophoresis using a Wizard Sv gel clean-up system (Promega). Then was used in an in-vitro transcription reaction at 37 °C for 16 hours (Hi Scribe, NEB) using 2 μL of each ribonucleotide (100 mM), 2 μL of 10X Reaction Buffer (NEB), 0.5 μL of RNAsin (Promega), 0.5 μL of Pyrophosphatase (NEB), 2 μL of T7 polymerase Mix (NEB) and 7μL of N gene dsDNA linear fragment (40ng/μ). Subsequently, a DNAse I treatment was made by adding 70μL of ultrapure water, 10μL 10X DNAse I Buffer, and 2 μL of DNAse I (NEB). Finally, the sample was incubated at 37°C for 15 minutes to remove DNA.

The RNA product was purified with an RNeasy kit following the provider’s instruction (Qiagen), obtaining 50μL of 1000ng/μL of the viral N RNA that was then serial diluted to the concentrations tested.

RT-qPCR analysis using synthetic RNA

Dye-based Open One-Step RT-qPCR (reaction mix and cycling conditions described in Tables 1 and 2, respectively) and probe-based Open One-Step RT-qPCR (reaction mix and cycling conditions described in Tables 3 and 4, respectively) were evaluated on in vitro-transcribed N gene RNA samples. For dye-based RT-qPCR, EvaGreen DNA-binding agent and specific primers for SARS-CoV-2 nucleocapsid (N) coding gene and the human RNAse P gene were used (Table 6). Probe-based RT-qPCR was performed using double-quenched probes (N1, N2 and RNAse P 5’ FAM / ZEN™ / 3’ lowa Black™ FQ probes, Table 6). For both assays, the homemade RT-qPCR reaction buffer (150 mM Tris-HCl pH 8.4, 50 mM KCl, 50 mM NH4OAc, 15 mM MgSO4, 0.5 % Triton X-100 and 0.5 mg/mL BSA.) and 100 mM DTT were used. Serial dilutions in the range of 1.4 × 107 to 1.4 × 101 copies of synthetic RNA were used as templates. RT-qPCR reaction mixes were prepared just before use and kept on ice until transferred to a StepOnePlus Real-Time PCR System instrument (Applied Biosystems).

View this table:
Table 6. Primers used in One-Step qRT-PCR reactions

RT-qPCR analysis using clinical samples

Clinical samples previously analyzed with commercial RT-qPCR kits were used to validate the results obtained with the synthetic RNA. As previously described, primers and probes for the N1 and N2 targets of the SARS-CoV-2 were used. In addition, the human RNAse P gene target was used as an endogenous internal control (Table 6). A total of 40 clinical samples (20 positive and 20 negatives) were reevaluated using a commercial kit (TaqPath ™ 1-Step RT-qPCR Master Mix) and a probe-based RT-qPCR reaction mix according to the conditions indicated by the supplier and previously described (Tables 3 and 4), respectively. Additionally, 33 samples were retested using a commercial kit (TaqPath™ 1-Step RT-qPCR Master Mix) and a probe-based RT-qPCR reaction mix (Tables 1 and 2). One-Step RT-qPCR reaction mixes were prepared before use and kept on ice until transferred to a StepOnePlus Real-Time PCR System instrument (Applied Biosystems). Cutoff points for Cq values required to decide whether a result is COVID-19 positive or negative were specified by CDC. To report a positive result, both viral targets N1 and N2 must be Cq < 40. To report a negative result, both viral targets must be Cq ≥ 40. If one of the viral targets is Cq < 40 and the other is Cq ≥ 40, the result was reported as undetermined. In addition, the samples, were considered valid only if Cq of the RNase P target was ≤ 35.

Statistical analysis

Mean Cq values obtained through both methods for each target gene were analyzed in pairwise comparisons using a paired Student’s t-test. The analysis was performed using GraphPad Prism 8 software.

Data Availability

All data produced in the present work are contained in the manuscript.

Author Contributions

AC, CI, RG, CARS and FF designed research; MR, JR, PB, JA and AS performed research; AC, CI, MR, GA, CARS and FF analyzed data; AC, GA, MR, FC, RG, CARS and FF wrote the manuscript. All authors reviewed the manuscript.

Additional Information

The authors declare no competing interests.

Funding

This work was supported by the National Agency for Research and Development (ANID) through the ANID Millennium Science Initiative Program (ICN17_022), Fondo de Desarrollo de Areas Prioritarias (Center for Genome Regulation; ANID/FONDAP/15090007), Fondo de Desarrollo Científico y Tecnológico (FONDECYT 1201684 awarded to C.A. Ramírez-Sarmiento, FONDECYT Regular 1211218 to Fernan Federici, and FONDECYT 3190731 to Maira Rivera), and an International Cooperation Program with Consejo Nacional de Ciencia, Tecnología e Innovación Tecnológica (ANID-CONCYTEC covbio0012 awarded to F. Federici and C.A. Ramírez-Sarmiento). This work was also supported by the National Institutes of Health NIAID training grant (Training Program in Immunology; T32-AI07405) awarded to A. J. Brown. P. Blázquez-Sánchez and J. Reyes were supported by ANID Doctoral Scholarships (21191979 and 21191684, respectively).

Acknowledgments

Thanks to Gina Dailey, Thomas Graham, Robert Tjian and Xavier Darzacq (University of California Berkeley) for sharing the Taq DNA pol plasmid. We also acknowledge the gracious help of Amparo Nuñez during the protein purification processes in the final stages of this work. We thank members of the gLAMP consortium, ReClone forum (https://reclone.org/), Open Bioeconomy Lab (https://openbioeconomy.org/) and the JOGL OpenCOVID community (https://app.jogl.io/program/opencovid19) for the advice and for sharing relevant information for the establishment of diagnostic reactions for SARS-CoV-2. Finally, we also thank to Laura Delgado and Alejandro Fonseca for helping with the manuscript.

References

  1. 1.↵
    Mohamadian, M., et al., COVID-19: Virology, biology and novel laboratory diagnosis. J Gene Med, 2021. 23(2): p. e3303.
    OpenUrlPubMed
  2. 2.↵
    Dharavath, B., et al., A one-step, one-tube real-time RT-PCR based assay with an automated analysis for detection of SARS-CoV-2. Heliyon, 2020. 6(7): p. e04405.
    OpenUrl
  3. 3.↵
    Corman, V.M., et al., Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Euro Surveill, 2020. 25(3).
  4. 4.↵
    Caruana, G., et al., Diagnostic strategies for SARS-CoV-2 infection and interpretation of microbiological results. Clin Microbiol Infect, 2020. 26(9): p. 1178–1182.
    OpenUrl
  5. 5.↵
    Gupta, E., et al., Pooled RNA sample reverse transcriptase real time PCR assay for SARS CoV-2 infection: A reliable, faster and economical method. PLoS One, 2020. 15(7): p. e0236859.
    OpenUrl
  6. 6.↵
    CDC. CDC 2019-Novel Coronavirus (2019-nCoV) Real-Time RT-PCR Diagnostic Panel. 2020 [cited 2021; Available from: https://www.fda.gov/media/134922/download.
  7. 7.↵
    Lu, X., et al., US CDC Real-Time Reverse Transcription PCR Panel for Detection of Severe Acute Respiratory Syndrome Coronavirus 2.. Emerging Infectious Diseases, 2020. 26(8): p. 1654–1665.
    OpenUrl
  8. 8.↵
    Lippi, G., A.M. Simundic, and M. Plebani, Potential preanalytical and analytical vulnerabilities in the laboratory diagnosis of coronavirus disease 2019 (COVID-19). Clinical Chemistry and Laboratory Medicine (CCLM), 2020. 58(7): p. 1070–1076.
    OpenUrl
  9. 9.↵
    Avaniss-Aghajani, E., et al., Validation of the Hologic Aptima Unisex and Multitest Specimen Collection Kits Used for Endocervical and Male Urethral Swab Specimens (Aptima Swabs) for Collection of Samples from SARS-CoV-2-Infected Patients. J Clin Microbiol, 2020. 58(8).
  10. 10.↵
  11. 11.↵
    Guglielmi, G., The explosion of new coronavirus tests that could help to end the pandemic. Nature, 2020. 583(7817): p. 506–509.
    OpenUrlCrossRefPubMed
  12. 12.↵
    Fisher, J., et al., Community, work, and family in times of COVID-19. Community, Work & Family, 2020. 23(3): p. 247–252.
    OpenUrl
  13. 13.
    Oldekop, J.A., et al., COVID-19 and the case for global development. World Development, 2020. 134: p. 105044.
    OpenUrl
  14. 14.↵
    Shamasunder, S., et al., COVID-19 reveals weak health systems by design: Why we must re-make global health in this historic moment. Global Public Health, 2020. 15(7): p. 1083–1089.
    OpenUrl
  15. 15.↵
    Tietze, F., et al., Crisis-Critical Intellectual Property: Findings From the COVID-19 Pandemic. IEEE Transactions on Engineering Management, 2020: p. 1–18.
  16. 16.↵
    Frazer, J.S., A. Shard, and J. Herdman, Involvement of the open-source community in combating the worldwide COVID-19 pandemic: a review. J Med Eng Technol, 2020. 44(4): p. 169–176.
    OpenUrl
  17. 17.↵
    Maia Chagas, A., et al., Leveraging open hardware to alleviate the burden of COVID-19 on global health systems. PLOS Biology, 2020. 18(4): p. e3000730.
    OpenUrlPubMed
  18. 18.↵
    Graham, T.G.W., et al., Open-source RNA extraction and RT-qPCR methods for SARS-CoV-2 detection. PloS one, 2021. 16(2): p. e0246647–e0246647.
    OpenUrl
  19. 19.↵
    Alcántara, R., et al., Unlocking SARS-CoV-2 detection in low-and middle-income countries. Cell Reports Methods, 2021. 1(7).
  20. 20.↵
    Mascuch, S.J., et al., A blueprint for academic laboratories to produce SARS-CoV-2 quantitative RT-PCR test kits. J Biol Chem, 2020. 295(46): p. 15438–15453.
    OpenUrlAbstract/FREE Full Text
  21. 21.↵
    Vonesch, S.C., et al., McQ – An open-source multiplexed SARS-CoV-2 quantification platform. medRxiv, 2020: p. 2020.12.02.20242628.
  22. 22.↵
    Bhadra, S., et al., A One-enzyme RT-qPCR Assay for SARS-CoV-2, and Procedures for Reagent Production. Bio-protocol, 2021. 11(2): p. e3898.
    OpenUrl
  23. 23.↵
    Bhadra, S., et al., One-Enzyme Reverse Transcription qPCR Using Taq DNA Polymerase. Biochemistry, 2020. 59(49): p. 4638–4645.
    OpenUrl
  24. 24.↵
    Kahl, L., et al., Opening options for material transfer. Nature Biotechnology, 2018. 36(10): p. 923–927.
    OpenUrl
  25. 25.↵
    Martinez, R.M., Clinical Samples for SARS-CoV-2 Detection: Review of the Early Literature. Clinical Microbiology Newsletter, 2020. 42(15): p. 121–127.
    OpenUrl
  26. 26.↵
    Murphy, J. and S.A. Bustin, Reliability of real-time reverse-transcription PCR in clinical diagnostics: gold standard or substandard? Expert Review of Molecular Diagnostics, 2009. 9(2): p. 187–197.
    OpenUrlCrossRefPubMed
  27. 27.↵
    Tahamtan, A. and A. Ardebili, Real-time RT-PCR in COVID-19 detection: issues affecting the results. Expert Rev Mol Diagn, 2020. 20(5): p. 453–454.
    OpenUrlCrossRefPubMed
  28. 28.↵
    Wang, Y., et al., A novel strategy to engineer DNA polymerases for enhanced processivity and improved performance in vitro. Nucleic Acids Res, 2004. 32(3): p. 1197–207.
    OpenUrlCrossRefPubMedWeb of Science
  29. 29.↵
    Matute, T., et al., Homebrew reagents for low cost RT-LAMP. medRxiv, 2021.
  30. 30.↵
    Nkengasong, J., Let Africa into the market for COVID-19 diagnostics. Nature, 2020. 580(7805): p. 565.
    OpenUrlCrossRefPubMed
  31. 31.↵
    GaudiLabs. PocketPCR. 2021; Available from: http://gaudi.ch/PocketPCR/.
  32. 32.
    Miroculus. Miriam for isothermal amplification reactions. 2021 [cited 2021; Available from: https://miroculus.com/miriam/.
  33. 33.↵
    Mulberry, G., et al., 3D printing and milling a real-time PCR device for infectious disease diagnostics. PLoS One, 2017. 12(6): p. e0179133.
    OpenUrl
  34. 34.↵
    Villanueva-Cañas, J.L., et al., Implementation of an open-source robotic platform for SARS-CoV-2 testing by real-time RT-PCR. PloS one, 2021. 16(7): p. e0252509–e0252509.
    OpenUrl
  35. 35.↵
    Alekseenko, A., et al., Direct detection of SARS-CoV-2 using non-commercial RT-LAMP reagents on heat-inactivated samples. Scientific reports, 2021. 11(1): p. 1820–1820.
    OpenUrl
  36. 36.
    Bhadra, S., et al., High-surety isothermal amplification and detection of SARS-CoV-2, including with crude enzymes. bioRxiv, 2020: p. 2020.04.13.039941.
  37. 37.
    Graham, T.G.W., et al., Inexpensive, versatile and open-source methods for SARS-CoV-2 detection. medRxiv, 2020: p. 2020.09.16.20193466.
  38. 38.
    Kellner, M.J., et al., A rapid, highly sensitive and open-access SARS-CoV-2 detection assay for laboratory and home testing. bioRxiv, 2020: p. 2020.06.23.166397.
  39. 39.↵
    Sherrill-Mix, S., et al., Detection of SARS-CoV-2 RNA using RT-LAMP and molecular beacons. Genome Biology, 2021. 22(1): p. 169.
    OpenUrl
  40. 40.↵
    http://Reclone.org. Reagent Collaboration Network. [cited 2021; Available from: https://reclone.org/.
  41. 41.↵
    Ruiz-Villalba, A., et al., Amplification of nonspecific products in quantitative polymerase chain reactions (qPCR). Biomolecular detection and quantification, 2017. 14: p. 7–18.
    OpenUrl
  42. 42.↵
    Burkardt, H.-J., Standardization and Quality Control of PCR Analyses. 2000. 38(2): p. 87–91.
    OpenUrl
  43. 43.↵
    Longo, M.C., M.S. Berninger, and J.L. Hartley, Use of uracil DNA glycosylase to control carry-over contamination in polymerase chain reactions. Gene, 1990. 93(1): p. 125–128.
    OpenUrlCrossRefPubMedWeb of Science
  44. 44.↵
    Wong, W., et al., Probe-based Real-time PCR Approaches for Quantitative Measurement of microRNAs. Journal of visualized experiments : JoVE, 2015(98): p. 52586.
  45. 45.↵
    Vogels, C.B.F., et al., Analytical sensitivity and efficiency comparisons of SARS-CoV-2 RT–qPCR primer–probe sets. Nature Microbiology, 2020. 5(10): p. 1299–1305.
    OpenUrl
  46. 46.↵
    Baranauskas, A., et al., Generation and characterization of new highly thermostable and processive M-MuLV reverse transcriptase variants. Protein Engineering, Design and Selection, 2012. 25(10): p. 657–668.
    OpenUrlCrossRefPubMedWeb of Science
  47. 47.↵
    Rivera, M.R., et al. Recombinant protein expression and purification of codon-optimized Pfu-Sso7d. 2020; Available from: https://protocols.io/view/recombinant-protein-expression-and-purification-of-bhcrj2v6
  48. 48.
    Rivera, M.R., et al. Recombinant expression and purification of codon-optimized M-MLV and Mashup. 2021 [cited 2021; Available from: https://dx.doi.org/10.17504/protocols.io.bsernbd6.
  49. 49.↵
    Rivera, M.R., et al. Recombinant protein expression and purification of Taq DNA polymerase. 2021; Available from: dx.doi.org/10.17504/protocols.io.bya3psgn.
    OpenUrlCrossRef
  50. 50.
    CDC. Research Use Only 2019-Novel Coronavirus (2019-nCoV) Real-time RT-PCR Primers and Probes. 2020 [cited 2021 March 2021]; Available from: https://www.cdc.gov/coronavirus/2019-ncov/lab/rt-pcr-panel-primer-probes.html.
Back to top
Posted December 07, 2021.
Download PDF

Supplementary Material

Data/Code
Email
A One-Step open RT-qPCR for SARS-CoV-2 detection
Ariel Cerda, Maira Rivera, Grace Armijo, Catalina Ibarra-Henriquez, Javiera Reyes, Paula Blázquez-Sánchez, Javiera Avilés, Aníbal Arce, Aldo Seguel, Alexander J. Brown, Yesseny Vásquez, Marcelo Cortez-San Martín, Francisco Cubillos, Patricia García, Marcela Ferres, César A. Ramírez-Sarmiento, Fernán Federici, Rodrigo A. Gutiérrez
medRxiv 2021.11.29.21267000; doi: https://doi.org/10.1101/2021.11.29.21267000
Reddit logo Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools

Subject Area