Principles and test methods of non-contact body thermometry

1.1 Foundational Literature - Physiology

We turn now to the physiology underlying the relationship between human skin and core temperatures, because this is essential to obtaining an accurate core body temperature with non-contact methods in real-world scenarios. The physiologic offset is the reduction in temperature between core body temperature and a well-chosen patch of skin. For the inner canthus, which has minimal thickness and vasomotion of tissue over the underlying primary artery, the slope of the relationship is driven by the thermal resistivity and the environmental conditions. To first order approximation, the total heat flux from core blood in the angular artery through this area of skin to the environment is set by the thermal resistivity of the skin, the conductivity and convection of mostly still air and the radiation balance to the surrounding environment. Extensive modeling of heat flow from core to skin has evolved since the original Pennes bio-heat equation{22,23,24}, but in the human inner canthi, the effect is approximated well by a linear or low-order polynomial relationship, which has been documented in the literature over typical ambient air temperatures between 30F and 100F{19}. See for example Figures 10 and 11 in this{21} 2003 US patent. (For context, further below we reference Section 2.1, which introduces Equation 1 to describe the effect of ambient temperature on the skin-to-core physiologic offset and Equation 2 to infer a core temperature from surface and ambient temperatures.)

Understanding the relationship between skin and core temperature is essential to monitoring core temperature, in particular for monitoring hypothermia during anesthesiology, which has been in active study for over 20 years. Ikeda et al{19} examined the impact of ambient air temperature on the offset between forehead skin and core temperature. They found a 20% dependence on ambient temperature, stating “Transferring a patient from a typical 20C operating room to a typical 25C postanesthesia care area, however, would reduce the core-to-forehead difference approximately 1C.”{19}.

The same research group, led by Dr Daniel Sessler, later performed a series of challenging and carefully run clinical studies of the monitoring and regulation of temperature, and made a series of valuable observations that are relevant and we collect them here. First, the essential performance required is ±0.5C for anesthesia purposes, because a 1C change has distinct physiologic consequences{25}. In further studies, the Sessler group examined several external thermometry methods (e.g. non-contact as well as contact) and compared accuracy against direct thermometry at various internal sites, such as the tympanic membrane{26,27,28}. It is important to point out that these anesthesiology studies that mention “tympanic” measurements use true tympanic measurements, which are highly challenging to perform as they involve directly contacting the tympanic membrane with a thermocouple and verifying actual contact, while studies outside of anesthesiology will refer to so-called “tympanic” measurements that are actually performed with aural NCIT thermometers that typically cannot see the tympanic membrane. In the field of thermography, cavity radiation is a useful tool to obtain accurate measurements, so the argument that an aural NCIT placed into a somewhat tortuous cavity enclosed at one end by the tympanic membrane could still accurately track true tympanic temperature is attractive (but see Daanen{29} for problems with aural geometry). However, these in-ear NCITs (as well as the temporal artery NCITs) have been shown by the same group to be significantly different from true tympanic temperatures, and unreliable for anesthesia monitoring purposes: “As normally used, that is directed into the aural canal or near the temporal artery, infrared systems are insufficiently accurate for clinical use … In light of their poor performance, it seems unfortunate that they have become so popular.”{25}. We mention this distinction because it is a real, yet widely unrecognized, difference between true tympanic measurement and the vastly more common aural NCIT so-called tympanic measurement. The widespread use of different non-core body sites as the “gold standard” is problematic when the terminology is inconsistent because it can lead to biased results and makes interpretation more difficult.

One such Sessler group study examined aural NCIT, oral thermometry, “deep forehead”, and a temporal artery (TA) NCIT against bladder core body thermometry{26}. The Bland-Altman plots (figure 3 from that study{26} shows TA minus bladder vs bladder) show clear bias-to-normal behavior for the TA NCIT, in particular having a negative slope, meaning a given range of actual core temperatures may be reported as lying within a much smaller range. All NCIT methods were insufficient, particularly the temporal artery NCIT, followed by aural NCIT then deep forehead, with only oral thermometry being acceptable for clinical purposes. Their focus on accurately monitoring core body temperature is driven by more than scientific curiosity: “Larach et al. estimated that the risk of dying after a malignant hyperthermia crises [sic] was an order of magnitude higher in unmonitored patients, presumably because temperature monitoring facilitated speedy diagnosis.”{30, 31}. Thus, at the risk of stating the obvious, there is a clear clinical mandate for maintaining accuracy to within ±0.5C. If NCITs are inaccurately estimating core body temperatures, it could have vital implications for quality in clinical care, because many practices rely on some form of NCIT for quick and easy body temperature measurement during vital signs assessment.

• Download figure
• Open in new tab

Fig. 3

a) Inner canthi skin surface temperatures at three ambient temperatures in a pilot study of 32 individuals. The spread in skin temperatures was confounded by improperly recorded air temperatures and an approximately 1C SSE confound that was not recognized at the time of the pilot study, while the 45F data collection was confounded by an equilibration time of under 2 minutes and therefore ignored in the fit. b) Data recorded in the present study showing inner canthi skin surface temperature in black squares, oral thermometry in blue and corrected body thermometry in red versus effective environmental temperature.

Several other groups have studied NCIT accuracy with similar findings of poor accuracy and bias-to-normal behavior{32,33,34}. In one particular study, Low et al{32} investigated the same TA NCIT used in the Langham study with a wireless ingestible core body thermometer and active manipulation of core body temperature and found similar behavior, specifically that the Exergen TA NCIT measurements were decoupled from actual core body temperature increases. Intriguingly, the authors examined US Patent 6292685{20} covering this device and reproduced the central equation that bears the same general form as found in other patent literature{21} and in a pilot study{35} we describe in Section 2.1. This equation has a linear dependence on ambient temperature, with the slope based on perfusion and specific heat capacity of blood.

The bottom line is that groups with expertise in physiology have shown that the ambient dependence of the physiologic offset is large enough that it must be accounted for to obtain a sufficiently accurate surface-to-core correction, which fortunately is easy to do with an appropriately careful air temperature measurement.

1.2 Foundational Literature - Non-contact Fever Screening using IRT systems

We turn now to the historical development of non-contact thermographic body temperature screening systems and the relevant international standard{7}. Studies of IRT for febrile screening lie in three groups: 1) post-hoc febrile discrimination (poor generalizability), 2) prospective discrimination and secondary screening of IRT-positive subjects (modest generalizability), and 3) prospective discrimination with secondary screening of all subjects (best generalizability). Post-hoc febrile discrimination refers to the practice of acquiring skin temperatures, deriving a correction or threshold from this same data that is then applied to this same data before deriving sensitivity and specificity (we will refer to this type of methodological error as a “post-hoc threshold”). This practice means the threshold may not apply to data collected by other groups, to data collected with different equipment (even of the same model) or even to the same group using the same equipment but on different days. Due to the dependence of IRT core body measurement on ambient temperature, SSE, DE, and other confounds, any post-hoc analysis like this is unlikely to generalize to later work, unless these confounds are all accounted for both in the threshold-setting study and all subsequent operations. Given two important confounds (SSE and DE) that can sum to several degrees Celsius were not published as relevant confounds for body thermometry until 2021, as well as the dependence of of the skin-to-core offset on ambient temperature, it is unclear how to interpret past studies relying on post-hoc threshold-determination{36,37,38}. While there is a considerable body of work addressing the SSE, it has apparently escaped notice by the developers, and most users, of the body thermometry standards.

A smaller list of studies did not rely on a post-hoc threshold. An 11-month study from 2005 of Dengue screening with IRT at Taiwan’s two main airports acquired over 8M passengers, with 22,000 identified as febrile by IRT{40} (IRT methods were not described). Of these, 3,011 were confirmed by aural NCIT and by serological analysis, 40 were identified as carrying Dengue, with 33 being actively viremic. The authors of that study used the statistics of the nationwide Dengue screening program to infer that these two screening stations captured the majority of Dengue infections traveling into Taiwan.

A particularly influential study was that of Ring et al.{41}, performed in a pediatric hospital in Poland in 2008 with 191 subjects between the ages of 1 and 18 yrs, with axilla contact thermometry and aural NCIT performed in parallel with IRT. Because this study was performed with deep knowledge of the ISO and IEC writing group’s early drafts defining the minimum standards needed for the use of a dedicated screening thermograph, it helped define{42} the underlying science and method in the final febrile screening standard{7}, using the data collected in this study and the impact of proper positioning of the head shown by the Glamorgan Protocol{43}.

The study of Nguyen et al. 2008{44} collected IRT, oral thermometry and self-reported fever in patients arriving at the emergency departments of hospitals in three major US cities. Out of 2873 patients, 476 self-reported fever (16.6%) but only 64 had oral-confirmed fever (2.2%). Thus, self-report, even in an emergency department, was found to be a poor predictor of febrility (sensitivity of 75% but PPV of only 10.1%). IRT performance varied across equipment, with only two of the three IRTs producing better sensitivity than self-report. OptoTherm and Device #5 were 91 and 90% sensitive and 86.0 and 80.0% specific, respectively, with r-values of 0.43 (Optotherm), 0.42 (Device #5), but only 0.14 for the third (Wahl).

In Zhou et al.{45}, the authors obtained IRT and 3-minute oral thermometry in 596 subjects, 47 of whom were febrile (>37.5C via oral) and generated 17 facial locations using hand-drawn regions-of-interest (ROI) for each subject. Receiver operating characteristic (ROC) analysis for each ROI provided the best sensitivities in three regions: 0.95 for inner canthi, 0.97 for full face and 0.865 for full forehead. The correlations for both inner canthi and full face ROIs were close to 0.8. Zhou et al. also made the point that low-grade fever would be the most likely target for screening, since mild and moderate febrile subjects are less likely to modify behaviors.

We are aware of only one prospective threshold study to have acquired viral status of both IRT-positive and IRT-negative subjects{46}, relying on an outbreak of Swine flu aboard a United States Navy vessel in the Pacific Ocean in 2011 by a co-author on the present paper, Dr Hinnerichs. In Hinnerichs et al., the threshold was set daily with a core-to-skin calibration on 10 afebrile individuals at the start of each day and the calibrated IRT and oral thermometry were collected in 320 subjects, 84 of whom were later confirmed positive for swine flu by laboratory PCR. The IRT operator was blinded as to the febrility status of each individual (and PCR results were not provided until after the IRT collection was complete). The environment was maintained between 71 and 73F during the data collection. The study achieved r-values of 0.90 and 0.87 (male and female respectively) and sensitivity of 84.5% with specificity of 97.5%. ROC analysis showed the ability to discriminate febrile from nonfebrile 91% of the time. The study further tested for any gender difference in sensitivity and found none, albeit in a relatively homogeneous population consisting predominantly of active service members.

Finally, due to the COVID-19 pandemic, the febrile screening market saw an explosion of supply of “automated” IRT systems from new vendors, many of whom were marketing low-resolution thermopile IRT systems that did not have an IR calibration source to obtain the required accuracy. A recent study (involving a co-author of this study, Dr Hinnerichs) examined the performance of seven recently-introduced automated IRT products and found all exhibited substantial bias-to-normal behavior, meaning their outputs “systematically biased to the mean human temperature”{47}. This publication was, to our knowledge, the first to identify and explicitly state the presence of a systematic bias that could produce harm to public health, and raised the point that no known physics or physiologic model could justify the use of a biasing algorithm with the normalization ranges as wide as those observed. The authors noted one limitation of the study was their inability to incorporate human subjects, due to the unavailability and safety of working with febrile (infectious) individuals. The authors pointed out their study should translate directly to real human faces, but as the work on SSE has shown{16}, it is likely the systems investigated would perform even worse on real human subject faces.

1.3 Medical Device Standards for Non-Contact Thermometry

The ISO standard for medical thermometry, IEC 80601-2-56:2017{8} (hereafter referred to as the “thermometry standard”), describes devices that fall into two primary categories: direct and adjusted. Direct relies on the sensor reaching equilibrium with the body site (e.g. oral, rectal, axillary) while adjusted takes one or more sensor readings and adjusts to a surrogate for a body site. Direct mode devices may be certified via laboratory testing, while adjusted mode devices must also undergo clinical testing for accuracy in human subjects against a direct-mode reference thermometer at a specified reference site (e.g. oral). NCIT and IRT both belong to adjusted-mode thermometry, although at time of writing, no IRT device has undergone testing to the thermometry standard. Note, we are aware of one such device that might be considered an IRT (relying on a multi-pixel sensor rather than a single-pixel) used for NCIT, the WelloStationX, but it is unclear whether this is the case given the sparse documentation available at this time. Rather, there is a separate standard describing the performance requirements for IRT devices intended solely for febrile screening, IEC 80601-2-59:2017{7} (hereafter referred to as the “screening standard”), which does not require clinical testing. The screening standard describes laboratory measurements for an IRT system comprised of either an IR camera with sufficient absolute accuracy or an IR camera paired with an IR calibration source (blackbody) to achieve a total system uncertainty of ±0.5C. Recent publications have pointed out shortcomings in the standard and proposed enhancements or modifications{16,17,18}. As discussed earlier in the introduction and results from our pilot study{36}, the lack of an ambient-dependent adjustment is a major confound for both IRT and NCIT devices. Further, the lack of a specification for the pixel independence or SSE{16} is a major problem for the reliability of IRT and NCIT devices.

We could end this section by drawing attention to the use of clinical testing for any adjusted-mode thermometry and urge the same clinical testing be required for the screening standard as well. Clinical testing should be used because it can provide validation of device performance while being operated as intended. However, devices that appear to exhibit severe bias-to-normal behavior have been cleared for marketing by the FDA as clinical electronic thermometers (product code FLL). The results in Section 2 indicate the reporting to the FDA of clinical tests performed on some devices may have been inadequate. Given the expertise and care required to perform clinical testing is beyond the reach of many manufacturers, it may be helpful to require clinical testing be performed by independent entities, but the testing currently specified is too challenging to be applied by independent test laboratories due to the reliance on convenience samples of febrility that may pose too much risk to be used outside of clinical environments. New test methods that avoid the need for febrile subjects could ameliorate these challenges.

2.1 Dependence of the Skin Surface Temperature on Core and Ambient Temperatures

The dependence of inner canthi skin temperature on core and ambient temperatures was briefly described in Sec. 1.1 with references to anesthesiology and physiology literature and US patent literature. The approximately linear form of this dependence has been confirmed by one of the authors in a pilot study incorporating 32 volunteers performed in June 2020{36}, where each individual’s facial maxima was recorded while the subject had been equilibrated at three stable ambient temperatures ranging from 45F to 85F (the 45F data was excluded due to incomplete equilibration) and this data compared with oral thermometry. This limited pilot study found a 24% dependence of the offset between skin and core (oral) temperature on ambient temperature, as shown in Fig. 3a.

A static analysis of the heat flow across skin (described in Sec. 2.3) results in conversions between inner canthi skin and core (oral as the surrogate) shown in the following equations: where T_core is the core (e.g. oral) body temperature, T_surface is the selected skin surface temperature, T_ambient is the environmental temperature, and F is the dependence of the physiologic offset on ambient temperature. The physiologic factor F is dependent on skin site and is expected to have some level of variability from person-to-person. For a particular IRT or NCIT, F may also incorporate some level of confound from SSE and DE and other equipment-specific characteristics, so it must be pointed out this factor is likely device-dependent but we expect it to be a fixed factor for a given device model over a reasonable ambient temperature range. In the pilot study described above, the slope of the fit in Fig. 3a resulted in F = 0.24. This has been revised to F = 0.1595 using the datasets collected in this study and described in Sec 3.1, all of which were collected after the implementation of a SSE correction{10} as shown in Fig. 3b, which shows inner canthi surface measurements, oral thermometry and corrected body temperatures, with the body correction derived from the linear fit of the offset between oral thermometry and surface temperature to the environmental temperature. Note the correction does not include the intercept from the linear fit so the resulting corrected body estimate is shifted slightly higher than the oral thermometry values from which the fit was derived. This factor is specific to the inner canthus, and other body sites have different factors, which is driven by the amount of insulation between fully perfused tissue and the environment. For example, an aural NCIT inspected in the next section (Fig. 4) appears to use a lower factor of F = 0.096 (the fitted slope minus 1), while the forehead site has slightly higher insulation and thus would use a higher value for F than the inner canthus (note, the forehead F-value was not measured here).

• Download figure
• Open in new tab

Fig. 4

Surface-to-Core transfer curves for 5 NCIT devices denoted in Table 1.

2.2 Observed Bias-to-Normal Algorithms

Some NCIT devices can be alternated between surface temperature and body temperature modes. This makes it straightforward to extract the surface-to-core transfer curve by directing a device at a calibration source and alternating between modes, while sweeping the calibration source through the expected surface temperature range in suitable steps. For those devices that do not have a surface temperature mode, a limited form of this process can be performed with body mode only in order to inspect the linearity, if not the exact numeric offset, of the surface-to-core curve. These can also be performed for some IRT devices, but not those that require the presence of a face in order to function. For this reason, we only investigated these transfer curves in NCITs. Once sufficient data is collected, an approximation of the internal algorithm can be reverse-engineered by piecewise fitting to arrive at a suitably close approximation.

The four forehead NCITs and one aural NCIT in Table 1 were examined for surface-to-core transfer functions (Note NCIT #3 is also represented in Table 2 as Device #3):. The first four devices exhibited strong bias-to-normal behavior, where the highest body temperature that would be biased into the normal range (outputting a body temperature below the device’s febrile threshold) is given in the last column of Table 1. The first three NCITs did not produce a temperature below mild hypothermia (95F, or 35C) for the lowest simulated body temperature used which was 92.34F. The aural NCIT and Device #5 and Device #1 (see Table 2) IRTs did not exhibit any bias-to-normal behavior. The Device #2 and Device #4 IRTs were not assessed in this manner. The aural NCIT (NCIT #5) uses a smaller physiologic offset (the factor in Eqn. 1 would be 5% for aural canal, as compared to the 20-25% used in Device #1 for inner canthi) due to the aural site being closer to core body temperature than the forehead which also has the consequence of utilizing a narrower range of output values. None of the NCIT devices had any apparent dependence of the physiologic offset on ambient temperature, while Device #1 is known to use Equation 2 shown as the 71.6F Surface-to-Core line and which shifts the transfer curve up and down depending on the ambient temperature.

An immediate need is a clinical test method capable of generating arbitrary realistic febrile faces, without requiring the risk or restrictions associated with using actual febrile subjects. The motivation for improving testing is to improve how the devices perform in the population they are most likely to be used within. The most severe fevers will either be reduced by antipyretics below the severe range or will remain out of the general population because they are aware they have a fever. Additionally, febrile screening that only performs at 103F and above while biasing temperatures between 100 and 103F to normal is actively harmful. Therefore, the clinical testing requirement must attempt to specify a more uniform range of febrile subjects (e.g. weighted more equally between mild, moderate and severe). The central challenges are that the availability of a useful range of febrile subjects cannot be guaranteed, and proximity to febrile subjects is associated with unacceptable risk.

2.3 A Test Method to Simulate Feverish Individuals

As discussed above, non-contact medical thermometry is based on a linkage between core temperature and skin temperature. When core temperature rises, skin temperature rises, and this offset between skin and core can be found using physiologic models of the flow of heat to the environment via the skin. To good approximation, the same result can be found with simple static thermal analysis (Fig. 5), by approximating the tissue above a near-surface primary artery as a passive thermal insulator. The flux of heat flowing across this insulator is dependent on the environmental insulation and temperature. Assuming fairly still air and a radiative temperature close to the air temperature, the flux is proportional to the difference in temperature divided by the total thermal resistance. This leads to Equations 1 and 2 in Section 1.1, which is calibrated for human inner canthi using one thermal camera system. It should be pointed out that the ambient dependence factor may differ due to subtle dependencies of the non-contact system used, including wavelength dependencies and image sensor spot size on target.

• Download figure
• Open in new tab

Fig. 5

Static thermal analysis of inner canthus. Note the lower inset which is contrasted with that from Fig. 2 above it, elevated air temperature causes elevated skin temperature in a manner that is measurably equivalent to elevated core temperature. This is a well-known confound that is taken advantage of in this study to devise a test of device performance without requiring febrile subjects.

This can be extended to the dynamic evolution of skin temperatures as the environmental temperature is changed. Fig. 6 shows Device #1 operated in body mode on a subject equilibrated to one room temperature (first column, 73F air temperature, 93.4F skin temperature and 98.2F calculated body temperature) and then moving to a second, elevated air temperature (86.6 to 87.3F air temperature, columns 2 through 4, showing time after first image). Initially, the skin temperature matches the previous skin temperature but rises rapidly, then slows and stabilizes. The skin temperature rises by over 3F during equilibration to stabilize at a value that produces a realistic calculated core body temperature, relative to that in the first panel, while the body temperature is underestimated when measured before equilibration has nearly finished.

• Download figure
• Open in new tab

Fig. 6

Exposure to different ambient (A) temperatures adjusts the surface (S) temperature, but may not affect core (C) temperatures. Example of an equilibrated subject moving from one environment (73F) to a warmer environment (86.6F) and the gradual evolution of the surface temperature. Calculation of core temperature based on Eq 1 (using F=0.234 from the pilot study) is correct only when the subject has equilibrated to the environment, as shown in the first and last panels only, with errors shown in the intervening panels.

This relationship (Eq. 1) combined with the underlying principle of non-contact thermometry suggests a method for generating test data in human subjects that would correspond to a nearly arbitrary range of actual core temperatures, while relying only on subjects with a normal range of core temperatures. In this method, subjects’ skin is allowed to equilibrate to a first environment with altered air temperature selected using Equations 1 and 2, and the simulated elevated body temperatures determined from the oral thermometry, Equations 1 and 2, and the elevated air temperature (or, if surface temperature is measured, this can be calculated using Equation 1). Next, the test equipment is operated from within a second controlled environment having a view into a protected portion of the first environment. The detected body temperatures are recorded from this second cooler environment upon confirmation of equilibration, by briefly opening a pathway to the subject’s skin, while carefully controlling to reduce equilibration to this second environment from air moving from the second to first environment. The devices can then be assessed for sensitivity, specificity and any other metric desired by comparing the elevated and detected body temperatures.

The equilibration process follows an approximately exponential decay between the previous environmental temperature to the new environmental temperature with a characteristic time constant. Fig. 7 shows a sequence of inner canthus skin temperature measurements after exposure to blown, heated air, which is not to be considered a suitable simulation of equilibration but used solely for illustration of time constants. For this test method, a minimum equilibration time of 10 minutes was set based on preliminary study confirming sufficient equilibration within several time constants. More work is planned to further elucidate the physiology of equilibration.

• Download figure
• Open in new tab

Fig. 7

Equilibration curve for skin exposed to a forced warm air for 60 seconds and then ambient air immediately prior to the start of acquisition. Fitting the exponential decay limited to short and longer time bounds found a time constant of 74 seconds for a faster component and 127 seconds for a slower component. Note, this synthetic scenario is not intended to replicate equilibration to moderately still air, only to illustrate the concept.

• Download figure
• Open in new tab

Fig. 8

Environment (tent) with opening cover removed and radiative shield lifted and tucked above the opening frame where each subject would stand for Snapshot measurements after completion of 10 minutes of Equilibration measurements. Device #4’s blackbody can be seen at the top right of the frame, Device #1 is shown to the right on a tripod stand, Device #2 is shown to the left. Not visible in this image: Device #3 and Device #5.

Several caveats are important to mention. First, the equilibration proceeds via conductive and radiative pathways, with more than half of the heat flow via radiation, so if the first environment’s radiative background temperature is significantly different from its air temperature, the equilibration will be to a different effective temperature. In many situations the background heat-carrying infrared radiation is coming from surfaces that are within a few degrees of the air temperature and as a result the effective temperature is within a degree of the air temperature. However this cannot be guaranteed in general and must be checked (see the Supplement - Radiative and Effective Temperature). This can be accounted for by taking an appropriately weighted sum of the radiative and air temperatures (e.g. 70% of radiative plus 30% of air) but it is easier in practice to use radiative shielding, such as sheets hung from a frame, suspended in air such that they match the air temperature. This can be verified during testing using an NCIT in surface temperature mode to verify radiative surfaces are within a suitable range of the air temperature, e.g. within 0.6C should produce an error no greater than 0.1C in calculated body temperature. Alternatively, an IRT used for tracking equilibration can serve the same purpose. Secondly, drafts will affect equilibration by reducing the boundary air layer at the skin so this must be minimized. Thirdly, if the temperature is not stable in both environments, the equilibration will not be stable, so the stability of the air temperature must also be monitored in both environments. If the radiative environment is uncoupled from the air temperature due to convection or too wide of a control window, the background radiance and air temperatures can be used to obtain an accurate estimate via a combined, effective temperature. Fourth, once equilibration is reached, the subject’s skin must be exposed to the test equipment, and this must be done without or before significant change in the subject’s skin due to exposure to the test equipment environment. Finally, the subject’s skin must be monitored during equilibration with a suitable IRT to measure the equilibration achieved.

2.4 Selection of Reference Body Measurement Site and Febrile Threshold

Medical thermometry devices have some latitude in specifying a febrile threshold for their device outputs. Some NCITs and IRTs specify thresholds as low as 99.5F. This work is concerned with assessing clinical suitability for febrile detection and because there are several body sites and thresholds to choose from a choice must be made for the reference site and febrile threshold. We aimed to select a site that balances accessibility and clinical accuracy and furthermore to select the most commonly used febrile threshold for this site and thus, we selected oral thermometry and 100.4F as the febrile threshold in this work. A deeper discussion of device-dependent thresholds is outside the scope of this study, but there are serious potential negative repercussions from relying on a lower threshold to ameliorate the loss of sensitivity from a strong bias-to-normal algorithm. Realistically, users will often ignore a reading between 99.5 and 100F or even higher.

It is important to point out the general concept of elevated body temperature detection has additional complexities we will not delve into deeply but briefly touch on a few here. Normal physiologic variability in core temperature is well established, with the time-of-day and menstrual cycle generally accepted as contributing the largest sources of variability, each ranging from 0.3 to 1F in magnitude{31}. The idea of an adjusted “average” body temperature, lower than the value of 37C or 98.6F arrived at in the 19th century, is still of considerable debate. Several possible causes or confounds are being investigated, such as increases in medication usage, changes in diet and lifestyle or decreases in latent infectious disease. However, accurate body thermometry is not simple, there is a range of rigor to these studies such as mining clinical records that may have been recorded with a variety of methods such as NCITs, and the view of some clinicians with routine involvement in close monitoring of body temperature (anesthesiology) is that 37C (98.6F) is a representative average{25, 31}. Ultimately it is not at all clear whether there is such a change.

3.1 Subjects and Test Summary

Under supervision of a St Olaf College’s IRB, we recruited 28 individuals (mean age 22.25 ± 7.45 yrs, 159.7 ± 36.5 lbs, 5.54 ± 0.32 ft, 12 males (43%); males mean age 23.58 ± 10.76 yrs, 161.6 ± 33.4 lbs, 5.75 ± 0.31 ft; females mean age 21.25 ± 2.84 yrs, 158.3 ± 36.5 lbs, 5.38 ± 0.22 ft) at St Olaf College from the student and faculty populations and, in five 2-hour sessions staggered over 8 days in August 2021, obtained oral temperature measurements and elevated temperature measurements after each individual stood for 10 minutes in each of four temperature-controlled tents, while being scanned with a thermal camera. After the 10 minutes concluded, the individual would stand at the rear wall inside the tent, behind a removable insulated cover and radiative shield, and through which they could be oriented towards a rotating table containing several secondary thermographic systems and the Device #3 thermometer. The tents were arranged around the rotating table (described below and in Fig. 9), configured to achieve the same distance and orientation of each system to each subject within an inch (tape measure to center console) for each tent, with markings indicated on the floor for rotational positioning. Each tent contained a space heater and relay controller configured for a 1F window about the following air temperatures: 76F, 80F, 84F, 88F (setpoints were approximate, each tent was monitored continuously via its own IRT device taking canthi measurements and each having an air temperature sensor calibrated to an accuracy of 0.1F).

• Download figure
• Open in new tab

Fig. 9

Environment (tent) with control of air temperature using a relay-controlled space heater. Equilibration is monitored using an equilibration scanner (IRT) acquiring sequential images over 10 minutes. NFC reader embedded in wall to activate equilibration scanner prior to entry of chamber and start of equilibration to synchronize acquisition of images and air temperature. Tent is modified for a second unoccluded pathway into the environment via a wall flap that is opened for a second NFC read and scan with scanner under test (“snapshot” scanner) and testing with other secondary scanners.

• Download figure
• Open in new tab

Fig. 10

Elevated temperature environments arranged uniformly about the scanners sited on a rotating lectern. Each environment is set to a different target temperature. A 4-foot tall lectern on freely rotating casters (the snapshot acquisition fixture) was used to hold the devices under test, with tape placed on the floor indicating the outlines of the lectern in order to maintain a consistent distance and orientation with respect to each tent. Each tent was arranged about this lectern with the opening of each tent facing the lectern and perpendicular to this axis as shown in Fig. 10, such that the distance from a subject’s face within each tent opening was maintained at 5 feet from Devices 2 and 4 mounted on the lectern. Device 5 was mounted on a separate tripod to the left and behind the lectern, Device 1 was mounted on a separate tripod to the right side of the lectern as shown in Fig. 8.

• Download figure
• Open in new tab

Fig. 11

Tent effective air temperature and simulated elevated body temperatures.

• Download figure
• Open in new tab

• Download figure
• Open in new tab

• Download figure
• Open in new tab

• Download figure
• Open in new tab

• Download figure
• Open in new tab

Fig. 12

Left column shows detected vs simulated elevated temperatures and right column shows Bland-Altman plot (using elevated temperatures instead of the average in the x-axis due to potential bias in the detected temperatures), for each of Devices #1-5.

3.2 Thermometry Devices

Oral thermometry was performed using a clinical heated probe oral thermometer (Welch Allyn SureTemp Plus 690) which had its accuracy tested in an immersion circulator water bath accurate to 0.1C traceable to NIST calibration standards (PolyScience SD7LR-20). This oral thermometer has a quick-read mode, returning an estimated core temperature in under 10 seconds, however this mode biases towards the population average (See Supplement Fig. S3), so for clinical study this thermometer must be switched into a direct mode and the operator must wait for several time constants, which in this study was taken to be 2 minutes past the quick-read result (approximately 2.25 minutes). Subjects and operators were trained on proper preparation and placement of the oral probe to the sublingual pocket using the manufacturer’s printed operating instructions.

The non-contact devices included four IRTs and one NCIT, as shown in Table 2, with each device’s specified (claimed) accuracy, febrile alarm threshold and approximate time required to obtain a temperature. Note, Device #5 operates only in surface temperature mode, and furthermore relies on a +1C (1.8F) threshold, based on the average of the last up-to-8 subjects within a time window. Because the moving average of subjects would be biased upwards by the elevated temperatures seen, the moving threshold was not used. Testing of the device indicated an average of 35.1C with non-elevated face temperatures in ambient temperatures of 22C, thus a threshold of 36.1C was used for Device #5 in subsequent analyses.

The Device #1 devices inside the tents and at the center testing station were configured to start measurement once the first face was detected after a USB-attached near-field communication (NFC) reader was activated by a valid NFC sticker. Each face measurement was initiated at a regular interval (minimum of 3 seconds after the previous for snapshot measurements at the end of the 10 minute equilibration and minimum of 4 seconds for equilibration measurements, and in each case requiring a further second to complete the face measurement), and relied on the product face measurement algorithm which records unix timestamps, distance to target, ambient temperature, inner canthus surface temperature, calculated body temperature, and various system statistics was modified to save additional data not normally recorded for every face measurement, including a secondary 0.1F-calibrated ambient temperature probe, face detection bounding boxes, and the set of thermal images used in each measurement. All Device #1 devices were synchronized via NTP and were monitored from a laptop at the center console which also operated the Device #4 Scan software (version 1.9.1.0).

3.3 Testing and Adjusted Environments

The testing environment was a 3rd floor college lecture room with concrete flooring and insulative walls and ceiling, with four environmental chambers arranged around a testing station. The testing room radiative temperature was approximately within 1C of the ambient air temperature, which was maintained at 71.2±0.73F (during each test session the air temperatures were within 0.56 to 1.46F of a setpoint). The environmental chambers consisted of a 300 denier nylon free-standing tent with no floor (THUNDERBAY Ice Cube 3 Man Portable Ice Shelter), one zippered entrance flap at a corner and four removable windows, three of which were covered and one of which was modified to accommodate a cover designed for easy removal.

Each tent contained a portable space heater (Andily Electric Space Heater at 750W setting) controlled by an on/off relay system (Ketotek KT4000) configured to operate with a 1F window below the setpoint. Setpoints were arranged to approximate a range of elevated body temperatures from 99.5F to 102.5F (see github repository for additional photographs of the test environments). The temperature probe was suspended approximately above where the subjects head would be positioned (between 6” and 1 foot of the ceiling), intended to avoid gradients due to the tent itself or to any vertical air column gradients within the tent. Radiative shielding (Mybecca 100% unbleached Cotton Muslin Fabric) hung from the ceiling edges plus insulation above the ceiling and on the concrete floor are used to more closely match the effective (radiative background) temperature to the actual air temperature inside the tent. A ¼” polycarbonate trapezoidal frame was adhered to the tent over the wall flap opening and a ¼” foam-core removable cover was designed to insert into this opening. The radiative shield over the wall flap was cut to match the bottom 3 sides of the trapezoidal wall flap opening and the bottom of the radiative shield pinned to the tent at the bottom of the wall flap opening so that volunteers could access the window.

3.4 Study Procedures

Each subject was given a data collection sheet having an NFC sticker and alphanumeric indicator (incrementing the letter by study group and incrementing the number by individual, e.g. C1 through C8 refers to the 8 subjects being acquired during the third study group). This sheet was used to collect demographic and potential confound information and the entries for oral thermometry acquired before the start of the study and oral thermometry in quick-read and monitor mode after each equilibration session. After completing their demographic information, each subject practiced using a dedicated training scanner having a live view of the thermal images. After practice was complete, the first subject (e.g. C1) was given a tent to proceed to (e.g. “volunteer C1 please proceed to tent A”), at which point the subject scanned the NFC sticker on their sheet on an NFC reader embedded in the outer wall prior to entry (note, the first five groups were acquired without a practice session and with the NFC reader inside the tent, which led to uncertainty between the start of equilibration and the first image acquired during equilibration). After entering the tent, assisted by study personnel, the subject stood in front of the equilibration scanner for 10 minutes until its display indicated completion, which was also monitored remotely by personnel at the snapshot acquisition fixture. On completion, personnel prepared by orienting the fixture to the completed tent, placed the reference blackbody for Device #4 on the wall frame, before removing the wall frame cover and passing the second NFC reader under a cutout in the radiative shield inside the tent. The subject scanned their data collection NFC sticker on this NFC reader and the personnel lifted the radiative shield cutout, instructed the subject to stand just inside the opening, and personnel then proceeded to acquire a single acquisition using each scanner being tested. Device #1 automatically acquires data once a face is observed and the snapshot scanner was configured to acquire three snapshots. Devices #2 and #5 both continuously report a body temperature and surface temperature respectively. Device #4 automatically reports a body temperature once a face is observed with the blackbody in view next to the face. Device #3 (NCIT) was manually operated by study personnel by swiping the probe head across the subject’s forehead-to-temple region following manufacturer’s instructions and sanitized with an EPA-approved (List N for COVID) disinfecting wipe after use. All temperatures except for the Device #1 were recorded manually by study personnel. Once the test snapshots are complete, the subject is directed to exit the tent via the entrance flap and receive an oral thermometry measurement, with both the quick-read and a two-minute monitor-mode temperature recorded. After all four tent acquisitions and oral temperatures, subject participation was complete.

3.5 Study Analyses

For each equilibration dataset, the inner canthus, background radiance and calibrated ambient air temperatures were filtered with a 3-wide, 1st-order Savitsky-Golay filter over time and re-aligned to a unified end-of-equilibration time grid using the unix timestamps for each measurement. Background radiance was estimated using the median of the coldest 10% of pixels, verified in each measurement to correspond to the radiative shield material. Inspection of the ambient temperatures revealed a difference between the air temperatures and background radiance, the effect of which was approximated using a weighted sum of radiative and ambient air temperatures in order to derive an effective environmental temperature timecourse (see Supplement - Radiative and Effective Temperature).

Simulated elevated body temperatures were calculated using the inside-tent measured inner canthi surface temperature at the conclusion of equilibration, prior to exposure to the outside-tent scanning devices. In this method, the equilibrated surface temperature is inserted into Equation 1 with the ambient outside-tent air temperature to obtain an simulated elevated body temperature.

For each Device #1 snapshot dataset taken from the outer testing environment, the three inner canthus measurements were taken over 8 seconds (every 4 seconds) and the average was used to calculate the Device #1 detected body temperature using the outer testing environment’s air temperature. For the Device #5 scanner, the reported detected facial surface temperature was recorded without any extrapolation. All other devices reported a detected body temperature.

For each device’s manufacturer-specified febrile threshold (100.4F for Devices #1 and #4, 99.14F for Device #2, 100.1F for Device #3, and an approximation of the moving-average threshold for Device #5), the true positive rate (TPR) was determined from the percentage of simulated elevated body temperatures over the device’s threshold (number of febrile) that were detected as above the device’s threshold (detected as febrile). The false positive rate (FPR) was determined by dividing the number of below-threshold simulated body temperatures that were detected above the device’s threshold (false positives) by the total number of below-threshold simulated body temperatures (number of non-febrile). The detected body temperature output versus the simulated elevated body temperature was plotted for each device (using surface instead of body temperatures for Device #5 only), and a linear fit performed, with the fit coefficients inspected for under- or over-estimation of body temperatures. Bland-Altman plots of detected minus simulated elevated temperatures versus simulated elevated temperatures were plotted for each device with 2 sigma bounds, or limits of acceptability (LA) and the average difference, or clinical bias (CB), denoted by dashed lines.

Note, Device #5 calculates the moving average of the last 8 subjects and sets a changing threshold of 1C greater than this average. This threshold would be manipulated by the presence of elevated face temperatures, whether simulated or actual, and thus cannot be used to assess efficacy without a pool of subjects with normal body temperatures on standby to continually reset the moving average, which was not possible with the limit on the number of subjects allowed in the testing environment in the present study. To compare devices we calculated the surface temperature corresponding to the average oral temperature of the study population and added 1C (1.8F) to use as the selection threshold to identify simulated febrile or non-febrile, and to obtain the device detection threshold we used Eq 2 to extrapolate a surface temperature for the average oral temperature and added 1C (1.8F). For all other devices, we used each manufacturer-specified febrile threshold as given in Table 2.

4.1 Oral and simulated elevated temperatures

The oral temperatures were averaged post-tent exposure (the average oral temperatures per tent, in increasing tent temperature order, were 98.33F, 98.31F, 98.33F and 98.45F) and no trend was observed in oral temperatures after elevated air exposure. Note however the average oral thermometry in the last exposure was 0.12F greater (2.28 standard deviations) than the average of the oral thermometry after the first three exposures.

Twenty-eight subjects in four chambers produced 112 elevated body temperatures (one measurement corrupted), enumerated in Table 3 and in Supplemental Table S1.

4.2 Detected versus simulated elevated temperatures

These figures show what may be bias-to-normal behavior in devices 2 and 3. The ideal Bland-Altman plot should show no trendline, but in these cases, a negative slope in the Bland-Altman plot for devices #2 and #3 indicates that as the object temperature increases, the reported temperature will tend to the same value. Note, we did not plot the difference versus the average because in the presence of a bias-to-normal, the average could obscure these effects.

The results shown in Table 4 are similar to those produced in a recent study{48} that used oral thermometry, IRTs and careful adherence to ISO guidelines. In particular, Figure 3 of that study shows the IRT vs oral temperatures for three IRT systems, each of which show a narrower range of IRT-reported body temperatures compared to the oral temperatures. Of note, Camera 2 is the same device and model as Device #5 in the present study, and its surface temperature output found to fit the body temperatures with a similar squared correlation coefficient and slope with a slope of 0.5 here versus 0.6186 in {48}, both of which refer to linear fitting of the surface temperature output, not the calculated body temperature. Of note, the present study did not use a felt board behind the subject for Device #5 due to the tent design, which may have contributed to this difference (see figure 1 in {48}). The average temperatures detected by each device is shown in Table 5, with the average elevated temperature for each tent for comparison.

Note the distribution of elevated temperatures straddled the febrile thresholds used and as a consequence, the false positive rate can be expected to be high for the thresholds used here, yet the real-world false positive rate would be expected to be far lower when it is weighted by the distribution of real temperatures near the threshold.

Disclosures

Dr Beall is a founder and CEO of Thermal Diagnostics LLC, which makes and sells the Device #1 IRT used in this study and Dr Beall has a financial stake in the company. Dr Schmidt is an Engineering Fellow at Fluke Corporation, which manufactures IRTs for industrial applications; his involvement in this research is independent of his work at Fluke Corporation and the views expressed here do not necessarily represent the views of Fluke. Dr Hinnerichs’ research is independent of the Department of Defense and does not reflect the opinion of the United States Army. Dr Adolph declares and attests she has no competing interests to declare. Mr Askegaard and Mr Berkesch are students at St Olaf College and declare and attest they have no competing interests.