Elsevier

NeuroImage

Volume 49, Issue 2, 15 January 2010, Pages 1677-1689
NeuroImage

Brain networks involved in haptic and visual identification of facial expressions of emotion: An fMRI study

https://doi.org/10.1016/j.neuroimage.2009.09.014Get rights and content

Abstract

Previous neurophysiological and neuroimaging studies have shown that a cortical network involving the inferior frontal gyrus (IFG), inferior parietal lobe (IPL) and cortical areas in and around the posterior superior temporal sulcus (pSTS) region is employed in action understanding by vision and audition. However, the brain regions that are involved in action understanding by touch are unknown. Lederman et al. (2007) recently demonstrated that humans can haptically recognize facial expressions of emotion (FEE) surprisingly well. Here, we report a functional magnetic resonance imaging (fMRI) study in which we test the hypothesis that the IFG, IPL and pSTS regions are involved in haptic, as well as visual, FEE identification. Twenty subjects haptically or visually identified facemasks with three different FEEs (disgust, neutral and happiness) and casts of shoes (shoes) of three different types. The left posterior middle temporal gyrus, IPL, IFG and bilateral precentral gyrus were activated by FEE identification relative to that of shoes, regardless of sensory modality. By contrast, an inferomedial part of the left superior parietal lobule was activated by haptic, but not visual, FEE identification. Other brain regions, including the lingual gyrus and superior frontal gyrus, were activated by visual identification of FEEs, relative to haptic identification of FEEs. These results suggest that haptic and visual FEE identification rely on distinct but overlapping neural substrates including the IFG, IPL and pSTS region.

Introduction

A recent study shows that with minimal training humans can haptically recognize facial expressions of emotion (FEE) surprisingly well (Lederman et al., 2007). This result demonstrates that touch can be useful for recognizing the actions of another person. Haptic face recognition may help congenitally blind individuals improve voluntary production of FEEs, which are more poorly recognized than those of sighted individuals (Rinn, 1991, Galati et al., 1997). However, little is known about the neural mechanisms underlying haptic recognition of nonverbal gestures. The present study focused on the neural substrates responsible for haptic recognition of FEEs by sighted individuals.

Previous studies suggest a shared representation for the execution and observation of actions. For instance, the inferior frontal gyrus (IFG) and inferior parietal lobule (IPL) are activated both by imitation and by visual recognition of FEEs (Carr et al., 2003, Montgomery and Haxby, 2008). These results are in accord with other neuroimaging studies suggesting that these regions may constitute the human homologues of the mirror-neuron system (Grafton et al., 1996, Iacoboni et al., 1999, Nishitani and Hari, 2002) in which neurons are activated by both execution and observation of an action (Rizzolatti et al., 1996, Ferrari et al., 2003, Fogassi et al., 2005).

A wealth of neurophysiological studies has shown that cortical areas in and near the superior temporal sulcus (STS region) are sensitive to stimuli that signal the actions of another individual (Perrett et al., 1985, Allison et al., 2000, Haxby et al., 2000). For example, visual recognition of static FEEs activates the posterior STS region (pSTS region) including the middle temporal gyrus (Gorno-Tempini et al., 2001) as well as the superior temporal sulcus (Narumoto et al., 2001). Furthermore, the IFG and STS regions in primates are indirectly connected with each other, via the IPL (Cavada and Goldman-Rakic, 1989, Seltzer and Pandya, 1994). Consequently, it has been proposed that the IFG, IPL and STS regions together constitute a network for action understanding (Rizzolatti et al., 2001).

If the brain network involving the IFG, IPL and STS region is critical for action understanding, such a network should be involved in action understanding, regardless of the sensory modality. Indeed, neurophysiological and neuroimaging studies have shown that the auditory perception of actions also activates IFG and IPL (Kohler et al., 2002, Lahav et al., 2007) and pSTS (Beauchamp et al., 2004) regions. However, to our knowledge, no study has examined whether this brain network contributes to haptic perception of the actions of others.

In the present study, we utilized functional magnetic imaging (fMRI) to test the hypothesis that the IFG, IPL and pSTS regions are involved in haptic identification of FEEs. On each trial, subjects used either touch or vision to identify one of three emotions (disgust, neutral and happiness) expressed on 3D facemasks, or they identified one of three kinds of 3D casts of (dress shoe, running shoe and hiking boot). We first identify brain regions activated by identification of FEEs relative to identification of shoes in each sensory modality. Then, we tested whether common regions are activated by both sensory modalities. Finally, we examine brain regions involved in haptic and visual identification of specific FEEs.

Section snippets

Subjects

Twenty healthy volunteers (9 male and 11 female) aged 18–31 years participated in the fMRI study. All participants were right-handed according to the Edinburgh Handedness Inventory (Oldfield, 1971). None of the volunteers had a history of symptoms requiring neurological, psychological or other medical care. All subjects gave written informed consent. The study was approved by the local medical ethics committee at Queen's University (Canada).

Stimuli

We used two different basic-level (Rosch, 1976)

Haptic object-identification task

Performance accuracy was similar for faces and control objects: 92.3 ± 1.2% (mean ± SEM) for faces and 92.1 ± 1.4% for control objects (Fig. 3). A paired t test on performance accuracy comparing faces and control objects showed no significant difference (p > 0.9). On the other hand, the same test on response time showed that faces produced longer response times than control objects (1090 ± 96 and 1020 ± 89 ms, respectively; t19 = 2.73, p < 0.05).

Visual object-identification task

Performance accuracy was comparable for faces and control

Discussion

The present study tested the hypothesis that the IFG, IPL and pSTS regions participate in a network that supports haptic and visual FEE identification. Our group-average analysis showed that activity in all of these regions was greater for FEE than for shoe identification, regardless of sensory modality. In addition to these commonalities, activity in some regions depended on sensory modality. An inferomedial part of the superior parietal lobule was activated by haptic, but not visual, FEE

Acknowledgments

This study was supported by a Postdoctoral Fellowship for Research Abroad from the Japan Society for the Promotion of Science and a Grant-in-Aid for Scientific Research on Innovative Areas, “Face perception and recognition” (21119524) from the Ministry of Education, Science, Sports and Culture, Japan, to R.K. and grants from the Natural Sciences and Engineering Research Council of Canada and the Canadian Institutes of Health Research to S.L. We thank S. David for assistance with neuroimaging,

References (73)

  • IzumaK. et al.

    Processing of social and monetary rewards in the human striatum

    Neuron

    (2008)
  • JamesT.W. et al.

    Haptic study of three-dimensional objects activates extrastriate visual areas

    Neuropsychologia

    (2002)
  • JamesT.W. et al.

    The influence of familiarity on brain activation during haptic exploration of 3-D facemasks

    Neurosci. Lett.

    (2006)
  • Kesler-WestM.L. et al.

    Neural substrates of facial emotion processing using fMRI

    Brain Res. Cogn. Brain Res.

    (2001)
  • KilgourA.R. et al.

    Haptic face identification activates ventral occipital and temporal areas: an fMRI study

    Brain Cogn.

    (2005)
  • KitadaR. et al.

    Tactile estimation of the roughness of gratings yields a graded response in the human brain: an fMRI study

    NeuroImage

    (2005)
  • LedermanS.J. et al.

    Hand movements: a window into haptic object recognition

    Cogn. Psychol.

    (1987)
  • LedermanS.J. et al.

    Haptic identification of common objects: knowledge-driven exploration

    Cogn. Psychol.

    (1990)
  • LeslieK.R. et al.

    Functional imaging of face and hand imitation: towards a motor theory of empathy

    NeuroImage

    (2004)
  • NarumotoJ. et al.

    Attention to emotion modulates fMRI activity in human right superior temporal sulcus

    Brain Res. Cogn. Brain Res.

    (2001)
  • NicholsT. et al.

    Valid conjunction inference with the minimum statistic

    NeuroImage

    (2005)
  • NishitaniN. et al.

    Viewing lip forms: cortical dynamics

    Neuron

    (2002)
  • NumminenJ. et al.

    Cortical activation during a spatiotemporal tactile comparison task

    NeuroImage

    (2004)
  • OldfieldR.C.

    The assessment and analysis of handedness: The Edinburgh Inventory

    Neuropsychologia

    (1971)
  • RizzolattiG. et al.

    Premotor cortex and the recognition of motor actions

    Brain Res. Cogn. Brain Res.

    (1996)
  • RoschE.H.

    Basic objects in natural categories

    Cogn. Psychol.

    (1976)
  • ShattuckD.W. et al.

    Construction of a 3D probabilistic atlas of human cortical structures

    NeuroImage

    (2008)
  • WagerT.D. et al.

    Optimization of experimental design in fMRI: a general framework using a genetic algorithm

    NeuroImage

    (2003)
  • WorsleyK.J. et al.

    Analysis of fMRI time-series revisited—again

    NeuroImage

    (1995)
  • AmediA. et al.

    Visuo-haptic object-related activation in the ventral visual pathway

    Nat. Neurosci.

    (2001)
  • AmuntsK. et al.

    Broca's region revisited: cytoarchitecture and intersubject variability

    J. Comp. Neurol.

    (1999)
  • BeauchampM.S. et al.

    Unraveling multisensory integration: patchy organization within human STS multisensory cortex

    Nat. Neurosci.

    (2004)
  • CalderA.J. et al.

    Impaired recognition and experience of disgust following brain injury

    Nat. Neurosci.

    (2000)
  • CalderA.J. et al.

    Neuropsychology of fear and loathing

    Nat. Rev. Neurosci.

    (2001)
  • CarrL. et al.

    Neural mechanisms of empathy in humans: a relay from neural systems for imitation to limbic areas

    Proc. Natl. Acad. Sci. U. S. A.

    (2003)
  • CavadaC. et al.

    Posterior parietal cortex in rhesus monkey: II. Evidence for segregated corticocortical networks linking sensory and limbic areas with the frontal lobe

    J. Comp. Neurol.

    (1989)
  • Cited by (86)

    • Altered spontaneous brain activity in patients with comitant exotropia before and after surgery: A resting-state fMRI study

      2022, Experimental Eye Research
      Citation Excerpt :

      2) At one month after surgery, the patients showed increased ALFF values in bilateral cuneus, calcarine sulcus and lingual gyrus relative to preoperative collection, while still reduced ALFF values in the cluster involving left lingual gyrus and calcarine sulcus compared with HCs. The calcarine sulcus, lingual gyrus and cuneus are major components of secondary visual cortex (V2), which plays a critical role in visual memory and various higher order functions of vision, including processing of spatial visual information, perception of color and identification of facial expressions of emotions (Bi et al., 2011; Kitada et al., 2010; Roland and Gulyas, 1995). A previous positron emission tomography study has reported V2 to be highly activated in infantile exotropia patients, suggesting a potential role in optical fusion (Yang et al., 2014).

    View all citing articles on Scopus
    View full text