Volume 15, Issue 3
  • ISSN 1568-1475
  • E-ISSN: 1569-9773
Buy:$35.00 + Taxes
  • Embodiment and American Sign Language

    Exploring sensory-motor influences in the recognition of American Sign Language

  • Author(s): David P. Corina 1  and Eva Gutierrez 2
  • View Affiliations Hide Affiliations
    1 Center for Mind and Brain, University of California, Davis
    2 Departamento de Metodología de las Ciencias del Comportamiento, Universitat de València
  • Source: Gesture, Volume 15, Issue 3, Jan 2016, p. 291 - 305
  • DOI: https://doi.org/10.1075/gest.15.3.01cor
    • Version of Record published : 28 Nov 2016


Little is known about how individual signs that occur in naturally produced signed languages are recognized. Here we examine whether sign understanding may be grounded in sensorimotor properties by evaluating a signer’s ability to make lexical decisions to American Sign Language (ASL) signs that are articulated either congruent with or incongruent with the observer’s own handedness. Our results show little evidence for handedness congruency effects for native signers’ perception of ASL, however handedness congruency effects were seen in non-native late learners of ASL and hearing ASL-English bilinguals. The data are compatible with a theory of sign recognition that makes reference to internally simulated articulatory control signals — a forward model based upon sensory-motor properties of one’s owns body. The data suggest that sign recognition may rely upon an internal body schema when processing is non-optimal as a result of having learned ASL later in life. Native signers however may have developed representations of signs which are less bound to the hand with which it is performed, suggesting a different engagement of an internal forward model for rapid lexical decisions.


Article metrics loading...

Loading full text...

Full text loading...


  1. Adank, Patti , Shirley-Ann Rueschemeyer , & Harold Bekkering
    (2013) The role of accent imitation in sensorimotor integration during processing of intelligible speech. Frontiers in Human Neuroscience, 7, Article 634. doi: 10.3389/fnhum.2013.00634
    https://doi.org/10.3389/fnhum.2013.00634 [Google Scholar]
  2. Balota, David A. & James I. Chumbley
    (1984) Are lexical decisions a good measure of lexical access? The role of word frequency in the neglected decision stage. Journal of Experimental Psychology: Human Perception and Performance, 10 (3), 340–357. doi: 10.1037/0096‑1523.10.3.340
    https://doi.org/10.1037/0096-1523.10.3.340 [Google Scholar]
  3. Callan, Akiko M. , Daniel E. Callan , Keiichi Tajima , & Reiko Akahane-Yamada
    (2006) Neural processes involved with perception of non-native durational contrasts. Neuroreport, 17 (12), 1353–1357. doi: 10.1097/01.wnr.0000224774.66904.29
    https://doi.org/10.1097/01.wnr.0000224774.66904.29 [Google Scholar]
  4. Callan, Daniel E. , Jeffery A. Jones , Akiko M. Callan , & Reiko Akahane-Yamada
    (2004) Phonetic perceptual identification by native- and second-language speakers differentially activates brain regions involved with acoustic phonetic processing and those involved with articulatory-auditory/orosensory internal models. Neuroimage, 22 (3), 1182–1194. doi: 10.1016/j.neuroimage.2004.03.006
    https://doi.org/10.1016/j.neuroimage.2004.03.006 [Google Scholar]
  5. Callan, Daniel , Akiko Callan , & Jeffery A. Jones
    (2014) Speech motor brain regions are differentially recruited during perception of native and foreign-accented phonemes for first and second language listeners. Frontiers in Neuroscience, 8, 1–15. doi: 10.3389/fnins.2014.00275
    https://doi.org/10.3389/fnins.2014.00275 [Google Scholar]
  6. Corina, David & Heather Knapp
    (2006) Sign language processing and the mirror neuron system. Cortex, 42 (4), 529–539. doi: 10.1016/S0010‑9452(08)70393‑9
    https://doi.org/10.1016/S0010-9452(08)70393-9 [Google Scholar]
  7. Du, Yi , Bradley R. Buchsbaum , Cheryl L. Grady , & Claude Alain
    (2014) Noise differentially impacts phoneme representations in the auditory and speech motor systems. Proceedings of the National Academy of Sciences, 111 (19), 7126–7131. doi: 10.1073/pnas.1318738111.
    https://doi.org/10.1073/pnas.1318738111 [Google Scholar]
  8. Emmorey, Karen
    (2002) Language, cognition, and the brain: Insights from sign language research. Mahwah, NJ: Lawrence Erlbaum.
    [Google Scholar]
  9. Emmorey, Karen , Jennifer A. Petrich , & Tamar H. Gollan
    (2012) Bilingual processing of ASL–English code-blends: The consequences of accessing two lexical representations simultaneously. Journal of Memory and Language, 67 (1), 199–210. doi: 10.1016/j.jml.2012.04.005
    https://doi.org/10.1016/j.jml.2012.04.005 [Google Scholar]
  10. Galantucci, Bruno , Carol A. Fowler , & Michael T. Turvey
    (2006) The motor theory of speech perception reviewed. Psychonomic Bulletin & Review, 13 (3), 361–377. doi: 10.3758/BF03193857
    https://doi.org/10.3758/BF03193857 [Google Scholar]
  11. Hickok, Greg
    (2010) The role of mirror neurons in speech perception and action word semantics. Language and Cognitive Processes, 25 (6), 749–776. doi: 10.1080/01690961003595572
    https://doi.org/10.1080/01690961003595572 [Google Scholar]
  12. Iacoboni, Marco
    (2008) The role of premotor cortex in speech perception: Evidence from fMRI and rTMS. Journal of Physiology – Paris, 102 (1), 31–34. doi: 10.1016/j.jphysparis.2008.03.003
    https://doi.org/10.1016/j.jphysparis.2008.03.003 [Google Scholar]
  13. Liberman, Alvin M. , Franklin S. Cooper , Donald P. Shankweiler , & Michael Studdert-Kennedy
    (1967) Perception of the speech code. Psychological Review, 74 (6), 431–461. doi: 10.1037/h0020279
    https://doi.org/10.1037/h0020279 [Google Scholar]
  14. Liberman, Alvin M. & Ignatius G. Mattingly
    (1985) The motor theory of speech perception revised. Cognition, 21 (1), 1–36. doi: 10.1016/0010‑0277(85)90021‑6
    https://doi.org/10.1016/0010-0277(85)90021-6 [Google Scholar]
  15. MacSweeney, Mairéad , Bencie Woll , Ruth Campbell , Philip K. McGuire , Anthony S. David , Steven C.R. Williams , John Suckling , Gemma A. Calvert , & Michael J. Brammer
    (2002) Neural systems underlying British Sign Language and audio‐visual English processing in native users. Brain, 125 (7), 1583–1593. doi: 10.1093/brain/awf153
    https://doi.org/10.1093/brain/awf153 [Google Scholar]
  16. Miall, R. Chris
    (2003) Connecting mirror neurons and forward models. Neurorepor, 14 (17), 2135–2137. doi: 10.1097/01.wnr.0000098751.87269.77
    https://doi.org/10.1097/01.wnr.0000098751.87269.77 [Google Scholar]
  17. Neville, Helen J. , Daphne Bavelier , David Corina , Josef Rauschecker , Avi Karni , Anil Lalwani , Allen Braun , Vince Clark , Peter Jezzard , & Robert Turner
    (1998) Cerebral organization for language in deaf and hearing subjects: Biological constraints and effects of experience. Proceedings of the National Academy of Sciences, 95 (3), 922–929. doi: 10.1073/pnas.95.3.922
    https://doi.org/10.1073/pnas.95.3.922 [Google Scholar]
  18. Poeppel, David , William J. Idsardi , & Virginie van Wassenhove
    (2008) Speech perception at the interface of neurobiology and linguistics. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 363 (1493), 1071–1086. doi: 10.1098/rstb.2007.2160
    https://doi.org/10.1098/rstb.2007.2160 [Google Scholar]
  19. Pulvermüller, Friedemann , Martina Huss , Ferath Kherif , Fermin Moscoso del Prado Martin , Olaf Hauk , & Yury Shtyrov
    (2006) Motor cortex maps articulatory features of speech sounds. Proceedings of the National Academy of Sciences of the United States of America, 103 (20), 7865–7870. doi: 10.1073/pnas.0509989103
    https://doi.org/10.1073/pnas.0509989103 [Google Scholar]
  20. Rauschecker, Josef P
    (2011) An expanded role for the dorsal auditory pathway in sensorimotor control and integration. Hearing Research, 271 (1), 16–25. doi: 10.1016/j.heares.2010.09.001
    https://doi.org/10.1016/j.heares.2010.09.001 [Google Scholar]
  21. Schwartz, Jean-Luc , Anahita Basirat , Lucie Ménard , & Marc Sato
    (2012) The Perception-for-Action-Control Theory (PACT): A perceptuo-motor theory of speech perception. Journal of Neurolinguistics, 25 (5), 336–354. doi: 10.1016/j.jneuroling.2009.12.004
    https://doi.org/10.1016/j.jneuroling.2009.12.004 [Google Scholar]
  22. Skipper, Jeremy I. , Susan Goldin-Meadow , Howard Nusbaum , & Steven Small
    (2007) Speech associated gestures, Broca’s area and the human mirror system. Brain and Language, 101 (3), 260–277. doi: 10.1016/j.bandl.2007.02.008
    https://doi.org/10.1016/j.bandl.2007.02.008 [Google Scholar]
  23. Thompson, Robin & Gabriella Vigliocco
    (2013) The relationship between sign production and sign comprehension: What the hands reveal. Presentation at the Conference Theoretical Issues in Sign Language Research 2013.
    [Google Scholar]
  24. Tremblay, Pascale & Steven L. Small
    (2011) On the context-dependent nature of the contribution of the ventral premotor cortex to speech perception. NeuroImage, 57 (4), 1561–1571. doi: 10.1016/j.neuroimage.2011.05.067
    https://doi.org/10.1016/j.neuroimage.2011.05.067 [Google Scholar]
  25. Wang, Yue , Joan A. Sereno , Allard Jongman , & Joy Hirsch
    (2003) fMRI evidence for cortical modification during learning of mandarin lexical tone. Journal of Cognitive Neuroscience, 15 (7), 1019–1027. doi: 10.1162/089892903770007407
    https://doi.org/10.1162/089892903770007407 [Google Scholar]
  26. Willems, Roel M. & Peter Hagoort
    (2007) Neural evidence for the interplay between language, gesture, and action: A review. Brain and Language, 101 (3), 278–289. doi: 10.1016/j.bandl.2007.03.004
    https://doi.org/10.1016/j.bandl.2007.03.004 [Google Scholar]
  27. Willems, Roel M. , Peter Hagoort , & Daniel Casasanto
    (2010) Body-specific representations of action verbs: Neural evidence from right-and left-handers. Psychological Science, 21 (1), 67–74. doi: 10.1177/0956797609354072
    https://doi.org/10.1177/0956797609354072 [Google Scholar]
  28. Wilson, Stephen M. , Ayse P. Saygin , Martin I. Sereno , & Marco Iacoboni
    (2004) Listening to speech activates motor areas involved in speech production. Nature Neuroscience, 7 (7), 701–702. doi: 10.1038/nn1263
    https://doi.org/10.1038/nn1263 [Google Scholar]
  29. Wilson, Stephen M. & Marco Iacoboni
    (2006) Neural responses to non-native phonemes varying in producibility: Evidence for the sensorimotor nature of speech perception. NeuroImage, 33 (1), 316–325. doi: 10.1016/j.neuroimage.2006.05.032
    https://doi.org/10.1016/j.neuroimage.2006.05.032 [Google Scholar]
  30. Wolpert, Daniel , Kenji Doya , & Mitsuo Kawato
    (2003) A unifying computational framework for motor control and social interaction. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 358 (1431), 593–602. doi: 10.1098/rstb.2002.1238
    https://doi.org/10.1098/rstb.2002.1238 [Google Scholar]
  • Article Type: Research Article
Keyword(s): American Sign Language; deaf; embodiment; handedness
This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error