1887
  1. Home
  2. e-Journals
  3. Gesture
  4. Volume 17, Issue 2
  5. Article
Volume 17, Issue 2
  • ISSN 1568-1475
  • E-ISSN: 1569-9773
USD
Buy:$35.00 + Taxes

Abstract

Abstract

We examined the cognitive resources involved in processing speech with gesture compared to the same speech without gesture across four studies using a dual-task paradigm. Participants viewed videos of a woman describing spatial arrays either with gesture or without. They then attempted to choose the target array from among four choices. Participants’ cognitive load was measured as they completed this comprehension task by measuring how well they could remember the location and identity of digits in a secondary task. We found that addressees experience additional visuospatial load when processing gestures compared to speech alone, and that the load primarily comes when addressees attempt to use their memory of the descriptions with gesture to choose the target array. However, this cost only occurs when gestures about horizontal spatial relations (i.e., left and right) are produced from the speaker’s egocentric perspective.

© John Benjamins Publishing Company
Loading

Article metrics loading...

/content/journals/10.1075/gest.17017.hos
2019-06-26
2024-10-04

  • From This Site

    /content/journals/10.1075/gest.17017.hos
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Alibali, Martha W.
    (2005) Gesture in spatial cognition: Expressing, communicating, and thinking about spatial information. Spatial Cognition & Computation, 5, 307–331. doi:  10.1207/s15427633scc0504_2
     https://doi.org/10.1207/s15427633scc0504_2 [Google Scholar]
  2. Baddeley, Alan
    (1992) Working memory. Science, 255, 556–559. 10.1126/science.1736359
     https://doi.org/10.1126/science.1736359 [Google Scholar]
  3. Goldin-Meadow, Susan, Howard Nusbaum, Spencer D. Kelly, & Susan Wagner
    (2001) Explaining math: Gesturing lightens the load. Psychological Science, 12, 516–522. doi:  10.1111/1467‑9280.00395
     https://doi.org/10.1111/1467-9280.00395 [Google Scholar]
  4. Holler, Judith, Heather Shovelton, & Geoffrey Beattie
    (2009) Do iconic hand gestures really contribute to the communication of semantic information in a face-to-face context?Journal of Nonverbal Behavior, 33, 73–88. doi: 10.1007/s10910‑008‑0063‑9
     https://doi.org/10.1007/s10910-008-0063-9 [Google Scholar]
  5. Hostetter, Autumn B.
    (2011) When do gestures communicate? A meta-analysis. Psychological Bulletin, 137, 297–315. doi:  10.1037/a0022128
     https://doi.org/10.1037/a0022128 [Google Scholar]
  6. Kelly, Spencer D., Peter Creigh, & James Bartolotti
    (2009) Integrating speech and iconic gestures in a Stroop-like task: Evidence for automatic processing. Journal of Cognitive Neuroscience, 22, 683–694. 10.1162/jocn.2009.21254
     https://doi.org/10.1162/jocn.2009.21254 [Google Scholar]
  7. Kelly, Spencer D. & Leslie H. Goldsmith
    (2004) Gesture and right hemisphere involvement in evaluating lecture material. Gesture, 4, 25–42. 10.1075/gest.4.1.03kel
     https://doi.org/10.1075/gest.4.1.03kel [Google Scholar]
  8. Kelly, Spencer, Meghan Healey, Aslı Özyürek, & Judith Holler
    (2015) The processing of speech, gesture, and action during language comprehension. Psychonomic Bulletin & Review, 22, 517–523. doi:  10.3758/s13423‑014‑0681‑7
     https://doi.org/10.3758/s13423-014-0681-7 [Google Scholar]
  9. Kelly, Spencer D., Corinne Kravitz, & Michael Hopkins
    (2004) Neural correlates of bimodal speech and gesture comprehension. Brain and Language, 89, 253–260. doi:  10.1016/S0093‑934X(03)00335‑3
     https://doi.org/10.1016/S0093-934X(03)00335-3 [Google Scholar]
  10. Kelly, Spencer D., Aslı Özyürek, & Eric Maris
    (2010) Two sides of the same coin: Speech and gesture mutually interact to enhance comprehension. Psychological Science, 21, 260–267. 10.1177/0956797609357327
     https://doi.org/10.1177/0956797609357327 [Google Scholar]
  11. Kelly, Spencer D., Sarah Ward, Peter Creigh, & James Bartolotti
    (2007) An intentional stance modulates the integration of gesture and speech during comprehension. Brain and Language, 101, 222–233. doi:  10.1016/j.bandl.2006.07.008
     https://doi.org/10.1016/j.bandl.2006.07.008 [Google Scholar]
  12. Kendon, Adam
    (1994) Do gestures communicate? A review. Research on Language and Social Interaction, 27, 175–200. doi:  10.1207/s15327973rlsi2703_2
     https://doi.org/10.1207/s15327973rlsi2703_2 [Google Scholar]
  13. Lakens, Daniël
    (2013) Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863. doi:  10.3389/fpsyg.2013.00863
     https://doi.org/10.3389/fpsyg.2013.00863 [Google Scholar]
  14. Maricchiolo, Fridanna, Augusto Gnisci, Marino Bonaiuto, & Gianluca Ficca
    (2009) Effects of different types of hand gestures in persuasive speech on receivers’ evaluations. Language and Cognitive Processes, 24, 239–266. doi: 10.1080/0169096080215992
     https://doi.org/10.1080/0169096080215992 [Google Scholar]
  15. Paivio, Allan
    (1991) Dual coding theory: Retrospect and current status. Canadian Journal of Psychology, 45, 255–287. 10.1037/h0084295
     https://doi.org/10.1037/h0084295 [Google Scholar]
  16. Post, Lysanne S., Tamara van Gog, Fred Paas, & Rolf A. Zwaan
    (2013) Effects of simultaneously observing and making gestures while studying grammar animations on cognitive load and learning. Computers in Human Behavior, 29, 1450–1455. 10.1016/j.chb.2013.01.005
     https://doi.org/10.1016/j.chb.2013.01.005 [Google Scholar]
  17. Pyers, Jennie E., Pamela Perniss, & Karen Emmorey
    (2015) Viewpoint in the visual-spatial modality: The coordination of spatial perspective. Spatial Cognition & Computation: An Interdisciplinary Journal, 15, 143–169. doi:  10.1080/13875868.2014.1003933
     https://doi.org/10.1080/13875868.2014.1003933 [Google Scholar]
  18. Rueckert, Linda, Ruth Breckinridge Church, Andrea Avila, & Theresa Trejo
    (2017) Gesture enhances learning of a complex statistical concept. Cognitive Research, 2, 1–6. doi:  10.1186/s41235‑016‑0036‑1
     https://doi.org/10.1186/s41235-016-0036-1 [Google Scholar]
  19. Sueyoshi, Ayano & Debra M. Hardison
    (2005) The role of gestures and facial cues in second language listening comprehension. Language Learning, 55, 661–699. doi:  10.1111/j.0023‑8333.2005.00320.x
     https://doi.org/10.1111/j.0023-8333.2005.00320.x [Google Scholar]
  20. Sweller, John
    (2005) The redundancy principle in multimedia learning. InRichard E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp.159–168). Cambridge University Press: Cambridge, UK. 10.1017/CBO9780511816819.011
     https://doi.org/10.1017/CBO9780511816819.011 [Google Scholar]
  21. Wagner, Susan M., Howard Nusbaum, & Susan Goldin-Meadow
    (2004) Probing the mental representation of gesture: Is hand-waving spatial?Journal of Memory and Language, 50, 395–407. doi:  10.1016/j.jml.2004.01.002
     https://doi.org/10.1016/j.jml.2004.01.002 [Google Scholar]
  22. Wickelgren, Wayne A.
    (1965) Acoustic similarity and intrusion errors in short-term memory. Journal of Experimental Psychology, 70, 102–108. doi:  10.1037/h0022015
     https://doi.org/10.1037/h0022015 [Google Scholar]
  23. Wu, Ying Choon & Seana Coulson
    (2014) Co-speech iconic gestures and visuo-spatial working memory. Acta Psychologica, 153, 39–50. doi:  10.1016/j.actpsy.2014.09.002
     https://doi.org/10.1016/j.actpsy.2014.09.002 [Google Scholar]
/content/journals/10.1075/gest.17017.hos
Loading
Does seeing gesture lighten or increase the load?
Gesture 17, 268 (2018); https://doi.org/10.1075/gest.17017.hos
/content/journals/10.1075/gest.17017.hos
/content/journals/10.1075/gest.17017.hos
Loading

Data & Media loading...

  • Article Type: Research Article
Keyword(s): cognitive load; gesture comprehension; horizontal spatial relations; verbal resources; visuospatial resources

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error