1887
Volume 17, Issue 2
  • ISSN 1568-1475
  • E-ISSN: 1569-9773
USD
Buy:$35.00 + Taxes

Abstract

Abstract

We examined the cognitive resources involved in processing speech with gesture compared to the same speech without gesture across four studies using a dual-task paradigm. Participants viewed videos of a woman describing spatial arrays either with gesture or without. They then attempted to choose the target array from among four choices. Participants’ cognitive load was measured as they completed this comprehension task by measuring how well they could remember the location and identity of digits in a secondary task. We found that addressees experience additional visuospatial load when processing gestures compared to speech alone, and that the load primarily comes when addressees attempt to use their memory of the descriptions with gesture to choose the target array. However, this cost only occurs when gestures about horizontal spatial relations (i.e., left and right) are produced from the speaker’s egocentric perspective.

Loading

Article metrics loading...

/content/journals/10.1075/gest.17017.hos
2019-06-26
2019-10-18
Loading full text...

Full text loading...

References

  1. Alibali, Martha W.
    (2005) Gesture in spatial cognition: Expressing, communicating, and thinking about spatial information. Spatial Cognition & Computation, 5, 307–331. doi:  10.1207/s15427633scc0504_2
    https://doi.org/10.1207/s15427633scc0504_2 [Google Scholar]
  2. Baddeley, Alan
    (1992) Working memory. Science, 255, 556–559. 10.1126/science.1736359
    https://doi.org/10.1126/science.1736359 [Google Scholar]
  3. Goldin-Meadow, Susan, Howard Nusbaum, Spencer D. Kelly, & Susan Wagner
    (2001) Explaining math: Gesturing lightens the load. Psychological Science, 12, 516–522. doi:  10.1111/1467‑9280.00395
    https://doi.org/10.1111/1467-9280.00395 [Google Scholar]
  4. Holler, Judith, Heather Shovelton, & Geoffrey Beattie
    (2009) Do iconic hand gestures really contribute to the communication of semantic information in a face-to-face context?Journal of Nonverbal Behavior, 33, 73–88. doi: 10.1007/s10910‑008‑0063‑9
    https://doi.org/10.1007/s10910-008-0063-9 [Google Scholar]
  5. Hostetter, Autumn B.
    (2011) When do gestures communicate? A meta-analysis. Psychological Bulletin, 137, 297–315. doi:  10.1037/a0022128
    https://doi.org/10.1037/a0022128 [Google Scholar]
  6. Kelly, Spencer D., Peter Creigh, & James Bartolotti
    (2009) Integrating speech and iconic gestures in a Stroop-like task: Evidence for automatic processing. Journal of Cognitive Neuroscience, 22, 683–694. 10.1162/jocn.2009.21254
    https://doi.org/10.1162/jocn.2009.21254 [Google Scholar]
  7. Kelly, Spencer D. & Leslie H. Goldsmith
    (2004) Gesture and right hemisphere involvement in evaluating lecture material. Gesture, 4, 25–42. 10.1075/gest.4.1.03kel
    https://doi.org/10.1075/gest.4.1.03kel [Google Scholar]
  8. Kelly, Spencer, Meghan Healey, Aslı Özyürek, & Judith Holler
    (2015) The processing of speech, gesture, and action during language comprehension. Psychonomic Bulletin & Review, 22, 517–523. doi:  10.3758/s13423‑014‑0681‑7
    https://doi.org/10.3758/s13423-014-0681-7 [Google Scholar]
  9. Kelly, Spencer D., Corinne Kravitz, & Michael Hopkins
    (2004) Neural correlates of bimodal speech and gesture comprehension. Brain and Language, 89, 253–260. doi:  10.1016/S0093‑934X(03)00335‑3
    https://doi.org/10.1016/S0093-934X(03)00335-3 [Google Scholar]
  10. Kelly, Spencer D., Aslı Özyürek, & Eric Maris
    (2010) Two sides of the same coin: Speech and gesture mutually interact to enhance comprehension. Psychological Science, 21, 260–267. 10.1177/0956797609357327
    https://doi.org/10.1177/0956797609357327 [Google Scholar]
  11. Kelly, Spencer D., Sarah Ward, Peter Creigh, & James Bartolotti
    (2007) An intentional stance modulates the integration of gesture and speech during comprehension. Brain and Language, 101, 222–233. doi:  10.1016/j.bandl.2006.07.008
    https://doi.org/10.1016/j.bandl.2006.07.008 [Google Scholar]
  12. Kendon, Adam
    (1994) Do gestures communicate? A review. Research on Language and Social Interaction, 27, 175–200. doi:  10.1207/s15327973rlsi2703_2
    https://doi.org/10.1207/s15327973rlsi2703_2 [Google Scholar]
  13. Lakens, Daniël
    (2013) Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863. doi:  10.3389/fpsyg.2013.00863
    https://doi.org/10.3389/fpsyg.2013.00863 [Google Scholar]
  14. Maricchiolo, Fridanna, Augusto Gnisci, Marino Bonaiuto, & Gianluca Ficca
    (2009) Effects of different types of hand gestures in persuasive speech on receivers’ evaluations. Language and Cognitive Processes, 24, 239–266. doi: 10.1080/0169096080215992
    https://doi.org/10.1080/0169096080215992 [Google Scholar]
  15. Paivio, Allan
    (1991) Dual coding theory: Retrospect and current status. Canadian Journal of Psychology, 45, 255–287. 10.1037/h0084295
    https://doi.org/10.1037/h0084295 [Google Scholar]
  16. Post, Lysanne S., Tamara van Gog, Fred Paas, & Rolf A. Zwaan
    (2013) Effects of simultaneously observing and making gestures while studying grammar animations on cognitive load and learning. Computers in Human Behavior, 29, 1450–1455. 10.1016/j.chb.2013.01.005
    https://doi.org/10.1016/j.chb.2013.01.005 [Google Scholar]
  17. Pyers, Jennie E., Pamela Perniss, & Karen Emmorey
    (2015) Viewpoint in the visual-spatial modality: The coordination of spatial perspective. Spatial Cognition & Computation: An Interdisciplinary Journal, 15, 143–169. doi:  10.1080/13875868.2014.1003933
    https://doi.org/10.1080/13875868.2014.1003933 [Google Scholar]
  18. Rueckert, Linda, Ruth Breckinridge Church, Andrea Avila, & Theresa Trejo
    (2017) Gesture enhances learning of a complex statistical concept. Cognitive Research, 2, 1–6. doi:  10.1186/s41235‑016‑0036‑1
    https://doi.org/10.1186/s41235-016-0036-1 [Google Scholar]
  19. Sueyoshi, Ayano & Debra M. Hardison
    (2005) The role of gestures and facial cues in second language listening comprehension. Language Learning, 55, 661–699. doi:  10.1111/j.0023‑8333.2005.00320.x
    https://doi.org/10.1111/j.0023-8333.2005.00320.x [Google Scholar]
  20. Sweller, John
    (2005) The redundancy principle in multimedia learning. InRichard E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp.159–168). Cambridge University Press: Cambridge, UK. 10.1017/CBO9780511816819.011
    https://doi.org/10.1017/CBO9780511816819.011 [Google Scholar]
  21. Wagner, Susan M., Howard Nusbaum, & Susan Goldin-Meadow
    (2004) Probing the mental representation of gesture: Is hand-waving spatial?Journal of Memory and Language, 50, 395–407. doi:  10.1016/j.jml.2004.01.002
    https://doi.org/10.1016/j.jml.2004.01.002 [Google Scholar]
  22. Wickelgren, Wayne A.
    (1965) Acoustic similarity and intrusion errors in short-term memory. Journal of Experimental Psychology, 70, 102–108. doi:  10.1037/h0022015
    https://doi.org/10.1037/h0022015 [Google Scholar]
  23. Wu, Ying Choon & Seana Coulson
    (2014) Co-speech iconic gestures and visuo-spatial working memory. Acta Psychologica, 153, 39–50. doi:  10.1016/j.actpsy.2014.09.002
    https://doi.org/10.1016/j.actpsy.2014.09.002 [Google Scholar]
http://instance.metastore.ingenta.com/content/journals/10.1075/gest.17017.hos
Loading
/content/journals/10.1075/gest.17017.hos
Loading

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error