1887
  1. Home
  2. e-Journals
  3. Gesture
  4. Volume 20, Issue 3
  5. Article
Volume 20, Issue 3
  • ISSN 1568-1475
  • E-ISSN: 1569-9773
USD
Buy:$35.00 + Taxes

Abstract

Abstract

While previous studies have shown the importance of visuo-spatial working memory in the processing of co-speech iconic gestures, clear evidence for a potential involvement of the verbal working memory (vWM) is currently lacking. To address this issue, participants in the present study were presented with a dual task paradigm. The main outcome variable was the performance on a Stroop-like gesture task which provides a behavioural index of gesture-speech integration. Participants performed this task under conditions of either high or low concurrent vWM load. Unlike in previous studies, the number of words to remember in the high load condition was determined by their individual verbal span rather than being fixed. Results showed reaction time costs in the form of longer reaction times for semantically incongruent gesture-speech combinations as compared to congruent combinations. However, this semantic congruency effect disappeared when the vWM load increased. This result suggests a causal involvement of verbal working memory capacity in gesture-speech integration.

© John Benjamins Publishing Company
Loading

Article metrics loading...

/content/journals/10.1075/gest.20028.kan
2022-08-04
2024-09-12

  • From This Site

    /content/journals/10.1075/gest.20028.kan
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Alibali, Martha W.
    (2005) Gesture in spatial cognition: Expressing, communicating, and thinking about spatial information. Spatial Cognition and Computation, 5 (4), 307–331. 10.1207/s15427633scc0504_2
     https://doi.org/10.1207/s15427633scc0504_2 [Google Scholar]
  2. Alibali, Martha. W., Flevares, Lucia M., & Goldin-Meadow, Susan
    (1997) Assessing knowledge conveyed in gesture: Do teachers have the upper hand?Journal of Educational Psychology, 89 (1), 183–193. 10.1037/0022‑0663.89.1.183
     https://doi.org/10.1037/0022-0663.89.1.183 [Google Scholar]
  3. Baddeley, Alan D.
    (1992) Working memory. Science, 225(5044), 556–559. 10.1126/science.1736359
     https://doi.org/10.1126/science.1736359 [Google Scholar]
  4. (2012) Working memory: Theories, Models and Controversies. Annual Reviews of Psychology, 631, 1–29. 10.1146/annurev‑psych‑120710‑100422
     https://doi.org/10.1146/annurev-psych-120710-100422 [Google Scholar]
  5. Baddeley, Alan D., & Hitch, Graham J.
    (1974) Working memory. InGordon H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol.81, pp. 47–89). London: Academic Press.
     [Google Scholar]
  6. Beattie, Geoffrey & Shovelton, Heather
    (2002) What properties of talk are associated with the generation of spontaneous hand gestures?British Journal of Social Psychology, 411, 403–417. 10.1348/014466602760344287
     https://doi.org/10.1348/014466602760344287 [Google Scholar]
  7. Besner, Derek & Stolz, Jennifer A.
    (1999) Unconsciously controlled processing: The Stroop effect reconsidered. Psychonomic Bulletin & Review, 6 (3), 449–455. 10.3758/BF03210834
     https://doi.org/10.3758/BF03210834 [Google Scholar]
  8. Boehringer, Andreas, Macher, Katja, Dukart, Juergen, Villringer, Arno, & Pleger, Burkhard
    (2013) Cerebellar transcranial direct stimulation modulates verbal working memory. Brain Stimulation, 61, 649–653. 10.1016/j.brs.2012.10.001
     https://doi.org/10.1016/j.brs.2012.10.001 [Google Scholar]
  9. Chu, Minguyan, Meyer, Antje, Foulkes, Lucy, & Kita, Sotaro
    (2014) Individual differences in frequency and saliency of speech-accompanying gestures: The role of cognitive abilities and empathy. Journal of Experimental Psychology: General, 143 (2), 694–709. 10.1037/a0033861
     https://doi.org/10.1037/a0033861 [Google Scholar]
  10. Daneman, Meredith & Carpenter, Patricia A.
    (1980) Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, 191, 450–466. 10.1016/S0022‑5371(80)90312‑6
     https://doi.org/10.1016/S0022-5371(80)90312-6 [Google Scholar]
  11. Delaloye, Christophe, Ludwig, Catherine, Borella, Erika, Chicherio, Christian, & de Ribaupierre, Anik
    (2008) The reading span as a measure of working memory capacity: Norms based on a French speaking population of 775 younger and older adults. Revue européenne de Psychologie Appliquée, 581, 89–103. 10.1016/j.erap.2006.12.004
     https://doi.org/10.1016/j.erap.2006.12.004 [Google Scholar]
  12. Dick, Anthony, Goldin-Meadow, Susan, Hasson, Uri, Skipper, Jeremy, & Small, Steven L.
    (2009) Co-speech gestures influence neural activity in brain regions associated with processing semantic information. Human Brain Mapping, 301, 3509–3526. 10.1002/hbm.20774
     https://doi.org/10.1002/hbm.20774 [Google Scholar]
  13. Engel, Randall W.
    (2002) Working memory capacity as executive attention. Current Directions in Psychological Science, 11 (1), 19–23. 10.1111/1467‑8721.00160
     https://doi.org/10.1111/1467-8721.00160 [Google Scholar]
  14. Feyereisen, Pierre & Havard, Isabelle
    (1999) Mental imagery and production of hand gestures while speaking in younger and older adults. Journal of Non-Verbal Behavior, 231, 153–171. 10.1023/A:1021487510204
     https://doi.org/10.1023/A:1021487510204 [Google Scholar]
  15. Gauthier, Janel & Bouchard, Stéphane
    (1993) A French-Canadian adaptation of the revised version of Spielberger’s State-Trait Anxiety Inventory. Canadian Journal of Behavioural Science, 25 (4), 559–578. 10.1037/h0078881
     https://doi.org/10.1037/h0078881 [Google Scholar]
  16. Gillepsie, Maureen, James, Ariel N., Federmeier, Kara D., & Watson, Duane G.
    (2014) Verbal working memory predicts co-speech gesture: Evidence from individual differences. Cognition, 1321, 174–180. 10.1016/j.cognition.2014.03.012
     https://doi.org/10.1016/j.cognition.2014.03.012 [Google Scholar]
  17. Green, Antonia, Straube, Benjamin, Weis, Susanne, Jansen, Andreas, Wilmes, Klaus, Konrad, Kerstin, & Kircher, Tilo
    (2009) Neural integration of iconic and unrelated co-verbal gestures: A functional MRI study. Human Brain Mapping, 30 (10), 3309–3324. 10.1002/hbm.20753
     https://doi.org/10.1002/hbm.20753 [Google Scholar]
  18. Habets, Boukje, Kita, Sotaro, Shao, Zeshu, Özyürek, Aslı, & Hagoort, Peter
    (2010) The role of synchrony and ambiguity in speech-gesture integration during comprehension. Journal of Cognitive Neuroscience, 23 (8), 1845–1854. 10.1162/jocn.2010.21462
     https://doi.org/10.1162/jocn.2010.21462 [Google Scholar]
  19. Hadar, Uri & Butterworth, Brian
    (1997) Iconic gestures, imagery and word retrieval in speech. Semiotica, 115 (1/2), 147–172. 10.1515/semi.1997.115.1‑2.147
     https://doi.org/10.1515/semi.1997.115.1-2.147 [Google Scholar]
  20. Hadar, Uri & Pinchas-Zamir, Lian
    (2004) The semantic specificity of gesture. Implications for gesture classification and function. Journal of Language and Social Psychology, 23 (2), 204–214. 10.1177/0261927X04263825
     https://doi.org/10.1177/0261927X04263825 [Google Scholar]
  21. Hadwin, Julie, Brogan, Joanna, & Stevenson, Jim
    (2005) State anxiety and working memory in children: A test of processing efficiency theory. Educational Psychology, 25 (4), 379–393. 10.1080/01443410500041607
     https://doi.org/10.1080/01443410500041607 [Google Scholar]
  22. Holle, Henning & Gunter, Thomas C.
    (2007) The role of iconic gestures in speech disambiguation: ERP evidence. Journal of Cognitive Neuroscience, 19 (7), 1175–1192. 10.1162/jocn.2007.19.7.1175
     https://doi.org/10.1162/jocn.2007.19.7.1175 [Google Scholar]
  23. Holle, Henning, Gunter, Thomas C., Rueschemeyer, Shirley, Hennenlotter, Andreas, & Iacoboni, Marco
    (2008) Neural correlates of the processing of co-speech gestures. NeuroImage, 39 (4), 2010–2024. 10.1016/j.neuroimage.2007.10.055
     https://doi.org/10.1016/j.neuroimage.2007.10.055 [Google Scholar]
  24. Holler, Judith, Shovelton, Heather, & Beattie, Geoffrey
    (2009) Do iconic hand gestures really contribute to the communication of semantic information in face-to-face context?Journal of Non-Verbal Behavior, 33 (2), 73–88. 10.1007/s10919‑008‑0063‑9
     https://doi.org/10.1007/s10919-008-0063-9 [Google Scholar]
  25. Hostetter, Autumn B.
    (2011) When do gestures communicate? A meta-analysis. Psychological Bulletin, 137 (2), 297–315. 10.1037/a0022128
     https://doi.org/10.1037/a0022128 [Google Scholar]
  26. Kelly, Spencer D., Barr, Dale J., Church, R. Breckinridge, & Lynch, Katheryn
    (1999) Offering a hand to pragmatic understanding: The role of speech and gesture in comprehension and memory. Journal of Memory and Langage, 401, 577–592. 10.1006/jmla.1999.2634
     https://doi.org/10.1006/jmla.1999.2634 [Google Scholar]
  27. Kelly, Spencer D., Creigh, Peter, & Bartolotti, James
    (2009) Integrating speech and iconic gestures in a stroop-like task: Evidence for automatic processing. Journal of Cognitive Neuroscience, 22 (4), 683–694. 10.1162/jocn.2009.21254
     https://doi.org/10.1162/jocn.2009.21254 [Google Scholar]
  28. Kelly, Spencer, Kravitz, Corinne, & Hopkins, Michael
    (2004) Neural correlates of bimodal speech and gesture comprehension. Brain and Language, 89 (1), 253–260. 10.1016/S0093‑934X(03)00335‑3
     https://doi.org/10.1016/S0093-934X(03)00335-3 [Google Scholar]
  29. Kelly, Spencer, Özyürek, Aslı, & Maris, Eric
    (2010) Two sides of the same coin: Speech and gesture mutually interact to enhance communication. Psychological Science, 21 (2), 260–267. 10.1177/0956797609357327
     https://doi.org/10.1177/0956797609357327 [Google Scholar]
  30. Kelly, Spencer, Ward, Sarah, Creigh, Peter, & Bartolotti, James
    (2007) An intentional stance modulates the integration of gesture and speech during comprehension. Brain and Language, 101 (3), 222–233. 10.1016/j.bandl.2006.07.008
     https://doi.org/10.1016/j.bandl.2006.07.008 [Google Scholar]
  31. Kita, Sotaro
    (2000) How representational gestures help speaking. InDavid McNeill (Ed.), Language and gesture (pp. 162–185). Cambridge, UK: Cambridge University Press. 10.1017/CBO9780511620850.011
     https://doi.org/10.1017/CBO9780511620850.011 [Google Scholar]
  32. Kita, Sotaro & Özyürek, Aslı
    (2003) What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language, 48 (1), 16–32. 10.1016/S0749‑596X(02)00505‑3
     https://doi.org/10.1016/S0749-596X(02)00505-3 [Google Scholar]
  33. Krauss, Robert M., Chen, Yihsiu, & Chawla, Purnima
    (1996) Non-verbal behavior and non-verbal communication: What do conversational hand gesture tell us?InM. Zanna (Ed.), Advances in experimental social psychology (pp. 389–450). San Diego: Academic Press.
     [Google Scholar]
  34. Krauss, Robert M., Morrel-Samuels, Palmer, & Colasante, Christina
    (1991) Do conversational hand gestures communicate?Journal of Personality and Social Psychology, 61 (5), 743–754. 10.1037/0022‑3514.61.5.743
     https://doi.org/10.1037/0022-3514.61.5.743 [Google Scholar]
  35. McNeill, David
    (1985) So you think gestures are non-verbal?Psychological Review, 92 (3), 350–371. 10.1037/0033‑295X.92.3.350
     https://doi.org/10.1037/0033-295X.92.3.350 [Google Scholar]
  36. (1992) Hand and mind. Chicago: University of Chicago Press.
     [Google Scholar]
  37. McNeill, David, Cassell, Justine, & McCullough, Karl-Erik
    (1994) Communicative effects of speech-mismatched gestures. Research on Language and Social Interaction, 27 (3), 223–237. 10.1207/s15327973rlsi2703_4
     https://doi.org/10.1207/s15327973rlsi2703_4 [Google Scholar]
  38. Momsen, Jacob, Gordon, Jared, Wu, Ying Choon, & Coulson, Seana
    (2020) Verbal working memory and co- speech gesture processing. Brain and Cognition, 1461, 105640. 10.1016/j.bandc.2020.105640
     https://doi.org/10.1016/j.bandc.2020.105640 [Google Scholar]
  39. Morrel-Samuels, Palmer & Krauss, Robert M.
    (1992) Word familiarity predicts asynchrony of hand gestures and speech. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18 (3), 615–622.
     [Google Scholar]
  40. Navon, David & Miller, Jeff
    (2002) Queuing or sharing? A critical evaluation of the single- bottleneck notion. Cognitive Psychology, 441, 193–251. 10.1006/cogp.2001.0767
     https://doi.org/10.1006/cogp.2001.0767 [Google Scholar]
  41. Obermeier, Christian, Holle, Henning, & Gunter, Thomas
    (2011) What iconic gesture fragments reveal about gesture-speech integration: When synchrony is lost, memory can help. Journal of Cognitive Neuroscience, 23 (7), 1648–1663. 10.1162/jocn.2010.21498
     https://doi.org/10.1162/jocn.2010.21498 [Google Scholar]
  42. Ovroye, Acacia L. & Storm, Benjamin C.
    (2019) Remembering what was said and done: The activation and facilitation of memory for gesture as a consequence of retrieval. Journal of Experimental Psychology: Learning, Memory and Cognition, 45 (3), 526–534.
     [Google Scholar]
  43. Owens, Matthews, Stevenson, Jim, Hadwin, Julie A., & Norgate, Roger
    (2014) When does anxiety help or hinder test performance? The role of working memory capacity. British Journal of Psychology, 105 (1), 92–101. 10.1111/bjop.12009
     https://doi.org/10.1111/bjop.12009 [Google Scholar]
  44. Özer, Demet & Göksun, Tilbe
    (2020a) Gesture use and processing: A review on individual differences in cognitive resources. Frontiers in Psychology, 111, 573555. 10.3389/fpsyg.2020.573555
     https://doi.org/10.3389/fpsyg.2020.573555 [Google Scholar]
  45. (2020b) Visual-spatial and verbal abilities differentially affect processing of gestural vs. spoken expressions. Language, Cognition and Neuroscience, 35 (7), 896–914. 10.1080/23273798.2019.1703016
     https://doi.org/10.1080/23273798.2019.1703016 [Google Scholar]
  46. Özyürek, Aslı, Willems, Roel M., Kita, Sotaro, & Hagoort, Peter
    (2007) On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials. Journal of Cognitive Neuroscience, 191, 605–616. 10.1162/jocn.2007.19.4.605
     https://doi.org/10.1162/jocn.2007.19.4.605 [Google Scholar]
  47. Smyth, Mary M. & Pendleton, Lindsay R.
    (1990) Space and movement in working memory. The Quarterly Journal of Experimental Psychology, Section A: Human Experimental Psychology, 42 (2), 291–304. 10.1080/14640749008401223
     https://doi.org/10.1080/14640749008401223 [Google Scholar]
  48. Vytal, Katherine E., Cornwell, Brian R., Arkin, Nicole Esther, Letkiewicz, Allison M., & Grillon, Christian
    (2013) The complex interaction between anxiety and cognition: insight from spatial and verbal working memory. Frontiers in Human Neuroscience, 71, 93. 10.3389/fnhum.2013.00093
     https://doi.org/10.3389/fnhum.2013.00093 [Google Scholar]
  49. Wagner, Susan M., Nusbaum, Howard, & Goldin-Meadow, Susan
    (2004) Probing the mental representation of gesture: Is handwaving spatial?Journal of Memory and Language, 501, 395–407. 10.1016/j.jml.2004.01.002
     https://doi.org/10.1016/j.jml.2004.01.002 [Google Scholar]
  50. Wechsler, David
    (2008) Wechsler adult intelligence scale – fourth edition.
     [Google Scholar]
  51. Wolf, Dhana, Rekittke, Linn-Marlen, Mittelberg, Irene, Klasen, Martin, & Mathiak, Klaus
    (2017) Perceived conventionality in co-speech gestures involves the fronto-temporal language network. Frontiers in Human Neuroscience, 111, 573. 10.3389/fnhum.2017.00573
     https://doi.org/10.3389/fnhum.2017.00573 [Google Scholar]
  52. Wu, Ying Choon & Coulson, Seana
    (2007) How iconic gestures enhance communication: An ERP study. Brain & Language, 1011, 234–245. 10.1016/j.bandl.2006.12.003
     https://doi.org/10.1016/j.bandl.2006.12.003 [Google Scholar]
  53. (2011) Are depictive gestures like pictures? Commonalities and differences in semantic processing. Brain & Language, 1191, 184–195. 10.1016/j.bandl.2011.07.002
     https://doi.org/10.1016/j.bandl.2011.07.002 [Google Scholar]
  54. (2014) Co-speech iconic gestures and visuo-spatial working memory. Acta Psychologia, 1531, 39–50. 10.1016/j.actpsy.2014.09.002
     https://doi.org/10.1016/j.actpsy.2014.09.002 [Google Scholar]
  55. (2015) Iconic gestures facilitate discourse comprehension in individuals with superior immediate memory for body configurations. Psychological Science, 26 (11), 1717–1727. 10.1177/0956797615597671
     https://doi.org/10.1177/0956797615597671 [Google Scholar]
  56. Zhao, Wanying, Riggs, Kevin, Schindler, Igor, & Holle, Henning
    (2018) Transcranial Magnetic Stimulation over left inferior frontal and posterior temporal cortex disrupts gesture-speech integration. The Journal of Neuroscience, 38 (8), 1891–1900. 10.1523/JNEUROSCI.1748‑17.2017
     https://doi.org/10.1523/JNEUROSCI.1748-17.2017 [Google Scholar]
/content/journals/10.1075/gest.20028.kan
Loading
High verbal working memory load impairs gesture-speech integration
Gesture 20, 354 (2021); https://doi.org/10.1075/gest.20028.kan
/content/journals/10.1075/gest.20028.kan
/content/journals/10.1075/gest.20028.kan
Loading

Data & Media loading...

  • Article Type: Research Article
Keyword(s): gesture-speech comprehension; gesture-speech integration; iconic gestures; verbal resources hypothesis; verbal working memory

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error