1887
image of High verbal working memory load impairs gesture-speech integration
USD
Buy:$35.00 + Taxes

Abstract

Abstract

While previous studies have shown the importance of visuo-spatial working memory in the processing of co-speech iconic gestures, clear evidence for a potential involvement of the verbal working memory (vWM) is currently lacking. To address this issue, participants in the present study were presented with a dual task paradigm. The main outcome variable was the performance on a Stroop-like gesture task which provides a behavioural index of gesture-speech integration. Participants performed this task under conditions of either high or low concurrent vWM load. Unlike in previous studies, the number of words to remember in the high load condition was determined by their individual verbal span rather than being fixed. Results showed reaction time costs in the form of longer reaction times for semantically incongruent gesture-speech combinations as compared to congruent combinations. However, this semantic congruency effect disappeared when the vWM load increased. This result suggests a causal involvement of verbal working memory capacity in gesture-speech integration.

Loading

Article metrics loading...

/content/journals/10.1075/gest.20028.kan
2022-08-04
2022-08-12
Loading full text...

Full text loading...

References

  1. Alibali, Martha W.
    (2005) Gesture in spatial cognition: Expressing, communicating, and thinking about spatial information. Spatial Cognition and Computation, 5 (4), 307–331. 10.1207/s15427633scc0504_2
    https://doi.org/10.1207/s15427633scc0504_2 [Google Scholar]
  2. Alibali, Martha. W., Flevares, Lucia M., & Goldin-Meadow, Susan
    (1997) Assessing knowledge conveyed in gesture: Do teachers have the upper hand?Journal of Educational Psychology, 89 (1), 183–193. 10.1037/0022‑0663.89.1.183
    https://doi.org/10.1037/0022-0663.89.1.183 [Google Scholar]
  3. Baddeley, Alan D.
    (1992) Working memory. Science, 225(5044), 556–559. 10.1126/science.1736359
    https://doi.org/10.1126/science.1736359 [Google Scholar]
  4. (2012) Working memory: Theories, Models and Controversies. Annual Reviews of Psychology, 63, 1–29. 10.1146/annurev‑psych‑120710‑100422
    https://doi.org/10.1146/annurev-psych-120710-100422 [Google Scholar]
  5. Baddeley, Alan D., & Hitch, Graham J.
    (1974) Working memory. InGordon H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol.8, pp. 47–89). London: Academic Press.
    [Google Scholar]
  6. Beattie, Geoffrey & Shovelton, Heather
    (2002) What properties of talk are associated with the generation of spontaneous hand gestures?British Journal of Social Psychology, 41, 403–417. 10.1348/014466602760344287
    https://doi.org/10.1348/014466602760344287 [Google Scholar]
  7. Besner, Derek & Stolz, Jennifer A.
    (1999) Unconsciously controlled processing: The Stroop effect reconsidered. Psychonomic Bulletin & Review, 6 (3), 449–455. 10.3758/BF03210834
    https://doi.org/10.3758/BF03210834 [Google Scholar]
  8. Boehringer, Andreas, Macher, Katja, Dukart, Juergen, Villringer, Arno, & Pleger, Burkhard
    (2013) Cerebellar transcranial direct stimulation modulates verbal working memory. Brain Stimulation, 6, 649–653. 10.1016/j.brs.2012.10.001
    https://doi.org/10.1016/j.brs.2012.10.001 [Google Scholar]
  9. Chu, Minguyan, Meyer, Antje, Foulkes, Lucy, & Kita, Sotaro
    (2014) Individual differences in frequency and saliency of speech-accompanying gestures: The role of cognitive abilities and empathy. Journal of Experimental Psychology: General, 143 (2), 694–709. 10.1037/a0033861
    https://doi.org/10.1037/a0033861 [Google Scholar]
  10. Daneman, Meredith & Carpenter, Patricia A.
    (1980) Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, 19, 450–466. 10.1016/S0022‑5371(80)90312‑6
    https://doi.org/10.1016/S0022-5371(80)90312-6 [Google Scholar]
  11. Delaloye, Christophe, Ludwig, Catherine, Borella, Erika, Chicherio, Christian, & de Ribaupierre, Anik
    (2008) The reading span as a measure of working memory capacity: Norms based on a French speaking population of 775 younger and older adults. Revue européenne de Psychologie Appliquée, 58, 89–103. 10.1016/j.erap.2006.12.004
    https://doi.org/10.1016/j.erap.2006.12.004 [Google Scholar]
  12. Dick, Anthony, Goldin-Meadow, Susan, Hasson, Uri, Skipper, Jeremy, & Small, Steven L.
    (2009) Co-speech gestures influence neural activity in brain regions associated with processing semantic information. Human Brain Mapping, 30, 3509–3526. 10.1002/hbm.20774
    https://doi.org/10.1002/hbm.20774 [Google Scholar]
  13. Engel, Randall W.
    (2002) Working memory capacity as executive attention. Current Directions in Psychological Science, 11 (1), 19–23. 10.1111/1467‑8721.00160
    https://doi.org/10.1111/1467-8721.00160 [Google Scholar]
  14. Feyereisen, Pierre & Havard, Isabelle
    (1999) Mental imagery and production of hand gestures while speaking in younger and older adults. Journal of Non-Verbal Behavior, 23, 153–171. 10.1023/A:1021487510204
    https://doi.org/10.1023/A:1021487510204 [Google Scholar]
  15. Gauthier, Janel & Bouchard, Stéphane
    (1993) A French-Canadian adaptation of the revised version of Spielberger’s State-Trait Anxiety Inventory. Canadian Journal of Behavioural Science, 25 (4), 559–578. 10.1037/h0078881
    https://doi.org/10.1037/h0078881 [Google Scholar]
  16. Gillepsie, Maureen, James, Ariel N., Federmeier, Kara D., & Watson, Duane G.
    (2014) Verbal working memory predicts co-speech gesture: Evidence from individual differences. Cognition, 132, 174–180. 10.1016/j.cognition.2014.03.012
    https://doi.org/10.1016/j.cognition.2014.03.012 [Google Scholar]
  17. Green, Antonia, Straube, Benjamin, Weis, Susanne, Jansen, Andreas, Wilmes, Klaus, Konrad, Kerstin, & Kircher, Tilo
    (2009) Neural integration of iconic and unrelated co-verbal gestures: A functional MRI study. Human Brain Mapping, 30 (10), 3309–3324. 10.1002/hbm.20753
    https://doi.org/10.1002/hbm.20753 [Google Scholar]
  18. Habets, Boukje, Kita, Sotaro, Shao, Zeshu, Özyürek, Aslı, & Hagoort, Peter
    (2010) The role of synchrony and ambiguity in speech-gesture integration during comprehension. Journal of Cognitive Neuroscience, 23 (8), 1845–1854. 10.1162/jocn.2010.21462
    https://doi.org/10.1162/jocn.2010.21462 [Google Scholar]
  19. Hadar, Uri & Butterworth, Brian
    (1997) Iconic gestures, imagery and word retrieval in speech. Semiotica, 115 (1/2), 147–172. 10.1515/semi.1997.115.1‑2.147
    https://doi.org/10.1515/semi.1997.115.1-2.147 [Google Scholar]
  20. Hadar, Uri & Pinchas-Zamir, Lian
    (2004) The semantic specificity of gesture. Implications for gesture classification and function. Journal of Language and Social Psychology, 23 (2), 204–214. 10.1177/0261927X04263825
    https://doi.org/10.1177/0261927X04263825 [Google Scholar]
  21. Hadwin, Julie, Brogan, Joanna, & Stevenson, Jim
    (2005) State anxiety and working memory in children: A test of processing efficiency theory. Educational Psychology, 25 (4), 379–393. 10.1080/01443410500041607
    https://doi.org/10.1080/01443410500041607 [Google Scholar]
  22. Holle, Henning & Gunter, Thomas C.
    (2007) The role of iconic gestures in speech disambiguation: ERP evidence. Journal of Cognitive Neuroscience, 19 (7), 1175–1192. 10.1162/jocn.2007.19.7.1175
    https://doi.org/10.1162/jocn.2007.19.7.1175 [Google Scholar]
  23. Holle, Henning, Gunter, Thomas C., Rueschemeyer, Shirley, Hennenlotter, Andreas, & Iacoboni, Marco
    (2008) Neural correlates of the processing of co-speech gestures. NeuroImage, 39 (4), 2010–2024. 10.1016/j.neuroimage.2007.10.055
    https://doi.org/10.1016/j.neuroimage.2007.10.055 [Google Scholar]
  24. Holler, Judith, Shovelton, Heather, & Beattie, Geoffrey
    (2009) Do iconic hand gestures really contribute to the communication of semantic information in face-to-face context?Journal of Non-Verbal Behavior, 33 (2), 73–88. 10.1007/s10919‑008‑0063‑9
    https://doi.org/10.1007/s10919-008-0063-9 [Google Scholar]
  25. Hostetter, Autumn B.
    (2011) When do gestures communicate? A meta-analysis. Psychological Bulletin, 137 (2), 297–315. 10.1037/a0022128
    https://doi.org/10.1037/a0022128 [Google Scholar]
  26. Kelly, Spencer D., Barr, Dale J., Church, R. Breckinridge, & Lynch, Katheryn
    (1999) Offering a hand to pragmatic understanding: The role of speech and gesture in comprehension and memory. Journal of Memory and Langage, 40, 577–592. 10.1006/jmla.1999.2634
    https://doi.org/10.1006/jmla.1999.2634 [Google Scholar]
  27. Kelly, Spencer D., Creigh, Peter, & Bartolotti, James
    (2009) Integrating speech and iconic gestures in a stroop-like task: Evidence for automatic processing. Journal of Cognitive Neuroscience, 22 (4), 683–694. 10.1162/jocn.2009.21254
    https://doi.org/10.1162/jocn.2009.21254 [Google Scholar]
  28. Kelly, Spencer, Kravitz, Corinne, & Hopkins, Michael
    (2004) Neural correlates of bimodal speech and gesture comprehension. Brain and Language, 89 (1), 253–260. 10.1016/S0093‑934X(03)00335‑3
    https://doi.org/10.1016/S0093-934X(03)00335-3 [Google Scholar]
  29. Kelly, Spencer, Özyürek, Aslı, & Maris, Eric
    (2010) Two sides of the same coin: Speech and gesture mutually interact to enhance communication. Psychological Science, 21 (2), 260–267. 10.1177/0956797609357327
    https://doi.org/10.1177/0956797609357327 [Google Scholar]
  30. Kelly, Spencer, Ward, Sarah, Creigh, Peter, & Bartolotti, James
    (2007) An intentional stance modulates the integration of gesture and speech during comprehension. Brain and Language, 101 (3), 222–233. 10.1016/j.bandl.2006.07.008
    https://doi.org/10.1016/j.bandl.2006.07.008 [Google Scholar]
  31. Kita, Sotaro
    (2000) How representational gestures help speaking. InDavid McNeill (Ed.), Language and gesture (pp. 162–185). Cambridge, UK: Cambridge University Press. 10.1017/CBO9780511620850.011
    https://doi.org/10.1017/CBO9780511620850.011 [Google Scholar]
  32. Kita, Sotaro & Özyürek, Aslı
    (2003) What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language, 48 (1), 16–32. 10.1016/S0749‑596X(02)00505‑3
    https://doi.org/10.1016/S0749-596X(02)00505-3 [Google Scholar]
  33. Krauss, Robert M., Chen, Yihsiu, & Chawla, Purnima
    (1996) Non-verbal behavior and non-verbal communication: What do conversational hand gesture tell us?InM. Zanna (Ed.), Advances in experimental social psychology (pp. 389–450). San Diego: Academic Press.
    [Google Scholar]
  34. Krauss, Robert M., Morrel-Samuels, Palmer, & Colasante, Christina
    (1991) Do conversational hand gestures communicate?Journal of Personality and Social Psychology, 61 (5), 743–754. 10.1037/0022‑3514.61.5.743
    https://doi.org/10.1037/0022-3514.61.5.743 [Google Scholar]
  35. McNeill, David
    (1985) So you think gestures are non-verbal?Psychological Review, 92 (3), 350–371. 10.1037/0033‑295X.92.3.350
    https://doi.org/10.1037/0033-295X.92.3.350 [Google Scholar]
  36. (1992) Hand and mind. Chicago: University of Chicago Press.
    [Google Scholar]
  37. McNeill, David, Cassell, Justine, & McCullough, Karl-Erik
    (1994) Communicative effects of speech-mismatched gestures. Research on Language and Social Interaction, 27 (3), 223–237. 10.1207/s15327973rlsi2703_4
    https://doi.org/10.1207/s15327973rlsi2703_4 [Google Scholar]
  38. Momsen, Jacob, Gordon, Jared, Wu, Ying Choon, & Coulson, Seana
    (2020) Verbal working memory and co- speech gesture processing. Brain and Cognition, 146, 105640. 10.1016/j.bandc.2020.105640
    https://doi.org/10.1016/j.bandc.2020.105640 [Google Scholar]
  39. Morrel-Samuels, Palmer & Krauss, Robert M.
    (1992) Word familiarity predicts asynchrony of hand gestures and speech. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18 (3), 615–622.
    [Google Scholar]
  40. Navon, David & Miller, Jeff
    (2002) Queuing or sharing? A critical evaluation of the single- bottleneck notion. Cognitive Psychology, 44, 193–251. 10.1006/cogp.2001.0767
    https://doi.org/10.1006/cogp.2001.0767 [Google Scholar]
  41. Obermeier, Christian, Holle, Henning, & Gunter, Thomas
    (2011) What iconic gesture fragments reveal about gesture-speech integration: When synchrony is lost, memory can help. Journal of Cognitive Neuroscience, 23 (7), 1648–1663. 10.1162/jocn.2010.21498
    https://doi.org/10.1162/jocn.2010.21498 [Google Scholar]
  42. Ovroye, Acacia L. & Storm, Benjamin C.
    (2019) Remembering what was said and done: The activation and facilitation of memory for gesture as a consequence of retrieval. Journal of Experimental Psychology: Learning, Memory and Cognition, 45 (3), 526–534.
    [Google Scholar]
  43. Owens, Matthews, Stevenson, Jim, Hadwin, Julie A., & Norgate, Roger
    (2014) When does anxiety help or hinder test performance? The role of working memory capacity. British Journal of Psychology, 105 (1), 92–101. 10.1111/bjop.12009
    https://doi.org/10.1111/bjop.12009 [Google Scholar]
  44. Özer, Demet & Göksun, Tilbe
    (2020a) Gesture use and processing: A review on individual differences in cognitive resources. Frontiers in Psychology, 11, 573555. 10.3389/fpsyg.2020.573555
    https://doi.org/10.3389/fpsyg.2020.573555 [Google Scholar]
  45. (2020b) Visual-spatial and verbal abilities differentially affect processing of gestural vs. spoken expressions. Language, Cognition and Neuroscience, 35 (7), 896–914. 10.1080/23273798.2019.1703016
    https://doi.org/10.1080/23273798.2019.1703016 [Google Scholar]
  46. Özyürek, Aslı, Willems, Roel M., Kita, Sotaro, & Hagoort, Peter
    (2007) On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials. Journal of Cognitive Neuroscience, 19, 605–616. 10.1162/jocn.2007.19.4.605
    https://doi.org/10.1162/jocn.2007.19.4.605 [Google Scholar]
  47. Smyth, Mary M. & Pendleton, Lindsay R.
    (1990) Space and movement in working memory. The Quarterly Journal of Experimental Psychology, Section A: Human Experimental Psychology, 42 (2), 291–304. 10.1080/14640749008401223
    https://doi.org/10.1080/14640749008401223 [Google Scholar]
  48. Vytal, Katherine E., Cornwell, Brian R., Arkin, Nicole Esther, Letkiewicz, Allison M., & Grillon, Christian
    (2013) The complex interaction between anxiety and cognition: insight from spatial and verbal working memory. Frontiers in Human Neuroscience, 7, 93. 10.3389/fnhum.2013.00093
    https://doi.org/10.3389/fnhum.2013.00093 [Google Scholar]
  49. Wagner, Susan M., Nusbaum, Howard, & Goldin-Meadow, Susan
    (2004) Probing the mental representation of gesture: Is handwaving spatial?Journal of Memory and Language, 50, 395–407. 10.1016/j.jml.2004.01.002
    https://doi.org/10.1016/j.jml.2004.01.002 [Google Scholar]
  50. Wechsler, David
    (2008) Wechsler adult intelligence scale – fourth edition.
    [Google Scholar]
  51. Wolf, Dhana, Rekittke, Linn-Marlen, Mittelberg, Irene, Klasen, Martin, & Mathiak, Klaus
    (2017) Perceived conventionality in co-speech gestures involves the fronto-temporal language network. Frontiers in Human Neuroscience, 11, 573. 10.3389/fnhum.2017.00573
    https://doi.org/10.3389/fnhum.2017.00573 [Google Scholar]
  52. Wu, Ying Choon & Coulson, Seana
    (2007) How iconic gestures enhance communication: An ERP study. Brain & Language, 101, 234–245. 10.1016/j.bandl.2006.12.003
    https://doi.org/10.1016/j.bandl.2006.12.003 [Google Scholar]
  53. (2011) Are depictive gestures like pictures? Commonalities and differences in semantic processing. Brain & Language, 119, 184–195. 10.1016/j.bandl.2011.07.002
    https://doi.org/10.1016/j.bandl.2011.07.002 [Google Scholar]
  54. (2014) Co-speech iconic gestures and visuo-spatial working memory. Acta Psychologia, 153, 39–50. 10.1016/j.actpsy.2014.09.002
    https://doi.org/10.1016/j.actpsy.2014.09.002 [Google Scholar]
  55. (2015) Iconic gestures facilitate discourse comprehension in individuals with superior immediate memory for body configurations. Psychological Science, 26 (11), 1717–1727. 10.1177/0956797615597671
    https://doi.org/10.1177/0956797615597671 [Google Scholar]
  56. Zhao, Wanying, Riggs, Kevin, Schindler, Igor, & Holle, Henning
    (2018) Transcranial Magnetic Stimulation over left inferior frontal and posterior temporal cortex disrupts gesture-speech integration. The Journal of Neuroscience, 38 (8), 1891–1900. 10.1523/JNEUROSCI.1748‑17.2017
    https://doi.org/10.1523/JNEUROSCI.1748-17.2017 [Google Scholar]
http://instance.metastore.ingenta.com/content/journals/10.1075/gest.20028.kan
Loading
/content/journals/10.1075/gest.20028.kan
Loading

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error