1887
  1. Home
  2. e-Journals
  3. Gesture
  4. Volume 15, Issue 1
  5. Article
Volume 15, Issue 1
  • ISSN 1568-1475
  • E-ISSN: 1569-9773
USD
Buy:$35.00 + Taxes

Abstract

Although it is widely acknowledged that gestures are complex functional elements of human communication, many current functional classification systems are rather rigid, implicitly assuming gestures to perform only one function at any given time. In this paper, we present a theoretical view on the inherent multifunctionality of speakers’ gestures, inspired by frameworks in structural-functional linguistics (Halliday’s Systemic Functional Grammar and Hengeveld & Mackenzie’s Functional Discourse Grammar). Building upon this view, we report on a large-scale internet-based gesture perception study, designed in a way open to the potential for complex multifunctionality of gestural expression. The results provide quantitative support for the view that speakers’ gestures typically contribute to multiple semantic and meta-communicative functions of the ongoing discourse simultaneously. Furthermore, we identify clusters of functions that tend to be combined, as well as correlations between pairs of them. As a whole, this paper achieves some degree of convergence between ecological and experimental views on gesture functionality.

© 2016 John Benjamins Publishing Company
Loading

Article metrics loading...

/content/journals/10.1075/gest.15.1.02kok
2016-05-19
2024-10-10

  • From This Site

    /content/journals/10.1075/gest.15.1.02kok
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Bavelas, Janet Beavin , Nicole Chovil , Douglas A. Lawrie , & Allan Wade
    (1992) Interactive gestures. Discourse Processes, 15 (4), 469–489. doi: 10.1080/01638539209544823
     https://doi.org/10.1080/01638539209544823 [Google Scholar]
  2. Beattie, Geoffrey & Heather Shovelton
    (1999) Mapping the range of information contained in the iconic hand gestures that accompany spontaneous speech. Journal of Language and Social Psychology, 18 (4), 438–462. doi: 10.1177/0261927X99018004005
     https://doi.org/10.1177/0261927X99018004005 [Google Scholar]
  3. (2001) An experimental investigation of the role of different types of iconic gesture in communication: A semantic feature approach. Gesture, 1 (2), 129–149. doi: 10.1075/gest.1.2.03bea
     https://doi.org/10.1075/gest.1.2.03bea [Google Scholar]
  4. Bergmann, Kirsten & Stefan Kopp
    (2006) Verbal or visual? How information is distributed across speech and gesture in spatial dialog. In David Schlangen & Raquel Fernández (Eds.), Proceedings of the 10th Workshop on the Semantics and Pragmatics of Dialogue (SemDial-10) (pp. 90–97). Potsdam, Germany: Universitätsverlag.
     [Google Scholar]
  5. (2009) Increasing expressiveness for virtual agents – Autonomous generation of speech and gesture for spatial description tasks. In Keith S. Decker , Jaime S. Sichman , Carles Sierra , & Cristiano Castelfranchi (Eds.), Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2009) (pp. 361–368). Ann Arbor, MI: IFAAMAS.
     [Google Scholar]
  6. Bergmann, Kirsten , Volkan Aksu , & Stefan Kopp
    (2011) The relation of speech and gestures: Temporal synchrony follows semantic synchrony. In Proceedings of the 2nd Workshop on Gesture and Speech in Interaction (GeSpIn 2011) . Bielefeld, Germany.
     [Google Scholar]
  7. Bressem, Jana , Silva H. Ladewig , & Cornelia Müller
    (2013) Linguistic Annotation System for Gestures (LASG). In Cornelia Müller , Alan Cienki , Ellen Fricke , Silva Ladewig , David McNeill , & Sedinha Teßendorf (Eds.), Body – language – communication: An international handbook on multimodality in human interaction, Vol. 1 (pp. 1098–1125). Berlin: Mouton de Gruyter.
     [Google Scholar]
  8. Bühler, Karl
    (1990/1934) Theory of language: The representational function of language, trans. Donald Fraser Goodwin . Amsterdam: John Benjamins (Sprachtheorie. Die Darstellungsfunktion der Sprache. Jena: Gustav Fischer). doi: 10.1075/fos.25
     https://doi.org/10.1075/fos.25 [Google Scholar]
  9. Clark, Herbert H. & Jean E. Fox Tree
    (2002) Using uh and um in spontaneous speaking. Cognition, 84 (1), 73–111. doi: 10.1016/S0010‑0277(02)00017‑3
     https://doi.org/10.1016/S0010-0277(02)00017-3 [Google Scholar]
  10. Connolly, John H
    (2010) Accommodating multimodality in Functional Discourse Grammar. Web Papers in Functional Discourse Grammar, 83, 1–18.
     [Google Scholar]
  11. Denis, Michel
    (1997) The description of routes: A cognitive approach to the production of spatial discourse. Current Psychology of Cognition, 16, 409–458.
     [Google Scholar]
  12. Dik, Simon C
    (1989) The theory of functional grammar, Part I: The structure of the clause. Dordrecht: Foris Publications.
     [Google Scholar]
  13. Efron, David
    (1972) Gesture, race and culture. The Hague: Mouton.
     [Google Scholar]
  14. Enfield, Nicholas J
    (2009) The anatomy of meaning: Speech, gesture, and composite utterances. Cambridge: Cambridge University Press. doi: 10.1017/CBO9780511576737
     https://doi.org/10.1017/CBO9780511576737 [Google Scholar]
  15. Halliday, Michael A.K
    (1985) An introduction to functional grammar. London: Edward Arnold.
     [Google Scholar]
  16. Hengeveld, Kees & J. Lachlan Mackenzie
    (2008) Functional Discourse Grammar: A typologically-based theory of language structure. Oxford: Oxford University Press. doi: 10.1093/acprof:oso/9780199278107.001.0001
     https://doi.org/10.1093/acprof:oso/9780199278107.001.0001 [Google Scholar]
  17. Holler, Judith & Geoffrey Beattie
    (2002) A micro-analytic investigation of how iconic gestures and speech represent core semantic features in talk. Semiotica, 142, 31–69.
     [Google Scholar]
  18. (2003) Pragmatic aspects of representational gestures: Do speakers use them to clarify verbal ambiguity for the listener?Gesture, 3 (2), 127–154. doi: 10.1075/gest.3.2.02hol
     https://doi.org/10.1075/gest.3.2.02hol [Google Scholar]
  19. Holler, Judith & Katie Wilkin
    (2009) Communicating common ground: How mutually shared knowledge influences speech and gesture in a narrative task. Language and Cognitive Processes, 24 (2), 267–289. doi: 10.1080/01690960802095545
     https://doi.org/10.1080/01690960802095545 [Google Scholar]
  20. Humboldt, Wilhelm von
    (1905) Gesammelte Schriften, Bd. 4 [The collected works of Wilhelm von Humboldt, Vol. 4]. Berlin: Behr. doi: 10.1515/9783110833751
     https://doi.org/10.1515/9783110833751 [Google Scholar]
  21. Kendon, Adam
    (1972) Some relationships between body motion and speech. Studies in Dyadic Communication, 7, 177–210. doi: 10.1016/B978‑0‑08‑015867‑9.50013‑7
     https://doi.org/10.1016/B978-0-08-015867-9.50013-7 [Google Scholar]
  22. (1980) Gesticulation and speech: Two aspects of the process of utterance. In Mary R. Key (Ed.), The relationship of verbal and nonverbal communication (pp. 207–227). The Hague: Mouton.
     [Google Scholar]
  23. (1981) Introduction: Current issues in the study of nonverbal communication. In Adam Kendon , Thomas A. Sebeok , & Jean Umiker-Sebeok (Eds.), Nonverbal communication, interaction, and gesture: Selections from Semiotica (pp. 1–53). Berlin: Walter de Gruyter. doi: 10.1515/9783110880021.1
     https://doi.org/10.1515/9783110880021.1 [Google Scholar]
  24. (1995) Gestures as illocutionary and discourse structure markers in Southern Italian conversation. Journal of pragmatics, 23 (3), 247–279. doi: 10.1016/0378‑2166(94)00037‑F
     https://doi.org/10.1016/0378-2166(94)00037-F [Google Scholar]
  25. (2004) Gesture: Visible action as utterance. Cambridge: Cambridge University Press. doi: 10.1017/cbo9780511807572
     https://doi.org/10.1017/cbo9780511807572 [Google Scholar]
  26. Kok, Kasper
    (in press). The grammatical potential of co-speech gesture: A Functional Discourse Grammar perspective. Functions of Language.
     [Google Scholar]
  27. Lausberg, Hedda & Han Sloetjes
    (2009) Coding gestural behavior with the NEUROGES-ELAN system. Behavior Research Methods, 41 (3), 841–849. doi: 10.3758/BRM.41.3.841
     https://doi.org/10.3758/BRM.41.3.841 [Google Scholar]
  28. Levy, Elena T. & David McNeill
    (1992) Speech, gesture, and discourse. Discourse Processes, 15 (3), 277–301. doi: 10.1080/01638539209544813
     https://doi.org/10.1080/01638539209544813 [Google Scholar]
  29. Loehr, Daniel P
    (2004)  Gesture and intonation . PhD thesis, Georgetown University.
  30. Lücking, Andy , Kirsten Bergman , Florian Hahn , Stefan Kopp , & Hannes Rieser
    (2013) Data-based analysis of speech and gesture: The Bielefeld Speech and Gesture Alignment Corpus (SaGA) and its applications. Journal on Multimodal User Interfaces, 7 (1/2), 5–18. doi: 10.1007/s12193‑012‑0106‑8
     https://doi.org/10.1007/s12193-012-0106-8 [Google Scholar]
  31. Martinec, Radan
    (2004) Gestures that co‐occur with speech as a systematic resource: The realization of experiential meanings in indexes. Social Semiotics, 14 (2), 193–213. doi: 10.1080/1035033042000238259
     https://doi.org/10.1080/1035033042000238259 [Google Scholar]
  32. McNeill, David
    (1992) Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.
     [Google Scholar]
  33. (2005) Gesture and thought. Chicago: University of Chicago Press. doi: 10.7208/chicago/9780226514642.001.0001
     https://doi.org/10.7208/chicago/9780226514642.001.0001 [Google Scholar]
  34. Müller, Cornelia
    (1998) Redebegleitende Gesten. Kulturgeschichte – Theorie – Sprachvergleich. Berlin: Berlin Verlag A. Spitz.
     [Google Scholar]
  35. (2013) Gestures as a medium of expression: The linguistic potential of gestures. In Cornelia Müller , Alan Cienki , Ellen Fricke , Silva Ladewig , David McNeill , & Sedinha Tessendorf (Eds.), Body – language – communication: An international handbook on multimodality in human interaction, Vol. 1 (pp. 202–217). Berlin: Mouton de Gruyter.
     [Google Scholar]
  36. Müller, Cornelia , Silva Ladewig , & Jana Bressem
    (2013) Gestures and speech from a linguistic perspective: A new field and its history. In Cornelia Müller , Alan Cienki , Ellen Fricke , Silva Ladewig , David McNeill , & Sedinha Tessendorf (Eds.), Body – language – communication: An international handbook on multimodality in human interaction, Vol. 1 (pp. 55–81). Berlin: Mouton de Gruyter.
     [Google Scholar]
  37. Muntigl, Peter
    (2004) Modelling multiple semiotic systems: The case of gesture and speech. In Eija Ventola , Cassily Charles , & Martin Kaltenbacher (Eds.), Perspectives on multimodality (pp. 31–49). Amsterdam: John Benjamins. doi: 10.1075/ddcs.6.04mun
     https://doi.org/10.1075/ddcs.6.04mun [Google Scholar]
  38. Parrill, Fey
    (2010) Viewpoint in speech–gesture integration: Linguistic structure, discourse structure, and event structure. Language and Cognitive Processes, 25 (5), 650–668. doi: 10.1080/01690960903424248
     https://doi.org/10.1080/01690960903424248 [Google Scholar]
  39. Sperber, Dan & Deirdre Wilson
    (1986) Relevance: Communication and cognition. Oxford: Blackwell.
     [Google Scholar]
  40. Streeck, Jürgen
    (2009) Gesturecraft: The manu-facture of meaning. Amsterdam: John Benjamins. doi: 10.1075/gs.2
     https://doi.org/10.1075/gs.2 [Google Scholar]
  41. Van Valin, Robert D., Jr
    (1993) Advances in role and reference grammar. Amsterdam: John Benjamins. doi: 10.1075/cilt.82
     https://doi.org/10.1075/cilt.82 [Google Scholar]
  42. Wundt, Wilhelm
    (1973) The language of gestures. The Hague: Mouton. doi: 10.1515/9783110808285
     https://doi.org/10.1515/9783110808285 [Google Scholar]
/content/journals/10.1075/gest.15.1.02kok
Loading
  • Article Type: Research Article
Keyword(s): communication; discourse; gestures; multifunctionality

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error