Gesture - Volume 21, Issue 2-3, 2022

This paper investigates gesture as a resource for marking politeness-related meanings. We asked 14 Korean and 14 Catalan participants to retell a cartoon, once to an unknown superior and once to a close friend. Participants in both languages curtail gestures when interacting with a socially distant superior. Speakers of both languages produced fewer gestures when addressing the superior, reduced their gesture space, decreased the encoding of manner, and reduced the use of character-viewpoint gestures. We see the decrease in gesture frequency and the less frequent encoding of manner as indicators of lower levels of iconicity when talking with status superiors. Curtailing gesture marks a less playful communicative context, and a more serious and deferential persona. Altogether, our research speaks to the importance of politeness in gesture production, and the social nature of gestures in human communication.

A consolidated tendency considers ‘gestures’ and ‘signs’ as distinct categories separated by a ‘cataclysmic break’. According to a different approach, gestures and signs have their common origin in actions, and are considered as part of language. The aim of this study was to compare the productions of preschool speaking hearing children and signing deaf children in response to the same visual stimuli. The execution parameters and representational strategies observed in gestures and signs were analyzed using the same coding. The results showed that hearing children exposed to Italian and deaf children exposed to Italian Sign Language are consistent in their productions of gestures and signs, respectively. Furthermore, the hearing children’s gestures and the deaf children’s signs for some items were produced with the same parameters and according to similar representational strategies. This indicates that these two forms of communication are not separate behaviors, but should rather be considered as a continuum.

Co-speech gestures can help the learning, processing, and memory of words and concepts, particularly motoric and spatial concepts such as verbs. The purpose of the present studies was to test whether co-speech gestures support the learning of words through gist traces of movement. We asked English monolinguals to learn 40 Cantonese words (20 verbs and 20 nouns). In two studies, we found support for the gist traces of congruent gestures being movement: participants who saw congruent gestures while hearing Cantonese words thought they had seen more verbs than participants in any other condition. However, gist traces were unrelated to the accurate recall of either nouns or verbs. In both studies, learning Cantonese words accompanied by congruent gestures tended to interfere with the learning of nouns (but not verbs). In Study 2, we ruled out the possibility that this interference was due either to gestures conveying representational information in another medium or to distraction from moving hands. We argue that gestures can interfere with learning foreign language words when they represent the referents (e.g., show shape or size) because learners must interpret the hands as something other than hands.

The present study examines the roles that the gesture of the Raised Index Finger (RIF) plays in Hebrew multimodal interaction. The study reveals that the RIF is associated with diverse linguistic phenomena and tends to appear in contexts in which the speaker presents a message or speech act that violates the hearer’s expectations (based on either general knowledge or prior discourse). The study suggests that the RIF serves the function of discourse deixis: Speakers point to their message, creating a referent in the extralinguistic context to which they refer as an object of their stance, evaluating the content of the utterance or speech act as unexpected by the hearer, and displaying epistemic authority. Setting up such a frame by which the information is to be interpreted provides the basis for a swifter update of the common ground in situations of (assumed) differences between the assumptions of the speaker and the hearer.

Previous studies using cross-modal semantic priming have found that iconic gestures prime target words that are related with the gestures. In the present study, two analogous experiments examined this priming effect presenting prime and targets in high synchrony. In Experiment 1, participants performed an auditory primed lexical decision task where target words (e.g., “push”) and pseudowords had to be discriminated, primed by overlapping iconic gestures that could be semantically related (e.g., moving both hands forward) or not with the words. Experiment 2 was similar but with both gestures and words presented visually. The grammatical category of the words was also manipulated: they were nouns and verbs. It was found that words related to gestures were recognized faster and with fewer errors than the unrelated ones in both experiments and similarly for both types of words.

This study presents an automatic tool that allows to trace smile intensities along a video record of conversational face-to-face interactions. The processed output proposes a sequence of adjusted time intervals labeled following the Smiling Intensity Scale (Gironzetti, Attardo, and Pickering, 2016), a 5 levels scale varying from neutral facial expression to laughing smile. The underlying statistical model of this tool is trained on a manually annotated corpus of conversations featuring spontaneous facial expressions. This model will be detailed in this study. This tool can be used with benefits for annotating smile in interactions. The results are twofold. First, the evaluation reveals an observed agreement of 68% between manual and automatic annotations. Second, manually correcting the labels and interval boundaries of the automatic outputs reduces by a factor 10 the annotation time as compared with the time spent for manually annotating smile intensities without pretreatment. Our annotation engine makes use of the state-of-the-art toolbox OpenFace for tracking the face and for measuring the intensities of the facial Action Units of interest all along the video. The documentation and the scripts of our tool, the SMAD software, are available to download at the HMAD open source project URL page https://github.com/srauzy/HMAD (last access 31 July 2023).