Languages in Contrast - Volume 22, Issue 2, 2022

For years, the study of spoken languages, on the basis of written and then also oral productions, was the only way to investigate the human language capacity. As an introduction to this first volume of Languages in Contrast devoted to the comparison of spoken and signed languages, we propose to look at the reasons for the late emergence of the consideration of signed languages and multimodality in language studies. Next, the main stages of the history of sign language research are summarized. We highlight the benefits of studying cross-modal and multimodal data, as opposed to the isolated investigation of signed or spoken languages, and point out the remaining methodological obstacles to this approach. This contextualization prefaces the presentation of the outline of the volume.

Users of signed and spoken languages regularly engage bodily enactment (commonly referred to as constructed action [CA] for signers and character viewpoint gestures [CVPT] for speakers) for the creation of meaning, but comparatively few studies have addressed how linguistic grammar interfaces with such gestural depictive devices across language modalities. CVPT gestures have been shown to co-occur with spoken language transitive verbs, and when a reference is definite or more accessible in the discourse. In sign, CA often alternates sequentially with fully conventionalized signs. In both CVPT and CA demonstrations, syntactic and pragmatic factors appear to be important. In this work, we consider these patterns by examining short retellings of video-based elicitation stimuli (silent-movie segments) from 10 deaf users of ASL (American Sign Language) and 20 hearing speakers of English. We describe examples of signs and words that co-occur with or precede specific instances of CA and CVPT. We also examine distributions and degrees of enactment across participants in order to consider the question of gesture threshold (Hostetter and Alibali, 2008, 2019). We provide various examples of how gestural material interfaces with linguistic grammar, which has implications for syntactic theory and possible grammatical constraints on such communicative devices.

Recent work has shown that ASL (American Sign Language) signers not only articulate the language in the space in front of and around them, they interact with that space bodily, such that those interactions are frequently viewpointed. At a basic level, signers use their bodies to depict the actions of characters, either themselves or others, in narrative retelling. These viewpointed instances seem to reflect “embodied cognition”, in that our construal of reality is largely due to the nature of our bodies (Evans and Green, 2006) and “embodied language” such that the symbols we use to communicate are “grounded in recurring patterns of bodily experience” (Gibbs, 2017: 450). But what about speakers of a spoken language such as English? While we know that meaning and structure for any language, whether spoken or signed, affect and are affected by the embodied mind (note that the bulk of research on embodied language has been about spoken, not signed, language), we can learn much about embodied cognition and viewpointed space when spoken languages are treated as multimodal. Here, we compare signed ASL and spoken, multimodal English discourse to examine whether the two languages incorporate viewpointed space in similar or different ways.

The aim of this study is to present a distributional portrait of forms of character-perspective sequences as produced by LSQ (Langue des signes québécoise) signers and Quebec French speakers, in relation to corporal and grammatical marking in a set of recorded discourses. Among the forms we examine are grammatical, corporal and rhythm markers. As for the types of character perspective shift examined, we focus on the nature of the event that is being enacted: speech, thought, action or gesture. The dataset employed in the study consists of short, elicited narratives using video sketches as stimuli. Both Deaf signers and French speakers were asked to describe short scenarios that were displayed without any signing or speech. Half of the stimuli were constructed from a series of factual events containing no emphatic reactions or actions, while the other half included emphatic elements. Twenty-four narratives produced by these two groups were transcribed and coded using ELAN to determine the distribution of character perspective shift sequences (CPS) used in terms of presence (duration) and frequency (occurrences). Further markers were also identified in terms of frequency, which was then analyzed with a factorial ANOVA statistical model. The overall finding of this study is that CPS is used in both language groups, despite their varying results in terms of the distribution of frequency and markers.

This study explores moments in signed and spoken conversation when manual production is on hold and its resulting interactive ramifications. Typically, the temporal structure of gesture and sign can be decomposed into a stream of distinct manual phases. There are moments, however, when this activity is stopped. This may happen for various reasons, e.g., when seeking attention, holding the floor or during overlaps. Holds have mostly been examined in sign languages regarding prosody, syntax, and corresponding to vowel lengthening in spoken languages. In gesture studies, they have been overlooked for not deemed relevant in the gesture-speech interface. By combining contrastive and multimodal analyses, this paper examines the relevance of holds as potential meaning-making practices deployed by LSFB signers and its comparison to Belgian French speakers. In 3 hours of video-recorded material drawn from 3 multimodal corpora, the following question is addressed: what are the roles of holds in the management of interaction within and across languages/modalities? While most of linguistic work considers manual movements to express referential content, the observations here push to reconsider the common boundary set between what constitutes gestural/linguistic phenomena in one language and what does not.

Reformulation is remarkably frequent in discourse and has been the subject of much work in spoken languages, both on written and oral data. Because of its metalinguistic nature, combined with its general aim of clarifying an expression, the act of reformulation offers a window to the way speakers process and adjust their expression in discourse. However, to date, the study of reformulation has hardly taken into account the now increasingly recognized multimodal and semiotically composite nature of language. This study aims to revisit the notion of reformulation from a multimodal perspective by comparing the use and semiotic composition of reformulations in the discourse of speakers and signers, as well as in the productions of interpreters. In doing so, we lay the foundations for a comparative study of discourse in signed and spoken language that accounts for the multimodality and semiotic complexity of language practices in different human ecologies.