Translation Spaces - Volume 11, Issue 1, 2022

Empirical research on the cognitive processing of audiovisual translation (AVT) products has been thriving over the past decade. While the use of cutting-edge experimental tools such as eye trackers has drawn increasing scholarly attention and accelerated the progress in understanding the complex mental processes involved in the reception of multimodal AVT products, relatively less attention has been devoted to the importance of establishing a theoretical framework or cognitive model that can explain and predict the behaviours observed in empirical experiments. By reviewing numerous theories or cognitive models relevant to AVT research in explaining how different perceptual and cognitive systems operate for understanding multimodal products, this paper calls for engagement with these theoretical frameworks and models to work towards a robust model that can generate testable hypotheses for the integration and interaction of multiple sources of information involved in the processing of AVT or other multimodal products.

Mixed methods have an established trajectory in the social sciences. Audiovisual Translation and Media Accessibility (MA) Studies are also increasingly applying the “third research paradigm” (Johnson et al., 2007, 112). Yet, publications in our field often fail to discuss the mixed-method nature of the study in depth, be it because of space limitations or a lack of deliberate integration of the methods. Concurrently, MA has seen a boom in experimental research, as descriptive approaches have given way to reception and user-centred studies that engage in the cognitive processes and immersion of audiences (Orero et al. 2018). This article proposes a methodological basis for MA researchers to design studies employing physiological instruments within a mixed methods framework. The core mixed methods designs (convergent, explanatory, and exploratory) are presented, and examples of their applications to research employing physiological instruments are discussed.

In this paper, we aim to promote the use of linear mixed models (LMMs) in eye-tracking research on subtitling. Using eye tracking to study viewers’ reading of subtitles often warrants controlling for many confounding variables. However, even assuming that these variables are known to researchers, such control may not be possible or desired. Traditional statistical methods such as t-tests or ANOVAs exacerbate the problem due to the use of aggregated data: each participant has one data point per dependent variable. As a solution, we propose the use of LMMs, which are better suited to account for a number of subtitle and participant characteristics, thus explaining more variance. We introduce essential theoretical aspects of LMMs and highlight some of their advantages over traditional statistical methods. To illustrate our point, we compare two analyses of the same dataset: one using a t-test; another using LMMs.

Recent developments in neural machine translation, and especially speech translation, are gradually but firmly entering the field of audiovisual translation (AVT). Automation in subtitling is extending from a machine translation (MT) component to fully automatic subtitling, which comprises MT, auto-spotting and automatic segmentation. The rise of this new paradigm renders MT-oriented experimental designs inadequate for the evaluation and investigation of automatic subtitling, since they fail to encompass the multimodal nature and technical requirements of subtitling. This paper highlights the methodological gaps to be addressed by multidisciplinary efforts in order to overcome these inadequacies and obtain metrics and methods that lead to rigorous experimental research in automatic subtitling. It presents a review of previous experimental designs in MT for subtitling, identifies their limitations for conducting research under the new paradigm and proposes a set of recommendations towards achieving replicability and reproducibility in experimental research at the crossroads between AVT and MT.

Empirical research on cognitive processing in AVT has been on the rise in recent years. A number of overarching works have recommended more standardised approaches and methodological frameworks to contribute to more streamlined, replicable, reproducible and valid future AVT research. To date, the issue of comparability of research materials (e.g., clips, subtitle tracks, comprehension questionnaires) and, more specifically, how to achieve comparability in quasi-experimental and experimental studies, particularly those involving repeated measures, has received little attention. This paper aims to address this knowledge gap by proposing a common-sense ten-step preparatory process for quasi-experimental and experimental subtitling studies. This preparatory process has previously been used in the S4AE project. The paper will focus on the final four steps, consisting of the preparation and comparison of multiple subtitle tracks. These steps were conceptualized taking into account the present research on subtitle parameters and the obstacles encountered while preparing comparable subtitle tracks.

Museums face a particular challenge in enabling blind and partially sighted (BPS) visitors to engage emotionally with the narrative(s) they present. In collaboration with a world-leading tourist attraction (Titanic Belfast) and the Royal National Institute of Blind People (RNIB), we have applied several different approaches for improving emotional engagement for BPS visitors. This paper addresses the critical challenge of how to obtain reliable evidence for evaluating the emotional response of BPS visitors to the museum’s audio description (AD) and overall experience. We consider six different methods for measuring emotional engagement, and consider their potential for providing reliable experimental evidence. Based on BPS-user feedback, we present a qualitative comparison of these methods, uniquely all applied to the same museum context.