1887
  1. Home
  2. e-Journals
  3. Translation, Cognition & Behavior
  4. Volume 7, Issue 1
  5. Article
Volume 7, Issue 1
  • ISSN 2542-5277
  • E-ISSN: 2542-5285
PDF
EPUB

Abstract

Abstract

This paper presents a conceptual replication of Kruger et al.’s (2017a) study, investigating the utility of heart rate (HR) and heart rate variability (HRV) indicators as objective measurements of psychological immersion in subtitled audiovisual products. The study employed a quasi-experimental design in ten participants viewing video clips with and without subtitles, while their physiological responses were monitored. Results suggest lower arousal levels in both conditions compared to baseline, with statistically significant differences observed in mean HR, mean RR, and RMSSD for the subtitles condition. While most inferential tests did not yield significant outcomes, descriptive trends indicate decreased arousal relative to the baseline. They also show increased arousal throughout the clip for most indicators and opposing trends for Mean HR, RMSSD and LF/HF between the two conditions. Methodological suggestions include using multiple data points for HR and HRV analysis, exploring ultra short-term measurements, selecting self-contained clips for viewing, and considering post-task baselines for a more accurate resting state representation. Despite limitations due to the small sample size, this study underscores the potential of HR and HRV as measures of immersion in translated audiovisual products, emphasising the need for larger sample sizes in future research to enhance statistical power and generalizability.

© 2024 John Benjamins Publishing Company
Available under the CC BY 4.0 license.
Loading

Article metrics loading...

/content/journals/10.1075/tcb.00090.gon
2024-10-10
2025-04-19

  • From This Site

    /content/journals/10.1075/tcb.00090.gon
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

/deliver/fulltext/tcb.00090.gon.html?itemId=/content/journals/10.1075/tcb.00090.gon&mimeType=html&fmt=ahah

References

  1. Agrawal, Sarvesh, Adèle Simon, Søren Bech, Klaus Bærentsen, and Søren Forchhammer
    2020 “Defining Immersion: Literature Review and Implications for Research on Audiovisual Experiences.” Journal of the Audio Engineering Society68 (6): 404–417. 10.17743/jaes.2020.0039
     https://doi.org/10.17743/jaes.2020.0039 [Google Scholar]
  2. Baek, Hyun J., Chul-Ho Cho, Jaegon Cho, and Jong-Min Woo
    2015 “Reliability of Ultra-Short-Term Analysis as a Surrogate of Standard 5-Min Analysis of Heart Rate Variability.” Telemed J E Health211: 404–414. 10.1089/tmj.2014.0104
     https://doi.org/10.1089/tmj.2014.0104 [Google Scholar]
  3. Baños, Rosa, Cristina Botella, Mariano Alcañiz, Víctor Liaño, Belén Guerrer, and Beatriz Rey
    2004 “Immersion and Emotion: Their Impact on the Sense of Presence.” Cyberpsychol. Behav. 71: 734–741. 10.1089/cpb.2004.7.734
     https://doi.org/10.1089/cpb.2004.7.734 [Google Scholar]
  4. Castaldo, Rossana, Paolo Melillo, Umberto Bracale, Marco Caserta, Maria Triassi, and Leandro Pecchia
    2015 “Acute Mental Stress Assessment via Short Term HRV Analysis in Healthy Adults: A Systematic Review with Meta-Analysis.” Biomedical Signal Processing and Control181: 370–377. 10.1016/j.bspc.2015.02.012
     https://doi.org/10.1016/j.bspc.2015.02.012 [Google Scholar]
  5. Chaume, Frederic
    2018 “An Overview of Audiovisual Translation: Four Methodological Turns in a Mature Discipline.” Journal of Audiovisual Translation1 (1): 40–63. 10.47476/jat.v1i1.43
     https://doi.org/10.47476/jat.v1i1.43 [Google Scholar]
  6. Díaz Cintas, Jorge, and Agnieszka Szarkowska
    2020 “Introduction: Experimental Research in Audiovisual Translation — Cognition, Reception, Production.” The Journal of Specialised Translation (33): 3–16.
     [Google Scholar]
  7. Di Lascio, Elena, Shkurta Gashi, and Silvia Santini
    2018 “Unobtrusive Assessment of Students’ Emotional Engagement during Lectures Using Electrodermal Activity Sensors.” Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2 (3): 1–21. 10.1145/3264913
     https://doi.org/10.1145/3264913 [Google Scholar]
  8. Fairclough, Stephen, Katie Ewing, and Jenna Roberts
    2009 “Measuring Task Engagement as an Input to Physiological Computing.” Proceedings — 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, ACII, 1–9. 10.1109/ACII.2009.5349483
     https://doi.org/10.1109/ACII.2009.5349483 [Google Scholar]
  9. Flis, Gabriela, Adam Sikorski, and Agnieszka Szarkowska
    2020 “Does the Dubbing Effect Apply to Voice-Over? A Conceptual Replication Study on Visual Attention and Immersion.” Journal of Specialised Translation (33): 41–69.
     [Google Scholar]
  10. Fryer, Louise
    2013 Putting it into Words: The Impact of Visual Impairment on Perception, Experience and Presence. PhD thesis, Goldsmiths College University of London.
     [Google Scholar]
  11. Fryer, Louise and Jonathan Freeman
    2012 “Cinematic Language and the Description of Film: Keeping AD Users in the Frame.” Perspectives: Studies in Translatology21 (3): 412–426. 10.1080/0907676X.2012.693108
     https://doi.org/10.1080/0907676X.2012.693108 [Google Scholar]
  12. Gambier, Yves, and Elena di Giovanni
    eds. 2018 Reception Studies and Audiovisual Translation. Amsterdam: John Benjamins. 10.1075/bct.95
     https://doi.org/10.1075/bct.95 [Google Scholar]
  13. Greco, Gian Maria, Anna Jankowska, and Agnieszka Szarkowska
    2022 “Addressing Methodological Issues in Experimental Research in Audiovisual Translation.” Translation Spaces11 (1): 1–11. 10.1075/ts.00030.int
     https://doi.org/10.1075/ts.00030.int [Google Scholar]
  14. Green, Melanie, and Timothy Brock
    2013 Transport narrative questionnaire. Measurement Instrument Database for the Social Science. Retrieved fromwww.midss.ie
     [Google Scholar]
  15. Iturregui-Gallardo, Gonzalo, and Anna Matamala
    2021 “Audio Subtitling: Dubbing and Voice-Over Effects and Their Impact on User Experience.” Perspectives29 (1): 64–83. 10.1080/0907676X.2019.1702065
     https://doi.org/10.1080/0907676X.2019.1702065 [Google Scholar]
  16. Jamovi project
    Jamovi project (2023) Jamovi (Version 2.3) [Computer Software]. https://www.jamovi.org
  17. Jankowska, Anna, Joanna Pilarczyk, Kinga Wołoszyn, and Michał Kuniecki
    2022 “Enough is Enough: How Much Intonation is Needed in the Vocal Delivery of Audio Description?” Perspectives31 (4): 705–723. 10.1080/0907676X.2022.2026423
     https://doi.org/10.1080/0907676X.2022.2026423 [Google Scholar]
  18. Kim, Taeyong, and Frank Biocca
    1997 “Telepresence via Television: Two Dimensions of Telepresence May Have Different Connections to Memory and Persuasion.” Journal of Computer-Mediated Communication3 (2). 10.1111/j.1083‑6101.1997.tb00073.x
     https://doi.org/10.1111/j.1083-6101.1997.tb00073.x [Google Scholar]
  19. Kim, Hye-Geum, Eun-Jin Cheon, Dai-Seg Bai, Young Lee, and Bon-Hoon Koo
    2018 “Stress and Heart Rate Variability: A Meta-Analysis and Review of the Literature.” Psychiatry Investigation15 (3): 235–245. 10.30773/pi.2017.08.17
     https://doi.org/10.30773/pi.2017.08.17 [Google Scholar]
  20. Kruger, Haidee, and Jan-Louis. Kruger
    2017 “Cognition and Reception.” InThe Handbook of Translation and Cognition, edited byFabio Alves, and Arnt Jakobsen, 71–89. New Jersey: John Wiley & Sons. 10.1002/9781119241485.ch4
     https://doi.org/10.1002/9781119241485.ch4 [Google Scholar]
  21. Kruger, Jan-Louis, María Soto-Sanfiel, Stephen Doherty, and Ronny Ibrahim
    2016 “Towards a Cognitive Audiovisual Translatology: Subtitles and Embodied Cognition”. InReembedding Translation Process Research, edited byRicardo Muñoz, 171–194. London: John Benjamins. 10.1075/btl.128.09kru
     https://doi.org/10.1075/btl.128.09kru [Google Scholar]
  22. Kruger, Jan-Louis, Stephen Doherty, and Ronny Ibrahim
    2017a “Electroencephalographic Beta Coherence as an Objective Measure of Psychological Immersion in Film.” Rivista Internazionale di Tecnica della Traduzione191, 99–111. 10.13137/2421‑6763/17353
     https://doi.org/10.13137/2421-6763/17353 [Google Scholar]
  23. Kruger, Jan-Louis, Stephen Doherty, and María Soto-Sanfiel
    2017b “Original Language Subtitles: Their Effects on the Native and Foreign Viewer.” Comunicar501, 23–32. 10.3916/C50‑2017‑02
     https://doi.org/10.3916/C50-2017-02 [Google Scholar]
  24. Laborde, Sylvain, Emma Mosley, and Julian Thayer
    2017 “Heart Rate Variability and Cardiac Vagal Tone in Psychophysiological Research — Recommendations for Experiment Planning, Data Analysis, and Data Reporting.” InFrontiers in Psychology81: 213. 10.3389/fpsyg.2017.00213
     https://doi.org/10.3389/fpsyg.2017.00213 [Google Scholar]
  25. Lemhöfer, Kristin, and Mirjam Broersma
    2012 “Introducing LexTALE: A Quick and Valid Lexical Test for Advanced Learners of English.” Behavior Research Methods44 (2), 325–343. 10.3758/s13428‑011‑0146‑0
     https://doi.org/10.3758/s13428-011-0146-0 [Google Scholar]
  26. Lemma, Anna, Bethany Kok, and Tania Singer
    2015 “Is Meditation Always Relaxing? Investigating Heart Rate, Heart Rate Variability, Experienced Effort and Likeability during Training of Three Types of Meditation.” International Journal of Psychophysiology97 (1): 38–45. 10.1016/j.ijpsycho.2015.04.017
     https://doi.org/10.1016/j.ijpsycho.2015.04.017 [Google Scholar]
  27. Lipponen, Jukka, and Mika Tarvainen
    2019 “A Robust Algorithm for Heart Rate Variability Time Series Artefact Correction Using Novel Beat Classification.” Journal of Medical Engineering & Technology43(3): 173–181. 10.1080/03091902.2019.1640306
     https://doi.org/10.1080/03091902.2019.1640306 [Google Scholar]
  28. Lohani, Monika, Brennan Payne, and David Strayer
    2019 “A Review of Psychophysiological Measures to Assess Cognitive States in Real-World Driving.” Frontiers in Human Neuroscience. 131: 57. 10.3389/fnhum.2019.00057
     https://doi.org/10.3389/fnhum.2019.00057 [Google Scholar]
  29. Loudon, Gareth, and Gina Deininger
    2016 “The Physiological Response During Divergent Thinking.” Journal of Behavioral and Brain Science6 (1): 28–37. 10.4236/jbbs.2016.61004
     https://doi.org/10.4236/jbbs.2016.61004 [Google Scholar]
  30. McCarthy, Cameron, Nikhilesh Pradhan, Calum Redpath, and Andy Adler
    2016 “Validation of the Empatica E4 wristband.” 2016 IEEEEMBS International Student Conference (ISC), Ottawa, ON, Canada, 1–4. 10.1109/EMBSISC.2016.7508621
     https://doi.org/10.1109/EMBSISC.2016.7508621 [Google Scholar]
  31. Moses, Ziev, Linda Luecken, and Eason, James
    2007 “Measuring Task-Related Changes in Heart Rate Variability.” 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society: 644–647. Lyon, France: IEEE. 10.1109/IEMBS.2007.4352372
     https://doi.org/10.1109/IEMBS.2007.4352372 [Google Scholar]
  32. Munoz M. Loreto, Arie van Roon, Harriëtte Riese, Chris Thio, Emma Oostenbroek, Iris Westrik, Eco de Geus, Ron Gansevoort, Joop Lefrandt, Ilja Nolte, and Harold Sneider
    2015 “Validity of (Ultra-) Short Recordings for Heart Rate Variability Measurements.” PLOS ONE10 (9). 10:e0138921.10.1371/journal.pone.0138921
     https://doi.org/10:e0138921.10.1371/journal.pone.0138921 [Google Scholar]
  33. Nussinovitch, Udi, Keren Elishkevitz, Keren Katz, Moshe Nussinovitch, Shlomo Segev, Benjamin Volovitz, and Naomi Nussinovitch
    2011 “Reliability of Ultra-Short ECG Indices for Heart Rate Variability.” Annals of Noninvasive Electrocardiology16 (2):117–22. 10.1111/j.1542‑474X.2011.00417.x
     https://doi.org/10.1111/j.1542-474X.2011.00417.x [Google Scholar]
  34. Olalla-Soler, Christian, Nicoletta Spinolo, and Ricardo Muñoz
    2023 “Under Pressure? A Study of Heart Rate and Heart-Rate Variability Using the SmarTerp CAI tool.” HERMES — Journal of Language and Communication in Business (63): 119–142. 10.7146/hjlcb.vi63.134292
     https://doi.org/10.7146/hjlcb.vi63.134292 [Google Scholar]
  35. Orero, Pilar, Stephen Doherty, Jan-Louis Kruger, Anna Matamala, Jan Pedersen, Elisa Perego, Pablo Romero Fresco, Sara Rovira-Esteva, Olga Soler, and Agnieszka Szarkowska
    2018 “Conducting Experimental Research in Audiovisual Translation (AVT): A Position Paper.” Journal of specialised translation301: 105–126. www.jostrans.org/issue30/art_orero_et_al.php
     [Google Scholar]
  36. Price, Mathew, Natasha Mehta, Erin Tone, and Page Anderson
    2011 “Does Engagement with Exposure Yield Better Outcomes? Components of Presence as a Predictor of Treatment Response for Virtual Reality Exposure Therapy for Social Phobia.” Journal of Anxiety Disorder. 251: 763–770. 10.1016/j.janxdis.2011.03.004
     https://doi.org/10.1016/j.janxdis.2011.03.004 [Google Scholar]
  37. Raffi, Francesca
    2020 “The Impact of Italian Dubbing on Viewers’ Immersive Experience: An Audience Reception Study.” Online Journal of Communication and Media Technologies10 (3). 10.30935/ojcmt/8371
     https://doi.org/10.30935/ojcmt/8371 [Google Scholar]
  38. Ramos Caro, Marina
    2015 “The Emotional Experience of Films: Does Audio Description Make a Difference?” Translator21 (1): 68–94. 10.1080/13556509.2014.994853
     https://doi.org/10.1080/13556509.2014.994853 [Google Scholar]
  39. 2016 “Testing Audio Narration: The Emotional Impact of Language in Audio Description.” Perspectives24 (4): 606–634. 10.1080/0907676X.2015.1120760
     https://doi.org/10.1080/0907676X.2015.1120760 [Google Scholar]
  40. Ramos Caro, Marina, Laura Espín López, and Ana Rojo López
    2021 “The Psychophysiological Impact of Audio Described Porn.” Onomázein Revista de Lingüística Filología y Traducción (8): 104–127. 10.7764/onomazein.ne8.06
     https://doi.org/10.7764/onomazein.ne8.06 [Google Scholar]
  41. Rojo Lopez, Ana, Marina Ramos Caro, and Laura Espin Lopez
    2021 “Audio Described vs. Audiovisual Porn: Cortisol, Heart Rate and Engagement in Visually Impaired vs. Sighted Participants.” Frontiers in Psychology121: 661452. 10.3389/fpsyg.2021.661452
     https://doi.org/10.3389/fpsyg.2021.661452 [Google Scholar]
  42. Rojo López, Ana, and Paweł Korpal
    2020 “Through Your Skin to Your Heart and Brain: A Critical Evaluation of Physiological Methods in Cognitive Translation and Interpreting Studies.” Linguistica Antverpiensia, New Series — Themes in Translation Studies191: 191–217. 10.52034/lanstts.v19i0.533
     https://doi.org/10.52034/lanstts.v19i0.533 [Google Scholar]
  43. Rojo López, Ana, Paula Cifuentes-Férez, and Laura Espín López
    2021 “The Influence of Time Pressure on Translation Trainees’ Performance: Testing the Relationship Between Self-Esteem, Salivary Cortisol and Subjective Stress Response.” PLOS ONE16 (9): e0257727. 10.1371/journal.pone.0257727
     https://doi.org/10.1371/journal.pone.0257727 [Google Scholar]
  44. Romero Fresco, Pablo
    2020 “The Dubbing Effect: An Eye-Tracking Study on how Viewers Make Dubbing Work.” The Journal of Specialised Translation33: 17–40.
     [Google Scholar]
  45. Salahuddin, Lizawati, Jaegeol Cho, Myeong G. Jeong, and Desok Kim
    2007 “Ultra Short-Term Analysis of Heart Rate Variability for Monitoring Mental Stress in Mobile Settings.” 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society: 4656–4659. 10.1109/IEMBS.2007.4353378
     https://doi.org/10.1109/IEMBS.2007.4353378 [Google Scholar]
  46. Schuurmans, Angela, Peter de Looff, Karin S. Nijhof, Catarina Rosada, Ron H. J. Scholte, Arne Popma, and Roy Otten
    2020 “Validity of the Empatica E4 wristband to Measure Heart Rate Variability (HRV)Pparameters: A Comparison to Electrocardiography (ECG).” Journal of Medical Systems44 (11): 190. 10.1007/s10916‑020‑01648‑w
     https://doi.org/10.1007/s10916-020-01648-w [Google Scholar]
  47. Shaffer, Fred, and J. P. Ginsberg
    2017 “An Overview of Heart Rate Variability Metrics and Norms.” Frontiers in Public Health51: 258. 10.3389/fpubh.2017.00258
     https://doi.org/10.3389/fpubh.2017.00258 [Google Scholar]
  48. Shaffer, Fredric, Steven Shearman, and Zachary Meehan
    2016 “The Promise Of Ultra-Short-Term (UST) Heart Rate Variability Measurements”. Biofeedback441: 229–33. 10.5298/1081‑5937‑44.3.09
     https://doi.org/10.5298/1081-5937-44.3.09 [Google Scholar]
  49. Spinolo, Nicoletta, Christian Olalla-Soler, and Ricardo Muñoz
    2022 “Finding a Way Into an Interpreter’s Heart: Methodological Considerations on Heart-Rate Variability Building on an Exploratory Study.” Interpreters Newsletter271, 63–87. 10.13137/2421‑714X/34392
     https://doi.org/10.13137/2421-714X/34392 [Google Scholar]
  50. Szarkowska, Agnieszka, and Julianna Boczkowska
    2020 “Colour Coding Subtitles in Multilingual Films — A Reception Study.” Perspectives30 (3): 520–536. 10.1080/0907676X.2020.1853186
     https://doi.org/10.1080/0907676X.2020.1853186 [Google Scholar]
  51. Tal-Or, Nurit, and Jonathan Cohen
    2010 “Understanding Audience Involvement: Conceptualizing and Manipulating Identification and Transportation.” Poetics381: 402–418. 10.1016/j.poetic.2010.05.004
     https://doi.org/10.1016/j.poetic.2010.05.004 [Google Scholar]
  52. Task Force of The European Society of Cardiology and The North American Society of Pacing and Electrophysiology
    Task Force of The European Society of Cardiology and The North American Society of Pacing and Electrophysiology (1996) “Heart Rate Variability.” Eur Heart J17/281.
     [Google Scholar]
  53. Veltman, J. A. and Anthony Gaillard
    1993 “Indices of Mental Workload in a Complex Task Environment.” Neuropsychobiology28(1–2): 72–75. 10.1159/000119003
     https://doi.org/10.1159/000119003 [Google Scholar]
  54. Walczak, Agnieszka, and Louis Fryer
    2017 “Creative Description: The Impact of Audio Description Style on Presence in Visually Impaired Audiences.” British Journal of Visual Impairment35 (1): 6–17. 10.1177/0264619616661603
     https://doi.org/10.1177/0264619616661603 [Google Scholar]
  55. 2017 “Vocal Delivery of Audio Description by Genre: Measuring Users’ Presence.” Perspectives261: 69–83. 10.1080/0907676X.2017.1298634
     https://doi.org/10.1080/0907676X.2017.1298634 [Google Scholar]
  56. Wissmath, Bartholomäus, David Weibel, and Rudolf Groner
    2009 “Dubbing or Subtitling?: Effects on Spatial Presence, Transportation, Flow, and Enjoyment.” Journal of Media Psychology21 (3): 114–125. 10.1027/1864‑1105.21.3.114
     https://doi.org/10.1027/1864-1105.21.3.114 [Google Scholar]
/content/journals/10.1075/tcb.00090.gon
Loading
Measuring psychological immersion through cardiovascular response measures in subtitled films
Translation, Cognition & Behavior 7, 90 (2024); https://doi.org/10.1075/tcb.00090.gon
/content/journals/10.1075/tcb.00090.gon
/content/journals/10.1075/tcb.00090.gon
Loading

Data & Media loading...

  • Article Type: Research Article
Keyword(s): heart rate (HR); heart rate variability (HRV); immersion; subtitling

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error