1887
image of Emoji in dialogue with images

Abstract

Abstract

This paper explores the visual nature of emoji and their potential to co-create meaning when in dialogue with other visual semiotic resources, such as images, paintings, videos, and photographs. Drawing on a multimodal social semiotic framework and studies that emphasise the independent meaning-making capabilities of visual communication, it challenges the traditional view that emoji are merely paralinguistic elements dependent on verbal co-texts. The study analyses a corpus of art museum social media posts, highlighting instances of intermodal convergence between the shared images and emoji used in the captions. This analysis reveals formal and semantic dialogic relations, showcasing how emoji can function as independent semiotic resources, capable of establishing meaningful interactions with other visual elements.

Available under the CC BY 4.0 license.
Loading

Article metrics loading...

/content/journals/10.1075/ld.00211.laz
2025-09-29
2025-11-09
Loading full text...

Full text loading...

/deliver/fulltext/10.1075/ld.00211.laz/ld.00211.laz.html?itemId=/content/journals/10.1075/ld.00211.laz&mimeType=html&fmt=ahah

References

  1. Abercrombie, David
    1968 “Paralanguage.” British Journal of Disorders of Communication(): –. 10.3109/13682826809011441
    https://doi.org/10.3109/13682826809011441 [Google Scholar]
  2. Bakhtin, Mikhail M.
    1986Speech Genres and Other Late Essays. Austin: University of Texas Press.
    [Google Scholar]
  3. Camillini, Gianluca, Marcello Barison, and Roberto Gigliotti
    2022 “Images in Dialogue: How They Talk and What They Say.” Img Journal: –.
    [Google Scholar]
  4. Częstochowska, Justyna, Kristina Gligorić, Maxime Peyrard, Yann Mentha, Michał Bień, Andrea Grütter, Anita Auer, Aris Xanthos, and Robert West
    2022 “On the Context-free Ambiguity of Emoji.” Proceedings of the International AAAI Conference on Web and Social Media: –. 10.1609/icwsm.v16i1.19393
    https://doi.org/10.1609/icwsm.v16i1.19393 [Google Scholar]
  5. Danesi, Marcel
    2016The Semiotics of Emoji. London: Bloomsbury Publishing.
    [Google Scholar]
  6. 2018 “Emojis: New Language or Technology-based Trend?” ETC: A Review of General Semantics(): –.
    [Google Scholar]
  7. Dürscheid, Christa, Dimitrios Meletis, and Yannis Haralambous
    2019 “Emojis: A grapholinguistic approach.” Grapholinguistics and Its Applications: –. 10.36824/2018‑graf‑duer
    https://doi.org/10.36824/2018-graf-duer [Google Scholar]
  8. Fulford, Jason and Tamara Shopsin
    2014This Equals That. New York: Aperture.
    [Google Scholar]
  9. Halliday, Michael Alexander Kirkwood
    1978Language as Social Semiotic. London: Hodder Arnold.
    [Google Scholar]
  10. Holquist, Michael
    2003Dialogism: Bakhtin and His World. London: Routledge. 10.4324/9780203425855
    https://doi.org/10.4324/9780203425855 [Google Scholar]
  11. Jovanovic, Danica and Theo Van Leeuwen
    2018 “Multimodal dialogue on social media.” Social Semiotics(): –. 10.1080/10350330.2018.1504732
    https://doi.org/10.1080/10350330.2018.1504732 [Google Scholar]
  12. Kessels, Erik
    2014In Almost Every Picture 13 — Attack Of The Giant Fingers. Amsterdam: Kesselskramer.
    [Google Scholar]
  13. Kress, Gunther and Theo Van Leeuwen
    2006Reading Images. The Grammar of Visual Design. Abingdon: Routledge.
    [Google Scholar]
  14. Kress, Gunther
    2009Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge. 10.4324/9780203970034
    https://doi.org/10.4324/9780203970034 [Google Scholar]
  15. Lazzeretti, Cecilia
    2023 “Exploring the Use of Emoji in Museum Social Network Sites.” Discourse, Context & Media, . 10.1016/j.dcm.2023.100697
    https://doi.org/10.1016/j.dcm.2023.100697 [Google Scholar]
  16. Lee, SooJin
    2018 “Emoji at MoMA: Considering the ‘original emoji’ as art.” First Monday(). 10.5210/fm.v23i9.9401
    https://doi.org/10.5210/fm.v23i9.9401 [Google Scholar]
  17. Linell, Per
    1998Approaching dialogue: Talk, interaction and contexts in dialogical perspectives. Amsterdam: John Benjamins. 10.1075/impact.3
    https://doi.org/10.1075/impact.3 [Google Scholar]
  18. Maier, Emar
    2023 “Emojis as Pictures.” Ergo: An Open Access Journal of Philosophy(): –.
    [Google Scholar]
  19. Martin, James R.
    2011 “Multimodal semiotics: Theoretical challenges.” InSemiotic margins: Meaning in multimodalities, ed. byShoshana Dreyfus, Maree Stenglin, and Susan Hood, –. London: Bloomsbury Publishing.
    [Google Scholar]
  20. Moya-Guijarro, Arsenio Jesús
    2024 “A Multimodal Analysis of Character-character Interaction in LGTB Picture Books and its Educational Implications.” Linguistics and Education: . 10.1016/j.linged.2024.101312
    https://doi.org/10.1016/j.linged.2024.101312 [Google Scholar]
  21. Painter, Clare, Jim Martin, and Len Unsworth
    2013Reading Visual Narratives: Image Analysis of Children’s Picture Books. Sheffield: Equinox Publishing Ltd.
    [Google Scholar]
  22. Ravelli, Louise J.
    2006 “Genre and the Museum Exhibition.” Linguistics and the Human Sciences(): –. 10.1558/lhs.v2i2.299
    https://doi.org/10.1558/lhs.v2i2.299 [Google Scholar]
  23. Sampietro, Agnese
    2016 “Exploring the Punctuating Effect of Emoji in Spanish Whatsapp Chats.” Lenguas Modernas: –.
    [Google Scholar]
  24. Seargeant, Philip
    2019The Emoji Revolution: How Technology is Shaping the Future of Communication. Cambridge: Cambridge University Press. 10.1017/9781108677387
    https://doi.org/10.1017/9781108677387 [Google Scholar]
  25. Sindoni, Maria Grazia
    2011Systemic-functional grammar and multimodal studies: An introduction with text analysis. Pavia: Ibis.
    [Google Scholar]
  26. 2019 “Mode-switching in Video-mediated Interaction: Integrating Linguistic Phenomena into Multimodal Transcription Tasks.” Linguistics and Education: –.
    [Google Scholar]
  27. Thomson, Samantha, Emily Kluftinger, and Jocelyn Wentland
    2018 “Are You Fluent in Sexual Emoji? 😉: Exploring the Use of Emoji in Romantic and Sexual Contexts.” The Canadian Journal of Human Sexuality(): –. 10.3138/cjhs.2018‑0020
    https://doi.org/10.3138/cjhs.2018-0020 [Google Scholar]
  28. Weigand, Edda
    2010 “Language as Dialogue.” Intercultural Pragmatics(): –. 10.1515/iprg.2010.022
    https://doi.org/10.1515/iprg.2010.022 [Google Scholar]
  29. Weissman, Benjamin
    2019 “Peaches and Eggplants or… Something Else? The Role of Context in Emoji Interpretations.” Proceedings of the Linguistic Society of America(): –. 10.3765/plsa.v4i1.4533
    https://doi.org/10.3765/plsa.v4i1.4533 [Google Scholar]
  30. Wicke, Philipp and Marianna Bolognesi
    2020 “Emoji-based Semantic Representations for Abstract and Concrete Concepts.” Cognitive Processing(): –. 10.1007/s10339‑020‑00971‑x
    https://doi.org/10.1007/s10339-020-00971-x [Google Scholar]
  31. Zappavigna, Michele and Lorenzo Logi
    2024Emoji and Social Media Paralanguage. Cambridge: Cambridge University Press. 10.1017/9781009179829
    https://doi.org/10.1017/9781009179829 [Google Scholar]
/content/journals/10.1075/ld.00211.laz
Loading
/content/journals/10.1075/ld.00211.laz
Loading

Data & Media loading...

  • Article Type: Research Article
Keywords: multimodality ; emoji ; multimodal dialogue ; social semiotics ; intermodal convergence
This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error