Volume 27, Issue 1
  • ISSN 0142-5471
  • E-ISSN: 1569-979X



Qualitative uncertainty refers to the implicit and underlying issues that are imbued in data, such as the circumstances of its collection, its storage or even biases and assumptions made by its authors. Although such uncertainty can jeopardize the validity of the data analysis, it is often overlooked in visualizations, due to it being indirect and non-quantifiable. In this paper we present two case studies within the digital humanities in which we examined how to integrate uncertainty in our visualization designs. Using these cases as a starting point we propose four considerations for data visualization research in relation to indirect, qualitative uncertainty: (1) we suggest that uncertainty in visualization should be examined within its socio-technological context, (2) we propose the use of interaction design patterns to design for it, (3) we argue for more attention to be paid to the data generation process in the humanities, and (4) we call for the further development of participatory activities specifically catered for understanding qualitative uncertainties. While our findings are grounded in the humanities, we believe that these considerations can be beneficial for other settings where indirect uncertainty plays an equally prevalent role.

Available under the CC BY-NC 4.0 license.

Article metrics loading...

Loading full text...

Full text loading...



  1. Boukhelifa, N., Perrin, M. E., Huron, S., & Eagan, J.
    (2017) How Data Workers Cope with Uncertainty: A Task Characterisation Study. Proceedings of the 2017 ACM Conference on Human Factors in Computing Systems (CHI) 2017–May, 3645–3656. 10.1145/3025453.3025738
    https://doi.org/10.1145/3025453.3025738 [Google Scholar]
  2. Boyd Davis, S., Vane, O., & Kräutli, F.
    (2021) Can I believe what I see? Data visualization and trust in the humanities. Interdisciplinary Science Reviews, 46(4), 522–546. 10.1080/03080188.2021.1872874
    https://doi.org/10.1080/03080188.2021.1872874 [Google Scholar]
  3. Braun, S.
    (2018) Critically Engaging with Data Visualization through an Information Literacy Framework. DHQ: Digital Humanities Quarterly, 12(4).
    [Google Scholar]
  4. Correll, M.
    (2019) Ethical Dimensions of Visualization Research. Proceedings of the 2019 ACM Conference on Human Factors in Computing Systems (CHI), 1–13. 10.1145/3290605.3300418
    https://doi.org/10.1145/3290605.3300418 [Google Scholar]
  5. D'Ignazio, C., & Klein, L.
    (2020) The Power Chapter. InData Feminism (pp.1–39). Retrieved fromhttps://datafeminism.mitpress.mit.edu/pub/vi8obxh7. 10.7551/mitpress/11805.003.0003
    https://doi.org/10.7551/mitpress/11805.003.0003 [Google Scholar]
  6. D’Ignazio, C.
    (2017) Creative data literacy: Bridging the Gap Between the Data-haves and Data-have Nots. Information Design Journal, 23(1), 6–18.
    [Google Scholar]
  7. Deitrick, S.
    (2012) Evaluating Implicit Visualization of Uncertainty for Public Policy Decision Support. Proceedings AutoCarto 2012, (Abbasi 2005), 161.
    [Google Scholar]
  8. Dimara, E., & Perin, C.
    (2020) What is Interaction for Data Visualization?IEEE Transactions on Visualization and Computer Graphics, 26(1), 119–129. 10.1109/TVCG.2019.2934283
    https://doi.org/10.1109/TVCG.2019.2934283 [Google Scholar]
  9. Dörk, M., Feng, P., Collins, C., & Carpendale, S.
    (2013) Critical InfoVis. CHI ’13 Extended Abstracts on Human Factors in Computing Systems on – CHI EA ’13, 2189. 10.1145/2468356.2468739
    https://doi.org/10.1145/2468356.2468739 [Google Scholar]
  10. Drucker, J.
    (2011) Humanities Approaches to Graphical Display. DHQ: Digital Humanities Quarterly, 5(1).
    [Google Scholar]
  11. (2015) Graphical Approaches to the Digital Humanities. InA New Companion to Digital Humanities. 10.1002/9781118680605.ch17
    https://doi.org/10.1002/9781118680605.ch17 [Google Scholar]
  12. Franke, M., Barczok, R., Koch, S., & Weltecke, D.
    (2019) Confidence as First-class Attribute in Digital Humanities Data. Proceedings of the Workshop on Visualization for the Digital Humanities (VIS4DH).
    [Google Scholar]
  13. Glinka, K., Pietsch, C., Dörk, M., & Marian Dörk
    (2017) Past Visions and Reconciling Views: Visualizing Time, Texture and Themes in Cultural Collections. DHQ: Digital Humanities Quarterly, 11(2), 1–19.
    [Google Scholar]
  14. Greis, M., Avci, E., Schmidt, A., & Machulla, T.
    (2017) Increasing Users’ Confidence in Uncertain Data by Aggregating Data from Multiple Sources. Proceedings of the 2017 ACM Conference on Human Factors in Computing Systems (CHI) 2017–May, 828–840. 10.1145/3025453.3025998
    https://doi.org/10.1145/3025453.3025998 [Google Scholar]
  15. Hall, K. W., Bradley, A. J., Hinrichs, U., Huron, S., Wood, J., Collins, C., & Carpendale, S.
    (2019) Design by Immersion: A Transdisciplinary Approach to Problem-Driven Visualizations. IEEE Transactions on Visualization and Computer Graphics, 26(1), 109–118. 10.1109/TVCG.2019.2934790
    https://doi.org/10.1109/TVCG.2019.2934790 [Google Scholar]
  16. Kale, A., Kay, M., & Hullman, J.
    (2019) Decision-Making Under Uncertainty in Research Synthesis: Designing for the Garden of Forking Paths. Proceedings of the 2019 ACM Conference on Human Factors in Computing Systems (CHI), 1–14. 10.1145/3290605.3300432
    https://doi.org/10.1145/3290605.3300432 [Google Scholar]
  17. Kale, A., Nguyen, F., Kay, M., & Hullman, J.
    (2019) Hypothetical Outcome Plots Help Untrained Observers Judge Trends in Ambiguous Data. IEEE Transactions on Visualization and Computer Graphics, 25(1), 892–902. 10.1109/TVCG.2018.2864909
    https://doi.org/10.1109/TVCG.2018.2864909 [Google Scholar]
  18. Kennedy, H., Hill, R. L., Aiello, G., & Allen, W.
    (2016) The Work that Visualisation Conventions Do. Information Communication and Society, 19(6), 715–735. 10.1080/1369118X.2016.1153126
    https://doi.org/10.1080/1369118X.2016.1153126 [Google Scholar]
  19. Kerzner, E., Goodwin, S., Dykes, J., Jones, S., & Meyer, M.
    (2018) A Framework for Creative VisualizationOpportunities Workshops. IEEE Transactions on Visualization and Computer Graphics, 25(1), 748–758. 10.1109/TVCG.2018.2865241
    https://doi.org/10.1109/TVCG.2018.2865241 [Google Scholar]
  20. Kitchin, R.
    (2021) Data Lives: How Data Are Made and Shape Our World. Policy Press.
    [Google Scholar]
  21. Lee, C., Yang, T., Inchoco, G., Jones, G. M., & Satyanarayan, A.
    (2021) Viral Visualizations: How Coronavirus Skeptics Use Orthodox Data Practices to Promote Unorthodox Science Online. Proceedings of the 2021 ACM Conference on Human Factors in Computing Systems (CHI). 10.1145/3411764.3445211
    https://doi.org/10.1145/3411764.3445211 [Google Scholar]
  22. Loukissas, Y.
    (2019) All Data are Local: Thinking Critically in a Data-Driven Society. MIT Press. 10.7551/mitpress/11543.001.0001
    https://doi.org/10.7551/mitpress/11543.001.0001 [Google Scholar]
  23. Manovich, L.
    (2011) What is Visualisation?Visual Studies, 26(1), 36–49. 10.1080/1472586X.2011.548488
    https://doi.org/10.1080/1472586X.2011.548488 [Google Scholar]
  24. Marai, G. E.
    (2018) Activity-Centered Domain Characterization for Problem-Driven Scientific Visualization. IEEE Transactions on Visualization and Computer Graphics, 24(1), 913–922. 10.1109/TVCG.2017.2744459
    https://doi.org/10.1109/TVCG.2017.2744459 [Google Scholar]
  25. McAllister, J. W.
    (2018) Scientists’ Reuse of Old Empirical Data: Epistemological Aspects. Philosophy of Science, 85(5), 755–766. 10.1086/699695
    https://doi.org/10.1086/699695 [Google Scholar]
  26. McCurdy, N., Gerdes, J., & Meyer, M.
    (2019) A Framework for Externalizing Implicit Error Using Visualization. IEEE Transactions on Visualization and Computer Graphics, 25(1), 925–935. 10.1109/TVCG.2018.2864913
    https://doi.org/10.1109/TVCG.2018.2864913 [Google Scholar]
  27. Meyer, M., & Dykes, J.
    (2019) Criteria for Rigor in Visualization Design Study. IEEE Transactions on Visualization and Computer Graphics, 26(1), 87–97. 10.1109/TVCG.2019.2934539
    https://doi.org/10.1109/TVCG.2019.2934539 [Google Scholar]
  28. Moretti, F.
    (2005) Graphs, Maps, Trees: Abstract Models for a Literary History.
    [Google Scholar]
  29. Padilla, L., Kay, M., & Hullman, J.
    (2020) Uncertainty Visualization. Handbook of Computational Statistics and Data Science. 10.31234/osf.io/ebd6r
    https://doi.org/10.31234/osf.io/ebd6r [Google Scholar]
  30. Panagiotidou, G., Poblome, J., Aerts, J., & Vande Moere, A.
    (2022) Designing a Data Visualisation for Interdisciplinary Scientists: How to Transparently Convey Data Frictions?Computer Supported Cooperative Work (CSCW). 10.1007/s10606‑022‑09432‑9
    https://doi.org/10.1007/s10606-022-09432-9 [Google Scholar]
  31. Panagiotidou, G., Vandam, R., Poblome, J., & Vande Moere, A.
    (2021) Implicit Error, Uncertainty and Confidence in Visualization: an Archaeological Case Study. IEEE Transactions on Visualization and Computer Graphics, 1–12. 10.1109/TVCG.2021.3088339
    https://doi.org/10.1109/TVCG.2021.3088339 [Google Scholar]
  32. Sacha, D., Senaratne, H., Kwon, B. C., Ellis, G., & Keim, D. A.
    (2016) The Role of Uncertainty, Awareness, and Trust in Visual Analytics. IEEE Transactions on Visualization and Computer Graphics, 22(1), 240–249. 10.1109/TVCG.2015.2467591
    https://doi.org/10.1109/TVCG.2015.2467591 [Google Scholar]
  33. Stoffel, F., Jentner, W., Behrisch, M., Fuchs, J., & Keim, D.
    (2017) Interactive Ambiguity Resolution of Named Entities in Fictional Literature. Computer Graphics Forum, 36(3), 189–200. 10.1111/cgf.13179
    https://doi.org/10.1111/cgf.13179 [Google Scholar]
  34. Tak, S., Toet, A., & Van Erp, J.
    (2014) The Perception of Visual Uncertainty Representation by Non-experts. IEEE Transactions on Visualization and Computer Graphics, 20(6), 935–943. 10.1109/TVCG.2013.247
    https://doi.org/10.1109/TVCG.2013.247 [Google Scholar]
  35. Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., & Spiegelhalter, D. J.
    (2019) Communicating Uncertainty about Facts, Numbers and Science. Royal Society Open Science, 6(5). 10.1098/rsos.181870
    https://doi.org/10.1098/rsos.181870 [Google Scholar]
  36. Windhager, F., Salisu, S., & Mayr, E.
    (2019) Exhibiting Uncertainty: Visualizing Data Quality Indicators for Cultural Collections. Informatics, 6(3). 10.3390/informatics6030029
    https://doi.org/10.3390/informatics6030029 [Google Scholar]
  37. Zimmerman, A.
    (2007) Not by metadata alone: The Use of Diverse Forms of Knowledge to Locate Data for Reuse. International Journal on Digital Libraries, 7(1–2), 5–16. 10.1007/s00799‑007‑0015‑8
    https://doi.org/10.1007/s00799-007-0015-8 [Google Scholar]

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error