Volume 27, Issue 1
  • ISSN 0142-5471
  • E-ISSN: 1569-979X



Surprise Machines is a project of experimental museology that sets out to visualize the entire image collection of the Harvard Art Museums, with a view to opening up unexpected vistas on more than 200,000 objects usually inaccessible to visitors. The project is part of the exhibition organized by metaLAB (at) Harvard entitled Curatorial A(i)gents and explores the limits of artificial intelligence to display a large set of images and create surprise among visitors. To achieve this feeling of surprise, a choreographic interface was designed to connect the audience’s movement with several unique views of the collection.

Available under the CC BY 4.0 license.

Article metrics loading...

Loading full text...

Full text loading...



  1. American Museum of Natural History (2022) Collectionscope [JavaScript]. https://github.com/amnh-sciviz/collectionscope (Original work published 2020)
    [Google Scholar]
  2. Barabási, A.-L. , Bello, M. , Kluge-Fabényi, J. , Forde, K. , Készman, J. , Meirelles, I. , Ratti, C. G. , Ritchie, M. , & Szántó, A. (2020) Hidden patterns: Visualizing networks at BarabásiLab ( A. Stang & P. Weibel , Eds.). Hatje Cantz Verlag.
    [Google Scholar]
  3. Benedetti, A. (2022, January5). Browsing and visualizing collections of images. https://www.youtube.com/watch?app=desktop&v=bprSgiqZj00undefined
    [Google Scholar]
  4. Birkle, C. , & Däwes, B. (2019) “Old media don’t go away, they mutate”: An interview with Jeffrey Schnapp. Amerikastudien/American Studies, 64 (1), 111–125. 10/ghm6cs
    https://doi.org/10/ghm6cs [Google Scholar]
  5. Bludau, M.-J. , Dörk, M. , & Heidmann, F. (2021) Relational perspectives as situated visualizations of art collections. Digital Scholarship in the Humanities, 36 (Supplement_2), ii17–ii29. 10.1093/llc/fqab003
    https://doi.org/10.1093/llc/fqab003 [Google Scholar]
  6. Börner, K. , Maltese, A. , Balliet, R. N. , & Heimlich, J. (2016) Investigating aspects of data visualization literacy using 20 information visualizations and 273 science museum visitors. Information Visualization, 15 (3), 198–213. 10.1177/1473871615594652
    https://doi.org/10.1177/1473871615594652 [Google Scholar]
  7. Bostock, M. , Ogievetsky, V. , & Heer, J. (2011) D3: Data-Driven Documents. IEEE Transactions on Visualization and Computer Graphics, 17 (12), 2301–2309. 10.1109/TVCG.2011.185
    https://doi.org/10.1109/TVCG.2011.185 [Google Scholar]
  8. Champion, E. M. (2016) Digital humanities is text heavy, visualization light, and simulation poor. Digital Scholarship in the Humanities, i25–i32. 10.1093/llc/fqw053
    https://doi.org/10.1093/llc/fqw053 [Google Scholar]
  9. Crawford, K. , & Paglen, T. (2019) Excavating AI: the politics of images in machine learning training sets. AI Now Institute. https://excavating.ai
    [Google Scholar]
  10. Crockett, D. (2019) IVPY: iconographic visualization inside computational notebooks. International Journal for Digital Art History, 4 1, 3.60–3.79. 10.11588/DAH.2019.4.66401
    https://doi.org/10.11588/DAH.2019.4.66401 [Google Scholar]
  11. Cuno, J. B. (Ed.) (1996) Harvard’s art museums: 100 years of collecting. Harvard University Museums.
    [Google Scholar]
  12. Danchilla, B. (2012) Beginning WebGL for HTML5. Apress. 10.1007/978‑1‑4302‑3997‑0
    https://doi.org/10.1007/978-1-4302-3997-0 [Google Scholar]
  13. Derry, L. , Kruguer, J. , Muelle, M. , & Schnapp, J. (2022) Designing a choreographic interface during covid-19. Movement and Computing Conference. 10.1145/3537972.3538020
    https://doi.org/10.1145/3537972.3538020 [Google Scholar]
  14. Diagne, C. , Barradeau, N. , & Doury, S. (2018) T-SNE Map. Experiments with Google. https://experiments.withgoogle.com/t-sne-map
    [Google Scholar]
  15. DiMaggio, P. , & Hargittai, E. (2001) From the “digital divide” to “digital inequality”: Studying internet use as penetration increases. Center for Arts and Cultural Policy Studies, Princeton University.
    [Google Scholar]
  16. Drucker, J. (2013) Performative materiality and theoretical approaches to interface. Digital Humanities Quarterly, 7 (1). www.digitalhumanities.org/dhq/vol/7/1/000143/000143.html
    [Google Scholar]
  17. Duhaime, D. (2021) PixPlot. Yale Digital Humanities Lab. https://github.com/YaleDHLab/pix-plot (Original work published 2017)
    [Google Scholar]
  18. Foster, H. (2011) The art-architecture complex. Verso.
    [Google Scholar]
  19. Geismar, H. (2018) Museum object lessons for the digital age. UCL Press. 10.2307/j.ctv1xz0wz
    https://doi.org/10.2307/j.ctv1xz0wz [Google Scholar]
  20. Harvard Art Museums (2012) International Image Interoperability Framework at Harvard University. https://iiif.harvard.edu/
    [Google Scholar]
  21. Impett, L. , & Moretti, F. (2017) Totentanz. Operationalizing Aby Warburg’s Pathosformeln. New Left Review, 107 1, 68–97.
    [Google Scholar]
  22. Jacomy, M. , Venturini, T. , Heymann, S. , & Bastian, M. (2014) ForceAtlas2, a continuous graph layout algorithm for handy network visualization designed for the Gephi software. PLoS ONE, 9 (6), e98679. 10.1371/journal.pone.0098679
    https://doi.org/10.1371/journal.pone.0098679 [Google Scholar]
  23. Jodidio, P. (2014) Piano: Renzo Piano building workshop, complete works, 1966-today. Taschen.
    [Google Scholar]
  24. Karjus, A. , Solà, M. C. , Ohm, T. , Ahnert, S. E. , & Schich, M. (2022) Compression ensembles quantify aesthetic complexity and the evolution of visual art. CitetononCRdoi:10.48550/ARXIV.2205.10271
    https://doi.org/Cite to nonCR doi: 10.48550/ARXIV.2205.10271 [Google Scholar]
  25. Kenderdine, S. , Mason, I. , & Hibberd, L. (2021) Computational archives for experimental museology, 3–18. 10.1007/978‑3‑030‑83647‑4_1
    https://doi.org/10.1007/978-3-030-83647-4_1 [Google Scholar]
  26. Klinke, H. (2021, October28). V-lab workshop: Visual analysis in cultural data – image plots and t-SNE maps made easy. History of Art Department. https://arthistory.berkeley.edu/events/v-lab-workshop-visual-analysis-in-cultural-data-image-plots-and-t-sne-maps-made-easy/
    [Google Scholar]
  27. Kräutli, F. (2016) Visualising cultural data: Exploring digital collections through timeline visualisations. Royal College of Art.
    [Google Scholar]
  28. Latour, B. (1988) The pasteurization of France ( A. Sheridan & J. Law , Trans.; English edition). Harvard University Press.
    [Google Scholar]
  29. Lima, M. (2011) Visual complexity: Mapping patterns of information. Princeton Architectural Press.
    [Google Scholar]
  30. Maaten, L. van der , & Hinton, G. (2008) Visualizing data using t-SNE. Journal of Machine Learning Research, 9 (86), 2579–2605. jmlr.org/papers/v9/vandermaaten08a.html
    [Google Scholar]
  31. Maizels, M. , & Qiu, C. (Eds.) (2020) Curatorial A(i)gents. metaLAB (at) Harvard. https://www.printedmatter.org/catalog/57243/
    [Google Scholar]
  32. Manovich, L. (2008) Data visualization as new abstraction and anti-sublime. In B. Hawk , D. M. Rieder , & O. O. Oviedo (Eds.), Small tech: The culture of digital tools. University of Minnesota Press.
    [Google Scholar]
  33. Manovich, L. (2020) Cultural analytics. MIT Press. 10.7551/mitpress/11214.001.0001
    https://doi.org/10.7551/mitpress/11214.001.0001 [Google Scholar]
  34. Manzini, E. (2016) Design culture and dialogic design. Design Issues, 32 (1), 52–59. 10.1162/DESI_a_00364
    https://doi.org/10.1162/DESI_a_00364 [Google Scholar]
  35. McCully, E. A. (2019) Dreaming in code: Ada Byron Lovelace, computer pioneer. Candlewick Press.
    [Google Scholar]
  36. McInnes, L. , Healy, J. , & Melville, J. (2018) UMAP: Uniform Manifold Approximation and Projection for dimension reduction. ArXiv.Org, stat.ML. https://arxiv.org/pdf/1802.03426.pdf
    [Google Scholar]
  37. metaLAB (2022) MetaLAB (at) Harvard & FU Berlin. https://mlml.io/
    [Google Scholar]
  38. Moon, C. Y. E. , & Rodighiero, D. (2020) Mapping as a contemporary instrument for orientation in conferences. Atti Del IX Convegno Annuale AIUCD. La Svolta Inevitabile: Sfide e Prospettive per l’informatica Umanistica. CitetononCRdoi:10.5281/zenodo.3611340
    https://doi.org/Cite to nonCR doi: 10.5281/zenodo.3611340 [Google Scholar]
  39. O’Shea, K. , & Nash, R. (2015) An introduction to convolutional neural networks. ArXiv:1511.08458 [Cs]. arxiv.org/abs/1511.08458
    [Google Scholar]
  40. Pietsch, C. (2022) VIKUS viewer [JavaScript]. https://github.com/cpietsch/vikus-viewer (Original work published 2018)
    [Google Scholar]
  41. Rodighiero, D. (2021a) Mapping affinities: Democratizing data visualization (Open-access English edition). Métis Presses. 10.37866/0563‑99‑9
    https://doi.org/10.37866/0563-99-9 [Google Scholar]
  42. Rodighiero, D. (2021b, August18). Ars memorativa as the genesis of information design: A conversation with Manuel Lima. Nightingale. https://nightingaledvs.com/ars-memorativa-as-the-genesis-of-information-design-a-conversation-with-manuel-lima/. 10.31235/osf.io/k5unq
    https://doi.org/10.31235/osf.io/k5unq [Google Scholar]
  43. Rodighiero, D. , Wandl-Vogt, E. , & Carsenat, E. (2021) Making visible the invisible work of scientists during the Covid-19 pandemic. Visual Culture Studies, 2 1. 10.31235/osf.io/m4uht
    https://doi.org/10.31235/osf.io/m4uht [Google Scholar]
  44. Rodighiero, D. , Wandl-Vogt, E. , & Carsenat, E. (2022) A visual translation of the pandemic. Leonardo, 55 (3). 10.1162/leon_a_02203
    https://doi.org/10.1162/leon_a_02203 [Google Scholar]
  45. Russakovsky, O. , Deng, J. , Su, H. , Krause, J. , Satheesh, S. , Ma, S. , Huang, Z. , Karpathy, A. , Khosla, A. , Bernstein, M. , Berg, A. C. , & Fei-Fei, L. (2015) Imagenet large scale visual recognition challenge. ArXiv:1409.0575 [Cs]. arxiv.org/abs/1409.0575. 10.1007/s11263‑015‑0816‑y
    https://doi.org/10.1007/s11263-015-0816-y [Google Scholar]
  46. Seguin, B. (2018) The Replica project: Building a visual search engine for art historians. XRDS: Crossroads, The ACM Magazine for Students, 24 (3), 24–29. 10.1145/3186653
    https://doi.org/10.1145/3186653 [Google Scholar]
  47. Steward, J. (2021) API documentation. Harvard Art Museums. https://github.com/harvardartmuseums/api-docs (Original work published 2015)
    [Google Scholar]
  48. Turing, A. M. (1950) Computing machinery and intelligence. Mind, LIX (236), 433–460. 10.1093/mind/LIX.236.433
    https://doi.org/10.1093/mind/LIX.236.433 [Google Scholar]
  49. Vane, O. (2019) Timeline design for visualising cultural heritage data. Royal College of Art.
    [Google Scholar]
  50. Weaver, W. (1948) Science and complexity. American Scientist, 36 (4), 536–544. JSTOR. www.jstor.org/stable/27826254
    [Google Scholar]

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error