HASTAC 2023 Conference

This paper identifies potential educational implications of teaching and learning with augmented reality (AR). We will present results from classroom testing of Booksnake, a mobile app we are developing that uses AR to transform existing digitized archival materials (DAM) for embodied exploration. We are comparing student interactions with archival materials across three modalities: (a) using Booksnake and AR to interact with DAM in physical space, (b) using traditional Web-based viewers to interact with DAM on flat screens, and (c) using a class visit to Special Collections to interact with physical archival materials in physical space.

Our guiding research questions are:
(1) Does student motivation to interact with DAM vary depending on interface (flat screens versus AR)?
(2) Does student motivation to interact with archival materials vary depending on whether they are interacting with physical archival materials versus digitized archival materials?
(3) What difficulties or barriers does AR technology present for use as an educational tool in classroom settings?

Our classroom testing is currently in progress at the University of Southern California and at California State University, San Marcos. HASTAC will be the first venue in which we publicly discuss the results of large-scale classroom testing.

The Booksnake project starts from the premise that close engagement with primary sources is foundational to humanities teaching and learning (Falbo 2000, Schmiesing & Hollis 2002, Toner 1993). Digitization initiatives at galleries, libraries, archives, and museums have brought millions of archival materials online (Rosenzweig 2003, Solberg 2012, Ramsay 2014, Putnam 2016). But interacting with digitized materials in a Web browser fails to replicate the close engagement possible during in-person research (Nolan 2013, Szpiech 2014, Kropf 2017, van Lit 2020).
In contrast, a user treats Booksnake like a lens, looking through their device’s screen at a virtual object blended into the physical world. This process, known as augmented reality (AR), makes it possible to interact with virtual objects in physical space (Azuma et al. 2001). Booksnake works by using the International Image Interoperability Framework (IIIF) to download a digitized item’s images and metadata. Booksnake next creates a custom virtual object replicating the physical item’s dimensions and appearance, then dynamically inserts this virtual object into the live camera view on a mobile phone or tablet. As the user moves, Booksnake continually adjusts the virtual object’s relative position and size in the camera view, so the virtual object appears to remain stationary in physical space. (The project takes its name from book snakes, the weighted fabric tubes used to hold archival material in place.)

Booksnake has particular potential to support student learning through movement, perspective, and scale. Embodied interaction is key to apprehending cultural heritage materials in their full complexity (Kai-Kee, Latina & Sadoyan 2020) and systematic reviews show AR can support student learning gains, motivation, and knowledge transfer (Bacca et al. 2014). Virtualization technologies like AR are especially effective for educational scenarios that would otherwise be rare, impractical, destructive, or expensive (Bailenson 2018).

Booksnake utilizes existing mobile devices, lowering barriers to access and use. By virtualizing the experience of embodied interaction, Booksnake makes it possible for people who cannot visit an archive (due to cost, distance, disability, or other reasons) to physically engage with digitized materials. But, unless thoughtfully developed, technologies like Booksnake also risk entrenching existing archival biases (because Booksnake can only present already-digitized materials) and widening the digital divide (because Booksnake requires access to a smartphone or tablet).

Finally, Booksnake asks us to think creatively about mobile devices as platforms for innovative humanities pedagogy. Booksnake is a suggestive experiment in how embodied interaction and spatial interfaces can activate archival collections, enhance discoverability, and improve accessibility.