Exploring Gigapixel Image Environments for Science Communication and Learning in Museums


Ahmed Ansari, USA, Illah Nourbakhsh, USA, Chris Bartley, USA, Marti Louw, USA

Abstract

This paper describes an ongoing collaboration between the Carnegie Museum of Natural History, Carnegie Mellon University, and the University of Pittsburgh to explore and study how emerging gigapixel image technology can be used to support science communication and learning in natural history museums. Multitouch based explorations of dynamic visual information spaces present unique user interface and interaction design opportunities and challenges. This case study focuses on a human centered design approach to technology innovation, by describing the design research methodologies used and the series of prototypes developed. We share findings from a series of user studies to optimize navigation, annotation and exploration in a zoomable user interface, and highlight the unique affordances of multiscalar image platforms to scaffold disciplinary observation and engagement with scientific content. Lastly, we discuss future design directions and offer some frameworks that can be adopted by museums when seeking to incorporate explorable, high-resolution imagery into exhibit experiences, both on the museum floor and on-line.

Keywords: Interaction design, high-resolution images, gigapixel, multiscalar, zoomable user interfaces, (ZUIs), annotations, petroglyphs, natural history museums, collections, science communication

1. Background

Creating rich learning environments with gigapixel technology

Gigapixel image technology is an emerging class of explorable interactive media. High-resolution images embedded in zoomable visual environments enable users to move from full panoramic views to incredibly deep close-ups often comprising of billions of pixels. These multiscalar image environments are most familiar in online mapping and geographic browsing applications such as Yahoo! Maps and Google Earth, but the information they can organize extends well beyond maps, including composite photographic images and even computationally rendered data visualizations. Deep zooming, or zoomable user interfaces (ZUIs), offer a spatial way to display and organize large amounts of information in a single interface using scroll, pan, and zoom controls (Bederson, 2011). Text, images, graphics, audio, and video can be embedded at spatial locations and zoom levels within an image, creating localized sites for commenting and conversation (Drucker et al., 2008).  Compared to earlier forms of interactive media, this technology affords users deeper navigational agency and exploratory control in a shared visual space.

Gigapixel image technology initially gained popularity with the photography and scientific communities enthused by a newfound ability to easily document and present subject matter with an extremely high degree of image resolution and fidelity (see the Fine Foundation Blog for Science Research). Increasingly, gigapixel image platforms are being used by museums and media professionals to facilitate public engagement and learning experiences with visual data. Media design groups like Ideum and xRez are developing multitouch viewers and gesture-based interfaces for navigating and exploring deep zoom image spaces. Through its Digital Narratives initiative, Microsoft Research is developing new user-driven narrative controls and viewers that enhance the storytelling aspects of inherently nonlinear rich interactive media. The Applied Laboratory for Interactive Visualization and Embodiment (ALiVE) of Hong Kong’s City University is designing and testing  large-scale architectural solutions to the problem of presenting  immersive, interactive, whole-body kinesthetic experiences to museum-goers using museums’ large-scale visual datasets of landscapes, archaeological sites and sacred texts (Kenderdine, 2010). Central to all these projects is the effort to invent new interfaces for interacting with explorable images, pushing beyond unmediated free exploration and into the realm of rich mediation and socially engaged, shared exploration.

Our project team has focused on design research that uncovers the affordances of gigapixel environments to support deep observation, engagement and learning. In a recent paper, Louw and Crowley describe the potential for multiscalar image platforms to support participatory learning experiences around visual evidence, highlighting the distinctive attributes of these networked visual environments for engaged science communication and learning (Louw & Crowley, 2013). In terms of museum applications, gigapixel image-based platforms for learning exhibit five promising affordances:

  1. Deep looking and noticing in a shared observational space: Users can move seamlessly from the macro to the micro within an image, enabling unseen details to emerge, and encouraging exploration and observation. In her article discussing the impact of the Google Art Project, which showcases high-resolution online reproductions of artworks from over 46 museums around the world, The Smithsonian’s Nancy Proctor points to the incredibly positive responses from museums, and notes that these online engagements with gigapixel images offer “intimate encounters with images at visual depths not possible even in the galleries” (Proctor, 2011).
  2. Democratizing a tool of science: Gigapixel image technology makes the process of taking, storing, sharing, and interpreting high-resolution images—a domain once the purview of surveillance and defense industries—available to the general public. The hardware and software required to easily create high-resolution images is now commercially available, with much of the code open source. Websites like GigaPan.org and Photosynth.net invite gigapixel image makers from all over the world to upload their content to be viewed, annotated, geolocated, commented on, and shared globally.
  3. Encouraging participatory learning: As a networked platform, the technology generates new possibilities for participatory learning and direct communication between and among disciplinary experts and the public, through crowdsourcing, open forums, and sharing of observations and interpretation. In 2010, the North Carolina State University Insect Museum began using gigapixel image technology to document their extensive insect collection, and has released over 2,700 panoramas of rare and fragile specimen drawers to the general public online. In a paper documenting the results and insights from the project, the NCSU team reported that the platform afforded them a new kind of public engagement opportunity whereby museum scientists and users could interact and have conversational exchanges about insect biology. (Bertone et al., 2012).
  4. Offering new visuospatial ways to curate collections and environments: Multiscalar images enable content to be organized and accessed visually along an explorable spatial dimension. Thus, depth and distance cues in visual perception can be used to order information, in addition to more traditional means of organizing exhibit content (thematic, chronological, geographic, hierarchical, etc.) to guide and encourage close looking. The Nature Valley Trail View is a prime example of an explorable visuospatial environment made possible by combining underlying image browsing technologies such as Streetview and GigaPan with geospatial data maps in unified visual information space. This project enables users to virtually explore and walk along trails at the Grand Canyon, Great Smoky Mountains and Yellowstone National Parks. Along the way contextual “call outs” provide additional, interactive media overlays for a more dynamic experience.
  5. Enabling context-dependent annotations and mediation: Embedded information can be revealed depending on user interactions and locations within a three-dimensional image space, dynamically tying information to user exploration (Luan et al. 2008). A playful example of this type of context-dependent mediation can be found on the portfolio website of the Canadian design firm Castor, which displays clever media-rich annotations as a user zooms in to see details of various inspirational objects on display.

Clearly, a gigapixel media environment has the potential to extend how a museum visitor approaches and engages with interactive media. But to date there has been little research or evaluation work done assessing the merits of this type of technology platform for learning and engagement in museum contexts. Facilitating scientific or disciplinary based forms of observation so that museumgoers begin to engage with artifacts and collections more critically, analytically, and systematically is a challenging prospect that requires specific scaffolding and mediation (Eberbach & Crowley, in press; Louw & Crowley, 2013), and there are considerable interaction and information design challenges that zoomable user interfaces present (Bederson, 2008).

Museum exhibit experiences must also account for myriad interests, needs, goals and expectations that the public brings to these settings (Falk and Sheppard, 2006). Asnatural’ and gesture-based user interfaces become the standard (Gammon & Burch, 2008), the question becomes vitally important of how to develop intuitive interaction spaces that cater to disparate types of users, giving them deeper agency and choice in how to move through content in ways that are personally relevant and support coherent meaning making (Kidd, Ntalla & Lyons, 2011). The following case study describes an effort to meet these design challenges in the context of a specific gigapixel application: the exploration, observation and appreciation of petroglyphs in a natural history museum setting.

2. Case Study: Stories in the Rock

Exploring the design of gigapixel image viewer for a museum environment

The University of Pittsburgh Center for Learning in Out-of-School Environments (UPCLOSE), in collaboration with Carnegie Mellon University and Carnegie Museum of Natural History, is developing an NSF-funded set of demonstration projects to study the affordances of gigapixel based image technology for science communication and learning.  Our work is guided by three distinct approaches to designing engaging public-science learning experiences: Public Understanding of Science, Public Participation in Scientific Research, and Public Engagement in Science, each of which are described in the report “Many experts, many audiences: public engagement with science and informal science education” published by the Center for the Advancement of Informal Science Education (McCallie et al. 2009).  The following case study presents the first of three demonstration projects and highlights the ‘Public Understanding of Science’ engagement approach which emphasizes knowledge building and activities designed to enable learners to shape their own inquiry and actively construct meaning.

Dr. Sandra Olsen, Head of Anthropology, and Director of the Center for World Cultures at the Carnegie Museum of Natural History, leads the Arabian Rock Art Heritage Project with a multinational team of researchers investigating a set of little known petroglyph sites found in remote regions of the Saudi Arabian desert. Etched into large rock faces are remarkable scenes of prehistoric people hunting wild animals, herding cattle, riding camels, and engaging in warfare on horseback. Some of these ancient rock art engravings date back to the Holocene Wet Phase (8,000 BCE–3,500 BCE), when the environment in the Arabian Peninsula was more like a savannah. During these expeditions, Dr. Olsen’s team documented dozens of rock art sites using GigaPan composite photography, texture mapping, and three-dimensional laser scanning. These visual data resources function not only as research evidence, but also work as fascinating windows into remote and inaccessible cultural sites of great importance; and they served as the inspiration for our Stories in the Rock project.

The Stories in the Rock demonstration project has three central aims. First, to study applications of multiscalar image environments to communicate and engage visitors in museum research and collections. Second, to support disciplinary-based ways of looking, noticing and observing features in an ancient petroglyph site. Third, to support user engagement through self-directed exploration and agency.

We next present a summary of our design research process and findings. The activities build towards fulfilling project goals, framing the possibilities and limits of gigapixel image interactions in a museum exhibit setting, generating frameworks and design concepts that meet both usability and learning objectives, and finally developing an iterative series of working prototypes that we deployed for testing and evaluation.

Phase I | Exploratory research

The initial project phase focused on formative design research to understand user and stakeholder goals, needs, and expectations, as well as research to define user interaction scenarios by observing engagement with existing gigapixel viewer-based technology. We used a combination of methods including individual interviews, think-aloud protocols (Lewis, 1982), a card sort activity (Nielsen, Sano, 1994), heuristic evaluation techniques (Nielsen, 1992), and contextual inquiry methods (Holtzblatt, 2004) to elicit and understand user needs and desires and to evaluate existing systems. The collected data was then synthesized and analyzed to establish key findings and insights, and several theoretical design frameworks were applied to help generate a set of solutions for an interactive exhibit experience based on explorations, interactions and mediation of high-resolution images.

Understanding curator needs & goals

A central goal of this project lies in facilitating communication between scientists and publics. As a starting point, we needed to learn more about the subject matter, its interpretation, and the curator’s communication and public engagement goals. To do this we conducted several semi-structured interviews with Dr. Olsen, and used a modified contextual inquiry method to understand her observational and interpretation practices, as well as how she utilizes gigapixel image data in her research. To support this inquiry, we printed a large 3” x 6” photograph of the Shuwaymis West petroglyph site as a prompt, and used post-it notes to spatially mark points of interest and identify important topics. This physical exercise functioned both as a way to capture rich information, and as means to begin thinking about translating scientific interpretation into a mediated exhibit experience

Findings

One of the insights that emerged from this work was the curator’s need to not only examine petroglyph scenes contained within a single panel, but also to make connections between images of different sites to illustrate, for example, carving tools and techniques, evolving representational strategies, and changing depictions of animal assemblages over time. To interpret the sometimes ambiguous figures depicted in the rock art, she frequently referenced modern-day equivalents to help differentiate a leopard from a cheetah figure, or to compare the great size and shape of ancient aurochsen horns to domesticated descendants of long-horn cattle. She noted the importance of finding ways to help museum visitors notice details, pick out salient features and make personal connections to topics of interest. These conversations provided ideas for interesting ways to mediate, tell stories, and help users observe these petroglyph images with the eye of an archaeologist.

Understanding user interactions with a gigapixel image viewer

To study user interactions with an explorable high-resolution image environments we began by examining an existing gigapixel image viewer.  Drawing on GigaPan images of petroglyph sites selected from Dr. Olsen’s Arabian Rock Art Heritage project, we set-up a 50” plasma display with an attached touch panel module to support single-touch inputs.  The user interface included a set of simple navigation elements affording zooming and panning on one side, and a series of snapshots at the bottom that would zoom into key hotspots within the image.

To assess the appeal and usability of this design, we used a standard think-aloud technique while observing and logging user interactions. Museum visitors were encouraged to talk about their experience while using the gigapixel viewer, and respond to questions about the interface, navigation, and content presentation. Each user group was also invited to participate in a subsequent card-sort activity to explore content themes, potential taxonomies and structure. Participants were given a number of topic cards showing a petroglyph figure associated with a subject theme (e.g. hunting, art & craft, animals, culture) and then asked to rank the cards from most to least interesting. On a provided form, participants wrote down any questions that they had relating themes, and suggested additional topic areas of interest.

Findings

These user studies generated several key insights about the interaction and information design challenges associated with gigapixel image viewers. We found that participants, in particular older individuals and groups, found navigating the single touchscreen-based image viewer to be non-intuitive and somewhat frustrating. Younger audiences expected the touchscreen to support familiar multitouch scroll, pinch, swipe inputs. These younger users adapted quickly to the interface navigation elements. Nevertheless, poor touchscreen responsiveness led most users to rely on zoom sliders, snapshots, and button controls for navigation, which many felt were cumbersome and sluggish. Many users resorted to serially using snapshot selections to move around. Users, especially younger ones, would often walk away from the exhibit display if they found it difficult to navigate.

The existing user interface we tested did not encourage free exploration of the petroglyph scenes, nor support observational talk and gesturing about what was seen in the image. Users reported that the system gave them no sense of how to explore the image, or any kind of contextual information about the petroglyph site. In general, most of our participants wished for more interpretive and explanatory content rather than descriptions of what was being seen visually. Younger audiences repeatedly expressed a desire for more engaging audio-visual or interactive media to obtain information on particular image details or themes, while older audiences indicated that more in-depth textual information and visual supplements like photographs or illustrations would help make the exhibit more engaging.

Several important insights emerged from the card sort activities. Many of the responses revolved around background information and interpretation. For instance when shown an image of a bow hunter with leashed dogs pursuing wild cattle, some users wanted to know more about hunting in the Neolithic period. Others raised questions about rock art carving techniques, ancient cultures, or the geology of the site. We were able to identify popular and particular themes of interest (e.g. nomadic culture and lifestyle). Users also provided suggestions for additional topics of interest, such as rock art carving tools and techniques. During this activity, users indicated a clear desire to have more control over how to explore content. They made requests for a more organized or structured approach to the information, and they wanted to hear from the curator about points of interest within the petroglyph image. These findings would guide our content development and information design process in the next phase.

Conducting a heuristics evaluation of the museum’s current gigapixel viewer

Next, we conducted a heuristic evaluation to test to the functionality, usability, aesthetics, and content presentation of an existing kiosk-based gigapixel image viewer that had been developed for the Museum’s M is for Museum exhibition.  The viewer was designed to showcase seven GigaPan images taken by museum scientists, including both artifacts and specimens from the museum’s collections. The gigapixel image viewer provided simple navigation elements, including a zoom/pan slider control on the left side of the image – below the image was a row of snapshots with short text annotations that when selected zoomed into a key frame within the image, and a set of arrow buttons to select between images (Fig. 1).

Figure 1a: The M is for Museum Gigapixel Image Viewer, Carnegie Museum of Natural History

Figure 1: The M is for Museum Gigapixel Image Viewer, Carnegie Museum of Natural History

Findings

The heuristic evaluation technique confirmed many of the findings from earlier studies about the general lack of user friendliness, content issues, and the problems users had navigating through the visual environment. Additionally, we felt that the user interface elements took up too great a portion of screen real estate (a full third of the screen area was used), and there was redundancy in the way multiple interface elements performed similar tasks. Moreover, the snapshot annotations were external to the image, separating the close-up detail from the text annotations and other potential mediation. Again, we found that the text descriptions associated with the snapshots were overly brief, giving little scientific explanation, background or interpretive clues about what to look for in the image.

Insights & design implications

The above findings would guide our content development and information design process in the next phase, and are summarized below:

  • Develop smooth and responsive touchscreen elements with support for multitouch gestures;
  • Explore different approaches to navigation and content control elements;
  • Streamline user interface elements in order to maximize the screen area available for image viewing and exploration;
  • Add multimedia content (audio, video, animations, graphics, diagrams) that supports user observations and engagement with the subject matter;
  • Establish a clear visual hierarchy and ordering of information, revealing what type of content is available, and where it lies spatially within the gigapixel image; and
  • Frame interactions with the viewer as a mediated conversation with a knowledgeable guide.

Phase 2 | Concept development & design

The objective for the next phase of work was to generate design solutions that would address many of the issues and opportunities identified during the exploratory phase. Using a set of guiding frameworks to develop user experience models, we produced a number of wireframes and high fidelity mockups that we tested with museumgoers utilizing A/B testing protocols (Nielsen, 2005). Further refinements were made based on feedback, narrowing down various iterations to a final prototype that would undergo full development and implementation. The prototype was then subjected to further pilot testing in think-aloud protocols, before being deployed for an external summative evaluation.

Before diving into a description of some of the choices made in designing and developing the mockups and prototypes, it would be useful to draw attention to the guiding frameworks that we used to better frame the design challenges we faced. In particular, we identified three core opportunities that became the cornerstones of our design process moving forward.

Translating user mental models into an engaging user experience cycle

In our user studies, we found that the biggest hurdle to getting users to use the image viewer and immerse themselves within the content lie in the lack of a properly managed and streamlined user experience. We found that the best way to bridge our project goals with user and stakeholder expectations was to treat the user experience as a form of mediated conversation between an expert (i.e. the museum scientist or curator) and the learner. For this, we turned to studies on conversation analysis, particularly work drawing on Erving Goffman’s model of interpersonal communication (Goffman, 1963, Tang, 2007). An effective communication loop based on this model follows an “initiate—sustain—leaving-taking” cycle of interaction. In terms of a mediated interactive system, it requires an effort by the system to initiate contact by presenting the user with just enough information about a potential engagement to create awareness and attract a response, then sustain an engagement by creating an engaging user experience that provides a sense of agency and exchange in interaction without overwhelming the user with choices or content, and lastly, offering a proper leave-taking sequence that closes the loop while presenting opportunities for future interactions and dialogue.  This model informed several aspects of our design.

To entice users to approach the gigapixel kiosk, we developed a start screen (Fig. 2) that presented basic with contextual information (title, site location). A/B testing revealed that some users had difficulties in reading the start screen approach as a prompt for interaction, and they preferred the actual gigapixel panorama to be shown as a better indicator of the experience to follow.

Figure 2. Initial start screen mockup, © CREATE Lab

Figure 2. Initial start screen mockup, © UPCLOSE

These changes were reflected in the final prototype (Fig. 3). We also introduced a feedback loop within the viewer code, whereby the interface would revert back to the start screen after 90 seconds of inactivity.

Figure 3. Final prototype start screen, © CREATE Lab

Figure 3. Final prototype start screen, © CREATE Lab

 

To sustain engagement, we wanted to find ways to provide more navigational agency, support visual inquiry and offer content choices based on user interests. We began by finding ways to refine and streamline the user interface. By iteratively simplifying the kinds of overlays that we were using by eliminating the need for explicit onscreen pan-zoom controls given the multitouch screen interface and adding a dual function “navmap” that tracks a user’s spatial location and zoom depth, we were able to reduce the number of onscreen elements and allocate the entire screen to the gigapixel image. We were also able to add the ability to embed rich media content, i.e. video and audio, into the information overlays (Fig. 4).

Figure 4. An example of text and video playing in the final protoype, © CREATE Lab

Figure 4. An example of text and video playing in the final prototype, © CREATE Lab

Furthermore, as the design solution enabling tactile navigation, we settled on three multitouch gestures: a ‘tap’ to select, ‘swipe’ to pan, and a two-finger ‘pinch’ gesture to control zoom level. As an additional support, we provided an explanatory graphic beside the touchscreen describing how to use the multitouch gesture controls.  We found these gestures intuitive to many museumgoers across a wide range of ages, and the ability to lay hands on the image to virtually dive deep into it provides a compelling visceral experience unmatched by using traditional proxy devices such as a mouse or track pad.

Scaffolding user observation through nested content structures

Our initial findings had outlined that users wanted to have a sense of agency in identifying and choosing both what kinds of information was available, and how it was presented to them. This need aligned both with the goal to design the kinds of mediated conversations that sustain interest-based exchanges, and the stakeholder’s requests to find ways to scaffold and direct user attention to facilitate observation. In the first viewer that the museum was using, there was no way to determine what was important about the selected snapshots within an image, what kinds of information different snapshots linked to, and how different regions within the gigapixel image were related to each other.

Because of the way in which information can be located in the visual plane of a high-resolution multiscalar image, in the form of point annotations or hotspots, it affords two distinct modes of organization: taxonomic, and visuospatial. The gigapixel image space offer two orthogonal dimensions for structuring content – one lies at the surface of the screen, and lends itself ordered taxonomies based on the nature of the subject matter. The second dimension lies within the image space, where the spatial location of the content embedded in x-y-z spaces gives a perspective where and on how the taxonomic structure(s) is distributed within the image. So how themes or hotspots clustered within regions of the image becomes an additional visual information cue. Activating these modes of content organization enables a powerful way for users to more deeply explore based on personal interest.

Using Richard Saul Wurman’s information organizing principles (Wurman, 1989), we identified location (i.e. petroglyph sites), chronology (i.e. Neolithic, Holocene), categories (i.e. disciplinary topics or themes), hierarchy (i.e. alphabetical, scalar) as basic ways to structure the information design. In addition, we identified as additional organization filters such as media types (text, audio, video) or user type (i.e. information tailored for kids, adults etc.) as further ways we could structure content. While sorting by location, time periods, categories and themes are central to the ways in which scientists observe and identify important patterns depending on their specific discipline, allowing content to be accessed through forms of hierarchy like popularity, or through the annotation mediums in which it is presented could also be appealing to certain kinds of subject matter and users.

We prototyped several design solutions under different content structuring regimes to see how annotations, and the corresponding ancillary information, would be used. For example, we created mockups testing the use of a top menu bar that would give users a choice in how they wanted to sort through content, by allowing them to select a taxonomic meta-structure following one of our identified organizing principles, as well as determining how content could be accessed from within that structure. (Fig. 5).

Figure 5. Mockup showing chronological sorting using a timeline, © UPCLOSE

Figure 5. Mockup showing chronological sorting using a timeline, © UPCLOSE

For thematic content, we created several different solutions testing different modes of interaction, the layout and interactivity of interface elements, and exploring design variables like color, shape, and icon to denote distinct categories of information (Fig. 6).

Figure 6. An example of the user interface mockups using themed content structures, © UPCLOSE

Figure 6. An example of the user interface mockups using themed content structures, © UPCLOSE

After evaluating the different prototypes through A/B testing with museumgoers, we found that users responded most favorably to a retractable ‘content drawer’ on the side of the left side of the screen, with a single color denoting a ‘selected/unselected’ theme state, so the corresponding annotations would reflect the chosen theme color and indicate the location and distribution of that theme within the image. This solution was adopted for the final prototype (Fig. 7).

Figure 7. Final prototype content drawers showing a selected theme, © CREATE Lab

Figure 7. Final prototype content drawers showing a selected theme, © CREATE Lab

Redefining annotations

Finally, we focused how to best annotate content in a dynamic multiscalar image. We propose that annotations, whether textual or otherwise are a powerful tool to give users further control over information within a zoomable visual environment. The first museum gigapixel viewer used a ‘snapshot’ metaphor where a sub-sampled frame within the image served as the navigation and annotation element. The problem with snapshot interface elements is that by separating snapshots from their spatial location within the image, it dislocates the information from its context, giving users little sense of how the spread of information and its location within the image matters. Framed snapshots also act as poor indicators of significance or markers of hard-to-see details within an image.

We chose to explore an ‘interest spot’ metaphor as a more precise way to pinpoint and contextualize annotations within a multiscalar zoomable user interface. Interest spots act as both a specific indicator to where visual details lie as one zooms in, and become the point from which content overlays can be centered and presented on screen. Our experiences suggest that these point annotations offered a flexible and compelling information design strategy.

Across several design iterations, we explored the idea of layering interest points with additional functionality and choice, giving users control over content modality, links to external sources, and access to social media via share buttons (Fig. 8).We also considered supporting “interest spot” based conversations between users and the museum staff. These strategies would enhance the “mediated conversation” and leaving-taking aspects of the interaction cycle we aspired to. However, due to project scoping requirements and time constraints, we had to leave out development of these enhanced interest points in the final prototype. It is worth noting that over the course of our A/B testing participants generally liked the idea of added functionality, and this suggests a promising area for future investigation.

Figure 8: Enhancing interest points with added functionality, © UPCLOSE

Figure 8: Enhancing interest points with added functionality, © UPCLOSE

Phase 3: Prototype implementation, testing & evaluation

Throughout the design process, we worked closely with our technology partner, the CREATE Lab at Carnegie Mellon University, to vet ideas and understand the software engineering implications and feasibility of various design choices. The final prototype system needed to withstand the demands of a museum setting, which imposed several technical constraints: the system needed to run with minimal maintenance, it had to run locally without an Internet connection and cloud storage, it needed to be able to self-recover from a browser crash, and finally require minimal effort for the museum staff to start-up and shut down.

The final design was implemented in JavaScript and HTML5, rendered in the Mozilla Firefox web browser, running on Windows 7 Pro 64-bit. Since all assets are stored locally, the kiosk computer is running a local instance of the Apache HTTP Server to which the web browser connects to load all content for the interactive. Running the assets locally resulted in faster access times, and meant fewer memory leaks and crashes, which ensured that the viewer ran with minimal maintenance for the duration of the study. The gigapixel image viewer was created in a modified version of Microsoft Seadragon Ajax, which supported loading gigapixel images, making use of several open-source JavaScript libraries like jQuery, Hogan, and Less. We also chose hardware which would provide an optimal, lag free user experience, utilizing a quad core Intel i7 CPU, 8 gigabytes of RAM, and a solid-state hard drive for reliable and fast access to the image files, and a 24” multitouch touchscreen which was installed in a customized freestanding kiosk case.

Furthermore, the viewer incorporated an event tracking system which records every user action, so we have exact statistics on which parts of the gigapixel image are most explored, which audio/video elements are most popular, active usage time vs. idle time, etc. The statistics on usage patterns gathered from the data logs gathered through the interactive viewer have been very useful in complementing our observational data. These statistics have been compiled and visualized using Google’s Fusion Tables software.

In order to test the effectiveness of the new viewer in the context of an exhibition setting, we worked with Laser Labs, a local design and fabrication studio, to develop the supporting Stories in the Rock exhibit environment (Fig. 9), which housed both the interactive kiosk testing, and showcased a large print GigaPan print, wall panel graphics, and “stone phone” audio handsets. We also developed a set of tabletop rock art tracing and card matching activities for families to help looking at petroglyphs with an archaeologist’s eye.

Figure 9: Panorama of the exhibit space, © UPCLOSE

Figure 9: Gigapixel image of the Stories in the Rock exhibit space, © UPCLOSE

The exhibit was launched as a demonstration project in late November 2012.

Case study results

While we are currently in the process of synthesizing both qualitative and quantitative findings, initial observations and our customized analytics program yield promising results.  As of January 2013, we logged 1136 user sessions over a 57-day period, with a mean time of 4 minutes logged per session.  This average session duration shows a considerable improvement over the mean time observed with the interactive museum viewer that existed at the start of the project. We were also able measure which interest points and themes were most frequently selected by users, giving us an indication navigation preferences and the kinds of content users preferred. Interestingly, the ratio of theme to interest spot clicks is quite low, suggesting that users are favoring interest-spot over theme-based explorations (Fig. 10). We were also able to track playable media usage – 37.4% of users watched or listened to media in the content overlays, indicating that multimedia content is clearly something that users find engaging. Audio was played slightly more frequently than the videos.

Figure 10: The ratio of interest point (blue) to theme selection (red) activations, © UPCLOSE

Figure 10: The ratio of interest point (blue) to theme selection (red) activations, © UPCLOSE

3. Future directions

Our design research findings indicate that the new developed Stories in the Rock gigapixel viewer has \ successful meet many of the project goals.  User engagement with the gigapixel image viewer has increased across the board, both in terms of the average length of user session times, and in the use of supporting text and media annotations.  Based on preliminary observational data tracking conversations and gestures, the experience appears to be increasing scientific forms of observation and talk, and users are better able to explore the subject matter based on their personal interests. The summative evaluation currently being conducted by Rockman et al. will further deepen our understanding of whether engagement, observation and learning goals have been met.

We also want to highlight several promising future directions. Enhancing annotations with additional functionality is certainly something that requires deployment and further user testing. We also feel that the current prototype did not fully exploit the opportunity to use the gigapixel image platform as a social media space. As we continue to develop this system, we will look for ways to support more direct engagement and discussion between museum experts and a communities of users/learners—and following Goffman’s cycle of engagement—to sustain dialogic exchanges. Currently no functionality exists for leaving comments or observations within the multiscalar image space, or for sharing information via social media channels, and these possibilities merit further exploration. Adding the ability to share or tag content that users found particularly interesting, or would like others to connected with, would also help to strengthen the leave-taking aspect of the user experience cycle that we have proposed, giving users an incentive to revisit or connect to new content.

While our current project focused mainly on illuminating non-linear navigational control and content structuring within a single gigapixel image, we recognize another area of need based on our meetings with the museum scientists; that is developing ways to navigate and structure content relationships between gigapixel images, and for that matter, other coordinating with other forms of high-resolution visual data (e.g. geospatial, simulations, visualizations).  Moreover, research groups are inventing evermore powerful interactive visual browsing tools for high resolution media. Our CREATE Lab technology partner has developed GigaPan Timemachine which extends gigapixel image exploration over time – adding another dimension of animation and complexity to the design space.

Through our work and conversations with museum staff, we have been made aware that there is considerable interest using gigapixel image technology to bring other aspects of museum research and collections to the public. Our next demonstration project focuses on working with the entomology section to develop a “digital teaching collection” to help overcome learning challenges inherent in aquatic macroinvertebrate identification and classification activities. “Taxonomic ID bottlenecks” are endemic to many citizen science and biomonitororing programs, and our work here will look for ways to use explorable, annotatable high-resolution macrophotography to make identifying and learning about the insects found in freshwater streams easier and more engaging.

Our hope is that the compelling visual nature of these networked explorable image environments evolve into a fully realized cyberlearning infrastructure that deepens observational practices and rich exchanges between experts and publics in a shared visual environment.

Acknowledgements

Gigapan images courtesy of Dr. Sandra Olsen, Richard T. Bryant and the Layan Cultural Foundation.  We would also like to thank Mary Ann Steiner, Kevin Crowley, Camellia Sanford, and the scientists and staff at the Carnegie Museum of Natural History for their many contributions to this project.

This material is based upon work supported by the National Science Foundation under Grant No. 1114476. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

References

Bederson, B. B. (2011). “The promise of zoomable user interfaces”. Behaviour & Information Technology, 30(6), 853-866.

Bertone, M. A., Blinn, R. L., Dew, T. M. S. J., Seltmann, K. C., & Deans, A. R. (2012). “Results and insights from the NCSU Insect Museum GigaPan project”. ZooKeys, (209), 115.

Beyer, H., & Holtzblatt, K. (1997). “Contextual Design: Defining Customer Centered Systems”, Morgan Kaufmann: San Francisco.

Eberbach, C.Crowley, K(In Press) From seeing to observing: How parents and children learn to see science in a botanical gardenJournal of the Learning Sciences.

Falk, J.H., & Sheppard, B. (2006). “Thriving in the Knowledge Age: New Business Models for Museums and Other Cultural Institutions”, Lanham: Altamira Press.

Gammon, B., & Burch, A. (2008). “Designing mobile digital experiences”. Digital technologies and the museum experience: Handheld guides and other media, Lanham: Altamira Press, 35.

Goffman, E. 1963. Behavior in Public Places: Notes on the Social Organization of Gatherings, Glencoe: The Free Press

Kenderdine, S. (2010). “Immersive visualization architectures and situated embodiments of culture and heritage”, Information Visualisation (IV), 2010 14th International Conference (pp. 408-414). IEEE.

Kidd, J., Ntala, I., & Lyons, W. (2011). “Multi-touch interfaces in museum spaces: reporting preliminary findings on the nature of interaction”, Re-thinking Technology in Museums: Emerging Experiences, Ciolfi, Scott and Barbieri, University of Limerick.

Louw, M., & Crowley, K. (2013). “New Ways of Looking and Learning in Natural History Museums: The Use of Gigapixel Imaging to Bring Science and Publics Together”. The Museum Journal, 56(1), 87-104.

Lewis, C., & Rieman, J. (1993). “Task-centered user interface design: a practical introduction”, University of Colorado, Boulder, (This shareware book is available at ftp.cs.colorado.edu).

Luan, Q., Drucker, S. M., Kopf, J., Xu, Y. Q., & Cohen, M. F. (2008). “Annotating gigapixel images”. Proceedings of the 21st annual ACM symposium on User interface software and technology, ACM., 33-36.

McCallie, E., Bell, L., Lohwater, T., Falk, J. H., Lehr, J. L., Lewenstein, B. V., & Wiehe, B. (2009). “Many experts, many audiences: Public engagement with science and informal science education”. A CAISE Inquiry Group Report, 1.

Nielsen, J. & Sano, D. (1995). “SunWeb: User interface design for Sun Microsystem’s internal web”Computer Networks and ISDN Systems, 28(1), 179-188.

Nielsen, J. (1992, June). “Finding usability problems through heuristic evaluation”. Proceedings of the SIGCHI conference on Human factors in computing systems, ACM, 373 – 380.

Nielsen, J. (2005). “Putting A/B Testing in Its Place”, available at http://www.useit.com

Proctor, N. (2011). “The Google Art Project: A New Generation of Museums on the Web?”,  The Museum Journal, 54(2), 215-221.

Tang, J. C. (2007). “Approaching and leave-taking: Negotiating contact in computer-mediated communication”, ACM Transactions on Computer Human Interaction, 14(1).

Wurman, R. S. (1989), Information Anxiety, New York: Doubleday


Cite as:
A. Ansari, I. Nourbakhsh, C. Bartley and M. Louw, Exploring Gigapixel Image Environments for Science Communication and Learning in Museums. In Museums and the Web 2013, N. Proctor & R. Cherry (eds). Silver Spring, MD: Museums and the Web. Published January 30, 2013. Consulted .
http://mw2013.museumsandtheweb.com/paper/exploring-gigapixel-image-environments-for-science-communication-and-learning-in-museums/


Leave a Reply