The Web As Immersive: Data Is The New Object.
Paula Bray, Australia
Museums are experiencing change and cultural institutions need to be producing experiences for audiences that are connected, participatory, transparent and meaningful, that tell stories and value memory in a networked environment. More and more these experiences and conversations are going to be driven by digital interfaces that connect people to the web and back in our gallery spaces, and then again later when they get home. Museums should aim to:
Present programs and exhibitions that reflect the spirit of the times and explore new ways to engage with audiences that may challenge, involve experimentation or generate controversy. (Casey 2009)
It is no longer essential for audiences to consume knowledge through the lens of a curated narrative. The role of museum educator and gatekeeper is shifting and this paradigm has evolved due to the web and online social connections and the way communities share stories and information. But how we achieve this successfully requires a lot of thinking around why we should do something in the first place, and not just because it’s digital and includes some new and exciting, shiny technology. This is going to be an ongoing issue for museums due to the rapid pace of changing online social platforms and the way people are using them. We need to produce content and experiences for ‘now’ because soon enough they may be deemed irrelevant and not particularly engaging if the experiences are not unique and different to what is on offer through devices in the home.
Museums are very good at producing exhibitions based on a formula, with the traditional, interpretative labels accompanying every object, didactic narratives and even assumed directions in which way the audience should traverse through the gallery space. Throw a digital experience into this mix and more often than not it is an add-on to the overall vision. There needs to be a fundamental shift in the way that exhibitions are processed and produced to really capture what the web has been able to do for communities and connections online. Exhibition development teams that are producing innovative projects that are participatory and involve social technologies, that present information about digital collections, exhibitions and stories. We need to learn how audiences are consuming and using deeper layers of information, and what connections they are making with that information. There is a need for realignment of museum processes to enable new technologies to deliver experiences to audiences that we may not be comfortable with or that challenge our pre-described exhibition outcomes.
As exhibition design, which includes digital technologies, achieves ever increasing importance as a cultural medium that can create and shape history, practitioners of museum technologies must address social implications with greater attention and sophistication. (Casey 2003)
Audiences are changing and this has surfaced quickly with the rise of new technologies, social media and mobile devices. Museums face important challenges in this new era of museological practice. Our audiences have pockets full of mobile devices ready to connect with museums, our content and collections, to participate and then share their experiences with their own communities.
Our audiences are now networked in a way that we have never experienced before, whether that is through their mobile devices in the gallery or the ability to share their experiences so readily during and after they leave the museum. There is a necessity to conceive of and deliver a new type of exhibition formula and process, and sometimes these may just include a small display powered by data that allows our audiences to experience content in a way that they may not be able to be experienced anywhere else. Is this the role of cultural institutions now: to think up and provide exhibitions/experiences that are unique and not available in the home or workplace? What if these do not involve didactic leanings and the desire to make sure we inform our audiences of the facts we know?
Why has data become so important in museums?
Museums are full of data and our collection databases are the obvious first places to look into exposing and sharing this. There are very limited cultural institutions collections that can be physically seen by the public. The statistics would vary for each institution but generally for public use of the actual object it is around 4% that is available for viewing in the gallery. Do our audiences actually know that our collections exist if they cannot access them? Museums collections are extensive but few are able to experience them as they reside in our collections stores. So how do we engage our audiences with our collections if they are generally not accessible? The digital form of our collections has never been so important. Allowing access to the digital representation of our collections is a must for current museological practices and potentially the only way some audience members will become aware that they exist. Bringing our data out into the public spaces, whether this is in the physical onsite experience or online, is crucial to allow our collections to exist, be used, shared, researched and included in innovative projects. The behind-the-scenes of the museum is sometimes far more interesting than what is available for public viewing only and this can be realised through the full potential of the value in our data.
There is a rapidly emerging consensus that the most successful museums of the future will be places to hang out, engage and contribute: museums that blur the boundaries between “back of the house” and the public side. They will be moderators and filters of contributed wisdom and diverse perspectives, in addition to being sources of scholarship and opinion. (Farrell et al 2010)
The role of the museum is being challenged through the wider networked environment that we currently find ourselves in. Our incredibly rich and structured collection data and content is far more utilised and understood if it is available and the use of the collection will be more meaningful (Oomen et al 2012) even if this data can sometimes be problematic and, according to Chan, ‘sucks’ (Chan 2012). This brings up a good point in that if we don’t use our own datasets to trial, test and aim to produce meaningful experiences with it, then how can we expect communities to use them in innovative ways as well? Will they be faced with the same problems we are?
- What data do I need and can I find it?
- Will the experience be unique and responsive?
- Can I participate with and contribute to this data and content?
- Can I select a story or journey to reveal?
- Can I share this experience?
- How do I make this experience meaningful?
But as suggested by Dearnley (2011), if we don’t release our data then how will we know what the issues are if our audiences can’t reveal them to us? And more importantly, how “would we have seen people make new apps with our data”? This is very chicken and egg stuff, but one that points me to “it’s best to release now so we can discover the issues” as has been widely discussed by the Gov 2.0 Taskforce in Australia. The Taskforce (2010) surmised that if public sector information is liberated as a key national asset, then there is the potential for invention and creativity so that the public can engage, innovate and create new public value with our data and content. Perhaps museums can take a lead and use the data in innovative ways to inspire others to do the same, particularly in their gallery spaces.
2. The Immersive in Museums: Do audiences want this type of experience/interaction?
Museum audiences are becoming more critical of the traditional museum exhibition, particularly the younger and harder to engage audience of around 16-25 years old. As Farrell et al. state, what the younger audience members request of museums is an interactive, immersive and participatory activity.
They want to be more than outside observers looking in. And the museum attributes they value most highly are uniqueness, novelty and authenticity. (Farrell et al 2010)
Museums are not so agile and are generally wired to provide didactic experiences that pre-suppose that the visitor will learn something by the time they leave the gallery. Museums are now finding that they need to provide for experiences that start online, continue in the gallery and then return back online once the visitor has left the institution. We are now thinking of and delivering digital experiences that can provide for this cycle. The Ann Arbor Hands-On Museum has delivered a Liquid Galaxy experience on the floor of their institution and they recognise the importance of this cycle by stating:
Since Google Liquid Galaxy is powered by Google, visitors to the Ann Arbor Hands-On Museum can continue their adventure after leaving the Museum by simply accessing these platforms on their own computers. Google Earth is free. (Ann Arbor Hands-On Museum 2012)
So why use Liquid Galaxy for an in-gallery experience?
Liquid Galaxy, developed by Google engineers in their 20% time, provides the opportunity to display Google Earth over multiple screens in an immersive experience.
The rich satellite imagery and 3D building models of Google Earth come to life when experienced in stunning large-scale panorama. The Liquid Galaxy lets you sail around the globe with the included 6-axis controller, allowing you instantly zoom in, out, and around in completely fluid motion. You can also search and navigate to specific locations on autopilot using the touch screen interface. (Liquid Galaxy 2012)
This platform provides a new way to experience data that you cannot achieve currently at home. The question with this technology is really “What sort of engaging and unique experience can you deliver to an on-site audience?” People can get delightfully lost in surfing locations all around the world, but how long can you sustain the interest of your audience? One of the problems is that experiencing data this way can provoke motion sickness and the user can get lost very quickly and then may not continue to explore using this technology. Is it the responsibility of the museum to curate an educational journey or particular narrative using this technology, or is it simply enough to allow audiences to make their own journey through locations of interest to them, usually a personal one?
Almost everybody wants to see their own house first; but then they start to explore, and we can never guess where they’ll choose to go next. (Holt 2010)
This is the big question we are facing at the Powerhouse Museum: what should we provide to our audiences using this technology? Putting this technology into our galleries now with the audience driving the controls doesn’t seem to be the right process to take. The controls can be difficult to navigate unless you are a natural gamer. As Jones discusses in an interview about creating immersive media and stories:
The best projects I’m seeing are those that are very focused, very specific, not offering platforms for platforms sake, but a clearly defined experience. (Jones 2012)
The Ann Arbor Hands-On Museum has found out early on, after one week of their Liquid Galaxy display being available in the gallery, that the visitor needs to be less in control of the technology.
We have decided for the next few months to permit visitors only exploration with the 3D joystick. We plan to offer facilitated tours of the moon and Mars. The reason for this is that allowing cursor controls (in our case, Suzo-Happ Controls trackball and select button) opens up the system so that visitors can escape Liquid Galaxy and surf the web and, worse, leave the exhibit on a web page for the next visitor. (Stout 2012)
We are currently debating providing a ‘tour-like’ experience that will lead the visitor on a meaningful and engaging journey, but we are aware that we shouldn’t just provide this technology in our galleries because it is shiny and new. Watching what visitors do with technology in many museums, there frequently seems to be a desire to touch the technology even if the purpose is not interactive: monitors are covered in fingerprints and visitors will pursue an interactive until it no longer provides them with a desire to explore further or the story just doesn’t provide engagement.
Having said this, there is also the need to make sure that the experiences provided allow for choice. The choice of the location to discover, the choice of the story that will be revealed through a curated narrative because we know that our audiences want to participate in the curation of content and that they want to be able to contribute a story or memory. This is where the collection can become the driver for the content of this technology. There is a desire to use mapping technology and data around a familiar subject: “where I live or grew up” is about the personal location. Museums have traditionally not been so good at providing for the personal experiences in their exhibitions.
The Powerhouse Museum has been making some of our historic, glass plate negative photographic collections available for use and reuse over a number of years now and in multiple platforms on the web. Exposing our collections to a broader audience online has been successful and we have seen the images researched, tagged, remixed and innovation has occurred using this content. One of the exciting elements of making images available is the mapping potential when a location is known or can be suggested. Mapping our images has made it possible for the Museum’s collection to exist in multiple platforms such as Flickr, Historypin and Sepia town. Geolocated content has become very powerful in terms of finding the content, providing stories, memories, tours and use in new technologies such as Google’s Liquid Galaxy.
One of the main reasons to explore Liquid Galaxy technology is our intention to produce an in-gallery immersive experience using content from Historypin that will be located in one of our galleries at the Powerhouse Museum. We aim to make this experience participatory, to pull in live data that is being pinned in Historypin across Australia and show this in a meaningful way, plus recreate the ‘tour’ function in the platform that also allows you to slide in and out of ‘then and now’ photography in their Streetview tool but in a curated way. The main problem we are facing with this experience is that we are ultimately trying to make an online experience become a physical one in a set space. This is problematic and needs thinking through very carefully. As van Dinther states: “Google Earth is simply not reliable enough to use in a museum environment. It crashes too often.”
3. First steps
This project has involved a lot of pre-testing so that we can be sure that what we want to deliver on the floor will be well considered for our visitors. We have started a project with students at the University of Western Sydney under the lead of e-researcher Andrew Leahy. The students were set the challenge of setting up the Museums Liquid Galaxy display as a test demo in our office and then let loose on our collection API (Application Programming Interface) to produce interesting content experienced in Google Earth across three vertical screens. The first problem: our API is messy and the students didn’t know what data to use that would be meaningful (or, as Chan noted, ‘sucks’). One aim was to find object location data and then map that according to the many locations that could be associated with the object. Then we were faced with the question of whether it is of interest to know where an object has been – its provenance – but more importantly whether we have enough data to make this a compelling experience. The first lesson learnt was that the students found we didn’t have the data in the API that they wanted to utilise around object location; however, the geo-located photographic images that we have loaded in Flickr, Historypin and Sepia town were the best source of data that they could experiment with. We had seen the public making innovative projects with the images previously since they were freely available online.
The students were then able to map all 450 of our geo-located images using Google Earth and 3D modelling so that the images appear to be floating in the orientation that they appear on the map according to their locative data. Once we had seen the compelling way our historic images looked mapped in this way, the questions kept coming: is this enough? How can we make this a great and compelling experience in the gallery space and link this to a experience that our visitor can have when they leave the museum, to continue the cycle? As the Ann Arbour Hands-On Museum claims, “Our new Google Liquid Galaxy exhibit is educational, visually stunning and absolutely engaging.”
As this technology has not been in our gallery before, we were making assumptions about how our audience would use/experience it. The decision to test and experiment with this in the physical location was taken and we felt that we needed to ask visitors what they would like to see used with this technology. The three screen demo-version will be set up in our educational digital lab, ‘Thinkspace,’ where we aim to trial the use of the gaming platform ‘Minecraft’ with this technology as it fits with some educational programs that the lab has been running. We will be observing and interviewing visitors as they use the technology to get an understanding of the following things:
- Do you find this experience unique and engaging?
- Would you like to have a curated tour of a certain location?
- What would you like to see this technology used for?
This evaluation should help us to drive a more defined outcome for using Liquid Galaxy in our gallery space during the year.
4. Search becomes art (a concept for an in-gallery experience)
On the 13th September 2012 the Powerhouse Museum’s online public access collection database received 9288 search terms. This is a mass of search data about what our online audience are looking at in relation to the objects in our collection, yet in the physical gallery space in the Museum some visitors may not know the museum even possesses over 400,000 objects. We rarely offer our visitors a way of searching the collection in the galleries and have only recently started to allow deeper layers of information that link to the collection via QR codes and short urls.
Many museums claim to be unique in part due to their rare and distinctive collections. We have become much better at making our collections available online (although more work is needed), yet we neglect the physical space in relation to searching the online collection. What are our collections if they are not available, or for that matter even viewable, and why have they been only really made physically available for the privileged few? Does the online collection now outweigh the physical collection due to access? There is one part of the Powerhouse Museum’s Strategic plan that states;
We are to deliver programs which explore creativity, technology and the sciences in ways which stimulate learning and innovation. We are to dissolve the boundaries between exhibitions, programs, publications and web content, plus to enable new interactions for visitors within our spaces and online. (Casey 2009)
This is actually quite a challenge to achieve and one that needs attention particularly in relation to our collections of which there is only a small amount of the physical objects that are actually ever available in the galleries. The majority of the collection is sitting on shelves in our collection stores and museums are generally not large enough to showcase all of their collection. Therefore what our online visitors are doing with our collections can be the basis for an in-gallery digital experience that aims to bring the onsite visitors closer to and engaged with the Museum’s collection. Perhaps for some, this will even make them aware that a collection exists. The big question for this really is “Do our audiences want to discover and participate with our collections in a deeper way?” Perhaps this is an experiment that is worth pursuing to find out.
As stated by Parry, museums need to change and evolve with digital and that we cannot just rely on our analogue formulas for distribution, especially since our framework for making sense of collections is rapidly progressing.
The choice we need to make in the digital age is this. Do we hold on to qualities that are defiantly analogue and rely on people being present at a venue, and having to ritualistically cross the threshold to get there as a social encounter that involves us looking mainly at physical objects? Or, do museums choose to change and converge in a digital age where content is distributed, people are networked and everyone can have a voice to create or produce? (Parry 2012)
The aim of ‘Search as art’ (an experience in ‘critical thinking’ stage) is to create a data visualisation/in-gallery projection of the daily search terms that are fed into our online collection search box and to project the terms on several lightweight fabric screens in one of the galleries. These projections will rely on audience ‘search’ participation online and onsite and only happen as a result of people searching the museums collection: the visitor is responsible for the results of the display, not the museum. This is essentially a crowd-curated exhibition that we hope to allow our visitors to find patterns and connections that are of interest to them, no matter what the search term is. Our data should not just be for the privileged few such, as academics and highly skilled coders, nor should it just be experienced in a flat search return list.
People’s search results will be projected along with their image results in a physical art-like digital experience. The words will be moving, not static, and the more popular terms will appear larger and will relate to images that are in a search result. Tablet devices would be made available so that visitors could get real-time results of their searches and see them through the digital experience. We will start to see trends and results and the most popular search term could be made larger than terms that are not as popular.
On September 13th the most popular search term in our online collection was ‘feet.’ Now this may have not just be related to the human anatomy but perhaps also to the measurement. Second was ‘ship,’ then ‘Australian,’, ‘design,’ ‘Sydney’ and ‘blue’ – and the list goes on. When you search ‘feet’ at the online collection the result is 1510 objects that are so diverse, from footwear, costumes, model aircraft, teapots right through to trains. So the first challenge begins and a big question is what would our visitors want to see when the top search terms return over 1000 objects, not all with images?
The display is anticipated to be different everyday and visitors may return to see how it changes. Once this exhibition is over, the code could be used elsewhere in the museum as an ongoing source of information for our visitors that can be seen anywhere including foyer screens, online etc. The simple premise of this experience will be to engage our onsite audience to search our online collection, OPAC, and learn more about the collections we hold. Hopefully it will also encourage further participation with our collections once they are home, keeping the cycle going. But how we do this needs considerable thought as we don’t want to create a ‘broken feedback loop’ as suggested by Simon: many institutions don’t spend enough time thinking about this part of participatory experiences. Our proposed experience, using search, needs to have a feedback mechanism that “will embrace the full spectrum of participatory behavior and provide responsive value to those who engage” (Simon 2012).
As Museum are grappling with transformational change in a time when digital and technologies are outpacing our capacity to keep up and produce new and innovative in-gallery experiences, we need to experiment more with our collections and this includes our data. We possibly should be rethinking the concept of the museum in general, why we exist and what our collections mean to us and our audiences. If we provide more social spaces for onsite engagement with our ‘unseen’ collections, then perhaps we are going to be in a better position of knowing what our ‘non academic’ audiences want with our significant and unique collections and data sets. Collections need to be utilised more than for academic research: we need to explore collections and our data in new and unexpected ways to reveal different ways of looking at them. Museums have a responsibility to make their collections and data associated with them not only accessible but to allow audiences to engage and respond with and to them. Audiences want to interact with us and our collections, and having an object in a showcase with a label may not be enough in the fast pace of changing, networked and participatory visitors.
Thinking through the two digital experiences for the Powerhouse Museum using different data sets has resulted in the need for the experiences to deliver on many aspects. It is not enough to put these experiences into our galleries and hope that our visitors can navigate the technology and find a story or memory that they want to investigate, share and respond to it. We will also have to make sure that they deliver:
- A unique and responsive experience;
- Allow the visitor to participate with and contribute to this data and content;
- The visitor will be able to choose a story or journey to take so that it can be personal;
- They will be able to share the experience with friends through their devices;
- To continue the experience when they leave the institution.
As these two projects are in the early concept stage there is a lot more user testing and visitor engagement analysis that will have to unfold before we can see if they have been beneficial to our audience but also to the Museum. We need to learn more about using technology with audience participation in our galleries so that we can provide better digital experiences that transgress from web to floor and back again.
Ann Arbor Hands-On Museum 2012 published at http://www.aahom.org/about-us/press/news/ann-arbor-hands-museum-unveils-google-liquid-galaxy-exhibit Accessed 22nd December 2012)
Casey, D. Powerhouse Museum Strategic Plan 2009 published at http://www.powerhousemuseum.com/pdf/about/strategic_plan_2009-2012.pdf (Accessed 5th December 2012)
Casey, V. ‘The Museum Effect’ presented at the Louvre in September 2003 at ICHIM 03: The Seventh International Cultural Heritage Informatics Meeting. Published at http://www.valcasey.com/thesis/thesis_effect.html (Accessed 5th December 2012)
Chan, S. ‘More on Museum Datasets, un-comprehensive-ness, data mining’ 2012 published at http://www.freshandnew.org/2012/08/museum-datasets-un-comprehensive-ness-data-mining/ (Accessed 23rd December 2012)
Dearnley, L. ‘Reprogramming The Museum’ 2011 published at http://www.museumsandtheweb.com/mw2011/papers/reprogramming_the_museum (Accessed 23rd December 2012)
Farrell, B et al. ‘Demographic Transformation and the Future of Museums’ 2010 published at http://culturalpolicy.uchicago.edu/publications/Demographic-Transformation.pdf (Accessed 23rd December 2012)
Gov 2.0 Taskforce ‘Engage, getting on with government 2.0’ published at http://www.finance.gov.au/publications/gov20taskforcereport/doc/Government20TaskforceReport.pdf (Accessed 23rd December 2012)
Holt, Jason. ‘A Galaxy of your own’ 2010 published at http://google-opensource.blogspot.com.au/2010/09/galaxy-of-your-own.html (Accessed 22nd December 2012)
Jones, M. 2012 ‘On Storyworlds, Immersive Media, Narrative and Museums-An Interview with Mike Jones’. Published at http://www.freshandnew.org/2012/10/storyworlds-immersive-media-narrative-interview-mike-jones/ (Accessed 22nd December 2012)
Liquid Galaxy 2012 published at http://code.google.com/p/liquid-galaxy/(Accessed 22nd December 2012)
Liquid Galaxy 2012 published at http://liquidgalaxy.endpoint.com/capabilities# (Accessed 22nd December 2012)
Oomen, J et al. 2012 ‘Sharing cultural heritage the linked open data way – everyone’s invited’ published at http://www.museumsandtheweb.com/mw2012/papers/navigating_the_bumpy_road_a_tactical_approach_ (Accessed 23rd December 2012)
Parry, R. 2012 in Kahn, Y. ‘Museums in the information age: survival of the most digital? published at http://www.guardian.co.uk/culture-professionals-network/culture-professionals-blog/2012/dec/05/museums-adapting-digital-age (Accessed 4th January 2012)
Simon, N. 2012 ‘Responsiveness is the most important part of participation’ published at http://museumtwo.blogspot.com.au/2010/02/responsiveness-is-most-important-part.html (Accessed 7th January 2012)
Stout, C. 2012 Ann Arbor Hands-On Museum email interview with Paula Bray (Accessed 2 January 2012).
van Dinther, P. 2012 ‘Immerse yourself in Google Earth with the “A-tour” published at The Google Earth Blog http://www.gearthblog.com/blog/archives/2012/10/immerse_yourself_in_google_earth_wi.html (Accessed 31st December 2012)
P. Bray, The Web As Immersive: Data Is The New Object.. In Museums and the Web 2013, N. Proctor & R. Cherry (eds). Silver Spring, MD: Museums and the Web. Published January 31, 2013. Consulted .