Modeling the Meaning of Museum Stories
Annika Wolff, UK, Paul Mulholland, UK, Trevor Collins, UK
Museum professionals create narratives that tell stories about museum objects and across museum objects. Narratives are not limited to the physical exhibition, but include also catalogues, audio tours, educational materials and web sites. In this paper we will describe Storyscope, a web-based environment for the construction of museum narratives, which links a final narrative to the underlying story and plot, thus allowing end-users the ability to see the thought-processes of the curator and to develop their own personalised narratives. We focus on the reasoning processes embedded into Storyscope that support a narrative author in story building by proposing additional events, and relations between events, for developing a plot of the story. Intelligent support also draws on notions of dramatic narrative to suggest ways of narrating a story and plot eliciting different types of reader response.
Keywords: museum, ontology, story-building, events, plot
A museum narrative provides an explanation of the relationship between a set of objects according to the underlying stories (events) and plots (relations between events) that link them. The story and plot may be conveyed subtly through the grouping and placement of objects in a physical space, or more explicitly through a written version, as might appear in a museum catalog. Other examples of museum narratives include tours, audio guides, educational activities, and information handouts. It is important to note that the underlying story remains the same in all cases.
Storyscope is a web environment through which museum professionals can author museum narratives. Storyscope supports the creation of multiple narrative outputs from a single underlying story, thus reducing the need to re-author the story each time a different type of narrative output is required. This is similar to the approach behind 1001 stories and the PoliCultura Portal (Campione, Di Blas, Franciolli, Negrini, & Paolini, 2011; Di Blas, Paolini, & Spagnolo, 2012), which each support output through multiple narrative channels from a single underlying story. In addition to capturing the objects and themes of the stories, Storyscope models the stories in terms of the individual events, which are related together through a plot. Similarly, where PoliCultura attaches facets to stories for browsing a large number of stories, Storyscope uses properties and values to describe the individual events, giving additional options for browsing and visualizing parts of stories.
Using Storyscope, museum professionals in different roles can share the same working materials, yet still provide their own interpretation. The curate ontology (Mulholland, Wolff, & Collins, 2012) formally models the story, plot, and narrative in way that reflects the structure of the Storyscope authoring environment. This view on the distinction between story, plot, and narrative is consistent with structuralist views on narrative, such as those described by Chatman (1980).
Since the Storyscope approach aims to capture stories that can be told about and across objects, the process of organizing and locating additional content for a museum story is based on principles of storybuilding and narrative. This contrasts with other approaches, which commonly search for and sequence museum content based on metadata similarities, such as artist, time, or place. Thus, users not only benefit by being able to search for exhibition stories that span items, as well as individual objects, but the Storyscope approach also supports visitors in both seeing the story underlying an exhibition and in re-authoring it according to their own interest/perspective.
2. Dossier building
The main working space in Storyscope is called a dossier, where the author collects the objects that they want to form part of their narrative. Figure 1 shows a dossier based on the life and art collection of the art historian Denis Mahon, that has been input into Storyspace by the National Gallery of Ireland (NGI).
Figure 1. A dossier on the life and art collection of the art historian Denis Mahon
Each object in the dossier has one or more stories associated with it, each viewing the object from a different perspective. For example, the story may pertain to when and how it was made, what inspired its creation, or what is directly depicted by it. Figure 2 shows an example of an object story about the drawing Saint Helena with Two of Her Handmaidens by Guercino, an artist from the early 17th century. This object story interprets the drawing in the context of Denis Mahon’s interest in Guercino and his identification of the drawing as inspiration for a later work by the same artist. Alternative stories related to this object could include the story of its creation in pen and ink, by Guercino, or a story about the life of Saint Helena.
Figure 2. An object story related to Saint Helena with Two of Her Handmaidens by Guercino
As objects are added into a dossier, relationships may be found between the events of one object and those of another. Figure 3 shows two more object stories from the Mahon dossier. The objects that the stories relate to are etchings by the artist Bartolozzi, who lived and worked over a century after Guercino. The stories that are associated with the objects, in the context of the Mahon dossier, tell how Bartolozzi copied Guercino’s work.
Figure 3. Object stories related to the Assassination of Amnon and Four Women with a Child by Bartolozzi
These relationships form the basis of the stories that span multiple objects and are formalized within the plot of the story. A plot consists of a number of plot elements, each of which defines a relation between events—for example, that one set of events are related to each other, or that one or more events influenced one or more other events. Using figure 3 as an example, a plot could tell how Guercino’s work influenced Bartolozzi to create the etchings, or how the two etchings are related through the same medium and even the same artist.
Plots are subjective: different authors may understand the connections among events in different ways. A plot described in the text of the object story for Four Women with a Child can be used to illustrate this:
Mahon and Turner in the catalogue ‘The drawings of Guercino in the collection of Her Majesty the Queen at Windsor Castle’, have tentatively suggested two biblical subjects – Pharaoh’s daughter and the infant Moses or Sarah expelling Hagar and Ishmael.
When objects and plots are completed, a narrative structure determines the order of objects for the final narrative presentation. As an example, a component of a narrative related to Denis Mahon might focus on the Guercino/Bartolozzi plot described above, using the three objects shown in figures 2 and 3 and their associated story texts to illustrate a connection between the artists.
3. Author support in Storyscope provided by a reasoning engine
The role of the Storyscope reasoning is to support the author during different stages of narrative development. The key areas where reasoning support is available are:
Support for plot: reasoning can propose plot relations between existing events in the dossier, or propose new events that fit the current scope of the story. The scope is defined by the setting (time and place) and theme of the current story. In identifying emerging themes, reasoning can be used to detect patterns in events of objects added into the dossier that could otherwise be hard to perceive from amongst the large amount of information contained there. This can be used to propose new directions in which to expand the storyline.
Support for narrative: reasoning proposes how plot information should be organized to produce different dramatic effects and to find the best way to exploit potentially conflicting plot and theme information of events. The final narrative consists of an ordered set of content based on the underlying story events, theme, and plot.
Reasoning support is made possible through an ontology. This ontology, and the different types of reasoning it supports, is the focus of the remainder of this paper.
4. Curate: Representing events, plots, and narratives
The curate ontology provides a means for representing museum narratives, as well as their underlying story events and plots. For a detailed description of the ontology please see Overview of the Curate Ontology (http://people.kmi.open.ac.uk/paulm/curate-introduction-20120920.pdf). The following presents a broad overview of parts of the ontology necessary for discussing the reasoning support.
There are two types of story in the curate ontology: those associated with individual objects and those that span multiple objects in a dossier. Stories comprise individual events, which are explicitly represented in curate. This allows a single event to have a role in multiple stories (and also in multiple plots within a story). An event in the curate ontology consists of a title and description, as well as a set of properties for describing it.
In addition to the fairly standard event properties of time, agent, location, activity, and object, events also allow description of museum-related properties such as dimension, material, art historic period, and genre. There are 12 properties in total; they were obtained through discussion with museum partners and based on existing event schema.
Events can be entered manually by an author, derived from the metadata of an object when it is added into Storyscope, or created using external metadata sources such as Freebase or FactForge. Freebase contains structured information harvested from several sources (notably Wikipedia) that makes it easily searchable by both people and machines. There is a lot of information in Freebase pertaining to artists and their works, as well as more general historical figures and events. FactForge provides access to similar information, in a form that supports a machine in making inferences or connections across the available information. In the future, event-detection from text (such as that added by an author) will also be used for adding events.
Figure 4 shows an abstracted representation of the ontology describing objects, object stories, their events, and event properties within a dossier.
The plot of a story contains a number of elements that define groups of events and relations between those groups. The simplest plot element is related. A related plot element is a single set of related events. Relations that can be specified between two groups of events currently include influenced, motivated, in reaction to, and inspired. Each of these relations has a “source” component and “consequence” component. To illustrate, an influenced plot element has a source of influence containing one or more events, and a consequence of influence also containing one or more events.
Figure 5. A representation of a plot from the Denis Mahon dossier
In the same way that a story can be associated with either individual objects or dossiers, so too can a narrative. A museum narrative has a structure that informs how elements should be organized within the narrative. For example, a narrative structure for an exhibition might group and organize objects for placement in different rooms, with the same structure translating to section headings on a handout.
In the curate ontology, this narrative structure is represented as a set of narrative components, each of which specifies the location of that component within the narrative in terms of its underlying story and plot characteristics. A narrative component contains one or more narrative elements, each of which maps to a story element, which can be an individual object narrative, an event, or possibly reference materials. A narrative component can also contain sub-components, thereby creating a narrative hierarchy.
5. Storybuilding through theme and setting
A story has both settings (when and where it takes place) and themes, which are reflected by the events of the story. The number of themes and settings for a given story can vary dramatically. However, when a story has a large number of settings or themes, it is usually organized to limit the number that must be considered at any particular level of the story. For instance, if the events of a book happen in a day, the story might be broken into the regions of morning, noon, and night. Then, within each of these regions, we start to understand what happened in each hour. If the book occurs over a century, the regions might be early, mid-, and late-century, with each region broken down further into decades, which are then considered by year. The purpose is to maintain coherence: if too much information is presented from the start, the story becomes hard to follow.
These principles can be applied when using reasoning support to propose new events to an author during storybuilding in Storyscope. The property values of events in a dossier provide information that can be used to build a story incrementally by finding external events that are relevant to the themes and/or settings of the current story. Events that do not fit either the setting or background are unlikely to be of relevance. These new events can subtly shift the focus for a further iteration of suggesting events, or else they might consolidate an existing pattern. The important thing to note is that story expansion occurs in a series of small steps, to avoid large shifts in time, location, or theme. Also, it is possible to expand the story from different levels, from the very top level (encompassing all story events) or at the level of an individual related plot element (containing a focused subset of events).
Settings and themes are derived by clustering the event properties. The resulting values can be used to search external historical data sources in order to suggest new historical context for the story. To illustrate, the NGI chose a selection of objects from the Denis Mahon archive to act as a test case “seed” content from which to develop a story.
To find the setting of this story, it is necessary to find the critical time periods in which the story events occurred and locations that correspond to these times, or vice versa. This is done using K-means clustering, in which a set of items are grouped into k clusters (where k is a number given prior to clustering) based on their similarity to other items within the cluster. Each cluster has a centroid that defines the important features of the cluster against which each item is compared, to decide if it is similar enough to be a member of that cluster. In this case, K-means clusters on the time values of the story events then use the centroid of each cluster to give a set of time periods that are important in the story. For the Denis Mahon content, this gave: 1934–1934, 1956–1956, and 1550–1569.
Events are then filtered by these time values, and K-means clustering is repeated on each set of events,using the remaining event properties. The location centroid is added to the time value to give the setting. The remaining values of the centroid give the theme. Focusing on settings first, for Denis Mahon this gave:
- Setting 1: 1934–1934 in London
- Setting 2: 1956–1956 in Bologna
- Setting 3: 1550–1569 in Rome
The same process can be done the other way around, looking first at locations. This gives:
- Setting 4: 1934–1934 in London
- Setting 5: 1642–1642 in Bologna
- Setting 6: 1665–1665 in Rome
The purpose of the two-stage process, rather than merely clustering on all values at once, is to avoid finding a setting that doesn’t exist within the story (e.g., London 1642). In the above example, the value for k is 3. This gives a maximum of six distinct settings at any given level of the story. As mentioned previously, this aims to limit the amount of information under consideration at the given level of the story, without precluding that additional sub-themes and settings might emerge when the same process is applied to sub-regions of the story during the grouping of events into plot elements.
While the setting can be used to search for events that may be related to the story through time and location, the theme is used to order the proposed events according to how well they match the story. The theme-related property value centroids that were derived in each clustering step are combined to give a single set of property values that are prominent at that level of the story. Theme information is accumulated since a theme can run throughout a story, even though it might not be prominent all the time, especially during story-building when the events of a story are not all there. The accumulated theme information from the Denis Mahon dossier was:
- Denis Mahon
- Annibale Caracci
- Paola della Pergola
- Lives of the artists
A few online sources of historical events in machine-readable form do exist, in a form that could be searched and imported into Storyscope (Exner & Nugues, 2011; Hienert & Luciano, 2012). These sources derive events from texts or timelines found in Wikipedia. These would be ideal for use in expanding a story, as they provide historical information already represented in the form of events; however, currently the results returned from these sources are too sparse for the identified settings.
Instead, we used the FactForge Linked Open Data repository to identity data related to the setting. As mentioned previously, FactForge provides access to information from sources such as Wikipedia in a format that can be searched and reasoned over by machine. Data can be accessed in RDF (resource description framework) format, which describes information in the form of triples comprised of a subject, predicate, and object. For example, “subject:personA ,predicate: place_of_birth , object: locationB” would be a triple linking personA to locationB in which they were born. Queries can be constructed to look for data: a simple example is querying for all people who have place_of_birth locationB. As the returned information was not in the form of events, these were translated into an event schema for inclusion in the story. For example, information retrieved about a book that was published at the time of the setting could be represented as a publication event at the specified time.
Therefore, queries to FactForge were constructed to return data that could be used to build historical events. The queries are structured to find information related to the museum domain occurring in the time periods, and which is consistent with some of the main themes that emerged from the theme clustering (writing, painting, birth). The results of the queries can be ordered according to the themes found in the previous step. Other themes (e.g., agents such as Denis Mahon and Annibale Caracci) can be used to select relevant content from across all returned queries. These queries look for:
- Birth events in the specified time and with a link to the location
- Death events in the specified time and with a link to the location
- Artworks created in the time period where the creator is associated with the location
- Artworks that were acquired in the time period, where the artwork or owner were associated with the location
- Publications in the specified time period (including reprints) with a link to the location, and where the author has a link to art.
The queries can be adjusted to other domains; they can also be made more or less restrictive by altering the extent to which they are directly linked to a location. In each case, it is possible to make an event to add to the story, such as a birth or death event, an event for creating an artwork, an event for buying an artwork, or an event for publishing a work. A few examples of results returned by querying Factforge with the time and place settings ‘London 1934’ derived from the Denis Mahon example are:
- Don Bachardy was born
- John Collier (artist) died
- Escher created Still Life with a Spherical Mirror
- The Phillips Collection purchased The Round Table by Georges Braque in 1934
- Dylan Thomas published 18 poems
In this example set, events 1, 3, and 5 would be prioritized, as these are the query results related to the themes birth, painting, and writing that were established in the previous step.
6. Plot relations
External data sources can also be queried to propose plot relations among agents in the story. For example, Freebase contains information about influence and peer relations. Using the Denis Mahon test case, it can be found via Freebase that:
Gian Pietro Bellori is in a peer relationship with Nicolas Poussin.
Bellori and Poussin are both agents mentioned in the Mahon story. Storyscope entities are, where possible, mapped to Freebase IDs when they are entered. This makes it easy to use Freebase to look up possible relationships between agents in a story. The same process can be used to query against people proposed in the previous step: when querying on theme and setting, for example, the new agents suggested as artists who were born, died, created artwork, or had their artwork purchased might have a stronger connection to the story if it can also be found via Freebase that they were in a relationship with a) another agent being proposed and/or b) an existing agent in the story. This information can be used to prioritize returned results.
K-means clustering algorithms can also use event property data to propose related plot elements, based on clusters of similar events. In this case, the value for k is based on an optimal group size derived in discussion with the museums and based on feedback of various cluster outputs from the available content, which suggested that related plot element groupings should contain between 2 and 10 events. Smaller plot elements are easier to interpret when in the process of trying to develop the plot lines. Therefore, the story is divided into groups containing on average approximately 6 events.
Once a plot grouping has been created, it is possible to find more specific plot relations among events either within the related plot element or even spanning multiple plot elements. This approach is consistent with a favored working style, which has been suggested by the museum partners to the project, of making some general statements about events that seem to be related and using this as a guide for incrementally specializing the plot relations.
All of the above reasoning processes can be applied on subsections of stories (e.g., plot elements) or on expanded stories. The diagram in figure 6 illustrates the process of story expansion.
Figure 6. Widening and narrowing a story by adding new events and emplotment using external data sources
In figure 6, A shows an initial set of story events. B indicates widening the scope. Events in shaded area are close to the original story core. C shows finding entity relations. For example, A influenced B, and A was in a peer relationship with C. The agent can be in the original story scope or increased scope. D shows the narrowing of the scope: find related plot elements and repeat plot reasoning to this narrower story scope; or widen the scope again from the story, including events added in the previous step.
These reasoning processes may support an author by automatically populating a dossier with events that are relevant to the story, making them immediately viewable on a timeline or a map for little effort. They might also propose events that add an historical context that they had not previously considered. For example, the external information sources such as Freebase and Factforge contain information about plays, books, sporting events, military events, etc. A non-expert museum visitor using Storyscope may find these reasoning processes useful to extend a story in a direction more fitting with their own personal interests.
7. Structuring a narrative output
When a story is complete, it can be output in the form of a narrative. A narrative has a structure that determines how story information, such as plot and theme, are presented for an audience, and that determines the organization of the objects within the narrative presentation.
The reasoning support for narrative can assist an author in structuring a narrative to produce different dramatic effects and to deal with conflicts among thematic “related” plots and other types of plot information. These issues can occur since the author can specify any number of plot elements in the story plot, potentially using the same event in multiple plot elements.
While in an online space there is the possibility of dealing with these using hyperlinks, or by having an event (and a linked object) appear more than one time, in most cases there will be a need to present the narrative linearly and with a constraint that an object can appear only once. The narrative reasoning proposes which principles should be used for grouping the events of the story, and how events should be organized within this scheme. Since some events derive from the individual object stories, this in turn can be used to propose how objects are organized within the narrative.
The narrative reasoning suggests a primary plot-based organizing principle, categorizing the events among a selected set of plot elements. This gives an initial grouping of events. A secondary organizing principle can be found that reinforces this. This proposes how the plot elements should be ordered with respect to one another, and how their internal organization should be arranged. For example, it may be possible to organize the plot elements and their constituent events so that they largely follow a chronological ordering. Alternatively, it may be possible to align the plot elements with other event properties such as location or the type of activity that occurred in the event. This secondary organizing principle reinforces the initial organization and can be reflected in the narrative organization: for example, using a timeline or map alongside the thematic groupings of objects and associated events. This process can be repeated recursively on sub-components of the narrative.
8. Creating drama and increasing tension
The two-part plot relations (influenced, motivated, in reaction to, and inspired) each have a source and a consequence. One way to organize the narrative is to take the two-part plot relations and present all source events (and related objects) first, followed by all of the consequence events. A secondary organizing principle can determine the grouping/ordering of events/objects within the source group and the consequence group, chosen using statistical analysis to find the best complementary principle (e.g., organizing by time, or finding that all of the source events happened in Florence and Rome and all of the consequence events occurred in London). This narrative organization increases tension by introducing many plot elements before resolving them, to “find out what happened.” This narrative organization principle is illustrated in figure 7. Note that the time in the diagram refers to narrative time, the time span in which the events and objects are encountered during the viewing of the narrative. The story time, when events of the story were occurring, may or may not be linearized in the narrative presentation.
Figure 7. Increasing tension in a narrative.
Conversely, if each plot element is completed in its entirety before the next one starts, this minimizes the tension (figure 8). In this example, the secondary organizing principle shown is the story time.
Figure 8. Minimizing tension in a narrative
A third approach for plot-based organization is to escalate or diminish tension over time through the choice of plot-element type. This is based on a hierarchy of plot relations, with motivated being assigned a “low drama” value and influenced a “high drama” value. Therefore, if motivated plot elements are all presented first and then influence plot elements come later, the tension increases over time. As in the previous case, a secondary organizing principle can be suggested to complement this plot organization. The different plot types can also be presented, as in the case for maximizing tension: for example, by introducing all sources of motivation, then all consequences, followed by all sources of influence and all consequence of influence (figure 9, example 1); or else each individual plot element can be introduced and then completed (figure 9, example 2). The first case would be considered more dramatic than the second.
Figure 9. An example of building tension using different plot types
9. Related/thematic organizing
The related plot elements can themselves be used as the primary organizing principle. Again, statistical analysis can be used to find the strongest corresponding secondary organizing principle. A related plot element may contain many subplots, either other more specific related plot elements, or two-part plot elements specifying an influence or motivation across two sets of events. In the latter case, a further narrative organization within the component might determine the best dramatic presentation of these subplots, according to the principles outlined in the previous sections.
However, in many cases plots will overlap. For example, there might be a strong interaction between two related plot elements and the locations in which they occur, such that in a related plot about Denis Mahon’s interest in the painters Caracci and Guercino, most of the events might occur in Rome and Bologna; whereas in another related plot element about Denis Mahon’s collection of Bartolozzi engravings, the majority of events occur in London. Spanning these two might be the plot element that was explored earlier (see figure 3) showing the relation between Guercino and Bartolozzi. This is illustrated in figure 10, which also shows how an event ordering might produce an organization of associated objects, where events in the organized narrative derived from the original object stories that were introduced in figures 2 and 3.
Figure 10. A narrative organized by related plots, showing how event ordering can produce an organization of objects for a narrative output
10. Presenting the final narrative
The form of the final narrative can be varied. Similarly, the extent to which the underlying story events and plot are explicitly conveyed alongside the objects can differ: for example, a case where objects are presented on their own without any story-related information but still organized by the underlying story principles; or a case where the story and plot are written out in their entirety. Whatever the case, a key aim of Storyscope is to allow the authoring of narratives that can be linked back to their originating story and dossier, even if a physical connection no longer exists. To this end, the contents of a dossier can be published along with a narrative, including the reference materials, unused objects, and events. It is even possible that users can create their own versions of a dossier within which they can change, reduce, and expand the published story, and from which they can produce their own narrative. End users have access to the same reasoning support as original authors. Therefore, as they choose a different perspective (e.g., by filtering out objects that they are not currently interested in), the theme and setting of the story can change, thus altering the externally suggested events and plot relations.
We would like to acknowledge the contribution of the National Gallery of Ireland, the Irish Museum of Modern Art, and the Dublin Institute of Technology. This work is conducted as part of Decipher, an EU Framework Programme 7 project in the area of Digital Libraries and Digital Preservation.
Campione, P., N. Di Blas, M. Franciolli, M. Negrini, & P. Paolini. (2011). “A “smart” authoring and delivery tool for multichannel communication.” Museums and the Web 2011. Philadelphia, PA.
Chatman, S. (1980). Story and discourse: Narrative structure in fiction and film. Cornell University.
Di Blas, N., P. Paolini, & L. Spagnolo. (2012). “PoliCultura Portal: 17,000 students tell their stories about cultural heritage.” Museums and the Web 2012. San Diego, CA.
Exner, P., & P. Nugues. (2011). “Using semantic role labeling to extract events from Wikipedia.” In Proceedings of the Workshop on Detection, Representation, and Exploitation of Events in the Semantic Web (DeRiVE 2011). Workshop in conjunction with the 10th International Semantic Web Conference 2011, Bonn, Germany.
Hienert, D., & F. Luciano. (2012). “Extraction of historical events from Wikipedia.” In Proceedings of the First International Workshop on Knowledge Discovery and Data Mining Meets Linked Open Data, Heraklion, Crete.
Mulholland, P., A. Wolff, A., & T. Collins. (2012). “Curate and storyspace: An ontology and web-based environment for describing curatorial narratives.” In: 9th Extended Semantic Web Conference (ESWC 2012), Heraklion, Greece.
A. Wolff, P. Mulholland and T. Collins, Modeling the Meaning of Museum Stories. In Museums and the Web 2013, N. Proctor & R. Cherry (eds). Silver Spring, MD: Museums and the Web. Published January 30, 2013. Consulted .