Tangible and Social Interaction in a Museum Exhibition for Children

Authors: Francesca Rizzo and Franca Garzotto


The paper discusses the design and evaluation of a museum exhibition named The Fire and The Mountain, where we exploited hybrid (i.e., digital and physical) artifacts as well as the paradigm of tangible interaction to enhance children’ experience and to support engagement, learning, and social behavior.

Tangible Computing, Museum, Edutainment


Tangible User Interfaces (TUIs) are receiving increasing interest in museums as a mean to create hybrid installations that support visitors manipulating physical and digital artefacts, and improve their experience within an exhibition. Designing TUI requires not only designing the digital but also the physical, and their interrelations within hybrid ensembles, as well as designing forms of interaction that can be characterized as full-body, haptic and spatial. While a number of studies can be found that report the design and evaluation of TUIs in a number of traditional educational domains such as schools [3], [7], [8], relatively few empirical research exist that discuss TUIs in museum contexts especially for the support of young visitor experiences [1], [6].

This paper provides a contribution by discussing our experience in The Fire and the Mountain – a temporary museum exhibition held in 2006 that involved the TUI paradigm in a creative way. The exhibition was mainly conceived as a learning experience for visitors, to raise awareness and to foster a better understanding, especially among the young generations, on the cultural heritage of the people living on territory around the Como Lake.

The subject of The Fire and The Mountain was the relationship between human beings and the fire from prehistory to present times in the Como valleys, an area that has remained relatively untouched by the massive industrial growth of the city basin and preserves a wealth of traditions, handicrafts, ways of life, oral heritage that local institutions are attempting to make survive. Coal mines and the use of fire date back to the first human appearance in the area. They shaped not only the forests around the Como lake, but also the culture and habits of the people that lived in this area till 19th Century – the “Magnani” people, whose job was to make iron stoves using the fire and the coal miners.

The Fire and The Mountain aimed at exploiting ICT in museums in a non-conventional way, and to experiment/evaluate new forms of technology enhanced learning and social interaction in public spaces, thus promoting these concepts in local cultural end educational players and sponsors. The exhibition was composed of four digital touchable installations that support tangible interaction and are “immersed” in a physical space designed to create an emotional relationship with the exhibition subjects, by means of light and sound effects.

Next sessions introduce the principles that informed the overall design of the user experience and discuss the details of each installation. We then discuss the field study we conducted with 62 children aging from 8 to 11, to validate our design and to identify guidelines for similar museum experiences in the future.


The Fire and the Mountain was created as a joint effort of museum curators, communication and new media experts, local experts of the subject matters, video and sound artists, interior designers, interaction designers, and computer engineers. The design of the exhibition was informed by a conceptual framework based on four central design concerns, partially inspired to previous work [6], [1]:

Emotional Involvement – making young visitor experience suggestive, enjoyable, meaningful, engaging, and fluid [9], [10], by means of an enriched physical space in which art and technology as well as light and sound effects intersection is exploited to foster curiosity and to make environment “magical” to be experienced;

multi-modality: supporting different sensory modalities and engaging multiple senses – tactile, auditory, visual; discovery Learning: exposing children to a variety of contents “hidden” in the various digital media installations, and promoting their discovery by means of multiple activities, stimuli, and interaction paradigms. social Learning: supporting learning as a social process, and fostering mutual collaboration and group discussion during the execution of the different activities.


All of the four interactive installations composing The Fire and the Mountain integrate text, video, music and sound. Although they can be freely explored in any order, we conceived a narrative flow that progressively bring young visitors along an in depth exploration of the “culture of fire”. Each installation was located in a different museum room and for each of them our digital artists designed a specific sound track, to create audio effects that evoke the natural environment each of the installation refers to.

The first installation, called Virtual Book (Figure 1 and Figure 2), provides visitors with an overview of the territory around Como, its origins and history. It is a Flash application shaped as an old book whose pages can be turned by hands on a horizontal smart board; with the same interaction paradigm, kids can also manipulate videos and pictures on the pages, e.g., moving them around by “drag&drop” (Figure 1b), to create their own “personalized” multimedia book.

The second installation is the Talking Dictionary (Figure 3). It allows visitors to discover rungin – the ancient idiom spoken by the people that lived in the Como valleys until a century ago. A character (the image of an old man from the valley) on a vertical smart can be “asked to speak rungin”. Kids can drag-and-drop an Italian term on his mouth and the character starts telling a short story about the term meaning.

The third installation called Research Tables is composed of two tables equipped with horizontal smart boards (Figure 4). They allow visitors to interact with 3D objects in order to discover the characteristics of the underground of the Como territory. On the first table, different pieces of coal are reproduced on the smart board and kids may interact with them by hands (moving, rotating, asking details). The second table provides a 3D map of the geological layers that compose the underground from the prehistoric era to our days. By using different kinds of physical objects on the various levels, kids can trigger different phenomena on the environment: for example, using a wood stick, they can see how trees on the land surface have been transformed by time into coal.

The fourth installation consists in an augmented environment that reproduces a Coal Cave (Figure 5). By using a video projector, we created a virtual reproduction of an existing cave, which can be explored using a tactile mouse. The tactile mouse is a system that transforms common thin panels into tangible interfaces using four low cost microphones: the user can interact with his fingers, wooden sticks, pens, etc. and the system tracks the interaction by analysing the propagation of acoustic waves in the panel. This yields to “virtual boards” built with materials as medium density fibreboards, Plexiglas, glass, etc, where the user can write or draw and the result is displayed in real time on a screen or saved in a PC memory.

The resulting interaction space provides a hybrid environment where the representation of a real natural environment (the coal cave) is integrated with points of interactivity (the tactile mice) to interact with the environment and receive information about it. In the Virtual Coal Cave, our interior designers elaborated a space concept consisting in black walls and a series of leds, also mixing visual and audio cues in order to evocate the obscurity of the cave and the light of the fire, as well as the auditory perception that a person may experiment inside a real cave (e.g., water dropping along the rocks, or wood burned by the fire).


The objective of our field study was to evaluate the design of the overall experience by detecting potential sources of disorientation and frustration, but also to identify those aspects of the installations that were most enjoyable and beneficial for learning for young visitors. Our work exploited a design ethnography approach [2] and involved 62 children ageing from 8 to 11 who visited the exhibition and interacted with the different exhibits. Children’ interactions was observed in presence and
video-recorded, and kids were shortly interviewed during a debriefing session after the visit. Qualitative data analysis methods were applied to verbal and visual data (observer’ notes, video recordings, kids answers). The data cluster and their analysis highlighted a number of interesting issues, discussed in the rest of this section.

The role of “Technological Expertise” vs. Affordance Children did not exhibit any difficulties to interact with the installations even though they never used touch based technologies before. Still, some of them (in some cases, the “most” experts in computers) did not realize immediately that they could manipulate some objects by hands, e.g., moving pictures and movies on the Virtual Book, or using some objects (as in the Research Table).

These data highlight the role of affordance in the design of interactive tactile or tangible interfaces, suggesting the importance of rendering the actions that can be performed on interactive objects in a clear and natural way in order to reduce how much it is possible the distance between the model of interaction implemented in the digital object in use and the way in which the correspondent physical objects is used in natural/real contexts.

Expanding the Experience

Physical manipulation of a virtual environment may stimulate an attitude to expand the virtual space by means of tangible artefacts. Some 11 years old children, for example, attempted to modify the Virtual Book by placing a real paper sheet aside a virtual page: by effect of smart board projection, they indeed achieved this visual effect, as shown in Figure 6.

This data cluster highlights that physical objects can be used not only as interaction tools, as it happens in most TUIs, but also as tools to expand the digital representation of the information world users are dealing with.

Age and Social Experience 

Apparently, the tendency towards socialization in a digitally augmented physical space is affected by age, While older children (11 years old) tended to exhibit solipsistic fruition of the installations, younger children (8 years old) preferred to work in group (Figure 7). This difference was explained by children’s comments collected during the debriefing session.

Most of the older children declared that they were interested in experimenting the technology peruse, since they “wanted to see if it works as video-games”. Younger children, less exposed to technology at school and at home and more incline to physical interaction among peers, were more interested in playing and discovering multimedia effects together, sharing this pleasure with their mates. When interviewed, they reported that they “liked to touch stuff together with their best friend”.
Tangible interaction vs. instruction-driven interaction Once children realized that they could manipulate some objects by hands or by means of other objects, they found it very natural and easy, and tend to apply this model of interaction in any situation. They experienced some difficulties in interacting with objects in the “normal way” we are used in desktop interfaces, i.e., expressing a “command” by selecting an icon or a text label. We observed that children who had used tangible interaction and direct manipulation experienced a series of breakdown interacting with the second research table that required them to touch instructions such as continue; close, start to trigger some behaviours. They tried repeatedly to touch the available images on the screen to make something happen, trying to apply the model of the interaction they have learned, consciously or unconsciously avoiding to read and interpret the instructions.

It seems that while the shift of interaction paradigm from an instruction-driven model (in which interaction is mediated by a textual or iconic element that expresses the meaning of the action to trigger) to a physical manipulation model is easy and natural but the shift in the opposite direction is not so easy to be performed.

Contextual Affordance

Children moved in the space because they were driven by both intentional motivations (e.g., personal interests and preferences) and the properties of the environment. In other words, the flow of children’ activities and interactions with the digitally augmented space seemed to be largely driven by the contextual affordances, i.e., the designed characteristics of the physical place – such as wall colors, the lights of the installations and their positions, the audio cues and sound tracks in the different environments, the archi-tectonical elements (e.g., the points of entrance to the exhibition, the location of the different installations in the different museum rooms) – as well as the social context, e.g., the number of children in front of an installation. In the context of digitally augmented physical spaces for museums, designing effective and coherent contextual affordances, which are appropriate for the visitors’ profile and the subject matter of the exhibition itself, is surely a critical and challenging issue.

In this paper, we have discussed the design experience of The Fire and the Mountain, a digitally augmented museum exhibition in which we exploited interaction paradigms based on hybrid artifacts, which support visitors manipulating and interacting with, and by means of, physical, and digital objects [4], [5], [9], [6].

The field study that we conducted to evaluate The Fire and The Mountains with young children suggests a number of directions for research in cultural heritage communication, learning in museum contexts, and, more generally, HCI.


Authors are grateful to Prof. Piero Fraternali which led “The Fire and the Mountain” project, and to all members of the design team.


1. Bannon, L., Benford, S. & Heath C. (2005). Hybrid Design Creates Innovative Museum Experience. Communication of the ACM, Vol. 48, No 3, 62-65.

2. Beyer, H. and Holzblatt, K. (1998). Contextual design. Defining customer-centered systems. San Francisco: Morgan Kaufmann.

3. Bruckman, A., Druin, A., Inkpen, K., & Preece, J. (2001) “The Children’s Challenge: New Technologies to Support Co-Located and Distributed Collaboration, Report on the CSCW 2000 Panel. In Computers and Kids (Ed. Druin, A.) SIGCHI Bulletin, March/April 2001, pp. 5-6.

4. Csikszentmihalyi, M., & Hermanson, K. (1994) Intrinsic motivation in museums: why does one want to learn? In The Educational Role of the Museum. (Ed. Hooper-Greenhill, E.) London: Routledge, pp.146-160.

5. Falk, J.H., Dierking, L.D. (2000), Learning from Museums: Visitor Experiences & the Making of Meaning, Walnut Creek, Altamira Press, CA.

6. Hall, T. & Bannon, L. (2005). Designing Ubiquitous Computing to Enhance Children’s Interaction in Museum, Proceeding of 4th International Conference on Interaction Design for Children, Boulder, Colorado.

7. Lepper, M. R., Malone, Th. W. Making learning fun: A taxonomy of intrinsic motivations of learning. In R. E. Snow & M. J. Farr (Eds.), Aptitude, learning, and instruction. Cognitive and Affective Process Analysis Vol. 3, Hillsdale, NJ: Lawrence Erlbaum, 1987

8. Ormerod, F., & Ivanic, R. (2002) Materiality in children ’s meaning-making practices. Visual Communication, Vol. 1, No. 1 (February 2002), 65-

9. Rogers, Y. et al (2004) Ambient wood: designing new forms of digital augmentation for learning outdoors, in Proceedings of IDC Conference 2004, USA.

10.Wood, D. & O’Malley, C. Collaborative learning between peers: An overview. Educational Psychology in Practice (1996), 11 (4), 4-9.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s