Enriching multimedia expression with tangible objects: the learning benefits for preschoolers

ProcediaSocial and Behavioral Sciences Procedia - Social and Behavioral Sciences 00 (2012) 000–000

www.elsevier.com/locate/procedia

INTERNATIONAL EDUCATIONAL TECHNOLOGY CONFERENCE IETC2012

Enriching multimedia expression with tangible objects: the learning benefits for preschoolers

Chau Kien Tsonga,*, Toh Seong Chongb, Zarina Samsudinc

[email protected], [email protected], [email protected] For Instructional Technology and Multimedia,Universiti Sains Malaysia, 11800 USM, Pulau Pinang, Malaysia

Abstract

The pedagogical benefits of tangible objects are abundant. However, the adaptation of tangible objects into multimedia learning for preschoolers is still remained absent. In view of this, this paper is produced with objective to discuss the advantages of using tangible objects by way of literature review. For tangible objects embraced in multimedia realm, multimedia objects can complement them meaningfully by providing fascinating multimedia expression and effects like games and versatile clues for preschoolers to know what kind of interaction is expected for tangible objects during the course of learning. A relevant prototype named TangiLearn has been developed for a case study. Its finding is briefly reported.

© 2011 Published by Elsevier Ltd. Selection and/or peer-review under responsibility of [name organizer]

multimedia; multimedia object; tangible object; tangible multimedia.

* Corresponding author. Tel.: 010-5059150.E-mail address: [email protected].

mailto:[email protected]

mailto:[email protected]

2 Chau, Toh, & Zarina / Procedia - Social and Behavioral Sciences 00 (2012) 000–000

1. Introduction

Developing a truly beneficial multimedia learning system for preschoolers need extra considerations. Preschoolers are in a category where much behavior is described by its own specific characteristics. They have limited vocabulary, limited degree of motor acuity (Huang, Smith, Spreen, & Jones, 2008; Read, MacFarlane, & Casey, 2002), and their reasoning facilities are not yet fully developed. They are at the “preoperational state” of cognitive development structure where their cognitive state develops through senses stimulated by external concrete stimuli (Piaget, 1952, 1972). Although the efficacy of the multimedia systems nowadays is confirmed in many different lines of research (Vickneasvari, 2007; Chen, 2005), they may not be equally suitable for preschoolers who have very varied needs and limitations described above. For example, virtual reality (VR) has been established for sometimes as a tool efficient for many aspects, including education (Hole & Schull, 2009; Chen, 2005). However, navigating a scene in VR is apparently a formidable task for preschoolers. Many systems are overwhelmed by a large quantity of instructions that often exceed the learning abilities of the children (Mohamad, Majid, Souriyavongsa, Chin, & Ong, 2011). As such, we observe a learning gap between the preschoolers and multimedia system.

Fig. 1 Gap between multimedia learning system and preschoolers

A new paradigm shift in multimedia system that truly adapted to their characteristics, hand motor skills, and underlying cognitive developmental abilities has to be sorted out. As tangible objects surrounding children are characterized by offering full range of sensory qualities, they can serve as an excellent candidate to bridge the learning gap.

Chau, Toh, & Zarina / Procedia - Social and Behavioral Sciences 00 (2012) 000–000 3

Fig. 2 Tangible multimedia bridges learning gap


However, some may doubt about the advantages and pedagogical possibility of tangible objects in multimedia realm. To convince them of the need of such adaptation, we thus produce this paper with an objective to discuss the learning benefits of using tangible objects by way of literature, and then discuss why the merging of tangible objects and multimedia objects is required. We propose to name such multimedia “tangible multimedia learning system”, or in short, tangible multimedia. Prior to actual experimental research (Chau, Toh, & Zarina, 2011), we have conducted a relevant case study and this paper also provides brief report of the case study.

2. The Learning Benefits of Incorporating Tangible Objects for Preschoolers

Tangible object denotes an intuitive and natural ways of learning (Ishii & Ullmer, 1997). Even though the so-called “natural” way is difficult to measure objectively, and is subject to interpretation (Forlines, Wigdor, Shen, & Balakrishnan, 2007), one can be aware that it has something to do with the engagement of primary senses of the human, particularly hands. With tangible objects, the preschoolers will have to directly hold objects with one or both hands, point, grasp, move, and release. Klemmer, Hartmann, and Takayama (2006) said “A natural place to start is with our hands, as they are simultaneously a means for complex expression and sensation …” (p. 143). Morsella and Krauss (2004) said, if hands are prevented from learning, various cognitive operations will be degraded. Indeed, hands are crucial for children in learning (Klemmer, Hartmann, & Takayama, 2006). Children learn through hands as natural motor response in human daily living, just like toddlers, they like to touch things, and without realizing, they learn.

Tangible objects bring tangibility experience into learning. With tangible objects, the children can concretize the abstract concepts that they never see before. According to Papert (1991), children learn most when they are engaged with concrete activities meaningfully to them. Zuckerman & Resnick (2003) asked, “How do we ‘concretize the abstract’? A physical, tangible interface can help. Children can touch it, can tinker with it …..” (p. 811). In defining the term “abstract”, Manches (2010) said, “abstract is in the sense of being disconnected from perceptual experience” (p. 72). It is only the concrete objects that can give perceptual experience. Tangible objects can give rise to mental images in the children mind, thus making abstract concepts more visible, accessible and learnable (Piaget, 1962; Hengeveld, Hummels, & Overbeeke, 2009; Antle, Droumeva, & Ha, 2009). This idea is also reflected in Montessori’s education where she stated that “Children build their mental image of the world, through the action and motor responses; and, with physical handling, they become conscious of reality” (Burnett, 1962, as cited in Marco, Cerezo, Baldassarri, Mazzone, & Read, 2009, p. 105). Empirical researches showed that children were able to perform symbol manipulation tasks with concrete physical objects when they failed to perform using abstract representations (O’Malley & Fraser, 2004). For example, Hughes’ (1986) research found that young children, who were unable to solve an addition problem presented symbolically, were able to do so when the same question was presented with concrete referents (Manches, 2010). Similar outcomes produced in research conducted by Canobi, Reeve, & Pattison (2002) on additive composition on 50 children aged 5 to 6 using physical objects. There exists a theoretical viewpoint called “embodied cognition”. It gives different view between concrete and abstract. Accordingly, concrete and abstract are inseparable. Abstract thinking is actually grounded in perceptual and sensory experiences (Manches, 2010), and this further supports the need of using tangible objects for concrete learning in multimedia realm.


Tangible objects are a logical choice for spatial activities (Xie, 2008). Piaget (1968) stated, “A world without objects would not present the character of spatial homogeneity and of coherence in displacements that marks our universe” (p. 3). Spatiality is human innate ability to interact with concrete objects in physical space (Sharlin, Watson, Kitamura, Kishino, & Itoh, 2004). In our context, spatiality means spatial information such as depth and position of physical objects required in the holistic processing of working memory system (Xie, 2008). Picture, on the other hand, can be recognized, but have no substantial spatial organization (Piaget, 1968). Patten and Ishii’s (2000) comparative research on space usage in a TUI (Tangible user interface) and GUI (Graphical user interface) showed that TUI users outperformed GUI users in the location recall task. In their research, many of the TUI users were found to use the spatial relationship between testing blocks and surrounding to assist them to recite the blocks content. Xu (2005) contended that GUI blocks human natural abilities (Xu, 2005). Spatiality in our context also means space for hand gestural operation. Tangible objects evoke space for trial-and-error activity, combine and recombine things, and perform spatial organization with more degree of freedom. All these open up window for enhancing the understanding of the preschoolers (Kelly, Singer, Hicks, & Goldin-Meadow, 2002; Manches, O'Malley, & Benford, 2009). In a study conducted on 84 children, Kelly, Singer, Hicks, & Goldin-Meadow (2002) discovered that children who need to gesture retain the mathematical knowledge better than children who only speak, but not gesture. Alibali and DiRusso (1999) in a research on 25 children discovered that children count objects most accurately if they gesture when counting.

Tangible objects offer the only way for tactile emotional experiences. Klemmer, Hartmann, and Takayama (2006) said “... they (hands) allow for complicated movement but their skin also has the highest tactile acuity of our extremities” (p. 143). Touch can convey additional meaningful information about its context (Wimmer, 2011) not accessible visually such as the softness and weight of materials. This can be evidenced from a situation where without the aid of visual, a person can recognize an object merely through touching (Jetsu, 2008). Learning merely through visual and auditory channels, like in the way the digital multimedia systems nowadays deliver information, the children cognition process to make sense of the concepts outside of children’s immediate context will be weaker. Cognition does not exist only in mind, but also during interaction between the materials and mind (Crook, 1992).

Fig. 3 Engagement of vision and touch via tangible object


Another important advantage of using tangible objects is their potential of supporting the working memory of a learner (Manches, 2010). This may be due to the fact that tactile information helps children to offload some cognitive burden and mental operation in learning (Manches, 2010; Antle, Droumeva, & Ha, 2009). This can be evidenced from the capability of abacus experts. Using various memory tasks, Hatano and Osawa (1983) tested 3 abacus experts and found that they had a very large memory span of numerical digit strings and arithmetic abilities than the average. Accordingly, due to long use of abacus, the experts have internalized a mental model called “mental abacus” that mimics the structure of an abacus to keep visuo-spatially a number comprising long digits. Chao, Haxby, & Martin (1999) called this mental model the “tool of mental sight” (Marco, Cerezo, Baldassarri, Mazzone, & Read, 2009).

3. Complementing tangible objects with multimedia

If the preschoolers are to learn the real world, tangible objects are obviously a good choice. However, the sole adoption of tangible objects in learning imposes certain constraints that hinder learning. One such constraint is the behavior of tangible objects that is governed by the laws of physics. As a natural form of material that have substance, that are permanent and of constant dimensions (Piaget, 1968), tangible objects cannot be passed trough, made disappear, moved on their own, or have their gravitational forces degraded. The manipulation of tangible objects is continuous, where its effects are gradual rather than discrete (Manches, O'Malley, & Benford, 2009), hence, the preschoolers may not be able to observe the outcomes clearly.

Another constraint is that actions performed on tangible objects are not traceable. Tangible objects do not provide any means for children to revisit any record of the previous solution for revision purposes, compare the past and present action, and to examine the relationship between different solutions for the tasks that they are attempting (Manches, 2010). There is suggestion of using video recording and written log of action. However, such suggestion is not viable because replay and search for relevant video files consume time whereas the demand of writing for written log exceeds the linguistic ability of young children (Manches, O'Malley, & Benford, 2009).

Multimedia expression overcome the aforesaid constraints by providing various multimedia effects such as pop up message, animated feedback, hints, quizzes, games, and versatile clue so that the preschoolers know what kind of action or interaction expected for the tangible objects. Through animation, invisible abstract concepts or materials in “intangible” form (like water molecule, chemical reaction) can be made visible, and behavior of tangible objects changed against physics laws (making the effects on objects discreet, colliding objects made passed through).


Fig. 4 Tangible and multimedia objects complement each other

Our proposed tangible multimedia is different from many tangible systems that aimed for facilitating human-computer interaction. Tangible multimedia in our context is that tangible objects complement digital multimedia objects for the purpose of making the multimedia contents tangible. For example, actual tangible spoon is used to represent virtual spoon in an animated movie. Via such real-life objects, digital multimedia objects are “externalized” meaningfully, and thereby gain their “tangibility”. For this reason, tangible objects are rightfully cognitive-friendly system to preschool whereas tangible systems are more as interaction-friendly system to users.

Fig. 5 Making digital multimedia objects tangible

4. Findings of case study for TangiLearn

Evidence for the pedagogical benefits of the tangible objects is abundant. This drives us to develop a prototype of tangible multimedia, TangiLearn for preschoolers (Chau, Toh, & Zarina, 2012a). To gather preliminary evidence to support the assumption that tangible multimedia is beneficial to preschoolers, a one-day case study on six preschoolers aged six from a kindergarten in Kuala Lumpur has been conducted (Chau, Toh, & Zarina, 2012b). Unstructured observation and questionnaires were utilized. The tangible and multimedia objects binding were implemented using Quick Response (QR) code marker and ActionScripts 3.0 library.

TangiLearn was a learning environment consisted of many virtual and tangible learning objects, such as animals and household items placed randomly in front of computer (Figure 6).


Fig. 6 Randomly-placed learning objects in TangiLearn using QR marker in case study

The participants were required to grasp a tangible object, and align it to the webcam to play corresponding animations and videos about the object on the computer screen. With this, the learning activities commenced. Upon completion of learning session, the participants would need to answer the quiz by identifying and picking up the correct tangible object.

The study revealed that 4 participants rated TangiLearn “enjoyed very much” and 3 participants rated that the quiz was very easy. Through observation, we noticed a more natural form of learning between the participants and multimedia. They grasped the tangible objects firmly and spent some time tinkered with them. We believed that the good performance of the participants in quiz, which require recitation of key terms learned in the system, were attributed to iterative tangibility sense of learning experiences.

A few technical problems arose during the case study. The most obvious problem was related to the issue of physical alignment of markers. Most of the participants have difficulties in orientating the visual markers to the camera precisely. By average, they took 35 seconds to get the visual marker recognized. Besides, they were very curious about the markers, thus they tended to tear them down for fun. The markers were huge, and thereby blocked some parts of the tangible objects. We also observed that using such markers, the participants could not move the tangible objects too close or too far from the camera. All these problems have to be addressed in the full-scale experimental research.

Problems encountered above prompted us to explore alternative technology for implementing the tangible-multimedia binding. Considering the choice of technology should rest on its usefulness to the students as learning aids, we plan to deploy RFID and sensors technology in the final experimental research. Because of its cost, they were not deployed in the case study. For the purpose of reducing the need of precise alignment, RFID tags and reader are deployed for objects recognition. To do this, RFID tag is inserted into a tangible object. When the tangible object is moved towards the radio wave field generated by a compatible reader, the chip in the tag will transmit the stored information to the reader, thereby establishing mutual communication that allows the computer to identify the object (Figure 7).


Fig. 7 Implementation of RFID technology

To further enhance the tactile attribute of digital objects in physical space, a type of sensor called force sensor is utilized. The force sensor is designed in a way that the sensor is glued on the tangible objects. The more the preschooler grasps and presses the sensor via the tangible objects, the more the digital multimedia objects in TangiLearn will react, such as movie clip moves further in the learning scene. We also plan to use various types of sensors and controller (e.g. slider, wii mote controller) in TangiLearn. With excessive use of RFID and sensors technology, it is expected that the aforesaid learning benefits of tangible objects in multimedia realm can be garnered in TangiLearn.

Fig. 8 Implementation of force sensor technology

5. Conclusion

In this paper, we discuss how tangible objects are beneficial to preschoolers by way of literature review. Taking the strength of physical sensation, TangiLearn, a manifestation of tangible multimedia, is designed. Evidence of improving self-report of enjoyment and learning outcomes in the case study reflect the need of such system for preschoolers. In the case study, we were also informed of the change required for the technical implementation of the TangiLearn system, from QR code to RFID technology.

References

Alibali, M. W., & DiRusso, A. A. (1999). The function of gesture in learning to count: More than keeping track. Cognitive Development, 14(1), 37-56.

Antle, A. N., Droumeva, M., & Ha, D. (2009). Hands on what?: Comparing children’s mouse-based and tangible-based Interaction. Proceedings from IDC ’09: The 8th International Conference on Interaction Design and Children, 80-88.

Burnett, A. (1962). Montessori education today and yesterday. The Elementary School Journal, 63, 71-77.


Canobi, K. H., Reeve, R. A., & Pattison, P. E. (2002). Young children’s understanding of addition concepts. Educational Psychology: An International Journal of Experimental Educational Psychology, 22(5), 513-532.

Chao, L. L., Haxby, J. V., & Martin, A. (1999). Attribute-based neural substrates in posterior temporal cortex for perceiving and knowing about objects. Nature Neurosci, 913–919.

Chau, K. T., Toh, S. C., & Zarina Samsudin. (2011). Tangible multimedia technology: A research proposal for bringing tangibility into multimedia learning amongst preschool children. Proceedings of the 5th International Malaysian Educational Technology Convention (IMETC 2011), Kuantan, Malaysia, October 16-19, 2011.

Chau, K. T., Toh, S. C., & Zarina Samsudin. (2012a). Designing tangible multimedia for preschool children based on multimedia design theories. Proceedings of the International Conference for Academic Research 2012, Kuala Lumpur, Malaysia June 8 – 11, 2012.

Chau, K. T., Toh, S. C., & Zarina Samsudin. (2012b). Tangible multimedia: A case study for bringing tangibility into multimedia learning. Proceedings of the International Educational Technology (IETC 2012), Taoyuan, Taiwan, 11 – 13, 2012.

Chen C. J. (2005). The design, development and evaluation of a virtual reality (VR)-based learning environment: Its efficacy in novice car driver instruction. Unpublished doctoral dissertation. Universiti Sains Malaysia, Penang.

Crook, C. (1992). Cultural Artefacts in social development: The case of computers. In McGurk, H. (Eds), Childhood Social Development: Contemporary Perspectives, 207-231. East Sussex, UK: Lawrence Earlbaum Associates.

Forlines, C., Wigdor, D., Shen, C. & Balakrishnan, R. (2007). Direct-touch vs. mouse input for tabletop displays. Proceedings of CHI ’07 (2007), 647-656.

Hatano, G., & Osawa, K. (1983). Digit memory of grand experts in abacus-derived mental calculation. Cognition, 15(1-3), 95-110. Amsterdam, Netherlands: Elsevier Science.

Hengeveld, B., Hummels, C., & Overbeeke, K. (2008). Let Me Actuate You. Proceedings from TEI’08: The Second International Conference on Tangible and Embedded Interaction, 159-166.

Hole, J., & Schull J. (2009). Inexpensive Immersive Environment. Proceedings from Immerscom ’09: The 2nd International Conference on Immersive Telecommunications, 16.

Huang, K., Smith, J., Spreen, K., & Jones, M. F. (2008). Breaking the sound barrier: Designing an interactive tool for language acquisition in preschool deaf children. Proceedings from IDC ’08: The 7th International Conference on Interaction Design and Children, 210-216.

Hughes, M. (1986). Children and number. Oxford: Blackwell Publishers.

Ishii, H. & Ullmer, B. (1997). Tangible bits: Towards seamless interfaces between people, bits and atoms. Proceedings from CHI’97: The SIGCHI Conference on Human Factors in Computing Systems, 234–241.

Jetsu I. (2008). Tangible User Interfaces and Programming. Master Dissertation. Department of Computer Science and Statistics, University of Joensuu, Finland.

Kelly, S. D., Singer, M., Hicks, J. & Goldin-Meadow, S. (2002). A helping hand in assessing children's knowledge: Instructing adults to attend to gesture. Cognition and Instruction, 20 (1), 1-26.


Klemmer, S. R., Hartmann, B., & Takayama, L. (2006). How bodies matter: Five themes for Interaction Design. Proceedings from DIS ’06: The 6th Conference on Designing Interactive Systems, 140-149.

Manches, A. (2010). The effect of physical manipulation on children’s numerical strategies: Evaluating the potential for tangible technology. Unpublished doctoral dissertation. University of Nottingham, UK.

Manches, A., O'Malley, C., & Benford, S. (2009). Physical manipulation: Evaluating the potential for tangible designs. Proceedings from TEI ’09: The 3rd International Conference on Tangible and Embedded Interaction, 77-84.

Marco, J., Cerezo, E., Baldassarri, S. Mazzone, E., & Read, J.C. (2009). Bringing tabletop technologies to kindergarten children. HCI 2009 – People and Computers XXIII, 103-111. UK: British Computer Society. p. 105

Mohamad Jafre Zainol Abidin, Majid Pour-Mohammadi, Souriyavongsa, T., Chin Da, & Ong, L. K. (2011). Improving Listening Comprehension among Malay Preschool Children Using Digital Stories. International Journal of Humanities and Social Science, 1(14), 159-164.

Morsella, E. & Krauss, R. M. (2004). The role of gestures in spatial working memory and speech. American Journal of Psychology 117(3), 411- 424.

O’Malley, C., & Fraser D. S. (2004). Futurelab Series Report 12: Literature review in learning with tangible technologies. Bristol, United Kingdom: NESTA Futurelab. [Online]. Available: http://www.futurelab.org.uk/download/pdfs/research/lit_reviews/futurelab_review_12.pdf

Papert, S. (1991). Situating constructionism. In Papert & Harel (Eds.), Constructionism. Cambridge, MA: MIT Press.

Patten, J., & Ishii, H. (2000). A comparison of spatial organization strategies in graphical and tangible user interfaces. Proceedings of DARE 2000 on Designing Augmented Reality Environments, 41-50.

Piaget, J. (1952). The origins of intelligence in children. New York, USA: University Press.

Piaget, J. (1962). Play, dreams, and imitation in childhood. New York: W. W. Norton.

Piaget, J. (1968). The construction of reality in the child. London: Routledge & Kegan Paul Ltd.

Piaget, J. (1972). The principles of genetic epistemology. New York: Basic Books.

Read, J. C., MacFarlane, S. J. & Casey, C. (2002). Endurability, Engagement and Expectations: Measuring Children's Fun. Proceedings from IDC'02. ACM Press.

Sharlin, E., Watson, B., Kitamura, Y., Kishino, F., & Itoh, Y. (2004). On tangible user interfaces, human and spatiality. Personal and Ubiquitous Computing (8) 5, 338 - 346.

Vickneasvari A/P Krishnasamy, (2007). The effects of a multimedia constructivist environment on students’ achievement and motivation in the learning of chemistry. Unpublished doctoral dissertation. Universiti Sains Malaysia, Penang.

Wimmer, R. (2011). Grasp sensing for human-computer interaction. Proceedings from TEI ’11: The Fifth International Conference on Tangible, Embedded, and Embodied Interaction, 221-228.

Xie, Z. (2008). Comparing children's enjoyment and engagement using physical, graphical and tangible user interfaces. Master dissertation. School of Interactive Arts and Technology, Simon Fraser University, Canada.


Xu, D. (2005). Tangible user interface for children: An overview [Technical report]. Preston, UK: University of Central Lancashire.

Zuckerman, O, & Resnick M. (2003). System blocks: A physical interface for system dynamics learning. Proceedings of the 21st International System Dynamics Conference, 810-811. USA: ACM Press.

Enriching multimedia expression with tangible objects: the learning benefits for preschoolers

Documents

Transcript of Enriching multimedia expression with tangible objects: the learning benefits for preschoolers