Realistic versus Schematic Interactive Visualizations for...

13
62 International Journal of Information and Communication Technology Education, 10(2), 62-74, April-June 2014 Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. ABSTRACT Many benefits have been claimed for visualizations, a general assumption being that learning is facilitated. However, several researchers argue that little is known about the cognitive value of graphical representa- tions, be they schematic visualizations, such as diagrams or more realistic, such as virtual reality. The study reported in the paper investigated whether the type of visualization (schematic versus realistic) has an effect on undergraduate students’ learning of surveying practices (specifically, ‘chaining’). The study compared two interactive virtual learning environments, one containing realistic visualizations of terrains and instruments, and one containing schematic graphical representations. Results of an experiment with 62 students show that there were not significant differences in learning between students who were exposed and interacted with the realistic visualizations versus those who interacted with the schematic ones. Realistic versus Schematic Interactive Visualizations for Learning Surveying Practices: A Comparative Study Hazar Dib, College of Technology, Purdue University, West Lafayette, IN, USA Nicoletta Adamo-Villani, College of Technology, Purdue University, West Lafayette, IN, USA Stephen Garver, College of Technology, Purdue University, West Lafayette, IN, USA Keywords: Multimedia Learning, Realistic Visualizations, Schematic Visualizations, Surveying Education, Surveying Practices INTRODUCTION Surveying is “…the science and art of mak- ing all essential measurements to determine the relative position of points and/or physical and cultural details above, on, or beneath the surface of the Earth, and to depict them in a usable form, or to establish the position of points and/or details…” (American Congress on Surveying and Mapping - ACSM). Teaching construction surveying presents many challenges such as limited students’ access to instruments, limited availability of terrains on which to practice, dependence on DOI: 10.4018/ijicte.2014040106

Transcript of Realistic versus Schematic Interactive Visualizations for...

Page 1: Realistic versus Schematic Interactive Visualizations for ...hpcg.purdue.edu/idealab/pubs/IJICTE14.pdfthat are processed in the visual sensory system” (Scheiter et al., 2008, p.

62 International Journal of Information and Communication Technology Education, 10(2), 62-74, April-June 2014

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

ABSTRACTMany benefits have been claimed for visualizations, a general assumption being that learning is facilitated. However, several researchers argue that little is known about the cognitive value of graphical representa-tions, be they schematic visualizations, such as diagrams or more realistic, such as virtual reality. The study reported in the paper investigated whether the type of visualization (schematic versus realistic) has an effect on undergraduate students’ learning of surveying practices (specifically, ‘chaining’). The study compared two interactive virtual learning environments, one containing realistic visualizations of terrains and instruments, and one containing schematic graphical representations. Results of an experiment with 62 students show that there were not significant differences in learning between students who were exposed and interacted with the realistic visualizations versus those who interacted with the schematic ones.

Realistic versus Schematic Interactive Visualizations for

Learning Surveying Practices:A Comparative Study

Hazar Dib, College of Technology, Purdue University, West Lafayette, IN, USA

Nicoletta Adamo-Villani, College of Technology, Purdue University, West Lafayette, IN, USA

Stephen Garver, College of Technology, Purdue University, West Lafayette, IN, USA

Keywords: Multimedia Learning, Realistic Visualizations, Schematic Visualizations, Surveying Education, Surveying Practices

INTRODUCTION

Surveying is “…the science and art of mak-ing all essential measurements to determine the relative position of points and/or physical and cultural details above, on, or beneath the surface of the Earth, and to depict them in a

usable form, or to establish the position of points and/or details…” (American Congress on Surveying and Mapping - ACSM).

Teaching construction surveying presents many challenges such as limited students’ access to instruments, limited availability of terrains on which to practice, dependence on

DOI: 10.4018/ijicte.2014040106

Page 2: Realistic versus Schematic Interactive Visualizations for ...hpcg.purdue.edu/idealab/pubs/IJICTE14.pdfthat are processed in the visual sensory system” (Scheiter et al., 2008, p.

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

International Journal of Information and Communication Technology Education, 10(2), 62-74, April-June 2014 63

weather conditions, need for one to one training, difficulty in assessing the individual student’s performance with accuracy, and more. Recently, several researchers have recognized the poten-tial of interactive visualizations for enhancing students’ learning of surveying concepts and practices. However, no study reported in the literature has investigated which specific as-pects of visualizations are most effective for learning surveying. Although visualizations can have fundamentally different structural features, serve diverse functions and convey different content for different target audiences, in educational research, they are often treated as a single, uniform entity and, as a result, “..reviews on learning with visualizations are equivocal, with studies showing widely vary-ing effects (negative to positive) on learning.” (Scheiter et al., 2009).

In the context of surveying education, all visualizations described in the literature pres-ent a high degree of realism. However, it is not known yet whether a realistic visualization that represents objects and processes with high fidelity is more effective at facilitating learning of surveying practices than a schematic visu-alization that illustrates the same objects and processes with diagrams and line drawings. The study reported in the paper fills this knowledge gap by answering the question of whether the amount of realistic detail of interactive visual-izations has an effect on undergraduate students’ procedural learning of chaining. The findings of the study reported in the paper are important as they can help educational researchers and visu-alization designers decide if they should take on the substantial cost and time-consuming effort to develop a highly realistic visualization (such as a photorealistic virtual learning environment) when simple line drawings or diagrams might be as or more effective for those particular learning objectives and target users.

The paper is organized as follows: in section 2 we review existing visualization taxonomies, discuss prior experiments that compared realis-tic versus schematic visualizations, and present

a review of recently developed visualizations for surveying education. In section 3 we describe the study and in section 4 we discuss the find-ings and outline future work.

BACKGROUND

Classification of Visualizations

In this paper, we use the definition of Scheiter and colleagues, who characterized visualiza-tions as “external representations that are intended to communicate information by using a visuo-spatial layout of this information and that are processed in the visual sensory system” (Scheiter et al., 2008, p. 3). Visualizations range from simple diagrams and black and white line drawings, to complex animations, multimedia and virtual reality. Various classifications of visualizations have been proposed. Some tax-onomies focus on the structural features of the visualizations (e.g. form and physical aspects that can be observed objectively), some are based on their functional features (e.g. use and purpose) and a few take into consideration also content features. Examples of functional clas-sifications include the ones by Macdonald Ross (1973) who suggested how numbers should be shown based on task criteria and purpose of communication, and Tufte (1983). Structural classifications include the one by Rankin (1990) who proposed structural categories of graphs, and the one by Lohse et al. (1994) who clas-sified visualizations based on subjects’ ratings of the visual similarity between graphical rep-resentations. Lohse and colleagues identified 6 basic categories of visual representations (e.g. graphs, tables, maps, diagrams, network and icons) and proposed two dimensions that further distinguish these categories. One dimension suggested that a graphical representation can express either continuous or discrete informa-tion; while the second dimension suggested that some visualizations are more efficient than others for conveying information. Imhof et al.

Page 3: Realistic versus Schematic Interactive Visualizations for ...hpcg.purdue.edu/idealab/pubs/IJICTE14.pdfthat are processed in the visual sensory system” (Scheiter et al., 2008, p.

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

64 International Journal of Information and Communication Technology Education, 10(2), 62-74, April-June 2014

(2009) proposed a classification system cover-ing the structural features, the functions, and the depicted contents of visualizations. Their taxonomy is based on seven structural dimen-sions, five functional features and seven content features. The seven structural dimensions are semiotic code, production technique, degree of dynamism, form of interactivity, combination with text and/or audio, amount of realistic detail, and presence of additional cues like arrows or highlights. The functional features include the affective functions, the attention-guiding functions, the (text-/picture-) supplementation functions, the working memory supporting functions, and the long-term memory supporting functions. The content features include genre (expository, narrative, or procedural), degree of realism (realistic versus fictional content), degree of identification, coherence, complexity, amount of detail of the depiction, and domain.

As previously mentioned, very few stud-ies found in the literature focused on specific aspects of the visualizations and studied their effect on learning. The study reported in the paper was concerned with one specific structural feature of visualizations, e.g. amount of realistic detail, and investigated its effect on students’ procedural knowledge of surveying practices.

Realistic versus Schematic Visualizations

Realism is the presentation of objects and pro-cesses as they actually are, without idealization or abstraction. In realistic visualizations, the similarity between real-world and represented objects is achieved by imitating the real-world referent with respect to color, shape, textures, spatial relationships, or movement (Scheiter et al., 2009). One of the advantages of realistic visualizations (such as VR) is that objects can be constructed to have a higher level of fidelity with the objects they represent when compared with other kinds of visualizations, e.g. diagrams and line drawings. Schematic visualizations, on the other hand, may consist of static line drawings with little or no resemblance to the

real-world objects as far as shape, color, texture, or motion (Scheiter et al., 2009).

It is not clear yet what are the cognitive advantages of representing the world at higher levels of realism than at higher levels of abstrac-tion and the few research studies that compared realistic and schematic visualizations have yielded conflicting results.

Those researchers who believe in the cognitive benefits of realistic visualizations argue that learning and training applications can benefit through having a greater fidelity with the represented world (e.g. Cover et al., 1993). One suggestion is that the transfer of training to the real world could be more direct, and therefore with less errors.

Due to the high resemblance of the depicted objects with the real ones, realistic visualizations may facilitate their recognition and they might especially be helpful for learning, if the learn-ing objective requires interacting with concrete objects in real-world settings (Betrancourt & Tversky, 2000).

Gonzales (1996) conducted a study that investigated the effects of image type, transi-tions, and interactivity styles used in animated interfaces in two decision-making domains. In-terfaces used either realistic or abstract images. Results showed that subjects performed better with animated interfaces based on realistic rather than abstract images. Van Gendt and Verhagen (2001) explored the effect of level of realism of the visualization on students’ performance on a visual test on the anatomy of a rat. The results showed that realistic pictures proved superior for recognizing anatomical features, whereas draw-ings yielded better results for understanding the relationship between anatomical structures. In an experiment by Goldstone and Sakamoto (2003) participants showed better understanding of the abstract principles underlying computer simulations of complex adaptive systems when the simulation included concrete rather than schematic graphical elements. However, for poor performers, the abstract version of the simulation transferred better to a new simula-

Page 4: Realistic versus Schematic Interactive Visualizations for ...hpcg.purdue.edu/idealab/pubs/IJICTE14.pdfthat are processed in the visual sensory system” (Scheiter et al., 2008, p.

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

International Journal of Information and Communication Technology Education, 10(2), 62-74, April-June 2014 65

tion governed by the same abstraction. Höffler and Leutner (2007) conducted a meta-analysis of 26 primary studies that compared dynamic and static visualizations with varying degree of realistic detail for acquiring different types of knowledge. Highly realistic animations proved superior to the schematic ones, especially when procedural-motor knowledge was to be acquired. Rieber et al. (2004) found that realis-tic simulations of Newtonian physics promote implicit learning, however this experience must be augmented with verbal explanations in order promote conceptual understanding.

Researchers who support the cognitive advantages of schematic representations have argued that simplified, relatively abstract representations are useful for extracting the es-sence of a situation (Gianutsos, 1994). Realistic visualizations entail more irrelevant details and therefore might distract learners and take their attention away from the fundamental aspects (Dwyer, 1976). By schematizing visualizations, key elements, which might be difficult to filter in reality, can be easily perceived by the view-ers. Schematic visualizations may facilitate symbolic interpretations of represented objects (Goldstone & Son, 2005) and abstract reasoning (Schwartz, 1995). Schwartz (1995; Schwartz & Black, 1996) conducted a study in which subjects were asked to determine whether marks on hinges and gears would come to contact if the mechanisms were put in motion. When the representation of the mechanisms was photo-realistic, participants were able to simulate the physical behavior of the hinge through analog imagery (e.g. closing or rotating into position). In contrast, when the representation of the mechanisms was a diagram, subjects were able to extract the static features of the display such as line lengths and angles. Schwartz and Moore (1998) found that a proportional reasoning problem illustrated in a diagram benefitted mathematical reasoning more than when displayed in a photograph. Although these experiments do not provide scientific evidence of the benefits of abstract diagrams over more realistic representations, they suggest that an advantage of schematization might be that it

promotes more abstract reasoning that is less influenced by physical constraints.

Dwyer and colleagues conducted a study in which students learned the anatomy of the human heart using static pictures and drawings that had different amounts of realistic detail (Dwyer, 1968). Results showed that students who were exposed to the simple line drawings performed better in the drawing test which assessed learning of specific locations of the various patterns, structures, and positions of the parts of the heart, as well as on the total criteria test that measured the student’s overall under-standing of the concepts presented. In contrast, realistic photographs proved superior for the identification test, which measured transfer of learning, i.e., the ability to identify numbered parts on a diagram of the heart. Findings of a study by Arnold and Dwyer (1975) suggested that “the simple line drawings are most effec-tive in facilitating achievement for learning objectives measuring cognitive abilities higher than the knowledge level of understanding. At these levels of cognitive ability it appears that the superfluous information contained in the realistic visualizations is not germane to concept attainment. The data indicate that for learning objectives at the level of understand-ing, increases in the amount of visual detail …improve achievement”. Scheiter et al. (2009) conducted an experiment in which students learned about cell mitosis watching either realistic or schematic dynamic visualizations; findings showed the superiority of the schematic visualizations.

In summary, there is no consensus on which type of visualization is more effective for learning. The results achieved by the stud-ies conducted so far suggest that schematic visualizations tend to benefit more abstract reasoning based on a symbolic interpretation of the depicted object, while realistic visualiza-tions tend to enhance more analogical problem solving skills. The studies also indicate that there is a strong interaction between the type of visualization and the learning objectives, as well as the type of visualization and the learner characteristics.

Page 5: Realistic versus Schematic Interactive Visualizations for ...hpcg.purdue.edu/idealab/pubs/IJICTE14.pdfthat are processed in the visual sensory system” (Scheiter et al., 2008, p.

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

66 International Journal of Information and Communication Technology Education, 10(2), 62-74, April-June 2014

Surveying Education

Surveying is a fundamental course in the Civil Engineering, Building Construction Manage-ment, Geomatics, Geography, Agriculture & Forestry, and Landscape Architecture cur-ricula. Surveying is a core course in the Civil Engineering, Building Construction Manage-ment, Geomatics, Geography, Agriculture and Forestry, and Landscape Architecture curricula. Traditionally, a surveying course includes three components: (1) the theoretical foundation of surveying, which includes math, trigonometry, geometry and physics concepts (often instruc-tors teach fundamental concepts of surveying using examples from textbooks and illustrations on the chalkboard); (2) instructor demonstration of functionality and manipulation of real sur-veying instruments; and (3) student practice (in groups) with real instruments. This traditional way of teaching construction surveying presents several challenges.

First, many surveying instruments are re-quired and the cost of purchasing and maintain-ing the instruments can be very high. Second, the effectiveness of the lesson can be influenced by the weather, location and time of day. Third, because many operations involve fine actions, the instructors often face the difficulty of clearly demonstrating each step to every student in the field (Dib et al., 2010). Fourth, students who rely on distance learning or are enrolled in programs at smaller/satellite campuses (these smaller campuses usually cannot afford to buy the surveying equipment) are currently deprived of opportunities to practice with the instruments and have to make several trips to the main campus during the semester in order to learn best surveying practices or do without.

Computer graphics technologies can help overcome many of these challenges and several researchers argue that surveying education can be significantly improved with the use of interactive visualizations. Kuo et al. (2007) have recently developed a realistic virtual survey instrument (SimuSurvey) and studied the feasibility of introducing SimuSurvey in regular surveyor training courses. Results of

the study indicated improved student ability to use the instruments and positive attitude toward including SimuSurvey in regular surveyor training courses. At Leeds Metropolitan Uni-versity, UK, Ellis et al. (2006) have developed an undergraduate VR surveying application. The interactive software includes 360-degree panoramic realistic visualizations of sites and makes use of QuickTime VR technol-ogy. The application was evaluated with 192 undergraduate students; findings suggest that the interactive tool complements traditional learning approaches, maintains student interest, and reinforces understanding. At University of New Castle, UK, Mills and Barber (2008) have implemented a realistic virtual surveying field course that includes both a virtual fieldtrip and a virtual interactive traverse learning tool (VITLT). The goal of the tool is to improve understanding of surveying methods for first year students in the Geomatics degree. The ap-plication was evaluated by several Geomatics students; all subjects highlighted the potential of VITLT to help the learning and understand-ing of a traverse. However, the students did not see the e-learning tool as a replacement for a traverse observation as carried out on the field-course, but suggested that it could be used as a preparation and revision tool. At Purdue University, Dib and Adamo-Villani (2013) have developed a realistic interactive virtual environment whose objective is to help undergraduate students learn and review the concepts and practices of differential leveling. Results of a summative study with 48 under-graduate students showed that using the realistic VLE led to an increase in subjects’ declarative knowledge by 28% and procedural knowledge by 30%. Compared to traditional practice in the field, using the VLE led to significantly higher declarative knowledge gains, whereas differ-ences in procedural knowledge gains (e.g. the ability to perform the leveling exercise in the field) were not significant.

In the context of undergraduate surveying education, all visualizations described in the literature are realistic, and the studies report experiments that aimed to evaluate the overall

Page 6: Realistic versus Schematic Interactive Visualizations for ...hpcg.purdue.edu/idealab/pubs/IJICTE14.pdfthat are processed in the visual sensory system” (Scheiter et al., 2008, p.

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

International Journal of Information and Communication Technology Education, 10(2), 62-74, April-June 2014 67

educational efficacy or perceived usefulness of these realistic visualizations. To our knowl-edge, no study has attempted to identify which specific aspects of the visualizations are most beneficial in surveying education. In particular, no study has attempted to investigate whether the level of realism of the visualization makes a difference in students’ learning of surveying practices. The experiment reported in the paper aims to fill this knowledge gap.

DESCRIPTION OF THE STUDY

The study aimed to answer the following re-search questions:

1. Does the type of visualization (schematic versus realistic) have an effect on under-graduate students’ learning of surveying practices (specifically, ‘chaining’)?

2. Does the type of visualization (schematic versus realistic) have an effect on students’ perceived helpfulness and willingness to use?

Learning is a multidimensional construct that includes cognitive, metacognitive, and motivational components (Mayer, 1998). Our experimental study focused on the cogni-tive aspects, and specifically on procedural knowledge, e.g. the ability to apply acquired knowledge to perform a given task (know how) (Gagné, 1985). In particular, the study aimed to assess whether the degree of realism of the vi-sualization had an effect on the students’ ability to correctly perform a chaining exercise in the field. We measured this learning goal using pre and post educational intervention competency testing. The testing instrument consisted of a lab practical to be performed in the field. The pre-test and post-test were identical.

We note that a prior study by the authors (Dib & Adamo-Villani, 2013) demonstrated the efficacy of interactive visualizations for learn-ing surveying practices. It also showed that the

visualizations were as effective as practice in the field. Thus, in this study we did not compare the visualizations with traditional field practice.

The study also aimed to assess whether the degree of realism of the visualization had an effect on students’ perception of the benefit of using the visualization, and on their willingness to use the visualization again. We measured perceived helpfulness and willingness to use with an online questionnaire that included five 7-point Likert-scale rating questions.

Subjects

Sixty-two (62) undergraduate students enrolled in Building Construction Management at Purdue University.

Stimuli

The stimuli of the experiment included two interactive visualizations that illustrate the surveying practice of chaining. Chaining (or taping) is the measurement of the horizontal distance between two points. It requires the use of several instruments (e.g. steel tape, chaining pins, plumb bobs, hand level, tension handles, clamp handles, and lining rods) and is usually performed in six steps: lining in, applying ten-sion, plumbing, marking tape lengths, reading tape, and recording the distance. The visualiza-tions guide the students through these six steps and help them understand how to measure the distance between two given points on a horizontal plane, steep slope and rough terrain using the proper techniques and instruments. The two interactive visualizations have been programmed to allow for 1/16th of an inch variation, that is, if the student sets up perfectly at the point of interest two times in a row, the plumb bob is within 1/16th of an inch from the previous location (this replicates real life set-tings where the plumb bob will be swinging and will always be at a very small distance from the point). If all the criteria are followed correctly, two consecutive measurements will vary within

Page 7: Realistic versus Schematic Interactive Visualizations for ...hpcg.purdue.edu/idealab/pubs/IJICTE14.pdfthat are processed in the visual sensory system” (Scheiter et al., 2008, p.

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

68 International Journal of Information and Communication Technology Education, 10(2), 62-74, April-June 2014

a 1/8th of an inch. Hence, in the visualizations, precision is reached if the same measurement or measurements within 1/8th of an inch or 1/100th of a foot are achieved multiple times. Accuracy is achieved by repeating multiple measurements, and therefore compensating for the random 1/8th of inch variation generated by the visualization.

The two visualizations have identical functional and content dimensions and differ only in one structural feature: amount of real-istic detail. Figure 1 shows screenshots of the schematic visualization; Figure 2 shows the same screenshots extracted from the realistic visualization.

PROCEDURE

All students received a 30-minute lecture and a 30-minute lab demonstration on chaining. During lecture, students were introduced to basic chaining concepts, methods and defini-tions of terminology. In the lab, students were introduced to chaining equipment, were given a basic demonstration on how to operate the instruments, and were shown how to read the measurements accurately and record the data correctly. After the lecture and lab demo, the students were administered a pre-test. The pre-test consisted of a lab practical, e.g., a chaining exercise to be performed in the field. Students

Figure 1. Screenshots extracted from the schematic visualization. Point selection (frame 1); plumb bob, hand level and tension meter setup (frame 2); reading tape (frame 3).

Page 8: Realistic versus Schematic Interactive Visualizations for ...hpcg.purdue.edu/idealab/pubs/IJICTE14.pdfthat are processed in the visual sensory system” (Scheiter et al., 2008, p.

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

International Journal of Information and Communication Technology Education, 10(2), 62-74, April-June 2014 69

were assigned two points on the terrain outside the building and were asked to measure the horizontal distance between them. Several instructors observed the students performing the exercise and graded the exercise based on the following criteria: (1) selection of needed tools; (2) setup and operation of instruments; (3) precision of the measurements (adherence to correct procedures); (4) accuracy of the mea-surements (adherence to correct procedures); (5) recording of the measurements; and (6) calculation of random error.

After the pre-test, a randomized complete block design was used to divide the subjects into two groups with similar pre-knowledge. In other words, we used the pre-test scores to group individuals in terms of pre-knowledge

and then made sure these groups of individuals were equally assigned to the two intervention groups, e.g. group A (realistic visualization), and group B (schematic visualization). Two weeks after the pre-test, group A worked in one of the labs using the realistic visualization; group B worked in a different lab using the schematic visualization. An instructor was present in each lab to address possible technical problems with the software. Participants interacted with the visualizations for a maximum of 45 minutes and then were asked to complete the online questionnaire. Both groups were tested for chaining competency after the two educational interventions; the post-test was the same as the pre-test. The differences in scores between the pre and post-test for group A and group B, as

Figure 2. Screenshots extracted from the realistic visualization. Point selection (frame 1); plumb bob setup (frame 2); tension meter setup (frame 3); hand level setup (frame 4); reading tape (frame 5).

Page 9: Realistic versus Schematic Interactive Visualizations for ...hpcg.purdue.edu/idealab/pubs/IJICTE14.pdfthat are processed in the visual sensory system” (Scheiter et al., 2008, p.

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

70 International Journal of Information and Communication Technology Education, 10(2), 62-74, April-June 2014

well as the answers to the online survey, were then analyzed and compared.

A video demonstration of the schematic visualization can be accessed at: http://hpcg.purdue.edu/idealab/VELS/ . A demo version of the realistic visualization can be accessed at: https://engineering.purdue.edu/IVM/Web%20Player%20Ver.4/Web%20Player%20Ver.html.

FINDINGS

1. Does the type of visualization (schematic versus realistic) have an effect on under-graduate students’ learning of surveying practices (specifically, ‘chaining’)?

Results of the pre and post-test show that subjects in group A increased their procedural knowledge of chaining by 40% after using the realistic visualization; subjects in group B increased their procedural knowledge by 37% after interacting with the schematic visu-alization. Hence, the difference in procedural knowledge gain between group A and group B was 3%, with group A being 3% higher. Table 1 reports Mean and Standard Deviation calculated for the pre and post-test scores for each group.

A 2-sample t-test was performed to deter-mine if the difference in procedural learning gains between group A and group B was sta-tistically significant. Results of the statistical analysis show that the difference in procedural knowledge gains between the two groups is not

statistically significant (P-value= 0.32; MEAN (group A)= 23.73; STD (group A) =6.24; MEAN (group B) = 20.96; STD (group B) = 5.47):

2. Does the type of visualization (schematic versus realistic) have an effect on students’ perceived helpfulness and willingness to use?

Answers to the rating questions showed differences between group A and group B (see Table 2).

Overall, the realistic visualization received higher ratings than the schematic one. T-tests were performed for each question to determine if the difference in ratings between group A and group B was statistically significant. For ques-tions 1, 3, 4 and 5 the difference in ratings is not statistically significant (p-value=0.12 (Q1); p-value= 0.93 (Q3); p-value= 0.87 (Q4); p-value =0.94 (Q5)). For question 2 (The visualization helps me understand how to set up and oper-ate the instruments properly) the difference is statistically significant (p-value=0.034).

DISCUSSION AND FUTURE WORK

The results of this study indicate that:

1. Students who used the schematic visual-ization were able to perform the chaining exercise with the same level of correctness,

Table 1. Summary of descriptive data analysis: Mean and standard deviation calculated for the pre and post-test scores

N Pre-Test Post-Test

Group A (Realistic Visualization)

Mean 31 58.20 81.93

STD 31 6.52 7.55

Group B (Schematic Visualization)

Mean 31 57.56 78.52

STD 31 7.02 8.35

nicolettaadamo-villani
Highlight
nicolettaadamo-villani
Highlight
This http address is wrong. it should be http://engineering.purdue.edu/IVM/VELS/VELS.html
Page 10: Realistic versus Schematic Interactive Visualizations for ...hpcg.purdue.edu/idealab/pubs/IJICTE14.pdfthat are processed in the visual sensory system” (Scheiter et al., 2008, p.

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

International Journal of Information and Communication Technology Education, 10(2), 62-74, April-June 2014 71

accuracy and precision as those who were exposed and interacted with the realistic one. Although there is a small difference in procedural knowledge gains between group A and group B, this difference is not statistically significant;

2. Students’ subjective ratings of the per-ceived helpfulness of the visualizations and willingness to use were consistently high for both visualizations (>6). The only significant difference in ratings was noted for Q2-- students perceived the realistic visualization as being more effective for learning how to set up and operate the required instruments. This difference is in line with prior research suggesting that realistic visualizations might especially be helpful for learning, if the learning ob-jective requires interacting with concrete objects in real-world settings (Betrancourt & Tversky, 2000).

The results of the comparative study on learning efficacy contradict the findings of prior studies that suggested that a higher degree of realism might be especially beneficial when procedural knowledge is to be acquired (Höf-

fler & Leutner, 2007; Arnold & Dwyer, 1975). In the case of learning the surveying practice of chaining, the degree of realism did not have an effect on students’ performance. Hence, the intuition that perceiving and acting in a virtual environment that is designed to simulate reality as closely as possible will provide a better learn-ing experience through facilitating the transfer of the experience to the physical world proved to be incorrect. However, we believe that the results of the study might have been affected by two factors: (a) the level of complexity of the learning objective, and (b) the educational context.

Chaining is a fairly simple exercise that does not require the use of highly sophisticated instruments or complex procedures. Hence, a schematic representation of the instruments and procedures that does not show a high level of fidelity with the world referents might be suffi-cient for transferring the acquired knowledge to the real world. In future work we will extend the comparative studies to more complex surveying practices and will explore whether there is an interaction between the level of complexity of the learning objective and the degree of realism of the visualization.

Table 2. Reports the mean rating for each question and for each group

Rating Question (1 = Strongly Disagree; 7 = Strongly Agree)

MEAN Rating (Group A) (N=31)

MEAN Rating (Group B) (N=31)

Q1. The visualization helps me understand how to measure the horizontal distance between two points using the proper techniques and instruments

6.82 6.14

Q2. The visualization helps me understand how to set up and operate the instruments properly 6.63 5.12

Q3. The visualization helps me understand how to measure the horizontal distance versus the slope distance

6.37 6.28

Q4. The visualization helps me understand how to choose the intermediate points so that the measurement is completed successfully with minimum number of instruments set up

6.31 6.12

Q5. If I were to take the surveying course again, I would use this visualization as a learning tool for chaining

6.86 6.78

Page 11: Realistic versus Schematic Interactive Visualizations for ...hpcg.purdue.edu/idealab/pubs/IJICTE14.pdfthat are processed in the visual sensory system” (Scheiter et al., 2008, p.

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

72 International Journal of Information and Communication Technology Education, 10(2), 62-74, April-June 2014

In our study, the visualizations were used as supplementary materials to traditional lecture and laboratory demonstration during which the students were shown the physical instruments and were given a demonstration of the proce-dures. In future work we will explore whether there is a correlation between the educational setting (e.g. visualization as supplementary learning material versus visualization as the only learning tool) and the degree of realism of the visualization. In other words, it could be possible that a highly realistic visualization illustrating a surveying practice is superior when the visualization is the only learning material available to the students, whereas a schematic visualization might prove as or more effective when the visualization is used as a supplemen-tary instructional tool.

The relatively small sample size was an-other limitation of the study. In order to build stronger evidence, additional studies with larger pools of participants and in different settings will need to be conducted in the future. We also acknowledge that different people learn in different fashions and that their education should be tailored to the way they learn the best. For instance, it could it be possible that some people learn best with schematics whereas oth-ers learn best with more realistic visualizations. Learning efficacy may also vary with levels of the learners’ motivation and learning abilities, e.g., when learner maturity or motivation level is low, more visual detail may be helpful. We plan to conduct further studies with different populations in order to investigate the effect of the degree of realism of the visualization on different types of learners. Lastly, the quality of the graphics could have affected the results of the study. Hence, another future work may be to measure the effects of the quality of the graphical visualizations (based on learners’ rating or more objective criteria) on students’ learning of surveying.

In summary, despite the aforementioned limitations, the work reported in the paper al-lows us to make the following generalizations:

(1) students perceive visualizations as useful supplementary instructional tools for learning surveying; and (2) when the learning objective is to acquire procedural knowledge of low-complexity surveying practices, the degree of realism of the visualization does not have an effect on students’ performance.

ACKNOWLEDGMENT

This research is supported by the National Sci-ence Foundation (NSF) – TUES program, award # 1140514 (June 2012 - Dec. 2014).

REFERENCES

American Congress on Surveying and Mapping—ACSM. (n.d.). Retrieved from http://www.acsm.net/

Arnold, T., & Dwyer, F. M. (1975). Realism in visu-alized instruction. Perceptual and Motor Skills, 40, 369–370. doi:10.2466/pms.1975.40.2.369

Betrancourt, M., & Tversky, B. (2000). Effect of computer animation on users’ performance: A review. Le Travail Humain, 63, 311–329.

Cover, A. S., Ezquerra, N. F., O’Brien, J. F., Rowe, R., Gadacz, T., & Palm, E. (1993). Interactively deformable models for surgery simulation. IEEE Computer Graphics and Applications, 13, 68–85. doi:10.1109/38.252559

Dib, H., Adamo-Villani, N., & Garver, S. (2013). An interactive virtual environment for learning dif-ferential leveling: Development and initial findings. Advances in Engineering Education.

Dib, H., Adamo-Villani, N., Keith, D., & Crawford, W. (2010). Virtual reality for learning construction surveying. In Proc. of ICCCEB: International Confer-ence, Nottingham, UK. Retrieved from http://www.engineering.nottingham.ac.uk/icccbe/proceedings/pdf/pf293.pdf

Dwyer, F. M. (1968). Effect of varying amount of realistic detail in visual illustrations designed to complement programmed instruction. Percep-tual and Motor Skills, 27, 351–354. doi:10.2466/pms.1968.27.2.351

Page 12: Realistic versus Schematic Interactive Visualizations for ...hpcg.purdue.edu/idealab/pubs/IJICTE14.pdfthat are processed in the visual sensory system” (Scheiter et al., 2008, p.

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

International Journal of Information and Communication Technology Education, 10(2), 62-74, April-June 2014 73

Dwyer, F. M. (1976). Adapting media attributes for effective learning. Educational Technology, 16, 7–13.

Ellis, R. C. T., Dickinson, I., Green, M., & Smith, M. (2006) The implementation and evaluation of an undergraduate virtual reality surveying applica-tion. In Proc. of BEECON 2006 Built Environment Education Conference, London, UK.

Gagné, R. M. (1985). The conditions of learning and theory of instruction. New York, NY: Holt, Rinehart and Winston.

Gianutsos, R. (1994). Driving advisement with the elemental driving simulator (EDS): When less suf-fices. Behavior Research Methods, Instruments, & Computers, 26, 183–186. doi:10.3758/BF03204615

Goldstone, R. L., & Sakamoto, Y. (2003). The transfer of abstract principles governing complex adaptive systems. Cognitive Psychology, 46, 414–466. doi:10.1016/S0010-0285(02)00519-4 PMID:12809681

Goldstone, R. L., & Son, J. Y. (2005). The transfer of scientific principles using concrete and idealized simulations. Journal of the Learning Sciences, 14, 69–110. doi:10.1207/s15327809jls1401_4

Gonzales, C. (1996). Does animation in user inter-faces improve decision making? In R. Bilger, S. Guest, & M. J. Tauber (Eds.), Proc. of the CHI 96 Conference on Human Factors in Computing Systems (pp. 27-34). New York, NY: ACM Press.

Hoffler, T. N., & Leutner, D. (2007). Instructional ani-mation versus static pictures: A meta-analysis. Learn-ing and Instruction, 17, 722–738. doi:10.1016/j.learninstruc.2007.09.013

Imhof, B., Jarodzka, H., & Gerjets, P. (2009). Clas-sifying instructional visualizations: A psychological approach. In Proceedings of the IMAGE 10. Retrieved from http://www.gib.uni-tuebingen.de/image/ausgaben-3?function=fnArticle&showArticle=158

Kuo, H. L., Kang, S. C., Lu, C. C., Hsieh, S. H., & Lin, Y. H. (2007). Feasibility study: Using a virtual surveying instrument in surveying training. In Proc. of ICEE 2007, Coimbra Portugal.

Lohse, G. J., Biolsi, K., Walker, N., & Rueter, H. H. (1994). A classification of visual representa-tions. Communications of the ACM, 37, 36–49. doi:10.1145/198366.198376

MacDonald-Ross, M. (1977). How numbers are shown: A review of research on the presentation of quantitative data in texts. AV Communication, 25, 359–404.

Mayer, R. E. (1998). Cognitive, metacognitive, and motivational aspects of problem solving. Instructional Science, 26, 49–63. doi:10.1023/A:1003088013286

Mills, H., & Barber, D. (2008). A virtual surveying field-course for traversing. E-learning in surveying, geo-information sciences and land administration. In Proc. of FIG International Workshop 2008, Enschede, The Netherlands.

Rankin, R. (1990). A taxonomy of graph types. In-formation Design Journal, 6, 147–159. doi:10.1075/idj.6.2.03ran

Rieber, L. P., Tzeng, S. C., & Tribble, K. (2004). Discovery learning, representation, and explanation within a computer-based simulation: Finding the right mix. Learning and Instruction, 14(3), 307–323. doi:10.1016/j.learninstruc.2004.06.008

Scheiter, K., Gerjets, P., Huk, T., Imhof, B., & Kammerer, Y. (2009). The effects of realism in learning with dynamic visualizations. Learning and Instruction, 19, 481–494. doi:10.1016/j.learn-instruc.2008.08.001

Scheiter, K., Wiebe, E., & Holsanov, J. (2008). Learning with visualizations: Theoretical and meth-odological aspects. In R. Zheng (Ed.), Cognitive ef-fects of multimedia learning (pp. 67–88). IGI Global. doi:10.4018/978-1-60566-158-2.ch005

Schwartz, D. L. (1995). Reasoning about the refer-ent of a picture versus reasoning about the picture as a referent: An effect of visual realism. Memory & Cognition, 23, 709–722. doi:10.3758/BF03200924 PMID:8538444

Schwartz, D. L., & Black, J. B. (1996). Analog im-agery in mental model reasoning: Depictive models. Cognitive Psychology, 30, 154–219. doi:10.1006/cogp.1996.0006 PMID:8660784

Schwartz, D. L., & Moore, J. L. (1998). The role of mathematics in explaining the material world: Mental models for proportional reasoning. Cognitive Science, 22, 471–516. doi:10.1207/s15516709cog2204_3

Tufte, E. R. (1983). The visual display of quantitative information. Cheshire, CT: Graphics.

Van Gendt, K., & Verhagen, P. (2001). Visual testing. Searching for guidelines. In Proc. of the 24th National Convention of the Association for Educational Com-munications and Technology, Atlanta, GA.

Page 13: Realistic versus Schematic Interactive Visualizations for ...hpcg.purdue.edu/idealab/pubs/IJICTE14.pdfthat are processed in the visual sensory system” (Scheiter et al., 2008, p.

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

74 International Journal of Information and Communication Technology Education, 10(2), 62-74, April-June 2014

Hazar Dib, MSCE, Ph.D. received his BSc and MSc in Civil Engineering in 2002 from the University of Balamand (Lebanon) and his PhD in Building Construction Management from the University of Florida (USA) in 2007 with emphasis on Information Sharing and Informa-tion Management. He is currently an Assistant Professor of Building Information Management in the College of Technology at Purdue University. His research areas include the integration of 3D technologies in information modeling techniques, development of knowledge acquisition methods and strategies for examining and capturing expert design knowledge, sharing lessons learned and historical data to improve design, increase efficiency in the construction processes, and improve the classroom experience of teaching construction management.

Nicoletta Adamo-Villani is an Associate Professor of Computer Graphics Technology in the College of Technology at Purdue University and a University Faculty Scholar. She has an MS in Architecture from University of Florence, Italy and is a certified animator and instructor for Autodesk. Prof. Adamo-Villani’s area of expertise is in character animation and character design and her research interests focus on the application of 3D animation technology to education, HCC (Human Computer Communication), and visualization. Prof. Adamo-Villani is author or co-author of more than 80 scientific publications, and is an award-winning animator and graphic designer. She is co-founder and director of the IDeaLaboratory.

Stephen Garver is a first year graduate student at Purdue University, pursuing a Master of Science in Computer Graphics. He completed his undergraduate degree at Purdue University, earning a Bachelor of Science in Computer Graphics Technology, with Distinction. Currently, Stephen works as a Research Assistant at the IDEA Laboratory at Purdue University. His research interests include real-time procedural modeling, animation, and game design.

nicolettaadamo-villani
Highlight
Building Construction Management