Online Synchronous vs. Asynchronous Software Training Through

Journal of Distance Education Technologies, 4(4), ��-102, October-December 2006 ��

Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

may grow to $24 billion ($18 billion in the U.S.) by 2006 with a compound annual growth rate of 35.6% (IDC, 2002). The burgeoning online learning/training market, and the increasing training budgets of businesses and schools has provided these key users of online training and marketing tools with practical reasons, as well as compelling research motives, to investigate the effectiveness of training and education in different online formats.

Online learning differs primarily from the traditional face-to-face (F2F) learning in that it is a user-centered, rather than instructor-centered, learning mode. Other benefits of substituting online learning for F2F learning include (1) self-paced instruction; (2) the ability to incorporate text, graphics, audio and video into the training; (3) opportunity for high levels of interactivity; (4) a written record of discussions and instruc-tions; (5) low-cost operation; and (6) access to a worldwide audiences (Aniebonam, 2000). In addition, online learning can remove a certain degree of space and time limitations, speed up the learning process for motivated learners, lower economic costs of attending F2F classes and have higher information accessibility and availability.

Although IT has changed the training and educational approaches and environments, the ultimate goal of learning has not changed, that is, to transfer knowledge to students and allow them to apply the acquired knowledge in real situations. In the field of IT, the suc-cess of software training can be assessed with a trainee’s IT skills of, and knowledge of the use of, particular software to solve problems. Surprisingly, after attending a training session, very few students know how to properly ap-ply the acquired knowledge and skills to real situations. This raises an important issue, that is, how to improve knowledge transfer capa-bility of learners in different online learning environments.

The importance of knowledge transfer is self-evident. However, the knowledge transfer process does not occur naturally. There is a need to assist learners in transferring their acquired knowledge into future applications. One effec-

tive approach to assisting the learning transfer process is “behavior modeling” (BM). This ap-proach teaches learners through demonstration and hands-on experience. Simon, Grover, Teng, and Whitcomb (1996) and Compeau and Hig-gins (1995) found that in the field of information technology, BM is the most effective approach compared to the other two knowledge transfer approaches: exploration — teaching through practice on relevant example, and instruction — teaching software characteristics.

Distance education is defined as “teaching through the use of telecommunications technol-ogies to transmit and receive various materials through voice, video and data” (Bielefield & Cheeseman, 1997, p. 141). In the same token, Leidner and Jarvenpaa (1995) define distance learning as “the transmission of a course from one location to another” (p. 274). These defini-tions provide an analogy to distance learning in the field of information technology or online software training. Online software training can be the transmission of instructional IT program-ming or contents to geographically dispersed individuals or groups.

There are two general modes of online learning: synchronous and asynchronous modes. Each mode can be marshaled with IT tools to deliver software training. Case in point, audio and video conferences are two types of online synchronous training mediums. Online asynchronous training mediums range from Web pages, file download, e-mail, e-mail list, news-group, forum, chat, response pad, whiteboard and to screen sharing. Built on his personal distance training and education experiences since 197 -+ 1, Horton (2000) suggests that online synchronous and asynchronous learning and training be designed for different purposes. Incorporating synchronous learning demands the control of schedule, time, people, class size, video and audio equipment and place. These factors constrain the possibility of reaching large numbers of students at any given time and in any given place.

However, the BM approach for trainees can be a problem in online asynchronous and synchronous training. For instance, any dem-

�0 Journal of Distance Education Technologies, 4(4), ��-102, October-December 2006


onstration presented by a live instructor would need to be replaced with a scripted or videotaped demonstration in asynchronous mode, and live transmission or Webcam in synchronous mode. This raises several important questions. Can the scripted, videotaped, live transmission or Webcam approach still be as effective as the traditional classroom? How receptive are stu-dents to different online learning environments with differential degrees of student-centered interaction compared to an instructor-centered F2F environment? Most importantly, it is an unknown but interesting question to ask whether knowledge can be effectively transferred in different online environments. This research is to address these important issues faced by any instructor who intends to apply the BM approach in either online synchronous and asynchronous environments.

BEHAVIORAL MODELING AND KNOWLEDGE TRANSFER

Social learning theory is the basis of the behavior modeling approach. Therefore, it is important to assess the applicability of the theory and approach in the online learning en-vironment. Learning outcomes can be measured by different types of knowledge transfer and end-user satisfaction.

Behavior Modeling in Online Environments

Bandura (1977) proposed the Social Learning Theory to explain the interactive learning process between individuals and their social environment. He asserted a series of social learning needs take place to direct an individual from biological and self-centered response to social and group behaviors. Since the social learning process takes place within a society, individuals learn to establish their behavior models by observing and imitating other individuals’ behaviors or through the enforcement of the media and environment. Online learning in different environments needs to be delivered via different media. Different online learning environments, therefore, may

have different degrees of enforcement to learn-ers’ individual behaviors.

Learning by modeling or observing people’s behaviors may be more effective than learning by trial-and-error because the former approach can avoid unnecessary mistakes and harm. Modeling an instructor’s behaviors em-powers students to (1) learn new behavior from the instructor, (2) self-evaluate their behaviors against the instructor’s and (3) enforce students’ current behavior.

Learning by modeling takes place in four sequential steps: (1) attention, (2) retention, (3) motor reproduction and (4) motivation and reinforcement (Bandura, 1977). Enforce-ment forces, such as the duration of training, praise, motivation and attention of others, al-lows learning to move along these four steps against counter forces. Enforcement forces, such as retention enhancement and practice, can contribute to better cognitive learning (Yi & Davis, 2001).

Lewin (1951) argued that the effectiveness of Behavior Modeling is a function of people interacting within an environment. The BM approach is different from learning by adap-tation. The former approach teaches through demonstration, while the latter approach influ-ences the behaviors of learners by reward and punishment (Skinner, 1938). The BM approach was first applied in the training of interpersonal communication and management skills (Decker & Nathan, 1985). Gist, Schwoerer and Rosen (1989) further applied the training to the context of information technology.

BM may be readily employed in face-to-face instruction, but cannot be easily simulated in online asynchronous instruction, which lacks the interactive immediacy necessary for opti-mally effective instructor demonstration and correction. The richness of information media in online synchronous instruction is another constraint and may also have less enforcement force than F2F instruction to the learning out-comes. For example, in a live training class, the instructor is able to demonstrate a software process and immediately ask the students to repeat the activity under the instructor’s close

Journal of Distance Education Technologies, 4(4), ��-102, October-December 2006 �1


supervision. However, in an online asynchro-nous situation where there is no live instructor, the demonstration loses the benefit of that im-mediate feedback. In the same token, in an online synchronous situation bandwidth constraints and compromised reciprocity may undermine the enforcement force of the demonstration. In both online environments, enforcement forces can be further compromised with the missing of “learning by doing,” another key element of F2F BM training (McGehee & Tullar, 1978).

Therefore, there is a strong possibility that the BM approach cannot be fully replicated in either the online synchronous or asynchronous situation and will not be as effective a method in online training as in the traditional environ-ment.

Knowledge Transfer Knowledge transfer is the application

of acquired skills and knowledge into differ-ent situations. Unless the transferring process occurs, learning has little value. The applied situations could be similar or novel to the learning situation. Depending on the situation, knowledge transfer can take place in different formats. In general, there are four different types of knowledge transfer.

Positive Transfer vs. Negative Transfer Positive transfer of learning means that

learning in one situation stimulates and helps learning in another situation. Negative transfer of learning hinders the application of learning in one situation to other situations. Positive learn-ing experience can be enhanced via analogy, informed instruction (Paris, Cross & Lipson, 1984), tutorial (Morris, Shaw & Perney, 1990) and so forth. Learning effectiveness can be improved by triggering positive learning and mitigating negative learning experience.

Near Transfer vs. Far Transfer Salomon and Perkins (1988) argued that

transfer of learning could have a differential degree of transfer. The effectiveness of near-transfer learning depends on the learner’s ability to solve problems similar to those encountered

in the learning context. For instance, learning how to add two digit numbers allows learners to add three digit numbers. Near-transfer learn-ing occurs in two similar situations and at a lower level. Therefore, the level of learning is more easily acquired and applied. In contrast, applying the acquired skills and knowledge in two dissimilar and sometimes novel situations is much harder to achieve. For instance, a table tennis player can apply skills of playing pinball to playing tennis. Although both sports look similar on the surface, the techniques to control pinballs and tennis balls are very different. The learning transfer is much harder to be acquired and retained. Therefore, the transfer is defined as far-transfer learning. Near-transfer and far-transfer of knowledge seem to be the most widely used measures of learning outcomes in the field of information technology since learners must utilize the knowledge learned in a computing environment.

Specific Transfer and General TransferDepending on learning content, there are

two different learning transfers: specific transfer and general transfer (Bruner, 1996). The former refers to the extension and association of habit and skills. The latter refers to the transfer of prin-ciples and attitudes that can be used to deepen the understanding of basic concepts.

Lateral Transfer and Vertical TransferGagne (1992) asserted that the transfer of

learning includes lateral and vertical transfers. Lateral learning is to apply one domain of knowledge to another domain. Lateral learning does not follow step-by-step instruction and is considered as provocative learning. Vertical learning means that a higher level of learning needs to be created by integrating acquired skills, and experiences with new situations. Vertical transfer of learning is analytical and sequential.

Other Knowledge Transfer TheoriesTheories related to knowledge transfer

are not limited to the above mentioned ones. For instance, the theory of identical elements



asserts that the more identical elements dif-ferent learning contains, the more efficient the transfer of learning (Thorndike, 1949). Baldwin and Ford (1988) proposed a general training theory to classify three categories of factors affecting transfer of training: (1) training inputs, (2) training outputs and (3) conditions of transfer. The situated learning theory argues that individuals are affected by learning environ-ment when trying to solve practical problems. Therefore, the interaction between learners and the environment is an important factor that needs to be taken into account when measuring the transfer of learning. Finally, the theory of formal discipline argues that knowledge transfer skills can be acquired by training learner’s sensual-ity, such as thinking, judgment, classification, imagination, creation and so forth.

The objective of this study was to inves-tigate the impacts of the learning environment in online and offline formats on the transfer of learning. The situational changes rationalize the adoption of situated learning theory. To accomplish this objective, we sought to train end-user to learn how to use Microsoft SQL server 2000 software. Therefore, we adopted the near-transfer and far-transfer measures of learning outcomes for our information technol-ogy related experiment.

HYPOTHESESHypotheses are formulated to investigate

whether the BM approach is as effective in online synchronous and asynchronous environments as in the traditional face-to-face environment. We measured learning outcomes by trainees’ perfor-mances in near-transfer and far-transfer tasks, as well as overall satisfaction levels. The study also considered the importance of time variant. Hence, training and performance measurement were conducted over five weeks.

Knowledge Near-Transfer (KNT) TasksH1: End-users trained using F2F behavior

modeling perform near-transfer informa-tion system tasks better than those trained in asynchronous behavior modeling.

H2: End-users trained in F2F behavior mod-eling perform near-transfer information system tasks better than those trained in synchronous behavior modeling.

H3: End-users trained in synchronous be-havior modeling perform near-transfer information system tasks better than those trained in asynchronous behavior modeling.

Knowledge Far-Transfer (KFT) TasksH4: End-users trained in F2F behavior mod-

eling perform far-transfer information system tasks better than those trained in asynchronous behavior modeling.

H5: End-users trained in F2F behavior mod-eling perform far-transfer information system tasks better than those trained in synchronous behavior modeling.

H6: End-users trained in synchronous be-havior modeling perform far-transfer information system tasks better than those trained in asynchronous behavior modeling.

Overall SatisfactionH7: End-users trained in synchronous be-

havior modeling have a higher overall satisfaction level than those trained in asynchronous behavior modeling.

RESEARCH DESIGNThis study applied Simon, Grover, Teng

and Whitcomb’s (1996) well-constructed software training theory to experimentally test behavior modeling training in three learning environments — F2F, online asynchronous and online synchronous environments. In doing so, it should be possible to detect the effects of the single independent variable (training environ-ment) on training outcomes. The experiment was conducted in a field setting that enabled the study to garner greater external validity than would be the case with a laboratory experiment. A field experiment methodology has the merits of “testing theory” and “obtaining answers to practical questions” (Kerlinger & Lee, 2000). The exploratory nature of the study requires



that variables (e.g., training environments and subject areas of study) under investigation be manipulated.

Subjects ControlThe setting for the field experiment was

the Tamkang University in Taiwan. The experi-ment was prompted by the need of 96 college sophomores, who are Management Information Systems (MIS) majors, to learn a Microsoft SQL Server 2000 software program in a database processing course. The schedule agreed on with the faculty at Tamkang University was to run the experiment for an hour training each week for four weeks. The author’s graduate assistant Ms. Lin helped administer the experiment to collect the data. The subject pool had a mean age of 22 years. Subjects who participated in the structured experiment had little database-related experience. Their intellectual levels are relatively the same because subjects scored the same range of scores in a national entrance exam. The national entrance exam system has been adopted for more than 40 years in Taiwan and is considered a relatively reliable test. Subjects’ individual backgrounds should not have influ-ence on learning outcomes.

For the purposes of this study, subjects were chosen if they lacked a theoretical and procedural understanding of the particular subject area being tested. Participants were given a pretraining questionnaire that includes important study units on Microsoft SQL Server 2000. Two experts of the domain administered the Delphi study to finalize the study units and questionnaires. This is to improve the content validity. The subjects voluntarily answered whether they knew those study units and answered their database-related experiences. Based on their answers, a correlation test of database and usage experience of the target system showed no significant differences among three experimental groups. Subjects of the study may be considered representative of novice end-users. Many studies (Ahrens & Sankar 1993; Santhanam & Sein, 1994) sup-port using students as experimental subjects to represent the general populations. Hence, all

subjects’ questionnaires were used for further data analysis. This segmentation was used to mitigate the effects of computer literacy and experience on the findings, thereby improving the internal validity of the study.

Training Treatments Face-to-face BM (FBM) is instructor-

centered training while online Asynchronous BM (ABM) and Synchronous BM (SBM) are learner-centered training. Course materials used in online learning environments were created to properly reflect the key elements of a behavior modeling approach. AniCam simulation soft-ware was used to record the demonstration of instruction. Hyperlink structure was used to help users assimilate nonlateral conceptual, and procedural knowledge.

Feedback activities of behavior modeling approach in online asynchronous environment are supported with e-mail and hyperlinks. SBM differs from ABM in providing feedback func-tions via real-time discussion forums. Training materials integrate key elements of behavior modeling approach: (1) control of three dif-ferent learning environments, (2) demonstra-tion of the instructor, (3) continuous feedback (verbal feedback in F2F and online synchronous environments; e-mail feedback in the online asynchronous environment). Three training environments were designed to maximize the effect of size on their differences (Figure 1).

Training ProceduresThe experimental study lasted for four

weeks. There was a 50 minute training session each week for each class. Figure 2 shows the experimental procedures used at each time period. The X’s, Y’s and Z’s represent online asynchronous BM training, online synchronous BM training and F2F BM training methods, respectively. The subscripts next to each al-phabet indicate the ith observation or training session, respectively. Before executing experi-mental treatments (the pretest period O1), the instructor asked the subjects to complete a short questionnaire soliciting demographic informa-tion, database software-related experience and



attitudes towards learning in the subject’s as-signed online learning environment (Pretest). Approximately one-third of the subjects pooled received the same experimental treatment for four straight weeks (Week1 to Week4). The as-signing process was random on the class basis. Randomizing the execution of O4 and O5 in Week2 and Week3 for Group A and Group B can help avoid possible confounding results from the interactive effects of the pretest of O1 and O3. This randomization process can further ensure that difference in learning outcomes of O6 is not possibly due to the sensitization of the participants after the pretest and the interaction of their sensitization, O4 and O5 (Kerlinger & Lee, 2000).

Before or after each training session, sub-jects were asked to complete database design tasks using the MS SQL commands to assess their prior knowledge in the trained subjects

and immediate learning outcomes that involve both near-transfer and far-transfer knowledge. On week five, students were evaluated again for their attitude changes towards the e-learning ses-sions and performance in near and far-transfer tasks (Post-test). The final exam concludes the five-week training sessions.

Training materials were designed to integrate key elements of the three training environments, as illustrated in Figure 3. Course materials used in the online asynchronous train-ing session were stored on the school’s server for students to learn at their own pace after each training session was completed. At the end of the experiment, students were asked about their affect for their learning environments.

Outcomes MeasurementRegardless of the teaching environment,

computer training is intended to instill in users a

Figure 1. Differences of behavior modeling approach in three learning modes

Online Learning Environments Off-line Learning EnvironmentAsynchronous BM (ABM) Synchronous BM (SBM) Face-to-Face BM (FBM)• Scripted demonstration

of step-by-step instructions

• Deductive/inductive complementary learning

• Trainees choose one of two relevant examples to practice

• Without online reference sources

• Trainee control

• Webcam-delivered demonstration of step-by-step instructions


• Instructor chooses examples that are relevant to trainees’ majors


• Trainer/trainee partially control

• Demonstration of a live instructor to learn step-by-step


• Live instructor chooses examples that are relevant to trainees’ majors


• Trainer control

Figure 2. Experimental proceduresGROUP Pretest Week1 Week2 Week3 Week4 Post-testGroup A O1 O2 X1 O3 X2 O4 X3 O5 X4 O6Group B O1 O2 Y1 O3 O4 Y2 O5 Y3 Y4 O6Group C O1 O2 Z1 O3 O4 Z2 O5 Z3 Z4 O6Oi ＝ Questionnaire and TestsXi ＝ ABM (Online Asynchronous BM Training)Yi ＝ SBM (Online Synchronous BM Training)Zi ＝ FBM (F2F BM Training)

Journal of Distance Education Technologies, 4(4), ��-102, October-December 2006 �5


Online Learning Environments Off-line Learning EnvironmentAsynchronous BM (ABM) Synchronous BM (SBM) Face-to-Face BM (FBM)• Scripted demonstration

of step-by-step instructions


• Trainees choose one of two relevant examples to practice


• Trainee control

• Webcam-delivered demonstration of step-by-step instructions


• Instructor chooses examples that are relevant to trainees’ majors


• Trainer/trainee partially control

• Demonstration of a live instructor to learn step-by-step


• Live instructor chooses examples that are relevant to trainees’ majors


• Trainer control

GROUP Pretest Week1 Week2 Week3 Week4 Post-testGroup A O1 O2 X1 O3 X2 O4 X3 O5 X4 O6Group B O1 O2 Y1 O3 O4 Y2 O5 Y3 Y4 O6Group C O1 O2 Z1 O3 O4 Z2 O5 Z3 Z4 O6Oi ＝ Questionnaire and TestsXi ＝ ABM (Online Asynchronous BM Training)Yi ＝ SBM (Online Synchronous BM Training)Zi ＝ FBM (F2F BM Training)

level of competency in using the system and to improve their satisfaction with the system. A us-er’s competency in using a system is contingent upon the user’s knowledge absorption capacity. Ramsden (1988) finds that effective teaching needs to align students with situations where they are encouraged to think deeper and more holistically. Kirkpatrick (1967) also suggests that learning effectiveness needs to be evaluated by students’ reactions, learning and knowledge transfer. The levels of knowledge absorbed by students, Bayman and Mayer (1988) suggest, may include syntactic, semantic, schematic and

strategic knowledge. Mennecke, Crossland and Killingsworth (2000) believe that experts of one particular knowledge domain possess more strategic and semantic knowledge than novices. Knowledge levels, as Simon, Grover, Teng and Whitcomb (1996) suggest, can be categorized as near-transfer, far-transfer or problem solv-ing. Near-transfer knowledge is necessary for being able to understand software commands and procedures. This type of knowledge is im-portant for a trainee to be able to use software in a step-by-step fashion. Far-transfer knowledge seeks to ensure that a trainee has the ability to

Figure 3. Delivery mechanisms of behavior modeling approaches

FBM (F2F Behavior Modeling)

ABM (Asynchronous Behavior Modeling)

SBM (Synchronous Behavior Modeling)

Course Materials

Instructor demonstrates the use of software along with PowerPoint slides

Covered three study subjects within forty five minutes each week

Course materials covered by FBM was pre-recorded and stored in a server.

No instructor was present to assist the learning process of students. Students learned at their own path and completed their study within forty five minutes.

Instructor was present, but broadcasted steaming video from a broadcast room.

Instructor conducted the real-time discussion with students on a BBS station.

Information Systems Tools

Instructor, PowerPoint, and PC

AniCam, PowerPoint and Acrobat Reader

AniCam, PowerPoint, Stream Author v.2.5 (Authoring Tool) and Acrobat Reader

Target System SQL Server2000 Personal Edition

SQL Server2000 Personal Edition

SQL Server2000 Personal Edition

Pretest Questionnaire

Learning Experience and Style Questionnaires



The First Week First Training Session First Learning Outcomes Test

First Training Session First Learning Outcomes Test

First Training Session First Learning Outcomes Test

The Second Week

Second Training Session Second Learning Outcomes Test

Second Training Session Second Training Session Second Learning Outcomes Test

The Third Week

Third Training Session Third Training Session Second Learning Outcomes Test

Third Training Session

The Fourth Week

Comprehensive Test (Third Learning Outcomes Test)



Post-test Questionnaire

Measure End-User Satisfaction





combine two or more near-transfer tasks to solve more complicated problems.

Both the use of software and information systems and the satisfaction levels of using them are useful surrogates to properly measure the effectiveness of an information system (Ives, Olson, & Baroudi, 1983). The end-user satisfaction level has been widely adopted as an important factor contributing to the success of end-user software training. Since the study was to replicate Simon, Grover, Teng and Whitcomb’s (1996) research in a dissimilar environment, near-knowledge and far-knowl-edge transfer, and end-user overall satisfaction levels were adopted in this study to measure training outcomes. Cronbach’s alpha reliability for Simon et al.’s (1996) instrument to measure satisfaction is r = 0.98. Users need to use the Likert scale from one to five to answer 12 test items related to their satisfaction with the use of online system.

DATA ANALYSISTable 1 shows the means and standard de-

viations for the scores at each treatment period. Table 2 shows F and P values of the dependent variables (near-transfer and far-transfer task performances, and overall satisfaction) across treatment groups and in different times. Pretest scores (Q1, Q2 and Q4) in varying weeks were used to tell apart students with prior experiences and knowledge on the studied topics. After learning in a weekly session, students partici-pated in a post-test. Their scores (Q3 and Q5) were used for KNT effectiveness comparison across training sessions. Scores of Q6 are KFT effectiveness and end-user satisfaction levels. A cursory examination of means (Table 1) indicates that no patterns can be identified for near-transfer performance from time Week1 to Week5. Subjects in ABM performed better than those in FBM, followed by SBM at Week1 while at Week2 and Week3 the order was changed to FBM>ABM>SBM and SBM>FBM>ABM, respectively. The findings are not in agreement with a consistent pattern as predicted by Hy-potheses H1 and H2. For KFT tasks, subjects in ABM performed better than those in SBM,

followed by FBM. This is the reversed order of a pattern as predicted by Hypotheses H3 and H4. The measurement of overall satisfaction level somewhat follows the predicted patterns of Hypotheses H5 and H6.

We took a closer look at the mean differ-ence at the significance level of 0.05. The study used one-way ANCOVA to analyze the effects of behavior modeling approach on learning outcomes over different time. Levene’s Test (1960) was used to examine the variance homo-geneity of three groups. Its F-statistics showed that KNT was 3.04 (p=0.053) at Week1, 13.01 (p=0.000) at Week2, 1.71 (p=0.187) at Week3, and 0.47 (p=0.627) at the Post-test. In contrast, the F-statistics of Levene’s Test for KFT and OS were 7.64 (p=0.001) and 1.75 (p=0.191) at the Post-test. With the exception of KNT at Week2 and KNF at the Post-test, all dependent variables met the p > 0.05 criterion for assuring homo-geneity of variances. The heteroscedasticity of variances for these two exceptions suggested that the statistical test results may not be valid. As such, the following discussion will ignore these two variances and focus on KNT and OS. For other effects that show significance, the study adopts the Scheffe post-test to analyze data. In addition, Pearson Correlation Analysis was used to assess the carry-over effects of different training sessions.

ANCOVA was performed using the general linear model approach; the results are presented in Table 3. It shows that the treat-ment effects are significant for KNT (Week2) and KNT (Week3) with F-statistics of 2.415 (p=0.095) and 2.891 (p=0.061), confirming a univariate treatment effect of learning en-vironments on the dependent variable: KNT. However, the treatment effects are not salient for other dependent variables: KFT and OS. These lacks of effect may have been due to small effect sizes.

Least-Squares Deconvolution (LSD) was used to test cross-correlations for KNT (Week2) and KNT (Week3). LSD is a cross-correlation technique for computing average profiles. LSD is very similar to most other cross-correlation techniques, though slightly more sophisticated



in the sense that it cleans the crosscorrela-tion profile from the autocorrelation profile (Donati, 2003). For KNT (Week 2), the LSD results indicate that subjects in FBM perform

better than those in ABM (p=0.051) and SBM (p=0.069). This supported the Hypotheses 1 and 2. However, Hypothesis 3 cannot be supported because the mean difference between ABM and

Table 1. Descriptive statistics - means (Standard Deviations)

ABM (N=40) SBM (N=26) FBM (N=30) Overall (N=96)KNT (Week 1) 27.63 (5.77) 25.38 (7.06) 26.00 (7.24) 26.51 (6.61)KNT (Week 2) 28.50 (3.43) 28.46 (3.09) 29.83 (0.91) 28.91 (2.83)KNT (Week 3) 56.05 (14.06) 64.27 (17.52) 62.27 (12.71) 60.22 (14.98)KNT (Week 5) 71.70 (16.21) 70.50 (19.78) 67.00 (17.27) 69.91 (17.49)KFT (Post-test) 9.63 (1.33) 9.42 (1.63) 8.83 (2.15) 9.32 (1.72)OS (Post-test) 38.10 (6.87) 39.38 (9.21) 38.61 (7.83)

Table 2. Performance on differentlLearning outcomes over five weeksF p-value Power

KNT (Week 1) 1.035 0.359 0.337KNT (Week 2) 2.415 0.095* 0.605KNT (Week 3) 2.891 0.061* 0.677KNT (Post-test) 0.634 0.532 0.246KFT (Post-test) 1.913 0.153 0.517OS (Post-test) 0.420 0.519 0.169

Table 3. Results for training methodsVariable Hypothesis Result in Correct

Direction?Significant p-value? (n.s. - not significant)

Week1 H1: FBM > ABM F n.s. (p=0.312)H2: FBM > SBM T n.s. (p=0.729)H3: SBM > ABM F n.s. (p=0.182)

Week2 H1: FBM > ABM T p=0.051H2: FBM > SBM T p = 0.069H3: SBM > ABM T n.s. (p=0.956)

Week3 H1: FBM > ABM T p=0.083H2: FBM > SBM F n.s. (p=0.612)H3: SBM > ABM T p = 0.029

Week4 H1: FBM > ABM F n.s. (p=2.71)H2: FBM > SBM F n.s. (p=0.459)H3: SBM > ABM F n.s. (p=0.787)

Post Test (KFT)

H4: FBM > ABM F p=0.057

H5: FBM > SBM F n.s. (p=0.2)H6: SBM > ABM F n.s. (p=0.639)

Post Test (OS)

H7: SBM > ABM F n.s. (p=0.579)

�� Journal of Distance Education Technologies, 4(4), ��-102, October-December 2006


SBM is not significant. For KNT (Week3), the LSD results indicate that (1) subjects in FBM performed better than those in ABM (p=0.083), and (2) subjects in SBM performed better than those in ABM (p=0.029). Hypotheses 4 and 6 are supported. Worthy to be noted is that H4 is upheld but in the reversed direction. This indi-cates that ABM is a more effective method than FBM at improving knowledge far transfer.

Four out of nine hypotheses in total are supported. Although not all hypothesized relationships are fully supported, the results obtained are interesting. The most intriguing result is that although there is statistically-justified reason for preferring FBM to ABM or SBM or software training, the pattern of results is not persistent in the long run. FBM resulted in better outcomes than ABM and SBM at Week2, and than ABM at Week3 for KNT. Although it never does so at a statistically significant level, subjects in ABM performed better than those in SBM, followed by those in FBM for KNT (Post-test) and KFT (Post-test). One interpretation of this is that either ABM or SBM training is no worse than FBM training across all dependent variables. The pattern of results for FBM suggests that trainers might choose ABM or SBM, which should to be a less costly alternative to FBM, without making any significant sacrifices in either learning or trainee reaction outcomes.

Another result of interest is that, with respect to the three online asynchronous train-ing methods, the pattern of results suggests that FBM might be the best for KNT in the short term. Of the nine hypotheses concerning relationships between these methods, four are in the expected direction, and significantly so. This indicates that use of ABM or SBM may be a better – and certainly no worse – software training strategy in the long term.

IMPLICATIONS FOR RESEARCH

This article studied the impact of training duration on performance and trainee reactions. Trainees were exposed to the same training

methods with different degrees of social pres-ence for different durations. These findings indicated that training duration and social pres-ence have little impacts on learning outcomes. Despite this, the findings here raise additional questions for research.

It may be more important to investigate the impacts of information richness (Fulk, 1993) features of online training media on training outcomes. Future studies might vary the social presence features of training media or their combination with social presence fea-tures (e.g., with instructor’s feedbacks versus discussion boards, e-mail response or playback features). Information richness may be a more influential factor affecting the performance of training approaches.

It may also be useful to replicate the experimental equivalence of FBM, ABM and SBM methods of software training with dif-ferent software and subjects. Since in the long term different treatments have similar impacts on learning outcomes, it may be practical to demonstrate the cost-based advantage of ABM over SBM, and SBM over FBM for software training in practical settings.

Another way to improve the reliability of the study is to manipulate some useful block-ing variables. A series of comparative studies can be conducted to assess the impact of indi-vidualism as a cultural characteristic, computer self-efficacy, task complexity (simple tasks vs. fuzzy tasks), professional backgrounds and the ratio of the training duration to the quantity of information to be processed, among others.

Learning style may be an important factor to consider in the online learning environment. According to social learning theory, learners interact with the learning environment to change their behavior. Learning style is situational and can vary with different learning environments. Therefore, it is possible that the combination of training methods, learning style and social pres-ence information richness (SPIR) attributes may jointly determine learning outcomes. This is not the case for BM approach in F2F environment. The self-paced online learning environment may alter the assertion. Hence, it may be necessary



to conduct longitudinal studies of the influence of learning style on learning performance and trainee reaction.

IMPLICATIONS FOR PRACTICE

The largest implication for practice is that ABM and SBM may provide cost-effec-tive substitutes for FBM without significant reductions in training outcomes in the long term. While it may still be true that FBM is still the most effective approach to improve KNT in the short term, ABM and SBM have similar leverage in KFT in the short term and KNT in the long term. Regardless of training environ-ments, trainees have same satisfaction levels in the near- and long-term. These findings strongly indicate that the cost issue is more important than learning effectiveness. When given the options to decide which BM approach to take in the long term, nonperformance issues (teacher and facility availability, trainee’s preferences, location and convenience issues) have to be first taken into account.

CONCLUSIONThe success of an online training strategy

depends on its effectiveness in improving learn-ing outcomes. This study, built on well-accepted frameworks for training research (Bostrom, Olfman & Sein, 1990; Simon & Werner 1996), examines the relative effectiveness of the behav-ior modeling approach in online synchronous, online asynchronous and face-to-face environ-ments. The results from this experiment provide an empirical basis for the development of an online behavior modeling strategy: (1) FBM is more effective than ABM and SBM for knowl-edge transfer in the short term (KNT), and (2) ABM and SBM are as effective as FBM for knowledge transfer and overall satisfaction in the long term (KFT).

What is learned from this study can be summarized as follows: When conducting soft-ware training, it may be almost as effective to use online training (synchronous or asynchronous) as it is to use a more costly face-to-face training

in the long term. In the short term face-to-face knowledge transfer model still seems to be the most effective approach to improve knowledge transfer in the short term.

The limitation of this experimental study is that it was conducted with a homogeneous group with Taiwanese cultural and educational backgrounds. Therefore, this study may be con-strained with the generalizability of its findings to different cultural contexts.

Hofstede (1997) stated that the domains of education, management and organization have nurtured the values context that differs from one country to another. Cultural influences have been discerned in the study of Internet us-age (Lederer, Maupin, Sena & Zhuang, 2000; Moon & Kim, 2001; Straub, 1997) and Web site design (Chu, 1999; Svastisinha, 1999). Users from different cultures have different perceptions about the usefulness and ease of use regarding different information systems (Straub, 1994). E-learning systems may differ based on the cultural backgrounds of the learners to improve their satisfaction levels and cognitive gains. Benefits of the congruence may include the improvement of (1) global e-learning adop-tion rate and (2) learning outcomes (attitude and cognitive gains). From the perspective of research design (Kerlinger & Lee, 2000), a cross-cultural study to replicate the study with American or European subjects may further validate and extend the generalizability of the findings.

The study has accomplished its major goal; it provides evidence as to the relative effectiveness of the behavior modeling ap-proach in different learning environments for software training. This research somewhat improves the generalizability of theories on the behavior modeling approach in different learning environments.

REFERENCES

Ahrens, J. D., & Sankar, C. S. (1993). Tailor-ing database training for end users. MIS Quarterly, 17(4), 419-439.

Aniebonam, M. C. (2000, October). Effective

100 Journal of Distance Education Technologies, 4(4), ��-102, October-December 2006


distance learning methods as a curricu-lum delivery tool in diverse university environments: The case of traditional vs. historically black colleges and universi-ties. Communications of the Association for Information Systems, 4(8), 1-35.

Baldwin, T.T., & Ford, J.K. (1988). Transfer of training: A review and directions for future research. Personnel Psychology, 41, 63-105.

Bandura, A. (1977). Social learning theory. Morristown, NJ: General Learning Press.

Bayman, P., & Mayer, R. E. (1988). Using con-ceptual models to teach BASIC computer programming. Journal of Educational Psychology, 80(3), 291-298.

Bielefield, A., & Cheeseman, L. (1997). Tech-nology and copyright law. New York: Neal-Schuman Publishers, Inc.

Bostrom, R. P., Olfman, L., & Sein, M. K. (1990). The importance of learning style in end-user training. MIS Quarterly, 14(1), 101-109.

Bruner, J. (1996). Toward a theory of instruc-tion. New York: Norton.

Chu, G.-L. (1999). The relationships between cultural differences among American and Chinese university students and the design of personal pages on the World Wide Web. Unpublished doctoral dissertation, University of Georgia.

Compeau, D. R., & Higgins, C. A. (1995). Application of social cognitive theory to training for computer skills. Information Systems Research, 6(2), 118-143.

CyberAtlas. (2003). E-Learning market expand-ing beyond IT training. Jupiter Research. Retrieve July 28, 2006, from http://cy-beratlas.internet.com/markets/educa-tion/article/0,,5951_914901,00.html

Decker, P. J., & Nathan, B. R. (1985). Behavior modeling training. New York: Praeger.

Donati, J. (2003). Least-squares deconvolution of Stellar Spectra. Retrieved July 28, 2006, from http://webast.ast.obs-mip.fr/people/donati/multi.html

Fulk, J. (1993). Social construction of com-

munication technology. Academy of Management Journal, 36, 921-950.

Gagne, R. M. (1992). Principles of instructional design. New York: Holt, Rinehart and Winston, Inc.

Gist, M. E., Schwoerer, C., & Rosen, B. (1989). Effects of alternative training methods on self-efficacy and performance in computer software training. Journal of Applied Psychology, 74, 884-891.

Hofstede, G. (1997) Cultures and organiza-tions: Software of the mind. New York: McGraw-Hill.

Horton, W. (2000). Designing Web-based training: How to teach anyone anything anywhere anytime. New York: John Wiley & Sons.

IDC. (2002, September 30). While corporate training markets will not live up to earlier forecasts, IDC suggests reasons for opti-mism, particularly e-learning. Retrieved July 28, 2006, from http://www.idc.com/getdoc.jhtml?containerId=pr2002_ 09_17_150550

Ives, B., Olson, M., & Baroudi, S. (1983). The measurement of user information satis-faction. Communications of the ACM, 26, 785-793.

Kerlinger, F. N., & Lee, H. B. (2000). Founda-tions of behavioral research. New York: Harcourt Brace College Publishers.

Kirpatrick, D. L. (Ed.). (1967). Evaluation of training: Training and development handbook. New York: McGraw-Hill.

Lederer, A. L., Maupin, D. J., Maupin, M. P., Sena, M.P. & Zhuang, Y. (2000). The technology acceptance model and the World Wide Web. Decision Support Systems, 29, 269-282.

Leidner, D. E., & Jarvenpaa, S. L. (1995). The use of information technology to enhance management school education: A theoretical view. MIS Quarterly, 19, 265-291.

Levene, H. (1960). In I. Olkin et al. (Eds.) Con-tributions to probability and statistics: Essays in honor of Harold Hotelling. (pp. 278-292). Stanford University Press.

Journal of Distance Education Technologies, 4(4), ��-102, October-December 2006 101


Lewin, K. (1951). Field theory in social science: Selected theoretical papers. New York: Harper and Row.

McGehee, W., & Tullar, W. (1978). A note on evaluating behavior modification and behavior modeling as industrial training techniques. Personal Psychology, 31, 477-484.

Mennecke, B. E., Crossland, M. D., & Killing-sworth, B. L. (2000). Is a map more than a picture? The role of SDSS technology, subject characteristics, and problem com-plexity on map reading and problem solv-ing. MIS Quarterly, 24(4), 601-627.

Moon, J., & Kim, Y. (2001). Extending the TAM for a World Wide Web context. Informa-tion & Management, 38, 217-230.

Morris, D., Shaw, B., & Perney, J. (1990). Helping low readers in grades 2 and 3: An after-school volunteer tutoring pro-gram. The Elementary School Journal, 91, 133-150.

Paris, S. G., Cross, D. R., & Lipson, M. Y. (1984). Informed strategies for learning: A program to improve children’s reading awareness and comprehension. Journal of Educational Psychology, 7, 1239-1252.

Ramsden, P. (Ed.). (1988). Context and strategy: Situational influences on learning. In Learning strategies and learning styles. New York: Plenum Press.

Salomon, G., & Perkins, D. N. (1988). Teach-ing for transfer. Educational Leadership, 46(1), 22-35.

Santhanam, R., & Sein, M. K. (1994). Im-proving end-user proficiency: Effects of conceptual training and nature of inter-action. Information Systems Research, 5(4), 378-399.

Simon, S. J., Grover, V., Teng, J. T. C., & Whitcomb, K. (1996). The relationship of information system training methods and cognitive ability to end-user satisfac-tion, comprehension, and skill transfer: A longitudinal field study. Information Systems Research, 7(4), 466-490.

Simon, S. J., & Werner, J. M. (1996). Computer training through behavior modeling, self-paced, and instructional approaches: A field experiment. Journal of Applied Psychology, 81(6), 648-659.

Skinner, B. F. (1938). The behavior of organ-isms: An experimental analysis. New York: Appleton-Century Company, Incorporated.

Straub, D. W. (1994). The effect of culture on IT diffusion: E-mail and FAX in Japan and the U.S. Information Systems Research, 5(1), 23-47.

Straub, D., Keil, M., & Brenner, W. (1997). Testing the technology acceptance model across cultures: A three country study. In-formation and Management, 33, 1-11.

Svastisinha, R. W. (1999). Wahhn: Web-based design. Wind and human comfort for Thai-land. Unpublished doctoral dissertation, University of Southern California.

Thorndike, R. L. (1949). Personnel selection: Test and measurement techniques. New York: John Wiley & Sons.

Yi, M. Y., & Davis, F. D. (2001). Improving computer training effectiveness for de-cision technologies: Behavior modeling and retention enhancement. Decision Sciences, 32(3), 521-544.

102 Journal of Distance Education Technologies, 4(4), ��-102, October-December 2006


Dr. Charlie C. Chen is an assistant professor in the Department of Computer Information Systems at Appalachian State University. He received his Ph.D. in Management Information Systems from Claremont Graduate University in 2003. He is also a certified project management professional (PMP). Dr. Chen has authored more than 20 referred articles and proceedings, and presented at many professional conferences and venues.

Dr. Ruey-shiang Shaw is an associate professor and chairman of the Information Management department at Tamkang University in Taiwan. He received his Ph.D. in Business Administra-tion of Management Sciences from Tamkang University. His current main research areas are information security awareness, information education, and strategic information planning.

Online Synchronous vs. Asynchronous Software Training Through

Documents

Transcript of Online Synchronous vs. Asynchronous Software Training Through