1

Author: Kamper, Candace, E

Title: An E-Learning Usability Study: The Impact of Two Different Writing Styles on

Usability

The accompanying research report is submitted to the University of Wisconsin-Stout, Graduate School in partial

completion of the requirements for the

Graduate Degree/ Major: MS Training and Development

Research Adviser: Kat Lui, Ph.D.

Submission Term/Year: Spring, 2012

Number of Pages: 54

Style Manual Used: American Psychological Association, 6th

edition

I understand that this research report must be officially approved by the Graduate School and

that an electronic copy of the approved version will be made available through the University

Library website

I attest that the research report is my original work (that any copyrightable materials have been

used with the permission of the original authors), and as such, it is automatically protected by the

laws, rules, and regulations of the U.S. Copyright Office.

My research adviser has approved the content and quality of this paper.

STUDENT:

NAME Candace E Kamper DATE: 04/09/2012

ADVISER: (Committee Chair if MS Plan A or EdS Thesis or Field Project/Problem):

NAME Dr. Kat Lui DATE: 04/10/2012

----------------------------------------------------------------------------------------------------------------------------- ----

This section for MS Plan A Thesis or EdS Thesis/Field Project papers only

Committee members (other than your adviser who is listed in the section above)

1. CMTE MEMBER’S NAME: DATE:

2. CMTE MEMBER’S NAME: DATE:

3. CMTE MEMBER’S NAME: DATE:

---------------------------------------------------------------------------------------------------------------------------------

This section to be completed by the Graduate School This final research report has been approved by the Graduate School.

Director, Office of Graduate Studies: DATE:

2

Kamper, Candace E. An E-Learning Usability Study: The Impact of Two Different Writing

Styles on Usability

Abstract

Usability testing was conducted to determine if writing style in an e-learning lesson impacts

efficiency as measured by completion time. Two e-learning lessons teaching and simulating a

workplace software application were created. Lesson A had instructions written with the screen

location of the task and then the task to be completed as in, “In the first name field, enter ‘Ann.’”

Lesson B had instructions written with the task to be completed and then the screen location of

the task. An example of Lesson B style is, “Enter ‘Ann’ in the first name field.” Study

participants completed either Lesson A or B with 6 participants completing Lesson A and 6

different participants completing Lesson B. Time for completion was recorded. The writing

style in Lesson A, characterized by stating the screen location first and then the task to be

completed, had a 14.5% quicker mean time for completion and a 12.2% faster median time for

completion as compared to Lesson B. Based on this study, it is recommended that the writing

style used in Lesson A be used when writing e-learning lessons simulating a workplace software

application.

3

Table of Contents

.................................................................................................................................................... Page

Abstract ........................................................................................................................................... 2

Chapter I: Introduction .................................................................................................................... 5

Statement of the Problem .................................................................................................... 5

Purpose of the Study ........................................................................................................... 6

Assumptions of the Study ................................................................................................... 6

Definition of Terms............................................................................................................. 7

Limitations of the Study...................................................................................................... 7

Methodology ....................................................................................................................... 7

Chapter II: Literature Review ......................................................................................................... 9

Usability .............................................................................................................................. 9

Usability Evaluation Methods............................................................................................. 9

Usability Testing ............................................................................................................... 11

Quantitative and Qualitative Data ..................................................................................... 11

Planning for Usability Testing .......................................................................................... 12

Number of Participants ..................................................................................................... 13

Selection of Participants ................................................................................................... 13

Testing Environment ......................................................................................................... 14

Writing Style ..................................................................................................................... 15

Font and Color Selection .................................................................................................. 16

Time to Complete Task ..................................................................................................... 16

Transfer of Training .......................................................................................................... 17

4

Chapter III: Methodology ............................................................................................................. 18

Subject Selection and Description .................................................................................... 18

Instrumentation ................................................................................................................. 19

Data analysis. .................................................................................................................. 20

Limitations ........................................................................................................................ 20

Chapter IV: Results ....................................................................................................................... 21

Item Analysis .................................................................................................................... 21

Figure 1. Comparison of completion times for 12 test participants with 6 participants

completing Lesson A and 6 participants completing Lesson B. ....................................... 22

Figure 2. Mean completion time for Lessons A and B. A 95% confidence level is

represented by the error bars. ............................................................................................ 23

Chapter V: Discussion .................................................................................................................. 25

Limitations ........................................................................................................................ 25

Conclusions ....................................................................................................................... 26

Recommendations ............................................................................................................. 27

References ..................................................................................................................................... 28

Appendix A: Lessons A and B...................................................................................................... 32

5

Chapter I: Introduction

An employee participated in an average of 32 hours (Green & McGill, 2011) to 39 hours

(Training, 2011) per year of employer-sponsored training. Training may be delivered online, by

an instructor in a classroom, or through a blending of delivery methods. The average amount

spent per employee for training was $1,228 (Green & McGill, 2011) which doesn’t include the

cost of the employee’s time to participate in training. The use of online, employer-sponsored

self-study training programs was reported by 21.9% of companies in 2011 (Training, 2011).

Additionally, learning to use computer software is often done through e-learning with 59% of

training in the use of software done online. Software application training, done through the use

of some type of tool that simulates the software being used, was done by 46% of companies in

the Training (2011) study.

As online courses are developed to teach a particular software application, decisions

about topics, number of topics, and length of an e-learning lesson are made. Decisions are also

made about layout, font size and color, background color, and writing style. All decisions made

about course development impact how effectively the course meets the stated learning objectives

for the learner and business goals for the company.

Statement of the Problem

Development of e-learning lessons often includes instructional designers, subject matter

experts, project and team managers, and web developers (Bersin, 2004). The end-user of

e-learning may be entirely left out of the development process. Koohang (2004a) recommends

moving from the point of view of a web developer or instructional designer to the point of view

of a user who brings personal experience to e-learning. Usability testing of e-learning,

conducted throughout the development process, can identify opportunities for improvements that

6

can increase the probability of meeting the learner’s needs and of reaching a company’s business

objectives.

Font and font size, colors, and sentence length have been studied to increase ease of

reading on the computer screen. E-learning, in particular when teaching a software application,

includes step-by-step instructions for entering information and selecting menu choices. Can

instructions be written in such a manner as to increase the efficiency of completing the lesson?

Purpose of the Study

This study will attempt to identify a difference in efficiency, as measured by time to

complete tasks, between two different styles used in e-learning writing. Usability testing will be

done to collect the amount of time it takes a user to complete one of two e-learning lessons; each

lesson written with a different style. The writing style providing the higher efficiency can be

used in future e-learning lesson development.

Using a standard best-practice writing style can, in subsequent e-learning courses,

decrease development costs due to less rework. Employees completing e-learning lessons with

instructions written in a more efficient-to-complete style will spend less time to take an e-

learning course thereby saving employee time and company cost. Highly usable courses increase

the amount of learning taken from an e-learning course and applied in the workplace (Park and

Wentling, 2007).

Assumptions of the Study

Using a website and taking an e-learning lesson both include reading information on a

computer monitor, maneuvering on a screen, and entering information. For this study, an

assumption is that research concerning website usability applies to e-learning lessons.

7

Definition of Terms

E-Learning. Any planned learning experience that delivers information through the use

of computer technology (Horton, 2006). For purposes of this research, when the term e-learning

is used, it is further defined as a unit of study available on-demand for an individual’s use.

Qualitative data. Collected observations of a facilitator and comments of test

participants (Barnum, 2011).

Quantitative data. Collected data that can be tallied and analyzed using statistical tools

(Barnum, 2011).

User-centered design. The intentional use of processes to create products which are

highly usable because design decisions have centered on the end-user of the product (Rubin and

Chisnell, 2008).

Limitations of the Study

Adults read at different speeds. No attempt was made to measure the study participants’

speed of reading. Given the size of the study, naturally slower or faster readers could have taken

one of the two lessons and impacted the timed results.

Study participants’ prior experience with computers was not measured. Granic and

Cukisic (2011) identified two factors that lead to longer task completion times in e-learning. The

two factors are less experience with computers and less experience with the e-learning system.

This is not factored in to the study.

Methodology

A review of the literature describing usability testing and factors impacting the speed of

reading in e-learning will be presented. After study participants take one of the two e-learning

lessons written with different styles, a comparison will be made between the two groups’ times

8

of completion of their e-learning lesson. A recommendation for most efficient writing style will

be made, if the results indicate there is a more efficient writing style.

9

Chapter II: Literature Review

Usability

The term usability has been defined in a number of ways. In Alshamri and Mayhew’s

(2009) review of definitions from organizations such as the International Organization for

Standardization to a variety of other research studies, the definitions all include some aspect of

“efficiency, effectiveness, and user satisfaction” of a product (p. 402). Rubin and Chisnell

(2008) add usefulness and accessibility in their understanding of usability.

For Rubin and Chisnell (2008), usefulness is how well a product achieves the results

expected when used for its intended purpose by a user. Efficiency is the how fast the expected

results can be achieved with an acceptable error and completion rate. Effectiveness can be

determined by examining if the product works as anticipated and expected by the user. The

user’s opinions and perceptions about the product are included in a product’s satisfaction rating.

The final factor in usability for Rubin and Chisnell is how usable the product is for people with

disabilities.

Broadening the usability definition to apply to e-learning, Cooper, Colwell and Jelfs

(2007) add that e-learning usability must also include meeting the learning objectives intended

for a student’s use of e-learning.

Usability Evaluation Methods

The development of products with high usability is the goal of user-centered design

(UCD). Organizations employing a user-centered design approach can increase return on

investment by increasing sales, lowering development costs, and lowering ongoing costs to

support the product (Cost and return on investment). Usability evaluation methods are any

processes used to provide information for UCD according to Barnum (2011) and Rubin and

10

Chisnell (2008). Researchers have organized usability evaluation methods by different criteria.

Barnum (2011) classifies usability evaluation methods for UCD into three groups: pre-

development, during development, and post-development. Rubin and Chisnell (2008) do not sort

usability evaluation methods into groups, but see a progression of which component is most

helpful at different points of UCD. Classifying usability evaluation methods based on the

presence or no presence of the product, on the presence or no presence of a typical user, and if

the evaluation is based on observing a typical user or the observations of a professional usability

expert was suggested by Sing and Der-Thanq (2004).

Usability testing is one of a multitude of usability evaluation methods. Additional

usability evaluation methods presented by Barnum (2011) and Rubin and Chisnell (2008)

include: focus groups, card sorts, participatory designs, heuristic evaluations, and paper

prototyping. Barnum (2011) adds meetings, interviews, diary/photo study, contextual inquiry,

remote testing, design critique, data analysis, affinity diagramming, and task analysis to the list

of preferred evaluation methods. Additional note-worthy usability evaluation methods according

to Rubin and Chisnell (2008) are: ethnographic research, walk-throughs, surveys, and follow-up

studies. They comment that the suggested methods are not an exhaustive list, but a

representative sampling of usability evaluation methods.

Post-tests are customarily used in educational settings and are considered a method of

usability evaluation for De Villiers (2007). If e-learning was difficult to use, little learning

would take place, and results on post-tests would be low.

Regardless of the evaluation method used, Hornbæk (2010) argued that too often

usability evaluation results focus on identifying problems without offering clear directions to

11

solve issues. He maintains that offering solutions to problems are the most useful result of

whatever usability evaluation method is used.

Usability Testing

With usability testing being one type of usability evaluation method, usability testing is

defined as the act of watching study participants use a product as end-users would use it in real

life (Barnum, 2011). There is a critical difference between testing a product and usability testing

which is examining how the user interacts with the product (O’Bryan, et al 2009).

Barnum (2011) and Rubin and Chisnell (2008) place usability testing as a step or one

component in user-centered design. Conducting usability testing as frequently as possible is

essential during the development phase. It is also useful during the post-development phase of a

product in field testing (Barnum, 2011). Rubin and Chisnell, too, see usability testing as critical

to influence product development during the development cycle of test, prototype, and test again

until the product is ready for release. Krug (2000) differentiates usability testing from focus

groups. Focus groups are used very early in the development process to seek insights about the

needs and wants for a product and utilizes group interaction to generate ideas. Usability testing,

on the other hand, is conducted when a version of the product is available so that study

participants can be observed using the product.

Quantitative and Qualitative Data

Both quantitative and qualitative data can be gathered during usability testing.

Quantitative data can include two types: performance and preference (Barnum, 2011). Because

usability testing always looks at the interaction of a user with what is being tested, performance

testing measures some aspect of the user interaction. Types of measurements include: “time on

task; number of errors; recovery from errors; success or failure at task completion; use of help,

12

documentation, or embedded assistance; and so forth” (Barnum, 2011, p137). Often, after a test

is completed, study participants are given a questionnaire and asked to rate different aspects of

the product. Ratings are quantitative and are classified as preference data (Barnum, 2011).

Questionnaires can also be used to collect qualitative data from study participants

indicating their preferences. Qualitative data gathered as users’ preferences includes comments

and open-ended questions (Barnum, 2011, and Rubin & Chisnell, 2008). Other methods for

collecting qualitative data are noticing non-verbal reactions, taking note of what actions are taken

when using the product, and recording comments of participants as they think aloud (if asked to

do so) while testing the product (Barnum, 2011).

Planning for Usability Testing

One approach to planning for e-learning usability testing is to use the principles of project

management (Bartz, 2010). At each step in project management, the needs of the learners are

considered. During the execution of the project plan, at the point an e-learning lesson has been

developed, usability testing can be done. Plans for usability testing should be included in a

project’s scope, allotted budget, and timeframe.

The specifics of usability test planning are outlined by Barnum (2011) and Rubin and

Chisnell (2008). Barnum begins with developing an agenda for an initial meeting to which the

critical stakeholders are invited. Rubin and Chisnell and Barnum both highlight the need to

determine test goals, to identify characteristics of the testers, to decide qualitative and

quantitative data to be collected, and to choose how results will be reported to the critical

stakeholders. Logistics such as arranging for the testing location, setting up for testing, and

recruiting for study participants are responsibilities that must be assigned and carried out.

13

Number of Participants

Studies have been conducted to determine the number of participants required to validate

a usability test. Hwang and Salvendy (2010) reviewed 17 papers which included 27 experiments

for data collection. They determined that the number of testers required is dependent on the type

of testing method used. For instance, when using the Think Out Loud method, five users

discovered 80% of possible issues of a website. A different method called Cognitive

Walkthrough required 13 testers to identify issues in a valid test. Their recommendation of using

10±2 testers “can be applied to a general or basic evaluation situation” (p.133).

Becker (2011) is of the opinion that testing often with fewer participants provides “the

most valuable results” (p. 40). Krug (2000) concurs noting that testing early in product

development even with only one participant is better than doing no early testing

Selection of Participants

While an assumption may be that participants in a usability test must belong to the

intended end-user group, in fact for website testing, familiarity with using the web is all that is

necessary according to Krug (2000). Krug also believes that while it is best to recruit

participants with characteristics of end-users, if none are available, do not forgo testing. For

testing academic library websites, Becker (2011) recruited students enrolled in English/Speech

Communication classes and library school students. Both sources provided excellent

information from the testing.

Rubin and Chisnell (2008) disagree with this approach. They asserted that using test

participants who do not have the same characteristics as the intended users provides data that has

very limited significance. Instead, every effort must be made to recruit test participants who

match the end-user’s needed skills and experience. Krug’s view may be valid for his work

14

because his focus is solely on website testing. Rubin and Chisnell apply their usability testing

principles to a wide variety of products and this may account for the differing recommendation.

Another approach is suggested by Barnum (2011). As a step in test preparation, the

creation of personas – stories and graphics describing an imagined user which is based on

characteristics of a specific user group – is recommended. Any number of personas can be

created with the goal of creating more than two and up to 12 personas to represent primary user

groups. Once personas are finalized, test participants are recruited to represent the different

personas.

One aspect when testing an e-learning course that should also be taken into consideration

is having different learning styles represented in the test participants according to V. Beer (as

cited in Lee, Chamer, & Ely, 2005). A second aspect to consider is to have test participants have

a range of knowledge about the topic being taught to increase the likelihood that the e-learning

course is useful for a large segment of users.

Testing Environment

For website testing, the minimal requirements for a testing lab are a room large enough

for the testing participant, facilitator, table on which the computer sets, and two chairs (Barnum,

2011). It is best if the room is a private space where no interruptions will occur (Becker, 2011).

If observing how a user maneuvers around a website to locate information is one purpose of a

usability test, Krug (2000) also advocates the use of a video recorder and tripod connected to a

viewing room so those involved with development can observe the test. The recording can also

be viewed later, if necessary. Another advantage of having a video recorder broadcast the test

proceedings to another room is that it allows observers to discuss what is happening without

interfering with the test (Rubin & Chisnell, 2008).

15

Writing Style

Because reading text on a screen is 25% slower than reading a printed page (Campbell,

2004), changes to writing style have been found to improve ease of reading and increase content

retention. Morkes and Nielsen (1997) advocate for making content “concise, scannable, and

objective [as compared to text written in a “promotional” style]” (Conclusions, para 3). When

all three factors were present, users’ success on a memory test increased 62% in the Morkes and

Nielsen study of website reading. Campbell (2004) also recommends paragraphs contain five or

fewer sentences. Sentences containing a list of items should instead be listed as bulleted items.

Calhoun (2006) interviewed experienced e-learning developers aiming to learn how

e-learning writing was different from other writing styles. From his interview with Saul

Carliner, Calhoun heard of the need to provide learners with text that requires reading only once

to be comprehended. Also learned by Calhoun was the belief that writing has to be “short and

stimulating” (para 4).

Providing concise text for reading online is advocated by others. Morkes and Nielsen

(1998) point out that concise writing does not mean giving the reader incomplete information. In

one study, they found that users perceived the concisely written version of a website provided

more information than one with more text. Their supposition is that longer texts are scanned

rather than read, and information is missed when scanning. Lynch and Horton (2008) agreed

with this recommendation and added that more concise writing doesn’t mean the writing is

written as if the reader has a lower level of intelligence.

Using a conversational style of writing is advocated by Clark and Mayer (2008) based on

their review of published research. The conversational writing style uses a second person active

voice and a less formal style.

16

Font and Color Selection

Campbell (2004) suggested that fonts should be sans serif and no smaller than 12 point.

A limited use of font styles was recommended. Also emphasized by Campbell is the use of

upper and lowercase letters to ease word recognition. Online (2003) also recommends the use of

upper and lowercase letters to increase the speed at which viewers can read the text. Using lower

case letters allows readers to recognize the shape of words quickly. The use of fonts such as

Verdana, Helvetica, Times, and Arial which are simple in style and common on most computers

can speed reading as well.

The common practice on websites to underline words that are linked to other sites means

that underlining should not be done in e-learning writing (Text appearance). The speed of

reading can be impeded by up to 20% when the appearance of fonts change due to the use italics,

size, or bolding so such changes should be done only for emphasis.

The impact of the font and background color was investigated by Zufic and Kalpic

(2009). In their study, black with light yellow background yielded a 17.64% higher percentage

of remembered terms as compared to the more common black text with white background. To

aid the speed in reading, a high contrast between a plain background and font color should be

provided (Campbell, 2004).

Time to Complete Task

In addition to the cost of employees’ time to take corporate-sponsored e-learning, Granic

and Cukisic (2011) have shown users taking longer to complete tasks in an e-learning lesson

have lower scores in memory tests and higher self-reported frustration levels. Two factors

identified as leading to more time to complete tasks include less experience with computers and

less experience with the e-learning system. They recommend taking in to account the differing

17

abilities and skill levels of the intended audience. E-learning therefore must be built “in a way

that facilitates the learning experience” (p.119). Furthering this idea, De Villiers (2007) found

that users experienced with common software expect e-learning lessons to behave in the same

manner as common software; that is, pressing a key in an e-learning lesson should provide the

same results as pressing that key in common software. Otherwise, users must first learn how

e-learning works before the users can begin to learn the content of the e-learning lesson.

A study by Koohang (2004b) agreed with Granic and Cukisic findings that users with

more experience using computers and the Internet were more satisfied with e-learning

courseware. However, Koohang’s study found that those who spent more time completing

assignments using the courseware had higher positive perceptions towards the courseware.

Transfer of Training

The relationship between e-learning usability and using knowledge gained through taking

an e-learning lesson to job tasks and skills was shown to have a significant correlation by Park

and Wentling (2007). Usability for Park and Wentling included understandable directions and

satisfaction with the e-learning courseware which was perceived to be easy to learn how to use

and easy to use. Further, they found that the user’s perception of usability of e-learning “directly

affects their transfer of training” (p. 324).

18

Chapter III: Methodology

End-user needs may be considered during the development of e-learning lessons, but

end-users are usually not included in the development cycle. Usability testing of e-learning,

done throughout the development process, can identify opportunities for improvements that can

increase the probability of meeting the learner’s needs and of reaching a company’s business

objectives.

Step-by-step instructions included in e-learning lessons can be written in different styles.

How does the style used for instructions, specifically when teaching a workplace software

application, impact the time to complete the lesson? Is there a difference in efficiency when a

usability test is conducted? A benefit of using a writing style that increases efficiency may

include decreased development costs due to less rework. Employees completing e-learning

lessons with instructions written in a more efficient-to-complete style may spend less time taking

an e-learning lesson thereby saving employee time and company cost.

This study examines the use of two writing styles in an e-learning lesson. Study

participant selection, manner of testing, and method of data analysis are as follows.

Subject Selection and Description

As recommended by the literature, 12 people were recruited to participate in the study.

There was no access to the intended end-user of the e-learning lesson simulating a workplace

software application. Participants in this study were members of my peer group who volunteered

to participate in the study. Participants ranged in age between 30 and 50 years of age. The

participants had a high level of computer confidence based on knowledge of the participants’

work.

19

Instrumentation

Two e-learning lessons teaching a workplace software application were created for this

study. Lesson A had instructions written with the screen location of the task first and then the

task to be completed. An example of this writing style is, “In the first name field, enter ‘Ann.’”

Lesson B had instructions written with the task to be completed first and then the screen location

of the task. An example of Lesson B writing is, “Enter ‘Ann’ in the first name field." Except for

this change in writing style, all other writing and screens were identical. See Appendix A for the

two e-learning lessons.

Data Collection Procedures

Six slips of paper with a printed ‘A’ and six slips of paper with a printed ‘B’ were

prepared and placed in a box. A desk with a computer and two chairs were arranged in a private

room.

The purpose of the study was briefly explained to a study participant. Each study

participant signed a consent form of voluntary participation as required by the university’s

Institutional Review Board. To determine which lesson a participant would take, the participant

drew a prepared slip of paper from the box. If an 'A' slip was drawn, the participant opened

e-learning Lesson A on the computer; if 'B' was drawn, the participant opened Lesson B on the

computer. Each slip was drawn once and then discarded.

The study participant was instructed to complete the e-learning lesson accurately and at a

speed which was comfortable. When all questions about the study process were answered, the

participant was asked to begin working on the e-learning lesson. At that moment, the start time

was recorded. The end time was noted at the completion of the last screen’s instruction task.

The participant was thanked for his or her participation in the study.

20

Data analysis. A comparison was made of the times to complete the two lessons using

the data analysis tools available in Microsoft® Office Excel 2010.

Limitations

No attempt was made to measure the study participants’ speed of reading. Naturally

slower or naturally faster readers could have taken one of the two lessons more often and this

factor impacted the timed results.

Study participants’ prior experience with computers was not measured. Granic and

Cukisic (2011) have shown that two factors identified as leading to more time to complete tasks

include less experience with computers and less experience with the e-learning system.

Experience with computers was not factored in to the study.

Time recording was done with a stop watch. A computer-aided timing system was not

available. Variation on when the start and stop button was pressed on the stop watch could

impact the timed results.

21

Chapter IV: Results

During the creation of e-learning, the end-user of e-learning often times is entirely left

out of the development process. Usability testing of e-learning, conducted throughout the

development process, can identify opportunities for improvements that can increase the

probability of meeting the learner’s needs and of reaching a company’s business objectives.

E-learning, in particular when teaching a software application, includes step-by-step

instructions for entering information and selecting menu choices. Can instructions be written in

such a manner as to increase the efficiency of completing the lesson? This study attempted to

identify a difference in efficiency, as measured by time to complete tasks, between two different

styles used in e-learning writing. The writing style providing the higher efficiency can be used

in future e-learning lesson development.

A usability test was conducted to compare the efficiency of two writing styles. Two

e-learning lessons were created with task instructions written differently in each lesson. Study

participants completed one of the two e-learning lessons demonstrating a workplace software

application. The amount of time each participant took to finish the lesson was recorded.

Item Analysis

Seven females and five males volunteered to participate in this study. Because there was

no access to members of the intended user group, participants were members of my peer group.

Participants ranged in age between 30 and 50 years of age. The participants had a high level of

computer confidence based on knowledge of the participants’ work.

Lesson A, written with screen location first and then task instruction, recorded

completion times from 2 min 57 s to 5 min 19 s. Lesson B, written with task instruction first and

then screen location, recorded completion times from 3 min 20 s to 5 min 44 s. As shown in

22

Figure 1, the fastest completion time for Lesson A was less than the fastest completion time for

Lesson B. The second fastest completion time for Lesson A was again less than the second

fastest completion time for Lesson B. The difference in completion times was the greatest for

fifth slowest completion times, shown as completion times for Test Participants Number 5. Still,

the pattern of Lesson A having a faster completion time as compared to Lesson B was true for

each comparison point.

Figure 1. Comparison of completion times for 12 test participants with 6 participants completing

Lesson A and 6 participants completing Lesson B.

The standard deviation for Lesson A was 51 s and for Lesson B was 1 min 4 s. The

standard deviation differed between Lesson A and Lesson B mainly due to the difference in

completion times previously noted at the Test Participants comparison point 5.

23

The difference from fastest to slowest completion times for Lesson A was 2 min 22 s.

For Lesson B, the difference in completion times from fastest to slowest was 2 min 24 s.

Although no attempt was made to rate a participant’s natural reading speed, it is interesting to

note there was only a 2 second difference between Lesson A’s ranges of completion times to

Lesson B’s ranges of completion times.

The mean time for completion for Lesson A was 3 min 55 s and for Lesson B was 4 min

29 s. The median time for completion for Lesson A was 3 min 50 s and for Lesson B was 4 min

18 s. Therefore, Lesson A had a 14.5% quicker mean time for completion and a 12.2% faster

median time for completion as compared to Lesson B.

As shown on Figure 2, at a 95% confidence level, Lesson A had 54 s and Lesson B had

1 min 8 s.

Figure 2. Mean completion time for Lessons A and B. A 95% confidence level is represented by

the error bars.

24

Each lesson had 6 participants complete the lesson. The sum of the time taken to

complete Lesson A was 23 min 30 s. Lesson B completion time sum was 26 min 55 s. In a

workplace setting, using the median U.S. wage in 2010 (Social Security Online, 2012), wage

costs for 6 participants to complete Lesson A was $4.96 and to complete Lesson B was $5.69.

25

Chapter V: Discussion

E-learning offers employees the opportunity to learn new or required workplace skills

and allows companies to offer on-demand, take-anywhere courses. As companies develop e-

learning for their employees, testing can be done throughout the development process to increase

the usability of a course. Studies have shown that a highly usable course increases the amount of

learning taken from an e-learning course and applied in the workplace (Park and Wentling,

2007).

Usability testing was conducted to learn if writing style in an e-learning lesson impacts

efficiency as measured by completion time. Two e-learning lessons teaching a workplace

software application were created. The two lessons were written with different writing styles for

task completion. Study participants completed either Lesson A or B with 6 participants

completing Lesson A and 6 different participants completing Lesson B. Time for completion

was recorded.

Limitations

No attempt was made to measure the study participants’ speed of reading. Given the size

of the study, naturally slower or faster readers could have taken one of the two lessons and

impacted the timed results.

Study participants’ prior experience with computers was not measured. Granic and

Cukisic (2011) have shown that two factors identified as leading to more time to complete tasks

include less experience with computers and less experience with the e-learning system.

Experience with computers was not factored in the study.

26

Time recording was done with a stop watch. A computer-aided timing system was not

available. Variation on when the start and stop button was pushed on the stop watch could

impact the timed results.

Conclusions

Recommendations found in literature were followed in the planning, designing, and

testing for this study. Usability testing for efficiency proved to be a valuable method to

determine the writing style to be used in future e-learning lessons. Quantitative data was

collected from the participants’ test completion times. This data was analyzed to determine if

writing style impacted completion times.

Lesson A had a 14.5% quicker mean time for completion and a 12.2% faster median time

for completion as compared to Lesson B. In this usability test for efficiency, the Lesson A style

consisting of writing the screen location first and then the task instruction provided a more

efficient method for participants to complete an e-learning lesson.

There was a 2 second difference between the completion time span for Lesson A as

compared to Lesson B. Although the natural speed of reading for participants was not factored

in to the test, it is interesting to note that the difference between the two test spans was 2

seconds. Perhaps the random selection of Lesson A or Lesson B negated any differences in

participants’ individual natural speed of reading.

Companies adopting the Lesson A writing style for its e-learning development can

positively impact costs in two ways. First, less rework and rewriting will be done because the

most efficient style of writing has already been determined. Second, the time for employees to

take e-learning lessons will be reduced which will lessen indirect training costs. With the mean

27

time 12.4% quicker and the median time 14.5% quicker using Lesson A style, a reduction of

time in this range could be expected for employees to take e-learning lessons.

Recommendations

Based on the results of this usability test for efficiency, the Lesson A style consisting of

writing the screen location first and then the task instruction is recommended to be used when

creating e-learning demonstrating a workplace software application.

Additional research using test participants who more closely align with end-user

characteristics needs to be conducted to see if results from this study can be duplicated with an

authentic end-user group.

The e-learning lesson used in this study simulated a workplace software application.

E-learning lessons of a different nature could be created and tested for efficiency. This type of

testing could examine the question, “Does writing style impact the efficiency of e-learning

lessons created for different purposes?”

28

References

2011 Training industry report. (2011). Training 48(6), 22-35.

Alshamari, M., & Mayhew, P. (2009). Technical review: Current issues of usability testing. IETE

Technical Review, 26(6), 402-406. doi: 10.4103/0256-4602.57825

Barnum, C. M. (2011). Usability testing essentials: Ready, set…test!. Burlingame, CA: Elsevier

Inc.

Bartz, J. (2010). The learner’s place in e-learning project management. Journal of Distance

Education, 24(1), 43-54.

Becker, D. (2011). Usability testing on a shoestring: Test-driving your website. Online 35(3), 38-

41.

Bersin, J. (2004). The blended learning book: Best practices, proven methodologies, and lessons

learned. San Francisco, CA: Pfeiffer.

Calhoun, H. (2006). Writing for e-learning. Intercom 53(4), 14-17. Retrieved from

http://archive.stc.org/intercom/PDFs/2006/20064_14-17.pdf

Campbell, K. (2004). E-ffective writing for e-learning environments. Hershey, PA: Information

Science Publishing.

Clark, R. C. & Mayer, R. E. (2008). E-learning and the science of instruction. San Francisco,

CA: Pfeiffer.

Cooper, M., Colwell, C., & Jelfs, A. (2007). Embedding accessibility and usability:

Considerations for e-learning research and development projects. ALT-J: Research in

Learning Technology, 15(3), 231-245. doi: 10.1080/09687760701673659

29

Cost and return on investment. Retrieved from Usability.gov: An official U.S. government

website managed by the U.S. Department of Health & Human Services.

http://www.usability.gov/basics/usasaves/index.html

De Villiers, M. R. (2007). An action research approach to the design, development and

evaluation of an interactive e-learning tutorial in a cognitive domain. Journal of

Information Technology Education, 6, 455-479.

Granic, A., & Cukusic, M. (2011). Usability testing and expert inspections complemented by

educational evaluation: A case study of an e-learning platform. Journal of Educational

Technology & Society, 14(2), 107-123.

Green, M., & McGill, E. (2011). The 2011 state of the industry. T+D, 65(11), 44-50.

Hornbæk, K. (2010). Dogmas in the assessment of usability evaluation methods. Behaviour &

Information Technology, 29(1), 97-111. doi:10.1080/01449290801939400.

Horton, W. (2006). E-learning by design. San Francisco, CA: Pfeiffer.

Hwang, W., & Salvendy, G. (2010). Number of people required for usability evaluation: the

10±2 rule. Communications of the ACM, 53(5), 130-133.

Koohang, A. (2004a). Expanding the concept of usability. Informing Science Journal, 7, 129-

141.

Koohang, A. (2004b) A study of users’ perceptions toward e-learning courseware usability

[Abstract]. International Journal on E-Learning, 3(2), 10-17.

Krug, S. (2000). Don’t make me think!. Indianapolis, IN: New Riders Publishing.

Lee, D., Chamers, T., & Ely, T. (2005). Web-based training in corporations: Design issues.

International Journal of Instructional Media, 32(1), 27-35.

Lynch, P. & Horton, S. (2008). Web style guide (3rd

ed.). Kendalville, IN: Yale University Press.

30

Morkes, J. & Nielsen, J. (1997). Concise, scannable, and objective: How to write for the web.

Retrieved from http://www.useit.com/papers/webwriting/writing.html

Morkes, J. & Nielsen, J. (1998). Applying writing guidelines to web pages. Retrieved from

http://useit.com/papers/webwriting/rewriting.html

O’Bryan, C. A., Crandall, P. G., Shores-Ellis, K., Johnson, D. M., Ricke, S. C., & Marcy, J.

(2009). Optimizing web-based instruction: A case study using poultry processing unit

operations. Journal of Food Science Education, 8(4), 93-100. doi: 10.1111/j.1541-

4329.2009.00082.x

Online. (2003). Journal of Audiovisual Media in Medicine, 26(3), 126-127. doi:

10.1080/01405110310001606344

Park, J.-H., & Wentling, T. (2007). Factors associated with transfer of training in workplace e-

learning. Journal of Workplace Learning, 19(5), 311-329. doi:

10.1108/13665620710757860

Rubin, J., & Chisnell, D. (2008). Handbook of usability testing: How to plan, design, and

conduct effective tests. Indianapolis, IN: Wiley Publishing, Inc.

Sing, C. C. & Der-Thanq, V. C. (2004). A review of usability evaluation methods for

instructional multimedia: An analytical framework. International Journal of Instructional

Media, 31(3), 229-238.

Social Security Online (2011). Automatic increases: Measures of central tendency for wage data.

Retrieved from http://www.ssa.gov/oact/cola/central.html

Text appearance. Retrieved from Usability.gov: An official U.S. government website managed

by the U.S. Department of Health & Human Services.

http://www.usability.gov/pdfs/chapter11.pdf

31

Zufic, J., & Kalpic, D. (2009). More efficient e-learning through design: color of text and

background, Online Submission. World Conference on E-Learning in Corporate,

Government, Healthcare, and Higher Education (ELEARN) 2009. Retrieved from

http://www.eric.ed.gov/PDFS/ED510610.pdf

32

Appendix A: Lessons A and B

Lesson A

33

34

35

36

37

38

39

40

41

42

43

44

Lesson B

45

46

47

48

49

50

51

52

53

54