A Study Into The Effectiveness of Web-based Reading

LI Tianjiao FENG Chenyue ZHAO Fei

QU Qu YAN Li

Agenda

•Motivation and Background•Design of Experiment•Data Analysis and Conclusion

• The rapid development of computer and Internet technologies has made e-Learning become an important learning method.

• However, when we surfing on the internet, we usually fail to obtain information in an effective or efficient way.

• So we are desired to know how to obtain information effectively, accurately, and efficiently.

3

Motivation

4

• As the increasing number of tests (IBT, new CET 4 &6)converted from paper-based to internet-based, more attention is paid or will be paid to obtaining information, ideas, key points from reading materials.• There are so many factors influencing on obtaining information or

the leaning through web-pages, such as:• The layout of web-pages;• The amount & format of assistant information on web-pages;• The amount & format of irrelative information on web-pages;• The length & difficulty of reading materials themselves on web-

pages; • The English skill levels of different individuals;• So on……

Motivation

5

• Investigate the factors that contribute to the effectiveness of learning through web-pages;

• Figure out the way to build a easy-to-learn web learning system;

• Enhance the effectiveness and efficiency of web learning.

Objectives

6

Expert Knowledge/Experience

• One of the key characteristics of e-Learning is its capability to integrate different media, such as text, picture, audio, animation and video to create a multimedia instructional material, promoting the reading interests and willingness of the learner.

( Gillani & Relan, 1997; Vichuda, Ramamurthy, & Haseman, 2001)

• Some research shows that too much unnecessary multimedia elements in instructional material may distract learners and actually decrease learning performance.

(Bartscha & Cobern, 2003; Mayer et al., 2001; Park & Hopkins, 1993; Rieber, 1996;Sweller, van Merrienboer, & Paas, 1998)

Relevant Background

7

Ho, J. & Ko, L. (2003), Design for Experimenting in E-Learning: The Effects of Web Page Design on Students' Test Performance. In G. Richards (Ed.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2003 (pp. 1610-1613). Chesapeake, VA: AACE

Previous Experiments

Response variable:• Test scores

Control variable:• 1. Message area location• 2. Function area location

Relevant Background

8

Responsevariable(units)

Normaloperating level

& range

Meas. precision,accuracy

How known?

Relationship ofresponse variable to

objective

Time used to complete reading the corresponding passage

7 minutes;From 7-3 min to 7+3 min per passage considering different individuals.

By timer;Unit: second

The shorter the required time, the higher learning effectiveness on web-pages.

Correct rate

Correct rate around 0.6 to 1.0, in common per passage.

Correct rate(the ratio of the number of correct answers to all the answers)

The higher the correct rate, the higher learning effectiveness on web-pages.

Response Variables

9

• The layout of reading materials:• The number of columns of the reading materials;• One column with passage and questions;• Two columns, one column with passage only and the other

with questions only. • The existence of assistant information:• Whether there is an outline in front of reading materials;• Yes• No

• The existence of irrelative information:• Whether there are some irrelative pictures in reading materials;• Yes• No

Control Variables

10

• The length of reading materials:• Around 270 words;• 5 choice questions each with 4 choices.

• The difficulty of reading materials:• CET 4 passages;• Arguments with 5-6 paragraphs

• Environment factors:• Noise, illumination, temperature;

• Others:• Possible factors related to computers;• The time of experiment.

Can they be controlled constantly with deliberately selecting?

Held constant Variables

11

• The difference between different individuals:• The learning capability;• The English skill level;• The reading habit; • The gender, sight;• The mood.

• Block:• Different individuals can be treated as a set of blocks.

Nuisance Factors

Design of Experiment

Design of Experiment

Define Control Variables

Deal with Nuisance Variables

Verify Response Variables

13

Control and Response Variables

Reading timeCorrect rate

Number of columns

OutlineIrrelevant information

Define Control Variables

•Control V: Column, irrelevant information, outline

• Full Factorial Design32

Deal with Nuisance Factors

•Possible nuisance

•How to eliminate the effects of these variables?

•Environment? Held constant•Passage? Held constant. If can’t, block•People? Hard to be held constant. Block

•First, we want to know the effects of these variables

People

Passage

Environment

Trial Runs

•Objective: effects of PEOPLE, PASSAGES

•Controlled factors held constant (One column; No outline; No irrelevant Information)

•Environment held constant (No. 4 Teaching Building, 9:00-10:00PM)

•8 people•4 passages per person•We tried to find passages of the same

level

双因子方差分析 : time(min) 与 people, Passage

来源 自由度 SS MS F Ppeople 7 9.7587 1.39410 3.20 0.018Passage 3 2.2559 0.75197 1.73 0.192误差 21 9.1346 0.43498合计 31 21.1492

S = 0.6595 R-Sq = 56.81% R-Sq （调整） = 36.24%

• People are very different (block)

• Passage are the same (held constant)

Response: Correct rate双因子方差分析 : Correct rate 与 people, Passage

来源 自由度 SS MS F Ppeople 7 0.10875 0.0155357 0.69 0.683Passage 3 0.03375 0.0112500 0.50 0.689误差 21 0.47625 0.0226786合计 31 0.61875

S = 0.1506 R-Sq = 23.03% R-Sq （调整） = 0.00% • People are the

same(held constant)• Passages are the

same(held constant)

Response Variables: time and correct rate

• If they are related, correct rate may result from time, rather other control variables.

•We need to fix the response variable

Longer reading time reading

is difficult lower correct

rate

Longer reading time

more effort devoted

higher correct rate

相关 : Correct rate, time(min) Correct rate 和 time(min) 的 Pearson 相关系数 = -0.020 P 值 = 0.914

Not correlationNo need to fix the response

Correlation between Time and Correct rate? Yes

Find another response variable that eliminates the correlation.

X=Correct rate/ Time

No

Analysis two distinct response variables with control variables.

Design of Experiment(1)acabbc

Block 1

abcabc

Block 2

(1)acabbc

Block3

abcabc

Block4

32 Factorial design with ABC confounded

4 treatment per block, 8 blocks, 32 run totally

Time

(1)acabbc

Block 5

abcabc

Block 6

(1)acabbc

Block7

abcabc

Block8

32 Factorial design with ABC confounded

With no block, 32 runs totally

Correct Rate

Design of Experiment

Experiment Size

•The experiment is quite expensive▫The subjects can experiment with one

passage only once▫The reading of one passage takes 7±3

minutes, 4 passages will make the total time around 30 minutes

▫As the time increase, subjects may become impatient and tired, causing bias

Data Analysis and Conclusion

Analysis of Factorial Design

3210- 1- 2- 3

99

95

90

80

70

605040

30

20

10

5

1

标准化效应

百分比

A Bl ocksB Col umnC Out l i neD I r rel evant

因子 名称

不显著显著

效应类型

B

标准化效应的正态图（响应为 Ti me，Al pha = . 05)

拟合因子 : Time 与 Blocks, Column, Outline, Irrelevant * 注 * 此设计具有一些修补运行。将使用回归方法进行分析。Time 的效应和系数的估计（已编码单位）项 效应 系数 系数标准误 T P常量 5.9198 0.2378 24.89 0.000Blocks -1.2111 -0.6056 0.3633 -1.67 0.110Column 1.0396 0.5198 0.2378 2.19 0.040Outline 0.0279 0.0140 0.2378 0.06 0.954Irrelevant 0.1383 0.0692 0.2378 0.29 0.774Blocks*Column -0.1514 -0.0757 0.3723 -0.20 0.841Blocks*Outline -0.3718 -0.1859 0.3723 -0.50 0.623Blocks*Irrelevant 0.0157 0.0079 0.3723 0.02 0.983Column*Outline 0.5007 0.2503 0.2437 1.03 0.316Column*Irrelevant -0.6411 -0.3205 0.2437 -1.32 0.203Outline*Irrelevant 0.2233 0.1117 0.2437 0.46 0.65

When column= -1, reading time is less than column =1. Two column layout is efficient in receiving information from reading.

Time

Analysis of Factorial Design(cont’d)

Column*Outline = -1 format pattern is effective in receiving information from reading.

拟合因子 : Correctness 与 Column, Outline, Irrelevant Correctness 的效应和系数的估计（已编码单位）项 效应 系数 系数标准误 T P常量 0.81875 0.02453 33.38 0.000Column 0.06250 0.03125 0.02453 1.27 0.214Outline -0.03750 -0.01875 0.02453 -0.76 0.452Irrelevant 0.03750 0.01875 0.02453 0.76 0.452Column*Outline -0.11250 -0.05625 0.02453 -2.29 0.031Column*Irrelevant 0.06250 0.03125 0.02453 1.27 0.214Outline*Irrelevant 0.01250 0.00625 0.02453 0.25 0.801

1- 1 1- 1

0. 90

0. 84

0. 78

0. 90

0. 84

0. 78

Col umn

Out l i ne

I r rel evant

-11

Col umn

-11

Outl i ne

Correctness 交互作用图数据平均值

Correct Rate

Design Projection

• Discard D (Irrelevant information) unchanged results

Residual Checking:

Conclusion is reasonable;Underlying

assumptions are satisfied .

Discussion

• Q1: Why format with outline could not increase the efficiency of reading, that is, reduce the time on reading?

A: 1. Outline lies on the top of web page. 2. People don’t have habit of reading web pages with outline

given.

• Q2:Why irrelevant information isn’t significant in response time and correct rate?

A: 1. The effect of irrelevant information is less than system error. This problem could be solved through large amount of runs.

2. The effect of variable is actually slight and ignorable.

ConclusionTime Correct Rate

Better Condition Two column Two column, with outline

Application Passage is long;Retral part is relevant

to the preceding information.

Passage is long;Passage with many

paragraphs;Quick reading is

required.Conclusion and advice on designing the web page:1. Outline could be designed as a moving bar, so that people could see it anywhere.2. Irrelevant information such as advertisement at normal level won’t bother people and interfere the process of reading.