Eindhoven University of Technology MASTER Personalized E … · Vaessen, D. Award date: 2009 Link...
Transcript of Eindhoven University of Technology MASTER Personalized E … · Vaessen, D. Award date: 2009 Link...
Eindhoven University of Technology
MASTER
Personalized E-learningshorten the length of an E-learning program
Vaessen, D.
Award date:2009
Link to publication
DisclaimerThis document contains a student thesis (bachelor's or master's), as authored by a student at Eindhoven University of Technology. Studenttheses are made available in the TU/e repository upon obtaining the required degree. The grade received is not published on the documentas presented in the repository. The required complexity or quality of research of student theses may vary by program, and the requiredminimum study period may vary in duration.
General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.
• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain
Personalized E-Learning 1
EINDHOVEN UNIVERSITY OF TECHNOLOGY
Department of Mathematics and Computer Science
Personalized E-Learning
by
Dirk Vaessen
Shorten the length of an E-learning program
Supervisors:
Prof. dr. P.M.E. De Bra (Tu/e)
Drs. F.H.A. van Buul CISSP (InfoSecure)
Eindhoven, January 2009
Personalized E-Learning 2
Personalized E-Learning 3
Personalized E-Learning 4
Personalized E-Learning 5
Abstract
This graduation project provides a conceptual model for the design of a personalized E-Learning
module in AHA! (Adaptive Hypermedia Architecture). AHA! is an adaptive web-based system for
creating adaptive websites. Before explaining how to create the personalized E-Learning module, the
differences between AHA! and other technologies are explained and what the advantages of AHA! in
comparison with the other technologies are. There are different types and methods for adaptation.
These are described in this thesis and the most suitable adaptation method for a personalized E-
learning module is further analyzed before the implementation is explained. A fully described
analysis of the time benefits for each adaptation method is given. For the company InfoSecure that
designs E-Learning modules an existing module is adapted in a personalized module with the help of
AHA! and the differences between those two products are explained. The results of the adaptive and
the original module are analyzed. Not only the results, but also how to analyze the results is fully
described in this thesis.
Personalized E-Learning 6
Personalized E-Learning 7
Foreword
After finishing my bachelor on Computer Science and Engineering at the Eindhoven University of
Technology, I decided to continue with the Business Information Systems (BIS) master also at the
department of Mathematics and Computer Science. The emphasis on the relation between computer
sciences and industrial engineering and management sciences in this master appealed to me.
Creating the AHA! application itself connects greatly with my bachelor education. Analyzing what
adaptation to use within the created application, so that it best fits InfoSecure’s business goals is an
aspect where my master education comes into play. Analyzing the results and proposing a suitable
plan to use these results in such a way that InfoSecure can create even better E-learning programs in
the future is also one of the subjects where my master education was very helpful.
I hope that this thesis will be a contribution for InfoSecure and their E-Learning modules and a
contribution in their awareness of new methodologies and techniques that can be used to create E-
learning modules. I also hope that the reader will be challenged to read this report and will
understand the decisions made in the different steps.
Before I wish you good luck with reading this thesis, as a tradition, I like to thank some people who
made it possible to graduate. First of all I like to thank my supervisors, Drs. Frans van Buul and prof.
dr. Paul De Bra for their support, feedback, and suggestions during the project. I also want to thank
the director of InfoSecure, Melle Beverwijk, for giving me the chance to work at his company in
Leusden, which was a great experience for me.
At last I would like to thank all colleagues, both InfoSecure employees and fellow students for their
feedback and their time.
January 2009,
Dirk Vaessen
Personalized E-Learning 8
Personalized E-Learning 9
Contents and Terms
Summarized Contents List
Abstract ................................................................................................................................................... 5
Foreword ................................................................................................................................................. 7
Contents and Terms ................................................................................................................................ 9
1 Introduction ................................................................................................................................... 15
2 E-learning ....................................................................................................................................... 17
3 Technologies .................................................................................................................................. 20
4 Adaptation ..................................................................................................................................... 28
5 Introduction to Awareness Module .............................................................................................. 54
6 Module in AHA! ............................................................................................................................. 64
7 Extracting Test Data ...................................................................................................................... 82
8 Testing Adaptivity .......................................................................................................................... 88
9 Conclusion ..................................................................................................................................... 93
10 References ................................................................................................................................. 94
Appendices ............................................................................................................................................ 95
Extended Contents List
Abstract ................................................................................................................................................... 5
Foreword ................................................................................................................................................. 7
Contents and Terms ................................................................................................................................ 9
Summarized Contents List ................................................................................................................... 9
Extended Contents List ........................................................................................................................ 9
List of Figures and Diagrams.............................................................................................................. 13
Terms and Abbreviations .................................................................................................................. 14
1 Introduction ................................................................................................................................... 15
1.1 General Introduction ............................................................................................................. 15
1.2 Assignment Goals .................................................................................................................. 16
1.2.1 Goals for Adaptive E-Learning Module ......................................................................... 16
1.2.2 Goals for Final Thesis ..................................................................................................... 16
2 E-learning ....................................................................................................................................... 17
2.1 E-learning Programs InfoSecure ............................................................................................ 17
2.1.1 Introduction Program .................................................................................................... 18
2.1.2 Follow-up Program ........................................................................................................ 18
2.1.3 Special Topics Learning Programs ................................................................................. 18
2.1.4 Training for IT Professionals .......................................................................................... 18
2.2 Adaptation InfoSecure Program ............................................................................................ 18
3 Technologies .................................................................................................................................. 20
Personalized E-Learning 10
3.1 AHA!....................................................................................................................................... 20
3.1.1 AHA! Architecture ......................................................................................................... 21
3.2 SCORM ................................................................................................................................... 21
3.2.1 Organization of SCORM ................................................................................................. 22
3.3 SCORM to AHA! ..................................................................................................................... 23
3.4 Improve SCORM Code ........................................................................................................... 24
3.4.1 Sequencing and Navigation ........................................................................................... 25
3.4.2 Selftest Implementation ................................................................................................ 26
4 Adaptation ..................................................................................................................................... 28
4.1 Adaptation Types................................................................................................................... 28
4.1.1 Content Adaptation ....................................................................................................... 28
4.1.2 Link Adaptation ............................................................................................................. 28
4.1.3 Presentation Adaptation ............................................................................................... 29
4.1.4 Information Adaptation ................................................................................................. 29
4.2 Adaptation Rules ................................................................................................................... 29
4.3 Adaptation Methods ............................................................................................................. 30
4.3.1 Adaptation by Pretest Questions .................................................................................. 30
4.3.2 Adaptation regarding to HR ........................................................................................... 30
4.3.2.1 Diplomas and Certificates .......................................................................................... 30
4.3.2.2 Function ..................................................................................................................... 31
4.3.2.3 Department ............................................................................................................... 31
4.4 Analysis .................................................................................................................................. 31
4.4.1 Example Course ............................................................................................................. 31
4.4.2 Pretest Analysis ............................................................................................................. 32
4.4.2.1 Success Percentage ................................................................................................... 32
4.4.2.2 Success Percentage together with Pretest Correlation Percentage ......................... 33
4.4.2.3 Selftest Adaptation .................................................................................................... 37
4.4.3 HR Analysis .................................................................................................................... 37
4.4.3.1 Data Mining ............................................................................................................... 38
4.4.3.2 Breakeven Percentage ............................................................................................... 40
4.5 Scenarios and Time Benefits ................................................................................................. 43
4.5.1 Scenario’s ...................................................................................................................... 43
Scenario 1: Best Case ................................................................................................................. 43
Scenario 2: Best Case 2 .............................................................................................................. 44
Scenario 3: Average Case .......................................................................................................... 46
Scenario 4: Worst Case .............................................................................................................. 49
Scenario 5: Worst Case 2 ........................................................................................................... 51
4.5.2 Time Benefits ................................................................................................................. 51
Personalized E-Learning 11
4.6 Conclusion ............................................................................................................................. 53
5 Introduction to Awareness Module .............................................................................................. 54
5.1 Module Introduction ............................................................................................................. 54
5.2 Module Duration ................................................................................................................... 54
5.3 Time Distribution ................................................................................................................... 55
5.4 Module Construction ............................................................................................................. 55
5.5 Module Adaptation Locations ............................................................................................... 57
5.5.1 Explanation .................................................................................................................... 57
5.5.2 What is Information Security......................................................................................... 57
5.5.3 Status within the Company ........................................................................................... 57
5.5.4 About the Golden Rules ................................................................................................ 57
5.5.5 Selftest ........................................................................................................................... 58
5.5.6 Conclusion ..................................................................................................................... 58
5.5.7 Relevant Links and Contacts .......................................................................................... 58
5.6 Module Adaptation Techniques ............................................................................................ 58
5.6.1 Student Profile ............................................................................................................... 58
5.6.2 Adaptation based on HR Information ........................................................................... 58
5.6.2.1 Accepted Diplomas and Certificates ......................................................................... 59
5.6.3 Adaptation based on Pretest ......................................................................................... 59
5.6.4 Adaptation Results ........................................................................................................ 61
5.6.5 Scenario’s ...................................................................................................................... 62
5.6.6 Time Benefits ................................................................................................................. 63
5.6.7 Conclusion ..................................................................................................................... 63
6 Module in AHA! ............................................................................................................................. 64
6.1 Process of the Adaptive Module ........................................................................................... 64
6.2 Conceptual Structure............................................................................................................. 66
6.2.1 Design Concept Structure .............................................................................................. 66
6.2.2 Creating Concepts ......................................................................................................... 68
6.2.2.1 Menu Structure ......................................................................................................... 69
6.2.2.2 Attributes ................................................................................................................... 69
6.2.2.3 Example .aha file ....................................................................................................... 69
6.2.3 Concept Relationships ................................................................................................... 70
6.2.3.1 Used Attributes.......................................................................................................... 70
6.2.3.2 Adaptation Rules ....................................................................................................... 71
6.2.4 Implementing Concept Structure .................................................................................. 71
6.2.4.1 Hierarchy and Suitability ........................................................................................... 71
6.2.4.2 Access ........................................................................................................................ 74
6.3 Write Pages ........................................................................................................................... 75
Personalized E-Learning 12
6.3.1 Write Standard Pages .................................................................................................... 75
6.3.2 Write Adapting Pages .................................................................................................... 76
6.3.2.1 Pretest Questions ...................................................................................................... 76
6.3.2.2 About the Golden Rules ............................................................................................ 77
6.3.2.3 Selftest ....................................................................................................................... 78
6.4 Look and Feel ......................................................................................................................... 78
6.5 Authoring Tools ..................................................................................................................... 80
6.5.1 Graph Author ................................................................................................................. 80
6.5.2 Concept Editor ............................................................................................................... 81
6.5.3 Form Editor .................................................................................................................... 81
6.5.4 Test Editor ..................................................................................................................... 81
6.6 Other Methods ...................................................................................................................... 81
7 Extracting Test Data ...................................................................................................................... 82
7.1 Analyzing the AHA! Logs ........................................................................................................ 82
7.1.1 Correcting Data .............................................................................................................. 83
7.1.2 Data Group 1 vs. Data Group 2 ...................................................................................... 83
7.1.3 Pretest Ratio .................................................................................................................. 86
7.2 Analyzing the AHA! Profile Logs ............................................................................................ 87
7.3 Testing the Significance ......................................................................................................... 87
7.3.1 F-test Two-Sample for Variances ................................................................................... 87
7.3.2 T-test Two-Sample ......................................................................................................... 87
8 Testing Adaptivity .......................................................................................................................... 88
8.1 Test Group ............................................................................................................................. 88
8.2 Test Group Analysis ............................................................................................................... 88
8.3 Pretest Results ....................................................................................................................... 89
8.3.1.1 Persons that succeeded for selftest first time........................................................... 89
8.3.1.2 Persons that didn’t succeed for selftest first time .................................................... 90
8.3.1.3 Success and Pre-test Question Correctness Percentage ........................................... 90
8.3.1.4 Selftest Adjustments ................................................................................................. 90
8.3.2 Statistical Proof of Results ............................................................................................. 90
8.3.3 Overall Conclusion ......................................................................................................... 91
8.4 HR Results .............................................................................................................................. 92
9 Conclusion ..................................................................................................................................... 93
10 References ................................................................................................................................. 94
Appendices ............................................................................................................................................ 95
Appendix A: Selftest.xhtml ................................................................................................................ 95
Personalized E-Learning 13
List of Figures and Diagrams
Table 1 Terms and Abbreviations .......................................................................................................... 14
Table 2 Fraction current imsmanifest.xml............................................................................................. 24
Table 3 Implement Sequencing and Navigation.................................................................................... 26
Table 4 Selftest implementation ........................................................................................................... 26
Table 5 Pretest maximum percentages ................................................................................................. 36
Table 6 Minimum selftest succeed percentage .................................................................................... 37
Table 7 Pretest scorings percentage ..................................................................................................... 38
Table 8 Pretest scorings percentage with certain diploma ................................................................... 38
Table 9 Pretest scorings percentage with certain certificate ................................................................ 39
Table 10 Pretest scorings percentage with certain function ................................................................ 39
Table 11 Updated diploma scorings percentage ................................................................................... 39
Table 12 Scorings percentage with diploma and certification taken into account ............................... 40
Table 13 Optimized scorings percentages ............................................................................................. 40
Table 14 Average duration pretest adapted course scenario 2 ............................................................ 45
Table 15 Average duration pretest adapted course scenario 3 ............................................................ 47
Table 16 HR adaptation part 1 .............................................................................................................. 48
Table 17 HR adaptation part 2 .............................................................................................................. 49
Table 18 Average duration pretest adapted course scenario 4 ............................................................ 50
Table 19 Average duration pretest adapted course scenario 5 ............................................................ 51
Table 20 Time benefits per scenario ..................................................................................................... 51
Table 21 Individual time benefits .......................................................................................................... 52
Table 22 Duration Introduction Module ............................................................................................... 55
Table 23 Accepted Diplomas ................................................................................................................. 59
Table 24 Accepted Certificates .............................................................................................................. 59
Table 25 Time Benefits Scenario's ......................................................................................................... 63
Table 26 Initializing table for current module ....................................................................................... 65
Table 27 Fraction of question1.xhtml ................................................................................................... 77
Table 28 Example access_John Doe.xml ............................................................................................... 82
Table 29 Table with extra time column ................................................................................................. 83
Table 30 Timings from all test persons ................................................................................................. 84
Table 31 Timings from correct test persons .......................................................................................... 84
Table 32 Average time pretest .............................................................................................................. 86
Table 33 Average time golden rules ...................................................................................................... 86
Table 34 Average time selftest .............................................................................................................. 87
Figure 1 AHA! Architecture (De Bra, et al., 2003) ................................................................................. 21
Figure 2 SCORM Bookshelf (Advanced Distributed Learning (ADL), 2006a) ......................................... 23
Figure 3 Percentages that lead to time profit for subject 3 .................................................................. 35
Figure 4 Time Distribution Introduction Module .................................................................................. 55
Figure 5 Example Basic Module (Information) Security ........................................................................ 56
Figure 6 Example aha file ...................................................................................................................... 70
Figure 7 Frame Structure ....................................................................................................................... 79
Figure 8 Screenshot Module AHA! ........................................................................................................ 80
Diagram 1 E-learning Programs of InfoSecure ...................................................................................... 17
Diagram 2 Non adapted example course .............................................................................................. 32
Diagram 3 Pretest adapted example course ......................................................................................... 32
Diagram 4 Process Pretest Question 3 .................................................................................................. 34
Diagram 5 Scenario 1 pretest adapted course ...................................................................................... 43
Diagram 6 Scenario 1 HR adapted course ............................................................................................. 44
Personalized E-Learning 14
Diagram 7 Scenario 2b possible pretest adapted course ...................................................................... 44
Diagram 8 Scenario 2b possible HR adapted course ............................................................................. 44
Diagram 9 Best case scenario 3 HR adapted ......................................................................................... 47
Diagram 10 Page sequence ................................................................................................................... 56
Diagram 11 Example adaptive page sequence ...................................................................................... 60
Diagram 12 Example 2 adaptive page sequence ................................................................................... 60
Diagram 13 Adaptation Process ............................................................................................................ 62
Diagram 14 Adaptive process generic module ...................................................................................... 65
Terms and Abbreviations
The following terms and abbreviations are used in this thesis.
Table 1 Terms and Abbreviations
Term Description
ADL Advanced Distributed Learning
Developer and implementer of learning technologies across the Department of
Defense (DoD).
AHA! Adaptive Hypermedia Architecture
BIS Business Information Systems
Master program at the Eindhoven University of Technology
HR Human Resources
LMS Learning management system
SCORM Sharable Content Object Reference Model
SQL Common querying language to query results from a relation database and to
update information in this database.
TEL Technology Enhanced Learning
Tu/e Eindhoven University of Technology
XML eXtensible Mark-up Language
Mark-up language commonly used on the Internet to exchange
information that can be irregular.
Personalized E-Learning 15
1 Introduction
In this chapter a general introduction (see chapter 1.1) to the company InfoSecure and their E-
learning modules is given, together with the assignment goals of this final thesis (see chapter 1.2).
1.1 General Introduction
This thesis is the result of the graduation period of Dirk Vaessen, carried out at InfoSecure in Leusden
as a completion of the “Business Information Systems” master at the Eindhoven University of
Technology.
InfoSecure was founded in 1999. Since
then InfoSecure has built up a
worldwide clientele.
In addition to the head office in The
Netherlands, InfoSecure has offices in
Belgium, Germany, Great Britain and
Scandinavia. Additionally they also have
partners in Switzerland, Croatia, China,
Japan and Canada.
The company provides solutions in more than 80 countries and in 22 different languages. The unique
method used has proven itself in practice and is an international best seller.
InfoSecure Group is highly experienced in the field of awareness training programs, knowledge
testing and risk/compliance review.
Within the field of Awareness & Training
InfoSecure has multiple E-learning
modules described in chapter 2.1. The
module that will be adapted is the
Introduction Program, in this program
the users are made aware of the
potential risks that the organization can
face and the measures to be established
in respect to their activities.
Personalized E-Learning 16
1.2 Assignment Goals
Before explaining everything in detail the assignment goals of this final thesis are given. The goals are
given in two subchapters. First the goals of the adaptive module in chapter 1.2.1., afterwards the
goals for this final thesis in general in chapter 1.2.2.
1.2.1 Goals for Adaptive E-Learning Module
The most important goal for the Adaptive E-Learning module is to create a better learning
experience, with the same high standard of the original E-Learning module, in a shorter time period.
Such a module has a few characteristics:
Time benefit
This is most important for the new module because eventually this module hits the market and as
you all know: time is money. If this module saves for instances 10 minutes in comparison with the old
module and has the same high standard, this will be a great improvement. A company with 60.000
employees that work for an average of 50 euro an hour will save half a million euro with the help of
this new adaptive module.
Creating a time benefit without endangering the high standard of the original E-learning, will be done
with the help of adaptation. This adaptation will make sure that the module is personalized.
Personalization
The personalization is done with the help of adaptation. The students that follow the new adaptive
course will only be presented the information that is suitable for them. F.i. information they are
already familiar to will be skipped and only information that is important for the specific student will
be presented. This way the duration of the module is shortened.
Better learning experience
A better learning experience is automatically created for the student, because it will take him less
time to learn the same. No familiar information to the student is presented, but only the information
that is suitable. This creates a much better learning experience for the student.
1.2.2 Goals for Final Thesis
To create such an adaptive module that suits all the aspects described in the previous subchapter is
the main goal. How it is created and which technologies are used is fully described in this thesis.
Another goal is to explain how this adaptive module is build with the help of AHA! Also clearly explain
how to use AHA! on a conceptual level for another adaptive module is another important goal in this
thesis. Analyzing the results of the AHA! adaptive module, and explain how, is also a goal in this
thesis.
Personalized E-Learning
2 E-learning
Before explaining the E-learning modules of InfoSecure, E
Nowadays everybody talks about Technology Enhanced Learning (TEL).
technology enhanced learning spread very broad an
nature of this evolving research field. Hence, the definition of TEL must be as broad and general as
possible in order to capture all aspects:
technical innovations (also improving efficiency and cost effectiveness) for learning practices,
regarding individuals and organizations, independent of time, place and pace. The field of TEL
therefore describes the support of any learning activity through technology
elearning or eLearning) is the delivery of a learning, training or education program by electronic
means. E-learning involves the use of a computer or electronic device (e.g. a mobile phone) in some
way to provide training, educatio
Learning, the complete education program is on the computer (or other electronic device).
computer is only used for specific parts of a program it
Because nowadays every employee has a computer at work or at home,
more and more popular. InfoSecure develops E
computer. No other electronic dev
programs of InfoSecure are explained in the next subchapter.
2.1 E-learning Programs InfoSecure
As described in the introduction InfoSecure
Besides e-learning programs InfoSecure
consultancy, risk assessment etc.
InfoSecure has multiple E-learning
subjects described in the next diagram.
Diagram
learning modules of InfoSecure, E-learning itself needs to be exp
talks about Technology Enhanced Learning (TEL). The existing definitions for
technology enhanced learning spread very broad and change continuously due to the dynamic
nature of this evolving research field. Hence, the definition of TEL must be as broad and general as
possible in order to capture all aspects: Technology enhanced learning has the goal to provide socio
ovations (also improving efficiency and cost effectiveness) for learning practices,
regarding individuals and organizations, independent of time, place and pace. The field of TEL
therefore describes the support of any learning activity through technology. E-
he delivery of a learning, training or education program by electronic
learning involves the use of a computer or electronic device (e.g. a mobile phone) in some
way to provide training, educational or learning material (Der08). E-Learning is a part of TEL. With E
Learning, the complete education program is on the computer (or other electronic device).
used for specific parts of a program it is still called TEL, but not E
Because nowadays every employee has a computer at work or at home, TEL and e
more and more popular. InfoSecure develops E-learning programs that involve the use of a
computer. No other electronic devices are possible or necessary for following the
of InfoSecure are explained in the next subchapter.
InfoSecure
As described in the introduction InfoSecure plays a big role in the information security sector.
InfoSecure offers lots of solutions for your company, like workshops,
consultancy, risk assessment etc. During my graduation I will only focus on the E
learning programs for different end-users, that implement all kind of
subjects described in the next diagram.
Diagram 1 E-learning Programs of InfoSecure
17
learning itself needs to be explained.
The existing definitions for
d change continuously due to the dynamic
nature of this evolving research field. Hence, the definition of TEL must be as broad and general as
Technology enhanced learning has the goal to provide socio-
ovations (also improving efficiency and cost effectiveness) for learning practices,
regarding individuals and organizations, independent of time, place and pace. The field of TEL
-learning (also called
he delivery of a learning, training or education program by electronic
learning involves the use of a computer or electronic device (e.g. a mobile phone) in some
a part of TEL. With E-
Learning, the complete education program is on the computer (or other electronic device). When the
is still called TEL, but not E-Learning.
TEL and e-learning become
learning programs that involve the use of a
ices are possible or necessary for following the programs. The
in the information security sector.
offers lots of solutions for your company, like workshops,
on the E-learning programs.
users, that implement all kind of
Personalized E-Learning 18
2.1.1 Introduction Program
The InfoSecure Awareness concept is based on the Modular Training approach to different target
groups. It starts with an Introduction program for every end-user, as workshops and/or as e-learning.
The solution is used in combination with 6-7 business film clips, learning material and selftests, all
corresponding with Client’s selected Golden Rules/Information Security best practices. In this report
the introduction program made for the Dutch company KPN is used as the subject for adaptation.
KPN selected 9 golden rules, but more about this specific module is in Chapter 5.
Dedicated programs can be built for the target group Management (normally shorter program) and
for example programs for Mobile workers, IT-Professionals and persons handling sensitive
information.
2.1.2 Follow-up Program
In subsequent years a follow-up program can be implemented for every end-user, as workshops
and/or as e-learning. Also here on the Modular Training approach to different target groups is used.
The solution is an extension to the introduction program. Mostly summaries are used from the
Special Topic Learning modules.
Also this program is used in combination with 6-7 business film clips, learning material and selftests,
all corresponding with Client’s selected Golden Rules/Information Security best practices.
2.1.3 Special Topics Learning Programs
Special Topics are developed to learn more in depth about special topic subjects. These programs can
be used to give additional training to those target groups who need more instruction in the subject.
For example mobile working need more instruction about the topics “Loss of Laptops & PDAs” and
“Mobile working”. The target group handling sensitive information can for example be trained more
in “Data Classification”, “Privacy” and “Working with 3rd parties”.
Also this program is used in combination with 3-4 business film clips, learning material and selftests,
all corresponding with Client’s selected Golden Rules/Information Security best practices.
2.1.4 Training for IT Professionals
InfoSecure has developed a training / awareness learning program for information security for IT
Professionals. The training in e-Learning concept content consists of modules on 2 levels of
education; 5 modules for basic training (each 15 minutes) and 6 modules for advanced training (each
30-45 minutes).
Modules are available for Security Essentials, Security Management, Critical Business Applications,
Computer Installations, Networks and Systems Development.
The information in the program is based on ISO standards, other best practices and the in public
domain available ISF Standard of Good Practice v.2005.
2.2 Adaptation InfoSecure Program
As described above InfoSecure has lots of E-learning programs that are all especially made for clients.
In the assignment goals is described that shortening the time duration is one of the main goals of
adaptation. Therefore the program “Training for IT Professionals” is most suitable for adaptation.
Personalized E-Learning 19
This module is by far the most time consuming program and therefore a lot of time profit can be
gained. This is a very technical program, which is very suitable for adaptation.
After an analysis by InfoSecure of his own market group, it was concluded that the clients that
bought this program were not (yet) interested in an adapted version. The clients that bought (or are
going to buy) the introduction program were very enthusiast about an adapted version, and
therefore the switch from “Training for IT Professionals” to “Introduction Program” for commercial
reasons was made.
This program takes far less time than the previous program, but it is expected that still a time benefit
can be gained, albeit a smaller one. The exact module is named “Basic module (information)
Security” for the company KPN. How this module is adapted is described in detail in chapter 5.
Personalized E-Learning 20
3 Technologies
E-Learning becomes more and more popular by the years, but there is still no standard format for E-
learning content. The adaptive E-learning module created in this thesis is done with the help of AHA!
(see chapter 3.1). The E-learning modules of InfoSecure are SCORM compatible. This is a common
and widely used standard for E-learning (see chapter 3.2). How to use SCORM content in AHA! is
explained in chapter 3.3. In chapter 3.4 some improvement points for the current SCORM code of
InfoSecure are given.
3.1 AHA!
AHA! is developed at the Eindhoven University of Technology. After some initial experimental
versions AHA! was released as version 1.0 in 2000. AHA! excelled in the area of simplicity. AHA! has
since evolved into a much more powerful system (version 2.0, 3.0 and soon 4.0), but new versions
maintain that basic simplicity.
• A user model based on concepts: Each time you visit a page in an AHA! application the name
of the page is passed to the adaptation engine which updates the user model. A user model
consists of concepts that have attributes. A typical example of an AHA! action is that visiting
a page may increase a knowledge attribute for (the concept corresponding to) that page. This
knowledge update may propagate to the knowledge attribute of other concepts, perhaps
corresponding to a section or chapter of a textbook. In AHA! a concept can have arbitrarily
many attributes of types Boolean, integer or string.
• Adaptive link hiding or link annotation: The suitability of link destinations (pages) is
determined by an author-defined requirement. This is a (Boolean) expression using arbitrary
user model values. The requirements can express the common prerequisite relationships
between concepts but can be used for any other condition that can be expressed through
such a Boolean expression. When a page is generated, links marked as conditional (using the
link class “conditional”) are displayed differently depending on the suitability of the link
destination. If the expression is true the link is shown in blue (unvisited) or purple (visited),
and when the expression is false the link is shown in black, and not underlined. This results in
hiding the unsuitable or undesired links. The color scheme can also be altered by the end-
user to make all links visible, in different colors.
• Conditional inclusion of fragments: Like for the links to pages the author can also associate a
requirement with fragments in a page. This is done through an <if> tag, with one or two
fragments, enclosed by a <block> tag. If the expression is true the first fragment is shown to
the user, otherwise the second (optional) fragment is shown. This can be used to include
prerequisite explanations, or any other piece of content. Fragments can be external objects,
represented through the <object> tag. Such objects can themselves also be associated with
concepts and accessing them triggers user model updates just like for page accesses.
AHA! is delivered as open source software, implemented entirely in Java, and works with the Java-
based webserver Tomcat and with Java servlets. You need recent versions of Tomcat and of the Java
SDK to make AHA! work. On the browser side you should use recent versions (of for instance Mozilla
Firefox or Microsoft Internet Explorer) to ensure full support of (X)HTML and HTTP.
Personalized E-Learning 21
3.1.1 AHA! Architecture
Figure 1 AHA! Architecture (De Bra, et al., 2003)
The overall architecture of AHA! (see Figure 1) shows that AHA! consists of java servlets that serve
pages from external WWW servers or from the local file system. These servlets interact with the
DM/AM and the UM. A request to a page triggers adaptation rules that perform UM updates. When
UM is updated the requested page is parsed to perform the conditional inclusion of fragments. That
inclusion is based on the new state of UM.
AHA! stores DM/AM and UM (of all users) either as XML files or in a mySQL database. The choice
between these two is made by the Manager who configures AHA!.
AHA! applications mainly consist of a set of concepts, some of which are linked to pages or objects
(or fragments). Concepts can be used to represent topics of the application domain, e.g. subjects to
be studied in a course. In AHA! the author of an application can associate any number of (named)
attributes with a concept. Some attributes have a meaning for the system, like access (a Boolean
attribute that temporarily becomes true when a page is accessed), some have meaning for the
author (and user), like knowledge or interest, and some have meaning for both, like visited
(determining the link color). Since AHA! uses an overlay user model, all attributes of concepts in
DM/AM also appear in UM.
The adaptation rules define how the user model is updated. When the user accesses a page (or an
object included in a page) the rules associated with the access attribute are triggered.
More detailed information about AHA! and how to build an adaptive module in AHA! is given in
chapter 6.
3.2 SCORM
Sharable Content Object Reference Model (SCORM) is a specification of the Advanced Distributed
Learning (ADL) Initiative, which comes out of the Office of the United States Secretary of Defense.
SCORM is a collection of standards and specifications for web-based e-learning. It defines
communications between client side content and a host system called the run-time environment
(commonly a function of a learning management system (LMS)). SCORM also defines how content
may be packaged into a transferable ZIP file.
The essence of SCORM is that any content that conforms to the SCORM specifications will work with
any SCORM conformant LMS. SCORM operates behind the scenes to make things compatible.
Basically SCORM governs two things: packaging content and exchanging data at runtime.
Personalized E-Learning 22
Packaging content determines how a piece of content should be delivered in a physical sense. At the
core of SCORM packaging is a document titled the "imsmanifest.xml". This file contains every piece of
information required by the LMS to import and launch content without human intervention. This
manifest file contains XML that describes the structure of a course both from a learner’s perspective
and from a physical file system perspective. Questions like, "Which document should be launched?"
and "What is the name of this content?" are answered by this document (Advanced Distributed
Learning (ADL), 2006).
Runtime communication, or data exchange, specifies how the content ”talks” to the LMS while the
content is actually playing. This is the part of the equation described as delivery and tracking. There
are two major components to this communication. First, the content has to "find" the LMS. Once
the content has found it, it can then communicate through a series of "get" and "set" calls and an
associated vocabulary. Conceptually, these are things like "request the learner’s name" and "tell the
LMS that the learner scored 95% on this test." Based on the available SCORM vocabulary, many rich
interactive experiences can be communicated to the LMS (Advanced Distributed Learning (ADL),
2006b).
3.2.1 Organization of SCORM
SCORM is a collection, integration and harmonization of specifications and standards that have been
bundled into a collection of “technical books.” Nearly all of the specifications and guidelines are
taken from other organizations. These technical books are presently grouped under three main
topics: the “Run-time Environment (RTE)”, the “Content Aggregation Model (CAM)”, and
“Sequencing and Navigation (SN).”
Of the many organizations working on specifications related to e-learning, there are four in particular
that are key to SCORM. ADL encourages active participation in one or more of these organizations in
support of future specification development.
• Alliance of Remote Instructional Authoring & Distribution Networks for Europe (ARIADNE)
(http://www.ariadne-eu.org/)
• Aviation Industry CBT Committee (AICC) (http://www.aicc.org/)
• Institute of Electrical and Electronics Engineers (IEEE) Learning Technology Standards
Committee (LTSC) (http://ieeeltsc.org/)
• IMS Global Learning Consortium, Inc. (http://www.imsglobal.org/).
Personalized E-Learning 23
Figure 2 SCORM Bookshelf (Advanced Distributed Learning (ADL), 2006a)
The Run-Time Environment specifies how content should behave once it has been launched by the
LMS. The Content Aggregation Model specifies how you should package your content so that it can
be imported into an LMS. This involves creating XML files that an LMS can read and learn everything
it needs to know about your content. SCORM also describes a “Sequencing and Navigation” model
for the dynamic presentation of content based on learner needs. While these various SCORM
books summarized in Figure 2 can stand-alone, there are areas of overlap. For instance, while the
RTE book focuses primarily on communication between content and LMSs, it frequently refers to
the different types of content objects conducting that communication: Sharable Content Objects
(SCOs). More details about SCOs are found in the CAM book. Similarly, while the SN book covers
the details of SCORM sequencing and navigation, the RTE book deals with content delivery and
gives high-level information on how an LMS determines which piece of content to deliver at any
given time.
3.3 SCORM to AHA!
The department of Computer Science at the University of Cordoba developed an Upload SCORM tool
for AHA! (Cristóbal Romero Morales, 2005). With the help of this program, which is still in a beta
phase, the SCORM zip file can be uploaded and the result is a complete new course in AHA!
consisting of the uploaded SCORM data. All you need to enter is the course name, which will already
be the AHA! Web-tree directory where course content is going to be extracted. Unfortunately this
upload program was of no use for the specific SCORM modules of InfoSecure, because this code was
not the exact SCORM code the upload program had expected.
As described in the previous subchapter the SCORM content consists of a ZIP-file, which contains a
descriptor file (imsmanifest.xml) where content organization and resources are described and
referenced. AHA! contents are organized in a root directory (only one in a course), but this directory
only contains the course resources. Usually, this directory already contains course configuration files,
introduction and registration files, but the information about organization and relationship between
different concepts are managed by the AHA! system itself. An AHA! course has a “Concept List”
which links each concept to a resource, and AHA! concepts are organized as a hierarchy. Because of
these similarities between SCORM content and AHA! the upload program creates a “Concept List”
using the imsmanifest.xml file as a source.
The problem with the InfoSecure SCORM code is that it is not really the ideal SCORM code. SCORM is
a collection of standards and specifications for web-based e-learning. Even if the SCORM code applies
to these standards and specifications, the code can still be very confusing. The current SCORM code
Personalized E-Learning 24
consists of a ZIP-file which contains the mandatory imsmanifest file with most of the content
organization and resource information. But this file is not used fully to the abilities of SCORM. A
small fraction of this file is given in the next table.
<manifest>
…
<organizations default="FT02240">
<organization identifier="FT02240">
<title>Computer Installations Basic</title>
<item identifier='ItemFT02240' isvisible='true'>
<title>Computer Installations Basic</title>
<item identifier='ItemFT02241' isvisible='true'>
<title>Introduction</title>
<item identifier='ItemFT02242' isvisible='true' identifierref='SCOFT02242'>
<title>Introduction</title>
</item>
</item>
…
</item>
</organization>
</organizations>
<resources>
<resource identifier="SCOFT02242" type="webcontent"
href="SCOs/F10_RAL_Wrapper.htm?LaunchURL=FT02242.htm%26WindowDims=height=712,width=1014"
adlcp:scormtype="sco">
<metadata>
<schema>ADL SCORM</schema>
<schemaversion>1.2</schemaversion>
<adlcp:location>FT02242.xml</adlcp:location>
</metadata>
</resource>
…
</resources>
</manifest>
Table 2 Fraction current imsmanifest.xml
Without going in too much detail and complete explanation of this code. It is cleaner SCORM code if
only one unique item with one title exist in the organization, which is linked (via the identifierref) to a
resource. But the problem with creating the concept list with the help of the upload program is the
href of the resource. Instead of a single htm(l) file with proper html (without JavaScript) code, it is
linked to “SCOs/F10_RAL_Wrapper.htm?LaunchURL=FT02242.htm%26WindowDims=height=712,width=1014”
and this will not work in the concept list in AHA!.
What happens in the current code is that the htm file (F10_RAL_Wrapper.htm) has to launch another
htm file (FT02242.htm) with the help of JavaScript. This file launches again with a bunch of JavaScript
code all the subpages and sequence order for this subject. All the subpages are in htm, but instead of
proper htm code it is all javascript with htm code added in. If these pages where changed in proper
html pages and the sequence and navigation of the pages is done with the SCORM standard as will
be described in the next subchapter, the upload program would be able to create the AHA! program
automatically. Because the upload program was of no use with this code, the E-learning modules
were all manually converted to AHA! as described in chapter 6. To create the adaptive E-learning
modules manually, instead of changing the SCORM code and use the upload program has two
reasons. The first reason is a better understanding of AHA!, while the program is completely build
from scratch. The second reason is the upload program itself. This program is still in a beta phase,
and even with the improved SCORM code it is not guaranteed the conversion to AHA! will go
flawlessly.
3.4 Improve SCORM Code
The current code of the InfoSecure modules does not use all the SCORM possibilities fully, which can
be very helpful. As explained in the previous subchapter the actual pages have the extension .htm,
Personalized E-Learning 25
but are nothing more than a bunch of JavaScripts with the html code inside the JavaScript. This
should not be the case to keep a better overview. Some other improvement points are given in the
next subchapters.
3.4.1 Sequencing and Navigation
First the terms sequencing and navigation are explained and afterwards an impression of the desired
implementation is given.
Sequencing
In summary, SCORM Sequencing depends on: a defined structure of learning activities, the Activity
Tree; a defined sequencing strategy, the Sequencing Definition Model; and the application of defined
behavior to external and system triggered events, SCORM Sequencing Behaviors.
By default, if no sequencing and navigation prescription is defined, a learner may choose any content
item at will. Adding specific prescriptions can alter this default behavior. For example, adding a flow
prescription to the items in the content organization will direct the LMS to guide the navigation in
the order defined by the organization tree. More complex adaptive sequencing can be based on the
completion status of certain learning resources or on more complex computation of user preferences
or assessment results(Advanced Distributed Learning (ADL), 2006c).
Past versions of SCORM provided no specific sequencing capabilities, effectively allowing only pure
free play, because it is a difficult and complex subject that required more time to come up with
workable solutions. There are many, and often divergent, requirements in the learning design
community. No approach has been found to solve all possible use cases. However, the approach used
in SCORM, which is based on the IMS SS Specification [5], is flexible enough to allow a wide variety of
learning and instructional design approaches.
Navigation/Presentation
Navigation controls are user interface devices that provide the means for a learner to indicate
the desire to navigate away from the Current Activity in a particular manner. SCORM requires
that an LMS provides, at a minimum, navigation controls that trigger Continue, Previous, and
Choice navigation events, when the processing of those events will result in content identified
for delivery to the learner. In addition, SCORM requires that an LMS not provide navigation
controls that trigger Continue, Previous, and Choice navigation events, when the processing of
those events will result in a Sequencing pseudo-code exception – providing the controls would
enable the learner to trigger navigation events that could disrupt the learner experience. SCORM
does not define how Sequencing and Navigation (SN) provided navigation controls are rendered,
how they are triggered or what navigation events they trigger.
SCORM also provides the means (via <adlnav:presentation>) for a content developer to identify
that the content is providing navigation controls within the content. In these cases, the LMS is
required to honor the request of the content and to not provide any redundant and potentially
confusing user interface controls.
So the manifest in the next table will make sure that navigation menu buttons of the LMS are
disabled (make sure that the necessary navigation controls are implemented with the content),
so no confusing user interface controls appear. And by setting the sequencing control flow to
true a sequencing implementation will automatically evaluate the order in which the activity’s
children should be experienced based on Continue and Previous navigation requests.
Personalized E-Learning 26
<manifest>
…
<organizations default="FT02240">
<organization identifier="FT02240">
<title>Computer Installations Basic</title>
<item identifier='ItemFT02240' isvisible='true'>
<title>Introduction and Explanation</title>
<adlnav:presentation>
<adlnav:navigationInterface>
<adlnav:hideLMSUI>continue</adlnav:hideLMSUI>
<adlnav:hideLMSUI>previous</adlnav:hideLMSUI>
</adlnav:navigationInterface>
</adlnav:presentation>
<item identifier='ItemFT02242' isvisible='true' identifierref='SCOFT02242'>
<title>Introduction</title>
</item>
<item identifier='ItemFT02244' isvisible='true' identifierref='SCOFT02244'>
<title>Explanation</title>
</item>
<imsss:sequencing>
<imsss:controlMode choice="true" choiceExit="true" flow="true" forwardOnly="false"
useCurrentAttemptObjectiveInfo="true" useCurrentAttemptProgressInfo="true" />
</imsss:sequencing>
</item>
…
</manifest>
Table 3 Implement Sequencing and Navigation
This way the complete navigation and sequencing can be build for the complete module and is
SCORM compatible. Other options within the <imsss:sequencing> are conditioning rules (pre, exit,
and post), rollup-roles, and objectives that certain items need to answer to otherwise they will not
be in the sequencing order. With the help of these options pre-test adaptation can be made possible
in SCORM.
3.4.2 Selftest Implementation
The current selftest is completely implemented with JavaScript and makes no use of the standards of
SCORM. This is not the ideal implementation, because SCORM has perfect standards to implement
such a selftest. First of all, all the questions need to be in the manifest as sub items of the (item)
selftest, this way with the help of sequencing (as explained in the previous subchapter) it can be
made sure all the information is studied before making the selftest. Another option with the help of
randomizationcontrols is to randomly select a number of questions out of a bunch. The manifest for
randomly selecting 2 selftest questions out of the possibly 3 is given:
…
<item identifier="SELFTEST">
<title>Selftest</title>
<item identifier="SELFTEST_QUESTION1" isvisible = "false" identifierref="RESOURCE_QUESTION1">
<title>Question 1</title>
</item>
<item identifier="SELFTEST_QUESTION2" isvisible = "false" identifierref="RESOURCE_QUESTION2">
<title>Question 2</title>
</item>
<item identifier="SELFTEST_QUESTION3" isvisible = "false" identifierref="RESOURCE_QUESTION3">
<title>Question 3</title>
</item>
<imsss:sequencing>
<imsss:randomizationControls selectCount="2" selectionTiming="onEachNewAttempt" />
</imsss:sequencing>
</item>
…
Table 4 Selftest implementation
Personalized E-Learning 27
The selftest questions still need to be written with the help of fi. JavaScript, because there is no
standard in SCORM for this, but the variables used in this JavaScript can be used (indirectly) in the
manifest (f.i. to make sure the correct questions are asked and the correct sequence is followed).
Personalized E-Learning 28
4 Adaptation
According to the dictionary (Mer08), adaptation is the act or process of adapting. This dictionary
defines adapting as: to make fit (as for a specific or new use or situation) often by modification.
In this case if a course is adapted, it is completely made fit to a specific user.
In the next subchapter will be described what types of adaptation are possible and which of these
possibilities are suitable for the InfoSecure modules. These adaptation types make use of adaptation
rules as described in chapter 4.2. These adaptation rules base decisions depending on the knowledge
of the student. This knowledge of the student can be determined with the adaptation methods
described in chapter 4.3.
4.1 Adaptation Types
The course can be adapted on different ways. The different adaptation types that make the course fit
to a specific user are explained in the following subchapters.
4.1.1 Content Adaptation
Content adaptation is the action of transforming content to adapt to device capabilities. Content
adaptation is usually related to mobile devices that require special handling because of their limited
computational power, small screen size and constrained keyboard functionality.
Content adaptation could roughly be divided into two fields: Media content adaptation that adapts
media files and browsing content adaptation that adapts websites to mobile devices.
In this case content adaptation is not an option, because all the students use a laptop or desktop
(with a screen resolution of 800-600+) to follow this course, so content adaptation is not necessary.
4.1.2 Link Adaptation
The links in the content can be adapted in such a way, that the presentation style of the links is
associated with the status of these links to pages. This status is determined by a set of rules. It is
typical that some simple rules are used that associate the link presentation according to the
suitability of the link destination. E.g. the presentation can be:
• GOOD: the link points to a suitable page you have not visited before. A standard color for
such link anchors is blue.
• NEUTRAL: the link points to a suitable page you have visited before. A standard color for such
link anchors is purple.
• BAD: the link points to an unsuitable page. Whether or not you visited this page before the
standard color for such link anchors is black.
In addition to link colors the presentation of links may also include icons. The presentation style is
completely in hands of the developer, and the coloring can be completely adjusted to the style of the
course. There is also the possibility to let the user choose his desired colors in a setup menu (AHA08).
In this case link adaptation is an option, but is not of first interest. This is because the sequencing of
the course is already established and changing this will not lead to time benefits, which still is the
first priority.
Personalized E-Learning 29
4.1.3 Presentation Adaptation
In the previous subchapter is described how presentation adaptation is used for the links in the
pages, but presentation adaptation can also be used for the entire course. This way the style of the
complete course is adapted to the style of the user’s preferences. Text fonts, font-sizes, coloring, etc.
In this case presentation adaptation is not an option, because all the courses are adapted to the style
of the company which bought the course, or to the style of InfoSecure. Presentation adaptation will
not lead to great time benefits and the style of the company will be discarded, therefore
presentation adaptation will not be applied.
4.1.4 Information Adaptation
The most important adaptation in this case is the information adaptation, because with this type of
adaptation the most time benefit can be made. Information adaptation is basically skipping or
adjusting information that should be known to the student according to certain adaptation rules (see
chapter 4.2). This way information can be skipped on different levels. The complete course can be
skipped, complete pages can be skipped, or possible only a paragraph or a sentence on a page is
skipped. This way each page can be different for every single student. Instead of skipping, adjusting is
also a possibility. E.g. a 1 minute video instead of a 3 minute video, 2 paragraphs instead of 3
paragraphs etc.
Noteworthy is that skipping the whole page is actually not information adaptation, but link
adaptation, because the link to that specific page is not displayed at all, but the objective is the same:
less information is displayed and more time is saved.
4.2 Adaptation Rules
In the previous subchapters is described that according to adaptation rules certain decisions are
made. The syntax of these adaptation rules are in pseudo-code, because this is program dependant.
The course consist of multiple pages, when the student visits a page, an adaptation rule is triggered.
For instance a student has a certain knowledge level (between 0 and 100) about each page, after
visiting a page a knowledge level can change (possibly depending on answers, or time visited).
For instance the rule on page 1 that makes sure that the knowledge level of page 5 of the student will
change (30% more than the knowledge level of page 2) after reading page 1, looks like this:
page5.knowledge := page2.knowledge + 30
The outcome of these rules can be used to trigger (part of) pages. This will look something like this in
pseudo-code: if (page5.knowledge>40)
then “display this information”;
else “display other information”
end if
Of course answers to certain pretest questions will change the knowledge level of pages (according
to adaptation rules) as well and therefore adaptation is easily applied with these questions.
Adaptation according to the HR information is also possible, because adaptation rules can draw
conclusion out of a student’s HR information. For instance if a student has a CISSP diploma the
following rule will make sure that his knowledge about page 5 is 100%.
If student1.diploma=”CISSP” then page5.knowledge=100
Personalized E-Learning 30
4.3 Adaptation Methods
The above described adaptation rules can base decisions on knowledge of students. This knowledge
can be based on all kinds of information that is available for the student. two adaptation methods
that are most suitable for InfoSecure are described in this chapter. Adaptation by pretest questions,
where a student’s knowledge is based on answers to pretest questions, and adaptation regarding to
HR where a student’s knowledge is based on the information available in the Human Resource
Database of the company.
4.3.1 Adaptation by Pretest Questions
At the start of a course a student is asked a certain number of questions all covering parts of the
course. The parts related to the questions the student answers right, will now be skipped (or
adjusted) during the course. This way a time benefit is gain, if the questions take less time than the
relating parts. The student should only answer the questions of the pretest if he is certain about his
answer, otherwise he should answer the question with the option: “I am not familiar with this
subject and would like to view the information”. This is the standard answer, so if the student
doesn’t change this answer, he is assumed not to know the information. The student is also asked if
he wants to take part of the pretest questions beforehand, if not he gets the normal non-adapted
course, and therefore will not gain time.
The results of the pretest questions will be analyzed to make sure this adaptation method gains time
in comparison with the non adapted course (see chapter 4.4.2).
4.3.2 Adaptation regarding to HR
As described earlier, the adaptation rules use available information of the user. E.g. answers to
questions, pages already visited etc. But another important factor in our course is the information
out of the HR database of the company (e.g. diplomas, certificates, department, function within the
company). How can this information contribute to better adaptation of the course. Adapting the
course according to HR information is more difficult than adaptation on pretest questions, because
there are more aspects that have to be taken into account. All the different HR attributes are
described in the next subchapters. This adaptation method is only applied if it books a time benefit in
comparison with the pretest adapted course. This will be thoroughly analyzed in chapter 4.4.3.
4.3.2.1 Diplomas and Certificates
First of all, people can have the same diploma/certificate, but still have different knowledge on
different subjects. This can be because the students graduated at a different university, but still get
the same diploma/certificate. Another important factor is the year of graduation, because through
the years a lot can change according to the subject material. Even if the year of graduation is taken
into account, students with the same diploma and same certificate still can have different
knowledge, for whatever reason (interest, extra diploma’s, background, current job, etc.). Therefore
no knowledge conclusions can be drawn with a 100% certainty on any HR attribute, including
diplomas and certificates.
In chapter 4.4.3 will be described how this data still can be useful for adaptation, by analyzing the
results of a student’s pretest (and or selftest) in comparison with his HR attributes.
Noteworthy is that if the HR information is not available within the system, this information needs to
be entered once into the company’s database and can then be used for every course the student is
going to follow.
Within the research to adaptation according to diplomas, the year of graduation plays a role. The
university, where the student graduated will not play a role, because with a lot of students, the
variety will be too large. The year of graduation can eventually be divided into groups, e.g. 1975-
Personalized E-Learning 31
1980, 1980-1985. This will depend on the outcome of the analysis. But it is expected that parallels
can be drawn to graduation years that are close to each other.
4.3.2.2 Function
The function of the student can also play in important role for adaptation. A manager or a CEO has
probably more knowledge on certain subjects than a secretary. But the same as for diplomas and
certificates in the beginning no conclusions can be made. Therefore this needs to be investigated as
well.
4.3.2.3 Department
The department the student is working in can be of the same interest as his diplomas, but studying
this results will be different per company. Therefore it is not of first interest, because every company
will have different departments. But in the analysis the department of the student can be analyzed
the same way the other HR information is analyzed. The outcome of the analysis will show if time
benefit can be gained according to adaptation to department. Again this is only possible for large
companies, because than there is a big enough sample to draw conclusions to a student’s
department, before the rest of the students from that department take the course. Of course bigger
companies do have similar departments, but it is not guaranteed that the outcome for every
company will be the same. Possible conclusions can be drawn only after finishing the analysis within
several different companies.
During the analysis in the next subchapter, the student’s department is not taken into account.
4.4 Analysis
As described in the previous chapter there are two methods to adapt the InfoSecure modules. Both
methods need to be analyzed to make sure that a time benefit is booked. In chapter 4.4.2 the pretest
adaptation is analyzed and in chapter 4.4.3 the HR adapted course is analyzed.
To better clarify the analyses in these two chapters first an example course is given in chapter 4.4.1.
4.4.1 Example Course
Every course consists of at least 2 parts. Information pages and a selftest. This is the case if the
course is not yet adapted. If the course is adapted with a pretest it consists of 3 parts. Questions,
information pages, and a selftest. For clarification and simplicity reasons a couple assumptions are
made in this example course.
There is exactly 1 pretest question (QI) that is related to 1 information page (Ii) and the selftest (STi)
consist out of exactly the same number of questions as the pretest and also relates to the same
information pages. In the actual courses, there can be more pretest questions related to more
information pages or even part of information pages. The selftest probably has more questions and is
related to several information pages more than once, but a better clarification can be made with this
example. The selftest will take more time than the pretest, because in the real course it probably
consists out of more (and possibly more time consuming) questions. The student is familiar to the
information while taking his selftest, but still the duration of the selftest will approximately be 1,5
times the duration of the pretest, for twice as many questions. This number can differ per course,
but is roughly in this area.
Personalized E-Learning 32
The non adapted example course will look as follows:
Diagram 2 Non adapted example course
The pretest adapted example course will look as follows (information pages are shown based on the
answers of the pretest questions):
Diagram 3 Pretest adapted example course
The duration of these part is estimated, the real average duration time of (parts of) a course can be
calculated after enough students have finished the course.
4.4.2 Pretest Analysis
During this analysis is checked if the pretest adapted course is making a time benefit in comparison
with the non adapted course. A time benefit can only be made if enough students pass for the
pretest. This percentage is calculated in chapter 4.4.2.1. The correlation between the pre-test
answers and the selftest answers is also very important for calculating the breakeven percentages.
This is explained in chapter 4.4.2.2.
A possible adaptation can be made to the selftest, which will increase the quality of the entire
module (see chapter 4.4.2.3).
4.4.2.1 Success Percentage
The most important percentage for making a time benefit with a pretest adaptive test is the
percentage of students that fail for the pretest and have to study the accompanying information.
This percentage must be lower than its breakeven percentage, otherwise the course makes no time
benefits in comparison with the non adaptive course. The breakeven percentage is calculated as
follows:
���
���
= 1 −
With
: Percentage of students that succeeds for the pretest question i.
��� : Time Profit for question i. The time of the information page of question i minus the time of
pretest question i (Ii -Qi).
��� : Time lost for question i equals Qi.
Personalized E-Learning 33
In the example case pretest question 3 takes 30 seconds, therefore for this question time lost (��) is
30 seconds. Time profit (��) will be 150 seconds. This is the duration of the accompanying
information minus the duration of the pretest question (180-30). The breakeven percentage () for
pretest question 1 is �
�, because
150
30=
1 −16
16
If at least �
� of the students succeeds for pretest question 1, pretest adaptation will make a time
benefit. This is under the assumption that there is a 100% correlation between the pretest and the
selftest. In other words, all the students that succeed for the pretest must succeed for the selftest.
This is probably not the case, so in the next subchapter the success percentage is calculated again,
taking into account the pretest correlation percentage.
4.4.2.2 Success Percentage together with Pretest Correlation Percentage
In the previous subchapter the assumption is made that the correlation between the pretest and
selftest questions is 100%. The actual percentage can be calculated, and together with that the
success percentage explained in the previous subchapter can be calculated more precisely.
When calculating the correlation percentage it is checked if the student really knew the information
he was supposed to know according to his pretest answers. For instance if a majority of the students
answer question Q3 correctly, but fail for question ST3 (which must be quite similar to Q3 of course),
than the pretest question or the selftest question needs to be changed, or at least the correlation
between these questions needs to be altered. There will be a percentage of students that will pass
the pretest question, but will fail for the selftest on the same subject. This correlation percentage
needs to be analyzed for every pretest question, to guarantee a time benefit is gained. This
percentage shouldn’t be too high, otherwise the pretest (and/or selftest) question must be altered or
removed. The lower this percentage the better. The breakeven success percentage depends on the
correlation percentages. When the following formula is smaller than zero, the correct percentage for
both the success percentage, as the correlation percentage can be calculated. The formula will be
explained below:
��1��1 − � + ��2��1 − �� − ���� < 0
With:
: Percentage of students that succeeds for pretest question i.
� : Percentage of , that succeeds for selftest question i.
1- : Percentage of students that fails for pretest question i.
1-� : Percentage of , that fails for selftest question i.
� : Percentage of students that succeed for pretest question i and selftest question i.
(1-�� : Percentage that succeeds for pretest question i, but fails for selftest question i.
��1� : Time lost for question i in case the student fails for his pretest question i equals Qi.
��2� : Time lost for question i in case the student succeeds for his pretest i, but fails for his
selftest i. The time of the pretest question i plus the time of (part of) the selftest (Qi +
ST(i)). In case there is no adaptive selftest (which means the complete selftest has to be
done again in case of failure the first time, instead of only the selftest questions that
were answered incorrectly the first time), the time lost will be much higher and
therefore the breakeven percentage will change drastically. Take this in consideration
when calculating these percentages.
Personalized E-Learning 34
In the example course pretest question 3 takes 30 seconds, and the accompanying information
would take 180 seconds, a total time profit (TP3) of 150 seconds is booked if the student passes his
selftest. If the student fails for his selftest, he has to redo the course and (parts of) the selftest again.
The course has an adaptive selftest, so probably not the complete selftest is repeated. This will lead
to a total time lost (TL23) of at least 75 seconds is (30 + 360/8). This process is visualized in the next
diagram:
Diagram 4 Process Pretest Question 3
The students that don’t succeed for the selftest (1-x), will have a duration time of 255 (30+180+45)
seconds for pages related to subject 3. This is 30 seconds (TL13) more than the non-adaptive test,
which has a duration time of 225 (180+45) seconds. The students that succeed for the pre-test and
the selftest (xy) have a duration time of 75 (30+45) seconds, which is a time profit of 150 seconds
(TP3) . The students that succeed for the pre-test, but fail for the selftest ( (1-y)x) have a duration
time of 300 (30+45+180+45) seconds, which is 75 seconds (TL23) more than the non-adaptive test.
��� : Time Profit for question i. The time of the information page of question i minus the
time of pretest question i (Ii -Qi).
Personalized E-Learning 35
Diagram 4 explains the above function, which leads to the following figure for question 3:
Figure 3 Percentages that lead to time profit for subject 3
The above figure gives a clear indication of the possible percentages of students that succeed for the
pretest (success percentage, x), together with the possible percentages of students that fail for the
selftest, but did succeed for the pretest(correlation percentage, y).
The time profit area can be expanded with a few percent, because there are factors that can’t be
exactly calculated, but play a role in the calculation of the percentages in the time profit area:
1. The assumption is made that every student succeeds for his selftest after studying the
information pages. But a small percentage of the students will fail for these selftest (or
pretest) questions even if they would have studied the according information pages at first.
This percentage can be determined by looking at the results from the non adapted course.
The assumption can be made that this percentage isn’t too high, because otherwise the
quality of the course (or the effort of the student) is too low. Therefore the time profit area
can be widened with a few percent.
2. Possibly in the non adaptive course, a student may think he is familiar to the subjects and will
study the information pages only briefly, but will finally fail for the selftest. Because he has to
redo most of the course, he will take more time studying the information pages than in the
first place. While answering pretest questions first, it comes to the student’s attention, that
he’s not so familiar to the subjects as he thought he was, and therefore he will immediately
study the information pages more carefully. This will save quite some time, so therefore the
time profit area can be widened with a few percent.
3. A student also has the option to answer the pretest question with the option “I am not
familiar with this subject and would like to view the information”. This option will cost him
only about 10 seconds. This option is neglected in these calculations, because the
Personalized E-Learning 36
investigation is based on adaptation and if the student admits he is not familiar to the
information, no adaptation is applied. The percentage of students that answer the pretest
question with that option, must be analyzed. Of course the time this group loses in total
must be less than the time benefits of the students that did answer the pretest question.
4. There is also a small group of students that will fail for the specific selftest question, but will
have successfully finished the complete selftest (in case one or more mistakes are permitted
in the selftest). In this case it needs to be considered how this specific group of students is
handled during the calculation of the percentages. Is it more important that the pretest and
selftest questions are perfectly correlated or is time-benefit the most important factor. In
case time benefit is the most important factor, the students in this specific group are handled
as if they passed the specific pretest and the time profit area will expand.
Adaptation can specify this group real easy, and there is an option to handle the selftest
results of this group differently. See chapter 4.4.2.3.
For every question the explained formula should hold, with a little margin in the percentages
possible, because of the above mentioned reasons. The ultimate breakeven points for the success
percentage are given in Table 5, assuming that every student succeeds for his selftest. The ultimate
breakeven points for the correlation percentage are given in Table 6, assuming that every student
succeeds for his pretest.
Pretest question Minimum
percentage (x)
1 21%
2 21%
3 12%
4 21%
5 8%
6 22%
7 22%
8 10%
Table 5 Pretest maximum percentages
Ultimate minimum percentage for pretest question 3 is 12% (rounded of 1/6 according to function
with y=1 (or see top left in time profit area in Figure 3) minus 5% because of above mentioned
reasons). Notice that the number 1/6 is also calculated in chapter 4.4.2.1, because here is also
assumed that every student that succeeds for his pretest also succeeds for his selftest.
Personalized E-Learning 37
Pretest question Minimum
percentage (y)
1 41 %
2 41 %
3 29 %
4 41%
5 22%
6 29%
7 29%
8 25%
Table 6 Minimum selftest succeed percentage
Ultimate minimum percentage for pretest question 3 is 29% (rounded of 0,34 according to function
with x=1 (or see bottom right in time profit area in Figure 3) minus 5% because of above mentioned
reasons).
Try to aim for the highest possible percentage of y, because this means that the pretest questions
and the selftest questions are highly correlated, and the higher y is, the more student can fail for the
pretest with still booking a time profit overall. In other words the lower x can be.
4.4.2.3 Selftest Adaptation
As becomes clear from the analysis, possibly there is a group of students that correctly answers a
pretest question, will therefore not see the according information pages, but answers the according
selftest question incorrectly. Because this student answered all other selftest questions correctly he
passed the complete selftest with success. This is of course strange, because apparently there is
information which he has never seen and is not familiar with. With the help of adaptation and
changing the selftest this problem can be solved. Just change the selftest in such a way that every
answer for which the student hasn’t seen the according information must be correct, otherwise he
will fail for the complete selftest and has to visit the according information pages after.
Possibly in the non adaptive test there will also be students that click through the information
quickly, fail for this specific selftest question, but succeed for the complete selftest. This is also not
the desired result. The problem with the non adaptive module is that this group cannot be specified.
The advantage of the adaptive module is that this group can easily be specified and the selftest can
be adjusted accordingly.
4.4.3 HR Analysis
As described earlier it is impossible to draw knowledge conclusions based on HR attributes with a
100% certainty. Therefore the pretest answers of the student are analyzed. Possibly there are
parallels between the HR attributes and the answers of the students.
The best case scenario is of course that all the students with the same HR information answer the
same questions correctly and the same questions incorrectly, because than the questions are no
longer necessary, because all the information is in the HR database. Unfortunately this is not the
case. In chapter 4.4.3.1 all HR attributes in combination with the answers of the pretest questions
are analyzed. The outcome of the analysis will be that there are different groups of students (with
different HR attributes) that have significantly different answers than the average student. These
outcomes will be compared with a breakeven percentage (see chapter 4.4.3.2) to make sure a time
benefit can be booked. This is the most important outcome. If in total students don’t gain time with
this approach, it will not be applied.
Personalized E-Learning 38
4.4.3.1 Data Mining
Data mining (sometimes called data or knowledge discovery) is the process of analyzing data from
different perspectives and summarizing it into useful information - information that can be used to
increase revenue, cuts costs, or both. Data mining software is one of a number of analytical tools for
analyzing data. It allows users to analyze data from many different dimensions or angles, categorize
it, and summarize the relationships identified. Technically, data mining is the process of finding
correlations or patterns among dozens of fields in large relational databases.
In this case all answers by all students will be analyzed and afterwards data mining will take place to
find correlations between the answers of the students and their HR attributes. This way possibly time
can be saved and therefore costs are cut. In this chapter is described how data mining works for this
example case, without going into too many details, because the data is still hypothetical.
The starting percentages are easy to determine, as described in the next table:
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Number of
students
All student 60% 45% 80% 74% 34% 90% 70% 37% 14000
Table 7 Pretest scorings percentage
The percentages represent the percentage of students, that succeed for the pretest question and
succeed for the complete final selftest. To be clear 100% minus this percentage is the percentage of
these students that fail for the pretest question AND fail for selftest.
This is the basis to compare the rest of the analysis results with. The next tables are:
D:Diploma+year Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Number
of
students
A - 1990 80% 95% 60% 54% 94% 60% 30% 97% 140
A - 1991 85% 85% 30% 72% 92% 50% 80% 64% 150
… 98% 86% 20% 74% 90% 53% 83% 60% …
… 90% 86% 40% 71% 91% 50% 80% 64% …
C – 1975 90% 84% 20% 79% 90% 50% 80% 60%
… … … … … … … …
No diploma 68% 76% 10% 44% 50% 33% 43% 30%
Table 8 Pretest scorings percentage with certain diploma
C:Certificate Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Number of
students
A 95% 86% 50% 74% 90% 53% 83% 60% 120
B 94% 82% 50% 72% 94% 57% 90% 70% 130
… 88% 86% 55% 74% 90% 63% 83% 60% …
… 88% 84% 66% 73% 95% 57% 91% 67% …
E 87% 86% 56% 74% 90% 55% 82% 60%
… … … … … … … … …
No
certificate
68% 76% 10% 44% 50% 33% 43% 30%
Personalized E-Learning 39
Table 9 Pretest scorings percentage with certain certificate
F:Function Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Number of
students
CEO 98% 84% 20% 74% 88% 53% 83% 60% 110
Secretary 44% 42% 23% 66% 90% 44% 44% 33% 140
… 91% 86% 26% 74% 88% 66% 83% 60% …
… 92% 86% 29% 77% 83% 53% 83% 60% …
Manager 98% 86% 33% 74% 90% 66% 83% 60% 70
…. … … … … … … … …
Table 10 Pretest scorings percentage with certain function
The chi-square test (also chi-squared or χ2 test) will investigate if there are groups of students with
certain HR attributes that differ from the basic group of students. This test also makes sure the data
source (number of students) is large enough to draw conclusions on. In this chapter only the basis of
this test will be described.
The test is done by taking one variable at the time. In this case the investigation starts with Table 8,
because a diploma probably has the most influence on the results. The test analyses the results from
students with a certain diploma and compares them to the basic results. For every diploma is
determined if the results can be treated differently. According to the test, students with different
diplomas possibly end up with the same test results and can therefore be treated the same. So after
this part of the test Table 8 will be updated:
Group name D Diploma + year Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8
D1
A1990 – A1995
B1997 – B2004
C1980
D1981-D1999
80% 95% 60% 54% 94% 60% 30% 97%
D2
A1990 – A1995
B1997 – B2004
C1980
C1981
D1981-D1999
85% 94% 70% 60% 89% 70% 50% 80%
… …
D25 No diploma
E2006
68% 76% 10% 44% 50% 33% 43% 30%
Table 11 Updated diploma scorings percentage
Actually this table is not completely correct, because the group names described in Table 11 can very
well be different for each question. So actually 8 tables are necessary, one for each question with
possible different groups. But for clarification reasons in this example one table is given. Notice that
this table has no column named “number of students”, because in this table only data is taken into
account that satisfies the criteria of the chi-square test and therefore is useable. If the student’s
diploma is not in this table, the student will be treated as an average student (see Table 7).
After this analysis the next HR attribute (certification) will be investigated in combination with the
results of Table 11, which will result in a more detailed table:
Personalized E-Learning 40
Group name D Group name C Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8
D1 C1 85% 92% 63% 66% 94% 80% 90% 99%
D1 C2 75% 91% 70% 62% 89% 75% 55% 80%
… …
D25 C9 68% 76% 10% 44% 50% 33% 43% 30%
Table 12 Scorings percentage with diploma and certification taken into account
With group name C containing all certification possibilities. E.g. C1 is certification A or certification B,
and C9 is certification Q. This table is an extension of Table 11 and as you can imagine this table will
expend more by taking into account more HR attributes. Relatively the table will not expend that
much more by taking into account more HR attributes, because the results of the test must make an
significant difference and the first two HR attributes are the most important two and will take credit
for most of the difference.
The test started with the most significant HR attribute diploma, but it could also be possible that the
certification of the student has a bigger impact on the results. Therefore the test also has to be
executed with first certification and afterwards diploma and the other HR attributes. But without
going into too many details, after the complete test is executed a table (this will be a very large table,
in the real program an updated database) will be available:
D C F Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8
D3 C2 F4 100% 100% 100% 100% 100% 100% 100% 100%
D1 C1 F5 94% 95% 97% 95% 94% 94% 95% 94%
D2 C2 F1 98% 84% 80% 74% 88% 53% 83% 60%
D4 C2 F1 80% 95% 60% 54% 94% 60% 30% 97%
D1 C1 - ..% ..% ..% ..% ..% ..% ..% ..%
… … … ..% ..% ..% ..% ..% ..% ..% ..%
D4 C3 F2 30% 33% 30% 27% 24% 30% 30% 26%
D2 C2 F2 18% 18% 9% 18% 5% 9% 9% 7%
- C8 - ..% ..% ..% ..% ..% ..% ..% ..%
D25 C9 F9 ..% ..% ..% ..% ..% ..% ..% ..%
x x x 60% 45% 80% 74% 34% 90% 70% 37%
Table 13 Optimized scorings percentages
With - meaning that this group is not taken into account and x meaning no group. Therefore the last
column represents the percentages equal to Table 7. This is because if the student falls in no
category available in the table on all the different HR attributes (this is the case if the available data
sample, the number of students, is too little or that his HR attributes make no significant difference)
the percentage of the average student is used.
Table 13 is constructed in such a way that the best fitting case scenario is at the top and the worst
fitting case is at the bottom. A student fits in a at least one row, but the best case for this student is
always the highest ranked row in the table.
Noteworthy is that Table 13 also has to be constructed 8 times (once for each question), for the same
reasons as described earlier for Table 11.
4.4.3.2 Breakeven Percentage
If the percentages in Table 13 are higher than the breakeven percentage of that question, the specific
student can skip this question and accompanying information.
Personalized E-Learning 41
This percentage needs to be calculated for every pretest question and is done by the following
formula: ���
���
= �
100 − �
With
In the example case the percentage for pretest question 1 is 47�
��%, because
27
30=
477
19
100 − 477
19
Where the time profit (���) in this case is Q1 = 30, because the breakeven percentage in comparison
with the pretest adaptive test is calculated, and the time lost (���) is ((part of) ST)*1.25 – Q1 = 27,
because in this case the student has to answer the selftest questions immediately and is not familiar
to all the information, this is why the selftest takes approximately 25% more time. The pretest
question is not asked.
Therefore if more than 47�
��% of the students with a certain HR information attribute (diploma,
function, certificate or a combination of those) answer pretest question 1 correctly HR adaptation
will make a clear time benefit for this question and will be applied.
In the example case the percentages are the same for each question, because each pretest question
takes the same time. But again this percentage needs to be rounded off, because there are factors
that can’t be exactly calculated that play a role in the calculation of this percentage:
1. The time the student takes for the selftest is raised with 25%, this is an approximation and
not an exact figure.
2. These percentages are the minimal percentages necessary, assuming that every student will
succeed for the non adapted course after studying the information. This is probably not the
case, so these percentages can be lowered with a few percent.
3. Assumed is that the success percentage and the correlation percentages, explained in
chapter 4.4.2.2 are sufficient, so that the pretest gains time in comparison with the non
adaptive test.
Because of these reasons a correct minimal percentage that should finish the pretest question and
the selftest successfully for the example case is around 40%.
� : Percentage of students the finish pretest question i and the selftest with a good result.
��� : Time Profit for question i equals Qi. In case the breakeven percentages in comparison with
the non adaptive course instead of the pretest adapted course are calculated the time profit
is the time of the pretest question plus the according information pages (Qi + Ii).
��� : Time lost for question i is the time of (part of) the selftest minus the time of the pretest
question (ST(i)-Qi). In case there is no adaptive selftest the percentages will be much lower,
because the complete selftest needs to be done again and therefore the time lost will be
much higher. Take this in consideration when calculating these percentages.
Personalized E-Learning 42
Noteworthy is the fact that this percentage is much higher in case of a non adaptive selftest (around
90% for this example), because then the time lost will be much higher. In the current case, with an
adaptive test, the didactics of the test will change, and this needs attention. With the current
example data possibly 60% percent of the students can immediately make the selftest, fail for it,
have to read the information pages (according to the answers of the selftest), and make the selftest
again. In this case the first selftest acts as a pretest and therefore the actual selftest should consists
of different questions, because otherwise the selftest will be too easy to succeed without having the
required knowledge. There could be made a point for raising the breakeven percentage so that less
students get the same selftest question twice in short period of time, but it is best to make multiple
selftest questions with the same subject. With the help of adaptation this is easily implemented in
the selftest.
Another minus is that in this case 60% of the students possibly have to answer questions which they
don’t know and only afterwards the get the according information. This can be annoying. These
people also get a bad result for their first selftest, which will not motivate them to continue.
Considerations have to be made, when implementing the HR adaptive E-learning module, possibly a
bigger breakeven percentage needs to be established, because this way less students will follow a
course that is not perfectly suitable for them. Again this is a consideration between time benefits and
the didactics of the course.
Personalized E-Learning 43
4.5 Scenarios and Time Benefits
Let’s consider a couple of scenarios together with the time benefits. The time benefits are not
completely correct, because the assumption is made that if the pretest question is answered
correctly, the selftest question about this subject is also answered correctly. In other words, the
percentage explained in chapter 4.4.2.2 is assumed to be 100%. But the percentage of students that
belong to this group needs to be calculated and the time for these students to redo part of the
course and the selftest needs to be added by the total time of the course for these group of students.
But this percentage should be very low, and for these scenario’s it is neglected.
It is assumed that the selftest of the non adapted course will be succeeded by all students the first
time. This is not the case, so the estimated numbers are minimal, because there are also students
that fail for the non adapted course. Another assumption is that in the adaptive test, students after
viewing the necessary information, always succeed for the selftest. This of course needs to be tested
(see chapter 8), but the scenarios and time benefits are sketched to give a proper indication of all the
possibilities and what scenarios to expect during actual testing.
For calculation of the time benefits it is assumed the HR adapted test has a non adaptive selftest, so
(in scenario 2 en 3) the results are a higher than the results with an adaptive selftest would be, but
again these time benefits are just to give an indication.
In chapter 4.5.1 all different kind of scenarios are considered and the time benefits under the above
assumptions are calculated. An overview of these time benefits is given in chapter 4.5.2.
4.5.1 Scenario’s
Five different kind of scenario’s are considered in this case. For all scenario’s the non adaptive course
will take 1710 seconds (see chapter 4.4.1).
Scenario 1: Best Case
Every student that falls into the following categories: D3, C2, and F4 answered question 1 through 8
correctly (see Table 13) and therefore these questions don’t need to be asked or answered by this
group of students. This will save a lot of time, because not only the pretest questions, but also the
accompanying information will be skipped. The time benefit will be as follows:
The pretest adaptive course for this type of student will take 600 seconds, because all the pretest
questions are answered correctly and therefore the accompanying information is not shown, and the
course will look as follows:
Diagram 5 Scenario 1 pretest adapted course
And the HR adaptive course will take 360 seconds, because the pretest questions don’t need to be
asked, and the course will look as follows:
Personalized E-Learning 44
Diagram 6 Scenario 1 HR adapted course
Of course this scenario is not very likely, because a 100% pretest question scoring is very hard to
achieve, other scenarios, like scenario 2 are more likely.
Scenario 2: Best Case 2
Every student that falls into the following categories: D1, C1, and F5 will on average make a solid
time benefit. In Table 13 the percentages are displayed and it is clear that most information is known
by most of these students. Let’s split up this scenario:
Scenario 2a
Most of these students will pass all pretest questions, and therefore the pretest adapted course will
take as much time as scenario1, 600 seconds, as well as the HR adaptive course, 360 seconds. A small
percentage of the students will fail for the selftest and at least 1 pretest question, a possible scenario
is given as scenario 2b.
Scenario 2b
All kind of scenario´s are possible, let´s just assume that this particular student fails for pretest
questions 1 and 6. In this case the pretest adapted course will take him 900 seconds (see Diagram 7).
Diagram 7 Scenario 2b possible pretest adapted course
Noteworthy is that the student thought he know the information, otherwise he would have
answered the pretest questions 1 and 6 with the option “I am not familiar with this subject and
would like to view the information” and would only need 10 instead of 30 seconds for pretest
question 1 and 6 each.
In case of the HR adapted course the student needs 1110 seconds, as described in the next diagram:
Diagram 8 Scenario 2b possible HR adapted course
For this particular student the HR adapted course takes more time than the pretest adapted course,
but overall more students will make time profit and therefore the average student with this HR
attributes will make a clear profit:
Personalized E-Learning 45
Time benefit complete scenario pretest adapted course
For every student the course takes at least 600 seconds (see Diagram 5), but a certain percentage
(see Table 13) will fail for some pretest questions, and therefore has to view the according
information pages. In the case of these students the average duration will be 672 seconds as
described in this table:
pretest
fails / succeeds
%
Duration
information
page (sec)
Total average
duration
per student
(sec)
Total average
duration per
student after
question (sec)
Q1 6 +120 720 607,2
94 0 600
Q2 5 +120 727,2 613,2
95 0 607,2
Q3 3 +180 793,2 618,6
97 0 613,2
Q4 5 +120 738,6 624,6
95 0 618,6
Q5 6 +240 864,6 639
94 0 624,6
Q6 6 +180 819 649,8
94 0 639
Q7 5 +180 829,8 658,8
95 0 649,8
Q8 6 +210 868,8 671,4
94 0 658,8
average total ≈672 sec
Table 14 Average duration pretest adapted course scenario 2
Again, it is assumed that the student will succeed for their selftest after viewing the necessary
information pages first.
Time benefit complete scenario HR adapted course
In case of the HR adaptive test a same calculation can be constructed. This time the course takes at
least 360 seconds (see Diagram 6), but if a student fails for a pretest question, the complete selftest
needs to be done (450 seconds, because the student started with the selftest, and was not familiar
with all the information, it takes him 25% more time than the normal 360 seconds) again. In the case
of this group of students the average duration will be less than the 672 seconds (see Table 14),
because the pretest is passed in more than 90% for all questions (see chapter 4.4.3.2). The exact
calculation can be made on the same way the pretest time is calculated, only with a slide adjustment.
Assumed is that the total time will be 360 seconds plus 450 seconds (time of the complete selftest),
810 seconds. And afterwards, the overall percentage of all students that succeed the selftest the first
time is subtracted from the outcome, resulting in this table:
Personalized E-Learning 46
pretest
fails /
succeeds
%
Duration
information
page (sec)
Total average
duration
per student
(sec)
Total average
duration per
student after
question (sec)
Q1 6% 120 930 817,2
94% 0 810
Q2 5% 120 937,2 823,2
95% 0 817,2
Q3 3% 180 1003,2 828,6
97% 0 823,2
Q4 5% 120 948,6 834,6
95% 0 828,6
Q5 6% 240 1074,6 849,0
94% 0 834,6
Q6 6% 180 1029 859,8
94% 0 849
Q7 5% 180 1039,8 868,8
95% 0 859,8
Q8 6% 210 1078,8 881,4
94% 0 868,8
Percentage
all questions
correct
64,93% -450 431,4
280,1
Percentage
that has to do
selftest
35,07% 0 881,4
309,1
Sum 589,2
Average total ≈590 sec
The average total time for this students is 590 seconds.
Scenario 3: Average Case
Every student that falls into the following categories: D4, C2, and F1 will on average make a time
benefit. To calculate the time benefits for the pretest adapted course, the same table can be
constructed as in scenario 2 only with different percentages:
Personalized E-Learning 47
pretest
fails / succeeds
%
Duration
information
page (sec)
Total average
duration
per student
(sec)
Total average
duration per
student (sec)
Q1 20 +120 720 624
80 0 600
Q2 5 +120 744 630
95 0 624
Q3 40 +180 810 702
60 0 630
Q4 46 +120 822 757,2
54 0 702
Q5 6 +240 997,2 771,6
94 0 757,2
Q6 40 +180 951,6 843,6
60 0 771,6
Q7 70 +180 1023,6 969,6
30 0 843,6
Q8 3 +210 1179,6 975,9
97 0 969,6
Average total ≈ 976 sec
Table 15 Average duration pretest adapted course scenario 3
The average duration for this type of students is 976 seconds. In the HR Adapted course it is only
profitable to adapt the questions 2, 5, and 8, because their percentages exceed the breakeven
percentage of 90%. All the other pretest questions are asked. So the HR adapted course will in the
best case look as follows:
Diagram 9 Best case scenario 3 HR adapted
In the best case for this scenario the course will take 510 seconds. The average case will take 636
seconds. This is calculated in two steps, first is calculated what the average duration time of the
pretest questions and accompanying information is, on the same way as above, with temporarily
changing the HR-adapted questions to 100%:
Personalized E-Learning 48
pretest
fails / succeeds
%
Duration
information
page (sec)
Total average
duration
per student
(sec)
Total average
duration per
student (sec)
Q1 20% 120 630 534
80% 0 510
Q2 0% 120 654 534
100% 0 534
Q3 40% 180 714 606
60% 0 534
Q4 46% 120 726 661,2
54% 0 606
Q5 0% 240 901,2 661,2
100% 0 661,2
Q6 40% 180 841,2 733,2
60% 0 661,2
Q7 70% 180 913,2 859,2
30% 0 733,2
Q8 0% 210 1069,2 859,2
100% 0 859,2
Average total ≈ 860 sec
Table 16 HR adaptation part 1
The average time for the questions 1, 3, 4, 6, and 7 with the possibly accompanying information is
859,2 seconds. The added average time for the questions 2, 5, and 8 with HR adaptation will be 86,9
seconds according to the next table.
Personalized E-Learning 49
pretest
fails /
succeeds
%
Duration
information
page (sec)
Total average
duration
per student
(sec)
Total average
duration per
student (sec)
Q1 0% 120 120 0,0
100% 0 0
Q2 5% 120 120 6,0
95% 0 0
Q3 0% 180 186 6,0
100% 0 6
Q4 0% 120 126 6,0
100% 0 6
Q5 6% 240 246 20,4
94% 0 6
Q6 0% 180 200,4 20,4
100% 0 20,4
Q7 0% 180 200,4 20,4
100% 0 20,4
Q8 3% 210 230,4 26,7
97% 0 20,4
Percentage
all questions
correct (PC)
86,62%
+0 +26,7 23,13
1-PC 13,38% +450 +476,7 63,78
Sum 86,91
Average added
total ≈ 86,91 sec
Table 17 HR adaptation part 2
The total average time for a student in this group will therefore be 859,2 + 86,91 ≈ 946 seconds
based on HR adaptation.
Scenario 4: Worst Case
The percentages in Table 13 for a student with the following HR attributes D4, C3, F2 are very low.
Actually the percentages are just high enough to apply the pretest. Compare the row in Table 13 with
Table 6 and it is clear that the time benefit will not be too high for the pretest. HR adaptation will
clearly not make a time profit.
The average pretest time will be 1568 seconds as calculated in Table 18, which still is a minimal
average time benefit of 142 seconds.
Personalized E-Learning 50
pretest
fails / succeeds
%
Duration
information
page (sec)
Total average
duration
per student
(sec)
Total average
duration per
student after
question (sec)
Q1 70 120 720 684
30 0 600
Q2 67 120 804 764,4
33 0 684
Q3 70 180 944,4 890,4
30 0 764,4
Q4 73 120 1010,4 978
27 0 890,4
Q5 76 240 1218 1160,4
24 0 978
Q6 70 180 1340,4 1286,4
30 0 1160,4
Q7 70 180 1466,4 1412,4
30 0 1286,4
Q8 74 210 1622,4 1567,8
26 0 1412,4
average total ≈1568 sec
Table 18 Average duration pretest adapted course scenario 4
Personalized E-Learning 51
Scenario 5: Worst Case 2
The percentages in Table 13 for a student with the following HR attributes D2, C2, F2 are very low.
These percentages are too low to apply the pretest. Compare the row in Table 13 with Table 6 and it
is clear that no time benefit is gained. With these percentages HR adaptation is out of the question.
The average pretest time will be 1810 seconds as calculated in the next table, which is an average
time cost of 100 seconds.
pretest
fails / succeeds
%
Duration
information
page (sec)
Total average
duration
per student
(sec)
Total average
duration per
student after
question (sec)
Q1 82 120 720 698,4
18 0 600
Q2 82 120 818,4 796,8
18 0 698,4
Q3 91 180 976,8 960,6
9 0 796,8
Q4 82 120 1080,6 1059
18 0 960,6
Q5 95 240 1299 1287
5 0 1059
Q6 91 180 1467 1450,8
9 0 1287
Q7 91 180 1630,8 1614,6
9 0 1450,8
Q8 93 210 1824,6 1809,9
7 0 1614,6
average total ≈1810 sec
Table 19 Average duration pretest adapted course scenario 5
4.5.2 Time Benefits
As described in the scenarios above a clear overview of the time benefits is given:
Scenario Non Adaptive Course
(TT)
Pretest Adaptive Course
(TT / TP)
HR Adaptive Course
(TT / TP)
Scenario 1 1710 seconds 600 seconds
1110 seconds
360 seconds
1350 seconds
Scenario 2 1710 seconds 672 seconds
1038 seconds
590 seconds
1120 seconds
Scenario 3 1710 seconds 976 seconds
734 seconds
946 seconds
764 seconds
Scenario 4 1710 seconds 1568 seconds
142 seconds
Not applied
-
Scenario 5 1710 seconds 1810 seconds (don’t apply)
-100 seconds
Not applied
-
Table 20 Time benefits per scenario
Personalized E-Learning 52
With
TT = Total time of the course per average student
TP = Total time profit of the course per average student in relation to the non adaptive course
Of course the individual time benefits can be very different, as shown in the next table:
Student Non Adaptive Course
(TT)
Pretest Adaptive Course
(TT / TP)
HR Adaptive Course
(TT / TP)
Best student 1710 seconds 600 seconds
1110 seconds
360 seconds
1350 seconds
Honest
student 1
1710 seconds 1710 seconds
0 seconds
-
-
Honest
student 2
1710 seconds 1790 seconds
-80 seconds
-
-
Worst
student 1
1710 seconds 1950 seconds
-240 seconds
-
-
Worst
student 2
1710 seconds 1950 seconds
-240 seconds
1710 seconds
0 seconds
Worst
student 3
1710 seconds 1950 seconds
-240 seconds
2160 seconds
-450 seconds
Worst
student 4
1710 seconds 2310 seconds
-600 seconds
-
-
Worst
student 5
±2650 seconds ±1950 seconds
±700 seconds
-
-
Table 21 Individual time benefits
With
TT = Total time of the course
TP = Total time profit of the course in relation to the non adaptive course
And
Best student is a student who knows all the answers to the pretest questions (or has such a good HR
profile that all the pretest questions are skipped) and succeeds for the selftest.
Honest student 1 is a student that prefers to follow the complete course instead of the adaptive one.
This student selects this option at the beginning of the course, and the little time this will take is
neglected.
Honest student 2 is a student that answers all the pretest question with the option “I am not familiar
with this subject and would like to view the information” has the same results as the non adaptive
course, but needs 10 seconds per pretest question to answer, and therefore spends 80 seconds more
in this case.
Worst student 1 is a student that fails for all the pretest questions.
Worst student 2 is a student that fails for all the pretest questions, but this was expected according
to his HR attributes.
Worst student 3 is a student that is supposed to know all the pretest questions according to his HR
attributes, but fails for the selftest on all parts.
Personalized E-Learning 53
Worst student 4 is a student that knows all the pretest questions, but fails for the selftest on all
parts.
Worst student 5 is a student that thinks he knows all the information, reads the information pages
not thoroughly and therefore fails the selftest in the non adaptive course and has to redo the course.
In case of the pretest adaptive course, the student realizes, because of the pretest questions, he is
not so familiar to the information as he thought he was, and therefore immediately reads the
information pages more thoroughly.
4.6 Conclusion
It is impossible to come up with a list of diplomas/certificates that guarantee knowledge about
certain subjects, therefore first an analysis needs to be done, but at the end a list of diploma’s,
certificates, together with the year of graduation, and possibly function and department will provide
a perfect scheme for what part of the course needs to be done by which type of student. Before this
list is made, good pretest questions need to be made, so adaptation can be applied. Of course it is
still possible an individual student needs more time for an adapted course than for the non adapted
course, but overall most students will gain time benefit and therefore the company will gain a lot of
time and therefore money.
Personalized E-Learning 54
5 Introduction to Awareness Module
As described before, the first module that is subjected to personalization is the module “Basic
module (information) Security”.
Source: http://www.infosecuretools.com/awareness/demo/kpn/#is (only accessible with password
by InfoSecure staff).
Subject: E-learning awareness – English – Basic Module Employees.
5.1 Module Introduction
The module gives an introduction to (information) security within the company, in this case KPN. The
module explains what information security is, and what the objective and importance for the
company is. The module consists of roughly three parts, an explanation of information security, the
golden rules, and a selftest. The module has a duration of approximately 40 minutes including the
selftest, which checks if the student’s knowledge of information security is sufficient.
5.2 Module Duration
The duration of the module is approximately 40 minutes. The duration is distributed over several
chapters (see Table 22 Duration Introduction Module). The time taken per chapter depends on the
student, and his reading speed. For the estimation of the duration a mean reading speed of 200
words per minute is used. Some chapters have movie material and for the total estimation of the
duration the student is expected to watch all the movies completely. The module also contains
exercises. The duration time of these exercises is also estimated for an average student.
Again if a person reads 300 words per minute the complete course will take 30 minutes, but for the
analysis of this module the results of the adaptation will be presented according to an average
student.
Chapter Pages +/- time
Explanation 1 60 sec (reading)
What is Information Security 2 105 sec (reading)
Status within the company 2 116 sec (reading)
About the golden rules 1 60 sec (reading)
Golden rule 1 3 71 sec (reading)
50 sec (movie)
60 sec (exercise)
Golden rule 2 4 129 sec (reading)
38 sec (movie)
60 sec (exercise)
Golden rule 3 4 83 sec (reading)
53 sec (movie)
30 sec (exercise)
Golden rule 4 4 76 sec (reading)
72 sec (movie)
60 sec (exercise)
Golden rule 5 4 77 sec (reading)
49 sec (movie)
60 sec (exercise)
Golden rule 6 4 60 sec (reading)
Personalized E-Learning
Golden rule 7
Golden rule 8
Golden rule 9
Selftest
Conclusion
Relevant links and contacts
Total:
16 chapters
5.3 Time Distribution
The duration of the module can be divided in three parts: reading, movies, and exercises (see
22 Duration Introduction Module
average student spends on each of that parts.
Figure
5.4 Module Construction
The module consists of 16 chapters with a total of 57 pages. The
before completing the course. By using the menu on the left the
pages (see Figure 5 Example Basic Module (Information) Security
Reading
51 sec (movie)
60 sec (exercise)
3 96 sec (reading)
93 sec (movie)
60 sec (exercise)
3 77 sec (reading)
64 sec (movie)
60 sec (exercise)
4 148 sec (reading)
38 sec (movie)
60 sec (exercise)
16 33 sec (reading)
60 sec (exercise)
180 sec (selftest)
1 40 sec (reading)
Relevant links and contacts 1 39 sec (reading)
57 pages
2528 sec
±42 min
Table 22 Duration Introduction Module
can be divided in three parts: reading, movies, and exercises (see
Duration Introduction Module). In the following diagram it becomes visible how much time an
spends on each of that parts.
Figure 4 Time Distribution Introduction Module
The module consists of 16 chapters with a total of 57 pages. The student has to study all these pages,
ing the course. By using the menu on the left the student visits all the necessary
Example Basic Module (Information) Security).
1270 min
50%
508 min
20%
750 min
30%
Reading Movies Exercises
55
can be divided in three parts: reading, movies, and exercises (see Table
n the following diagram it becomes visible how much time an
has to study all these pages,
visits all the necessary
Personalized E-Learning 56
Figure 5 Example Basic Module (Information) Security
The sequence of the pages is completely free of choice. But most students will follow the menu from
top to bottom, by pressing next-button, on the bottom right of the screen. The standard sequence of
the course is described by the following diagram:
Diagram 10 Page sequence
Personalized E-Learning 57
5.5 Module Adaptation Locations
Basically adaptation can be applied throughout the complete course, except for the final selftest at
first, because this test checks if the student has sufficient knowledge about information security. The
selftest is similar for every student. Even if some subjects are not threaded during the course,
because of the adaptation, still questions about these subjects are asked. This is to check if the
adaptation is correctly applied and the student really has sufficient knowledge about these subjects.
If the student fails for his test, the course, or at least a part of the course, needs to be repeated.
Adaptation can be applied throughout the complete course. Every page can be adapted. In the
current module every chapter only consists of a few pages and will take little time (see Table 22
Duration Introduction Module). Adaptation within the pages will therefore gain very little time,
because the information given per page is very minimal. Therefore the adaptation needs to be
applied on the chapters. So the real question is; which chapters contain known information to the
student and which chapters don’t?
In the following subchapters each chapter of the module will be examined.
5.5.1 Explanation
This chapter explains how the e-learning module works. Every e-learning module of InfoSecure works
in the same way, so if a student is already familiar to e-learning modules of InfoSecure, this chapter
can be skipped and adaptation can make this possible.
5.5.2 What is Information Security
In this chapter information security is explained. This chapter is very important and can therefore
only be adapted if it is a 100% certain the student is familiar to information security.
5.5.3 Status within the Company
These two pages gave some statements about information security at KPN and the conclusion is that
the employees of KPN do not think and act alike when it comes to security.
This page can be adapted/skipped if the student knows the status within the company, and therefore
knows why it is important to study this course. This page will be skipped if the previous page (What is
Information Security) is skipped as well.
5.5.4 About the Golden Rules
This chapter explains the golden rules. These are the most important rules that play a role in security
and information security within the company. There are nine roles each extensively explained in a
individual chapter. This chapter must be adapted in such a way that only the necessary golden rules
for the student will be explained and presented in the menu on the left. The 9 golden rules are:
• Rule 1: Keep to the law and KPN’s rules of conduct
• Rule 2: Allot the correct classification to company information
• Rule 3: Prevent improper use or theft of company information
• Rule 4: Prevent improper use or theft of company equipment such as laptop, PDA and
mobile phone
• Rule 5: Pay attention when using e-mail and the Internet
• Rule 6: Use information from and about others with care
• Rule 7: Prevent unauthorized access to our buildings and systems
• Rule 8: Your own safety and that of others is first and foremost!
• Rule 9: Report incidents directly to the Helpdesk of KPN Security: 0800 - 4040 442
If the student is familiar with all the rules it can be skipped, otherwise not.
Personalized E-Learning 58
5.5.5 Selftest
As described earlier the selftest will not be adapted at first, because the selftest must be followed by
every student. This selftest consists out of 12 questions and are all asked. But this selftest can be
adapted if the student fails for the selftest the first time and has to follow the course again. This time
only the questions that were answered incorrectly the first time need to be asked, and this will save
time (an adaptive selftest). So actually the chapter selftest, is the only chapter that is adapted within
the chapter instead of the chapter itself.
5.5.6 Conclusion
This chapter gives a summary and some important telephone numbers, so this chapter cannot be
adapted.
5.5.7 Relevant Links and Contacts
This chapter gives links and contact information, this chapter cannot be adapted, but the student is
not obligated to read it.
5.6 Module Adaptation Techniques
In the previous subchapters is clearly explained where the adaptation can be applied. In this chapter
will be explained how it is applied. A profile of the student is made before taking the course. This
profile keeps updating itself after every (or during the) course taken by the student. The current
course “E-learning awareness: Basic Module Employees” will be taken by a lot of new employees and
therefore the profile still needs to be created.
5.6.1 Student Profile
Every student has a profile. If the student doesn’t have a profile it will be created before starting with
a course. The profile will be filled with human resources (HR) information available for every
employee (education, diploma’s, department). If this information is not available (in the system), the
student has to answer this questions one time before starting with his first course.
The profile also contains answers to pretest questions, that are asked before every course. The
answer to these question are very important. The student should only answer if he’s absolutely sure,
otherwise the student might possibly skip vital information. And this will cost extra time, because the
student will then fail for his selftest and has to redo the course.
Information about courses already taken will also be saved in the profile of the student.
A good profile is the most vital part of the adaptation. The profile information is saved on the server,
and is only used for the adaptation of the courses. The information is encrypted and thus protected
against misuse by third parties.
5.6.2 Adaptation based on HR Information
As described in the previous chapter it is impossible to adapt on HR information without the results
of an analysis first. In this case you might suspect that a student with a background (diploma,
function, or certificate) in the Information Security field should pass this basic introduction module
without studying the information first. But again this conclusion cannot be drawn without an
analysis.
Personalized E-Learning 59
5.6.2.1 Accepted Diplomas and Certificates
After the analysis the results of the accepted diplomas and certificates will possibly look like this.
A possible list of diploma’s that satisfy the criteria to skip the course is given below.
Bsc Computer Science (Information Security)
Msc Computer Science (Information Security)
Postgraduate Diploma in Information Security
Ing. Computer Science (Information Security)
Ir. Computer Science (Information Security)
Table 23 Accepted Diplomas
A possible list of certificates that satisfy the criteria to skip the course is given below.
Postgraduate Certificate in Information Security
Certified Information Systems Security Professional (CISSP)
Table 24 Accepted Certificates
In this example only a list is considered, for which the student can skip the complete course. Ideally
the analysis is much more detailed, and for every HR attribute appropriate adaptation steps can be
executed. For instance a certain golden rule(s) can be skipped with a certain diploma.
5.6.3 Adaptation based on Pretest
Because HR adaptation is only possible after an analysis (of the pretest answers of the students) the
focus is on pretest adaptation. Therefore pretest answers need to be asked. For every module holds
that the benefit of time must be substantially larger than the time that it takes the student to answer
the pretest questions. In this case the module already contains small exercises that perfectly can be
used as pretest questions. Therefore it will not cost extra time and will only gain time.
All the nine golden rules have an exercise part, ask this question before explaining the rule, with the
option “I am not familiar with this golden rule and I’d like to study this rule first”. If the student
chooses this option, the course stays exactly the same and the same exercise is asked after finishing
the chapter about the golden rule (see Diagram 10 Page sequence).
If the student completes the chapter-exercise without mistakes, this chapter can be skipped, and
that will save time. The following diagram shows an example of a student that is familiar to the
golden rules 3, 6, and 9 according to his answers given at the chapter questions.
Personalized E-Learning 60
Diagram 11 Example adaptive page sequence
It is also an option to let the student make all these exercises at the beginning of the module (This is
why it is called a pretest), and then adapt the course according to the answers. This way the student
is first answering questions, than (possibly) studying information, and after that doing his final
selftest. Diagram 12 will show the sequence for the student that is familiar to the golden rules 3, 6,
and 9 according to his pretest answers.
Diagram 12 Example 2 adaptive page sequence
This option is used in the implementation in AHA! as described in the next chapter.
Personalized E-Learning 61
5.6.4 Adaptation Results
With the pretest (and later the HR information) it is clear that the course can be adapted, resulting in
a course containing different number of chapters for each student. The chapter “Explanation” will
only be presented if a student hasn’t taken a course yet. In other words if the student has a profile
already this chapter can be skipped.
For the following two chapters “What is information security” and “status within the company”
adaptation will be harder. If somebody is familiar with information security the entire course can be
skipped together with these pages.
For the latter, the status of the company is explained to raise questions to the student. It can be
skipped, but again this has to do with the didactics of the module. If the student follows most of the
course, it is better to leave this chapter intact.
The other chapters (conclusion, relevant links and contacts), except for the golden rules chapters,
and the selftest described above, will not be adapted.
The following diagram gives an overview of how in the first implementation the adaptation will be
applied throughout the course after analyzing the HR attributes, and assuming that there is only a
correct list of diplomas and certificates as stated in chapter 5.6.2.1, for which a student can skip the
entire course and starts immediately with the selftest. In this diagram the method explained in
Diagram 11 is used. Notice that this is a different implementation than chosen in the next module in
AHA! with only the pre-test and no HR adaptation.
Personalized E-Learning 62
Diagram 13 Adaptation Process
*In the diagram example the selftest is failed by the student on the questions about chapter 2,3, and 5. Only these parts of the course need to be repeated.
**Update Profile: actually the profile is possibly updated after every step, and not only at the end of the module.
5.6.5 Scenario’s
The fictional company JOHANSSON has over 100.000 employees, every employee has to have
sufficient knowledge about information security and therefore all the employees need to follow the
adapted Basic Module (Information) Security.
Adam, the chief information officer of the company, who just graduated for his CISSP certificate has
all the knowledge about information security you expect him to have.
Personalized E-Learning 63
Rachel, the secretary of Adam, has worked for him for several years and has quite some experience
working with computers and dealing with company secrets.
Joey is working in the sales department of the company, but has a background in Computer Science.
35 years ago he graduated in Computer Science, but he found a job in a complete different direction,
so his knowledge about information security is pretty out-of-date.
5.6.6 Time Benefits
The time benefits that are booked with the adaptation are hard to predict, but different scenario’s
can be sketched. In total there are 1025 several routes to go through the course (go to the selftest
immediately, or visit all the obligated pages together with the 10 optional pages (explanation, golden
rule 1 through 9), 102.).
The best case scenario is Adam, according to his HR information he’s familiar to the content and
apparently he was. He made the selftest with absolutely no mistakes.
Rachel had followed a different course before, so the explanation part was not necessary for her, and
according to her answers of the pretest, she was familiar to golden rules 3,4,5, and 9. At the end she
successfully ends the selftest.
The worst case scenario is Joey, according to his HR information he should be familiar to all the
information, but according to his selftest, he wasn’t familiar to any of the subjects what so ever.
Therefore he had to do the entire course, after failing his selftest. After finishing the course, Joey
successfully ended the selftest. Assuming Joey would have succeed for the selftest the first time, if he
first studied the course, the adapted course took more time than the non adapted course. But this
scenario will almost never occur.
Adapted Course (sec) Non-adapted Course (sec) Time benefit (sec)
Adam 352 2528 2176 (±36 min)
Rachel 1872 2528 656 (±11 min)
Joey 2801 2528 -273 (±-4,5 min)
Table 25 Time Benefits Scenario's
These time benefits are based on the HR adaptive module (assuming that there is a correct list of
diplomas and certificates as stated in chapter 5.6.2.1, for which a student can skip the entire course
and starts immediately with the selftest.). In case of the pretest adaptive module the adapted course
will take a little more time for the pretest questions. Possibly therefore the result of Joey will be
better. Also if a better analysis of the HR attributes is done, it is very well possible that Joey will not
pass for skipping the entire course, but only parts of it.
5.6.7 Conclusion
Adaptation can be applied for this course, but it is an introduction course, so most employees, will
probably follow the complete course. There will be gaining of time thanks to the adaptation, but
concrete numbers are not available till some testing with employees is done. However in the
previous chapter becomes clear that in the best case scenario 36 minutes is saved, and in an average
scenario 11 minutes. In a few cases the adapted module will take a little more time, but this is only
the case if the selftest needs to be done more than once.
The course will not change that much, only the sequence of presented information and exercises will
change. Therefore it is guaranteed that the module is still of high standard.
Personalized E-Learning 64
6 Module in AHA!
In the previous chapter it is clearly explained for which parts of the module adaptation is possible. In
this chapter the structure of the adaptive module is explained. How to build an adaptive model with
the help of AHA! and how to build this exact module is explained in this chapter.
The adaptive AHA! module is build for test purposes. Therefore the module works with pretest
questions and adapts according to the answers of them. After enough students have followed this
adaptive course, similarities between the pretest answers and the HR attributes of the student can
be found. But this is not the first test purpose. The main test purpose is to find out if the pretest
adaptive module has sufficient time benefits in comparison with the non-adapted module. The
adaptive module is followed by a test group that has never followed this module before, therefore
no additional information about the students is known, apart from the information that they entered
at the startup screen. Another similar test group will follow the non adaptive version of the course.
This course is also rebuild in AHA!. The building of the non adaptive AHA! module will not be
explained, because building this version is similar to building the adaptive version, except the
concept structure is less complicated.
6.1 Process of the Adaptive Module
In this subchapter first the process of the specific adaptive module is described. Afterwards this
process is visualized with a diagram that is suitable for all adaptive modules of this kind. This diagram
will be explained and initialized for this specific module.
This adaptive module is build with nine pretest questions, that are all directly linked with the nine
golden rules (as in Diagram 12). According to these answers the module is constructed. Possibly the
student can immediately go to the selftest, in case he answers all the pretest questions correctly. In
the other cases the according information (the golden rules) based on the pretest answers is shown,
before the student can make his selftest.
After the selftest (12 questions) is succeeded (at least 10 questions correct), the last two pages with
additional link and contact information are shown. If the students fails for his selftest, the module is
rebuild with only the according pages based on the selftest answers. After studying the pages again
the selftest must be done again. Only the selftest questions that were answered incorrectly the first
time are asked again (an adaptive selftest). If the student answers more than 9 questions correct
(together with his previous attempts) the selftest is succeeded, otherwise the module is again rebuild
with only the according pages based on the selftest answers. This process continues until the student
has answered at least 10 questions correct from the selftest.
This process is visualized with the help of
Diagram 14. This diagram describes the process of an adaptive module that uses pretest questions
and a selftest. The explanation and the initialization of the diagram for this specific adaptive module
is given next.
Personalized E-Learning 65
Diagram 14 Adaptive process generic module
First the initialization of the module needs to be done. In case of this module, the number of pretest
questions (N) is set to 9 and the number of selftest questions (M) is set to 12. The other variables;
number of pretest questions answered correctly (PQT), number of selftest questions answered
correctly (STT), and a Boolean variable that determines if the selftest is made at least once (m) are all
set to 0 at initialization.
After visiting the pages “Explanation Module” and “Explanation Pretest” (which can be replaced with
different pages in a different module of course) the temporarily variable n makes sure all pretest
questions are asked and the Boolean function PQ(n) determines if question n is answered correctly. If
all 9 (n<N) pretest questions are asked the process continues. In case all these questions are
answered correctly (PQT=N) the selftest starts, otherwise the pages “What is IS”, “Status within
KPN”, and “About Golden Rules” are visited before the temporarily variable i determines which pages
(in this case golden rules) have to be visited and which pages can be skipped. This decision factor is
based on the answers of the pretest questions (PQ(i)), the answers of the selftest questions (ST(i)),
and the Boolean variable m. The relationships between those are given in the next table for the
current module.
i PQs(i) (Bool) Q(i) (Bool) Page(i) (const)
1 PQ(1) ST(1) AND (ST9) Golden rule 1
2 PQ(2) ST(2) AND (ST10) Golden rule 2
3 PQ(3) ST(7) Golden rule 3
4 PQ(4) ST(3) Golden rule 4
5 PQ(5) ST(5) Golden rule 5
6 PQ(6) ST(8) Golden rule 6
7 PQ(7) ST(11) Golden rule 7
8 PQ(8) ST(12) Golden rule 8
9 PQ(9) ST(4) AND ST(6) Golden rule 9
Table 26 Initializing table for current module
Personalized E-Learning 66
This table differs for every adaptive module, but in the current case the nine pages are the nine
golden rule pages and the pretest questions are all directly linked to the golden rules. Actually in this
case PQs(i) is the same as PQ(i), but if more pretest questions are used for one golden rule, this will
not be the case. Like with the selftest questions, multiple questions relate to one golden rule. F.i.
Selftest question 4 and 6 need to be answered correctly, only then golden rule 9 can be skipped. The
decision to visit page(i) depends on Q(i) if the selftest is done at least once (m=1) and depends on
PQs(i) if the selftest is not yet done (m=0).
After visiting the necessary golden rule pages the explanation of the selftest is given. Temporarily
variable m makes sure all the selftest questions are asked and Boolean function ST(m) determines if
question m is answered correctly. If all 12 (m<M) questions are asked the selftest results are
displayed. In case at least 10 selftest questions are answered correctly (STT>9) the pages
“conclusion” and “relevant links” are displayed before the module is ended. If less questions are
answered correctly the student has to visit some golden rules again (according to Table 26) and then
has to redo the selftest until at least 10 good answers are given.
To accomplishing such a process in AHA! first a conceptual structure needs to be created (see
chapter 6.2).
6.2 Conceptual Structure
Creating a conceptual structure is the most fundamental part of designing an adaptive hypermedia
application. First a concept structure/hierarchy needs to be designed (see chapter 6.2.1), afterwards
the concepts need to be created (see chapter 6.2.2). At last, and most important, the concept
relationships (see chapter 6.2.3) need to be created. The concepts, as well as the concept
relationships can be directly edited into an XML file that contains the application’s concept structure.
In chapter 6.2.4 will be explained for each concept how and which relations and attributes are
implemented.
AHA! has authoring tools (see chapter 6.5) that will make this process easier and will work less time
consuming than editing the XML file manually and leave little room for errors.
Before explaining how to create the conceptual structure, first the meaning of a concept is explained:
• something conceived in the mind : thought , notion ;
• an abstract or generic idea generalized from particular instances;
(Mer08).
AHA! uses concepts to implement the AHA! application on every level. So don’t be confused with the
meaning of the word concept, because AHA! uses concepts on every level, and for every page in
AHA! a concept should be created. Therefore in reference to AHA! it is better to speak of a concept
structure than a conceptual structure. Actually the conceptual structure of the application is given in
Diagram 14 and the concept structure is an overview of all the concepts and their hierarchy.
6.2.1 Design Concept Structure
Each AHA! application consists of a set of concepts. For each page there should be a concept, but
there may also be many other concepts. In the case of the adaptive module each page is linked with
exactly one concept. The concept structure of the adaptive module is the following, with the
concept name between brackets:
o Explanation (infosecure1)
o Pretest Questions (pretest)
Personalized E-Learning 67
• Question 1 of 9 (question1)
• Question 2 of 9 (question2)
• Question 3 of 9 (question3)
• Question 4 of 9 (question4)
• Question 5 of 9 (question5)
• Question 6 of 9 (question6)
• Question 7 of 9 (question7)
• Question 8 of 9 (question8)
• Question 9 of 9 (question9)
o What is Information Security (whatisism)
• Page 1 (whatisis)
• Page 2 (whatisis2)
o Status within KPN (statusm)
• Page 1 (status)
• Page 2 (status2)
o About the golden rules (about)
o Golden Rule 1 (goldenrule1m)
• Page 1 of 2 (goldenrule1)
• Page 2 of 2 (goldenrule12)
o Golden Rule 2 (goldenrule2m)
• Page 1 of 4 (goldenrule2)
• Page 2 of 4 (goldenrule22)
• Page 3 of 4 (goldenrule23)
• Page 4 of 4 (goldenrule24)
o Golden Rule 3 (goldenrule3m)
• Page 1 of 3 (goldenrule3)
• Page 2 of 3 (goldenrule32)
• Page 3 of 3 (goldenrule33)
o Golden Rule 4 (goldenrule4m)
• Page 1 of 3 (goldenrule4)
• Page 2 of 3 (goldenrule42)
• Page 3 of 3 (goldenrule43)
o Golden Rule 5 (goldenrule5m)
• Page 1 of 3 (goldenrule5)
• Page 2 of 3 (goldenrule52)
• Page 3 of 3 (goldenrule53)
o Golden Rule 6 (goldenrule6m)
• Page 1 of 3 (goldenrule6)
• Page 2 of 3 (goldenrule62)
• Page 3 of 3 (goldenrule63)
o Golden Rule 7 (goldenrule7m)
• Page 1 of 2 (goldenrule7)
• Page 2 of 2 (goldenrule72)
o Golden Rule 8 (goldenrule8m)
• Page 1 of 2 (goldenrule8)
• Page 2 of 2 (goldenrule82)
o Golden Rule 9 (goldenrule9m)
• Page 1 of 3 (goldenrule9)
• Page 2 of 3 (goldenrule92)
• Page 3 of 3 (goldenrule93)
Personalized E-Learning 68
o Selftest (selftest)
• Question 1 of 12 (selftest1)
• Question 1 of 12 confirmation (selftest1c)
• Question 2 of 12 (selftest2)
• Question 2 of 12 confirmation (selftest2c)
• Question 3 of 12 (selftest3)
• Question 3 of 12 confirmation (selftest3c)
• Question 4 of 12 (selftest4)
• Question 4 of 12 confirmation (selftest4c)
• Question 5 of 12 (selftest5)
• Question 5 of 12 confirmation (selftest5c)
• Question 6 of 12 (selftest6)
• Question 6 of 12 confirmation (selftest6c)
• Question 7 of 12 (selftest7)
• Question 7 of 12 confirmation (selftest7c)
• Question 8 of 12 (selftest8)
• Question 8 of 12 confirmation (selftest8c)
• Question 9 of 12 (selftest9)
• Question 9 of 12 confirmation (selftest9c)
• Question 10 of 12 (selftest10)
• Question 10 of 12 confirmation (selftest10c)
• Question 11 of 12 (selftest11)
• Question 11 of 12 confirmation (selftest11c)
• Question 12 of 12 (selftest12)
• Question 12 of 12 confirmation (selftest12c)
• Results (results)
o Conclusion (conclusion)
o Relevant links and contact (links)
The above table describes the concept structure of the adaptive module. As described in chapter
6.1, some concepts will not be showed in the menu, depending on the answers of the pretest
questions. This will be accomplished by creating concept relationships (see chapter 6.2.3). Every
concept is (in this case) linked to a page. This is described in detail in the next subchapter.
6.2.2 Creating Concepts
As explained above in this case every concept is linked to a page. A concept contains elements, the
necessary elements will be explained in this chapter. Each concept has a unique name (in AHA! this
must be a single word, alphanumeric and starting with a letter), a description, a resource, and a
concepttype. Description speaks for itself, this is a description of the concept. For linking the page to
the concept concepttype is set to “page” for every concept and for resource an URL is specified. For
instance: “file:/infosecure/page1.xhtml”.
Other elements are title and hierarchy, these elements are used to define the menu structure (see
chapter 6.2.2.1). Concepts also have attributes with their own properties (see chapter 6.2.2.2). After
all the concepts are created the xml file will have about 7000 lines and a fraction of this file is given in
chapter 6.2.2.3 to explain the representation in AHA!.
Personalized E-Learning 69
6.2.2.1 Menu Structure
As will be described in chapter 6.4 the module uses the StaticTreeView (standard in AHA!) to
generate a menu. Because of the automatic generation of the menu AHA! needs additional
information for each concept in the form of elements. Title is an element that holds the title of the
concept, that will be shown in the menu. And the element hierarchy has three sub-elements:
firstchild, nextsib, and parent. In order they describe the first child of the concept, the name of the
next sibling in the hierarchy and the name of the parent concept, if that exists.
6.2.2.2 Attributes
For each concept AHA! stores a number of attributes that may be different for every concept. Every
attribute has a number of properties and a set of adaptation rules. The properties are: name, type,
isPersistent, isSystem, isChangeable, description, default.
Name: the name of the attribute.
Type: the type of the attribute (Boolean, Integer, or String).
IsPersistent: a Boolean value that determines if the value of the attribute is remembered in the
user model or is only stored temporarily during the rule execution.
IsSystem: a Boolean value that determines if the attribute is system defined or not (only access
and visibility, these will be explained in chapter 6.2.3.1).
IsChangeable: a Boolean value that determines if the attribute can be changed with the help of
forms.
Description: a description of the attribute.
Default: the initial value of the attribute.
The attributes used for implementing the adaptive module will be explained in chapter 6.2.3.
6.2.2.3 Example .aha file
The complete .aha file which contains the concept structure with all his elements, attributes and
relationships is almost 7000 lines, a small fraction of this code is given below to explain the
representation of the concepts in AHA!:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE conceptList SYSTEM '../generatelist4.dtd'>
<conceptList>
<name>infosecure2</name>
<concept>
<name>infosecure2.infosecure2</name>
<description>An abstract concept to bind the concepts of the module</description>
<resource>file:/infosecure2/infosecure2.xhtml</resource>
<concepttype>page</concepttype>
<title>Adaptive Module InfoSecure</title>
<hierarchy>
<firstchild>infosecure2.whatisism</firstchild>
<nextsib></nextsib>
<parent></parent>
</hierarchy>
<attribute name="visited" type="int" isPersistent="true" isSystem="true"
isChangeable="true">
<description>has this page been visited?</description>
<default>0</default>
</attribute>
<attribute name="suitability" type="bool" isPersistent="false" isSystem="false"
isChangeable="false">
<description>the suitability of this page</description>
<default>true</default>
</attribute>
<attribute name="showability" type="int" isPersistent="true" isSystem="false"
Personalized E-Learning 70
isChangeable="true">
<description>showability this concept</description>
<default>0</default>
</attribute>
<attribute name="hierarchy" type="bool" isPersistent="true" isSystem="true"
isChangeable="false">
<description>the visibility of this concept in the hierarchy</description>
<default>true</default>
</attribute>
</concept>
………
</conceptList>
Figure 6 Example aha file
6.2.3 Concept Relationships
All the concepts described in the concept structure (see chapter 6.2.1) have relationships with each
other. As described in chapter 6.2.2.1 all the concepts have an element hierarchy with the explained
sub-elements to create the correct order of the concepts. The concepts also have an attribute
hierarchy that will make sure the concepts with this attribute set to true will be shown in the menu
(see chapter 6.4). This attribute and others will be explained in more detail in the next subchapter.
Each concept also has adaptation rules, these rules and how they are used to update the attributes
of concepts will be explained in chapter 6.2.3.2.
6.2.3.1 Used Attributes
As described in chapter 6.2.2.2 each concept stores attributes with different properties. AHA! uses
some standard attributes (see next subchapter) and some attributes are created especially for this
module (see chapter 6.2.3.1.2).
6.2.3.1.1 Standard Attributes
AHA! uses some standard attributes like Access, Knowledge, Visited, Suitability, Interest, Showability,
and Hierarchy. The ones used for this module are explained below:
Access: a non-persistent Boolean attribute that temporarily becomes true when the resource
associated with the concept is accessed.
Visited: a persistent integer attribute that counts the number of accesses to a concept.
Suitability: a Boolean attribute (that can be persistent or not) that determines whether the
concept is considered desirable or undesirable. AHA! can change the color of the links
to these concepts if they are desired or not. This feature is not used in this module.
However AHA! also has the option to implement a link to “next suitable concept” and
therefore this attribute is used.
Hierarchy: a Boolean attribute that determines if the concept is part of the hierarchy in the
menu.
6.2.3.1.2 Created Attributes
TempNumber1: For every pretest question Boolean attributes are created (temp1, temp2, temp3
etc.). The attributes determine which answer is given to the question. F.i. if temp2 of
concept question3 becomes 1, then the student answered option 2 for pretest
question3.
TempNumber2: It is also possible that integer attributes are created instead of Boolean ones, this
depends if multiple answer combinations are possible to one question. F.i. question1
consists out of 4 small questions, that each set a different temporarily attribute to a
Personalized E-Learning 71
different value. By adding these values together and compare this with a known
number, only one combination makes sure that all small questions are answered
correctly.
Done: For every pretest and selftest question a Boolean attribute is created that determines
if the question itself is been asked already.
Ff: For every selftest question an integer value is created that determines the answer of
the selftest question. If the value is set to 1 the answer was correct.
Temp1: For every selftest question this Boolean value determines if the selftest question was
answered correctly.
Complete: This Boolean value for selftest12 determines if the selftest is completed or not.
Result: The concepts results has an integer attribute that determines how many correct
answers are given so far to the selftest questions.
All above attributes are changeable, because with the help of forms these attribute values are
changed, and are persistent. The default value for all these attributes is false or 0.
6.2.3.2 Adaptation Rules
The updates to attributes of concepts in the user model are done through event-condition-action
rules. Every rule is associated with an attribute of a concept, and is "triggered" whenever the value of
that attribute changes. Every page has an access attribute which is (virtually) "changed" whenever
the end-user visits that page. This triggers the rules associated with this attribute. Every rule consists
of the following parts: condition, true-actions, false-actions, and propagation.
Condition: when the rule is triggered this Boolean condition is evaluated.
True-actions: when the condition is true, this set of actions is executed.
False-actions: when the condition is false, this set of actions is executed.
Propagation: when a rule is executed it updates some attribute(s) of some concept(s). The
propagation field indicates whether these updates will trigger the rules associated
with the updated attribute(s).
Adaptation rules are used in this module mainly if the access attribute was triggered, the
implementation of these rules and the use of the above explained attributes is explained in the next
subchapter.
6.2.4 Implementing Concept Structure
In the previous chapter is explained which attributes, elements, and adaptation rules can be used, in
this chapter will be explained how they are actually implemented in the concept structure for the
adaptive module.
6.2.4.1 Hierarchy and Suitability
These two attributes are very close to each other, as a matter of fact these two attributes are in this
module exactly the same, because there is no concept that is suitable and not in the hierarchy of the
menu or vice versa. Therefore for every concept that has another value than true for suitability, the
default value for hierarchy becomes suitability. This way it is sure that these two attributes have the
same value. It will also save time and leave little room for errors, if during programming the value of
suitability must be changed manually, the value of hierarchy will change automatically in the same
value.
Personalized E-Learning 72
Because of the adaptive module concepts are only shown (suitable/in the hierarchy) if these are
necessary for the user. And this mainly depends on the answers given to the pretest questions. Also
the order of the questions is determined with the help of the suitability attribute, because every
question needs to be asked and questions cannot be skipped. Therefore the module starts with only
the explanation starting page (this concept suitability is true, therefore always visible and in the
hierarchy) and the pretest explanation page. This concept has his suitability attribute set to:
infosecure1.whatisism.visited == 0 && infosecure1.selftest.visited == 0
Meaning that the concept whatisism (What is Information Security) must not be visited yet and the
concept selftest must not be visited yet. After the pretest is completed the next suitable concept will
be whatisism or selftest (according to yet to explain adaptation rules), therefore this concept
(pretest) will only be visible during the answering of the pretest questions thanks to this suitability
attribute.
Every pretest question is only suitable if it is not asked before and the previous question is asked
before. This is accomplished by setting the suitability attribute for every question. F.i. the suitability
attribute for pretest question2 is as follows:
infosecure1.question1.done && !infosecure1.question2.done
After finishing the pretest questions the next concept in line is whatism. This concept is suitable if
one of the golden rules is suitable or if the selftest is completed and not succeeded. This leads to the
following settings of the suitability attribute:
(infosecure1.goldenrule1m.suitability || infosecure1.goldenrule2m.suitability ||
infosecure1.goldenrule3m.suitability || infosecure1.goldenrule4m.suitability ||
infosecure1.goldenrule5m.suitability || infosecure1.goldenrule6m.suitability ||
infosecure1.goldenrule7m.suitability || infosecure1.goldenrule8m.suitability ||
infosecure1.goldenrule9m.suitability)
||
(infosecure1.results.result < 10 && infosecure1.selftest12.complete)
The concepts whatisis and whatisis2 are children of the concept whatisism and therefore
automatically only suitable if whatisism is suitable, and therefore no suitability attributes need to be
set here.
The concepts statusm and about get the same suitability attribute as whatisism, because the only are
suitable if whatisism is suitable.
The suitability attributes for the golden rules pages are most important, because they depend on the
answers given in the pretest and in case the selftest is complete they depend on the answers given in
the selftest. Therefore every golden rule has a unique suitability attribute, 2 golden rules concepts
are discussed in detail, goldenrule1m and goldenrule3m, because they are slightly different from
each other. The other suitability attributes of the golden rule concepts are constructed in the same
way.
In chapter 6.3.2 will be explained how certain temporarily attributes get certain values, but for now it
is enough to know that temporarily attributes get certain values that make it possible to know how
the pretest and selftest questions are answered (see chapter 6.2.3.1.2).
Concept question1 has among others the integer attributes temp1, temp2, temp3, and temp4. If
those four temporarily integer attributes together are 400 the question is answered correctly and
then concept goldenrule1m is not suitable. Also must be made sure that all pretest questions are
answered and the selftest is not yet made. In case the selftest is made, it must not be succeeded and
in this case question1 or question9 of the selftest must be answered incorrectly (because question 1
Personalized E-Learning 73
and 9 of the selftest are related with golden rule 1). Putting this all together the suitability attribute
of concept goldenrule1m becomes as follows:
(infosecure1.question1.temp1 +infosecure1.question1.temp2 + infosecure1.question1.temp3 +
infosecure1.question1.temp4 != 400 && infosecure1.question9.done &&
!infosecure1.selftest12.complete)
||
(infosecure1.results.result < 10 && infosecure1.selftest12.complete &&
(!infosecure1.selftest1.temp1 || !infosecure1.selftest9.temp1))
Concept question3 has among others the Boolean attributes temp1, temp2, temp3, and temp4. For
this question if temp1 and temp3 are true and the others are not true, than the question is answered
correctly and therefore goldenrule3m should not be suitable. Just as for question 1 it also must be
made sure that all pretest questions are answered and the selftest is not yet made. In case the
selftest is made, it must not be succeeded and in this case question7 of the selftest must be
answered incorrectly (because question 7 of the selftest is related with golden rule 3). Putting this all
together the suitability attribute of concept goldenrule3m becomes as follows:
(!(infosecure1.question3.temp1 && !infosecure1.question3.temp2 && infosecure1.question3.temp3
&& !infosecure1.question3.temp4) && infosecure1.question9.done &&
!infosecure1.selftest12.complete)
||
(infosecure1.results.result < 10 && infosecure1.selftest12.complete &&
!infosecure1.selftest7.temp1)
After the golden rules the selftest follows, the introduction page of the selftest is only showed if the
pretest is done, therefore the suitability attribute is:
Infosecure1.question9.done
The following selftest questions must be shown in order. Important is that no questions can be
skipped (therefore only one question at a time can be shown in the menu), which leads to a rather
long value for the suitability attributes of the selftest questions.
For selftest question 1 this is the following:
!infosecure1.selftest1.temp1 && !infosecure1.selftest1.done
The temp1 value is set in selftest1c, so this makes sure that the confirmation of the question is not
yet done and that the question isn’t asked already.
Selftest1c has his suitability attribute set to:
infosecure1.selftest1.done && !infosecure1.selftest1c.done
This makes sure the confirmation is not yet done and the selftest question is. This attribute is the
same for all selftest confirmation pages. The suitability attributes for the selftest questions becomes
longer for every question, because it must check if the previous questions are asked already.
Therefore for selftest2 the suitability attribute is set to:
!infosecure1.selftest2.temp1 && !infosecure1.selftest2.done && infosecure1.selftest1c.done
Until selftest12 that is set to:
Personalized E-Learning 74
!infosecure1.selftest12.temp1 && !infosecure1.selftest12.done &&
infosecure1.selftest1c.done && infosecure1.selftest2c.done && infosecure1.selftest3c.done &&
infosecure1.selftest4c.done && infosecure1.selftest5c.done && infosecure1.selftest6c.done &&
infosecure1.selftest7c.done && infosecure1.selftest8c.done && infosecure1.selftest9c.done &&
infosecure1.selftest10c.done && infosecure1.selftest11c.done
After all the questions are answered a result page comes up. The suitability attribute for this page is
set to the following, because all questions must be answered:
infosecure1.selftest1c.done && infosecure1.selftest2c.done && infosecure1.selftest3c.done &&
infosecure1.selftest4c.done && infosecure1.selftest5c.done && infosecure1.selftest6c.done &&
infosecure1.selftest7c.done && infosecure1.selftest8c.done && infosecure1.selftest9c.done &&
infosecure1.selftest10c.done && infosecure1.selftest11c.done && infosecure1.selftest12c.done
The last two pages (conclusion and links) are only displayed if the pretest is done and the selftest is
succeeded, this can easily be confirmed by:
infosecure1.question9.done && infosecure1.results.result > 9
In all the above values of suitability attributes are other attributes involved like results, done, and
complete that have a different value depending on time and answers given. How these values change
is explained in the next subchapter and in chapter 6.3.2 where is explained how adapted pages are
written and how these values change with the help of forms.
6.2.4.2 Access
Adaptation rules are used in this module if the access attribute of a concept is triggered. Every time
the confirmation of a selftest question is accessed, there is an adaption rule triggered that checks if
the result of the selftest is correct. If a question is answered correctly, the end result increases with
1. For selftest4c this leads to the following in the xml-file:
<concept>
<name>infosecure1.selftest4c</name>
…
<attribute name="access" type="bool" isPersistent="false" isSystem="true"
isChangeable="false">
<description>triggered by page access</description>
<default>false</default>
<generateListItem isPropagating="true">
<requirement>infosecure1.selftest4.ff==1</requirement>
<trueActions>
<action>
<conceptName>infosecure1.results</conceptName>
<attributeName>result</attributeName>
<expression>infosecure1.results.result + 1</expression>
</action>
</trueActions>
</generateListItem>
…
</attribute>
</concept>
For further references it will be described in pseudo-code as follows:
IF infosecure1.selftest4.ff == 1 THEN
infosecure1.results.result := infosecure1.results.result + 1
After the selftest the results are displayed. When these results are displayed most attributes need to
be reset, because possibly a student has to redo parts of the test. A student only has to redo the
questions of the selftest for which he fails, therefore if the value of the temp1 attribute of the
selftest questions equals false, the done attribute of the selftest and selftest confirmation concept
Personalized E-Learning 75
needs to be set to false, so the question will be repeated during the next selftest. Also the value of ff
needs to be set to zero again.
At this point the selftest is completed, so this attribute needs to be set to true. This will create the
following adaptation rule: IF infosecure1.selftest1.temp1 == false THEN {infosecure1.selftest1.done=false
;infosecure1.selftest1c.done=false;infosecure1.selftest1.ff=0}
IF infosecure1.selftest2.temp1 == false THEN {infosecure1.selftest2.done=false
;infosecure1.selftest2c.done=false;infosecure1.selftest2.ff=0}
IF infosecure1.selftest3.temp1 == false THEN {infosecure1.selftest3.done=false
;infosecure1.selftest3c.done=false;infosecure1.selftest3.ff=0}
IF infosecure1.selftest4.temp1 == false THEN {infosecure1.selftest4.done=false
;infosecure1.selftest4c.done=false;infosecure1.selftest4.ff=0}
IF infosecure1.selftest5.temp1 == false THEN {infosecure1.selftest5.done=false
;infosecure1.selftest5c.done=false;infosecure1.selftest5.ff=0}
IF infosecure1.selftest6.temp1 == false THEN {infosecure1.selftest6.done=false
;infosecure1.selftest6c.done=false;infosecure1.selftest6.ff=0}
IF infosecure1.selftest7.temp1 == false THEN {infosecure1.selftest7.done=false
;infosecure1.selftest7c.done=false;infosecure1.selftest7.ff=0}
IF infosecure1.selftest8.temp1 == false THEN {infosecure1.selftest8.done=false
;infosecure1.selftest8c.done=false;infosecure1.selftest8.ff=0}
IF infosecure1.selftest9.temp1 == false THEN {infosecure1.selftest9.done=false
;infosecure1.selftest9c.done=false;infosecure1.selftest9.ff=0}
IF infosecure1.selftest10.temp1 == false THEN {infosecure1.selftest10.done=false
;infosecure1.selftest10c.done=false;infosecure1.selftest10.ff=0}
IF infosecure1.selftest11.temp1 == false THEN {infosecure1.selftest11.done=false
;infosecure1.selftest11c.done=false;infosecure1.selftest11.ff=0}
IF infosecure1.selftest12.temp1 == false THEN {infosecure1.selftest12.done=false
;infosecure1.selftest12c.done=false;infosecure1.selftest12.ff=0}
infosecure1.selftest12.complete = true
In chapter 6.3.2 will be explained how the values of these attributes are changed.
6.3 Write Pages
In this chapter is explained how the actual pages are written. First is explained how the pages are
written that are exactly the same as in the non adapted version (see chapter 6.3.1). Afterwards is
explained how in some pages adaptation is encoded and why (see chapter 6.3.2).
All the pages are rewritten, because the styling of the original pages was done within the pages,
instead of using stylesheets. Also paragraphs (<p></p>) and divisions (<div></div>) are used, because
they are easy to adapt with the help of stylesheets. For more information about stylesheets, visit the
website of World wide web Consortium (W3C09). No information about writing XHTML is given in
this document, for more information, visit the same website (XHT09).
6.3.1 Write Standard Pages
As described in chapter 3 the import SCORM to AHA! program didn’t function as planned and
therefore the AHA! module needed to be build from scratch. While copying and pasting the pages
from the HTML pages to the for AHA! required XHTML pages, a few errors occurred. The HTML pages
used some tags with no end-tag, which is required for XHTML. E.g. <BR> was frequently used instead
of <BR></BR> or <BR />.
Another problem was the nesting of the tags. In HTML there is automatic endings of tags, and
therefore the following codes works fine:
<UL>
<LI>Option 1
<LI>Option 2
<LI>Option 3
<UL>
<LI>Suboption 1
<LI>Suboption 2
</UL>
<LI>Option 4
Personalized E-Learning 76
<LI>Option 5
But it has to be properly nested to work fine in XHTML, as shown in the next table:
<UL>
<LI>Option 1</LI>
<LI>Option 2</LI>
<LI>Option 3</LI>
<UL>
<LI>Suboption 1</LI>
<LI>Suboption 2</LI>
</UL>
<LI>Option 4</LI>
<LI>Option 5</LI>
</UL>
Another problem occurred while copying the text from the html source directly to the xhtml source.
Often the error: “invalid byte 1 of 1-byte utf-8 sequence” was displayed instead of the text. This error
occurs because rich characters, e.g. opening and closing quotations, were used in the text. These
characters needed to be replaced with the standard characters.
Because of these differences between HTML and XHTML basically all the pages were manually
rewritten, to make sure no errors were made. Also extra pages were necessary for the correct
adaptation, these pages will be explained in the following subchapter.
6.3.2 Write Adapting Pages
Most pages with adaptation were especially written and were no part of the non-adapted module,
like the pretest questions (see chapter 6.3.2.1) and the selftest (see chapter 6.3.2.3). Other pages
were only a little adjusted for a better learning experience, like the golden rules page (see chapter
6.3.2.2). In the next subchapters specific pages of the adaptive module are explained, with this
explanation it should be clear how to create your own adapting AHA! pages.
6.3.2.1 Pretest Questions
The pretest questions are asked at the beginning of the module. There are nine questions, that
means nine concepts, and nine pages. Each page is more or less the same, so by explaining two
pages (question1.xhtml and question3.xhtml) of respectively concept question1 and question3 the
principle of creating pretest questions becomes clear. Only the specific adaptations parts are
explained, the basic (X)HTML syntax is not explained. For creating the questions a form is created,
within this form AHA! attributes can be used, therefore the following is possible:
<form method="post" action="/aha/ViewGet/FormProcess?redirect=true">
<p>Fill in the correct terms. Use all terms exactly once.<br /><br />
<ul>
<li>The <select name="Element1.question1.temp1" size="1" >
<option value="0" name="0"> </option>
<option value="1" name="1">KPN company code</option>
<option value="100" name="2">KPN sub codes</option>
<option value="2" name="3">KPN key values</option>
<option value="3" name="4">KPN collective labour agreement</option>
</select> are concrete rules of behaviour on competition, integrity, safety and inside
knowledge.</li><br /><br />
<li>The <select name="Element1.question1.temp2" size="1" >
<option value="0" name="0"> </option>
<option value="100" name="1">KPN company code</option>
<option value="1" name="2">KPN sub codes</option>
<option value="2" name="3">KPN key values</option>
<option value="3" name="4">KPN collective labour agreement</option>
</select> describes that basic principles we apply within KPN in our daily
activities.</li><br /><br />
Personalized E-Learning 77
</ul>
</p>
<p class="nextlink"><input type="submit" value="Next Question"> </input>
</p>
<input type="hidden" name="Element1.question1.done" value="true"></input>
</form>
Table 27 Fraction of question1.xhtml
After submitting this form, by clicking on the “next question” button the attributes
infosecure1.question1.temp1 and infosecure1.question1.temp2 will be set to 0,1,2,3, or 100
depending on the answer selected. After submitting also the attribute infosecure1.question1.done
will be set to true thanks to the hidden input tag. With this information it is possible to adapt the
module as described in chapter 6.2.4.1.
The form will be sent to the internal FormProcess page of AHA!, this page will process the form in a
way AHA! can work with all the values of the attributes. Without the addition “?redirect=true” in the
action tag of the form a confirmation page is given, but with this addition, no confirmation is given
and the next suitable concept will show up in the same frame. In this case the next question.
Question 3 differs a little from question 1 because instead of selecting an answer, checkboxes are
used so multiple answers can be selected:
<p>
How can you best send confidential information by email? Select the correct answer(s)<br
/><br />
<input class="checkbox" type="checkbox" name="element1.question3.temp1" value="true"> Call
the receiver in advance</input><br />
<input class="checkbox" type="checkbox" name="element1.question3.temp2" value="true"> Via my
hotmail account</input><br />
<input class="checkbox" type="checkbox" name="element1.question3.temp3" value="true">
Encrypted</input><br />
<input class="checkbox" type="checkbox" name="element1.question3.temp4" value="true"> Does
not matter, as long as it is sent via the XP-working station</input>
</p>
This way the attributes infosecure1.question3.temp1 to infosecure1.question3.temp4 will be set to
either false or true, depending on selection.
6.3.2.2 About the Golden Rules
The page “about the golden rules” gives a summary about the golden rules. In case of the adaptive
test some golden rules are more important than others to specific users. Depending on the pretest
answers some golden rules can be skipped and therefore they need less attention on this page.
Therefore the important golden rules are displayed in bold, and the other rules just in plain test.
Because these rules differ per student adaptation is necessary. This adaptation is done with the help
of conditional fragment within the page. These conditional fragments let you conditionally include or
hide parts of a page. The condition is an expression using attributes of concepts and constants. In this
case the fragment looks like this:
Personalized E-Learning 78
<table border="0" width="100%">
<if expr="infosecure1.goldenrule1.suitability">
<block><tr><td width="40"><b>Rule 1: </b></td><td><b>Keep to the law and KPN's rules of
conduct</b></td></tr></block>
<block><tr><td width="40">Rule 1: </td><td>Keep to the law and KPN's rules of
conduct</td></tr></block>
</if>
<if expr="infosecure1.goldenrule2.suitability">
<block><tr><td width="40"><b>Rule 2: </b></td><td><b>Allot the correct classification to
company information</b></td></tr></block>
<block><tr><td width="40">Rule 2: </td><td>Allot the correct classification to company
information</td></tr></block></if>
…
</table>
This way the explanation of the golden rule is displayed bold if it’s concept is suitable.
6.3.2.3 Selftest
Before the selftest starts, a page with the explanation of the selftest is given. This page is also an
adapted page. If some golden rules are not read by the student, and these golden rules were advised
to read according to the pretest, the student will be reminded of that. If a student answers all pretest
questions correctly, he will immediately go to this page. In this case this will be displayed on the
page. If a student already made his selftest and he has to do the selftest again, he won’t need the
same explanation of the test as before. In this case a different (shorter) explanation of the selftest is
given. This page will look different depending on a lot of attributes, therefore a lot of conditional
fragments are used. Conditional fragments are also used inside other conditional fragments. To make
this more clear the complete page selftest.xhtml is given in Appendix A.
Some expressions within the conditional fragments contain & or < or > instead of &, <, or
>. Unfortunately this is necessary, otherwise it will confuse the XML parser.
As is visible in selftest.xhtml the “next page” button is also constructed with the help of conditional
fragments. This way the button is constructed in the complete module.
6.4 Look and Feel
Although the original module was in the style of KPN, the adaptive module is in the style of
InfoSecure, according to their demand. Each AHA! application defines its own look and feel. This
consists of a definition of layout, consisting of html frames and which information goes into which
frame, and a specification of concept presentation.
The files “ConceptTypeConfig.xml” and “LayoutConfig.xml” are created in such a way that the
module fits perfectly on a screen width a least a resolution of 800x600 pixels or more. The frame
structure looks like this with the corresponding pixels. If an asterisk instead of a number is given, this
means that the rest of the page is used, this depends per screen resolution:
Personalized E-Learning 79
Both the MainView, and the StaticTreeView are included in AHA!. The MainView automatically
presents the correct pages/concepts. The StaticTreeView automatically generates a menu in the form
of a tree according to the concept structure (see chapter 6.2). If the student clicks on a concept link
in the StaticTreeView, this concept will be shown in the MainView. To make sure these pages are
created in the InfoSecure style a stylesheet is automatically added to all the files and views that are
used in this program. This stylesheet is added to all the files by adding the following line into
LayoutConfig.xml:
<layoutconfig>
<stylesheet>infosecure.css</stylesheet>
</layoutconfig>
Figure 7 Frame Structure
*
Empty
88
*
20
190 610 *
HEADER
Main
View
FOOTER
Static
Tree
View
Empty
Personalized E-Learning 80
The end result looks like this:
Figure 8 Screenshot Module AHA!
6.5 Authoring Tools
Instead of writing all the code manually AHA! offers some authoring tools, which will lighten your
job. The Graph Author (see chapter 6.5.1) and the Concept Editor (see chapter 6.5.2) are both tools
for creating concepts, attributes, and concept relationships. The form editor (see chapter 6.5.3) lets
you create forms through which users can change part of their user model and at last the test editor
(see chapter 6.5.4) which lets you create multiple choice questions. For a complete description of
these authoring tools, visit the AHA! tutorial website (AHA08). In the following subchapters specific
aspects of the tools will be explained, also will be explained why this tool is used or not.
6.5.1 Graph Author
The Graph author is strongly recommended for creating the concept structure of an application. This
high level authoring tool is sufficient for almost all applications and is easy to use. Because of this
Personalized E-Learning 81
reason the Graph Author was first used to create the concept structure. But after creating the
structure the relationships need to established. Therefore the graph author was not ideally for this
specific module. As described in the previous subchapters the suitability and hierarchy attributes
depend on a lot of values and some concepts depend on a lot of other concepts. The graph author
works graphically. So literally all concepts that in one way or another depend on each other have to
be connected to each other with a specific relationship. This would be possible, but you probably
can’t see the forest for the trees anymore. Another important reason for not working with the graph
author anymore is that the module must be fully understand by the author. If the author wants to
make an adjustment in the XML file this must be possible. When working with the Graph author a
.gaf file is created which can be converted in the actual .aha xml file. The .aha file can’t be converted
to a .gaf file, so therefore changes made in the .aha file will not be visible in the Graph Author. And
after using the Graph Author again, changes manually made will be discarded. The Graph Author is
an easy to use tool, but for this purpose and better understanding of the concepts it is not used.
6.5.2 Concept Editor
The concept editor is a low level authoring tool which edits the .aha XML file with the help of concept
templates. It is an easy to use tool, but it leaves more room for errors than the graph author and it
doesn’t work that fast. If you are familiar to XML and if the same changes to a lot of concepts need to
be made, it is easier to work directly in the .aha file with the help of Notepad or another Editor.
While making the current module most changes are made directly in the XML file.
6.5.3 Form Editor
The Form editor is a tool that lets you create forms in an easy way. If you are already familiar with
the HTML forms this editor is unnecessary. But if you are unfamiliar with forms, this editor will make
your job easier. In this adaptive module, the pretest as well as the selftest consist of forms. The form
editor isn’t used while making these questions, because it was easier to make one question and copy
it for the other questions with some small adjustments.
6.5.4 Test Editor
AHA! has a very nice test editor for creating multiple choice questions with some great features, like
time spend, results analysis etc. which was perfectly suitable for this module. The only problem was
that the test made with this editor is a java applet and the author has no control over the
presentation of the test. Therefore all the test are made with forms instead of the test editor,
because that way the tests are in the same style as the rest of the module, which was a requirement
of InfoSecure.
6.6 Other Methods
In the previous subchapters is clearly explained how the module is build and which tools are used. Of
course there are multiple ways to create the same module in AHA!. The used method was that no
new templates or views were created and that AHA! his built-in views and relations were fully used.
Another possibility would have been to create a complete different view for the menu, instead of
using the built-in StaticTreeView. This would have saved time in setting all the suitability and
hierarchy attributes right, but again creating the new view would have cost time.
Another possibility would have been to use the NextView, this view will automatically display a
button to the next suitable page. This would have saved quite some time, because now on every
page the “next page” link is created with the help of conditional fragments. But again a button was
not in the correct style, but adjusting the NextView (and the corresponding ConceptTypeConfig.xml)
in such a way that the button becomes a link is a good option.
Personalized E-Learning 82
There is also the possibility to create templates. In the graph author tool you can create concepts and
concept relationships of different types. The list of available types is based on templates. The
functionality of the graph author can be extended by creating new templates for concept
relationship types and/or changing existing ones.
There are a lot of different methods for creating AHA! applications, no one is better than the other,
but be aware of the possibilities.
7 Extracting Test Data
Before explaining the results of the test in the next chapter, in this chapter will be explained how the
data from the AHA! log files is used, with the help of which programs and why.
7.1 Analyzing the AHA! Logs
In the aha subdirectory xmlroot/log are all the log files of all the persons that have done the test. The
log file is named after the name the user entered before the test started. For Instance John Doe his
log file is named “access_John Doe.xml” and looks like this:
<?xml version="1.0"?>
<!DOCTYPE log SYSTEM 'access.dtd'>
<log>
<user>John Doe</user>
<record>
<accessdate>Thu Sep 04 09:15:00 CEST 2008</accessdate>
<sessionid>6D3B224E69A3BE221E8803D0379032DD</sessionid>
<name>file:/infosecure1/infosecure1.xhtml</name>
<fragment>false</fragment>
</record>
<record>
<accessdate>Thu Sep 04 09:15:20 CEST 2008</accessdate>
<sessionid>6D3B224E69A3BE221E8803D0379032DD</sessionid>
<name>file:/infosecure1/pretest.xhtml</name>
<fragment>false</fragment>
</record>
<record>
<accessdate>Thu Sep 04 09:15:34 CEST 2008</accessdate>
<sessionid>6D3B224E69A3BE221E8803D0379032DD</sessionid>
<name>file:/infosecure1/question1.xhtml</name>
<fragment>false</fragment>
</record>
…
Table 28 Example access_John Doe.xml
As is visible from the example log file it is easy to calculate the time a person takes for each page by
subtracting the access dates from each other. In case of the example John Doe took 14 seconds at
the page “file:/infosecure1/pretest.xhtml”. These calculations can easily be made by a spreadsheet
program (f.i. Microsoft Excel 2007), this is why the xml log files were imported in Excel. An extra
column with the outcome of subtracting the two access dates from each other is created and the
following table is created for every user:
User Access date Name Time
John Doe Thu Sep 04
09:15:00 CEST
2008
file:/infosecure1/infosecure1.xhtml 00:00:20
John Doe Sep 04
09:15:20 CEST
2008
file:/infosecure1/pretest.xhtml 00:00:14
Personalized E-Learning 83
John Doe Thu Sep 04
09:15:34 CEST
2008
File:/infosecure1/question1.xhtml 00:00:22
… … … …
Table 29 Table with extra time column1
These tables (for every user) are imported in a database program (f.i. Microsoft Access 2007) in a
single table so the calculations can easily be made. This table (named p1) has 1720 records and all
the desired calculations with the help of SQL statements can be made.
7.1.1 Correcting Data
With the help of a simple SQL query (“SELECT time, name FROM p1 ORDER BY Kolom1 DESC;”) that
orders all the pages from most time consuming to less time consuming. The few pages that took
some users more than 5 minutes are replaced by the average time a user needed for this specific
page. Assumed is that the user was distracted (phone call etc.) and his time-value was incorrect. The
average time for every page is easily calculated by the following query:
SELECT AVG(SECOND(p1.Time)+60*MINUTE(p1.Time)) AS som, name FROM p1 GROUP BY name;
7.1.2 Data Group 1 vs. Data Group 2
With the help of the following two SQL statements:
SELECT SUM(SECOND(p1.Time)+60*MINUTE(p1.Time)) AS som, p1.[user]
FROM p1
WHERE name LIKE 'file:/infosecure1/*'
GROUP BY p1.[user];
And
SELECT SUM(SECOND(p1.Time)+60*MINUTE(p1.Time)) AS som, p1.[user]
FROM p1
WHERE name LIKE 'file:/infosecure2/*'
GROUP BY p1.[user];
the total time per user per group is displayed (these exact SQL statements are only capable of
calculating times with a maximum of 3599 seconds, which in this is case is more than enough). The
results are displayed in the following two tables, with the names of the persons anonymous.
Som
(sec)
User Som
(sec)
User
469 Person 1A 523 Person 2A
633 Person 1B 567 Person 2B
690 Person 1C 718 Person 2C
877 Person 1D 1355 Person 2D
1 As a matter of fact the actual table contained more columns, such as fragment, sessionid, and some
temporary columns. These columns aren’t used in the calculations and therefore left out in this document for a
better overview.
Personalized E-Learning 84
1015 Person 1E 1374 Person 2E
1107 Person 1F 1421 Person 2F
1166 Person 1G 1450 Person 2G
1256 Person 1H 1527 Person 2H
1586 Person 1I 1583 Person 2I
2018 Person 1J 1588 Person 2J
2206 Person 1K 1624 Person 2K
2318 Person 1L 1683 Person 2L
2401 Person 1M 2707 Person 2M
2653 Person 1N 3363 Person 2N
Group 1 Group 2
1456 Average 1535 Average
Table 30 Timings from all test persons
The figures in these tables are not the final figures, by further analyzing the log files, person 1A, 1B,
1C and 1D only made the pretest and quit the module afterwards, so for calculating the average time
for group 1 these figures need to be discarded. Person 1H didn’t make the final selftest, so his time
his discarded as well. Therefore the average time for group 1 will be 1830 seconds (30,6 minutes).
In group 2 only person 2A didn’t finish the complete module, so only his time is discarded. This gives
group 2 an average time of 1612 seconds (26,9 minutes).
Som
(sec)
User Som
(sec)
User
1015 Person 1E 567 Person 2B
1107 Person 1F 718 Person 2C
1166 Person 1G 1355 Person 2D
1586 Person 1I 1374 Person 2E
2018 Person 1J 1421 Person 2F
2206 Person 1K 1450 Person 2G
2318 Person 1L 1527 Person 2H
2401 Person 1M 1583 Person 2I
2653 Person 1N 1588 Person 2J
1624 Person 2K
1683 Person 2L
2707 Person 2M
3363 Person 2N
Group 1 Group 2
1830 Average 1612 Average
Table 31 Timings from correct test persons
These figures are not the figures that prove the adaptive test is less time consuming than the non-
adaptive test. On the contrary, the adaptive test takes on average almost 4 minutes more than the
non-adaptive test, according to the test results.
As will be described in chapter 8.2 most persons that made the test were all familiar to the content
and therefore the percentage that succeeded for the selftest the first time was pretty high. In the
next table the persons that succeed for the first time (1x), second time (2x) or third time (3x) are
Personalized E-Learning 85
given. There were also two persons that made the entire test, but didn’t succeed for the selftest the
first time, but neglected do to the selftest again (2x*). See next table.
Som
(sec)
User Selftest Som
(sec)
User Selftest
1015 Person 1E 1x 567 Person 2B 1x
1107 Person 1F 1x 718 Person 2C 1x
1166 Person 1G 1x 1355 Person 2D 1x
1586 Person 1I 1x 1374 Person 2E 1x
2018 Person 1J 1x 1421 Person 2F 1x
2206 Person 1K 2x 1450 Person 2G 1x
2318 Person 1L 1x 1527 Person 2H 1x
2401 Person 1M 2x 1583 Person 2I 1x
2653 Person 1N 1x 1588 Person 2J 1x
1624 Person 2K 2x*
1683 Person 2L 1x
2707 Person 2M 2x*
3363 Person 2N 3x
Group 1 Group 2
1830 Average All 1612 Average All
1695 Average 1x 1327 Average 1x
2304 Average 2x 2565 Average 2x* / 3x
The average time for persons that succeed for the selftest the first time is 1695 seconds in group 1
and 1327 seconds in group 2. Group 2, the persons that followed the non-adaptive test, are more
than 6 minutes faster than the persons who followed the adaptive test, and this is not the desired
result. For more information about these result see the next chapter.
The average time for persons that not succeed for the selftest the first time is 2304 seconds in group
1 and 2565 seconds2 in group 2. Group 1, the persons that followed the adaptive test, are 4,5
minutes faster than the persons who followed the non-adaptive test. This is the desired result. For
more information about these result see the next chapter.
2 This number is on the low end, because person 2K and 2M didn’t finish the test, therefore at least 240
seconds can be added to this number.
Personalized E-Learning 86
7.1.3 Pretest Ratio
During the analysis of the results the ratio between the pretest, the actual time persons study the
information pages, and the selftest is necessary, therefore first the average time of the pretest needs
to be established. This can be done with the following SQL statement:
SELECT AVG(SECOND(p1.Kolom1)+60*MINUTE(p1.Kolom1)) AS som, name
FROM p1
WHERE name LIKE 'file:/infosecure1/question*'
GROUP BY name
ORDER BY som DESC;
And the following table is the result.
Som
(sec)
Name
80 file:/infosecure1/question1.xhtml 55 file:/infosecure1/question2.xhtml 18 file:/infosecure1/question3.xhtml 53 file:/infosecure1/question4.xhtml 62 file:/infosecure1/question5.xhtml 37 file:/infosecure1/question6.xhtml 43 file:/infosecure1/question7.xhtml 49 file:/infosecure1/question8.xhtml 66 file:/infosecure1/question9.xhtml
463 Total
Table 32 Average time pretest
By executing a couple of queries the average time per golden rule is calculated as follows:
Average
time (sec)
Golden Rule
66 Golden Rule 1
129 Golden Rule 2
81 Golden Rule 3
99 Golden Rule 4
86 Golden Rule 5
34 Golden Rule 6
38 Golden Rule 7
102 Golden Rule 8
98 Golden Rule 9
733 Total
Table 33 Average time golden rules
The average time of the selftest can be calculated the same way and is:
Average
time (sec)
Selftest
Personalized E-Learning 87
34 Selftest 1
19 Selftest 2
48 Selftest 3
21 Selftest 4
32 Selftest 5
27 Selftest 6
24 Selftest 7
27 Selftest 8
29 Selftest 9
15 Selftest 10
20 Selftest 11
24 Selftest 12
320 Total
Table 34 Average time selftest
7.2 Analyzing the AHA! Profile Logs
In the aha subdirectory xmlroot/profile are all the profile files of the persons that have done the test.
The profile files are named with a number. There is also an index file which has an overview of all the
numbers coupled with the names of the students. This way it is possible to know which profile file
belongs to which student. In these xml formatted files all the latest values of all attributes used are
stored. In this file it is easy to determine which selftest questions were answered correctly (f.i. if the
value of infosecure1.selftest1.temp1 is true, this question was answered correctly) and to find all
latest values of all attributes in the user model.
7.3 Testing the Significance
With the help of two test the significance of the data is tested.
7.3.1 F-test Two-Sample for Variances
The F-test Two-Sample for Variances analysis performs a two-sample F-Test to compare two
population variances. This test can easily be executed with the help of a Excel, this spreadsheet
program has the option to load the analysis toolpak, and with the help of this toolpak the f-test can
be executed. The tool provides the result of a test of the null hypothesis that these two samples
come from distributions with equal variances, against the alternative that the variances are not equal
in the underlying distributions.
The tool calculates the value f of an F-statistic (or F-ratio). A value of f close to 1 provides evidence
that the underlying population variances are equal. In the output table, if f < 1 “P(F<=f) one tail” gives
the probability of observing a value of the F-statistic less than f when population variances are equal,
and “F Critical one-tail” gives the critical value less than 1 for the chosen significance level, Alpha. If
f>1, “P(F<=f) one tail” gives the probability of observing a value of the F-statistic greater than f when
population variances are equal, and “F Critical one-tail” gives the critical value greater than 1 for
Alpha.
7.3.2 T-test Two-Sample
The Two-Sample t-Test analysis tools test for equality of the population means that underlie each
sample. A t-Statistic value, t, is computed and shown as “t Stat” in the output value. If t < 0, “P(T<=t)
Personalized E-Learning 88
one tail” gives the probability that a value of the t-Statistic would be observed that is more negative
than t. If t > 0, “P(T<=t) one tail” gives the probability that a value of the t-Statistic would be observed
that is more positive than t. “t Critical one-tail” gives the cutoff value, so that the probability of
observing a value of the t-Statistic greater than or equal to “t Critical one-tail” is Alpha.
“P(T<=t) two tail” gives the probability that a value of the t-Statistic would be observed that is larger
in absolute value than t. “P critical two-tail” gives the cutoff value, so that the probability of an
observed t-statistic larger in absolute value than “P critical two-tail” is Alpha.
8 Testing Adaptivity
As mentioned in chapter 6, the described module was build with test purposes. A test group (see
chapter 8.1 and 8.2) has followed the adaptive module and another similar test group has followed
the non-adapted module. In chapter 8.3 will be described what were the time benefits for the
adapted version in comparison with the non-adapted version. In chapter 8.4 the HR results will also
be investigated, so be it in a nutshell, because a much larger test-group is necessary for this
investigation.
8.1 Test Group
First of all, the larger the test group, the more reliable the test results will be. Because this will be the
first test, and minor errors are still possible, it is better to start with a test group that is large enough
for correct results, without the risks of showing errors to possible customers. These possible errors
will be detected during this first test. A good number of test persons will be around 40, this is large
enough for reliable information, and good conclusions for testing the pretest adaptation can be
drawn.
The actual test group consists out of InfoSecure employees and customers of InfoSecure. This test
group is divided in two similar test groups. One group follows the non-adaptive version and the other
test group will follow the adaptive version. These two test groups have to be similar, therefore the
InfoSecure employees and the customers of InfoSecure are equally distributed over the two groups.
Because the InfoSecure employees are all well known to InfoSecure, it is possible to divide persons
with the same function over the two groups. This makes sure the two test groups are as similar to
each other as possible. After the tests are done, it will be analyzed in chapter 8.2 if indeed both test
groups were similar to each other.
The groups are divided fifty-fifty, because the results of both tests are equally important. For both
tests the results are extracted from the time taken per page and the sequence order of the pages in
the module. The analysis of the results is completely anonymous. The name (and additional entered
information) of the person will not be used in the primary analysis. The results will be discussed in
chapter 8.3. The additional information entered by the test person (so not his/her name) will be used
for the analysis of the HR results in chapter 8.4. Again this will be a very concise analysis, because the
test group of 25 persons is too little for this purpose.
8.2 Test Group Analysis
25 Persons were invited to make the adaptive test and another 25 to make the non-adaptive test.
Because the test takes around 30 minutes not all persons complied to this invitation, but after
another friendly reminder at least 14 persons in each group made the test and this is sufficient to
draw some first conclusions. The first conclusions that can be made is about the high percentage of
persons that succeed for the selftest. This percentage is high on both test groups. For group 2, the
persons that made the non-adaptive test 10 out of 14 persons succeeded for the selftest the first
time. And in group 1, the persons that made the adaptive test, 7 out of 9 persons that made the
Personalized E-Learning 89
selftest succeeded the first time. These are both high percentages, which possibly are not average,
because most of the test persons are employees of InfoSecure and have an above average
knowledge about Information Security. Therefore the test results in the next subchapter are mainly
based on persons that have sufficient knowledge of Information Security.
8.3 Pretest Results
In this subchapter the results of group 1 (the adaptive test) and group 2 (the non-adaptive test) will
be analyzed. In the previous chapter is explained how the data is extracted and in this chapter the
results are discussed.
The first and most important conclusion is that the desired result of the adaptive test gaining a
overall time benefit is not the case according to the test results. The conclusion can be divided in two
parts. First the persons that immediately succeeded for the selftest (see chapter 8.3.1.1) and
afterwards the persons that didn’t immediately succeed for the selftest (see chapter 8.3.1.2). The
percentages explained in chapter 4.4.2 are discussed in chapter 8.3.1.3 Afterwards a statistical proof
of the results is given (see chapter 8.3.2) and at last the overall conclusion (see chapter 8.3.3).
8.3.1.1 Persons that succeeded for selftest first time
As described in chapter 7.1.2 the average time for persons that succeed for the selftest the first time
is 1695 seconds in group 1 and 1327 seconds in group 2. The test group consisted out of mainly
InfoSecure personnel and they were familiar to most of the topics. Therefore some persons (f.i.
person 2B and 2C) were able to literally click through the information (only briefly observing the
information, and sometimes on own insights take a little more time to really study the information),
and immediately succeed for the final selftest. The persons in group 1 had to follow the pretest. With
this pretest still a high percentage (7 out of 9) succeeded for the selftest immediately, but too much
time was taken for the pretest to get decent percentages as described in chapter 4.4.2.2.
As described in chapter 7.1.3 the ratio for the pretest and the actual information pages is too high. In
the test example in chapter 4.4.1 the ratio is only 240/1350≈0,18 and the current ratio is 0,63.
Therefore the conclusion is that the pretest is taking too much time (in comparison with the other
information pages). People take too much time reading/analyzing the pretest, while it actually is
important that people answer the pretest questions quickly and skip the question quickly if they are
not sure of the answer. This should become more clear to the users of the module, so they take less
time answering the pretest questions.
Some pretest questions on their own can do with little answer alternatives, therefore gaining more
time benefit. The question itself becomes a little easier to answer, but this shouldn’t be a problem if
users only answer in case the know the correct answer for sure.
The ratio is too high, also because the test persons didn’t take as much time as expected for every
golden rule. F.i. the average time of golden rule 7 is 38 seconds, while the business clip alone has a
duration of 93 seconds, not to speak of another complete page with textual information. It is clear
that not all users view the business clip and do not completely read the information on the pages.
That being said the conclusion is that persons that are familiar to the information (which the majority
of the test persons apparently was) click through the information rather quick and still succeed for
the selftest.
Another problem with the current module is that the subject is not too technical or specific, and with
some common sense most questions can be answered. Therefore possibly questions that ask for the
knowledge estimation instead of questions that actually test the knowledge of the student are
necessary. Important in this case is that the student has to estimate his own knowledge really
well/honest. If not, he will fail for the selftest and will definitely make no time benefit. In case of a
more technical module a pretest with knowledge questions is the way to go. In this case the
questions are too difficult to answer if the student is not familiar with the content and easy to
answer if the student is familiar with the information. Therefore the pretest will not take too much
time.
Personalized E-Learning 90
To summarize the pretest in his current form does not guarantee a time benefit for persons that are
familiar to most of the information because:
1) Persons that are familiar to all the information are able to click through the information
rather quick, and make more time profit than with the help of this pretest.
2) The pretest takes too much time in comparison with the information pages. The ratio is
around 63% and ideally should be around 10%. The solution for creating a better pretest is:
a. Make sure the people know that the pretest is not the selftest and that incorrect
answers are no problem. People have to really know the answers to the pretest
question, otherwise just skip it and save time;
b. Shorten the length of the pretest questions, by changing the question or the number
of answer alternatives.
c. Change its form depending on the module (technical or not)
8.3.1.2 Persons that didn’t succeed for selftest first time
Persons that didn’t succeed for the selftest the first time, have a time benefit with the adaptive test.
2206 seconds vs. 2565 seconds. This is because the selftest is adaptive as well and questions of the
selftest that were rightly answered the first time, aren’t asked again in the selftest the next time.
Another great advantage of the adaptive module is that only the golden rules related to the wrongly
answered selftest question are repeated in the module the second (or even third or more) time. A
big side note is that the test data for persons that didn’t succeed for the selftest the first time is
rather small, so an extra investigation is necessary to validate this outcome a 100%. But expected is
that for people not too familiar with all the content, a pretest makes a solid time benefit, especially
when the ratio of the pretest to the information pages as described above becomes smaller.
8.3.1.3 Success and Pre-test Question Correctness Percentage
The pretest correlation percentage and the success percentages (see chapter 4.4.2.2) are highly
depend on the time of the pre-test questions. Since all the questions of the pretest are taking too
much time in comparison with the Information Pages and the selftest, it’s no use to calculate these
percentages, because even without calculation it is clear that the percentages will be too high or (in
case the pretest takes more time than the Information page) negative/impossible. When the pre-test
is adjusted, these percentages need to be calculated as explained in chapter 4.
8.3.1.4 Selftest Adjustments
As explained in chapter 4.4.2.3, there is possibly a group of students that succeed for a pretest
question and fail for the corresponding selftest question(s), but succeed for the complete selftest.
This is not a desired situation and with the help of the AHA! profiles, this can be analyzed for the
current test sample (see chapter 7.2). Seven persons succeeded for the selftest, and 2 of them
succeeded with answering at least one question of the selftest incorrect, without ever viewing the
according information pages. According to these numbers, it is better to adjust the selftest and let
these specific students redo parts of the test, to improve their knowledge.
8.3.2 Statistical Proof of Results
All the conclusions are based on the fact that the results of the two tests are significant enough. This
of course needs to be proven, and this can be done with the help of the f-test explained in chapter
7.3.1 and t-test explained in chapter 7.3.1.
Personalized E-Learning 91
The results for the f-test of group 1 and 2 with alpha (significance level) equaling 0,05 are:
Group 1 Group 2
Mean 1830 1612,307692
Variance 388202,5 529627,5641
Observations 9 13
F 0,732972614
P(F<=f) one-tail 0,33724584
F Critical one-tail 0,304512355
As explained above, F is 0,73, so this proves that the population variances are not equal, but this
conclusion cannot be drawn from this sample, because the value of P(F<=f) one-tail is much to great
and therefore the population variances of the two groups need to be considered equal. But with
more test results, it might very well be possible that the population variances are different for both
groups.
More important is the equality of the population means, which can be calculated with the help of the
t-test explained in chapter 7.3.1. The results for this test are as follows:
Group 1 Group 2
Mean 1830 1612,308
Variance 388202,5 529627,6
Observations 9 13
T Stat 0,751672923
P(T<=t) one-tail 0,230729075
t Critical one-tail 1,729132792
P(T<=t) two-tail 0,46145815
t Critical two-tail 2,09302405
T Stat is 0,75. Therefore the population means of the two groups are not equal. But again P(T<=t)
one-tail is 0,23 and P(T<=t) two-tail is 0,46. Which both are way too high and therefore the null
hypothesis is rejected. Even though the mean of group 1 is 218 seconds more in this test sample,
this difference is no significant difference because of the high dispersion.
It would be possible to find a significant difference, but therefore the variation needs to be much
lower in the sample data.
8.3.3 Overall Conclusion
Overall the mean of the adaptive test is equal (according to t-test) to the mean of the non-adaptive
test. This has a couple of reasons fully described above. Another reason the adaptive test didn’t gain
time as expected, is actually also described above. The test group consisted out of many persons that
did succeed for the selftest the first time, and consisted out of few persons that didn´t. This ratio is
also a factor of the overall result, because persons that don´t succeed for the selftest the first time
will probably gain time by following the adaptive module.
Personalized E-Learning 92
8.4 HR Results
There are no Human Resources results with this current test, because the test group is too small.
Before analyzing the HR results, first the pretest should be adjusted with the above mentioned
improvements. During this analysis it is also necessary to investigate the HR data of the persons that
did the non adaptive test and clicked through the information really quickly and succeeded for the
selftest (in this case persons 2B and 2C). Because these persons are better of immediately making the
selftest as well.
For the adaptive test analyze the results of the pretest answers with the HR information of the
student as described in chapter 4.4.3.
Personalized E-Learning 93
9 Conclusion
After finishing this thesis multiple conclusions can be drawn, but there are also some conclusions
that cannot be drawn yet according to this research. Before investigating the similarities between the
HR attributes of a student and the answers of his pretest questions as explained in chapter 4, the
current pretest needs to be improved as suggested in chapter 8. The pretest in its current form didn’t
lead to time benefit, this is why it needs to be adjusted. But still even with the adjusted pretest the
question remains if the current module is suitable for this kind of adaptation. Better adaptation is
definitely possible for a more technical module, which actually was the first intention of InfoSecure,
but was changed because of marketing reasons as explained in chapter 2.
Definitely more time benefit can be gained with the help of adaptive E-Learning on technical
subjects, but even on the less technical subjects time benefits can be gained, albeit smaller ones. To
test these time benefits a larger test group than used in chapter 8 is necessary. This is because with a
larger test group, smaller time benefits can be found.
AHA! is a suitable program to develop the adaptive E-learning programs as described in chapter 6,
and because of the automatic creation of logs, it is also ideal for test purposes as described in
chapter 7. In chapter 3 is described that with the help of the navigation and sequencings possibilities
of SCORM it is also possible to create an adaptive e-learning module with a pretest. But
unfortunately this is not personalized, so adjustments in the actual code need to be made to store
values for each student. Eventually with this adjustment it will become possible to link the HR
attributes of the student to the sequencing and navigation possibilities of the module and create the
desired adaptive e-learning module. Using SCORM content doesn’t automatically create log files, so
for test purposes additional code needs to be written as well. Therefore AHA! has the upper hand in
creating and testing adaptive e-learning modules, because of its simplicity and built-in possibilities.
Personalized E-Learning 94
10 References
Advanced Distributed Learning (ADL) SCORM® 2004 3rd Edition Content Aggregation Model (CAM)
version 1.0 [Report]. - 2006.
Advanced Distributed Learning (ADL) SCORM® 2004 3rd Edition Run-Time Environment (RTE) version
1.0 [Report]. - 2006b.
Advanced Distributed Learning (ADL) SCORM® 2004 3rd Edition Sequencing and Navigation (SN)
Version 1.0 [Report]. - 2006c.
Advanced Distributed Learning (ADL) Sharable Content Object Reference Model (SCORM)® 2004 3rd
Edition Overview version 1.0 [Report]. - 2006a.
AHA! Tutorial [Online]. - 2008. - http://aha.win.tue.nl.
Cristóbal Romero Morales Sebastián Ventura Soto, Cesar Hervás Martínez, Paul de Bra Extending
AHA! [Report]. - 2005.
De Bra Paul [et al.] AHA! The Adaptive Hypermedia Architecture [Report]. - 2003.
Derek Stockley [Online]. - 2008. - http://derekstockley.com.au/elearning-definition.html.
Merriam-Webster [Online]. - 2008. - http://www.merriam-webster.com/dictionary/.
W3C's overview of Web style sheets: CSS. [Online] // World Wide Web Consortium - Web
Standards. - 2009. - http://www.w3.org/Style/CSS/.
XHTML 1.0: The Extensible HyperText Markup Language (Second Edition) [Online] // World Wide
Web Consortium - Web Standards. - 2009. - http://www.w3.org/TR/xhtml1/.
Personalized E-Learning 95
Appendices
Appendix A: Selftest.xhtml
<!DOCTYPE html SYSTEM "/aha/AHAstandard/xhtml-ahaext-1.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<link href="stylesheets/infosecure.css" rel="stylesheet" type="text/css" />
<title>Selftest</title>
</head>
<body>
<h1>Selftest</h1>
<div class="left">
<if expr="!infosecure1.selftest12.done">
<block>
<p><if expr="(infosecure1.goldenrule1.suitability || infosecure1.goldenrule2.suitability ||
infosecure1.goldenrule3.suitability || infosecure1.goldenrule4.suitability ||
infosecure1.goldenrule5.suitability || infosecure1.goldenrule6.suitability ||
infosecure1.goldenrule7.suitability || infosecure1.goldenrule8.suitability ||
infosecure1.goldenrule9.suitability) == false">
<block>
You have answered all the pretest questions correctly and therefore immediately can do the
selftest.
</block>
<block>
<if expr="(infosecure1.goldenrule1.visited == 0 &&
infosecure1.goldenrule1.suitability) || (infosecure1.goldenrule2.visited == 0 &&
infosecure1.goldenrule2.suitability) || (infosecure1.goldenrule3.visited == 0 &&
infosecure1.goldenrule3.suitability) || (infosecure1.goldenrule4.visited == 0 &&
infosecure1.goldenrule4.suitability) || (infosecure1.goldenrule5.visited == 0 &&
infosecure1.goldenrule5.suitability) || (infosecure1.goldenrule6.visited == 0 &&
infosecure1.goldenrule6.suitability) || (infosecure1.goldenrule7.visited == 0 &&
infosecure1.goldenrule7.suitability) || (infosecure1.goldenrule8.visited == 0 &&
infosecure1.goldenrule8.suitability) || (infosecure1.goldenrule9.visited == 0 &&
infosecure1.goldenrule9.suitability)">
<block>
You've not read through all the golden rules yet, you're advised to read through the
following golden rules, before taking the test:<br />
<if expr="infosecure1.goldenrule1.visited == 0 &&
infosecure1.goldenrule1.suitability"><block><a href="infosecure1.goldenrule1"
class="conditional">Golden Rule 1</a><br /></block></if>
<if expr="infosecure1.goldenrule2.visited == 0 &&
infosecure1.goldenrule2.suitability"><block><a href="infosecure1.goldenrule2"
class="conditional">Golden Rule 2</a><br /></block></if>
<if expr="infosecure1.goldenrule3.visited == 0 &&
infosecure1.goldenrule3.suitability"><block><a href="infosecure1.goldenrule3"
class="conditional">Golden Rule 3</a><br /></block></if>
<if expr="infosecure1.goldenrule4.visited == 0 &&
infosecure1.goldenrule4.suitability"><block><a href="infosecure1.goldenrule4"
class="conditional">Golden Rule 4</a><br /></block></if>
<if expr="infosecure1.goldenrule5.visited == 0 &&
infosecure1.goldenrule5.suitability"><block><a href="infosecure1.goldenrule5"
class="conditional">Golden Rule 5</a><br /></block></if>
<if expr="infosecure1.goldenrule6.visited == 0 &&
infosecure1.goldenrule6.suitability"><block><a href="infosecure1.goldenrule6"
class="conditional">Golden Rule 6</a><br /></block></if>
<if expr="infosecure1.goldenrule7.visited == 0 &&
infosecure1.goldenrule7.suitability"><block><a href="infosecure1.goldenrule7"
class="conditional">Golden Rule 7</a><br /></block></if>
<if expr="infosecure1.goldenrule8.visited == 0 &&
Personalized E-Learning 96
infosecure1.goldenrule8.suitability"><block><a href="infosecure1.goldenrule8"
class="conditional">Golden Rule 8</a><br /></block></if>
<if expr="infosecure1.goldenrule9.visited == 0 &&
infosecure1.goldenrule9.suitability"><block><a href="infosecure1.goldenrule9"
class="conditional">Golden Rule 9</a><br /></block></if>
</block>
<block>
Now that you have read through all the necessary information, you can do the selftest.
</block>
</if>
</block>
</if>
With the test you can establish for yourself whether you have understood everything, the
level of your knowledge and how you deal with information security in practice.</p>
<p>The test consists of 12 questions, if you have given too many incorrect answers, you will
have failed the test. You will have to do (part of) the test again, after you have finished
(part of) the module again. </p>
<p>If you answered enough questions correct, you have passed the test and your manager will
be informed.</p>
<p>After pressing the "next question" button, it is stated if you have answered the question
correctly or not.</p>
<p>Good Luck</p>
<p class="nextlink">
<if expr="infosecure1.selftest1.suitability"><block><a href="infosecure1.selftest1"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest2.suitability"><block><a href="infosecure1.selftest2"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest3.suitability"><block><a href="infosecure1.selftest3"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest4.suitability"><block><a href="infosecure1.selftest4"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest5.suitability"><block><a href="infosecure1.selftest5"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest6.suitability"><block><a href="infosecure1.selftest6"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest7.suitability"><block><a href="infosecure1.selftest7"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest8.suitability"><block><a href="infosecure1.selftest8"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest9.suitability"><block><a href="infosecure1.selftest9"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest10.suitability"><block><a href="infosecure1.selftest10"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest11.suitability"><block><a href="infosecure1.selftest11"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest12.suitability"><block><a href="infosecure1.selftest12"
class="conditional">next page</a></block>
</if>
</block>
</if>
</block>
</if>
</block>
</if>
</block>
</if>
</block>
</if>
</block>
</if>
</block>
</if>
Personalized E-Learning 97
</block>
</if>
</block>
</if>
</block>
</if>
</block>
</if>.
</p>
</block>
<block>
You are familiar with the selftest, so no explanation is needed. Only the questions are
asked which were wrongly answered the previous time.<br /><br />
Answer enough questions correcly and you will succeed for the test.
<p class="nextlink">
<if expr="infosecure1.selftest1.suitability"><block><a href="infosecure1.selftest1"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest2.suitability"><block><a href="infosecure1.selftest2"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest3.suitability"><block><a href="infosecure1.selftest3"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest4.suitability"><block><a href="infosecure1.selftest4"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest5.suitability"><block><a href="infosecure1.selftest5"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest6.suitability"><block><a href="infosecure1.selftest6"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest7.suitability"><block><a href="infosecure1.selftest7"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest8.suitability"><block><a href="infosecure1.selftest8"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest9.suitability"><block><a href="infosecure1.selftest9"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest10.suitability"><block><a href="infosecure1.selftest10"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest11.suitability"><block><a href="infosecure1.selftest11"
class="conditional">next page</a></block>
<block>
<if expr="infosecure1.selftest12.suitability"><block><a href="infosecure1.selftest12"
class="conditional">next page</a></block>
</if>
</block>
</if>
</block>
</if>
</block>
</if>
</block>
</if>
</block>
</if>
</block>
</if>
</block>
</if>
</block>
</if>
</block>
</if>
</block>
</if>
</block>
</if>.
</p>
</block>
Personalized E-Learning 98
</if>
</div>
<div class="right">
<img src="images/conclusion.jpg"/>
</div>
</body>
</html>