ExamAdmin Draft FDS · PDF fileExamAdmin Draft FDS ... This document details a draft...

ExamAdmin Draft FDSRick Graves

MIS

1

ExamAdmin Draft Functional Design Specification

August 2008

Rick Graves Management Information Services


MIS

2

1. Table of Contents

1. Table of Contents ................................................................................................................ 2

2. Introduction ........................................................................................................................ 3 2.1 Scope Of This Document ......................................................................................................................................... 3 2.2 System Architecture ................................................................................................................................................ 3 2.3 Proposed Functionality............................................................................................................................................ 3

3. System Architecture ............................................................................................................ 4 3.1 ExamAdmin Core Component [ ] .......................................................................................................................... 5

3.1a ExamAdmin Application ............................................................................................................................... 5 3.1b ExamAdmin Database .................................................................................................................................. 5 3.1c ExamAdmin Web Service ............................................................................................................................. 5 3.1d Data Access Layer......................................................................................................................................... 5

3.2 SITS Vision Component [ ]..................................................................................................................................... 6 3.2a SAINT Database............................................................................................................................................ 6 3.2b eVision.......................................................................................................................................................... 6

3.3 Syllabus+ Plugin Module [ ]................................................................................................................................... 6 3.3a Syllabus+ Module ......................................................................................................................................... 6 3.3b UEF File......................................................................................................................................................... 6 3.3c Syllabus+ ...................................................................................................................................................... 6 3.3d IMG File ........................................................................................................................................................ 7

3.4 QMP Plugin Module [ ].......................................................................................................................................... 7 3.4a QMP Module ................................................................................................................................................ 7 3.4b QMWise Web Service................................................................................................................................... 7 3.4c QMP Database ............................................................................................................................................. 7

3.5 Other Plugin Modules [ ]....................................................................................................................................... 7

4. System Functionality ........................................................................................................... 8 4.1 ExamAdmin Core Component [ ] .......................................................................................................................... 8

4.1a Main Menu................................................................................................................................................... 8 4.1b User Access & Restrictions ........................................................................................................................... 8 4.1c Extract Profiles ............................................................................................................................................. 9 4.1d Reports ....................................................................................................................................................... 11 4.1e Manage Published Timetables ................................................................................................................... 11

4.2 SITS Vision Component [ ]................................................................................................................................... 12 4.3 Syllabus+ Plugin Module [ ]................................................................................................................................. 13

4.3a Options for Syllabus+ Implementation ...................................................................................................... 13 4.4 QMP Plugin Module [ ]........................................................................................................................................ 14

4.4a Schedule Exams.......................................................................................................................................... 14 4.4b Transfer Exam Marks to SAINT................................................................................................................... 14

5. Appendix A – AAM Statistics ............................................................................................. 16 5.1 Students Requiring Extra Time .............................................................................................................................. 16 5.2 Additional Requirements....................................................................................................................................... 16


MIS

3

2. Introduction This document details a draft functional design specification for ExamAdmin – an application to enhance the level of automation in the examination scheduling processes at the University of Bradford.

2.1 Scope Of This Document

The functionality described herein is a preliminary attempt to address the problems with and requirements for exam scheduling. Part of the functionality is concerned with integrating QMP, SAINT and the Exam Scheduler (either WASP or Syllabus+) – this functionality is required as part of the JISC project, and would require development in any case. The remaining functionality is intended to replace (and improve) what was expected from WASP (that it appears currently to be unable to deliver), whilst continuing to use Syllabus+.

This is a draft specification, and so any comments on the design or requirements of the system are encouraged.

2.2 System Architecture

ExamAdmin will comprise of a central core component, which links directly to the SAINT database. This core component will provide all the central functions of the system, and expose its functionality via a web service. The Syllabus+ and QMP modules will be written as separate plugin modules, which talk to the central core component via its web service. This permits a more extensible architecture, where future modules can be written and bolted onto the system, using any web service compatible technology.

The central core component, and the Syllabus+ and QMP plugin modules, will be written in ASP.net (using C#), and hence ExamAdmin will be a browser based application.

2.3 Proposed Functionality

ExamAdmin will provide the following basic functionality:

• Extract examination data from SAINT under the user’s control, exporting it to Syllabus+. • Extract students’ disability requirements from SAINT, exporting them to Syllabus+. • Provide examination reports for each department, detailing the modules and students that will be

examined. • Schedule online assessments in QuestionMark Perception (QMP), derived from the Syllabus+ timetable. • Transfer online assessment marks from QMP into SAINT. • Produce individual exam timetables for students, that may be viewed online.

Further enhancements to the system, at a later stage of development, could include:

• The production of Exam Board reports. • Scheduling and administration of word processed exams. • Distribution of individual timetables to mobile phones, email or via other electronic means.


MIS

4

3. System Architecture

The schematic below represents how ExamAdmin will link with other elements in the system.

Figure 1 ‐ ExamAdmin Schematic


MIS

5

3.1 ExamAdmin Core Component [ ]

This component is the heart of the system, into which all other modules are “plugged”.

3.1a ExamAdmin Application

This application contains all the core functionality of the system, such as:

• Managing user access and restrictions. • Managing the creation of profiles used to extract examination information from SAINT. • Reporting upon the examination information in SAINT.

It will be written in ASP.net/C# as a browser application.

3.1b ExamAdmin Database

ExamAdmin will have its own database (SQL Server). It will use this to:

• Store user access details – logins and permissions. • Store SAINT extract profiles and templates. • Keep copies of published Syllabus+ timetables. • Track details of exams that have been scheduled in QMP, used in the configuration of Sunray cards. • Store definitions of any reports that the system generates. • Anything else that ExamAdmin has to “remember”!

3.1c ExamAdmin Web Service

The Web Service will allow external applications to have access to the core functionality of ExamAdmin. Using this architecture will allow “plugin” modules to be created for the system, thus making it highly extensible from both an in‐house and, especially important since this work will be part of a JISC project, from an external (i.e. other institutions) perspective.

The Web Service architecture allows modules to be written using any technology that supports web services, such as:

• ASP.net using any of Visual Basic, C#, C++ or J#. • Java • Perl • PHP • Python

The two plugin modules that will be written initially for ExamAdmin will be ones that integrate with QMP and Syllabus+. These will be authored in C#.

3.1d Data Access Layer

This is a common layer of code that both the ExamAdmin application and Web Service will use to communicate with the SAINT database and ExamAdmin’s own database. It will deployed on the server as a DLL (Dynamic Link Library), authored in C#.


MIS

6

3.2 SITS Vision Component [ ]

Writing an application that can communicate with a variety of student record systems is prohibitively complex, especially when applied to the goals and timescales of this project. Hence, ExamAdmin will be written to work exclusively with SITS Vision. Since SITS Vision is a market leader in its field, other SITS customers could use and develop ExamAdmin, especially given its web service based extensibility. This should keep JISC happy!

3.2a SAINT Database

The SAINT database is implemented in Oracle. However, SITS vision can also be deployed in SQL Server. Initially the Data Access Layer will be written to accommodate Oracle only, but with built in extensibility to accommodate SQL Server at a later date.

3.2b eVision

eVision, the web interface to SAINT, can be used to display individual student timetables. See section 4.2 on page 12.

3.3 Syllabus+ Plugin Module [ ]

The Syllabus+ Exam Scheduler developed by Scientia, is currently used in the Exams Office to schedule all exams, but with certain limitations. The Syllabus+ plugin module of ExamAdmin will aim to overcome these limitations and provide enhanced automation of the exam scheduling process.

3.3a Syllabus+ Module

The Syllabus+ module will be a browser based application that links to the ExamAdmin web service. It will provide functionality to:

• Transfer examination information from SAINT to Syllabus+ for scheduling. • Transfer students’ disability information from SAINT to Syllabus+. • Transfer the completed exam timetable back into ExamAdmin.

3.3b UEF File

Syllabus+ can import and export files that use UEF (Universal Exchange Format). UEF files are delimited text files, which use a standardised coding format to describe the data they contain. For example, the current SAINT extract, run from a Unix shell script, creates a UEF file that the Exams Office import into Syllabus+.

Whilst transferring data between Syllabus+ and ExamAdmin using intermediate UEF files introduces an extra manual import/export step, the process will normally only be run twice per examination period; once to import the exam data from SAINT, and once to transfer the completed timetable back to ExamAdmin. Also, Syllabus+’s implementation of UEF is comprehensive, so we have complete access to the Syllabus+ data set.

3.3c Syllabus+

The Exams Office currently use version 1.9 of the Exam Scheduler. Under the terms of the licensing agreement with Scientia, Bradford are eligible for free upgrades. Therefore we have two licences for the new version 2.1 that we can deploy in the Exams Office. This new version has an enhanced interface and a few functional improvements, and is backwards compatible with IMG files created in previous versions.


MIS

7

3.3d IMG File

Syllabus+ stores its data in IMG files which can be located on any local or network drive. These files work just like Word documents – i.e. you load an IMG file when you start Syllabus+, do some scheduling/editing, and then save it again. The drawback to this is that two people cannot modify a timetable at the same time. Both people can load the same IMG file, but the person who saves it last will overwrite the other person’s changes.

3.4 QMP Plugin Module [ ]

QMP (QuestionMark Perception) is the University’s online assessment solution.

3.4a QMP Module

The ExamAdmin plugin module for QMP will facilitate:

• The scheduling of online exams in QMP. • The transfer of exam marks from QMP into SAINT.

3.4b QMWise Web Service

QuestionMark provide QMWISe, which is a web service interface to QMP. This is the preferred method of communication with QMP, as direct manipulation of the QMP Database may invalidate support agreements.

3.4c QMP Database

The QMP database is hosted on SQL server.

3.5 Other Plugin Modules [ ]

Other modules for ExamAdmin that could be developed at a later date include:

• The production of Exam Board reports. • Scheduling and administration of word processed exams. • Distribution of individual timetables to mobile phones, email or via other electronic means.

Using the web service approach, these modules can be developed in isolation to the ExamAdmin core, therefore negating the need to recompile and redistribute the core for every enhancement to the system.


MIS

8

4. System Functionality

4.1 ExamAdmin Core Component [ ]

4.1a Main Menu

The Main Menu (shown below) is the gateway to all ExamAdmin functions.

Figure 2 ‐ The Main Menu

It lists all core functions, as well as all installed plugin modules. Clicking on a button will take the user to the relevant page.

Only those areas of the system that a user has access to will be shown.

4.1b User Access & Restrictions

The User Access page (shown below) allows control over which areas of the system a user has access to.


MIS

9

Figure 3 ‐ User Access

Users require a logon to access ExamAdmin. Two options are available here:

• If the user has an LDAP login, this can be used to access the system. • If the user does not have an LDAP login, a user Login and Password can be assigned to them.

Access to all core and plugin features can be assigned via this screen, and this determines what is available to the user on the Main Menu.

4.1c Extract Profiles

Any number of Extract Profiles can be created in the system, although there will usually be one for each examination period. An Extract Profile determines which modules and students should be included for a particular examination period. It allows all the flexibility of the current SQL extract, but via an easy to use interface, as shown below.


MIS

10

Figure 4 ‐ Extract Profiles

The Extract Profile is comprised of two sections:

• The Include section – one or more sets of assessments that are to be examined. • The Exclude section – these are assessments that are not being examined in the examination period, but

would be brought in by the Include section. For example, you could exclude all Management modules that are being taken overseas, or particular modules that have had their exam date moved for whatever reason.

Both sections use the same sets of criteria to define the modules and students to be included/excluded. These criteria allow:

• The use of the “*” wildcard – e.g. MAN* for all Management modules. • “<” and “>” comparators – e.g. Module sequence “<900”. • A “but not” qualifier, e.g. Assessment type is “EX*” but not “EXAMP” or “EXAMV”


MIS

11

4.1d Reports

Initially the system will provide one report that allows the Exams Office and schools/departments to check which modules and students will be examined in a particular period. The report will take its data from a particular Extract Profile (see section above), and will therefore allow correction of SAINT data or amendment of the Extract Profile to be carried out without the need to provide a draft timetable. This means that when the Syllabus+ module gets its data from an Extract Profile, it has already been checked, and should drastically reduce the need for the timetable rescheduling that currently exists.

An example of how the report will be presented is shown below:

Figure 5 ‐ Exam Report

Other reports that might be beneficial to include at a later date include:

• A report that lists all the exams being taken by a particular student. • A report that shows why a particular module or student is not included in an Extract Profile – e.g. There

is no assessment record set up for them, or which condition in the Extract Profile the module/student is breaking.

4.1e Manage Published Timetables

ExamAdmin will also include the facility to manage Published Timetables. The Syllabus+ module (or a plugin module written for another scheduler) will get its data from an Extract Profile to import into Syllabus+. Once Syllabus+ has scheduled the exams, the Published Timetable can be uploaded to ExamAdmin. This allows


MIS

12

students to view their individual timetables, and also allows QMP to access the date, time and room designation for any exam.

The Manage Published Timetables section of ExamAdmin allows an administrator to control which Published Timetables are available in the system.

Although Syllabus+ can produce its own reports, it may well be beneficial to have ExamAdmin producing examination reports in its own right.

4.2 SITS Vision Component [ ]

Rather than have another bespoke system to display online student timetables, with all its security and logon concerns, we can use eVision. What is proposed is the following:

• ExamAdmin will populate a custom table in the SAINT database with the data from a Published Timetable. • RQH/RQI entries are created in SAINT to access this custom table using SQL (as recommended by Tribal). • SRL scripts are written to format and display the results from an RQH. • eVision pages are created to use these SRL’s to display an individual student timetable.

Testing has shown that this process works, as shown below:

Figure 6 ‐ Timetables via eVision


MIS

13

What may also be worth considering is the following:

• Before a timetable is published, students can view which modules they will be examined in via eVision. The data for this will be taken from an Extract Profile, and can thus give a timescale of months for students to check that they are on the correct module and if the Extract Profile needs “tweaking”, rather than the current timescale of weeks. Since most timetable changes are informed by students themselves, this will help nail down data quality well before scheduling begins.

• Online timetables, and the pre‐timetable module list above, can be viewed in Blackboard – students use this more than eVision.

4.3 Syllabus+ Plugin Module [ ]

This module has two functions:

• To get the data from an Extract Profile and save it as a UEF file. This UEF file can then be imported into Syllabus+.

• The completed timetable will be saved from Syllabus+ as another UEF file. This file can then be uploaded to ExamAdmin as a Published Timetable.

4.3a Options for Syllabus+ Implementation

Extensive research into how Syllabus+ can be better used for exam scheduling has been undertaken. Whilst Syllabus+ presents far fewer problems and is much more flexible than WASP, it is let down by its lack of comprehensive special needs support. However, these problems can be overcome by changing the way that Syllabus+ is used in the Exams Office. Since a change of process has risk implications, three options are presented below in increasing order of comprehensiveness, but also risk.

Option 1 – Carry on as usual

• The Exams Office continue to use version 1.9 of Syllabus+. • ExamAdmin will populate the UEF file with disability information, removing the need to do this manually. • Students with special requirements will continue to be scheduled in the main room, but also scheduled

outside of Syllabus+ into special rooms – i.e. they have “two seats”.

Option 2 – Special Requirements done inside Syllabus+.

• The Exams Office upgrade to version 2.1 of Syllabus+.

Students with special requirements will initially be scheduled in the main room. However, they will then be moved into special rooms using Syllabus+, thus removing the current “two seats” and manual process problems. This post‐process approach is the one recommended by Scientia, but there is no automatic procedure to do this. Students will have to be moved around one at a time, and manually, using Syllabus+. There can be almost 1000 special sittings in a main exam period (Jan or May).

Also, using this option, Syllabus+ cannot take account of students requiring extra time in its overloading constraints.

Option 3 – Fully Automatic

ExamAdmin treats those exams that have students with special requirements as different exams. For example, on exam “MAN0101M” if there are some students who need 15 minutes extra per hour, they will be put on exam “MAN0101M +ET15”, whose length will be adjusted to take account of the extra time. If there are additional students who require a separate room, they will be put on exam “MAN0101M +SEP”. The exported UEF file will contain information that says that these exams should be scheduled at the same time, and are in


MIS

14

fact the same paper. The correct rooms can then be assigned to each exam, and the timetable can then be scheduled completely automatically.

The only drawback to this is that the list of exams will be longer. However, based on the May 2008 exam period which contained 516 exams, research has shown that with all the special exams, there would be 923 – not even double the number.

Also, using this option, if a student decided not to take advantage of their additional requirements for one or more exams, they could simply be moved back onto the main exam and rescheduled without the need for moving them manually around rooms.

4.4 QMP Plugin Module [ ]

4.4a Schedule Exams

This function allows the user to browse a Published Timetable for exams that meet a certain criteria (usually those with a specific assessment type), and then create a schedule for them inside QMP.

Figure 7 ‐ Schedule Exams

4.4b Transfer Exam Marks to SAINT

There are two methods to transfer marks from QMP to SAINT, but both will rely on the participant name in QMP being the same as the student’s LDAP login.

Import a CSV file using the SAINT client

The QMP module can create a CSV file containing all the marks for a particular exam. This file can then be imported directly, using the SAINT client, on the SAS screen. This is shown below:


MIS

15

Figure 8 ‐ Exam Mark import to SAINT

The benefit of this approach is that SAINT will automatically calculate the correct grade (i.e. pass, fail, etc…) for each mark upon import. Also, multiple CSV files can be imported at once.

This has been tested in SAINT, on exam marks supplied from QMP, and found to be successful.

Direct transfer of marks to the SAINT database

This can be achieved, but we will also have to calculate the grade inside ExamAdmin, based upon the Mark Scheme employed by the assessment. Failure to supply a grade, as well as a mark, will cause subsequent exam board processing in SAINT to fail. The rules determining a grade are complicated, and it may well be better to let SAINT do the calculations itself by CSV import, rather than trying to duplicate the procedure outside of SAINT.


MIS

16

5. Appendix A – AAM Statistics

The following statistics are based on a snapshot taken on 12th August 2008, for all students registered in the AAM module – not just current students.

5.1 Students Requiring Extra Time

Minutes per Hour Extra Number of Students % of Students

10 1 0.1%

15 967 95.7%

18 1 0.1%

20 12 1.2%

25 1 0.1%

30 22 2.2%

60 4 0.4%

15 Hours ! 2 0.2%

5.2 Additional Requirements

Requirement Students

Amanuensis in examinations 72

Arrangements for written exams only 1

Clock stopped time in exams 57

Comfortable Chair required for exams 65

Exam paper text to be unambiguous 6

Exam Scripts in large print (see notes) 16

Exam scripts on tape recorder for student

4

Examination breaks 49

Fan required in exam room 1

Height Adjustable table for W/Chair 2

IT Training Required 1

Large print exam scripts required 3

Needs to walk/stretch/lie down 23

No flickering lights in room 4

Option of amanuensis 2

Option of Scribe or PC 6

Paper modified for Deaf student 1

papers transcribed after exam with student

2

Requirement Students

PC in Examinations 113

Proof reader required for essays/assignments etc

2

Reader in examinations 50

Regular drinks required 7

Requires Coloured paper see NAS 24

Room close to a wc 18

Separate room required for exams 190

Short clear instructions, no long paragraphs

2

Sit close to door/WC 4

Speakers to face student when speaking 1

Special lighting required 8

TFT Screen in PC based exams 3

toilet breaks as and when required 19

Use of Dictionary eg: Paper or Electronic 1

Wrist Rest and Mouse Rest 1

Writing slope / board required 3

ExamAdmin Draft FDS · PDF fileExamAdmin Draft FDS ... This document details a draft...

Documents

Transcript of ExamAdmin Draft FDS · PDF fileExamAdmin Draft FDS ... This document details a draft...