CAMEL

About Assessment and evaluation at a Distance…

introduction

• Distance courses for 13 years. Actually, with over 2000 students KATHO became a leading institution in distance education in Flanders.

• While a regular F2F course takes 3 years and 60 study points each year, the distance course takes, depending on accreditation of prior experiential learning of the students, between 2 and 4 years.

• The digital learning environment is based on Blackboard and consists of digital courses, assignments, discussion boards,…

introduction• Although students can study at home and enjoy

their practice in schools within their own environment, they still need to come to the university campus when it comes to assessment, oral or written exams.

• Since 2007, some Belgian students living abroad, showed their interest in becoming primary school teachers by means of the distance course. Those candidates could not be accepted due to practical problems concerning assessment and follow up of their practical training.

introduction• In September 2008 the Expertise Centre distance

learning of KATHO-Tielt (MAD), started up a small research project on online assessing: the Camel project was born

• 5 students signed a letter in which they agreed to take part in this experiment, with all due risks.

• The concrete challenges of this project: translating and adapting documents concerning their teaching practice, development of good instructions and instruction movies, and most of all, a good scenario for the assessment, both for the teachers and the students involved.

introduction• The students participate in the online courses,

make the assignments and participate in the discussion board on Blackboard.

• On top of that, these students also get access to a Blackboard community with specific information.

• Anyhow, the project is very demanding for the study coach, who is – regardless of the specific disciplines - responsible for all the students, but spends more hours on the students of this particular project…

• When it comes to assessment and evaluation at a distance… creating a setting with real reliability and objectivity is a basic condition.

• We recommend open book exam as a first option to avoid problems with cheating. But that is not enough. We need to be able to create an environment for assessment and evaluation, that takes the criteria of the basic condition of objectivity very seriously on the one hand and is flexible enough on the other hand to use a wide range of question types and different ways of support and feedback from the teacher instructor.

about the challenges

the scenario• We chose Adobe Connect to work out the best

setting for an exam on distance. This videoconferencing program allows to talk and see, to share documents, to chat and to take over the screen of the student’s computer.

• Prior to the first exam by means of Adobe Connect, both student and teacher can learn to work with the program by watching a screen casted instruction movie on the specific community in Blackboard.

the scenario

• After the first six months, as a result of trials and errors, we summarized a brief description of 5 formats within the scenario for assessment at a distance. Formats that both students and teachers experienced as rather comfortable and reliable settings for assessment.

the scenario: 5 formats

• The pure oral exam

As if it was a face2face setting, the teacher and the student plan an individual meeting for the exam. The teacher is asking the question, the student responds in real time. In this case, the quality of webcam and headset is very important. When it comes to a good sound, sometimes Skype revealed to be the most efficient tool for VOIP (Voice Over Internet Protocol).

• Written exam, digital and synchronous

Again basically a digital face2face setting. In this format, the teacher shares the test as an online document in Adobe Connect. The student can write ‘live’ in the document. Sometimes there is a little delay, but the result is very satisfying and makes full profit of the extra value of the videoconferencing tool. The shared document can be an Office Word document but also an Excel or a Powerpoint. A huge difference with ordinary chat!

• Written exam in a traditional way

The student prints the test, fills it out, scans it and sends it back.

=> In case you want to avoid any technical difficulties or e.g. when students need to make complicated drawing. Math exams are solved this way.

• Written exam in the learning management system

The teacher makes the test available in his course within Blackboard.

The advantages of a pool with questions and automatic correction will facilitate the task of the teacher in the long run.

• Written exam with the use of assessment software.

The teacher makes the test available in Edumatic, comparable to the internationally well known authoring tool Questionmark.This tool allows the use of a wide range of question types.

Findings and implications for future application.

• Teachers of the distance education currently use the five described formats of the scenario. They are used to teach and coach students in an online environment. Once both student and teacher are familiar with the setting, the preparation takes less and less time. However, the very individual process of assessing on a distance is very time consuming for the teacher...

Findings and implications for future application.

• Among the concrete objectives for 2010, the detailed description of the scenarios is a priority as well as more tutorials and instructional screen casts for both teachers and students involved in assessment and evaluation at a distance. In the mean time, based on literature on identical experiences and new research in this area, we need to refresh our TPACK (Technological Pedagogical and Content Knowledge) to refine and improve the project.

Conclusions

• As this is a work-in-progress project, we hope to learn from our experiences and implement some of the new techniques in the “normal” distance program.

• Since October 2009, we have 9 students in the project, of whom 3 are living in Belgium… Yes, it is clear that this is a project with lots of potential and without any doubt, lots of candidates.

• On the other hand, it is clear that the actual approach is too time consuming to maintain if we accept a larger group of students…

Conclusions…• The most important conclusion is that the whole

project helps us to reflect about the meaning of assessment. It leads us to a critical approach and even towards questioning traditional forms of assessment. It challenges us to find out, to invent completely new forms of assessment…

• Perhaps the biggest challenge is to involve the students in this urgent construction of new assessment and the design of the distance education of the future with richer and more accurate forms of assessment.

Thank you

for your attention !!!

filip.vervenne@katho.be