A generative computer language to customize online learning assessment

A generative computer language to customize online learning assessment

A generative computer language to customize

online learning assessment

Antonio Balderas, Iván Ruíz-Rube, Juan M. Dodero, Manuel Palomo-Duarte


Table of contents

» Introduction» EvalCourse» Case study» Conclusions and future research


Introduction

– Importance of competences in Education:• Subject specific (knowledge)• Generic (abilities, capacities and knowledge that any student

should develop regardless of his/her subject area)

– Assessing generic skills could not be a scalable task:• We hardly have enough time to meet planning

• More difficult as the number of activities to evaluate increases


Introduction

– The scope of our work is related to LMS (Learning Management System)

– Each file, access or assignment done by a student is registered in the system

– This collected information would be used as objective indicators of students' proficiency

– Indicators are automatically-obtained statements that help determine if the competence level is met


Introduction

» We have developed a Domain-Specific Language to customize online learning assessments in Moodle: EvalCourse

» Using EvalCourse lecturers can easily retrieve indicators from information stored in the Moodle activity logs without any technical knowledge on databases or computer programming


EvalCourse

» EvalCourse is a plugin for the Eclipse IDE that processes queries in a domain-specific language that we have developed to get indicators


EvalCourse


EvalCourse

» Example of use:– Students have been submitting several tasks during the course

– We are going to asses their performance in the competence of planning and time managment from the number of tasks that have been uploaded in time

Evidence Milestones_list:get studentsshow milestonesin assignment.


EvalCourse

» List of quantitative information

Id Username Total In-time Overdue In-Progress In-time rate

1 John 5 3 2 0 60

2 Mary 5 1 0 4 20

3 Jane 5 5 0 0 100

4 Michael 5 3 2 0 60

5 Walter 5 4 1 0 80


EvalCourse

» A group of graphs that gives an overview of the results that can support the interpretation of the data


EvalCourse

» A Pentaho transformation file


Case study

» The case study we analyze was developed in University of Cadiz, Spain

» In a compulsory course on Language Processors II of its degree on Computer Science

» 36 students enrolled the course in 2012/13 academic year

» The course evaluation was done manually and we later applied EvalCourse


Case study

The competences we wanted to assess were:

» Planning and time management» Teamwork » Interpersonal skills

» Leadership


Case study

Extraction of indicators from milestones results

– Students had to work in small teams– Every team had to carry out five milestones during the course– Every milestone had a deadline – For each team, only one member had to upload each activity


Case study


– Planning and time management• All their milestones submitted in time

– Teamwork• Five activities submitted exactly


Case study


Evidence PlanTeam: get students show milestones in assignment.


Case study

Extraction of indicators from forum results

– Students had to work in small teams– Each team in the course had a forum for internal

communication– As the indicator of Interpersonal skills of each student we

calculate the total forum participation– As the indicator of Leadership we took into account the amount

of debates that each student started


Case study

Extraction of indicators from forum results

Evidence forum_participation: get students show participation in forum.


Case study

– The grades that the supervisor manually calculated for every

student were closed to the indicators:• The best works were also those who performed better on planning

and time management and teamwork skills• Few students who participated in the forum, but students with

good marks in both indicators really stood out on the competences


Conclusions and future research

– EvalCourse: a generative computer language to customize online learning assessments

– Lecturers were allowed to retrieve desired indicators from information in the LMS using a very simple syntax

– In the case study, we assessed students' performance in the competences of Leadership, interpersonal skills, Planning and Teamwork



– Results are promising

– The interpretation that we did about indicators was closely related to the results that the lecturer noticed during the course

– Further study are needed to get a strong conclusion on the validity of the specific usage of the information retrieved by EvalCourse



We have two future lines of work ahead: – Identify others indicators stored in LMS that can be used to

assess competences– Expand our generative computer language to be used in others

LMS


Thank you very much for your attention.

Questions? Comments?

A generative computer language to customize

online learning assessment

http://assembla.com/spaces/evalcourse

Acknowledgments:

A generative computer language to customize online learning assessment

Software

Transcript of A generative computer language to customize online learning assessment