Icem&siie 2011 peer-assessment

Loureiro et al. (2011). E-assessment in collaborative blended learning

E-assessment in collaborative blended learning: evolving practices and students’

perceptions

Loureiro, MªJoão, Pombo, L. Balula, A. Moreira, A.


Sumary

• problem/research questions• context• user generated content experience

– objectives– methodology

• results• final considerations


interactioninteraction

how to improve?

practice research

• problem

policy/management


bLearning•nature

•teaching strategies

•learning

•e-assessment

focus

impact

Online

context

practices (evolve, improve, innovate)

research


• evolution/research questions

EES(MsD)

DE(PhD 2008/09)

DE(PhD 2010711)

evauation of teaching

(students’ perceptions)

evauation of PA activities

assessment of coll. work


assessment of coll. work


quality of peer assessment

evauation of PA activities


– PhD program in Multimedia in Education module• Distance Education (DE) – two editions (2008/09 and 2010/11)

– blearning regime • face-to-face and at a distance• group work

– DE module aim - develop research competences – • search, select, systematize, synthesize and literature related with DE;• communication, collaborative work and assessment competences (self-

and peer- assessment) “good enough doctorate” (Wisker, 2010)


– PA benefits:

• promote higher order thinking, developing students’ critical thinking, communication, lifelong learning, problem solving…

• increase the amount of feedback, from teachers and from peers• enhance students’ sense of ownership, responsibility and students’

motivation, since they find it useful, attractive and enjoyable• support active and autonomous learning • can avoid the involvement of free-riders in group work, i.e,

facilitate the identification of individual contributions

Boud and Falchikov (2007), Joordens et al. (2009), Li et al. (2009, 2010), Peng (2010), Rourke et al. (2008), Sluijsmans et al. (2004), Topping (2000, 2008, 2010),

Zundert et al. (2010)


– PA pitfalls:

• students’ attitudes toward PA may hinder the above mentioned benefits – unconfidence towards PA –> resistant toward PA

• PA can be subjective – students might be too hard, critical or boasting, or not used to assess their peers -> increase students’ anxiety

• PA is time-consuming – training, preparation, and monitoring are needed

– PA advantages increase – formative feedback

Boud and Falchikov (2007), Joordens et al. (2009), Li et al. (2009, 2010), Peng (2010), Rourke et al. (2008), Sluijsmans et al. (2004), Topping (2000, 2008, 2010),

Zundert et al. (2010)


• User Generated Content (UGC) experience and methodology

– case studies (blearning courses/user generated content – literature review ODL) / action research (reflexion on/after action).

– strategies - project work, problem solving, collaborative learning and assessment for learning -> quality in online learning.

– duration of the DE module, participants & tools:• 2008/09: five-week period – 24 students – Wiki (

http://ead0809.wetpaint.com), Google Docs, EndNote Web…;• 2010/11: four weeks) – 18 students – Ning (http://

labconteudosdigitais.ning.com), Google Docs, EndNote Web…


• changes in 2010/11 edition:

– provided extra opportunities for shared PA and use of assessment criteria of the group work • it was used for formative e-assessment of three versions of

the ongoing work instead of one;

– assignment made by email -> confidentiality -> authenticity.


• results (difficulty and relevance of PA activities)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Self and peer-assessment of the group work and of the develloped competences

Weekly workgroup reflexive analysis

Survey of perceptions about how to make a literature search…

2010/11 Academic Year

Difficulty Relevance

2008/09 20010/11


• results (privacy, participants and components )

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%

Collabarative work should be assessed as a whole with the same mark for all elements

Collabarative work should beassessed with several components

Collabarative work should be assessed individually, by teachers and by colleagues

Collabarative work should be assessed only by teachers

Self and peer assessment of the group works should be published in the module's site

Self and peer assessment of developed competences (inside each group) should be published in the

module's site

Academic Year - 2010/11

completely disagree disagree agree completely agree don't know

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%

Collabarative work should be assessed as a whole with the same mark for all elements

Collabarative work should beassessed with several components

Collabarative work should be assessed individuall, by teachers and by colleagues

Collabarative work should be assessed only by teachers

Self and peer assessment of the group works should be published in the module's site

Self and peer assessment of developed competences (inside each group) should be published in the

module's site

Academic Year - 2008/09

completely disagree disagree agree completely agree don't know

2008/09 20010/11


• final considerations - connections with UGC state of art1. definition of UGC

content created by postgraduate students (literature reviews); shared freely; content has not been evaluated through a process of formal peer

review (journals); students performed self-reflexive task concerning the groupwork

and the developed competencies.

2. quality in elearning– activities fit the students’ needs and enriched learning context,

active involvement in the production of a literature review (product and processes).


• final considerations - relations with the CONCEDE framework

– CONCEDE framework for peer creation - peer-validation• writing process involved several activities: authoring, editing,

updating, revising, reflecting…; • DE module has been evaluated by different stakeholders:

– students;– university professor not involved in the module - peer-evaluation; – formal peer-assessment - submitted for publication (journals and

international conferences, the experience has been evaluated at the first and second level of the quality pyramid.


• final considerations– limitations - personal experience, few contexts, number of

students/comparability, qualitative approach…– further work

• content analysis (PA, reflexive activities, products…), triangulation• share, dissemination -> transfer to other context

– old - improve students learning - meets new???• does the available ICT tools help, how, in what circumstances? • is its use sustainable?

– environment, economical, social development impact???– inclusion or exclusion??)


questions and comments???

[email protected]@[email protected]

[email protected]


%0 20 40 60 80 100

Use of the negotiated criteria

Adequacy of the chosen vocabulary

Provision of constructive feedback(criticisms)

Provision of constructive feedback(questions)

Provision of constructive feedback(suggestions for improvement)

Fairness of the score

Adequacy of the final considerations

• results (1th edition DE) – quality of PA


modules organisation (DocProg)

• module organisation

Icem&siie 2011 peer-assessment

Education

Transcript of Icem&siie 2011 peer-assessment