MINIMIZING BIAS IN ASSESSMENTProf Balasubramanian Thiagarajan MS DLO

Assessment Vs Evaluation

Assessment This is a feed back provided to the

student by the faculty regarding his / her performance

This provides valuable inputs to the student to enable them to take corrective measures

Student is not penalized with failure Improves student competence in

core and other related competency areas

Evaluation Analysis and use of data by faculty

to make judgement about a student’s performance

Competency of a student is assessed

Student is penalized with fail marks Ensures that our teaching program

produce graduates with optimal clinical skills and theoretical knowledge

Evaluation Ideal scenario

This is an analysis and use of data by faculty to make judgements about performance of a medical student

Evaluation should be accurate Evaluation should be performance based It should not have any bias for or against the student Ideal evaluation scenario should be able to identify gaps between

competency based goals and individual performance

Differences between approach of internal and external examiners

Internal examiner Has intimate knowledge of the

student’s academic capability Is involved in mentoring the student

hence the student is familiar with the questions posed by internal examiner

Positive and negative bias are common

Can influence the external examiner

External examiner Has absolutely no knowledge about

student capability Not involved directly in mentoring

the student and hence student has no knowledge about the questions posed by them

External examiners show no bias Can be influenced by internal

examiner

Bias

Two types of Bias are encountered during evaluation process:Cognitive bias – Distortion in a way reality / information is perceivedCognitive response bias – Evaluators expect the examinee to answer questions in a way they expect it to be answered which could result in positive / negative bias towards the candidate.Only a proper tool box designed for the process of evaluation would minimize chances of bias.

Designing evaluation tools basic steps

Step I – Create the evaluation planStep II – Deploying the toolStep III – Analyzing the responseStep IV - Act

Halo effect

Throndike was the first to introduce this conceptArises from the inability of people to think of other individuals in mixed terms, instead they are categorized roughly good or bad across all categories of measurements. Examiner’s expectation varies according to the institution the candidate is fromAbility of a candidate is assessed from the initial stages of performanceBlind spots ensures that the examiner is not aware of Halo effect

Reverse Halo effect

Also known as Devil effectA single weak point of a candidate revealed early during the process of exam is sufficient to cause undesirable resultIdeal examination tool created should avoid Devil effectMajority of Bias with negative impact is caused by Reverse Halo effect

Our Experience

Characteristics of ideal student evaluation tool

It should be comprehensive It should cover multiple parameters Should be generalizable It should be reliable and reproducible

Components of ideal examination tool box

Written examination Clinical examination OSCE OSPE Ward rounds Log book evaluation

Theory Examination

Essay and short notes 70% weightage

MCQ (Negative marks, multiple correct answers) 30% weightage

Thank you