Transforming High-stake Examinations DALJEET SINGH,

HEAD – GOVERNMENT BUSINESS, MeritTrac Services

Jharkhand – India’s one of the fastest growing destination

Ranked 3rd in “ease of doing

business” in the country.

Governance Education

launched the Sarva Shiksha

Abhiyan, thereby consolidating

primary education to all up to the age

of 14.

InfrastructureTotal reserve of

3218.08 million tons hematite areas

accounting for about 30 per cent of the national reserves.

IndustryHouses 2 major steel

plants.And has about 93000 manufacturing MSME

units employing approximately 3 lakh

persons

HealthcareDetermined to provide quality

health care services,

particularly for those in remote and

difficult areas in order to reduce morbidity and

mortality.Source - http://js.leadersspeak.in/

ESTIMATION OF INCREMENTAL MANPOWER DEMAND IN JHARKHAND2012 – 2022 IS 44 LAKHS

As per the Deloitte Research in NSDC study Report

Building Capacity to Service this growth

Capacity Building = Quality Recruitment + Periodic Training Programs

Key components to revolutionize the State Development -• Leveraging the internal Capacities of people within the Government

through adequate training

• Setting up right Mechanism for High stake examinations in

identification of best people and skill set by ensuring the best

“Quality + Fairness + Security + Convenience”

Delivering High Stake Examination in Jharkhand - Key Considerations

Quality Test Content in all Languages

Delivering Examination Across the State

On Time & accurate Result Processing

Secured & efficientLogistics

Mitigating Examination Fraud

It is important to leverage best practices for computer based or paper based test delivery

Effective Examination Delivery – People, Process & Technology

People

Process Technology

Mitigating FraudDescriptive

Script Evaluation

Exam Management

TRANSFORMING EXAMINATION

As per the Deloitte Research in NSDC study Report

Benefits of using Computer Based Tests

• Multi-batch Exams• Simple to administer• Reduce risk• Inclusive

• Question paper SECURITY before, during and after the exam• Increased ACCURACY through limited manual intervention• TIME-BOUND; ability to process results faster• Greater TRANSPARENCY; ability to respond to RTI queries accurately and in

time• Better quality of candidates to serve the Government for long time to come• Peace of mind to Administrators

Local Self-

Governmen

ts

Police Departments

Irrigation Department

Municipal

Corporatio

n

Power Distribution

Large number of Exam Bodies have already Administered Large Computer Based Exams…Breaking the Myth

Best Practices in Computer Based Exams have been leveraged to administer computer based exams for lakhs of candidates in 300+ Indian cities, thus breaking the myth about India’s readiness to use technology for exams

STREAMLININGDESCRIPTIVE ANSWER EVALUATION

As per the Deloitte Research in NSDC study Report

Evaluation of Descriptive Answer Scripts have always been a Nightmare for Examination Bodies

Time of evaluators spent on administration/operations (Score Management)

Errors in evaluation due to wrong tabulation of scores

Declining quality of evaluations due to lack of control over the process

Logistics and paper movement

Lack of availability of evaluators on time

Multiple evaluations of answer scripts

Benefits of On-Screen Marking (OSM) technology for Exam Board

Section-wise evaluation in parallel, enabling speed with accuracy.

Secure evaluation from anywhere possible

Greater quality of evaluation through dual or more evaluation at

the same time

Live status report for COE and Chairman’s office for continuous monitoring,

higher operational visibility and control

100% accuracy; Zero totalling error; no missing questions.

Fairness in evaluation

Automated in-built security check. No movement of

physical answer scripts; No local storage of answer

scripts at evaluation centre; Anonymity of answer scripts

superior than existing model

Greater transparency - Option for candidates to avail the additional service to see their answer sheets through

a digital copy

MITIGATINGIMPERSONATION

As per the Deloitte Research in NSDC study Report

Risks of Impersonation in High Stake Exams Methods of committing fraud is higher in high stake exams; impersonation is an organized fraud that has often marred large exams

Leverage Technology to prevent ImpersonationFOOL PROOF IN CLASS ROOM CANDIDATE AUTHENTICATION

Biometric Candidate Authentication• F

ingerprint registration and verification

• Capture photo for second level of authentication

Advantage• A

utomated & error-free fingerprint verification

• Ability to manage large volumes

Guaranteed• Z

ero impersonation

• Unedited data and audit trail for RTI queries

As per the Deloitte Research in NSDC study Report

Telling

This was one of single largest exams in the state. 4.5 Lakh candidates took the test distributed into 3 batches.The successful administration of the test is a testimony to the state’s ability to deliver high quality examinations of such magnitude

Recruitment Test - Constable Competitive Examination 2015 administered across the state of Jharkhand

Volume• 7129 positions to be filled• ~4.5 lakh candidates scheduled for the test• Tests administered in 320 Test Centers across the state• Supported 19 Tribal Languages in the main Examination

Solution• Online applications• Issue of Admit Cards, • Online question Representation • Content Management (including tribal Languages)• Test Conduction• Result Processing

Benefits• Use of technology for processing of Applications reduced errors and

streamlined the test registration.• High quality content made available in 19 tribal languages.• Efficient logistics across the state for greater security and time bound

test conduct.• Faster and accurate result processing

• The Department wanted to recruit candidates for 7129 Positions in Jharkhand.

• Wanted to conduct the examination across the state including provision for tribal languages.

Large Paper Based Recruitment Test in Jharkhand

One of Largest computer based exams delivered in a single state. A local body took a progressive step by adopting best practices like Multi-batch exam and equating to make the exam process effective

The Dept. of Local Bodies of a large progressive state Volume

• 1947 vacancies to be filled in 17 positions• ~4 lakh candidates scheduled for the test• Project executed in 4 months• Tests administered in 10 cities in the state

Solution• Online applications• Issue of Admit Cards, • Mock Test for Candidates, • Computer Based Exam, • Online representations from Candidates• Result Declaration

Benefits• Logistical challenges compared to Offline exam got reduced• Received High level of Secrecy and confidentiality • Highly secure and transparent• IT-Enabled recruitment process – Online form, Admit cards,

Emails/SMS Alerts, Online Exam, Online Representations and Results

• The Department wanted to recruit candidates for 17 Posts in a large progressive state.

• Wanted to migrate exam to Computer Based mode but had to administer tests in the state only.

• Examination best practices had to be adopted to be able to deliver such a large test in the state at highest SECURITY, TRANSPARENCY and QUALITY

Large Computer Based Recruitment Test for a Local Body

Examination prevented impersonation at for a very high stake & scale Exam.The data later used for counselling and admissions.

A large entrance examination uses in-classroom Biometric finger print authentication to prevent impersonation

Volume• 1.5 lakh candidates authenticated during

examination in one day• 1750 devices used by 2200 resources in 100 cities• Counselling for 36,000 candidates at 16 cities

Solution• Use of solution (patent pending) in-classroom fingerprint

biometric verification solution. • Registration of finger print captured at the classroom

during the examination with minimum disruption and then verification during counselling & admission using the same data.

Benefits• Prevention of impersonation• No change in process (seamless enrolment

during examinations)• Secure & Convenient• Reliable: ISO Standards based data capture

Impersonation in examinations is fast emerging as a major threat in high stake examinations in education and recruitment.

MeritTrac Solution: To prevent challenges of

impersonation To make the exam fair. Ensure that the method used caused

zero disruption and zero anxiety among the stakeholders.

Ensure that the method used is accurate and scalable for large number of test takers

Most Prestigious Engineering Entrance in the Country Prevents Impersonation

Achieving Faster Turnaround Time with TechnologyThe Public Service Commission has taken the lead in implementing a technology enabled solution setting benchmark for examination process in the country : • 100% accuracy; Zero

totalling error; no missing questions. Fairness in evaluation

• Greater transparency - Option for candidates to avail the additional service to see their answer sheets through a digital copy

• Greater quality of evaluation through dual or more evaluation at the same time

Volume• 25,000 candidates, ~1 lakh scripts, 36 lakh

images• 4 evaluation centers, 210 evaluators, all PHDs• Evaluation done on 7 days a week

Fast tracking adoption

• On the job training provided to evaluators

• Technical support at each center to assist when needed

Acceptance

• Results published in under 40 days, from erstwhile 9 months. Highly convenient for commission

• Appreciation from evaluators for ease of evaluation; evaluators included 87 year old Subject Matter Expert

• Future adoption confirmed. Commission contemplating marking from home for next phase

• The Public Service Commission of a large state wanted to reduce the turnaround time of descriptive script evaluations for its Civil Services Examinations.

• They used On-Screen Marking to improve transparency, speed, and quality of evaluations

A Public Service Commission Leverages On-Screen Marking for Descriptive Exams

Jharkhand has made Significant Progress…Tests have been delivered in 19

Tribal & regional languages

Tests have been delivered in over

450 centres

Technology has been used in paper

based examinations for greater accuracy

and speed

Introduced Online

Applications much earlier than many

neighbouring states

Displayed ability to

deliver large examinations successfully

THANK YOUINFO@MERITTRAC.COM | WWW.MERITTRAC.COM