Essential Elements of Test Security

C O N F E R E N C E 2 0 1 9

Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Essential Elements of Test Security

John Kleeman, Executive Director and Founder

David Hunt, Information Security Officer

Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

In many industries, assessments increasingly important

Ensure workers knowledgeable, safe, competent in practices

Ensures skills and competence certified

Threat of an unscrupulous person who chooses to cheat putting co-workers / organization / public at harm

Why Test Security Matters

Slide 2


General security

• The concept of risk and how it can drive security

• General security good approaches

Nine common risks to test security

• What they are

• Real life examples

• Mitigations

Practical demonstration of

some security risks

Using Questionmark

securely

What we are going to cover today

Slide 3


Data Security

System Security

Slide 4

Trustable Results

Planning Assessment

Authoring Items

Assembling assessment

Pilot and Review

Delivery

Analyze Results

Trustable Results Rely on a Trustable & Secure Process


Data Security

System Security

Slide 5

Trustable Results

Planning Assessment

Authoring Items


Pilot and Review

Delivery

Analyze Results

Content theft

Content theft

Test center disruption

Identity fraud

Unauthorized disclosure

Tampering

Some Risks to Trustable Results

Unauthorized aids / help


Cheating on Exams a Problem Worldwide

Slide 6


Meanwhile Data Breaches Rise …

Slide 7


A. Content theft / leakage

B. Unauthorized aids for cheating during exams

C. Identity fraud

D. Results tampering

E. Personal information breached

F. Something else

Question: What keeps you up at night?

Slide 8



Managing risk is the key to being secure

Slide 9


Confidentiality

IntegrityAvailability

Confidentiality

• Keep questions secure -avoid content theft

• Results only available to those who should know

Integrity

• Right person takes assessment

• Assessment process fair and robust

• No cheating

Availability

• Assessments can be taken when needed

• Results are stored safely

First Start with Your Security Objectives

Slide 10


Formal process can identify all threats, less likely to miss key threats

Quantifying risk allows you to prioritize actions that will reduce risk

Recommended by all respected security authorities: ISO 27001, NIST & many more

Example▪ 2017 Verizon data breach report reported 81% of hacking-related

breaches involved weak or stolen passwords.▪ For most organizations, a risk assessment is going to highlight this

and suggest some mitigations

Why Assess Risks?

Slide 11


No News Driven Security

1. Assets (What's important)

2. Asset data▪ Value

▪ Impact

▪ Threats/Vulnerabilities

▪ Likelihood and Frequency

3. Repeatable risk assessment process

4. Report and action findings

5. Monitor and Measure

Risk Based Security Decisions

Slide 12


High probabilityLow impact

High probabilityHigh impact

Low probability.Low impact

Low probability.High impact

LOW HIGH

LOW

HIG

H

Impact

Pro

bab

ility

Then look at Risk Probability and Impact

Prioritize security mitigations based on risk impact and probability


Security

vs

UsabilityVS

Security vs

Usability

The Great Balancing Act

Slide 14


After delivery

Tampering with results Unauthorized disclosure of results

During test delivery

Identity fraudContent theft /

harvestingCopy answers from

another Getting help from

othersUnauthorized test aids

Before delivery

Content theft from item bank Disruption at test center

9 Common Risks to Consider



Risk:

Content theft from item bank

Authoring Items



Content theft from item bank

Slide 17

Author leaks content deliberately to help their students

Author leaks content by mistake or by using poor security

Technical vulnerability in item bank

Potentially very high: requires rewriting all items

Could invalidate test for all test takers

Threats Risk impact Real world example


More real world examples

Slide 18


Use a secure cloud service (Look for compliance with known security standards)▪ Avoids local files▪ Avoids email of questions▪ ISO 27001 and good technical security

Good HR practices with authors▪ Training and education▪ Confidentiality agreements▪ Remove access when people leave project▪ Good passwords▪ Extend to translators too if test is translated

Use permissions and roles▪ Authors should only have access to the minimum they need for a project▪ Restrict access to questions/assessments they do not need▪ Single sign-on

Some mitigations to prevent item bank leakage

Slide 19


Just enough capability to get the job done

In assessment management systems:▪ Use roles and permissions to give

people the minimum capabilities they need

▪ Restrict access to questions to those who need

▪ Have a separate user account for privileged (high capability) actions

One person

requests

Another person

approves

Activity authorized

Two important security principles help

Slide 20

Least privilege Segregation of duties

By requiring collusion it becomes more difficult than one rogue person!



Risk

Disruption at test center

Slide 21

Pilot and Review Delivery


Disruption at a test center

Slide 22

Poor technical security at test center allows questions to be seen

Test center proctors corrupt

Test center proctors / teachers have incentive to help their students

Can result in a batch of test takers at the test center all being helped




Slide 23


Don’t allow download of questions to test center in advance

Consider not using test centers in pilot process

Data analysis / forensics

Incident response plan in place and tested

Using good technical measures▪ High quality TLS encryption

▪ Proctors unable to see questions

Use online proctoring▪ Remote proctors who do not know test takers and cannot receive a bribe less likely

to collude with test takers

Possible mitigations

Slide 24


Traditional Proctoring•Proctor observes test taker physically in

person / in the same room Live Online Proctoring• Proctor observing test taker live via webcam

Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.



A practical demonstration

How secure are our normal devices?

Slide 26



Risk

Identity fraud

Slide 27

Delivery


Identity fraud

Slide 28

Candidate gets a friend to take an exam for him/her

Candidate pays someone to take an exam for him/her

At least one result is meaningless

Can threaten acceptance of programme



Proctoring▪ Proctor checks government issued photo-ID▪ If concerned about proctors colluding with candidates, use online

proctoring

Frequent testing▪ Series of tests harder to get someone to take for you than single large test▪ With online assessment and online proctoring, much more realistic to have

more frequent, smaller tests

Use single sign-on (SAML)▪ People less keen to share credentials if they allow access to all their other

data

Reduce the motivation to cheat

Some ways to mitigate candidate impersonation

Slide 29


Fraud needs:▪ Motivation▪ Opportunity▪ Rationalization

For a high stakes test, there may be Motivation

Anti-cheating measures seek to reduce Opportunity

How can we reduce Rationalization?

Fraud triangle

Slide 30


Mitigating Rationalization

Slide 31

Rationalization

“Everyone else cheats so why shouldn't I?”

“I have no alternative but to cheat and I have to pass because …?”

“I can get away with this”

“I didn’t know I was doing anything wrong”

• Explain the facts:• Most people do not cheat• Consequences of cheating

Mitigation

• Provide environments for the candidate to be able to pass the exam honestly

• Explain the security measures in place and the consequences if they’re caught

• Explain and gain positive agreement to theo Honesty Codeo Code of ethicso Non-disclosure



Risk

Content theft / harvesting

Slide 32

Pilot and Review Delivery


Content theft / harvesting

Slide 33

Screenshot or other technical copy of test questions

Group harvesting, everyone remembers a few questions and shares online

Someone pays to take test just to see and remember questions

Slow degradation of integrity of test

Expensive as need to write new items



Use proctoring but don’t allow proctors to see content

Secure browsers like Questionmark Secure

Large and changing item banks with randomized test content

Reduce motivation to cheat / honour code

Use video/audio as part of stimulus to make harder to copy

Prevent harvesters from taking tests (pre-requisites, limit retakes)

Keep time limit as short as sensible

Possible mitigations

Slide 34


A secure browser reduces content theft and cheating

While using a secure browser, it is difficult for participants to▪ Make screenshots▪ Copy content▪ Run other programs▪ Search the Internet or use chat

Questionmark has a Questionmark Secure App and other vendors have other approaches

Secure browsers



Risk

Copy answers from another

Slide 36

Delivery


Copy answers from another

Slide 37

Candidate looks at the screen of the person next door to him/her and copies their answers

In a take-home exam, two people do it together

Organized answer sharing

Invalid test results for a candidate

Devalues credibility for others



Appropriate separation between workstations

Randomization of question order / question selections / choice order

Use “Privacy Screen Filters” to reduce visibility angles

How to mitigate copying answers

Slide 38



Risk

Getting help from others

Slide 39

Delivery


Getting help from others

Slide 40

Use a cellphone or instant messenging to get live help from a friend or coach

Someone sits by test taker to give coaching

Invalidates one result

If prevalent, degrades program




Slide 41


Use a secure browser (e.g. Questionmark Secure) for online tests

Proctoring (especially online)


Frequent testing

Forensics / analysis

Consider other approaches than essays

Do not tie test-taker performance to Instructor compensation

Some mitigations

Slide 42



Risk

Unauthorized test aids

Slide 43

Delivery


Unauthorized test aids

Slide 44

Cheat sheets / access to notes

Test taker Googles the answers

Calculators / other tools

Makes results less valid




Slide 45


Secure browsers

Proctoring

Make exam open book▪ In the real world, people have access to reference material, is it fair and valid to deny

it during an exam?

Ask higher level questions (e.g. Situational judgement)


Limit or prohibit personal items in test area

Mitigations

Slide 46



Risk

Tampering with results

Slide 47

DeliveryAnalyze Results


Tampering with results

Slide 48

An insider changes someone’s score out of favouritism

Participant bribes someone to change score

Technical vulnerability exploited to adjust scores

Ranges from just one score awry to whole exam invalidated



Robust delivery and results platform

ISO 27001 or similar security management

Store results in an online cloud system

Permissions / roles / segregation of duty

Robust audit trail

Care with results when leave the assessment system

How to mitigate tampering with results

Slide 49



Risk

Unauthorized results disclosure

Slide 50

Analyze Results


Unauthorized results disclosure

Slide 51

Mistake by administrators discloses data

Malware or other technical vulnerability discloses data

Serious embarrassment and negative publicity

Government fines

Invasion of privacy



Ensure that your supplier(s) have strong security▪ ISO 27001 is good

▪ Check they remain committed to security

Train your personnel well including security awareness

Remove access from personnel who leave organization/project

Ensure your systems are well secured if results go there

Strong passwords and single sign on

Permissions / roles / least privilege

Mitigations

Slide 52


After delivery

Tampering with results Unauthorized disclosure of results

During test delivery

Identity fraudContent theft /

harvestingCopy answers from

another Getting help from

othersUnauthorized test aids

Before delivery

Content theft from item bank Disruption at test center

The 9 risks we have looked at


It’s important to have Defense in Depth

Slide 54

Layers of protection rather than a single point of failure!


Many of the mitigations for one risk also apply to other risks

A single mitigation or control is vulnerable to bypassing

A defense in depth strategy involves having several controls, for example:▪ No cellphone policy

▪ Proctoring

▪ Lock down browser

▪ ID checks

▪ Random selection of questions

All complement each other

Why defend in depth?

Slide 55


Identify goals of your assessment program

Look at risks that apply to you

Implement mitigation measures based on importance of those risks

Improved security gives better data for better decisions

Improved security also increases trust from stakeholders

Summary

Slide 56



Some additional Questionmark specific advice

Slide 57


If you were to lose your data▪ You’d have to re-create all your questions and assessments▪ You would have lost all your past participant results – no evidence, no

item analysis, no records▪ Your assessment program would be down for a considerable period

If you use Questionmark OnDemand, we do it for you

If you use Perception / On Premise▪ Set up a reliable backup

▪ Conduct regular restore tests to check the backups can be restored

Backup

Slide 58

Ris

ksH

ow

to

dea

l w

ith


If you give someone a wide set of privileges, this can cause damage▪ If their credentials are stolen▪ If they make a mistake (e.g. delete something)▪ If they become untrustworthy

Limit “super user” permissions If you need “super users”, create two accounts

▪ One for normal access▪ One for privileged access

Reduces risk of super user account causing damage

Least privilege

Slide 59

Ris

ksH

ow

to

dea

l w

ith


Define roles that match needs

Allocate users to roles

Consider▪ Least privilege

▪ Segregation of duties

Use role-based security in Questionmark OnDemand:

Use roles and permissions

Slide 60


Passwords can be compromised in many ways including:▪ Weak passwords being guessed or cracked▪ People writing down passwords insecurely▪ Phishing attacks▪ Malware (e.g. keylogging)

Upgrade to the latest version (OD3 or Questionmark OnPremise) so you can set a good password policy (passphrases!)

Train your users on passwords and general IT security Limit rights to the least privilege that people need Consider SAML which allows two-factor authentication

Password weaknesses

Slide 61

Ris

ksH

ow

to

dea

l w

ith


▪ Common for organizations to fail to remove permissions from leaving or transferring employees/contractors

▪ Suppose someone finds a way to create themselves an administrator user or give themselves more permissions without you knowing?

▪ At regular intervals▪ Review all Questionmark administrators

Are they who you expect?

Are they still employed and/or allocated to your project?

Do they still need all the access they have?

▪ SAML can be another approach

Review usage rights

Slide 62

Ris

ksH

ow

to

dea

l w

ith

Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 63

Results Forensics: Assessment Completion

Time report

• Spot anomalous completion times

• Identify potential cheating

• Flag potential content theft

• Determine if allotted time is sufficient for completing the assessment

Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 64

Results Forensics: Assessment Results

Over Time

• Spot anomalous results for specific participants

• Mean score

• Minimum score

• Maximum score

• 95% confidence interval


Most secure as updated monthly, often fixing security issues

If you’re not already there, please move to OD3

Authoring Manager Is less secure

If you can’t move to OnDemand, you should move to Questionmark OnPremise▪ Removes Authoring Manager

▪ Allows stronger passwords and SAML

▪ Many, many security improvements

Upgrading to the latest version doesn’t just give you more features, it also makes you more secure

Slide 65

Questionmark OnDemand On Premise


Data Security

System Security

Slide 66

Trustable Results

Planning Assessment

Authoring Items


Pilot and Review

Delivery

Analyze Results

Trustable Results Rely on a Trustable & Secure Process



Your questions

Slide 67

Essential Elements of Test Security

Documents

Transcript of Essential Elements of Test Security