Caveon Webinar Series - Lessons Learned at ATP 2016 - 04202016
24
www.caveon.com 1 Caveon Webinar Series Test Security Lessons Learned at ATP’s Innovations in Testing 2016 April 20, 2016
-
Upload
caveon-test-security -
Category
Education
-
view
204 -
download
0
Transcript of Caveon Webinar Series - Lessons Learned at ATP 2016 - 04202016
www.caveon.com 1
Caveon Webinar SeriesTest Security Lessons
Learned at ATP’s Innovations in Testing 2016
April 20, 2016
www.caveon.com 2
Today’s Presenters
Steve Addicott Jamie Mulkey
www.caveon.com 3
Agenda for Today• Keynote Address• Caveon Impressions• Security Sessions’ Key Points• Plenary • Summary• Q&A
www.caveon.com 4
Keynote SpeakerHuman-Centered Innovation
David Viotti, Founder and CEO, SMALLIFY
• Make little breakthroughs until you get to your big breakthrough
• Make big problems smaller• Use small bets to move ideas into action• Use ‘Yes, and…’ to add to a solution’s
ideas and keep the process positive
www.caveon.com 5
Caveon Impressions• Security concerns are
on the rise (Many security sessions)
• Online Proctoring momentum growing
• Understanding “Workforce skills”
• Loss of public trust in testing
• Be proactive and use secure design practices to thwart cheating
www.caveon.com 6
Security-Related Sessions• Dumping the Dopes Who Use Braindump Sites:
How IBM Turned the Tables Using Data Forensics • Test Security SMACKDOWN! • The Complete Equation for Workforce Skills
Credentialing• The Media is Knock, Knock, Knocking at Your
Door…Now What???• Real-World High-Stakes Test Security
Investigations• A Case Study – Considering Security from
Program Inception• Shutting the Barn Door after the Horse has
Bolted
www.caveon.com 7
Session #1Dumping the Dopes Who Use Braindump Sites: How IBM Turned the Tables Using Data Forensics• Kristin Gibson, IBM Solutions Certification Strategy and
Program Manager • Jamie Mulkey, Vice President, Caveon Test Security
www.caveon.com 8
Key PointsDumping the Dopes Who Use Braindump Sites: How IBM Turned the Tables Using Data Forensics• Miskeyed braindump answer keys help
identify test cheaters• Exam data can be used to catch the test
“thief” as well as those who purchased braindump content
• There are recommended automation tools and legal considerations in test program score invalidation
• There are several score invalidation principles testing programs can use
www.caveon.com 9
Session #2Test Security SMACKDOWN!
• Christy Frederes, Director of Operations & Security, Ascend Learning
• Tara Miller, Manager, Test Security and Administration, Ascend Learning
• Jamie Mulkey, Vice President, Caveon Test Security
www.caveon.com 10
Key PointsTest Security SMACKDOWN!• Utilize test security best practices
o Have a Security Plan in placeo Have a plan for addressing test
security incidenceso Develop a communications strategy
for developing test security awareness
• Practical implementation through a competitive game!
www.caveon.com 11
Session #3The Complete Equation for
Workforce Skills Credentialing• Rachel Schoenig, Assistant Vice President, Head
of Test Security, ACT• Jennifer Geraets, Sr Test Security Analyst, ACT• Jamie Mulkey, Vice President, Caveon Test
Security
www.caveon.com 12
Key PointsThe Complete Equation for
Workforce Skills Credentialing• The integrity of the assessment tool is
not the complete equation for valid, reliable results
• A common framework for credentialing is needed
• The Security subcommittee is helping the industry provide the missing test security variable
www.caveon.com 13
Session #4The Media is Knock, Knock, Knocking at Your Door…Now What???• Walt Drane, Director of Research and
Development Operations, Mississippi Department of Education
• Steve Addicott, Vice President, Caveon Test Security
• Marc J. Weinstein, Vice President, Investigative Services, Caveon Test Security
www.caveon.com 14
Key PointsThe Media is Knock, Knock, Knocking at Your Door…Now What???• Media relations begins BEFORE a crisis
occurs• Have a crisis communications response
plan readyo Roles and responsibilities
• Be proactive with the media when a breach occurs
• Be the source for your story, or someone else will
• Cover your bases professionally and legally
www.caveon.com 15
Session #5Real-World High-Stakes Test Security Investigations• Walt Drane, Director of Research and Development
Operations, Mississippi Department of Education• A. Benjamin Mannes, Director of the Office of
Investigations, American Board of Internal Medicine• Marc J. Weinstein, Vice President, Investigative
Services, Caveon Test Security
www.caveon.com 16
Key PointsReal-World High-Stakes Test Security InvestigationsReal World Example:• Whistleblower draws media attention• Data forensics and on-the-ground
interviews of students and educators ensued
• Two educators have licenses revoked• Investigation still ongoing
www.caveon.com 17
Key PointsReal-World High-Stakes Test Security Investigations• All testing organizations are
vulnerable to threats• Identify/ determine the
scope/severity of an incident• Conducting thorough, fair, and
effective test security investigations is key
www.caveon.com 18
Session #6A Case Study – Considering Security from Program Inception
• Kathy Powers, Certification Program Manager, Lenovo
• Susan Weaver, Test Development Manager, Caveon Test Security
www.caveon.com 19
Key PointsA Case Study – Considering Security from Program Inception
Lenovo Case Study:o Every system and process in the
Certification program impacts test security (and vice versa)
o Before determining “what” and “how much” security is needed, identify what is at risk
o Make a plan - Work the plan
www.caveon.com 20
Session #7Shutting the Barn Door after the
Horse has Bolted• David Foster, CEO, Caveon (sub for Kris Vezina,
Google) • Jim Wollack, Associate Professor & Director,
Testing & Evaluation Services, U of Wisconsin-Madison
• Kelli Foster, VP, Secure Exam Development and Support, Caveon Test Security
www.caveon.com 21
Key Points
Shutting the Barn Door after the Horse has Bolted
• Strong item design is key for securityo Secure item typeso Security-enhanced item stems o Item rotation o Trojan horses
• Limiting item exposure
www.caveon.com 22
Closing PlenaryReclaiming the Conversation -
Influencing How the World Talks and Thinks About Testing
• Andre Allen, General Dynamics Information Technology
• Eric D'Astolfo, Pearson VUE• Nikki Eatchel, Scantron• Ashok Sarathy, GMAC• Rachel Schoenig, ACT
www.caveon.com 23
Closing Plenary• The backlash of testing –
what’s happening• Testing is getting a bad
wrap• Working together as a
community to encourage the value and purpose of testing
• The need for testing has never been greater, and our industry’s ability has never been stronger
• Creating a positive testing public relations plan
www.caveon.com 24
Thank You!Follow Caveon on twitter @caveonCheck out our blog www.caveon.com/blogLinkedIn Group “Caveon Test Security”Steve Addicott
Dr. Jamie [email protected]
