Questionmark s Assessment Platform
Transcript of Questionmark s Assessment Platform
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Introduction toQuestionmark’s Assessment Platform
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Key Drivers for Assessment
Assessment Platform Overview▪ Collaborative Authoring
Demonstration
▪ Blended & Secure Delivery
Demonstration
▪ Reporting and Analytics
Demonstration
▪ Integration
Q&A
Agenda
Slide 2
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
• Founded in 1988
• Assessment solutions that enable organizations to measure knowledge, skills, abilities and attitudes securely for certification, regulatory compliance and improved learning outcomes
• ISO/IEC 27001 Certified (Learn more: www.questionmark.com/trust)
Background
• Enterprise customers worldwide
• Business, Government; Higher Education; Certification Providers; Awarding Bodies
Customers
About Questionmark
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Assessment Platform
Collaborative, secure authoring of surveys, quizzes, tests, and exams
Reporting to analyze results and inform stakeholders
Blended delivery to support a broad range of devices and security needs
Multilingual: 36 Languages
Global 24/7 Support
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
OnDemandOnDemand for Government
OnPremise
Deployment Options
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Measure and Enhance Learning
•Measure knowledge, skills and attitudes –before, during and after learning
•Correct misconceptions and increase knowledge retention
Qualification and Certification
•Deliver reliable, defensible tests to certify employees
•Ensure sound personnel decisions
Regulatory Compliance
•Ensure knowledge of essential laws, regulations and health/safety guidelines
•Demonstrate compliance and reduce risk
Sales/Technical Channel
•Competence/product expertise of sales and technical professionals
•Increase revenues, customer satisfaction
•Reduce costs
Credentialing
•Efficiently create and maintain valid, reliable tests
•Generate revenue from exams and practice tests
Higher Education
•Higher Education
•Increase student satisfaction
•Reduce cheating
•Improve learning
Key Drivers for using Assessments
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Authoring and Item Banking
Slide 8
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Browser-based Role-based access control
▪ Secure, collaborative authoring
Author items (questions) Author assessments Publish assessments for delivery Manage item bank
▪ By topic, subtopic and metatag▪ Item version control▪ Import items & assessments
Manage translations
Authoring and Item Banking
Slide 9
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Slide 10
Many item types to choose from…
Multiple choice
Multiple Response
Matching
Ranking
Pull-down List
Select-a-blank
Matching
Job Task Analysis
Knowledge Matrix
Survey Matrix
Likert scale
True/False
Yes/No
Text Match
Drag-and-Drop
Hotspot
Numeric
Essay
Fill-in-the-blanks
Explanation
File Upload
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Demonstration
Slide 11
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Question-level feedback
Slide 18
Question with participant’s answer
Corrective feedback
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Question
Topic
Assessment Electrical Skills
AC Motors
Item Item
Synchronous Motors
Item Item
Switchgear
Item Item
Feedback and Scoring at Topic Level
Slide 19
Topic FeedbackTopics Topic Score
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Provide users role-based access to content
Provide users role-based access to content
Roles & Permission Management
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Item version history and rollback
Slide 21
Audit trail of edits and option to “rollback”
Compare revisions side-by-sideCompare revisions side-by-side
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Search description, stimulus, choices, status, meta tags,
dates and more
Item Search Search item bank to find items that contain certain words and/or tags
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Translation Management for Multilingual Assessments
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Blended, Secure Delivery
Slide 28
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Author Once… Deliver Anywhere
Slide 29
Single results set
Available Securely
Author once Deliver to any devicein context
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Responsive Design, Blended Delivery
One Assessment…
…many devicesand browsers
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Demonstration
Slide 32
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Print your own bubble sheets
Participants enter responses
Scan and upload for scoring
Analyze and report on results
Printing and Scanning:Deliver assessments online or on paper
Slide 34
Deliver assessments on paper or online Print your own question and bubble sheets Use your own scanner to read responses Upload scanned bubble sheets for scoring Scanned assessment results available for reporting
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Delivering high stakes assessments
Quizzes & Surveys
Diagnostic Tests
Low Stakes Tests
Medium Stakes Tests
High Stakes Exams
S e c u r i t y R e q u i r e m e n t s
Sta
ke
s o
f A
ss
es
sm
en
t
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Set password expiration periods
Force password changes
Set password requirements/constraints by▪ Length
▪ Complexity
▪ Alphanumeric
▪ Punctuation
▪ Upper/lower case
▪ Digits/Digit placement
▪ Exclude previous passwords
Manage and Enforce Strong Password Policies
Slide 38
Password112345678qwertyFootballabc123
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Set date ranges for availability of assessment(s)
Set “maximum number of attempts”
Set Scheduling Restrictions to limit access
Slide 40
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Secure browser for high-stakes assessments
Reduce content theft, cheating; improve exam integrity
Questionmark SecureOne Setting – Multiple Platforms
Slide 41
Windows Mac iOS Android
PrintingScreen captureTask switching
Control/system keysRight-click / context menu
optionsView source
Accidentally exiting exam
Prevents….
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Require Proctoring
Slide 42
Assessment Schedules
Assessment Settings
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
• Proctors can control and intervene in exam delivery: • Generate “unlock” code for test
• Start, pause, un-pause and stop
• Dashboard to monitor exam status of each test taker
• Works with Questionmark Secure
OnSite Proctoring
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Convenience of online exam delivery at home or workplace under the watchful eye of a proctor▪ Works with most standard webcams/audio
▪ Live proctor can start, pause, and stop exam
More info:▪ Website
▪ Your account manager
Live Online Proctoring
Slide 44
www.questionmark.com/proctoring
Powered by
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Accessibility
Also support for…- Non-mouse navigation; alternate pointing devices- Screen readers
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Single-Sign-On
• SAML
• LDAP
Launch and Track via AICC or SCORM
• SuccessFactors, Cornerstone, etc.
Learning Tools for Interoperability (LTI)
• Blackboard, Canvas, D2L, Moodle, etc.
Web services API
• QMWISe
OData APIs
• Results API
Integration
Slide 46
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Reporting and Analytics
Slide 48
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Reporting and Analytics
Slide 49
What can we learn from assessments?
Did they like it?
Did they learn it?
Can they do it?
Can we prove it?
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
What you can learn from Questionmarkreporting and analytics
How are our Items and Assessments doing? How are our Participants doing?
Did the learning investment impact
performance?Where are the skills and knowledge gaps in my
organization?
Are my questions and tests valid and reliable?
What did participants think of the learning
experience?
What did participants learn from the training
or course?
How do I coach participants to improve
performance?
How did one group of test takers compare to
another?
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Item and Test Analysis Item Analysis report Test Analysis report Score list report Test center analysis report Assessment completion time report Question frequency analysis report Demographic report Question type report Assessment Overview report Assessment results over time report
Participant-focused reports Score list report Coaching Report Transcript Report Grade Book Report
Survey Reports Survey Frequency Analysis Job-task-analysis (JTA)
▪ JTA Summary▪ JTA dimension by demographic
Course Evaluation▪ Course Summary Report▪ Instructor Summary Report▪ Class Summary Report▪ Class Detail Report
Types of reports available
Slide 51
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Score List Report
Slide 52
Demonstration
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
•~ % answered correctly
What it measures:
•Number from 0.0 to 1.0
How it is expressed:
•Too high: Question is too easy.
•Too low: Might be Too difficult…or confusing, ambiguous wording… or more than one possible correct answer… or miscoded
How to interpret:
• Correlation between item and assessment scores
What it measures:
• Number between -1.0 and +1.0.
How it is expressed:
• Generally…
• < 0.2 low
• > 0.3 good
How to interpret:
Item Analysis: Reliability of Question Results Data
Difficulty index (aka “P value”) Item correlation discrimination:
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 54
Difficulty/P-Value
Correlation -Discrimination
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 55
Option Analysis
Difficulty (p-value)
Discrimination
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 56
Coaching ReportCompare against
benchmarks or other participants
Participant
Topiclevel
Overall
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Job Task Analysis (JTA) Reports
Slide 57
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Course SummaryInstructor Summary
Class Summary Class Detail
Four Course Evaluation Reports:
Slide 58
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Built-in Reports Custom Reports and Dashboards
Microsoft ExcelPowerPivot
BI Tools
Big Data Applications: Results API
Slide 59
R e s u l t s W a r e h o u s e
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
•Planning
• Installation
• Integration
Deployment
•Assessments
• Importing items
•Templates / Look & feel
Authoring
• Item reports
•Other custom reports
Reporting
•Testing and Assessment Portals
•People Data syncing
•Results Data syncing
•Self Registration
•Auto Scheduling
•Assessment Clustering
•Certificate Generation
•Distributed Administration
•Assessment Catalogs
Solutions & Integration
Consulting / Solution Services
Slide 60
w w w. q u e s t i o n m a r k . c o m / c o n s u l t i n g
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Summary of key benefits
Collaborative Authoring
• Secure
• Browser-based
• Available globally
Blended Delivery
• Responsive Design
• Browser support
• Tablets/smartphones
• Accessibility
• Secure delivery for high stakes exams
Reporting and analytics
• How are my participants doing?
• How are my tests doing?
• Results API
Multilingual
• Translation Management
• Screens/Controls in 36 languages
• Right-to-left languages
Integration
• Supports industry standards
• APIs
• Connectors
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
What questions do you have?
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
w w w.quest ionmark .com/learn ingresources
eBooks, White Papers, Infographics and more!
Slide 65
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Connect!
Thank you for attending!
We hope to see you at a future webinarKeep up to date at blog.questionmark.com!