NSAS Voucher Program Guide_Layout 1

Voucher Program Guide

NBMESelf-Assessment

Services

Overview ...........................................................................................................................................3

Background ......................................................................................................................................................3

NBME Self-Assessment Services ..........................................................................................................................3

Ownership and Copyright ..................................................................................................................................3

Terms, Conditions, and Disclaimer ......................................................................................................................3

Performance Feedback.......................................................................................................................................4

Voucher Ordering..............................................................................................................................5

NBME Store Account Setup ................................................................................................................................5

Voucher Payment Methods..................................................................................................................................7

Voucher Order Cancellations .............................................................................................................................8

NBME Store Account Credit................................................................................................................................8

Voucher Information Program (VIP) .................................................................................................9

Exam Form Assignment ....................................................................................................................................10

Download and Distribution of Vouchers .............................................................................................................10

Voucher Redemption and Administration.......................................................................................11

Voucher Redemption........................................................................................................................................11

Administration Support.....................................................................................................................................11

Standard Setting Administration...................................................................................................................11

Examine Administration ..............................................................................................................................12

Access to Score Performance Feedback ..........................................................................................13

Examinees Accessing Score Reports...................................................................................................................13

Institutions Accessing Score Reports ...................................................................................................................13

Contact Us........................................................................................................................................14

Table of Contents

NSAS Voucher Program Guide ￨ JUNE 2017

Copyright © 2006, 2009-2017 National Board of Medical Examiners® (NBME®). All rights reserved.

BACKGROUND

The NBME Self-Assessment Services (NSAS) Voucher Pro-gram is offered to medical schools, residency programs,and medical education organizations (“the Institution”)interested in purchasing self-assessments for their stu-dents. Institutions are able to:

• Order vouchers for the self-assessment through anonline NBME Store

• Assign a specific form number to voucher IDs

• Download prepaid voucher IDs and distribute themto students for requesting a self-assessment

• Track the redemption of vouchers (redeemed versusunredeemed)

• Track voucher expiration

• Monitor the completion of self-assessments(launched or completed)

• View performance feedback for completed self-assessments

SELF-ASSESSMENT SERVICES

NSAS is offered to both domestic and international med-ical students and graduates. Web-based self-assess-ments include:

• Comprehensive Basic Science, Clinical Science, andClinical Medicine self-assessments allow examineesto evaluate their readiness to take the USMLE Step 1,Step 2 Clinical Knowledge (CK), and Step 3 exami-nations.

• The Clinical Science Mastery Series helps examineesto gauge their comprehension of the clinical sciencescovered during a clerkship or medical educationcourse.

• Self-assessments for the International Foundations ofMedicine® (IFOM®) Clinical Science Examination arefor examinees preparing to take that exam or foranyone wishing to evaluate medical knowledge andunderstanding of clinical science considered essen-tial internationally for the provision of safe and effec-tive patient care.

OWNERSHIP AND COPYRIGHT

The materials presented on the self-assessments areowned and copyrighted by the NBME. Participants maynot transfer or reproduce self-assessment materials inany way without permission from the NBME. Any unau-thorized transfer or reproduction of these materials, byany means, including but not limited to, storage in a re-trieval system, transmission, printing, memorization, ordistribution is strictly prohibited. If the NBME believesthat any participant has engaged in or is engaging in theforegoing behavior, the NBME reserves the right to takeany and all further action necessary to protect the in-tegrity of the examination, including but not limited tonotifying the participant’s medical school or residencyprogram of the suspected unauthorized disclosure ofcopyrighted materials and taking legal action. Any ac-tions taken by a medical school or residency program onthe basis of such information are completely the respon-sibility of the medical school or residency program. Seefull disclaimer.

TERMS, CONDITIONS AND DISCLAIMERS

The material presented in this self-assessment is providedby the NBME for educational purposes only. The self-as-sessments are not intended to predict performance onthe United States Medical Licensing Examination®

(USMLE®) or the NBME subject examinations. Rather, theself-assessments are intended to be used as a tool to de-termine a participant’s strengths and weaknesses in gen-eral topic areas.

Overview

NSAS Voucher Program Guide ￨ JUNE 2017 3

PERFORMANCE FEEDBACK REPORTS

Participants receive immediate feedback in the form ofperformance profiles that indicate relative strengths andweaknesses in general topic areas. For CBSSA andCCSSA, participants receive a performance profile and ascore interpretation guide immediately after completinga self-assessment. These profiles are similar to those thatcandidates receive for USMLE Step 1 and Step 2 CK.The score interpretation guide allows participants to“translate” an assessment score to an approximate cor-responding USMLE Step 1 or Step 2 CK total score. ForCCMSA, participants receive a performance profile only.The performance profile provides an assessment scoreand graphically displays relative strengths and weak-nesses in general topic areas.

The NBME does not guarantee that any individual per-formance on NSAS will predict performance on USMLEStep 1, Step 2 CK, or Step 3, and cautions participantsagainst making any predictive inferences.

• An individual performance profile with diagnosticfeedback that highlights areas of strength and weakness; *

• A score interpretation guide with a conversion table that translates an assessment score to an approximated total score on the USMLE score scale;**

• The number of items answered incorrectly by contentcategory;

• The ability to review the text of items answered incorrectly with their correct answers (explanationsfor the correct answers will not be included); and,

• The ability to review longitudinal feedback is available with their correct answers with diagnosticfeedback.

NOTE: The correct answer feature will be available forthe Clinical Science Mastery Series (CSMS) and Interna-tional Foundations of Medicine (IFOM) Clinical Scienceself-assessments in 2018.

* Not available for Mastery Series and IFOM self-assessments.

** Not available or CCMSA.

Longitudinal Performance Feedback

Participants receive longitudinal performance profilesthat display results of the six most recent assessmentsthey have completed. This allows participants to tracktheir progress over time and view a graphical display oftheir progress in each content category. Longitudinalfeedback is not provided for the Clinical Science MasterySeries or the IFOM CSE self-assessment at this time.

Sample Reports and Additional Resources

Sample performance reports and the resources listedbelow are available online athttps://nsas.nbme.org/home:

• Fees, payment options and policies• Frequently asked questions• Sample self-assessment with a tutorial• System Requirements


Overview

NSP is a secure website and a primary source for authorized medical school users to obtain information about theirstudents for a variety of activities related to USMLE, NBME subject examinations, NSAS Voucher Program, and otherservices.

Each medical school has at least one User Account Administrator (UAA) who is responsible for assigning users theprivileges they need to access areas of the portal appropriate for their responsibilities. The UAA must assign users the NSAS Voucher Service permission to access the NBME Self-Assessment Services portlet, which houses the NBME Store and Voucher Information Program system.

NOTE: NBME Account Administrators are responsible for creating accounts for residency programs and medical or-ganizations as well as assigning user privileges to the NSAS Voucher Service.

NBME STORE & ACCOUNT SETUP

Self-assessment vouchers are ordered through an online NBME Store. The store will allow you to shopfor a variety of NBME services. Users will not be able to place an order if they do not have access to astore billing account. UAAs can create institution store billing accounts and assign local users access to them.NBME Account Administrators can do the same for residency programs and medical organizations. The NBME re-quires that an institution store billing account be set up for the purposes of placing an order and applying credit. In-structions for creating a NBME Store Billing Account is available on the NBME Store Portlet.

Voucher Ordering


After entering the online store, users will be able to browse by category and view a description of each product, add aproduct to their shopping cart, select a method of payment, track the status of orders, as well as view and downloadinvoices. Click here for self-assessment voucher fees.

NOTE: Participants receive longitudinal performance feedback at no cost when they take more than one self-assessment.


Voucher Ordering

PAYMENT METHODS

Payment for services may be made by:

• Credit card (Visa or MasterCard only)• Check or money order• Wire transfer• Available credit (Institution account with a credit balance)

Self-assessment vouchers ordered with a credit card or account credit will be made available within15 minutes of order placement. When using account credit, the credit must be equal to or less than the balancedue. Payments cannot be split into two different payment methods. Payment methods cannot be changed once theorder is submitted. However, if payment is remitted with a check or wire, the vouchers will not be madeavailable until the order is paid in full.

A confirmation e-mail with an invoice is provided immediately after order submission. Users can view all current andpast order history from the order history tab.

NOTE: Since vouchers are ordered through a separate platform than the web-based NBME subject examinations,funds cannot be transferred from one platform to another.


Voucher Ordering

ORDER CANCELLATIONS

Order cancellations may be made through the NBME Store. After users access the Order History page, they will beable to click on the order number and select the Cancel button. However, if the order has been fulfilledand the vouchers have been generated, users will not be able to select this option. They will need tosubmit a cancellation request to [email protected].

NBME STORE ACCOUNT CREDIT

Vouchers are good for one year from the NBME date of issue. Institutions will receive credit for unusedvouchers associated with orders submitted through the NBME store. This credit is applied to the institution store ac-count once the vouchers have expired. The credit may then be used toward future orders. User Account Administratorsand staff with permissions to order vouchers can review account credit by clicking on Manage Store Billing Accountsbutton on the NBME Store Portlet.


Voucher Ordering

The VIP system allows institutions to assign a form to vouchers, download and distribute voucher IDs to students, trackvoucher activity, monitor completion of assessments, and view performance reports.

When users access VIP, they will see a Voucher Order History page and by default be able to access voucherIDs for orders they have placed. Users have the option to select All Vouchers from the Select View drop-downmenu located in the left-hand corner above the voucher history table to view the entire voucher order history oftheir institution.

The Not Redeemed link associated with an order allows a user to view unused voucher IDs and their end or expira-tion date. The Redeemed link will allow a user to track voucher activity – examinee name, voucher ID used, self-as-sessment status (launched versus completed), form selected, timing mode, scaled score, and a performance feedbackreport.

Voucher Information Program (VIP)


FORM ASSIGNMENT

All vouchers are processed without a form designation. However, the institution has the capability to assign a specificform to voucher IDs through the VIP system. A drop-down menu will provide the number of the forms available; thehigher the number of the form, the more current. Located within VIP, under the General Instructions drop-down menu, are instructions for form assignment.

NOTE: If a form is not assigned, then the voucher IDs may be redeemed for “any form” of the self-assessment re-quested.

DOWNLOAD AND DISTRIBUTION OF VOUCHER IDS

After users click on the Not Redeemed link to view their voucher IDs, they can select the PDF iconin the right-hand corner. This feature will allow users to download and merge voucher IDs withredemption instructions into a PDF. Institutions may then distribute a voucher ID along with instructions to eachexaminee. Users also have the option to download voucher IDs into a spreadsheet by selecting the Excel icon.


Voucher Information Program

VOUCHER REDEMPTION

NBME self-assessments are purchased and launched through the NSAS interactive website(https://nsas.nbme.org/home). Resources and materials available on the website include a sample assessment,system requirements, and sample performance reports. Since the self-assessments are a non-secure service, a proctor-ing system is not required for administration as with subject examinations.

Examinees will be required to:

• Create and activate their account prior to acquiring a self-assessment;• Select the type of assessment and form for administration;• Choose the timing mode (standard-paced versus self-paced);• Select Voucher as the method of payment and enter the voucher ID; and• Start their self-assessment.

NOTE: The method of payment for a self-assessment cannot be split.

ADMINISTRATION AND SUPPORT

Examinees should be provided a voucher ID and voucher redemption instructions in advance.There are two different approaches to administration.

Standard Setting Administration

If you are going to have examinees take the self-assessment in a classroom setting, you will need to allow enoughtime for account setup (examinees will need to activate their account via e-mail), purchasing, the tutorial, and admin-istration of the assessment. Below is a table that displays the timing modes (excluding the 15-minute tutorial). Pleasevisit the NBME website to view timing modes.

Voucher Redemption and Administration


Standard-paced is selected by default as the timing mode when a self-assessment is chosen. Thetiming mode cannot be changed after an assessment is started.

Examinees will have to review an online tutorial before they begin their self-assessment. The tutorial will describe howto answer items, skip items, review responses, and change answers. Here are a few helpful tips for institutions:


Voucher Redemption and Administration

During Administration

Examinees will be prompted to review an online tutorial before they begin their assessments. They can skip the tuto-rial if they have previously completed a self-assessment. After students calibrate their monitor, the next page they willsee is the tutorial. The “SkipTutorial” option is located at the bottom of the screen, on the left-handside. Students should only skip the tutorial if they are familiar with taking a self-assessment.

Immediately Following Administration

After examinees complete their assessment, they will be able to view their performance immediately in the online ap-plication. After the students advance through the online application to review their feedback, they will have the op-tion to complete a survey or select “End.” If they select “End,” this will prompt the system to end the session andclose the window.

After exiting the test application, examinees should go to the Current Status page of their NSAS account. Theywill need to “Log Out” of their account and then exit the browser window.

If examinees “Pause” a section and resume it later, they will return to the same section and item in which they wereworking, provided the total administration time allotted has not run out. If participants select “Pause” while workingin a section, the time clock stops for that section only and the browser will close.

Before examinees complete a section, they must select “End” in order to proceed to the next section. If they end asection, or if time expires for a section, they will not be able to return to that section to review items or change an-swers.

Examinee Administration

The same rules apply to both standard setting and examinee administration. Examinees will have to set up an accountas well as select an assessment, form, and timing mode. Standard-paced is selected by default as the timing modewhen a self-assessment is chosen. However, examinees may choose self-paced for their assessment.

Once the examinees redeem the voucher, they will have 30 days to launch the self-assessment. Once launched, theexaminee will have 20 days to complete the self-assessment (an expiration date is provided within the ap-plication). Examinees will have to review an online tutorial before they begin their self-assessment.

EXAMINEES ACCESSING SCORE REPORTS ON NSAS SITE

Examinees will receive their feedback immediately after they have completed their self-assessment on-screen throughthe testing application.

INSTITUTIONS ACCESSING SCORE REPORTS ON NSP

Institutions will be able to view performance feedback for completed self-assessments by access-ing the Redeemed link in the VIP system. However, it will take up to 24 hours for the feedback results to beposted to the system. NSP users will be able to view the examinee name, voucher ID used, form selected, timingmode, and scaled score. When users click on the View link, they can view and download perform-ance feedback reports in a PDF format. Users can also download the information they see on-screen (excluding the PDF report) by clicking on the Excel icon.

NOTE: An aggregate report is not provided for the NBME Self-Assessments since this service is based on individualperformance.

Access to Score Performance Feedback


Please feel free to contact us if you have any questions. Various NBME Staff can provide the information you need orhelp solve a problem.

Contact Us


[email protected]

NBME Contact List

NBME Self-Assessment Voucher Program Questions OnlyAvailable Monday-Friday between 9:00am-5:00pm except for week-ends and holidays.

[email protected], option 1

Operational Support and All Other InquiriesAvailable 24 hours a day, 7 days a week except for holidays.

NSAS Voucher Program Guide_Layout 1

Documents

Transcript of NSAS Voucher Program Guide_Layout 1