Test Taking Skills - ACOFP€¦ · Test-Taking Skills Written Donald Sefcik, DO, ... (> 2.5 mm)...

Test-Taking Skills Donald J. Sefcik, DO, MS, MBA, FACOFP

7/28/2014

1

Test-Taking Skills

Written

Donald Sefcik, DO, MBA, FACOFP

Disclosure

Why are Standardized Tests so hard?

Faculty-generated tests Standardized tests

7/28/2014

2

Introductory Comments

• Do you know someone that:

– Studied LESS and scored HIGHER than you?

What’s the impact of study hours COMLEX Level 1 exam

Hours Studied <100 101-150 151-200 201-250 251-300 301-350 351-400 >400

Mean Score 552.75 533.50 536.17 530.43 545.95 549.85 523.00 536.25

538.2 538.8

MUSCOM Research; AACOM 2012

Hours <100 101-150 151-200 201-250 251-300 301-350 351-400 >400

Maximum 599 539 581 657 778 717 679 692

Minimum 480 528 428 428 415 382 397 367

Range 119 11 153 229 338 311 330 325

What the C1 mean doesn’t tell us

128 326

MUSCOM Research; AACOM 2012

7/28/2014

3

Higher Scorers

_

X

???

Neural Plasticity

Study

Techniques

Study

Techniques

Higher Scorers

_

X

7/28/2014

4

Overall View

“Have to…”

“Get to…”

Highest Scorers

Lowest Scorers

Study Techniques / Resources

Efficient

Effective

Highest Scorers

Lowest Scorers

Majority of Effort

Preparation

Exam Day

Highest Scorers

Lowest Scorers

7/28/2014

5

Skill Development

Test-

Wiseness

Test-

Taking

Highest Scorers

Lowest Scorers

Maximizing your SCORE

Test-Wiseness Test-Taking Skills

• Reading Accuracy

• Reading Fluency

• Guessing

• Changing Answers

• Test Blueprint

• Long-term Memory

• Transfer

• Self-Regulation

TEST

ANXIETY


EXAM DAY


• Reading Fluency

• Guessing


7/28/2014

6

Example

Infarction of the right ventricle occurs when the proximal

segment of the right coronary artery is occluded. Which of the

following is the most sensitive electrocardiographic (EKG)

finding of a right ventricular infarction?

A. ST segment elevation in I, aVL, V5 and V6

B. ST segment depression in V1, V2 and V3

C. ST segment elevation in V1, V2 and V3 with

ST segment depression in aVF

D. ST segment elevation in V4R with an upright T wave

E. ST segment elevation in V1 (> 2.5 mm) with a

right bundle-branch block

Example












STEM

OPTIONS

Example

Which of the










Interrogatory

7/28/2014

7

Example












Interrogatory

Distractors

Impact of Stem (Vignette)

Examinees with

Higher Scores

Examinees with

Lower Scores

No Vignette 99% 90%

Short Vignette 98% 82%

Long Vignette 98% 66%

http://www.nbme.org/pdf/itemwriting_2003/2003iwgwhole.pdf

Examinees with

Higher Scores

Examinees with

Lower Scores

No Vignette 99% 90%



Reading

A patient with hyponatremia and highly concentrated urine is noted

on CXR to have a centrally-located, suspicious-appearing, solitary

pulmonary nodule. Which of the following is the most likely explanation?

A. small cell carcinoma

B. squamous cell carcinoma

C. adenocarcinoma

D. large cell carcinoma

E. mesothelioma

EXAMPLE

7/28/2014

8

Pulmonary Masses Apical

TB

Peripheral

Large Cell Carcinoma

Adenocarcinoma Central

Small Cell Carcinoma

Squamous Cell Carcinoma

Pleural-Based

Mesothelioma

A patient with hyponatremia and highly concentrated urine is noted

on CXR to have a centrally-located, suspicious-appearing, solitary

pulmonary nodule. Which of the following is the most likely explanation?

A. small cell carcinoma (central)

B. squamous cell carcinoma (central)

C. adenocarcinoma (peripheral)

D. large cell carcinoma (peripheral)

E. mesothelioma (pleural-based)

EXAMPLE

SIADH

Reading Accuracy

Exam Day

Performance

• Selecting Answers

Know for Sure (Record & Confirm Choice)

Familiar but Not Sure (Maximize Probability)

No Idea (Test-Taking Skills; Guessing Better)


• Reducing Test Anxiety (Exam Site Issues)

7/28/2014

9

• Focus on the Stem – Find the BEST answer

– If you know if after reading the stem, “go for it”

• Read ALL options before recording answer – Avoid partly correct responses

• Go with instinctive (GUT) feeling – You picked it for a reason

• Once decide incorrect – Do not consider it again

• Avoid “reading into” the question – The stem sets the task

During the Examination

Question

What probable etiologies are suggested when drugges foss

in extremis?

A. Sloorgs thanceed with the droxemia

B. Potcoms fotted secondary to an oral deficiency

C. The dagts ptopted and the frodds roanzed

D. The dhiths buazzed secondary to gastrointestinal crooleets

E. Taggoks were removed from the juklets

Question

Patients suffering from strugleum frequently undertrosl the gruifreit

because:

A. all patients with strugleum have krodncurz.

B. strugleum always presents with qugruestful.

C. there are never cases of strugleum without feskabolose.

D. patients with strugleum commonly have gizzlehorst.

E. no fuegrools results in strugleum.

7/28/2014

10

Question

Along the eastern coast of Maine, the cases of huestoolduds are

most often secondary to:

A. poor nutrition.

B. the moisture in the frocklesters.

C. the abundance of greezles.

D. a relative abundance of kleasternezers.

E. an oversupply of fhoulsteens that occur directly as a result of

kjuoost, djurts and bhespernets maldoorneys.

Question

The drug of choice for patients experiencing

zhiglerastasis secondary to kweatlostosis is:

A. dragrootmycine.

B. brusterocilline.

C. noopkweetosporine.

D. zhiglerpurocine.

E. pneustodregcycline.

7/28/2014

11

When GUESSING is the ONLY alternative

General Alternative

If four of the five options are very specific and one is

very general…

Often the more general option is correct response



Longest Alternative

Often the exam writer wants to be sure you have

enough information to recognize the correct

Response.


Chase 1964



Longest Alternative

Question

Along the eastern coast of Maine, the cases of huestoolduds are

most often secondary to:

A. poor nutrition.

B. the moisture in the frocklesters.

C. the abundance of greezles.

D. a relative abundance of kleasternezers.

E. an oversupply of fhoulsteens that occur directly as a result of

kjuoost, djurts and bhespernets maldoorneys.

7/28/2014

12


Grammar Agreement The stem and responses should be grammatically

correct…

Singular stem should not be completed by plural

option


Broad & Whitney 1972



Grammar Agreement

Question

What probable etiologies are suggested when drugges foss

in extremis?

A. Sloorgs thanceed with the droxemia

B. Potcoms fotted secondary to an oral deficiency

C. The dagts ptopted and the frodds roanzed

D. The dhiths buazzed secondary to gastrointestinal crooleets

E. Taggoks were removed from the juklets


Specific Determiners The implication of absoluteness (always, never,

must, none, only…) is often incorrect.

Words that permit exception (seldom, usually, often,

perhaps, etc.) are more commonly associated with

the correct response.


Milman 1965

7/28/2014

13



Specific Determiners

Question

Patients suffering from strugleum frequently undertrosl the gruifreit

because:

A. all patients with strugleum have krodncurz.

B. strugleum always presents with qugruestful.

C. there are never cases of strugleum without feskabolose.

D. patients with strugleum commonly have gizzlehorst.

E. no fuegrools results in strugleum.


Clang Association Similarity between a word in the stem and correct

response.


Diamond & Evans 1972



Clang Association

Question

The drug of choice for patients experiencing

zhiglerastasis secondary to kweatlostosis is:

A. dragrootmycine.

B. brusterocilline.

C. noopkweetosporine.

D. zhiglerpurocine.

E. pneustodregcycline.

7/28/2014

14


Deductive Approach (Convergence) A. W & X

B. Y & Z

C. X & Y

D. V & Z

E. X & Z


Smith 1982


Deductive Approach (Convergence) A. W & X V = 1

B. Y & Z W = 1

C. X & Y X = 3

D. V & Z Y = 2

E. X & Z Z = 3


Smith 1982

A patient with a history of angina and syncope is noted to have

a systolic murmur. Which one of the following sets of findings

distinguishes chronic mitral regurgitation from other etiologies?

A. Laterally displaced PMI, normal S2, murmur intensity decreases with Valsalva

B. Normal PMI, split S2, murmur is loudest during midsystole

C. Laterally displaced PMI, normal S2, murmur is loudest during late systole

D. Sustained PMI (heave), normal S2, murmur radiates to apex

E. Laterally displaced PMI, split S2, murmur intensity decreases with Valsalva

Deductive Approach

7/28/2014

15

A patient with a history of angina and syncope is noted to have

a systolic murmur. Which one of the following sets of findings

distinguishes chronic mitral regurgitation from other etiologies?

A. Laterally displaced PMI, normal S2, murmur intensity decreases with Valsalva

B. Normal PMI, split S2, murmur is loudest during midsystole

C. Laterally displaced PMI, normal S2, murmur is loudest during late systole

D. Sustained PMI (heave), normal S2, murmur radiates to apex

E. Laterally displaced PMI, split S2, murmur intensity decreases with Valsalva

Deductive Approach

Guessing Strategies

• Changing Answers….

– 1/5 = What is percent chance right?

– If first guess = 20% (1/5)

– Second guess = 25% (1/4)

– BUT - If right and change it, now 100% wrong!

7/28/2014

16




• Reading Fluency

• Guessing


• Test Blueprint


• Transfer

• Self-Regulation

TEST

ANXIETY

Research indicates:

Optimal amount of stress (maximizes performance)

Yerkes-Dodson Law 1908

High

Performance Test Anxiety

Low

Low Stress High

Test Anxiety

Can Cause

• Psychological Changes – GI Symptoms, Insomnia, Headaches

• Emotional Changes – Irritability, Moodiness, Frustration, Anger

• Cognitive Changes – Memory Blocks, Attention Span Limitations

Test Anxiety

7/28/2014

17

Caused by

• Lack of Preparation – Cramming

– Procrastination

– Poor Study Habits

• Recurring Thoughts

– During the exam

– BEFORE the exam

Test Anxiety

Ways to Reduce

• Plan Studying & Improve Study Skills

• Understand the Examination – Test Blueprint (Table of Specifications)

– Pilot Items

• Build Confidence – Study effectively & efficiently

Test Anxiety



• Test Blueprint


• Transfer

• Self-Regulation

7/28/2014

18

Plan

Activity (a verb not a noun)

• Select Topics (AOBFP Exam Blueprint)

• When to Study (Metacognition & Self-Regulation)

• How to Study (Long-Term Memory & Transfer)

• Ways to Reduce Test Anxiety (Pilot Items)

PLANNING

for

Standardized

Tests

Scoring

Content

Skills

Timed

Guessing

Items

Standardized Tests - PLANNING

Test-Wise Test-Takers: First Step in Planning

7/28/2014

19

Blueprint – Why Important?

Exam Content %

Addiction Medicine 3

Adolescent Medicine 4

Behavioral Sciences 12

General Medicine 48

Geriatrics 5

Surgery 13

Obstetrics / Gynecology 4

Pediatrics 4

Sports Medicine 3

Women’s Issues 4

http://www.aobfp.org/cert-req/cert-app.html

Blueprint – Distribution of Items

Exam Content %



Behavioral Sciences 12 Weighting

General Medicine 48 Weighting

Geriatrics 5

Surgery 13 Weighting


Pediatrics 4

Sports Medicine 3

Women’s Issues 4


Breakdown by Discipline

General Medicine 48%

Allergy / Immunology / Rheumatology 5

Cardiology 5

Dermatology 5

Endocrinology 5

Gastroenterology 5

Hematology 4

Nephrology / Urology 4

Neurology 5

OPP 5

Pulmonology 5


7/28/2014

20


Behavioral Sciences 12%

Preventive Medicine 5

Psychiatry 4

Medical Jurisprudence 3



Surgery 13%

EENT 5

General Surgery 3

Orthopedics 5


Blueprint – Guides Study Order

Exam Content % Interest




General Medicine 48

Geriatrics 5

Surgery 13


Pediatrics 4

Sports Medicine 3

Women’s Issues 4


7/28/2014

21

Blueprint – Be Mindful of Priorities

Exam Content %




General Medicine 48 DS #10

Geriatrics 5

Surgery 13

Obstetrics / Gynecology 4 DS #1

Pediatrics 4

Sports Medicine 3

Women’s Issues 4

INVERSE of INTEREST

Ways to Enhance

MEMORY

Information

Storage

Retrieval

Recognition Recall

Information IN

Storage

Study Habits

Learning Style

Organization

Retrieval

Associations

Understanding

Cues

Information OUT

7/28/2014

22

Retrieval

Recognition Recall

Less Active

Factual

Cues Present

Repetition

Cramming

More Active

Problem Solving

Cue Absent

Bridges

Strategies

Ways to Enhance the Process

Remembering Transfer

30% 70%

PLANNING

for

Standardized

Tests

Scoring

Content

Skills

Timed

Guessing

Items


Pilot Items

1. Can’t identify them

2. Don’t count

3. Capitalize on presence

Preparation

PRACTICE

• Study Techniques (Clinical Vignettes)

Study Aids (Which Resources to Use)

Study Behaviors (Reading; Q&A Activities)

7/28/2014

23

Preparation

Improving Performance

Content Context (Information) (Abilities and Attitude)

Study Aids Study Behaviors

• Total-Time Hypothesis

Reading next day (Distributive Trials) • Reduces time required to “relearn it”

• Increase memory

Time Management

Ebbinghaus 1880; Medina 2008

7/28/2014

24

• Escape Syndrome

– Sitting around thinking about/talking about

“how bad it’s going to be”…

Time Management

• Understand & Associate (Meaning)

– Meaningless Facts memorized

• Day 5 = Recall ~ 40%

• Day 30 Recall < 10%

– Principles Understood with Meaning

• Day 5 = Recall > 95%

• Day 30 Recall > 90%

Self-Directed Learning Methods

Maddox 1964; Medina 2008


• Develop a Strategy

– Choose strategy to meet your needs

Reading

• 50% of time should be spent reflecting

• Additional Suggestions

– Paraphrasing (increase understanding)

– Retrieval (create notes from memory)

McWhorter 1998

7/28/2014

25

• Self-Evaluation

– Be sure “Closing the Gap”

• Question-and-Answer Resources

–Maximize potential


Impact of Stem (Vignette)

Examinees with

Higher Scores

Examinees with

Lower Scores

No Vignette 99% 90%



http://www.nbme.org/pdf/itemwriting_2003/2003iwgwhole.pdf

Examinees with

Higher Scores

Examinees with

Lower Scores

No Vignette 99% 90%



Q&A resources

• Fluency

– Practice reading faster

– 10 minutes = 10 items; 60 minutes = 60 items

– Do not look up any answers until finished!

– Better if global not by topic / content

7/28/2014

26

Q&A resources

• Fluency

– What else do we know about reading?

– When the answer is not readily apparent…

– Lower scorers: stem-options-stem-options

– Higher scorers: stem-stem-stem…

Q&A resources

• Accuracy

– Feedback is critical to your success!

– Read explanations before you score yourself

– When you calculate your score, break it down

Q&A resources

• Accuracy

– Break your score into component parts

– 1. A*

– 2. C^

– 3. D#

A* = I know this

C^ = I have some familiarity with this

D# = I have no idea

7/28/2014

27

Q&A resources

• Accuracy

– Feedback

– Overall score = 47/60 (78%)

– BUT…

* = 11/20 (55%)

^ = 17/20 (85%)

# = 19/20 (95%)

This is a problem!

Q&A resources

• Accuracy

– Anticipation is critical to your success!

– For every practice item manipulate the stem to

make each distractor become a correct answer.

Example

A 64yo male with thoracic complaints….

Historical and Physical features

EKG, Labs, X-rays, other…

A. Myocardial Infarction

B. Pericarditis

C. Pulmonary Embolism

D. Dissecting Thoracic Aortic Aneurysm

E. Somatic Dysfunction

7/28/2014

28

Example

A 64yo male with chest tightness….

Hx: HTN and Chol; PE: Carotid Bruits

ST Segment elevation (II, III & aVF); CK-MB


B. Pericarditis




Example

A 64yo male with acute pleuritic pain….

Hx: Virchow’s Triad; PE: Sinus Tachycardia

EKG: S1Q3 3; CXR: hemidiaphragm elev


B. Pericarditis




Why are Standardized Tests so hard?

Faculty-generated tests Standardized tests




• Reading Fluency

• Guessing


• Test Blueprint


• Transfer

• Self-Regulation

TEST

ANXIETY

PLANNING

for

Standardized

TestsScoring

Content

Skills

Timed

Guessing

Items


Test-Wise Test-Takers: First Step in Planning

7/28/2014

29

References 1. West DC et al. Critical Thinking in Graduate Medical Education.

JAMA. 2000;284(9):1105-1110

2. McGaghie WC, et al. What is the Impact of Commercial Test Preparation Courses on Medical Examination Performance? Teaching and Learning in Medicine. 2004;16(2):202-211.

3. Werner LS, Bull BS. The Effect of Three Commercial Coaching Courses on Step One USMLE Performance. Medical Education. 2003;37:527-531

4. Sefcik DJ, Prerost FJ, Arbet SE. Personality Types and Performance on Aptitude and Achievement Tests: Implications for Osteopathic Educators Journal of the American Osteopathic Association. 2009;109(6):296-301

5. Millman J, Bishop CH. An Analysis of Test-Wiseness Educational and Psychological Measurement. 1965;25(3):707-726

6. Evans W. Test Wiseness: An Examination of Cue-Using Strategies. Journal of Experimental Education. 1984;52(3):141-144

7. Dolly JP, Williams KS. Using Test-Taking Strategies to Maximize Multiple-Choice Test Scores. Educational and Psychological Measurement. 1986;46(3):619-625

8. Norman GR et al. Factors Underlying Performance on Written Tests of Knowledge. Medical Education. 1987;21:297-304

9. Wenghofer E et al. Doctor Scores on National Qualifying Examinations Predict Quality of Care in Future Practice. Medical Education 2009;43:1166-1173.

10. Baeten M, Kyndt E, Struyven K, Dochy F. Using Student-Centered Learning Environments to Stimulate Deep Approaches to Learning: Factors Encouraging or Discouraging Their Effectiveness. Educational Research Review 2010;5:243-260.

11. Skochelak SE. A Decade of Reports Calling for Change in Medical Education: What Do They Say? Academic Medicine 2010;85(9):S26-S33.

12. Prince M. Does Active Learning Work? A Review of the Research. Journal of Engineering Education. 2004;7:223-231.

13. Winne PH, Nesbit JC. The Psychology of Academic Achievement. Annual Review of Psychology 2010;61:653-678.

14. Laatsch L. Evaluation and Treatment of Students With Difficulties Passing the Step Examinations. Academic Medicine. 2009;84(5):677-683.

15. Jairam D, Kiewra KA. An Investigation of the SOAR Study Method. Journal of Advanced Academics 2009;20(4):602-629.

16. Michael J. Where's the Evidence that Active Learning Works? Advances in Physiological Education. 2006;30:159-167.

17. Light, Greg, Roy Cox, and Susanna Calkins. Learning and Teaching in Higher Education 2nd ed. London: SAGE, 2009.

18. Medina, J., Brain Rules. Seattle, WA: Pear Press, 2008.

19. Sefcik D, Bice G, Prerost F. How to Study for Standardized Tests. Jones & Bartlett Learning, 2013.

Test Taking Skills - ACOFP€¦ · Test-Taking Skills Written Donald Sefcik, DO, ... (> 2.5 mm)...

Documents

Transcript of Test Taking Skills - ACOFP€¦ · Test-Taking Skills Written Donald Sefcik, DO, ... (> 2.5 mm)...