Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP,...
-
Upload
lisandro-cartlidge -
Category
Documents
-
view
215 -
download
0
Transcript of Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP,...
![Page 1: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/1.jpg)
Psychometrics Made Easy
Certification Exam PreparationJanuary 15, 2015
F. Sam Griggs, Jr., CCP, Cost Engineering Consultant
![Page 2: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/2.jpg)
IntroductionWhat Is It?
How Did We Get Here?
What Is Its Language?
What is the Process?
How Can You Pass the Exam?
Your Technical Paper
![Page 3: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/3.jpg)
Measurement
Theory
Technique
Psychometrics
![Page 4: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/4.jpg)
Psychometrics
Psychometric
s
Statistical Research Effect on
Measurement
Objective Measurement of Skills and Knowledge
Key Concepts are Reliability
& Validity
![Page 5: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/5.jpg)
How Did We Get Here?
8 Hour Hand Written Exam
ComputerizedTesting
Statistical Data
![Page 6: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/6.jpg)
Psychometrics Review
Removed Difficult
Questions Most Got Wrong
Removed Easy
Questions Most Got
Right
![Page 7: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/7.jpg)
Grading:
To determine a passing grade of any test, it must be justified with empirical data.
It should be legally defensible.
Meet the Standards for Educational and Psychological Testing.
![Page 8: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/8.jpg)
Glossary of TermsAlpha Coefficient
Angoff Method
Distractor
Item Difficulty
Item Distraction
Cutscore
Key
![Page 9: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/9.jpg)
Glossary of TermsMean
Mean P
Mean Rpbis
P-value
Point Biserial Correlation (Rbpis)
Standard Deviation
Standard Error of Measurement (SEM)
![Page 10: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/10.jpg)
Glossary of TermsReliability
Validity
Test Bias
Negative Correlation
SME
![Page 11: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/11.jpg)
William H. Angoff Distinguished research scientist
at ETS for more than 40 years.
Angoff Method: One of his many contributions to psychometrics.
Author of “Scales, Norms, and Equivalent Scores”.
![Page 12: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/12.jpg)
Angoff Method:
Most widely used
• Determines the passing % or cutscore.
Relies on SME
• SME are subject matter experts who examine each question.
• Predict how many minimally qualified candidates would answer correctly.
• The average of the SME’s predictions becomes its predicted difficulty.
• The sum of the predicted difficulty values averaged is the recommended cut score
Goal• Rate each test item • To determine whether a minimally qualified
candidate would answer the item correctly or incorrectly.
![Page 13: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/13.jpg)
Mechanics of Psychometrics
Statistics
Validity
Reliability
![Page 14: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/14.jpg)
ReliabilityA Reliable Measure: Measures test items
consistently across time, individuals, and situations.
Reliability is necessary, but not sufficient for validity.
Alpha Coefficient
RetestTest
![Page 15: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/15.jpg)
Validity
Valid Measure: Measures what it is intended to measure.
Sample question: What is the capital of Texas?
This question is a reliable measure but would not be a valid measure for the CCP exam.
A valid answer might be Houston.
A valid measure for a CCP exam must measure only CCP components such as Cost Estimating, Scheduling, Project Management, etc.
![Page 16: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/16.jpg)
Negative Correlation
This Negative number indicates that those who did well on the overall exam actually performed very poorly on this particular item.. Sometimes it indicates that the answer might be keyed incorrectly.
Sample question: What is the capital of Texas?
In this case, some candidates who passed the overall exam did not get this question correct – as marked. They selected the distractor “Austin”.
A valid answer was thought to be “Houston” and was erroneously marked as correct answer.
This statistic shows that a dis-proportionate number of successful candidates missed this question. So, those questions with Negative Correlation are reviewed very closely. Sometimes, candidates “over-think” the question.
![Page 17: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/17.jpg)
Test Bias
A form of systematic (non-random) error which leads to examinees from one demographic group having an advantage over examinees from another demographic group.
Test bias may cause differences in average scores across demographic groups.
Example: USA uses different system of measurement than Europe. They use metric system so they may have problems with our questions re measurement.
But differences in group scores are not sufficient evidence that Test Bias is present.
AACE Certification Board has committee working on problem – Goal is to reduce Test Bias.
![Page 18: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/18.jpg)
StatisticsBoth Reliability & Validity can be assessed
Statistically.
P-Value (Pass Rate or Item Difficulty)Point Biserial Correlation (Rbpis)Negative Correlation
Alpha Correlation
Statistical Assessments:
![Page 19: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/19.jpg)
Easy Test Question90% tested answer correctly.
Sample Question:
What is the Capital of Florida?
a. Miamib. Tallahasseec. Jacksonvilled. Orlando
![Page 20: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/20.jpg)
Easy Test Question
Sample Question:
What is the Capital of Florida?
a. Miamib.Tallahasseec. Jacksonvilled. Orlando
![Page 21: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/21.jpg)
Hard Test Question50% tested answer incorrectly.
Sample Question:
What is the Capital of Michigan?
a. Detroitb. Ann Arborc. Lansingd. Kalamazoo
![Page 22: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/22.jpg)
Hard Test Question Alternative is to change
distractor(s)
Sample Question:
What is the Capital of Michigan?
a. Detroit – change to say, Bostonb. Ann Arbor – change to say,
Cloquetc.Lansingd. Kalamazoo
![Page 23: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/23.jpg)
Hard Test Question50% tested answer incorrectly but what if we
changed distractor answers?
Sample Question:
What is the Capital of Michigan?
a. Bostonb. Cloquetc. Lansingd. Kalamazoo
![Page 24: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/24.jpg)
Hard Test Question
Sample Question:
What is the Capital of Michigan?
a. Detroitb. Ann Arborc.Lansingd. Kalamazoo
![Page 25: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/25.jpg)
Sample of Math QuestionLess than 60% tested answer incorrectly.
Sample Math Question:
What is $5,100 invested at 12% compounded yearly, at end of 6 years?
a. $5,712b. $8,988c. $10,066d. $98,950
![Page 26: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/26.jpg)
Solution to Math Problem
Solution : There are a couple of different ways to work this.
Start Begin Rate End
Year 1 $5,100 1.12 $5,712
Year 2 $5,712 1.12 $6,397
Year 3 $6,397 1.12 $7,165
Year 4 $7,165 1.12 $8,025
Year 5 $8,025 1.12 $8,988
Year 6 $8,988 1.12 $10,066
![Page 27: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/27.jpg)
Sample Math Problem - Tips for Solution
There are a couple of different ways to work this example.
Solution : Work up a Table shown on previous slide.
Solution : Remember the formulas. There should be some basic formulas you want to remember.
Solution : Remember the ‘Rule of 72”. To determine the amount of time to double an investment, divide 72 by the interest rate. In this case, 72/12 = 6 years. The $5k should double and maybe answer is approx $10k.
Get rid of obviously wrong answers. These would be answer a and d. Answer a is too small and d is too high. That narrows it down to 2 plausible answers. If you had to guess, odds are 50% instead of 25%.
![Page 28: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/28.jpg)
Certification Board Review:
Review the question to confirm that the marked answer is correct.
Change the wording of the question.
Modify the distractor to add or decrease the difficulty.
Make no changes.
Retire the question.
Steps in Making the Question Easier or Harder
![Page 29: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/29.jpg)
Top Tips to Get a Great Score
Purpose •Understand the purpose of the exam.
Expect •Know what to expect in the timed testing environment.
Attitude •A positive attitude is a most.
![Page 30: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/30.jpg)
Top Tips to Get a Great Score
Practice•Especially necessary for anyone who has never taken a Psychometric Exam.•Find websites & books that have practice test.•Focus on tests that specialize in your area(s) of weakness.
Time Yourself •Test yourself working against the clock.
Broaden Sample Test•Don’t rely only on the study guides.•Use AACE Recommended Practices (RP) on their web page.•Many of the new questions on exams come from the new RPs.
![Page 31: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/31.jpg)
Top Tips to Get a Great Score
Refresh Math Skills
• Fact 1: AACE has Math Questions that other exams do not.
• Fact 2: Cost Engineers are expected to know the Rate of Return and other Math problems.
Get a Good Night’s Sleep• Make sure you get a good night’s sleep before
the exam so you are fully alert.
Follow Instructions• Carefully read or listen to the instructions on
taking the exam.
![Page 32: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/32.jpg)
Top Tips to Get a Great Score
Answering Questions
• Trust yourself…Usually your first answer is correct.• Answer easy questions first, skip those you get
stuck on.• At the end, return to the questions you skipped.• Do not leave any questions unanswered. Eliminate
all obviously wrong choices.• Last resort…Guess if you need to as you have a
25% chance of getting it right.
Complete the Feedback Section on Exam
• Complete the Feedback Section at the end of the exam.
• Your comments or concerns help the Reviewers modify and improve future exams.
Don’t Give Up
• Keep trying until you Pass!• Identify what areas that kept you from passing.• Study these problematic areas to improve your
skills.
![Page 33: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/33.jpg)
Conclusion:
Achieve
Certification
Use Angoff Method
Learn the Language
Implement ProcessesStudy &
Prepare for Exam
![Page 34: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/34.jpg)
Technical PaperImportant information you should know before writing your paper
![Page 35: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/35.jpg)
Tips for a Winning Paper
Table of Contents
Introduction: Purpose & Background
Basis & Analysis of Your Data
References
1
2
3
4
5
Abstract
Make sure your paper Includes:
![Page 36: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/36.jpg)
Tips for a Winning Paper
Graphs or Tables to Support Premise
Logical Conclusion
Do Word Count: Minimum 2,500 Req.
Extra Points for Appendix
6
7
8
9
10
Organization & Clarity
Make sure your paper Includes:
![Page 37: Psychometrics Made Easy Certification Exam Preparation January 15, 2015 F. Sam Griggs, Jr., CCP, Cost Engineering Consultant.](https://reader037.fdocuments.net/reader037/viewer/2022110320/56649c955503460f949523d7/html5/thumbnails/37.jpg)
Questions??