Welcome to Unit III Recognition Items
description
Transcript of Welcome to Unit III Recognition Items
Welcome to Unit III
Recognition Items
EDS 596 Assessment Techniques in Counseling
Usual DisclaimerAvoid prolonged contact with skin
Not to be taken internallyNo animals were harmed during the
production of this presentation
After viewing this presentation avoid operating mobile or dangerous equipment
This presentation contains graphic images which some viewers may find disturbing.
Definition of Test
“…a task or series of tasks used to collect a sample of human behavior assumed to be representative of a
large domain.”
Typically a test is used in an attempt assess or measure some quality or characteristic of an individual. Tests are used to judge, grade, or evaluation someone.
TestsTests are composed of items;
Each item is designed to elicit a response (simple or complex)
The response is evaluated or compared to a standard and a judgment made (right-wrong/correct-incorrect)
Tests are frequently used in the assessment of Achievement, Aptitude or Intelligence.
Items typically fall into one of two large families of item forms
Recognition and Supply
Recognition Items•Highly structured
•Restricted Response Freedom
•Objectively Scored
This family of items includes:The alternative response item
The Matching Exercise and …
The multiple choice/Interpretive Exercise
Some beginning thoughts….a test item represents a link in a chain and is used for one of two purposes….
1. To assess “mastery” of some body of material or skill….or
2. To measure differences in mastery or levels of achievement among students/clients…..
As a link in the chain each item must have one characteristic. Ideally, a client or student who does not know the material should miss it and a student who knows the material or has mastered the material should be able to get it right! The item should
discriminate.
Recognition Items……
Advantages:A. Permits excellent item sampling
B. Easy, quick, and objective scoring
C. Writing is minimized…Primitive Responses
D. Generally good for measuring simple outcomes…
E. No skill is needed for scoring
F. Quick return of scores and scored tests…
Recognition Items……
Disadvantages:A. Bright students may be penalized…
B. Guessing is always possible
C. Items may encourage rote learning….not always bad…
D. Sometime used to measure “trivial” material.
Recognition Items……General Guidelines
A. Test only for important ideas, information, & skills
B. Write items as simply and directly as possible
C. Include only ONE correct or best answer.
D. Sometime used to measure “trivial” material.
E. Avoid “Chaining” questions
F. Avoid specific determiners
G. Never “pull” questions or material from text.
H. Avoid “trick” questions
I. Always review items prior to printing test
The Alternative Response Item
Advantages:A. Excellent item sampling
B. Easy to write and score
B. Good for measuring “simple” outcomes.
C. Good item for clients with poor reading ability
Form: A simple declarative sentence requiring a dichotomous judgment.
The Alternative Response Item
Disadvantages:A. Ease of successful guessing (50%)
B. Encourages rote learning…not always bad
B. Inability to measure complex outcomes
C. Limited to dichotomous material
D. Susceptible to ambiguity and misinterpretation
E. Least discriminating of all test item forms..
F. May penalize bright and insightful students
Guidelines for Alternative Response Items
a. Insure that items are unequivocally true or false
b. Avoid the use of specific determiners
c. Avoid ambiguous, and indefinite terms of degree
d. Avoid negatively worded items..
e. Avoid “double barreled items”
f. Keep T’s and F’s about the same length
g. Avoid detectable response patterns
h. Avoid trivial material
Consider this Question…
True of False…The sun rose today.
Most would agree that the answer is true….it got light this morning.....
However, the more thoughtful among us may conclude that it is False…the sun only appears to rise and set as a function of the earth’s rotation on its axis, for example…..
Alternative Response Items
Poor: T F Because of dramatically increased levels of CO2 emissions from industrialized countries there has been a reduction in the size of the polar ice caps.
This is a double barreled item!
Better: T F Recently there has been a reduction in the size of the polar ice caps.
Alternative Response Items
True or False
Acceleration of a body due to gravity is not 9,8 m/sec2
This item is worded in the negative...
Alternative Response Items
True or False
Weather and climate always have the same meaning….
This item contains a specific determiner, the word “always”
What to do about guessing
• Guessing is possible anytime the correct answer is available for selection
• A student gets the same point if he/she knows the answer or simply get lucky with a guess
• Some students guess far more often than others• Guessing closes the range and makes differences
between students harder to see.
Any attempt to deal with guessing should be directed toward equalizing the impact of
guessing on all students.
Two choices are available
• Try to discourage guessing by imposing a penalty through the use of a score correction formula
•Try to encourage guessing by telling students not to leave any question blank (Probably the best option)
The Score Correction Formula
Tommy takes a 50 item, Alternative Response test, getting 42 correct. His score, correct for guessing would then be…..42-(8/2-1)….42-(8/1)….42-8=34
• That’s not so difficult, is it?
12
842cgS 34842
1
842
cgS
Confidence WeightingTrue or False?
The Avco-Lycoming O-235-C1C produces 130 bhp at 2950 rpm.
Please write down your answer….true or false…
Now…tell me how confident you are of your answer….write down:
1 if your answer is just a guess
2 if you’re kind of certain and
3 if you’re absolutely sure…..
The Matching Exercise
Form: Two columns of information the first of which is referred to as the list of items…the second column is referred to as the list of options. The task is to match the items with the appropriate options….This is the best type of item to use when trying to assess mastery of a paired-association.
Advantages of the Matching Exercise
• Ease of construction
• Excellent item sampling
• Ease of scoring (if properly constructed)
• Best item for paired-associations tasks
Disadvantages of the Matching Exercise
• Encourages simplistic and rote learning…not always bad…
• Basic inability to assess complex learning outcomes
• Guessing is always possible but this can be dealt with through proper construction.
Writing a Matching Exercise
• Items should be in the left column and options in the right.
• Each exercise should consist of homogeneous material.• Keep each exercise relatively short (8-10 items).• Avoid having an equal number of items and options.• Arrange options in some systematic fashion is possible• Avoid specific determiners• Explain clearly in the directions the basis upon which the
match is to be made.• Maintain grammatical consistency.
Match the famous compositions in Column I with the guys who wrote that kind of stuff in Column II. Each choice in Column II may be used one, or not at all.
__ 1. Les Quaaludes A. Offenbach
__2. Pictures of an Exhibitionist B. Rimsky-Korsakov
__3. Marche Slob C. Chopin
__4. Orpheus in His Underwear D. Orff
__5. Le Coq Au Vin E. Liszt
__6. Carmina Piranha F. Stravinsky
__7. Peter and the Moose G. Debussy
__8. Capriccio Escargot H. Prokofiev
__9. A Midsummer Night on Elm Street I. Beethoven
J. Tchaikovsky
K. Rachmaninoff
Match the famous Operas in Column I with the guys who wrote that kind of stuff in Column II. Each choice in Column II may be used once, more than once, or not at all.
__ 1. Rigatoni A. Offenbach
__2. La Boingggg B. Rimsky-Korsakov
__3. Madam Mothballs C. Chopin
__4. Il Fettuccine D. Orff
__5. The Cadillac de Saville E. Liszt
__6. La Sagna
__7. The Marriage of Orthogro
__8. Das Fahrvergnugun
__9. The Magic Galoot
Directions: On the line to the left of each achievement in column A, write the letter from column B that identifies the name of the man noted for that achievement. Each name from column B may be used once, more than
once, or not at all. This is POOR PRACTICE
Column A Column B
__ 1. Invented the telephone A. Alexander Graham Bell
__2. Discovered America B. John Glenn
__3. First U.S. Astronaut to orbit the earth C. Abraham Lincoln
__4. First President of the U.S. D. Ferdinand Magellan
E. Christopher Columbus
F. George Washington
G. Eli Whitney
The Multiple Guess Item
Form: Typically takes the form of a stem with four or five options only one of which is considered to be correct or the best answer. There are two forms…..
Correct answer type: One correct among a number of incorrect ones…..
Best answer type: Several correct responses which differ in DEGREE of correctness or completeness.
Multiple Guess Items
Advantages:
a. Excellent item sampling
b. Ease of scoring
c. Versatile across a range of difficulty
d. Student writing is minimized
e. Permits writing tests which can re-used
Multiple Guess Items
Disadvantages:a. Time consuming to write….for each stem
that is written a correct response must be created as well as three or four incorrect responses which appear as though they could be correct.
b. Students complain that more than one correct response
Multiple Guess Items
Guidelines/Suggestionsa. Stem should be brief, clear, and conciseb. Stem should completely set the taskc. The options should be brief, clear, and concised. Distractors should be plausiblee. Avoid specific determinersf. Correct responses should be randomly placed.g. Use “all of the above” or “none of the above”
only when appropriate and sparingly.
The two types of M/C Items
The Correct Answer Type:
Which of the following continents lies totally above the equator?
a. Africa c. Asia
b. Austrailia d. South America
The two types of M/C Items
The Best Answer Type:
If you found yourself on a desert island, which of the following would you need first to find?
a. Food
b. Water
c. Shelter
d. A “friendly” companion
Forms of M/C Stems
• Which of the following is the best explanation for why most people wet the bed during earthquakes? Interrogatory
• The principle reason for nocturnal enuresis in geologically active regions is: Incomplete sentence
• Mark the answer which best describes the reason for bed wetting in volcanic areas. Imperative
Question Format Stem
10.Which one of the following expressions used by the airlines is an OXYMORON?
A. Complete stop D. Airline food
B. Direct flight E. Friendly Skies
C. Nonstop flight
A specific determiner…
10. A “BLEAM” is an:
A. lizard with 200 eyes, 100 for distance and 100 for reading.
B. animal with the body of a crab and the head of a certified public accountant.
C. a singles game played with dice and a large tube of ointment.
D. savage beast with the head of a lion and the body of a lion, though not the same lion.
Avoid specific determiners.
10. The largest African antelope is an:
A. Hippo
B. Dik-Dik
C. Eland
D. Warthog
Vary the position of the correct response
10. The largest African antelope is a/an:
A. Hippo A. Dik-Dik
B. Dik-Dik B. Eland
C. Eland C. Hippo
D. Warthog D. Warthog
By arranging the options in alphabetical order the correct response has in now in the “B” position.
Vary the position of the correct response
10. The Magna Carta was signed in what year?
A. 1965 A. 1215
B. 1812 B. 1776
C. 1215 C. 1812
D. 1776 D. 1965
By arranging the options in chronological order the correct response has in now in the “A” position.
Avoid Implausible Distrators
10. About how many calories are recommended daily for a girl, age 14, height 1.57m, weight 50kg, moderately active?
A. 0 A. 1,500
B. 2,000 B. 2,000
C. 2,500 C. 2,500
D. 30,000 D. 3,000
Strive for Parallelism
10. The cell islets of the pancreas:
A. Contain ducts
B. Produce insulin
C. Disappear as one grows older
D. Are located around the margins of the pancreas
This is poor form…..
Strive for Parallelism and let the stem set the task…..
10. The cell islets of the pancreas secrete the substance called:
A. adrenalin
B. insulin
C. secretin
D. trypsin
This is much better!
Strive for Parallelism and let the stem set the task…..
10. South America:
A. Is a flat, arid, country
B. Imports coffee from the U.S.
C. Has a larger population than the U.S.
D. Was settled mainly by colonists from Spain
Strive for Parallelism and let the stem set the task…..
10. Most of South America was settled by colonists from:
A. England
B. France
C. Holland
D. Spain
It’s not wise to “chain” questions
Poor:
1. Scurvy is caused by a lack of vitamin:
a. A b. B1 c. B12 d. C
2. A good source of this vitamin is:a. Orange juiceb. Cod-liver oilc. Whole riced. liver
Avoid “Trick” Questions
Poor:
1. Which of the following substances is most essential for human life?
a. Protein b. Water c. Vitamins d. Minerals
1. Better: A healthy person is marooned on a deserted island. To survive, the person needs almost immediately to find a source of:a. Protein b. Water c. Vitamins d. Minerals
Check the Reading Level!
Poor: The promiscuous use of sprays, oils, and antiseptics in the nose during acute colds is a pernicious practice because it may have a deleterious on:
A. the sinuses.
B. red blood cells.
C. white blood cells.
D. the olfactory nerve.
Check the Reading Level!
Better: Frequent use of sprays, oils, and antiseptics in the nose during bad colds may result in:
A. Sinuses infections.
B. mucous membrane congestion.
C. harm to white blood cells.
D. olfactory nerve damage.
Eschew Verbosity
Poor: Cells of one kind belong to a particular group performing a specialized duty. We call this group of cells a tissue. All of us have different kinds of tissues in our bodies. Which of the following would be classified as epithelial tissue?
A. Tendons
B. Adenoids and tonsils
C. Mucous membranes
D. Cartilage
Eschew Verbosity
Better: Which of the following would be classified as epithelial tissue?
A. Tendons
B. Adenoids and tonsils
C. Mucous membranes
D. Cartilage
Think Positively
Poor: Which of the following structures of the ear is NOT concerned with hearing?
A. Eardrum
B. Oval window
C. Semicircular canals
D. Cochlea
Think Positively
Better: Which of the following structures of the ear helps us to maintain balance?
A. Eardrum
B. Oval window
C. Semicircular canals
D. Cochlea
Test only for important ideas
Poor: In 1998 the death rate from accidents of all types per 100,000 population in the 15-24 age group was?
A. 59.1
B. 59.2
C. 59.3
D. 59.4
Test only for important ideas
Better: In 1998 the leading cause of death in the 15-24 age group was from:
A. respiratory diseases.
B. accidents.
C. cancer.
D. rheumatic heart disease.
Measuring Complex AchievementThe Interpretive Exercise
• Apply a principle• Interpret relationships• Recognize and state inferences• Recognize the relevance of information• Develop/recognize tenable hypotheses• Formulate/recognize valid conclusions• Recognize the limitations of data• Design experimental procedures
Measuring Complex AchievementThe Interpretive Exercise
The form: The IE consists of a series of objective items the answers to which may be conjured from a common set of data. These data may be written material, maps, charts, graphs, tables or pictures.
This pie chart depicts Bill’s weekly allowance distribution
1. What is the ratio of the amount Bill spends for school supplies to the amount he spends of movies.
a. 7:2 b. 1:3 c. 2:7 d. 3:1
2. What would be the best title for this pie chart?
a. Bill’s weekly allowance c. Bill’s weekly expenditures
b. Bill’s money Chart d. Bill’s money planning
Interpretive Exercise
Objective: to recognize the relevance of information
Bill lost his overshoe on the way to school. He wanted to put a notice on the bulletin board so other children could help him find it. Which of the following sentences tells something that would help children find the overshoe?
Directions: Circle “yes” if it would help. Circle “no” if it would not help.
Yes No 1. The overshoe was black
Yes No 2. It was very warm
Yes No 3. It was for his right foot
Yes No 4. It was a Christmas present
Yes No 5. It was nice looking
Yes No 6. It had a zipper
Yes No 7. It had a gray lining