TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.
-
Upload
andrew-logan -
Category
Documents
-
view
215 -
download
1
Transcript of TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.
![Page 1: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/1.jpg)
![Page 2: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/2.jpg)
![Page 3: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/3.jpg)
![Page 4: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/4.jpg)
TOP 10 FLASHPOINTS IN THE
EVALUATION OF TEACHING
![Page 5: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/5.jpg)
![Page 6: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/6.jpg)
![Page 7: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/7.jpg)
KEY REFERENCES:
1. Arreola, R. A. (2007). Developing a
comprehensive faculty evaluation
system (3rd ed.). San Francisco:
Jossey-Bass (Anker).
2. Seldin, P. (Ed.). (2006). Evaluating
faculty performance. San Francisco:
Jossey-Bass (Anker).
![Page 8: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/8.jpg)
3. Hativa, N. (2013). Student ratings of
instruction: A practical approach to
designing, operating, and reporting.
Seattle, WA: CreateSpace.
4. Hativa, N. (2013). Student ratings of
instruction: Recognizing effective
teaching. Seattle, WA: CreateSpace.
![Page 9: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/9.jpg)
5. Spooren, P., Brockx, B., & Mortelmans,
D. (2013). On the validity of student
evaluation of teaching: The state of the
art. Review of Educational Research,
83(4), 598–642.
![Page 10: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/10.jpg)
TEACHING EVALUATION POLL
111 1. How many evaluate teaching in your university or college? Yes No 2. How many use a student rating scale? Yes No 3. How many use peer observation? Yes No 4. How many use other evidence? Yes No 5. How many would prefer to go eat again instead of listening to me? Yes No
TOTAL SCORE ____
![Page 11: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/11.jpg)
A FEW FACTOIDS: 1. 15,000+ studies on teaching
effectiveness (60+ yrs.)
2. 2000 refs on student ratings (90 yrs.)
3. 80+ year history on scaling
4. Student ratings have dominated
as frequently the only measure
![Page 12: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/12.jpg)
MAJOR OUTCOMES: YOU WILL BE
1. armed with the major problems &
issues in the evaluation of teaching;
2. able to decide on the most appropriate
course of action for the decisions you
make;
3. given answers to your burning questions.
![Page 13: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/13.jpg)
TYPES OF DECISIONS:
1. Formative: teaching improvement
2. Summative: contract renewal,
promotion/tenure, merit pay/increase
3. Program: curriculum revision
![Page 14: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/14.jpg)
WHY BOTHER? How important is teaching?
1. Faculty: to improve their courses
& teaching
2. Administrators: to make fair
& equitable decisions about
hiring, firing, promoting, and
demoting part- & full-time faculty
![Page 15: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/15.jpg)
PROFESSIONAL STANDARDS:
Personnel Evaluation Standards Program Evaluation Standards Standards for Educational &
Psychological Testing
![Page 16: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/16.jpg)
LEGAL STANDARDS:
U.S. Equal Employment
Opportunity Commission’s (EEOC)
Uniform Guidelines on Employment
Selection Procedures
![Page 17: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/17.jpg)
COURT DECISIONS: (faculty appointments, pay, contract
renewal, teaching awards, and
promotion and tenure)
1. Substantial or Academic
Deference
2. Limited Deference
3. 80/20 Rule for Adverse Impact
![Page 18: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/18.jpg)
![Page 19: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/19.jpg)
A FEW GOOD MEN (1992):
© Columbia Pictures Industries
and Castle Rock Entertainment
![Page 20: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/20.jpg)
TOP 10
FLASHPOINTS
![Page 21: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/21.jpg)
FLASHPOINT
a critical stage in a process, trouble spot, contentious issue, volatile hot button, or lowest temperature at which flammable liquid will give off enough vapor to ignite
Derived from 2 Latin words: flashus, meaning “your shorts,” and pointum, meaning “are on fire”
![Page 22: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/22.jpg)
A. Student ratings are limited &
incomplete
B. Student ratings are fallible
![Page 23: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/23.jpg)
Triangulate “complementary” multiple sources of evidence for each decision
![Page 24: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/24.jpg)
15 Sources of Evidence:
1. Student Ratings2. Peer Observations3. Peer Review of Course Materials4. External Expert Ratings5. Self-Ratings6. Videos7. Student Interviews8. Exit and Alumni Ratings
9. Employer Ratings10. Mentor’s Advice11. Administrator Ratings12. Teaching Scholarship13. Teaching Awards14. Learning Outcome Measures15. Teaching Portfolio
![Page 25: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/25.jpg)
A. Start with “formative decisions”
B. Then consider each type of
“summative decision”
![Page 26: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/26.jpg)
Tailor the combo of sources of evidence you pick to the decision being made
![Page 27: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/27.jpg)
360° ASSESSMENT OF A PROFESSOR
(Formative Decisions)
PROFESSOR
![Page 28: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/28.jpg)
360° ASSESSMENT OF A PROFESSOR
(Formative Decisions)
SELF-RATINGS
VIDEOSelf/Peer
MENTOR
PEERRATINGS
STUDENTINTERVIEWS
STUDENT RATINGS
PROFESSOR
![Page 29: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/29.jpg)
360° ASSESSMENT OF A PROFESSOR
(Summative Decisions)
PROFESSOR
![Page 30: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/30.jpg)
360° ASSESSMENT OF A PROFESSOR
(Summative Decisions)
SELF-RATINGS
VIDEO(optional)
MENTOR(optional)
PEERRATINGS(optional)
DEPT.CHAIR
STUDENTRATINGS
PROFESSOR
![Page 31: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/31.jpg)
A. Most “home-grown” scales are PUTRID
B. Commercially-developed scales
typically meet professional standards
![Page 32: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/32.jpg)
Whatever scale you
develop, do it right or don’t
do it at all.
![Page 33: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/33.jpg)
HOW MANY TYPES OF
ITEMS ARE THERE?
2
![Page 34: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/34.jpg)
ITEM WORLD
1. Test Items (correct and incorrect answers)
![Page 35: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/35.jpg)
If your urologist says, “You have a
kidney stone the size of Ohio,”
that is an example of a(an)
A. analogy.
B. hyperbole.
C. metaphor.
D. simile.
E. rather disturbing thought.
![Page 36: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/36.jpg)
2. Scale/Questionnaire Items: (response = answer)
a. Opinion/Attitude: no incorrect response b. Factual: response = fact
(1) Sociodemographic Characteristics (2) Actual Practices
![Page 37: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/37.jpg)
RATING ITEM AUTOPSY
STIMULUS RESPONSE
![Page 38: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/38.jpg)
![Page 39: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/39.jpg)
STIMULUS RESPONSE Sentence Anchors (My instructor is a knucklehead.) (Strongly Agree – Strongly Disagree)
Phrase Anchors (Enunciated clearly) (Excellent – Poor)
Word Anchors (Volume) (Effective – Ineffective)
![Page 40: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/40.jpg)
ANCHOR WORLD
1. Intensity
2. Evaluation
3. Frequency
4. Quantity
5. Comparison
![Page 41: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/41.jpg)
SAMPLE FLAWED STATEMENTS
• My instructor is a bade speler and a morron. (double-barreled content)
• My instructor treated all students with respect. (student not in a position to rate)
• My instructor entered class on a cable from the ceiling like Ethan Hunt in Mission: Impossible. (fact; not a behavior)
![Page 42: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/42.jpg)
A. Online advantages in admin., cost,
turnaround, & responses outweigh
p & p
B. Most online problems can be overcome
![Page 43: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/43.jpg)
3 Options:
1.in-house admin.
2.vendor admin. with “home-grown” scales
3.vendor admin. & rating scale
![Page 44: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/44.jpg)
![Page 45: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/45.jpg)
A. Scale must be admin. under identical
conditions to all students
B. Control time, place, conditions, &
situational factors
![Page 46: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/46.jpg)
Admin. within a narrow
window of 1–2 days before
or after final evaluation
![Page 47: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/47.jpg)
A. 30–50% rates provide an inadequate
data base for decisions
B. 20 strategies to boost response
rates
![Page 48: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/48.jpg)
Use a combination of admin., organizational, & incentive procedures (early posting of grades has the highest increase in rates)
![Page 49: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/49.jpg)
RESPONSE RATE Effective strategies:
a. Faculty and administrators communicate
importance of students’ input and how
results will be used
b. Assurance of anonymity
c. User-friendly system
d. Faculty “assign” students to complete evals.
e. Completion is part of course requirements
f. Provide extra credit, points, or other incentives
g. Draw raffle or lottery prizes
*h. Withhold students’ early access to final grades
![Page 50: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/50.jpg)
A. Global items provide the illusion of
simplicity, accurate & reliable info,
& pinpoint precision
B. They can be unreliable, unrepres. of
teaching behaviors, & illegal for
personnel decisions
![Page 51: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/51.jpg)
Cease & desist use of global items for summative decisions
Use total scale rating (mean/median) and subscale composite ratings
![Page 52: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/52.jpg)
COMMERCIAL BREAK
![Page 53: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/53.jpg)
![Page 54: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/54.jpg)
ANGER MANAGEMENT (2003):
© Revolution Studios Distribution Co.
and Columbia TriStar Entertainment
![Page 55: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/55.jpg)
We Now Rejoin Ron’s Presentation
Already in Progress.
![Page 56: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/56.jpg)
A. Eliminate “neutral” anchor
B. Use “NA” during field test only
C. Use “NO” in observation scales
D. Assign a midpoint score to blank
responses
![Page 57: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/57.jpg)
Clearly specify scoring procedure for each of 4 response options
![Page 58: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/58.jpg)
A. Link anchor, item, subscale, & total
scale ratings to specific decisions
B. Define purpose of C-R & N-R
interpretations, especially norms
![Page 59: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/59.jpg)
C-R ratings are collected WITHIN each course; N-R ratings are based on OUTSIDE course norms
DO NOT rank faculty under any conditions
![Page 60: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/60.jpg)
![Page 61: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/61.jpg)
A. Identify F2F & online common &
unique characteristics for teaching
effectiveness
B. Consider various options to create
scales to measure those characteristics
![Page 62: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/62.jpg)
Design or select scales that tap both what is in common & what is unique to F2F & online courses
![Page 63: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/63.jpg)
FINAL ADVICE
![Page 64: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/64.jpg)
1. Use multiple sources of evidence
for all faculty “job” decisions
2. Develop or select the best quality
measures to provide that evidence
3. Match the evidence to your decisions
4. Provide a narrow window to standardize
administrations
5. Use a combination of methods to maximize
response rate
BOTTOM LINE
![Page 65: TOP 10 FLASHPOINTS IN THE EVALUATION OF TEACHING.](https://reader036.fdocuments.net/reader036/viewer/2022081603/56649ec75503460f94bd2e06/html5/thumbnails/65.jpg)