Assessment of Information Skills. There is more teaching going on around here than learning and you...
-
Upload
gervais-reeves -
Category
Documents
-
view
217 -
download
0
Transcript of Assessment of Information Skills. There is more teaching going on around here than learning and you...
![Page 1: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/1.jpg)
Assessment of Assessment of Information SkillsInformation Skills
![Page 2: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/2.jpg)
There is more teaching going There is more teaching going on around here than learning on around here than learning and you ought to do something and you ought to do something about that.about that.
Graduating SeniorGraduating SeniorKing’s College, 1968King’s College, 1968
![Page 3: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/3.jpg)
AssessmentAssessment
A systematic ongoing process ofA systematic ongoing process of setting goals or asking questions,setting goals or asking questions, gathering information,gathering information, interpreting it, andinterpreting it, and usingusing it it
to improve student learning.to improve student learning.Barbara D. WrightBarbara D. Wright
![Page 4: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/4.jpg)
Why not use grades?Why not use grades?
““A grade is an inadequate report of an A grade is an inadequate report of an inaccurate judgment by a biased and inaccurate judgment by a biased and variable judge of the extent to which a variable judge of the extent to which a student has attained an undefined level student has attained an undefined level of mastery of an unknownof mastery of an unknown proportion of proportion of an indefinite material.”an indefinite material.”
Bill Boyle & Tom ChristieBill Boyle & Tom Christie
Issues in Setting StandardsIssues in Setting Standards
Falmer Press, 1996Falmer Press, 1996
![Page 5: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/5.jpg)
Grades may reflect many things besides Grades may reflect many things besides mastery of course objectives:mastery of course objectives:
Verbal abilityVerbal ability
ParticipationParticipation
CooperationCooperation
Extra creditExtra credit
AttendanceAttendance
EffortEffort
Criterion Performance Criterion Performance vs. vs. Value AddedValue Added
Myths regarding student evaluationsMyths regarding student evaluations
![Page 6: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/6.jpg)
Pay me nowPay me now
oror
Pay me laterPay me later
![Page 7: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/7.jpg)
Align your OutcomesAlign your Outcomes
Institutional outcomesInstitutional outcomes
Program outcomesProgram outcomes
Course outcomesCourse outcomes
Class outcomesClass outcomes
![Page 8: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/8.jpg)
Dimensions of AssessmentDimensions of Assessment
Formative - ProcessFormative - Process Feedback for improvementFeedback for improvement Corrective & diagnosticCorrective & diagnostic
Summative – Evaluative Summative – Evaluative JudgmentalJudgmental DecisionalDecisional
![Page 9: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/9.jpg)
Dimensions of AssessmentDimensions of Assessment
An individual An individual A cohortA cohort A classA class A courseA course A programA program An institutionAn institution
![Page 10: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/10.jpg)
Quantitative DataQuantitative Data
NumericalNumerical Statistically Statistically
reliablereliable Structured Structured
techniquestechniques ObjectiveObjective Assumes static Assumes static
realityreality
Usually can be Usually can be generalizedgeneralized
Allows for Allows for comparison and comparison and correlationcorrelation
Looks for facts Looks for facts and causesand causes
![Page 11: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/11.jpg)
Qualitative DataQualitative Data
Not numericalNot numerical Provides Provides
“richness”“richness” Allows for Allows for
ambiguitiesambiguities Structured or Structured or
semi-structuredsemi-structured Written, verbal, Written, verbal,
visual, etcvisual, etc
Often subjectiveOften subjective Can be Can be
exploratoryexploratory Assumes Assumes
dynamic realitydynamic reality Looks for Looks for
motivations and motivations and points of viewpoints of view
![Page 12: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/12.jpg)
AssumptionsAssumptions
Vs.Vs.
AssessmentAssessment
![Page 13: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/13.jpg)
AssessmentAssessment
Enhance student learning and Enhance student learning and institutional effectivenessinstitutional effectiveness
Student learning rather than Student learning rather than faculty evaluationfaculty evaluation
Assess the process, rather than Assess the process, rather than the outcome, for the purpose of the outcome, for the purpose of improving the outcome improving the outcome
![Page 14: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/14.jpg)
Measure What MattersMeasure What Matters
Measure what matters, Measure what matters, NOTNOT what is what is measurablemeasurable
Because what you measure Because what you measure becomes what you focus onbecomes what you focus on
![Page 15: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/15.jpg)
AssessmentAssessment
Knowing Knowing what what you are doingyou are doing Knowing Knowing whywhy you are doing it you are doing it Knowing what students are Knowing what students are
learninglearning as a result as a result ChangingChanging because of the because of the
informationinformation
Debra GilchristDebra Gilchrist
![Page 16: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/16.jpg)
Indirect Evidence of Student Indirect Evidence of Student LearningLearning
Surveys, self-reports & journalsSurveys, self-reports & journals Focus groups & interviewsFocus groups & interviews Alumni & employer surveysAlumni & employer surveys Percent of students entering Percent of students entering
graduate or professional schoolsgraduate or professional schools
![Page 17: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/17.jpg)
Limitations of Indirect MeasuresLimitations of Indirect Measures
Provide data on factors that predict or Provide data on factors that predict or mediate learningmediate learning
DO NOT evaluate learning DO NOT evaluate learning per seper se DO NOT NECESSARILY imply that value-DO NOT NECESSARILY imply that value-
added learning has or has not occurredadded learning has or has not occurred(e.g., enthusiasm or lack of interest may be caused (e.g., enthusiasm or lack of interest may be caused
by factors not related to the course)by factors not related to the course)
Oswald RatterayOswald Ratteray
![Page 18: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/18.jpg)
Students’ Reported Information Skills
Ability to Locate, Evaluate, and Select Information 1998 1999 2000 2001 2002 Average College Average 4.30 4.23 4.10 4.15 4.24 4.20 Accounting 4.26 4.33 4.07 4.21 4.32 4.24 Biology 4.13 4.48 4.23 4.35 4.14 4.27 Business 3.88 4.17 3.97 4.24 4.33 4.12 Chemistry 4.67 4.00 4.20 4.83 4.43 Communications 4.26 4.38 4.31 4.26 4.45 4.33 C. I. S. 4.33 4.17 4.21 4.16 4.23 4.22 Criminal Justice 4.50 4.13 3.96 4.00 4.24 4.17 Education 4.31 4.18 4.20 4.22 4.35 4.25 English 4.47 4.27 4.11 4.31 4.45 4.32
Scale: 5 (excellent) 4 (good) 3 (average) 2 (below average) 1 (poor)
![Page 19: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/19.jpg)
Direct Evidence of Student Direct Evidence of Student LearningLearning
Student assignmentsStudent assignments Standardized testsStandardized tests Course embedded assessmentsCourse embedded assessments Portfolios of students’ workPortfolios of students’ work Capstone experiencesCapstone experiences Student performances & exhibitsStudent performances & exhibits Other observations of student behaviorOther observations of student behavior
![Page 20: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/20.jpg)
Direct Evidence of Student Direct Evidence of Student LearningLearning
Rubrics & exemplarsRubrics & exemplars Concept mapsConcept maps Juried/peer review of student projectsJuried/peer review of student projects Performance on a case study or Performance on a case study or
problemproblem Locally devised testsLocally devised tests Commercially produced tests & examsCommercially produced tests & exams
![Page 21: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/21.jpg)
Limitations of Direct Limitations of Direct MeasuresMeasures
Indicate:Indicate: WHAT students learnedWHAT students learned HOW MUCH they learnedHOW MUCH they learned What they DID NOT learnWhat they DID NOT learn
DO NOT indicate:DO NOT indicate: WHY students learn or did not learnWHY students learn or did not learn
DO NOT necessarily indicate:DO NOT necessarily indicate: Whether VALUE-ADDED learning occurredWhether VALUE-ADDED learning occurred
Oswald Oswald RatterayRatteray
![Page 22: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/22.jpg)
Develop Rubrics to Assess Develop Rubrics to Assess Work:Work:
Levels of achievementLevels of achievement
Criteria that distinguish good work from poor Criteria that distinguish good work from poor workwork
Descriptions of criteria at each level of Descriptions of criteria at each level of achievementachievement
Peggy MakiPeggy Maki
![Page 23: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/23.jpg)
Basic Rubric Grid FormatBasic Rubric Grid Format
TitleTitle
Scale Level 1Scale Level 1 Scale Level 2Scale Level 2 Scale Level 3Scale Level 3
Dimension 1Dimension 1
Dimension 2Dimension 2
Dimension 3Dimension 3
Dimension 4Dimension 4
Task Description
![Page 24: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/24.jpg)
Why use a Rubric?Why use a Rubric?
Examine complex work efficientlyExamine complex work efficiently Clarifies faculty expectationsClarifies faculty expectations Communicates expectations Communicates expectations Improves students’ work Improves students’ work Criterion-referenced grades rather than Criterion-referenced grades rather than
normative-referencednormative-referenced Facilitates course or program Facilitates course or program
assessmentassessment
![Page 25: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/25.jpg)
ExamplesExamples
![Page 26: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/26.jpg)
PortfoliosPortfolios
Teaching portfolioTeaching portfolio Documents teaching over time Documents teaching over time
Course portfolioCourse portfolio Reflection & review of a single Reflection & review of a single
coursecourse Student portfolioStudent portfolio
Highlights student learningHighlights student learning
![Page 27: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/27.jpg)
Portfolio ElementsPortfolio Elements
Course design, philosophy, Course design, philosophy, rationalerationale
ImplementationImplementation ResultsResults ReflectionReflection
![Page 28: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/28.jpg)
Portfolio ElementsPortfolio Elements
Focus on the match between Focus on the match between assessment & course goalsassessment & course goals
Use existing assessments & data, when Use existing assessments & data, when possiblepossible
Include a variety of evidence typesInclude a variety of evidence types Be purposeful when selecting evidence Be purposeful when selecting evidence
about learningabout learning
![Page 29: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/29.jpg)
Assessment is not about us, Assessment is not about us,
it is about student learning.it is about student learning.
![Page 30: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/30.jpg)
Assessment is like Assessment is like scotch,scotch,
it is an acquired taste.it is an acquired taste.
![Page 31: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/31.jpg)
Questions?Questions?
![Page 32: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/32.jpg)
Seniors’ Information Skills by Standard : Mean Seniors’ Information Skills by Standard : Mean Scores/Percentage Correct - Fall 2004*Scores/Percentage Correct - Fall 2004*
GroupGroup NN Composite Composite ScoresScores
Std 1Std 1 Std 2Std 2 Std 3Std 3 Std 4Std 4 Std 5Std 5
142142 57.3257.32 60.5660.56 56.6256.62 60.0060.00 58.4558.45 50.9950.99
AA 2222 50.3650.36 52.7352.73 57.2757.27 52.7352.73 43.6443.64bb 45.4545.45
BB 3838 56.0056.00 58.9558.95 49.4749.47 57.3757.37 61.5861.58 52.6352.63
CC 2121 57.5257.52 55.2455.24 54.2954.29 61.9061.90 65.7165.71aa 50.4850.48
DD 1515 58.6758.67 65.3365.33 57.3357.33 66.6766.67 53.3353.33 50.6750.67
EE 2626 60.3160.31 63.0863.08 65.3865.38 60.0060.00 62.3162.31 50.7750.77
FF 1111 61.0961.09 69.0969.09 56.3656.36 63.6463.64 60.0060.00 56.3656.36
GG 99 64.0064.00 73.3373.33 64.4464.44 68.8968.89 60.0060.00 53.3353.33
ababRefers to comparisons within column where the MEAN scores of group Refers to comparisons within column where the MEAN scores of group a a are significantly (Pare significantly (P< < .05) higher than .05) higher than the MEAN scores of group the MEAN scores of group bb..*Please note that in some cases the small group size and number of questions per standard may preclude more *Please note that in some cases the small group size and number of questions per standard may preclude more meaningful statistical comparisons.meaningful statistical comparisons.
![Page 33: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/33.jpg)
Seniors’ Information Skills by Standard : Mean Seniors’ Information Skills by Standard : Mean Scores/Percentage Correct -Fall 2004*Scores/Percentage Correct -Fall 2004*
GroupGroup NN Composite Composite ScoresScores
KnowledgeKnowledge ApplicationApplication
SeniorsSeniors 142142 57.3257.32 61.8061.80 53.2053.20
AA 2222 50.3650.36 50.7650.76bb 50.0050.00
BB 3838 56.0056.00 58.5558.55 53.6453.64
CC 2121 57.5257.52 61.1161.11 54.2154.21
DD 1515 58.6758.67 67.7867.78 50.2650.26
EE 2626 60.3160.31 66.6766.67aa 54.4454.44
FF 1111 61.0961.09 68.1868.18 54.5554.55
GG 99 64.0064.00 72.2272.22aa 56.4156.41ababRefers to comparisons within column where the MEAN scores of group Refers to comparisons within column where the MEAN scores of group a a are significantly (Pare significantly (P< < .05) higher than .05) higher than the MEAN scores of group the MEAN scores of group bb..*Please note that in some cases the small group size and number of questions per standard may preclude more *Please note that in some cases the small group size and number of questions per standard may preclude more meaningful statistical comparisons.meaningful statistical comparisons.
![Page 34: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/34.jpg)
Information Skill of Seniors & Graduate Students from Three Northeast, Pennsylvania
Colleges by Standard: Mean Scores/Percentage - Fall 2004*
CollegeCollege NN Composite Composite ScoresScores
StdStd11
StdStd22
StdStd33
StdStd44
StdStd55
388388 57.2257.22 61.2461.24 54.5454.54 55.3155.31 58.0458.04 56.9656.96
1 (Srs)1 (Srs) 142142 57.3257.32 60.5660.56 56.6256.62 60.0060.00 58.4558.45 50.9950.99bb
2 (Grads)2 (Grads) 2929 56.2856.28 63.4563.45 48.2848.28 47.5947.59 57.2457.24 64.8364.83aa
3 (Srs)3 (Srs) 217217 57.2757.27 61.3861.38 54.0154.01 53.2753.27 57.8857.88 59.8259.82aa
ababRefers to comparisons within column where the MEAN scores of group Refers to comparisons within column where the MEAN scores of group a a are significantly (Pare significantly (P< < .05) higher than .05) higher than the MEAN scores of group the MEAN scores of group bb..*Please note that in some cases the small group size and number of questions per standard may preclude more *Please note that in some cases the small group size and number of questions per standard may preclude more meaningful statistical comparisons.meaningful statistical comparisons.
![Page 35: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/35.jpg)
Information Skill of Seniors & Graduate Students from Three Information Skill of Seniors & Graduate Students from Three Northeast, Pennsylvania Colleges by Standard : Mean Northeast, Pennsylvania Colleges by Standard : Mean
Scores/Percentage - Fall 2004*Scores/Percentage - Fall 2004*
CollegeCollege NN Composite Composite ScoresScores
KnowledgeKnowledge ApplicationApplication
388388 57.2257.22 58.9358.93 55.6355.63
1 (Srs)1 (Srs) 142142 57.3257.32 61.8061.80 53.2053.20
2 (Grads)2 (Grads) 2929 56.2856.28 57.1857.18 55.4455.44
3 (Srs)3 (Srs) 217217 57.2757.27 57.3057.30 57.2557.25
ababRefers to comparisons within column where the MEAN scores of group Refers to comparisons within column where the MEAN scores of group a a are significantly (Pare significantly (P< < .05) higher than .05) higher than the MEAN scores of group the MEAN scores of group bb..*Please note that in some cases the small group size and number of questions per standard may preclude *Please note that in some cases the small group size and number of questions per standard may preclude more meaningful statistical comparisons.more meaningful statistical comparisons.
![Page 36: Assessment of Information Skills. There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior.](https://reader030.fdocuments.net/reader030/viewer/2022012922/56649e405503460f94b30dc7/html5/thumbnails/36.jpg)
Five Stages of AssessmentFive Stages of Assessment(from Elisabeth Kubler-Ross)(from Elisabeth Kubler-Ross)
1.1. Denial Denial ““No, not me”No, not me”
2.2. Anger or resentment Anger or resentment ““Why me?”Why me?”
3.3. Bargaining Bargaining ““Yes, me, but…”Yes, me, but…”
4.4. Depression Depression ““Yes, me”Yes, me”
5.5. AcceptanceAcceptance