The Required Library Component: Assessing First-Year Teaching in the Small Academic Library
description
Transcript of The Required Library Component: Assessing First-Year Teaching in the Small Academic Library
![Page 1: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/1.jpg)
The Required Library Component: Assessing First-Year Teaching in the
Small Academic Library
Susan von Daum Tholl, PhD, DirectorDiane Zydlewski, Head of Reference
Anne Hancock, Collection Development Librarian
![Page 2: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/2.jpg)
Scope of our discussion
• Setting the stage—gaining administration approval for a required library component
• Development of the library curriculum within the FYS; one-on-one faculty collaboration and attendance at FYS faculty meetings
• Hybrid approach and revision of our online tutorial
• Continual modifications of assignments and library workshop
• Data gathering; faculty collaboration
• Statistical analysis, which is ongoing
• Use of our results to mold the future. . . .
![Page 3: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/3.jpg)
• Pre-2005 — Basic info lit session offered in non-credited venues
• Fall 2005 — First-Year Seminars required Library Component offered in both fall and spring semester
• 2005-Spring 2007 — Worked 1 on 1 with faculty on assignments
• 2007-2008 — Piloted customized assignments and pre- and post- testing by individual seminar
• Fall 2008 — Applied to IRB; approval received January 2009
• Fall 2009 — 3-Year Study began; all seminars offered in Fall semester
• Fall 2011 — Study completed
Timeline of Development
![Page 4: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/4.jpg)
Stated Goal of FYS
“engage first-year students in research
and learning the basic elements of
conducting research specifically using
the resources of the library”
![Page 5: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/5.jpg)
• Pre-2005 — Basic info lit session offered in non-credited venues
• Fall 2005 — First-Year Seminars required Library Component offered in both fall and spring semester
• 2005-Spring 2007 — Worked 1 on 1 with faculty on assignments
• 2007-2008 — Piloted customized assignments and pre- and post- testing by individual seminar
• Fall 2008 — Applied to IRB; approval received January 2009
• Fall 2009 — 3-Year Study began; all seminars offered in Fall semester
• Fall 2011 — Study completed
Timeline of Development
![Page 6: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/6.jpg)
Pre-TestOnline Tutorial and Welcome Assignment
Workshop & Workshop
Assignment – Clickers (Year 3)
Evaluation Assignment (Years 1 & 2)
Post-Test
Substance of the
Library Component
from fall 2009 to fall 2011
![Page 7: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/7.jpg)
Criteria for eligibility of participants:
• First-Year entering students
• Age (over 18 years of age at the pre-test)
• Completion of both pre- and post-test
• Consent letter explaining the process for opting out
Data collected for individual students:
• Pre- and Post-test scores
• Assignment grades
• Number of assignments completed
• Gender
![Page 8: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/8.jpg)
YearEligible
Participants/Enrollments
% students showing positive
change pre-to post-test for all participants
% increase pre- to pos-test for all
participants
Average pre-test grade
Average post-test
grade
Fall 2009 324/366(90.00%)
88.89% 45.01%MIN=-50.00%MAX=300.00%
10.00 MIN=2.00 MAX=18.00
13.65 MIN=1.00 MAX=18.00
Fall 2010 411/542(75.83%)
88.10% 48.65%MIN=-30.00%MAX=550.00%
10.11 MIN=2.00MAX=17.00
13.82 MIN=6.00 MAX=18.00
Fall 2011 443/551(80.40%)
97.97% 54.38%MIN= -28.75%MAX=500.00%
10.18MIN=2.00MAX=17.00
14.81MIN=5.00MAX=18.00
Rates of Improvement during the Study
![Page 9: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/9.jpg)
A Hybrid Approach:
Combining a Hands-On Workshop and an Online Tutorial
Diane Zydlewski, Head of Reference
![Page 10: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/10.jpg)
The Evolving Role of the Tutorial
Fall 2003Online PowerPoint Presentation accompanying a live presentation.
Fall 2005Web-based Tutorial completed independently or in a workshop.
Spring 2008Web-based Tutorial completed during a required workshop.
Fall 2008Combination of the tutorial and hands-on activities in the workshop.
![Page 11: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/11.jpg)
Pre-Test
Online Tutorial and
Welcome Assignment
Workshop with clickers
and Workshop /Evaluation
Assignment
Post-Test
Current Role of the Tutorial within the Library Component of FYS Program
![Page 12: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/12.jpg)
Scope of the Tutorial
![Page 13: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/13.jpg)
![Page 14: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/14.jpg)
Pop-up alert appears if the
wrong answer is selected.
![Page 15: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/15.jpg)
![Page 16: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/16.jpg)
![Page 17: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/17.jpg)
Technical and Design Aspectsof the tutorial
(See also the best practices handout.)
![Page 18: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/18.jpg)
Fall 2005
Simulation of Library Web Site.
![Page 19: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/19.jpg)
Fall 2011
![Page 20: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/20.jpg)
Spring 2007
![Page 21: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/21.jpg)
Fall 2011
![Page 22: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/22.jpg)
Assignments: A Dynamic Information Literacy Tool
Anne Hancock, Collection Development Librarian
![Page 23: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/23.jpg)
WelcomeAssignment
• Complete one week before workshop
• Register for Library Barcode
• Questions covered:• basic library info• creating search
strings• online catalog• online databases
WorkshopAssignment
• Hand out during Library Workshop
• Identify research topic
• Create search string
• Find book, article, and website
• Complete outside of class, due 1 week later
• Evaluate resources found in Workshop for content quality
• Create citation
Do
ne
Concept & Design : A “Stepped” Approach
Evaluation & Citation
Assignment
![Page 24: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/24.jpg)
PROBLEMS
• Too much material
• Assignment questions unclear
• Faculty confusion about assignment
• Students not reading instructions
The Welcome Assignment
SOLUTIONS
• Pared down scope
• Conformed assignment questions to tutorial language and SIMPLIFIED.
• Reinforced assignment process with faculty
• Made instructions stand out more
(increased spacing and font)
![Page 25: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/25.jpg)
PROBLEMS
• Students did not have enough time!!!!
• Workshop sessions of different lengths (50 or 75 minutes)
• Number and types of resources a challenge
• Search String CONFUSION!!!
The Workshop Assignment
SOLUTIONS
• Pared down number of resources we asked students to find.
• Simplified search string portion of assignment.
![Page 26: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/26.jpg)
PROBLEMS
• Questions open-ended
• Student answers incomplete
• Difficult to grade subjective questions in a standardized way
• Citations
• Faculty and Student perceptions
The Evaluation & Citation Assignment
SOLUTIONS
• Rewrote assignment
• Used guided evaluation questions
• Removed open-ended questions
• Removed formal citation and asked bibliographic questions instead.
• Eliminated assignment in 2011
![Page 27: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/27.jpg)
Prior to IRB studyFaculty
Last year of IRB studyOne Librarian
Grading Methods
![Page 28: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/28.jpg)
Data Gathering & Analysis
Data Collected:
• Pre- and Post-test scores and question by question answers
• Assignment grades
• Number of assignments completed
• Gender
Data Analysis:
• Created IBM SPSS database for each year
• Data entered into IBM SPSS
• Using IBM SPSS software to run statistical tests
![Page 29: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/29.jpg)
All Welcome only Workshop only Evaluation only None0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Assignment Completions by Year
2009
2010
2011
Assignments Completed
![Page 30: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/30.jpg)
All Welcome only Workshop only Evaluation only None0%
20%
40%
60%
80%
100%
120%
140%
Average % Change Pre to Post-Test by As-signment Completed
2009
2010
2011
Assignment Completed
Pe
rce
nta
ge
Ch
an
ge
![Page 31: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/31.jpg)
YearEligible
Participants/Enrollments
% students showing positive
change pre-to post-test for all participants
% increase pre- to pos-test for all
participants
Average pre-test grade
Average post-test
grade
Fall 2009 324/366(90.00%)
88.89% 45.01%MIN=-50.00%MAX=300.00%
10.00 MIN=2.00 MAX=18.00
13.65 MIN=1.00 MAX=18.00
Fall 2010 411/542(75.83%)
88.10% 48.65%MIN=-30.00%MAX=550.00%
10.11 MIN=2.00MAX=17.00
13.82 MIN=6.00 MAX=18.00
Fall 2011 443/551(80.40%)
97.97% 54.38%MIN= -28.75%MAX=500.00%
10.18MIN=2.00MAX=17.00
14.81MIN=5.00MAX=18.00
Rates of Improvement during the Study
![Page 32: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/32.jpg)
Use of Results to Mold the Future Library Component. . .
• Statistical analysis is ongoing.
• Analyze results of the pre- and post-test question-by- question to identify patterns of item difficulty to indicate what our teaching must address.
• We are looking at completion rates for the Assignments as they relate to individual student learning outcomes on the post-test.
![Page 33: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/33.jpg)
The Future . . .
• Rework the pre- and post-tests, clarifying some language and creating distractors to develop a stronger test.
• Aside from the IRB Study, anonymous “clicker” data on self-efficacy and student satisfaction from the workshops needs to be compiled and analyzed.
![Page 34: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/34.jpg)
The Study as a springboard for new initiatives . . . .
• Use statistical analysis to help to inform the various levels of our Information Literacy Program
• Formalize our required Library Workshop in the First- Year Writing Program, whose topic is a more advanced look at evaluation, especially online resources
• Apply to IRB for a follow-up study of participants in Years 2 and 3 of the FYS Study when they are Seniors
• Develop focus groups with the Instructional Assistants
• Fill a new FT position – Instructional Librarian
• Create a faculty development program
![Page 35: The Required Library Component: Assessing First-Year Teaching in the Small Academic Library](https://reader035.fdocuments.net/reader035/viewer/2022062809/56815752550346895dc4f647/html5/thumbnails/35.jpg)
Thank you!
Our Contact information:
Susan von Daum Tholl, [email protected]
Diane [email protected]
Anne [email protected]
http://www1.emmanuel.edu/library