Howard R. Mzumara Ranjita D. Shinde Denise L. Czachura IUPUI Testing Center 1 Presentation given at...

Howard R. Mzumara Ranjita D. Shinde Denise L. CzachuraIUPUI Testing Center

1

Presentation given at the IMLS/IUPUI Steering Committee Meeting at IUPUI, Indianapolis, IN (November 13, 2007)

I. Brief Review of Evaluation Objective and FocusII. Class Enrollment and Completion RatesIII. Survey Participant Demographic SummaryIV. Confidence, Attitude, and Benefit DimensionsV. SO Achievement Test SummaryVI. Overall Quality and Usefulness SummaryVII. Appendix

2

Note: Please refer to the Shaping Outcomes Evaluation Data PDF file posted on the Shaping Outcomes password protected webpage for a more in depth look at the evaluation findings.

Key Objective: Assess effectiveness and impact of the class◦ SO Version 1: Formative Evaluation

Improve design and delivery of SO class Track effectiveness of project development and

implementation◦ SO Version 2: Summative Evaluation

Assess impact on learners Measure knowledge gained by taking SO class Assure project objectives are achieved

3

Shaping Outcomes: Enrollment and Completion Rates * (June 2005 to mid-October 2007)

Embedded Tutorial† Stand-alone Course† Total†

Number of Students Enrolled 126 individuals 190 individuals &

34 teams316 individuals &

34 teams

Number of Students who Did Not Successfully Complete the Class**

7 individuals 126 individuals & 11 teams

133 individuals & 11 teams

Number of Students who Completed the Class

119 individuals (94.4% individual)


(33.7% individual & 67.6% team)



*Note: The numbers in parentheses represent completion percentages rounded off to the nearest tenth. ** Participants either withdrew from the class, were administratively withdrawn from the class, or otherwise did not complete the class successfully. †Classes were completed individually or in teams.

4

Shaping Outcomes: Enrollment and Completion Rates for Stand-alone Courses* (June 2005 to mid-October 2007)

Non-IMLS Grantees† IMLS Grantees† Total†

Number of Students Enrolled




Number of Students who Did Not Successfully Complete the Class**




Number of Students who Completed the Class

61 individuals & 1 team






*Note: The numbers in parentheses represent completion percentages rounded off to the nearest tenth. ** Participants either withdrew from the class, were administratively withdrawn from the class, or otherwise did not complete the class successfully. †Classes were completed individually or in teams. 5

Students enrolled in the embedded tutorials had the highest completion rate with 119 individuals completing the class. As SO embedded tutorials were a segment of university offered courses, students may have been more invested in completing the course.

IMLS Grantees had the highest team completion rate with 22 teams completing the course. Grantees who completed the course individually did not fair as well, with 3 out of 17 individuals completing the course.

8

Gender◦ 86.4% participants were female

Age◦ All age ranges were represented, with approximately 46% of

respondents between 19 and 39 and 54% over 40. Race/Ethnicity◦ 89.7% of participants were White

Area of Specialization◦ 61.6% of participants specialized in library sciences, 18.1% in

museum science, and 20.3% in other disciplines See detailed charts in the Appendix.

9

Participants’ confidence in performing OBPE activities greatly improved as reflected in the pre- to post-class mean scores. Participants enrolled in an embedded tutorial showed a larger improvement in confidence.

10

Participants’ attitude toward OBPE moderately improved as reflected in the pre- to post-class mean scores. Participants enrolled in an embedded tutorial showed a larger improvement in attitude.

11

Overall, most participants reported that the SO online class would benefit them and their work.

12

Paired t-test analysis indicated a statistically significant improvement in participants’ knowledge of OBPE as reflected in the pre- to post-class mean scores.

Overall, participants’ pre-class average score was 12.62 out of 20 questions and the post-class average score was 15.42 out of 20.

IMLS Grantees had the highest scores and showed greatest improvement in knowledge, with an increase of 3.63 points from pre- to post-class.

14

75.4% of all respondents reported that the overall quality of the SO modules was “Excellent” or “Above Average”.

15

77.2% of all respondents reported that the material covered in the SO online class was “Very Useful” or “Useful” .

16

Howard R. Mzumara Ranjita D. Shinde Denise L. Czachura IUPUI Testing Center 1 Presentation given at...

Documents

Transcript of Howard R. Mzumara Ranjita D. Shinde Denise L. Czachura IUPUI Testing Center 1 Presentation given at...