FAIMER 2007 1

Program Evaluation 3:How is it used?Evaluation as Research

Stewart Mennin

Bill Burdick

2 FAIMER 2007

HOW PROGRAM EVALUATION INFORMS & IMPROVES CURRICULUM

Faculty & students monitor & improve curriculum

Short-loop feedback ---continuous improvement

Confirms the extent to which curriculum is working

Education developed

3 FAIMER 2007

HOW PROGRAM EVALUATION INFORMS & IMPROVES CURRICULUM

Reveals stresses & strains on the faculty & students

Adjustments made in curriculum

Capture unintended outcomes

Compare student performance with 2° data (national, other)

4 FAIMER 2007

HOW PROGRAM EVALUATION INFORMS & IMPROVES CURRICULUM

Collect & analyze long-term measures

Specific issues studied upon request

Produce reports

Research & scholarship in medical education

5 FAIMER 2007

What Is Studied?About Students Characteristics, demographics 2° Noncognitive dimensions Professional identity, specialty choice,

location Cognitive behavior & performance data The work place (during & after

graduation)

6 FAIMER 2007

What Is Studied?

About Teachers Attitudes & experiences with

curriculum change and implementation Educational effort Perception of Students

7 FAIMER 2007

What Is Studied?

About Curriculum Perceptions, experiences, & attitudes Integration of basic & clinical sciences Performance on assessments Community-based learning Research projects Other

8 FAIMER 2007

What is Studied in Practice

Health Professionals as Health Professionals as Team MemberTeam Member

Health Professionals as Health Professionals as Managers in Patient CareManagers in Patient Care

Clinical ExpertiseClinical Expertise

Patient ManagementPatient Management

Patient CommunicationPatient Communication

Patient EducationPatient Education

Quality AssuranceQuality Assurance

Health Professionals Health Professionals as Managers of as Managers of ThemselvesThemselves

Insight Into Own LimitsInsight Into Own Limits

Self CareSelf Care

Lifelong LearningLifelong Learning

Continuing EducationContinuing Education

Personal DevelopmentPersonal Development

Health Professionals as Health Professionals as Managers of Their Managers of Their EnvironmentEnvironment

Peer CommunicationPeer Communication

Resource UtilizationResource Utilization

Systems ManagtementSystems Managtement

Clinical GovernanceClinical Governance

Respect for ColleaguesRespect for Colleagues

Health of PopulationsHealth of Populations

Teaching StudentsTeaching Students

Quality AssuranceQuality Assurance

Hays et al, 2002

9 FAIMER 2007

Evaluation Methods

Quasi - experimental Comparison groups? Triangulation Qualitative before quantitative 3-year minimum

10 FAIMER 2007

Data Comparisons

Standards & groups to which data are compared?

Previous classes Subgroups within the student population Curricula & students at other schools National or international data when available Criterion based standards developed by staff

11 FAIMER 2007

Other Key Considerations Support & resources

Grants 6-10% of budget

Safe data, storage Talent, skills, expertise Publication

Internal External

FAIMER 2007 12

Thank you very much