EILS_NSF Program Assessment

EILS_NSF Program Assessment

Strategies for obtaining meaningful & comparable

results

Strategies for obtaining meaningful & comparable

results

Discussion: Understanding

Assessment Needs

Discussion: Understanding

Assessment Needs

• Student learning outcomes vs. evaluation of program implementation

• Added twist: obtaining “comparable results”

NSF_TUES Guidelines NSF_TUES Guidelines

• Methods likely only to have local impact discouraged; encourage “broad based”

• Developing or testing newly developed approaches encouraged

• Collect and synthesize information about student understanding, reasoning, practical skills, interests, attitudes or other valued outcomes.

““Testing effectiveness of new materials and Testing effectiveness of new materials and instructional approaches”instructional approaches”

Program Implementation vs. Student Learning

Outcomes

Program Implementation vs. Student Learning

Outcomes How/Was the course taught

Students served, demographics etc.

Novelty, course context

Institutional Stability, Changes,

Connections

What did they learn?

What can they do (that they couldn’t do before)?

Skill set? Concepts? Perspectives? Practice?

What do results say about instruction and course implementation?

Comparative Cases?

Student Learning Outcomes Different

Measures

Student Learning Outcomes Different

Measures

Important to know and do

Worth being familiar with

Enduring understanding developing skill sets

Assessment Types

Traditional/ Known-Answer•quizzes & tests, •selected response•constructed response

Performance Tasks, Portfolios•Open ended responses•Demonstration, proof of concept•Authentic practice •Scoring strategies

UCSC/SEED MethodsUCSC/SEED Methods Student self-report surveys (attitudes, satisfaction, existing & new understandings)

Focus Group Interviews (supplemental to surveys)

External/participant reviews (mentor feedback, satisfaction)

Individual Lab assessments: Pre -post questionnaires (key concepts)

Pre -post problem sets (“back of the envelope reasoning”)

Pre-post concept mapping (LCA) Diagnostic Learning Activities

Also tracking student demographics, enrollment, majors, Also tracking student demographics, enrollment, majors, interest, course progressionsinterest, course progressions

Proposal for new and innovative assessment

strategies

Proposal for new and innovative assessment

strategies• Pre - post format but integrated within in-class activities or homework

• Online, performance-based • Emphasis on quantitative reasoning, data interpretation and problem solving skills

UCSC/SEED Results cont.

Indicator Categories

UCSC/SEED Results cont.

Indicator Categories

UCSC/SEED Results cont.

UCSC/SEED Results cont.

• Carbon Footprint subcategory (Radiation, Eco-toxicity, Pollution Lithosphere into Biosphere, Carbon Emissions, & Greenhouse Gases) = largest point spread

• Social Justice & equity = low totals and low point spread