Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G....

44
Using Findings to Foster Innovation Robert Pacheco, Ed.D. Assessment Chair, RP Group [email protected]

Transcript of Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G....

Page 1: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Using Findings to Foster

Innovation

Robert Pacheco, Ed.D.

Assessment Chair, RP Group

[email protected]

Page 2: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

At the end of our time together we will:

• Apply techniques to find innovative solutions to real learning problems using

results from student learning

• Utilize a gap analysis model to identify target areas of improvement to

demonstrate meeting our pledge

• Build an action plan to

Using Findings to Foster

Innovation

Page 3: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

A thank you.

Page 4: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

House Cleaning

Page 5: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

http://tinyurl.com/z7cbokp

Page 6: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...
Page 7: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...
Page 8: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

www.cccnext.net

Page 9: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...
Page 10: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...
Page 11: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...
Page 12: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...
Page 13: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Assessment Commons

www.assessmentcommons.org

Email me, [email protected] - and let’s chat.

Page 14: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Let’s Get Started

Page 15: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Completes SEP

Books in stock at Bookstore

Library Orientation

Effective degree audit

Effective Early Alert program

Faculty Letter of Recommendation /

intro to network

Effective Orientation

Good impression from

campus visit Talk to Univ. Rep / Employer

Intrusive Counseling

Meet with college outreach professional

Attends Lecture Series

Join club / participate in student Govt

Connecting with faculty outside

classroom

Clean petition process

User-friendly application

process

Leverages Learning Center resources

Powerful learning experience in classroom

Financial Aid Support

Successful Completion

Get accurate perception from HS counselor

Placement Test Prep

PROGRESSENTRY COMPLETION

Potential First Time Student

CONNECTION

RP Group Model, G. Stoup

Page 16: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Successful

Transfer/Graduation

Academic

Intervention

What is the Effect? (Hypothetical)

Sequence of course

material

Synthesis of course

content

Work group

oversight

Counseling supportLinkages to library

Coordination between

faculty & counselors Timeliness of

feedback on

progressClassroom

technology Examples in lesson

plan

Protocols for making

group assignments

Strong

Moderate

Weak

First Year

Student

Page 17: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Reflect and Chat

What are the biggest challenges you face as you

consider how to use results to innovate?

(Take 5 minutes!)

Page 18: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

The Thinking at the Time

(Schools) in the United States are, in a sense, a manufactory, doing a $2 billion dollar business each year

in trying to prepare future citizens for usefulness and efficiency in life. As such,

we use the same specifications in applying to the conduct of our business

The same principles of specialized production and manufacturing

efficiency which control other partsof the manufacturing world.

Cubberly, Stanford University

Page 19: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Modern Thinking (Innovation and

Design)

Every Enterprise has to become a learning organization (and) a teaching institution. Organizations that build in continuous learning in jobs will dominatethe twenty-first century. Peter Drucker

‘Learning organizations' are those organizations where people continually expand their capacity to create the results they truly desire, where new and expansive patterns of thinking are nurtured, where collective aspiration is set free, and where people are continually learning to see the whole together. Peter Senge

Page 20: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Hat Tip, Greg Stoup, RP Group

Augmented Accountability

Page 21: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

The Present Future

Page 22: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Data do not speak for themselves. What is also needed are occasions that bring educators together to:

• examine evidence about student learning,

• reflect on its meaning and

• identify approaches that yield better results.

– Carnegie Foundation (2009)

Bringing This to What We Do

Page 23: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Bringing This to Ground Level

Innovation

Page 24: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

The Five Reasons We Assess

• To become masters of our craft, studying the learning process in the discipline

and in the larger learning arenas and discovering new ways and methods to teach.

• To determine the extent to which the curriculum is working (design and implementation).

• To inform the decisions as to where time, energy and/or money should be

repurposed for continuous improvement in learning.

• To help us become a learning organization that is adaptive and nimble

for the 21st Century (Kezar USC, Drucker CGU, Senge MIT)

• To help demonstrate our quality assurance pledge to the community we serve.

Page 25: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Learning in the Aggregate

Source: Adapted from Jim

Nichols

Criteria Groucho Larry Curly Chico Moe Avg

Spelling 3 4 1 2 3 2.6

Grammar 2 5 3 2 4 3.4

Punctuation 4 5 2 3 4 3.6

Structure 3 2 3 5 3 3.2

Total 12 16 9 12 14

Student Grade C A D C B

Grades

Outcomes Assessment

Total down the column for individual grading. Analyze across the row for assessment of intended outcomes from the group.

Page 26: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Stanford Design School

http://dschool.stanford.edu/

Page 27: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Engaging the Findings

It is a two step process:

1. Probe the Data

2. Ideate on Possible Innovations WASC Assessment Leadership Academy, 2011

Be creative. Be scholars. Be peers. Be masters of your craft.

Page 28: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Step One: Probe the Data

• Ask questions and look at the data for complete

understanding (that story that the data tells)

• Engage in collaborative inquiry and information gathering

• Discuss possible reasons for the data—what factors are

responsible for the results? What explanation is there for

the findings? (This is the fun part.)

• Look at all the data

I wonder…

Page 29: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Step Two: Ideate Possible Innovations

• Collaboratively consider some possible next steps you would like to try to

close the learning gaps

• Reflect on other lines of inquiry we might want to pursue in future

assessment studies

• Examine whether the curriculum is working (design and implementation)

• Reflect on where time, money and efforts could be dedicated or repurposed

to improve learning.

• Think about how best to share what you learned so that we can learn from

your investigations

• Ponder how do we move from reviewing the findings to trying something

new

What if…

Page 30: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Ideation

Creating Choices

• Experiment and Take Risks

• Lead to Broad Learning Strategies

• Think Expansively

List Possibilities

• From the ethereal to the boots on the ground

• Identify themes from collective thought

Imagine Away Obstacles

• From the ethereal to the boots on the ground

• Identify themes from collective thought

Page 31: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

In Practice

Page 32: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...
Page 33: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Let’s Start With a “Design” Challenge

Here is some assessment results that a program has been

given for critical thinking skills. Take a look:

• Senior Capstone Projects: Moderate ratings in critical thinking

• Senior Self-Assessment on Critical Thinking: High ratings in critical

thinking

• Ratings of Critical Thinking for Recent Grads By Employers and

Transfer Universities: Low ratings in critical thinking

• Alumni/ae Ratings of Critical Thinking: Low ratings in critical thinking

Probe these Findings. Imagine Away the Limitations and Obstacles. Take Five Minutes!

I wonder…

Page 34: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...
Page 35: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Some Possible Ideas

• What critical thinking skills were rated in capstones?

• Were the critical thinking skills demonstrated in capstones related to the

critical thinking desired by employers?

• Were the critical thinking outcomes the same for all groups?

• What kind of situations required critical thinking in employment settings?

• What kind of situations were alumni/ae experiencing for their use of critical

thinking skills?

Page 36: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Step Two: Ideate Possible Innovations

• Senior Capstone Projects: Moderate ratings in critical thinking

• Senior Self-Assessment on Critical Thinking: High ratings in critical

thinking

• Employers’ Ratings of Critical Thinking for Recent Grads: Low ratings

in critical thinking

• Alumni/ae Ratings of Critical Thinking* Low ratings in critical thinking

Brainstorm some possible actions (five reasons we assess) to close the assessment loop. Take Five Minutes!

Page 37: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Step Two: Ideate Possible Innovations

• Senior Capstone Projects: Moderate ratings in critical thinking

• Senior Self-Assessment on Critical Thinking: High ratings in critical

thinking

• Employers’ Ratings of Critical Thinking for Recent Grads: Low ratings

in critical thinking

• Alumni/ae Ratings of Critical Thinking* Low ratings in critical thinking

Brainstorm some possible actions (five reasons we assess) to close the assessment loop. Take Five Minutes!

Page 38: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

The Five Reasons We Assess

• To become masters of our craft, studying the learning process in the discipline

and in the larger learning arenas and discovering new ways and methods to teach.

• To determine the extent to which the curriculum is working (design and implementation).

• To inform the decisions as to where time, energy and/or money should be

repurposed for continuous improvement in learning.

• To help us become a learning organization that is adaptive and nimble

for the 21st Century (Kezar USC, Drucker CGU, Senge MIT)

• To help demonstrate our quality assurance pledge to the community we serve.

Page 39: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

In Practice

Page 40: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...
Page 41: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Let’s Start With a “Design” Challenge

Here is some assessment results that a program has been

given for critical thinking skills. Take a look:

• Senior Capstone Projects: Moderate ratings in critical thinking

• Senior Self-Assessment on Critical Thinking: High ratings in critical

thinking

• Employers’ Ratings of Critical Thinking for Recent Grads: Low ratings

in critical thinking

• Alumni/ae Ratings of Critical Thinking: Low ratings in critical thinking

Probe these Findings. Imagine Away the Limitations and Obstacles. Take Five Minutes!

Page 42: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Some Possible Steps

• Engage employers in pedagogy and curriculum of program courses and

capstones planning and evaluation.

• Dialogue with students to make sure that are able to identify critical thinking

skills (meta analysis)

• Align curriculum and pedagogy with real life situations from employers.

• Involve students in self-regulatory progress of examining their learning and

development of critical thinking skills.

• Engage faculty in collaborative definition and articulation of critical thinking

skills and learning outcomes.

Page 43: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...
Page 44: Using Findings to Foster Innovationct-prod-wp.taftcollege.edu/iarp/download/...RP Group Model, G. Stoup. Successful Transfer/Graduation Academic Intervention What is the Effect? ...

Final Thoughts

How can you apply some of what we discussed today at

your sites?

How can you better share your findings in a meaningful

way that improves student work and your mastery of

your craft?

The biggest question I still have…

I am most excited to implement...

Other thoughts?