Post on 11-Jul-2020
Using Findings to Foster
Innovation
Robert Pacheco, Ed.D.
Assessment Chair, RP Group
bob@assessmentcommons.org
At the end of our time together we will:
• Apply techniques to find innovative solutions to real learning problems using
results from student learning
• Utilize a gap analysis model to identify target areas of improvement to
demonstrate meeting our pledge
• Build an action plan to
Using Findings to Foster
Innovation
A thank you.
House Cleaning
Assessment Commons
www.assessmentcommons.org
Email me, bob@assessmentcommons.org - and let’s chat.
Let’s Get Started
Completes SEP
Books in stock at Bookstore
Library Orientation
Effective degree audit
Effective Early Alert program
Faculty Letter of Recommendation /
intro to network
Effective Orientation
Good impression from
campus visit Talk to Univ. Rep / Employer
Intrusive Counseling
Meet with college outreach professional
Attends Lecture Series
Join club / participate in student Govt
Connecting with faculty outside
classroom
Clean petition process
User-friendly application
process
Leverages Learning Center resources
Powerful learning experience in classroom
Financial Aid Support
Successful Completion
Get accurate perception from HS counselor
Placement Test Prep
PROGRESSENTRY COMPLETION
Potential First Time Student
CONNECTION
RP Group Model, G. Stoup
Successful
Transfer/Graduation
Academic
Intervention
What is the Effect? (Hypothetical)
Sequence of course
material
Synthesis of course
content
Work group
oversight
Counseling supportLinkages to library
Coordination between
faculty & counselors Timeliness of
feedback on
progressClassroom
technology Examples in lesson
plan
Protocols for making
group assignments
Strong
Moderate
Weak
First Year
Student
Reflect and Chat
What are the biggest challenges you face as you
consider how to use results to innovate?
(Take 5 minutes!)
The Thinking at the Time
(Schools) in the United States are, in a sense, a manufactory, doing a $2 billion dollar business each year
in trying to prepare future citizens for usefulness and efficiency in life. As such,
we use the same specifications in applying to the conduct of our business
The same principles of specialized production and manufacturing
efficiency which control other partsof the manufacturing world.
Cubberly, Stanford University
Modern Thinking (Innovation and
Design)
Every Enterprise has to become a learning organization (and) a teaching institution. Organizations that build in continuous learning in jobs will dominatethe twenty-first century. Peter Drucker
‘Learning organizations' are those organizations where people continually expand their capacity to create the results they truly desire, where new and expansive patterns of thinking are nurtured, where collective aspiration is set free, and where people are continually learning to see the whole together. Peter Senge
Hat Tip, Greg Stoup, RP Group
Augmented Accountability
The Present Future
Data do not speak for themselves. What is also needed are occasions that bring educators together to:
• examine evidence about student learning,
• reflect on its meaning and
• identify approaches that yield better results.
– Carnegie Foundation (2009)
Bringing This to What We Do
Bringing This to Ground Level
Innovation
The Five Reasons We Assess
• To become masters of our craft, studying the learning process in the discipline
and in the larger learning arenas and discovering new ways and methods to teach.
• To determine the extent to which the curriculum is working (design and implementation).
• To inform the decisions as to where time, energy and/or money should be
repurposed for continuous improvement in learning.
• To help us become a learning organization that is adaptive and nimble
for the 21st Century (Kezar USC, Drucker CGU, Senge MIT)
• To help demonstrate our quality assurance pledge to the community we serve.
Learning in the Aggregate
Source: Adapted from Jim
Nichols
Criteria Groucho Larry Curly Chico Moe Avg
Spelling 3 4 1 2 3 2.6
Grammar 2 5 3 2 4 3.4
Punctuation 4 5 2 3 4 3.6
Structure 3 2 3 5 3 3.2
Total 12 16 9 12 14
Student Grade C A D C B
Grades
Outcomes Assessment
Total down the column for individual grading. Analyze across the row for assessment of intended outcomes from the group.
Stanford Design School
http://dschool.stanford.edu/
Engaging the Findings
It is a two step process:
1. Probe the Data
2. Ideate on Possible Innovations WASC Assessment Leadership Academy, 2011
Be creative. Be scholars. Be peers. Be masters of your craft.
Step One: Probe the Data
• Ask questions and look at the data for complete
understanding (that story that the data tells)
• Engage in collaborative inquiry and information gathering
• Discuss possible reasons for the data—what factors are
responsible for the results? What explanation is there for
the findings? (This is the fun part.)
• Look at all the data
I wonder…
Step Two: Ideate Possible Innovations
• Collaboratively consider some possible next steps you would like to try to
close the learning gaps
• Reflect on other lines of inquiry we might want to pursue in future
assessment studies
• Examine whether the curriculum is working (design and implementation)
• Reflect on where time, money and efforts could be dedicated or repurposed
to improve learning.
• Think about how best to share what you learned so that we can learn from
your investigations
• Ponder how do we move from reviewing the findings to trying something
new
What if…
Ideation
Creating Choices
• Experiment and Take Risks
• Lead to Broad Learning Strategies
• Think Expansively
List Possibilities
• From the ethereal to the boots on the ground
• Identify themes from collective thought
Imagine Away Obstacles
• From the ethereal to the boots on the ground
• Identify themes from collective thought
In Practice
Let’s Start With a “Design” Challenge
Here is some assessment results that a program has been
given for critical thinking skills. Take a look:
• Senior Capstone Projects: Moderate ratings in critical thinking
• Senior Self-Assessment on Critical Thinking: High ratings in critical
thinking
• Ratings of Critical Thinking for Recent Grads By Employers and
Transfer Universities: Low ratings in critical thinking
• Alumni/ae Ratings of Critical Thinking: Low ratings in critical thinking
Probe these Findings. Imagine Away the Limitations and Obstacles. Take Five Minutes!
I wonder…
Some Possible Ideas
• What critical thinking skills were rated in capstones?
• Were the critical thinking skills demonstrated in capstones related to the
critical thinking desired by employers?
• Were the critical thinking outcomes the same for all groups?
• What kind of situations required critical thinking in employment settings?
• What kind of situations were alumni/ae experiencing for their use of critical
thinking skills?
Step Two: Ideate Possible Innovations
• Senior Capstone Projects: Moderate ratings in critical thinking
• Senior Self-Assessment on Critical Thinking: High ratings in critical
thinking
• Employers’ Ratings of Critical Thinking for Recent Grads: Low ratings
in critical thinking
• Alumni/ae Ratings of Critical Thinking* Low ratings in critical thinking
Brainstorm some possible actions (five reasons we assess) to close the assessment loop. Take Five Minutes!
Step Two: Ideate Possible Innovations
• Senior Capstone Projects: Moderate ratings in critical thinking
• Senior Self-Assessment on Critical Thinking: High ratings in critical
thinking
• Employers’ Ratings of Critical Thinking for Recent Grads: Low ratings
in critical thinking
• Alumni/ae Ratings of Critical Thinking* Low ratings in critical thinking
Brainstorm some possible actions (five reasons we assess) to close the assessment loop. Take Five Minutes!
The Five Reasons We Assess
• To become masters of our craft, studying the learning process in the discipline
and in the larger learning arenas and discovering new ways and methods to teach.
• To determine the extent to which the curriculum is working (design and implementation).
• To inform the decisions as to where time, energy and/or money should be
repurposed for continuous improvement in learning.
• To help us become a learning organization that is adaptive and nimble
for the 21st Century (Kezar USC, Drucker CGU, Senge MIT)
• To help demonstrate our quality assurance pledge to the community we serve.
In Practice
Let’s Start With a “Design” Challenge
Here is some assessment results that a program has been
given for critical thinking skills. Take a look:
• Senior Capstone Projects: Moderate ratings in critical thinking
• Senior Self-Assessment on Critical Thinking: High ratings in critical
thinking
• Employers’ Ratings of Critical Thinking for Recent Grads: Low ratings
in critical thinking
• Alumni/ae Ratings of Critical Thinking: Low ratings in critical thinking
Probe these Findings. Imagine Away the Limitations and Obstacles. Take Five Minutes!
Some Possible Steps
• Engage employers in pedagogy and curriculum of program courses and
capstones planning and evaluation.
• Dialogue with students to make sure that are able to identify critical thinking
skills (meta analysis)
• Align curriculum and pedagogy with real life situations from employers.
• Involve students in self-regulatory progress of examining their learning and
development of critical thinking skills.
• Engage faculty in collaborative definition and articulation of critical thinking
skills and learning outcomes.
Final Thoughts
How can you apply some of what we discussed today at
your sites?
How can you better share your findings in a meaningful
way that improves student work and your mastery of
your craft?
The biggest question I still have…
I am most excited to implement...
Other thoughts?