Facilitators of School Improvement

Lisa Guzzardo AsaroDeane Spencer

February 2014

Rigor, DOK and Monkeys

Connector Activity

Why Rigor Matters

Good teaching can make a significant difference in student achievement, equal to one effect size (a standard deviation), which is also equivalent to the effect that demographic classifications can have on achievement.

Dr. Heather Hill, University of Michigan

Why Rigor Matters

If a student is in one of the most effective (rigorous) classrooms he or she will learn in 6 months what those in an average classroom will take a year to learn. And if a student is in one of the least effective classrooms in that school, the same amount of learning take 2 years.

Dr. Heather Hill, University of Michigan

Task Predicts Performance

If between 10 and 20% of the CCSS and questions on most standardized tests are at a low level of complexity, how much instructional time should we devote to DOK Level 1 and Level 2 questions in a lesson or an assessment?

Brenda Matthews, 2010

Task Predicts Performance

If as much as 80% of the CCSS and questions on most standardized tests (and in life) require moderate to high levels of complexity, how much of instructional time should we devote to DOK Level 3 or Level 4 questions in a lesson or on an assessment?

Brenda Matthews, 2010

Elbow TalkThink about what you do in the classroom.

What is it that makes a lesson rigorous?Is it difficulty?

Is it length of the assignment?What contributes to rigor in the classroom?

Cognitive DemandLevels of Complexity, Not DifficultyCognitive Demand

The kind and level of thinking required of students to successfully engage with and solve a task.

The way in which students interact with content.

Webb’s Depth of KnowledgeLevels of Cognitive Complexity

Level 1: Recall and ReproductionLevel 2: Skills & ConceptsLevel 3: Strategic ThinkingLevel 4: Extended Thinking

Webb’s Depth of Knowledge

The Depth of Knowledge framework was developed by Norman Webb at the University of Wisconsin to help align standards with assessments. Webb’s Depth of Knowledge was used by the Council of Chief State School Officers to align the assessments and as the basis for the Common Core State Standards.

Webb’s Depth of KnowledgeLevels of Cognitive Complexity

Task Thinking

Collecting data samples over several months Recall

Organizing the data in a chart Skills/Concepts

Using the chart to make and justify predictions Strategic Thinking

Developing a generalized model from this data and applying it to a new situation

Extended Thinking

Webb’s Depth of KnowledgeLevels of Cognitive Complexity

The Depth of Knowledge (DOK) is NOT determined by the way verbs are used, but instead by the context in which the verb is used and the depth of thinking required.

Webb’s Depth of KnowledgeLevels of Cognitive Complexity

Words like explain or analyze have to be considered in context.

“Explain to me where you live”is recall and recite.

Webb’s Depth of KnowledgeIt is about what follows the verb

“Analyze this sentence to decide if the commas have been used correctly”

•Does NOT meet criteria for high cognitive processing.•The student has been taught a rule for using commas and is using the rule to complete a task.

Webb’s Depth of KnowledgeThree uses of a verb = Three

DOK

DOK 1 – Describe three characteristics of metamorphic rocks.DOK 2 – Describe the difference between metamorphic and igneous rocks.DOK 3 – Describe a model that you might use to represent the relationships that exist within the rock cycle.

The DOK is NOT determined by the verb, but the context in which the verb is used and the depth of thinking required.

Webb’s Depth of KnowledgeDOK is about intended outcome,

not difficulty

DOK is a reference to the complexity of mental processing that occurs when students answer a question, perform a task, or generate products.

•Adding is a mental process.•Knowing the rule for adding is the intended outcome that influences the DOK.•Once someone learns the “rule” of how to add, 4 + 4 is DOK Level 1•Adding 4,658,985 + 9,993,274 is still DOK Level 1

Webb’s Depth of KnowledgeDOK is about complexity

The intended student learning outcome determines the DOK level. Every objective is assigned a DOK level. Classroom instruction and assessments reflect the DOK level of the objective or intended learning outcome.

Extending the length of an activity alone does not necessarily create rigor!

The DOK is NOT determined by how difficult the task may be, but by the complexity of the processing that is associated with the task.

Webb’s Depth of KnowledgeKey Points

•DOK is about what follows the verb.•DOK is different from task or item difficulty. (DOK 1 + DOK 1 + DOK 1 = DOK 1)•DOK is based on the cognitive complexity of the task students are being asked to do.

Webb’s Depth of KnowledgeLevels of Cognitive Complexity

Level 1: Recall and ReproductionLevel 2: Skills & ConceptsLevel 3: Strategic ThinkingLevel 4: Extended Thinking

Webb’s Depth of KnowledgeDOK 1 – Recall & Reproduction

Basic tasks that require students to use simple skills or abilities to recall or locate facts from the

text.•Identify who, what, where, when and why.•Recall facts, terms, trends, or concepts.•Define a word or concept.•Identify cause and effect.•Identify purposes.

Webb’s Depth of KnowledgeDOK 1 – Sample

Read the sentence:Undertaking the development of a nonprofit is a major

responsibility, but one that allows people to make a significant difference for a cause they truly care about.What is the meaning of the work undertaking as it is used in this sentence?

a. to give up c. to try harderb. to manage d. overachievement

Webb’s Depth of KnowledgeDOK 2 – Working with Skills and

ConceptsRequires both the initial comprehension and subsequent processing of text or portions of the

text.•Describe or explain how or why.•Apply learned concepts to new situations.•Provide an example.•Compare/contrast people, places, events, purposes, and concepts.•Classify or sort items into categories.•Convert information from one form to another.

Webb’s Depth of KnowledgeDOK 2 – Sample

What purpose does repeating the phrase, “and Brutus is an honorable man,” serve in Marc Anthony’s first speech?

a. to raise doubt about whether Brutus is honorableb. to emphasize that Marc Anthony agrees with Brutusc. to express Marc Anthony’s grief over Caesar’s deathd. to praise Caesar’s life and accomplishments

Webb’s Depth of KnowledgeDOK 3 – Short-Term Strategic

ThinkingDemands higher order thinking processes, such as analysis and evaluation. Requires knowledge and

skill from multiple subject-matter areas.•Explain Reasoning•Generate solutions in project-based settings•Support ideas with evidence•Recognize and explain misconceptions•Solve real-world problems with predictable outcomes

Webb’s Depth of KnowledgeDOK 3 – Sample

Read “A Dream Deferred” by Langston Hughes.Write 2 paragraphs to explain how the author’s use of imagery and figurative language contributes to the central theme of the poem. Cite at least 2 examples from the poem that support your reasoning and explain how they contribute to the central message.

Webb’s Depth of KnowledgeDOK 4 – Extended Strategic

ThinkingRequires student to use complex reasoning and strategic

thinking over a long period of time in order to solve a problem.

•Conduct investigations.•Gather and analyze data to support an idea or argument.•Solve real-world problems with unpredictable outcomes.•Apply and adapt information to real world situations.•Develop a logical argument.•Make predictions with evidence as support.•Design a plan of implementation.

Webb’s Depth of KnowledgeDOK 4 – Sample

Gather data in order to:•Define and describe the problems that acid rain could potentially cause in a specific farming community.•Include a proposal of alternate solutions to the problem.•Select the solution that you feel provides the best alternative and support your reasoning with evidence.•Design a plan of implementation for the solution you have proposed.

Rigor, DOK, and Monkeys – Using DOK

• Independently read the article Monkey Helpers by Tamra Orr

•As a table, generate and record on the handout:• One DOK level 1 question• Three DOK Level 2 questions• One DOK Level 3 question• One DOK Level 4 question

Rigor, DOK, and Monkeys – Sharing items

•Designate a scribe at your table.•Use the chart paper at your table to record your DOK questions.

(Recreate your handout on the chart paper)• Include your school name(s)• Record your questions with the intended DOK

noted.

Today’s Outcomes

32

•Engage in understanding Rigor and DOK as it relates to instruction

•Explore the School Systems Review, Interim Self-Assessment and Answer: Now What?

•Explore M-Step Updates

•Review the NEW School Data Analysis

•Program Evaluation Tool

•Explore MI School Data, Data Director, and Macombfsi.net

33

Today’s Roadmap

• Welcome • Connector: Rigor, DOK, and Monkeys

• School Systems Review or Interim Self- Assessment

• M-Step and Principal’s Packet• School Data Analysis• Program Evaluation Tool• MI School Data, Data Director, and Macombfsi.net

• Team Work

34

Key Working Agreements A Facilitation Tool

• Respect all Points of View

• Be Present and Engaged

• Honor Time Agreements

• Get All Voices in the Room

These breathe life into our Core Values

35

Parking LotA Facilitation Tool

•Rest questions that do not benefit the whole group

•Place questions that do not pertain to content at this time

•Place questions that pertain, but participants do not want to ask at this time

Action Required Chart

•Any request by you that I need to respond to must be placed on the Action Required Chart

•You need to PRINT your complete name, school, and email address

36

37

38

GATHERGet Ready

Collect DataBuild Profile

Stages One and Two

STUDYAnalyze Data

Set GoalsSet Measurable Objectives

Study Research

Process Data

DUE DATE:

40

AdvancED ISA MDE SSR

One or the Other; NOT BOTH

MDE: School Systems Review (SSR)

AdvancED: Interim Self Assessment (ISA)

TASKS:1.) Those schools that have had an EXTERNAL REVIEW this year, NOW WHAT? Use the ppt. and reports from AdvancED to identify action steps.

2.) All schools address the NOW WHAT?

3.) Complete either the School Systems Review, Interim Self Assessment or Self Assessment

42

43

GATHERGet Ready

Collect DataBuild ProfileFocus

Focus

Stage One GatherStep 2 Collect Data

Achievement Data

Principal Resource Packet

Division of Accountability Services (DAS)

•Spotlight News, January 15,2015

•M-Step Spring 2015 Transition Readiness Timeline

•M-STEP Resources

Division of Accountability Services (DAS) Assessment and Accountability 

http://www.michigan.gov/mde/0,1607,7-140-22709---,00.html

45

Sign up to receive

Stage Two StudyStep 4 Analyze Data (4 data points)

46

STUDYAnalyze Data

Set GoalsSet Measurable Objectives

Study Research

MDE School Data Analysis: ALL SCHOOLS

47

MDE School Data Analysis: ALL SCHOOLS

Not in ASSIST, yet!

FOUR SECTIONS in this ORDER• Demographic Data (questions 1-9)

• Process Data (questions 10-18)

• Achievement/Outcome Data (questions 19-23)

• Perception Data (questions 24-35)

48

MDE School Data Analysis: ALL SCHOOLS

Not in ASSIST, yet!SUMMARY SECTION

The intent of this summary is to synthesize the results of your data

analyses and to drive the construction of your school’s improvement plan or

reform/redesign plan.

Questions 36-38 *Question 38 is not included, yet.

49

50

MDE School Data Analysis: ALL SCHOOLS

Not in ASSIST, yet!

51

Stage Four DoStep 11 Evaluate Plan

DOImplement PlanMonitor Plan

Evaluate Plan

Deconstruct a STRATEGY at the ACTIVITY LevelCreated a Strategy Addendum

52

Program Evaluation Addendum

53

STRATEGY IMPLEMENTATION GUIDE

54

AdvancED and MDE RESOURCES

http://advanc-ed.org/partnership/mde

55

Located Here

PURPOSE ONEEvaluating the Fidelity of

Implementation

implemented as intended?

implemented consistently and with fidelity?

enough time and resources available and used for implementation?

Did the strategy work?

PURPOSE TWOEvaluating Impact (Student

Achievement)

increased student achievement?

unintended consequences (good and bad)?

be continued? discontinued? modified?

58

59

Program Evaluation Diagnostic: Due June 30

OCTOBER Work During FSI•Deconstructed a Strategy at the Activity Level•Revised the Strategy as needed•Created a PE Strategy Addendum•Complete Strategy/Program/Initiative Description(page 2 in ASSIST)

PAGE 2What is the name of the strategy/program/initiative being evaluated?Example: Classroom Instruction that Works: Non Negotiables

Provide a brief description of the strategy/program/initiative being evaluated?Example: Robert Marzano’s 2nd Edition elaborates on the 9 Strategies that work in a Classroom to include all effective instructional pedagological tools.

What is the need being addressed by the strategy/program/initiative?Describe what data analysis surfaced a student learning need that contributed to the selection of the strategy/program/initiative.

Provide the reason for selecting the strategy/program/initiative, include intended results.Describe the student learning results you expect to achieve due to implementing the strategy/program/initiative.

Cite the research supporting the strategy/program/initiative along with a brief summary.Include a brief summary explaining the strategy/program/initiative.

60

Program Evaluation Tool

January – February Work•Complete pages 5-6 on Readiness•Complete pages 7-8 on Knowledge and Skills•Complete pages 9-10 Opportunity

Program Evaluation Tool Implementation with FidelityWas the Strategy/Program/Initiative Implemented as Intended

61

IN AN IDEAL STRATEGY/PROGRAM/INITIATIVE, all

personnel involved in the program implement the strategies with fidelity

according to the research, carrying out responsibilities by their proposed timelines. They use clearly defined protocols to collect and review formative implementation data

to identify unintended consequences. Program leaders consider adjustments guided by implementation data while maintaining the integrity of results.

FOCUSFOCUS

ASSIST Program Evaluation Tool Section Screen Shot

62

ASSIST Program Evaluation Tool Response Screen Shot

63

ASSIST Program Evaluation Tool Evidence Screen Shot

64

ASSIST Program Evaluation Tool What Does the Evidence Show Screen Shot

65

66

Presenter: Dr. Jennifer Parker-Moore

MISchooldata.org Data Director

GATHERGet Ready

Collect Data Build Profile

STUDYAnalyze Data

Set GoalsSet Measurable Objectives

Research Best Practice

Team Work Time

•Create an PE Strategy Addendum•Complete page one- of the Program Evaluation Diagnostic in ASSIST•Network with Colleagues

67