RBT and DOK Alignment for Practitioners
-
Upload
jerrie-brown -
Category
Education
-
view
2.188 -
download
0
description
Transcript of RBT and DOK Alignment for Practitioners
Practical Alignment for Item / Test Developers:
Revised Bloom’s Taxonomy (RBT) and
Depth of Knowledge Levels (DOK)Staff Trainings, 2009
Jerrie W. Brown, Content Lead for Social Studies Assessments
North Carolina State University
Center for Urban & Community Services
Technical Outreach for Public Schools (TOPS)
“Lack of excellence in American schools is not caused by ineffective teaching, but mostly by misaligning what teachers teach, what they intend to teach, and what they assess as having been taught.”
-Prof. S. Alan Cohen, 1987 University of San Francisco
“Alignment also helps ensure the validity of test results. If standards and tests are not aligned, results might provide a misleading picture of how well students are attaining standards.”
-Robert Rothman, 2003Annenberg Institute for School Reform
What is alignment?The No Child Left Behind Act of 2001 (NCLB) requires that each state align assessments to content standards.
Once the enacted (instruction), intended (state standards), and assessed (state tests) curricula have been measured, questions can be asked about the extent to which content is similar across them.
To the extent content is the same, they are said to be aligned.
J. Brown, NCSU, 2009
Defining Alignment
If the content assessed is exactly the same as the content represented in the standards, alignment is perfect.
There are two ways in which alignment can be less than perfect:
(1) Content in the standards may not be assessed, and (2) content assessed may not be in the standards.
J. Brown, NCSU, 2009
Alignment Procedure
1. Review alignment criteria of Revised Bloom’s Taxonomy (RBT) and Depth of Knowledge (DOK) levels
2. Rate standards3. Rate test items4. Aggregate individual ratings5. Determine degree of RBT-based alignment6. Determine degree of DOK-based alignment
J. Brown, NCSU, 2009
J. Brown, NCSU, 2009 6
Alignment DimensionsDOKDOK – Two Dimensions1. Depth of Knowledge Levels2. Content Topics or Strands
(Webb, 2002)
RBTRBT – Two Dimensions1. Cognitive Processes2. Knowledge Types
(Anderson, 2001)
6. Create
5. Evaluate
4. Analyze
3. Apply
2. Understand
1. RememberContent
Topic/Strand A
4. Extended Thinking
3. Strategic Thinking
2. Skill/Concept
1. RecallA. Factual
B. Conceptual
C. Procedural
D. Meta-Cognitive
ContentTopic/Strand B
ContentTopic/Strand C
(Note: achieves content validity) (Note: achieves content validity and instructional validity)
7
Revised Bloom’s Taxonomy (RBT) Table for _____________________________
(Subject / Grade Level)
KnowledgeDimension
(Lower Order = 0%) Cognitive Process Dimension (Higher Order = 0%)
1.Remember
2.Understand
3.Apply
4.Analyze
5.Evaluate
6.Create
A. FactualKnowledge (0%)
B.ConceptualKnowledge(0%)
C.ProceduralKnowledge(0%)
D.Meta-CognitiveKnowledge(0%)
Adapted from Anderson, Lorin W. and David R. Krathwohl, et al., (2001).
J. Brown, NCSU, 2009
8
Depth of Knowledge (DOK) Levels for _____________________________
(Subject / Grade Level)
ContentCategories
Depth of Knowledge Levels
CategoricalConcurrence
Yes / No
1.Recall
2.Skill/Concept
3.StrategicThinking
4.ExtendedThinking
A. Topic/Strand__________(Specify)
B.Topic/Strand__________(Specify)
C.Topic/Strand__________(Specify)
Adapted from Norm Webb, (2002).
J. Brown, NCSU, 2009
RBT’s Knowledge Dimension:
Major Types Definitions
A Factual The basic elements students must know to be acquainted with a discipline or solve problems in it
B Conceptual The interrelationships among the basic elements within a larger structure that enable them to function together
C Procedural How to do something, methods of inquiry, and criteria for using skills, algorithms, techniques, and methods
D Metacognitive Knowledge of cognition in general as well as awareness and knowledge of one’s own cognition
J. Brown, NCSU, 2009
RBT’s Knowledge Dimension: Major Types Subtypes
A Factual AA. Knowledge of terminologyAB. Knowledge of specific details and elements
B Conceptual BA. Knowledge of classifications and categoriesBB. Knowledge of principles and generalizationsBC. Knowledge of theories, models, and structures
C Procedural CA. Knowledge of subject-specific skills and algorithmsCB. Knowledge of subject-specific techniques and methodsCC. Knowledge of criteria for determining when to use appropriate procedures
D Metacognitive DA. Strategic knowledgeDB. Knowledge about cognitive tasks, including appropriate contextual and conditional knowledgeDC. Self-knowledge
J. Brown, NCSU, 2009
RBT’s Cognitive Process Dimension:
Cognitive Process Categories
Definitions
1 Remember Retrieve relevant knowledge from long-term memory
2 Understand Construct meaning from instructional messages, including oral, written, and graphic communication
3 Apply Carry out or use a procedure in a given situation
4 Analyze Break material down into its constituent parts and determine how the parts relate to one another and to an overall structure or purpose
5 Evaluate Make judgments based on criteria and standards
6 Create Put elements together to form a coherent or functional whole; reorganize elements into a new pattern or structure
J. Brown, NCSU, 2009
RBT’s Cognitive Process Dimension: Cognitive Process Categories Cognitive Processes
1 Remember 1.1 Recognizing1.2 Recalling
2 Understand 2.1 Interpreting2.2 Exemplifying2.3 Classifying2.4 Summarizing
2.5 Inferring2.6 Comparing2.7 Explaining
3 Apply 3.1 Executing3.2 Implementing
4 Analyze 4.1 Differentiating4.2 Organizing4.3 Attributing
5 Evaluate 5.1 Checking5.2 Critiquing
6 Create 6.1 Generating6.2 Planning6.3 Producing
J. Brown, NCSU, 2009
Norm Webb’s Specific Alignment Criteria
Content FocusA. Categorical Concurrence: between goals and assessment is met if the same or consistent categories of content appear in both documents
B. Depth-of-Knowledge Consistency: at least 50% of the items corresponding to a goal had to be at or above the level of knowledge of the goal
C. Range-of-Knowledge Correspondence: fifty percent of the objectives for a goal must have at least one related assessment item
D. Balance of Representation: the degree to which one objective is given more emphasis on the assessment than another
Formal Alignment Studies and Depth of Knowledge Levels (DOKs)
J. Brown, NCSU, 2009
Depth of Knowledge LevelsNorm Webb (2002) Generic DOKs
Level 1: Recall– Recall of a fact, information, or procedure.
Level 2: Skill/Concept– Use information or conceptual knowledge, two or more steps, etc.
Level 3: Strategic Thinking– Requires reasoning, developing plan or a sequence of steps, some
complexity, more than one possible answer.
Level 4: Extended Thinking– Requires an investigation, time to think and process multiple
conditions of the problem.
J. Brown, NCSU, 2009
Subject-Specific DOKsScience Social Studies Math
Level 1Recall and Reproduction
Recall of Information
Recall
Level 2Skills and Concepts
Basic Reasoning
Skill/Concept
Level 3Strategic Thinking
Complex Reasoning
Strategic Thinking
Level 4Extended Thinking
Extended Reasoning
Extended Thinking
J. Brown, NCSU, 2009
Reading DOKs
J. Brown, NCSU, 2009
Level 1 :•Support ideas by reference to details in the text.•Use a dictionary to find the meaning of words.•Identify figurative language in a reading passage.
Level 2:•Use context cues to identify the meaning of unfamiliar words.•Predict a logical outcome based on information in a reading selection.•Identify and summarize the major events in a narrative.
Level 3:•Determine the author’s purpose and describe how it affects the interpretation of a reading selection.•Summarize information from multiple sources to address a specific topic.•Analyze and describe the characteristics of various types of literature.
Level 4: •Analyze and synthesize information from multiple sources.•Examine and explain alternative perspectives across a variety of sources. •Describe and illustrate how common themes are found across texts from different cultures.
Getting Started
Resources needed:– Knowledge types (a-d)– Cognitive processes (1-6)– Depth of knowledge levels (1-4)– Standards ratings forms– Test items ratings forms– RBT blank taxonomy tables
J. Brown, NCSU, 2009
Rating WA State Standards
J. Brown, NCSU, 2009
Plotting Standards
J. Brown, NCSU, 2009
KnowledgeDimension
(Lower Order = 0%) Cognitive Process Dimension (Higher Order = 0%)
1.Remember
2.Understand
3.Apply
4.Analyze
5.Evaluate
6.Create
A. FactualKnowledge (0%)
101, 5
9B.ConceptualKnowledge(0%)
2, 3
86
C.ProceduralKnowledge(0%)
74
D.Meta-CognitiveKnowledge(0%)
Rating ‘WASL’ Test Items
J. Brown, NCSU, 2009
Plotting Test Items
J. Brown, NCSU, 2009
KnowledgeDimension
(Lower Order = 0%) Cognitive Process Dimension (Higher Order = 0%)
1.Remember
2.Understand
3.Apply
4.Analyze
5.Evaluate
6.Create
A. FactualKnowledge (0%)
101, 5
9B.ConceptualKnowledge(0%)
2, 3
86
C.ProceduralKnowledge(0%)
74
D.Meta-CognitiveKnowledge(0%)
Individual and Group Results
Thank You for Participating!
23
FOR MORE INFORMATION:
Mr. Jerrie W. Brown, ConsultantNorth Carolina State UniversityCenter for Urban and Community ServicesTechnical Outreach for Public Schools1500 Blue Ridge RoadRaleigh, N. Carolina 27607
Tel: (919) 515-1125Email: [email protected]: http://www.cuacs.ncsu.edu