1
Oregon Content Standards Oregon Content Standards Evaluation Project, Evaluation Project,
Contract Amendment Phase: Contract Amendment Phase: Preliminary FindingsPreliminary Findings
Dr. Stanley Rabinowitz
WestEd
November 6, 2007
2
Background: Overview of Standards Evaluation Project
One of several key projects funded by the Gates Foundation
Independent evaluation by WestEd, a non-profit educational research, development, and service agency
3
Oregon Content Standards and Assessment System Evaluation
WestEd was contracted to Review and evaluate the content
standards (all grades, all academic content areas)
Evaluate alignment among the state assessments and the content standards
Make recommendations for improvement of Oregon’s comprehensive standards and assessment system
4
Contract Amendment Phase
Extended scope of work to assist the ODE in implementing key recommendations that emerged from the comprehensive evaluation
Builds upon current state initiatives Time frame: work will extend from mid-
October to December 31, 2007
5
Contract Amendment Scope of Work
Three key steps:1. Analysis of the degree to which Oregon’s
existing content standards support the draft Essential Skills
2. Analysis of the degree to which Oregon’s current test item pool (TESA/OAKS), work samples, and writing assessments assess proficiency in the Essential Skills
3. Identification of Core Standards and recommendations for addressing gaps in coverage of core concepts
6
Focus for Today: Step 1Support for Essential Skills in
Existing Standards
Based on draft 3.0 Essential Skills (ES) Evidence of degree to which current content
standards in ELA, mathematics, science, and social sciences support the ES
Backward mapping of support from CIM through benchmark grades 3, 5, and 8
Recommendations for improving/refining draft Essential Skills to ensure effective application across content areas
7
Step 1 analyses are grounded in and informed by Findings from the evaluation of Oregon’s
standards and assessments Responses to Oregon’s Critical Questions Review of research literature Trends in practices across other states WestEd staff expertise in the content areas and
in standards and assessment development, curriculum and instruction, measurement, and cognitive development
Input solicited from state stakeholder groups– Department of Education staff – Essential Skills Task Force
8
Draft 3.0 Essential Skills
1. Read and comprehend a variety of texts at different levels of difficulty.
2. Write clearly and accurately for a variety of purposes.
3. Listen actively, speak clearly, and present publicly.
4. Think critically.5. Apply math to solve problems in a variety of
settings.6. Use technology to learn, live, and work.7. Demonstrate civic and community engagement.8. Demonstrate global literacy.
9
Protocol for Step 1 Determine whether each CIM standard
supports a draft Essential Skill1. Which one (if any) of the eight ES does it primarily
support?2. What is the strength of that relationship?
• Strong- CIM standard supports a fundamental skill or concept as explicitly stated in the Essential Skill
• Partial- CIM standard supports the Essential Skill in a superficial way or at a lower complexity level
Backward map from the CIM standard to identify evidence of support for the ES in grades 3, 5, and 8
– Do standards in grades 3, 5, 8, and CIM support the Essential Skill in a logical and coherent sequence or progression?
10
Guidelines Used During Analyses
Analysts focused on the bold category-level statements that defined each Essential Skill.
Where provided, the numbered indicator statements were used to clarify the focus of each Essential Skill.
A CIM standard could support an Essential Skill without specifically addressing one of the indicator statements provided.
11
Preliminary Findings What percentage of the CIM content
standards in four content areas support the Essential Skills?
Number of CIM
Standards Analyzed
Number of Standards
Supporting an ES*
Percentage of Standards
Supporting an ES
ELA 105 105 100%
Mathematics 84 84 100%
Science 18 18 100%
Social Sciences 39 34 87%
total N = 246 241 98%
*Based on primary relationships (i.e., each standard’s support was recorded for primary link only).
12
More Preliminary Findings…
Overall, how was this support distributed across Essential Skills?
n = number of CIM standards, across four content areas, based on primary relationships
ES 1 ES 2 ES 3 ES 4 ES 5 ES 6 ES 7 ES 8
n=46 n=32 n=21 n=34 n=84 n=1 n=11 n=12
19% 13% 9% 14% 34% <1% 5% 5%
13
More Preliminary Findings… Within each content area, how do the CIM
standards support the Essential Skills?*
ES 1 ES 2 ES 3 ES 4 ES 5 ES 6 ES 7 ES 8
ELA(all standards
support an ES)
44% 30% 20% 5% 0 1% 0 0
Math(all standards
support an ES)
0 0 0 0 100% 0 0 0
Science(all standards
support an ES)
0 0 0 100% 0 0 0 0
Social Sciences
(34/39 standards
support an ES)
0 0 0 28% 0 0 28% 31%
*Based on primary relationships.
14
More Preliminary Findings… Of those CIM standards that support the
Essential Skills, what percentage was rated as providing strong support?
Number of CIM Standards
Supporting ES*
Number of Standards Providing
Strong Support
Percentage of Standards Providing
Strong Support
ELA 105 99 94%
Math 84 78 93%
Science 18 16 89%
Social Sciences 34 23 68%
total 241 216 90%
*Based on primary relationships.
15
More Preliminary Findings…
What percentage of CIM standards that support the ES show a coherent progression across grades 3, 5, 8, and CIM?
Number of CIM Standards Supporting
ES*
Number of Standards
Showing Coherent Progression
Percentage of Standards
Showing Coherent Progression
ELA 105 87 83%
Math 84 72 86%
Science 18 16 89%
Social Sciences
34 31 91%
total 241 206 85%
*Based on primary relationships.
16
Summary of Preliminary Findings
Overall, 98% of existing standards across the four content areas (ELA, math, science, and social sciences) primarily supported one of the draft Essential Skills.
Across content areas, the majority of relationships among supporting standards and the Essential Skills was strong.– ELA and math: >92% support was strong– Science: 89% support was strong– Social sciences: 68% support was strong
17
Summary of Preliminary Findings
The distribution of standards’ primary support across Essential Skills varied by content area.– Overall, less than 10% of standards’ primary support
was associated with ES 6, 7, and 8 combined.– Primary support in the ELA standards was balanced
across three ES (1, 2, 3).– 100% of math standards provided primary support for
ES 5.– 100% of science standards provided primary support
for ES 4.– Primary support in the social sciences standards was
balanced across three ES (4, 7, and 8). Across content areas, evidence of coherent progression
across grades ranged from 83-91% (85% overall).
18
Next Steps Analysis of the degree to which Oregon’s current
test item pool (TESA/OAKS), work samples, and writing assessments assess proficiency in the Essential Skills
Identification of Core Standards and recommendations for addressing gaps in coverage of core concepts
Formal report with findings and recommendations for improvement/ refinement emerging from the Amendment Phase
19
For more information about the standards and assessment evaluation:
www.ode.state.or.us/teachlearn/real/standards/Standards_Review.aspx
For more information about WestEd:
www.wested.org
Top Related