Transitioning to the Smarter Balanced Assessment System North Carolina Superintendent Association
Transitioning to the Smarter Balanced Assessment System North Carolina Superintendent Association...
-
Upload
jesus-preston -
Category
Documents
-
view
214 -
download
0
Transcript of Transitioning to the Smarter Balanced Assessment System North Carolina Superintendent Association...
Transitioning to the Smarter Balanced Assessment System
North Carolina Superintendent Association
Susan GendronSenior Fellow
International Center June 17 2012
Key to Effective Instruction Is Alignment
• Organizational Leadership• Instructional Leadership• Teaching
Teaching
Organ
izational
Lead
ersh
ipInstructional
Leadership
Student Achievement
What will our students need to:
Know Do
4
PISA 2009
1 Shanghai-China 556
2 Korea 539
3 Finland 536
4 Hong Kong-China 533
5 Singapore 526
6 Canada 524
7 New Zealand 521
8 Japan 520
9 Australia 515
10 Netherlands 508
17 United States 500
20 Germany 497
21 Ireland 496
22 France 496
25 United Kingdom 494
33 Spain 481
43 Russian Federation 459
48 Mexico 425
53 Brazil 412
57 Indonesia 402
Overall Reading
Scale
Significantly Above OECD Average
Not Significantly Different
(OECD Average 493)
Significantly below OECD Average
PISA 2009
Overall Math Scale
Significantly Above OECD Average
Not Significantly Different
(OECD Average 496)
Significantly below OECD Average
1 Shanghai-China 600
2 Singapore 562
3 Hong Kong-China 555
4 Korea 546
6 Finland 541
9 Japan 529
10 Canada 527
11 Netherlands 526
13 New Zealand 519
15 Australia 514
16 Germany 513
22 France 497
28 United Kingdom 492
31 United States 487
32 Ireland 487
34 Spain 483
38Russian Federation
468
51 Mexico 419
57 Brazil 386
61 Indonesia 371
PISA 2009
Overall Science
Scale
Significantly Above OECD Average
Not Significantly Different
(OECD Average 501)
Significantly below OECD Average
1 Shanghai-China 575
2 Finland 554
3 Hong Kong-China 549
4 Singapore 542
5 Japan 539
6 Korea 538
7 New Zealand 532
8 Canada 529
10 Australia 527
11 Netherlands 522
13 Germany 520
16 United Kingdom 514
20 Ireland 508
23 United States 502
27 France 498
36 Spain 488
39 Russian Federation 478
50 Mexico 416
53 Brazil 405
60 Indonesia 383
Reading Risk
Mapping State Proficiency Standards onto NAEP Scales, IES August 2011
NC
ProficiencyProficiencyGrade 4 Reading 2009Grade 4 Reading 2009
ProficientRequired
NAEP Score
North Carolina 69 % 204
Massachusetts 54 % 234
Missouri 47 % 229
New Hampshire 74% 211
Oregon 84 % 177
Washington 73 % 205
Vermont 70% 214
Reading Risk
Mapping State Proficiency Standards onto NAEP Scales, IES August 2011
NCNC
ProficiencyProficiencyGrade 8 Reading 2009Grade 8 Reading 2009
ProficientRequired
NAEP Score
North Carolina 66% 246
Minnesota 67% 259
Missouri 50% 267
Vermont 69% 259
Oregon 69% 250
Washington 68% 253
Math Risk
Mapping State Proficiency Standards onto NAEP Scales, IES August 2011
NCNC
ProficiencyProficiencyGrade 4 Mathematics 2009Grade 4 Mathematics 2009
ProficientRequired
NAEP Score
North Carolina 81 % 220
Massachusetts 48 % 255
Hawaii 50 % 239
New Hampshire 73 % 237
New Mexico 77 % 224
Washington 52 % 243
Missouri 45 % 246
Math Risk
Mapping State Proficiency Standards onto NAEP Scales, IES August 2011
NCNC
ProficiencyProficiencyGrade 8 Mathematics 2009Grade 8 Mathematics 2009
ProficientRequired
NAEP Score
North Carolina 80 % 253
Massachusetts 49 % 300
Missouri 47 % 287
Hawaii 39 % 286
Minnesota 58 % 287
Oregon 71 % 266
Washington 51 % 288
Elbow partner
• What are your risks?• How will you prepare your school for
a potential dip in scores?• Do you have a communication
strategy and a clear vision for the future?
16
Organ
izational
Lead
ersh
ip
StudentAchievement
Culture
Organizational Leadership
Levin and Elmore• Everyone needs to collaborate to
ensure that daily teaching and learning practices are the focus of the school
• All responsible for success• Principals and teachers are
fundamentally evaluators• Leaders responsible for cultural
changes – by displacing specific norms, structures, and processes by others
How will you create a culture in your schools of ALL students “college and career ready”?
20
Rigor and Relevance
Teaching
Rigor/Relevance Rigor/Relevance For For
All StudentsAll Students
22
A B
DC
1.1. AwarenessAwareness2.2. Comprehension Comprehension 3.3. ApplicationApplication4.4. AnalysisAnalysis5.5. Synthesis Synthesis 6.6. EvaluationEvaluation
Knowledge TaxonomyKnowledge Taxonomy
23
Application ModelApplication Model1.1. Knowledge in one disciplineKnowledge in one discipline
2. Application within one 2. Application within one disciplinediscipline
3. Application across disciplines3. Application across disciplines
4. Application to real-world 4. Application to real-world predictable situationspredictable situations
5. Application to real-world 5. Application to real-world unpredictable situationsunpredictable situations
24
25
Curriculum-Instruction-Assessment • What should students know and to be able to do?
•What should students learn?
•What should students be taught?
• What are students being taught? • How are students being taught?
• What have students learned? • What haven’t students learned?
Curriculum
Students
Instruction Assessment
Curriculum-Instruction-Assessment Connections
ObservationObservation InterpretationInterpretation
CognitionCognition
“AssessmentTriangle”
Evidence-Based Design Framework
Models of Cognition
• Describe how students acquire knowledge and develop competence in a particular area
• Reflect recent and credible scientific evidence of typical learning processes and informed experiences of expert teachers
• Describe typical learning progression toward competence, including milestones (benchmarks)
31
Observation Models
• A set of specifications for assessment tasks that will elicit illuminating responses from students
• The tasks or situations are linked to the cognitive model of learning and should prompt students to say, do, or create something that provides evidence to support inferences about students’ knowledge, skills, and cognitive processes
Interpretation
• Interpretations use the evidence from observations to make claims about what students understand and can do
• Claims– Frame a manageable number of learning goals
around which instruction can be organized– Guide the specification of appropriate evidence– Provides a basis for meaningful reporting to
different interested audiences
An Overview of SBAC’s Approach
Content Specifications …– Create a bridge between standards and assessment and,
ultimately, instruction
– Organize the standards around major constructs & big ideas
– Express what students should learn and be able to do
Each claim is described for assessment
• Rationale for each claim– Why is this learning goal important for College & Career
Readiness (CCR)?– What does the research say about learning in this area?
• What does ‘sufficient’ evidence look like?– What types of items/tasks?– What content/texts will be emphasized?
• What are some suggested reporting categories?
Summative Assessment Targets
• Indicate proposed prioritized content for the summative assessment- link CCSS to the kinds of items/tasks students will respond to
• Show how one or more (or parts) CCSS addresses the target – ‘bundles’ CCSS (examples on next slide)
– Standards or parts of standards that relate to same type of understanding & comparable rigor/DOK demands
– Several similar CCSS from different strands
Assessment Claims for English Language Arts/Literacy
Reading
Writing
Speaking/Listening
Research/Inquiry
(a/o Round 2 – released 9/20/11)
Draft Overall Assessment Claims for English Language Arts/Literacy
OVERALL 3-8
OVERALL 9-12
(a/o Round 2 – released 9/20/11)
Students can demonstrate
progress toward college and
career readiness in English
language arts and literacy.
Students can demonstrate
college and career readiness in
English language arts and
literacy.
English Language Arts and Literacy Standards “Roadmap”
READING WRITINGSPEAKING & LISTENING LANGUAGE
10 Anchor Standardsfor College and Career Readiness
10 Anchor Standardsfor College and Career
Readiness
6 Anchor Standards for CCR
6 Anchor Standards for CCR
ELA Standards
K-12
ELA Standards
K-12
ELA Standards
K-12
Literacy Standards
6-12
ELA Standards
K-12
Literacy Standards
6-12
Literary Text
Hist. / S.S.
Sci. / Tech Subj.
Inform Text
1K
2
3
4
5
6
7
8
9-10
11-12
9-10
11-12
6-8
9-10
11-12
6-8
11-12
1K
2
3
4
5
6
7
8
9-10 9-10
11-12
6-8
1K
2
3
4
5
6
7
8
9-10
11-12
1K
2
3
4
5
6
7
8
9-10
11-12
K
1
2
3
4
5
9-10
11-12
6
7
8
Found-ational Skills
1
2
3
4
5
K → → → → → → → → → → →→ → → → → → → → → → →→ → → → → → → → → → →
→ → → → → → → → → → →
→ → → → → → → → → → →→ → → → → → → → → → →
→ → → → → →
→ → → → → →
→ → → → → →
→ → → → → →
→ → → → → →
→ → → → → →
40
English Language Arts and Literary Shifts
engageny
Smarter BalancedAssessment Consortium
The Assessment Challenge
How do we get from here...
...to here?
All studentsleave high
school college and career
ready
All studentsleave high
school college and career
ready
Common Core State
Standards specify K-12 expectations
for college and career
readiness
Common Core State
Standards specify K-12 expectations
for college and career
readiness...and what can an
assessment system do to help?
Next Generation Assessments
• More rigorous tests measuring student progress toward “college and career readiness”
• Have common, comparable scores across member states, and across consortia
• Provide achievement and growth information to help make better educational decisions and professional development opportunities
• Assess all students, except those with “significant cognitive disabilities”
• Administer online, with timely results
• Use multiple measures
Source: Federal Register / Vol. 75, No. 68 / Friday, April 9, 2010 pp. 18171-85
Smarter BalancedBackground
A National Consortium of States
• 28 states representing 44% of K-12 students
• 21 governing, 7 advisory states
• Washington state is fiscal agent
Smarter BalancedApproach
Seven Key Principles
1. An integrated system2. Evidence-based approach3. Teacher involvement4. State-led with transparent
governance5. Focus: improving teaching and
learning6. Actionable information – multiple
measures7. Established professional standards
A Balanced Assessment System
Common Core State Standards specify
K-12 expectatio
ns for college and
career readiness
Common Core State Standards specify
K-12 expectatio
ns for college and
career readiness
All students
leave high
school college
and career ready
All students
leave high
school college
and career ready
Teachers and schools have information and tools
they need to improve
teaching and learning
Interim assessments Flexible, open,
used for actionable feedback
Summative assessments
Benchmarked to college and career
readiness
Teacher resources for
formative assessment
practicesto improve instruction
A Balanced Assessment System
School Year Last 12 weeks of the year*
DIGITAL CLEARINGHOUSE of formative tools, processes and exemplars; released items and tasks; model curriculum units; educator training; professional development tools and resources; scorer training modules; and teacher collaboration tools.
English Language Arts and Mathematics, Grades 3-8 and High School
Computer Adaptive
Assessment andPerformance
Tasks
Computer Adaptive
Assessment andPerformance
TasksScope, sequence, number and timing of interim assessments locally determined
*Time windows may be adjusted based on results from the research agenda and final implementation decisions.
SummativePerformance
TasksFor
Accountability• Reading• Writing• Math Re-take
option
Summative EndOf Year AdaptiveAssessment forAccountability
Optional Interim
Assessment
Optional Interim
Assessment
Using Computer Adaptive Technology for Summative and
Interim Assessments
Technology
IT readiness survey: Available January 2012
IT readiness survey: Available January 2012
System architecture: Available January 2012
System architecture: Available January 2012
Vendors start building the system: February 2012
Vendors start building the system: February 2012
Improve the technology throughout pilot and field test: 2012 and 2013
Improve the technology throughout pilot and field test: 2012 and 2013
Technology
New Purchases (as of April 2012)– Hardware – 1GHz processor, 1 GB RAM, 9.5 inch
screen size (10 inch class), screen resolution of 1024 x 768
• Must have tools to temporarily disable features ( i.e.. web browser, Bluetooth connections, application switching)
– Operating Systems – Windows 7, Mac 10.7, Linux (Ubuntu 11.10, Fedora 16), Chrome, iOS, Android 4.0
• Will consider older versions and Linux after survey data and cognitive labs
– Network – Must be able to connect to the Internet
Technology
New Purchases (as of April 2012)
– Form Factors –Desktops, laptops, netbooks, thin-client, and tablets ) iPad, Windows and Android) that meet the above specifications
– Additional Accessories –• Headphones may be required for audio support• Physical keyboards (as opposed to virtual) and/or
mice may be required for use with tablets
Technology
Anticipate by 2016-17 up to 25% of the mathematics assessment will comprise items/tasks requiring student-produced illustrations or calculations.
Students will need access to tablet (or other device) that employs a stylus for user input
Future required use will be described in the August 2012 guidelines
Technology Plan
• What is your vision?
• Designing an infrastructure – CoSN, SETDA, Horizon report, Maine Learning Technology Initiative
• Define the type of learning you want for your students.
• Professional Development – Teachers, Principals, Technology Coordinators
57
2012 Horizon Time to Adoption
• One Year or Less:– Mobiles and Apps– Tablet Computing
58
2012 Horizon Time to Adoption• Two to Three Years:
– Game-Based Learning– Personal Learning Environments
Horizon Report 2012 44
Personal Learning Environments
Horizon Report 2012
2012 Horizon Time to Adoption
• Four to Five Years:– Augmented Reality– Natural User Interfaces
61Horizon Report 2012
Key Trends (Horizon Report 2012)• Abundance of resources• Cost of technology dropping, • More common for students to bring
their own device• Schools including online learning,
hybrid learning and collaborative models
62
Key Trends (Horizon Report 2012)• One to one computing spreading
across the world and US• People expect to work, learn, and
study whenever• Technology profoundly affects the
way we work, collaborate, communicate and succeed
• New emphasis on challenged-based and active learning
63
Challenges (Horizon Report 2012)
• Demand for personal learning is not supported by current technology
• Lack of digital media literacy for teachers• Economic pressures – new models of
education – unprecedented competition• Institutional barriers • Blending of formal and informal learning
64
Challenges (Horizon Report 2012)
• Learning that incorporates real life learning – is undervalued
• Many activities related to learning take place outside the walls of classrooms
• Putting 21st century technology into 19th century schools is a major undertaking
• We don’t use digital media for formative assessment
65
What is on your Horizon?
• Download the NMC 2012 Horizon Report
• www.CoSN.org/Horizon
66
Assessment System Components
Assessment System Components
• Optional comprehensive and content-cluster assessment to help identify specific needs of each student
• Can be administered throughout the year
• Provides clear examples of expected performance on Common Core standards
• Includes a variety of question types: selected response, short constructed response, extended constructed response, technology enhanced, and performance tasks
• Aligned to and reported on the same scale as the summative assessments
• Fully accessible for instruction and professional development
Assessment System Components
The use of
performance
measures has been
found
to increase the
intellectual challenge
in classrooms
and to support
higher-quality
teaching.
- Linda Darling-Hammond and Frank Adamson, Stanford University
“
”
Performance Task Guidelines
• Integrate knowledge and skills across multiple standards or strands
• Measure capacities such as depth of understanding, research skills, complex analysis, and identification/providing of relevant evidence
• Require student-initiated planning, management of information and ideas, interaction with other materials
70
Performance Task Guidelines
• Require production of more extended responses (e.g., oral presentations, exhibitions, product development, in addition to more extended written responses which might be revised and edited
•Reflect a real-world task and/or scenario-based
•Allow for multiple approaches
•Represent content that is relevant and meaningful to students
71
Performance Task Guidelines
• Allow for multiple points of view and interpretations
• Require scoring that focuses on the essence of the task
• Be feasible for the school/classroom environment
72
Performance Task Guidelines
• Allow for demonstration of important knowledge & skills, including those that address 21st century skills such as critically analyzing, synthesizing media texts
73
Grade 08 ELA Sample CR Item
• Assessment Target- 11. REASONING & EVALUATION: Apply reasoning and a range of textual evidence to justify inferences or interpret author’s presentation of information (author’s line of reasoning; point of view/purpose; relevance of evidence and/or elaboration to support claims, concepts, ideas)
• Standards: RI-6, RI-8, RST 6
• DOK - 3
Item Prompt
Based on the text, what inference can be made about how tests and testing should occur to ensure an accurate measurement of overall water quality? Explain your inference using details from the text.
Questions to be answered in the speech
•In two sentences, use your own words to tell what a wonder is and explain how a person who helps others can be considered a wonder. •Write 2 or 3 sentences identifying a personal quality that both Mickey and Ana display. Give an example from both the video and the interview to support your answer.
Questions to be answered in the speech
• Tell which website you think would be most useful for learning about another young person that is a wonder because he or she helps others. Cite the web site by giving the web address. Use details from the website to support your answer.
Performance Tasks Sources (2 articles, 3 videos)
Article 1 •Metro Daily News June, 2011
The Rise of the Robot Pet by Elena Soto
Video 1: Fugitsu’s cute teddy-bear robot shows what it can do, May 2010. http://www.youtube.com/watch?v=AwWeN1ARy74--This is an introduction to the Fujitsu robot teddy bear. (1:58)
Video 2: Pleo: Robot, pet or both? December 2007 http://www.youtube.com/watch?v=F6LCEFr8SxQ--The host of a technology show provides an introduction to a specific robot pet. (3:10)
Article 2 Technology Trends Quarterly Fall, 2010 Love In the Time of Robots by Frank Mullin
Video 3: Maya’s Human Interaction – Sensors (A Genibo-QD film), February 2010. http:// www.youtube.com/watch?v=9b4jx5RzqAk --The Genibo robot dog displays some of its capabilities. (1:49)
Scoring
1.Establishing a claim2.Presentation of relevant evidence3.Organization4.Conventions5.Language, tone, purpose, audience6.Content
Draft Assessment Claims for Mathematics (a/o Round 2 – released 12/9/11)
A Schematic representation of CCSSM content
“Hurdles Race”
Think of the Content involved
• Interpreting distance-time graphs in a real-world context
• Realizing “to the left” is faster
• Understanding points of intersection in that context (they’re tied at the moment)
• Interpreting the horizontal line segment
• Putting all this together in an explanation
Think of the Practices involved
• Make sense of problems and persevere in solving them.
• Reason abstractly and quantitatively.
• Construct viable arguments…
• Model with mathematics.
• Use appropriate tools strategically.
• Attend to Precision.
• Look for and make use of structure.
• Look for and express regularity in repeated reasoning.
Item Exemplars: Technology Enhanced and Constructed
Response
Item Exemplars: Technology Enhanced and Constructed
Response
Item Exemplars: Technology Enhanced and Constructed
Response
Few initiatives are
backed by evidence
that they raise
achievement.
Formative
assessment is one of
the few approaches
proven to make a
difference.- Stephanie Hirsh, Learning Forward
Assessment System Components
“
”
145
Definition:Assessment that takes place continuously during the course of teaching and learning to provide teachers and students with feedback to close the gap between current learning and desired goals.
Assessment Reform Group, 2002; Bell & Cowie, 2001; Black et al., 2003; Black & Wiliam, 1998; OECD, 2005; Sadler, 1989; Shepard, 2000)
Advantages of Formative Assessment
• Students learn faster• Teachers know what students
already know & adjust instruction
• Students aware of progress• Most powerful moderator in
student achievement• Works for at risk students
Formative Assessment Strategies
(Black, Wiliam,1998; Sadler, 1998; Stiggins, 2007;Heritage, 2007)• Pre-assessing students
• Sharing Learning goals with students
• Co-creating classroom discourse & questioning
• Rich & challenging tasks elicit student response
• Identifying gaps
Formative Assessment Strategies
(Black, Wiliam,1998; Sadler, 1998; Stiggins, 2007;Heritage, 2007)
• Providing feedback/how to improve
• Self-assessments
• Peer- assessments
• Opportunities to close the gap
• Celebrations
Feedback
Strategies
• Traffic Light feedback• Gallery Walk• Portfolio• Concept Map• Ticket out the door
What do students say
• Class discussion• Debate• Oral presentation• story/event telling• Agree/disagree• Choral reading• Think-Pair-Share• You’re the Judge• Ask a question• Make a Statement• Radio Show
• Small group talk• Play/drama• Reciting a
poem/speech• Panel discussion• Music• Interviews• Think aloud• Answer specific• Podcasts• Read aloud• Other____
Take a Three Minute Pulse
• After 10-15 discussion, reading, lecture• Reflect, discuss what they learned
using higher order thinking skills • Suggested questions: (Marzano)
– How does this information relate to you?– How does what we’ve just learned relate to..– How is what we just learned similar or different to– Identify one thing you knew and one thing that was
new to you…
Exit Sheet
One Minute Response
Grades Supported
Grades Summative Interim(Optional)
Formative Tools and Professional Learning
(Optional)
✔ ✔ ✔
1-2 Performance Tasks as Required
to Cover CCSS
✔
EOC and Comprehensive
✔
✔ ✔
EOC and Comprehensive
✔
Optional ✔
EOC and Comprehensive
✔
3 8
9 10
11
12
Data are only useful if
people are able to
access, understand
and use them… For
information to be
useful, it must be
timely, readily
available, and easy to
understand.
- Data Quality
Campaign
Assessment System Components
• Static and dynamic reports, secure and public views
• Individual states retain jurisdiction over access and appearance of online reports
• Dashboard gives parents, students, practitioners, and policymakers access to assessment information
• Graphical display of learning progression status (interim assessment)
• Feedback and evaluation mechanism provides surveys, open feedback, and vetting of materials
“
”
Support for Special Populations
• Accurate measures of progress for students with disabilities and English Language Learners
• Accessibility and Accommodations Work Group engaged throughout development
• Outreach and collaboration with relevant associations
Common-Core Tests to Have Built-inAccommodations
- June 8, 2011
“
”
Timeline
Formative Processes, Tools, and Practices Development Begins
Formative Processes, Tools, and Practices Development Begins
Writing and Review of Pilot
Items/Tasks (including
Cognitive Labs and Small-Scale Trials)
Writing and Review of Pilot
Items/Tasks (including
Cognitive Labs and Small-Scale Trials)
Field Testing of Summative and
Interim Items/Tasks Conducted
Field Testing of Summative and
Interim Items/Tasks Conducted
Content and Item
Specifications
Development
Content and Item
Specifications
Development
Pilot Testing of Summative and
Interim Items/Tasks Conducted
Pilot Testing of Summative and
Interim Items/Tasks Conducted
Preliminary Achievement
Standards (Summative) Proposed and Other
Policy Definitions Adopted
Preliminary Achievement
Standards (Summative) Proposed and Other
Policy Definitions Adopted
Operational Summative Assessment
Administered
Operational Summative Assessment
Administered
Procurement Plan
Developed
Procurement Plan
Developed
Writing and Review of Field Test Items/Tasks
(throughout the school year)
Writing and Review of Field Test Items/Tasks
(throughout the school year)
Final Achievement Standards
(Summative) Verified and
Adopted
Final Achievement Standards
(Summative) Verified and
Adopted
Common Core State Standards Adopted
by All Member States
Common Core State Standards Adopted
by All Member States
Summative Master Work Plan Developed and Work Groups
Launched
Summative Master Work Plan Developed and Work Groups
Launched
Find Out More
The Smarter Balanced Assessment Consortium
can be found online athttp://www.smarterbalanced.org