Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational...
-
Upload
francis-francis -
Category
Documents
-
view
215 -
download
0
Transcript of Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational...
![Page 1: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/1.jpg)
Navigating the Next Generation of Assessments
![Page 2: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/2.jpg)
Dr. Marianne PerieCo-Director Center for Educational Testing University of Kansas
Dr. Scott SmithDirector of Standards and AssessmentsKansas State Department of education
Lee Jones Assessment CoordinatorKansas State Department of Education
Mark StephensonAssessment
![Page 3: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/3.jpg)
Everything is changing
SMARTER BALANCED ASSESSMENT CONSORTIUM (SBAC)
WISER HARMONIOUS ASSESSMENT OF KANSAS (WHAK)
(We are taking suggestions on naming this new assessment!)
![Page 4: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/4.jpg)
Transition assessment in Spring 2014 is the same as we have always planned
Spring of 2015, when we would have had SBAC for the first time, we will have an enhanced Kansas assessment.
By Spring of 2016, we will be very close to where we would have been with SBAC with difference due to choices made in Kansas.
But how much is changing, really?
![Page 5: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/5.jpg)
Transition Assessment Delivered on KITE Aligned with Kansas College and
Career Ready Standards for math, reading, and writing conventions
Follows similar blueprint to SBAC, including technology-enhanced items
Machine-scorable items only No performance task No writing prompt No listening items
Spring 2014
![Page 6: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/6.jpg)
The transitional test will serve as the backbone for the new Kansas College and Career Ready Assessment
Enhancements Listening section Writing prompt Math performance tasks
New Development in 2014–2015
![Page 7: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/7.jpg)
The core machine-scorable part of the test will be parallel to what was administered in 2014, but we will refresh the item pool.
We will field test the enhancements. Accountability for 2015 will be based on the
core portion of the test. Schools and districts will receive feedback on
the field-test portions to help gauge student readiness on the full set of standards.
Spring 2015
![Page 8: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/8.jpg)
After analyzing the field-test items, we will create the best form possible that includes all features of our future assessment.
We will set new achievement standards (cut scores and performance level descriptors) based on that form.
Using those new cut scores, KSDE will set new AMOs and communicate the new targets to schools and districts.
Summer 2015
![Page 9: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/9.jpg)
Now we have a fully enhanced test that covers all of the Kansas College and Career Ready Standards.
We should also have sufficient numbers of items to make the test adaptive this year as well.
Spring 2016
![Page 10: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/10.jpg)
The test will be adaptive to allow students to be measured on items close to their ability level.
We are considering stage adaptivity rather than item adaptivity that SBAC uses.
Benefits More reliable estimates Targets assessment to student level Requires fewer items than item-level
Adaptivity
![Page 11: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/11.jpg)
Model of a Stage Adaptive Test
![Page 12: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/12.jpg)
Summary
Core machine-scorable items aligned with KCCRS
Core machine-scorable items aligned with KCCRS
Core machine-scorable items aligned with KCCRS
Spring 2014
Spring 2015
Spring 2016
Listening Items
Complete enhancements (performance task, essay,
listening items)
EssayMath
Perform-ance Task
Adaptivity
![Page 13: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/13.jpg)
Adaptivity should help students who used to take the KAMM. They will be assessed on grade-level, but with easy items.
Many tools are available to all students (e.g., highlighter, notes, calculator)
Accommodations are available electronically (contrast, auto font)
TTS is available in a new and improved voice this year.
Special Education
![Page 14: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/14.jpg)
We want Kansas educator and stakeholder fingerprints all over these new assessments
Design decisions Item writing Item reviewing Range finding Scoring Standard setting
Kansas Fingerprints
![Page 15: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/15.jpg)
Administrators Support teacher involvement Voice in report decisions Cross-content development Kansas Board of Regents High school — we want to ensure that
tests truly predict college readiness Will be involved in design and review of
assessments and setting cut scores
Kansas Stakeholders
![Page 16: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/16.jpg)
Summative assessment ELA (complete in 2016) Math (complete in 2016) History/government (complete in 2016) Science (complete in 2017)
Formative tools Really more like sample items now Developing capacity for teachers to build test
forms using item pool In 2016, we can start building true formative,
instructionally-embedded tests. Will include science and H/G
Full Kansas Assessment Program
![Page 17: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/17.jpg)
Essay Covers both ELA and Science or
History/Government (HG) standards Students in grades where science/HG
is assessed would only need to write one essay aligned to both science/HG and literacy standards
Two rubrics provide two scores—one for each test
Things to Consider
![Page 18: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/18.jpg)
Students watch “I have a dream” and read an essay by Malcolm X. They are then asked to write an essay comparing and contrasting the two approaches to the civil rights movement. They may be asked to define and give examples of civil disobedience and discuss its efficacy.
HS Example
![Page 19: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/19.jpg)
Write informative/explanatory texts, including the narration of historical events, scientific procedures/ experiments, or technical processes.
a. Introduce a topic clearly, previewing what is to follow; organize ideas, concepts, and information, using strategies such as definition, classification, comparison/contrast, and cause/effect; include formatting (e.g., headings), graphics (e.g., charts, tables), and multimedia when useful to aiding comprehension.
b. Develop the topic with relevant facts, definitions, concrete details, quotations, or other information and examples.
c. Use appropriate transitions to create cohesion and clarify the relationships among ideas and concepts.
d. Use precise language and domain-specific vocabulary to inform about or explain the topic.
e. Establish and maintain a formal style and objective tone.f. Provide a concluding statement or section that follows from
and supports the information or explanation presented.
Writing Standards (G11)
![Page 20: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/20.jpg)
High school: Civil rights, social change Standard # 3: Societies are shaped by beliefs, ideas,
and diversity. 3.1 The student will recognize and evaluate
significant beliefs, contributions, and ideas of the many diverse peoples and groups and their impact on individuals, communities, states, and nations.
3.2 The student will draw conclusions about significant beliefs, contributions, and ideas, analyzing the origins and context under which these competing ideals were reached and the multiple perspectives from which they come.
Kansas History/Government Standards
![Page 21: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/21.jpg)
Adaptivity in writing prompt and performance task.
SBAC planned to have writing prompts and performance tasks written to three different levels of difficulty. The first part of the test would determine which writing prompt/performance task the student received.
Should we do the same? Would two levels be sufficient?
Things to consider …
![Page 22: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/22.jpg)
Scoring With a writing prompt and math
performance task, we will have student responses that need to be scored by hand.
We would like teachers to be involved in scoring.
What is the best model?
Things to consider …
![Page 23: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/23.jpg)
Fully distributed scoring Teachers score on the computer using
KITE. An independent activity. School or regional based scoring
Teachers score as a group and enter scores later.
Scoring center Only a few teachers come to Lawrence for
several days to score all student papers.
Scoring models
![Page 24: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/24.jpg)
Weighting the essay and performance task What are your thoughts as to how
much weight the essay should have on the ELA score or the math performance task should have on the math test?
MC items are worth one point each. TE items can be worth up to 3 points each. Math core is worth 60 points ELA core is worth 57-71 points,
depending on the grade.
Things to consider …
![Page 25: Navigating the Next Generation of Assessments. Dr. Marianne Perie Co-Director Center for Educational Testing University of Kansas Dr. Scott Smith Director.](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649cf45503460f949c2fae/html5/thumbnails/25.jpg)
KITE Help Desk [email protected] Phone: 785.864.3537 Toll Free: 855.277.9752
Marianne Perie [email protected]
Contact Information