What Does NAEP Grade 8 Data on School Experience Show about Student Learning? Wendy Geiger VA NAEP...
-
Upload
amos-williamson -
Category
Documents
-
view
215 -
download
0
Transcript of What Does NAEP Grade 8 Data on School Experience Show about Student Learning? Wendy Geiger VA NAEP...
11
What Does What Does NAEP Grade 8 NAEP Grade 8 Data on School Data on School
Experience Show Experience Show about Student about Student
Learning?Learning?
Wendy GeigerVA NAEP Coordinator
Carrie L. GiovannoneAZ NAEP Coordinator
Pam A. SandovalCO NAEP Coordinator
Michele SonnenfeldFL NAEP Coordinator
NATIONAL CONFERENCE ON STUDENT ASSESSMENT
Los Angeles, CaliforniaJune 23, 2009
2222
Overview of PresentationOverview of Presentation
Grade 8 PerspectiveGrade 8 Perspective Use of Various NAEP data results from Use of Various NAEP data results from
students, teachers and schools tostudents, teachers and schools to– Show new findingsShow new findings– Support current researchSupport current research– Make connections across research studiesMake connections across research studies
Give suggestions on how to use the Give suggestions on how to use the NAEP background dataNAEP background data
NAEP Reading Grade 8 NAEP Reading Grade 8 Comparing 2003 to 2007Comparing 2003 to 2007
3311DoDEA is Department of Defense Education ActivityDoDEA is Department of Defense Education Activity
Media characterization of Media characterization of middle-school studentsmiddle-school students
Self-CenteredSelf-Centered Hormone-DrivenHormone-Driven ThoughtlessThoughtless Undisciplined (Kinney, 2007)
55
Middle-school students Middle-school students can also becan also be
ConcernedConcerned CaringCaring Compassionate Compassionate (Kinney,
2007)
66
1960s Perception1960s Perception The brain of a child ages The brain of a child ages
12 to 14 does not grow. 12 to 14 does not grow. Teaching complex Teaching complex
material during the material during the middle-school years will middle-school years will have a damaging effect. have a damaging effect.
Middle school is a time for Middle school is a time for social adjustment, social adjustment, individual growth, and individual growth, and coping with early coping with early adolescence, not adolescence, not academic learning or self-academic learning or self-discipline. This is known discipline. This is known as the Life-Adjustment as the Life-Adjustment Movement. Movement. (Yecke, 2005)(Yecke, 2005)
77
2121stst Century Perception Century Perception Focus must be on Focus must be on
students’ acquisition of students’ acquisition of essential academic skills essential academic skills and knowledge. and knowledge. (ACT, 2008)
Talent can be practical, creative, communicative, enterprising, as well as analytical intelligence. (Prime Minster Brown, 2009)
Students must be instilled with a love of learning for its own sake. (Rose, 2009)
88
99
The Forgotten Middle
Middle schools should be dedicated to Middle schools should be dedicated to the the
goals of high academic standards and goals of high academic standards and achievement, challenging curriculum, achievement, challenging curriculum,
results-based accountability, and results-based accountability, and sound discipline. sound discipline. (ACT, 2008)
10101010
“…“…eighth-grade achievement is the eighth-grade achievement is the best predictor of students’ ultimate best predictor of students’ ultimate
level level of college and career readiness by of college and career readiness by high school graduationhigh school graduation–even more –even more than students’ family background, than students’ family background,
high school coursework, or high school coursework, or high school grade point average.”high school grade point average.”
-The Forgotten Middle(ACT, 2008)
What are the students What are the students saying?saying?
What students said:What students said: 92% indicated they would 92% indicated they would
definitely or probably attend definitely or probably attend college.college.
93% said there was no chance 93% said there was no chance they would drop out of high they would drop out of high school and not graduate.school and not graduate.
What really happens:What really happens: Only Only 66% begin college directly 66% begin college directly
after high school.after high school. Only 83% actually graduate Only 83% actually graduate
from high school. from high school. (NASSP/PDK, 2007)
1111
Why do Grade 8 students Why do Grade 8 students who say they will attend who say they will attend
college college fail to enroll?fail to enroll?They do not:They do not:
have the knowledge and skills needed to have the knowledge and skills needed to enter and succeed in high school;enter and succeed in high school;
have the academic and behavioral have the academic and behavioral discipline to be successful in high school; discipline to be successful in high school;
know which courses are required for know which courses are required for college entry;college entry;
have access to adequate funds to pay for have access to adequate funds to pay for college;college;
think the university is for them; have encouragement from home. (ACT,
2008)1212
13131313
What Does the NAEP Data Say What Does the NAEP Data Say about Grade 8 Student Learning?about Grade 8 Student Learning?
Main NAEP
Long Term Trend
High School Transcript StudyQuestionnaire Data
Average Scale Scores
Percentages
15151515
NAEP Student NAEP Student Questionnaires Questionnaires
Completed by students Completed by students and used to collect and used to collect background information.background information.
Collected information on Collected information on students' demographic students' demographic characteristics, characteristics, classroom experiences, classroom experiences, and educational support.and educational support.
16161616
NAEP Teacher NAEP Teacher Questionnaires Questionnaires
Completed by teachers and used to Completed by teachers and used to collect background information.collect background information.
Collected data on teachers’ Collected data on teachers’ background, training, and classroom-background, training, and classroom-by-classroom information.by-classroom information.
17171717
NAEP School Questionnaires NAEP School Questionnaires
Completed by school principal or other head Completed by school principal or other head administrator.administrator.
Collected information about school policies that Collected information about school policies that relate to or describe characteristics of schools.relate to or describe characteristics of schools.
Investigated the relationship Investigated the relationship between student achievement between student achievement and school factors that may and school factors that may influence achievement.influence achievement.
17171717
1818
Uses for the NAEP Background Data
Gives us “rich” data that can be linked to students’ NAEP scores
Can be corroborated with other research
Can reveal areas of interest for further research
1919
Limitations of the NAEP Background Data
Does not determine causality Must keep in mind the standard
errors Many explanations can play a role –
only used to raise “red flags” in the data
2020
““How hard was this test compared to most How hard was this test compared to most other tests you have taken this year in other tests you have taken this year in school?”school?”
Students chose from the following options:Students chose from the following options: Easier than other testsEasier than other tests About as hard as other testsAbout as hard as other tests Harder than other testsHarder than other tests Much harder than other testsMuch harder than other tests
2020
Affective Disposition/Affective Disposition/Difficulty of Assessment Difficulty of Assessment
2020
Student Questionnaire, Grade 8 NAEP 2007 Reading
Difficulty of AssessmentDifficulty of Assessment
2121
The data verifies that the students’ answers on the background questionnaires are valid.
These data show that the easier the test was perceived to be by the students, the higher their scale scores.
Student Questionnaire, Grade 8 NAEP 2007 Reading
Easier than
others
As hard as
others
Harder than
others
Much harder
62% 30% 6% 2%
t
Importance of Success on Importance of Success on NAEP Reading AssessmentNAEP Reading Assessment
As students placed greater importance on the NAEP test, the lower their scale scores.
Not Very Importan
t
Somewhat Important
Important Very important
16% 34% 30% 20%
2222
Student Questionnaire, Grade 8 NAEP 2007 Reading
23232323
Difficulty of this reading test
Importance of success on this reading test
Easier than others As hard as others
Not very important 263 259
Somewhat important 268 264
Important 266 262
Very important 261 256
The majority of the students are in the “easier than others” and “somewhat important”/”important” cross-tab cells.
The NAEP data shows that the more value the students placed on the test, the lower they scored.
Cross Tab - Difficulty and Cross Tab - Difficulty and ImportanceImportance
Student Questionnaire, Grade 8 NAEP 2007 Reading
2424
Teachers Play a Role in Student Learning
Teachers help Develop a students character and mind Instill an ambition to achieve and a love of
life-long learning Offer friendship to the students Build student confidence, resilience, and
determination Encourage students to plan ahead, work with
others, and stay on task Emphasize the importance of healthy living
25252525
““Teachers who can Teachers who can create a climate for create a climate for learning and focus on learning and focus on relationship-building relationship-building with their students with their students will encourage their will encourage their students to reach students to reach their future goals and their future goals and dreams.” dreams.” (ICLE, 2008)
““A curriculum is only A curriculum is only as good as those who as good as those who teach it.” teach it.” (Alexander, 2009)
2525
Pages Read in School and Pages Read in School and for Homeworkfor Homework
29% of Grade 8 students selected to take the NAEP Reading assessment reported that their teachers only asked them to read 5 or fewer pages in school and for homework.
The more pages Grade 8 students are asked to read for school or homework (up to 20 pages), the higher their NAEP Reading test score.
Percentage
*
*Indicates significantly lower than other categories
Student Questionnaire, Grade 8 NAEP 2007 Reading
*
2626
27272727
The 15 Key Elements of EffectiveThe 15 Key Elements of EffectiveAdolescent Literacy ProgramsAdolescent Literacy Programs
(Biancarosa & Snow, 2006)
What does the NAEP data tell us about 4 out of the 15 elements?
Direct, Explicit Comprehension InstructionDirect, Explicit Comprehension Instruction Effective adolescent literacy interventions must address Effective adolescent literacy interventions must address
reading comprehensionreading comprehension
Effective Instructional Principles Embedded in Effective Instructional Principles Embedded in ContentContent
Motivation and Self-Directed LearningMotivation and Self-Directed Learning Students choose the books they want to read in schoolStudents choose the books they want to read in school
Extended Time for LiteracyExtended Time for Literacy Need 2 to 4 hours of literacy-connected learning dailyNeed 2 to 4 hours of literacy-connected learning daily
Direct, Explicit Comprehension Direct, Explicit Comprehension InstructionInstruction
Language Arts Classes Most-Advanced Students Least-Advanced Students
Ask students to explain or support their understanding of what they have read
Those students asked to do this task “more than once a week” scored higher than those asked to explain what they read “only once a month”
No Difference in Scores
Ask students to make generalizations and draw inferences based on what they have read
No Difference in Scores Students that were asked to complete this task at least “once a month” scored higher than those “never or hardly ever” asked to make generalizations based on what they read
Ask students to describe the style or structure of the text they read
Students that were asked to complete this task at least “once a month” scored higher than those “never or hardly ever” asked to describe the style of the text they read
Teacher Questionnaire, Grade 8 NAEP 2007 Reading
2828
Teacher Questionnaire, Grade 8 NAEP 2007 Reading
Language arts is taught primarily as a discrete subject with little or no integration with instruction in other subjects.
Some language arts instruction is integrated with other subjects, and some language arts instruction is presented as a discrete subject [combined].
Language arts lessons are primarily integrated with instruction in other subjects.
Effective Instructional Principles Effective Instructional Principles Embedded in ContentEmbedded in Content
2929
“Which best describes how language arts instruction is organized
for 8th grade students at this school?” (Choose only one option.)
Discrete Combined Integrated
25% 65%* 10%*Significantly higher than the other categories
*
Teacher Questionnaire, Grade 8 NAEP 2007 Reading
Effective Instructional Principles Embedded in Content
Language Arts Class Organization
3030
NAEP 2007 data shows that 65% of Grade 8 students selected to take NAEP Reading were in “Combined” language arts classes.
*
*Significantly higher than “Integrated”
The students in “Integrated” courses scored lower than their peers.
**
**
Motivation and Self-Directed Motivation and Self-Directed LearningLearning
Never or HardlyEver
Once or Twice/Month
Once or Twice/Week
Almost Every Day
15%* 22% 31% 32%
NAEP 2007 data shows that “least advanced” Grade 8 students scored significantly lower when asked to read “almost every day” even though they chose their books.
There is no difference in the average scale scores of those who are given time in class to read books they chose when asked to read “once or twice a month” or “once or twice a week.”
Teacher Questionnaire, Grade 8 NAEP 2007 Reading
3131
*Significantly different from other categories
““Most advanced” Most advanced” students had similar students had similar results for average results for average scale scores.scale scores.
Total time teacher spends with one Grade 8 Language Arts class in typical week.
<3 hrs 3 - 4.9 hrs
5 - 6.9 hrs
7 - 9.9 hrs
10 hrs +
2% 48% 31% 15% 4%
Extended Time for LiteracyExtended Time for LiteracyTeacher Questionnaire, Grade 8 NAEP 2007 Reading
3232
Extended Time for LiteracyExtended Time for LiteracyTeacher Questionnaire, Grade 8 NAEP 2007 Reading
Reading Skills & Strategies
Appreciation & Analysis of Literature
Student Writing
Breakdown of
Language Arts
Instruction time
1-10 Percent
9% 11% 7%
11-40 Percent
46% 49% 47%
41-60 Percent
25% 26% 27%
61-90 Percent
13% 10% 13%
91+ Percent
6% 4% 5%
~100% 100% ~100%
255
263
265
264
262
269
265
259
257
255
257
263
263
262
261
3333
3434
How to Use the NAEP Data
Within State Departments To Support State Initiatives for Grant
Proposals For Teacher Professional Development To Guide Policy Development As an Integral Part of the Overall State
Assessment Program
3535
Opportunities in the 21st Century
“Education is no longer just a pathway to opportunity,
it’s a prerequisite for success.”
President Barack Obama’s speech to a joint session of Congress
February 24, 2009http://www.cbsnews.com/stories/2009/02/24/politics/
main4826494.shtml
36363636
ReferencesReferencesNAEP Data Explorer: http://nces.ed.gov/nationsreportcard/naepdata/
NAEP Questionnaires: http://nces.ed.gov/nationsreportcard/bgquest.asp
________________________________________________________________________ ACT. (2008). The forgotten middle: Ensuring that all students are on target for college and
career readiness before high school. Iowa City: ACT. Alexander, R.J. (2009) Towards a New Primary Curriculum: a report from the Cambridge
Primary Review. Part 2: The Future. Cambridge: University of Cambridge Faculty of Education.
Biancarosa, C., & Snow, C. E. (2006). Reading next: A vision for action and research in middle and high school literacy. (2nd Ed.). Washington, D.C.: Alliance for Excellent Education.
International Center for Leadership in Education. (2008). Florida middle school mathematics initiative, institute 6. Elementary-Middle School Transition Student Engagement Initiative in Review.
Kinney, P. (October, 2007). A voice from the middle: Middle level students care about their education and their future. Principal Leadership, 35-36.
National Association of Secondary School Principals & Phi Delta Kappa. (2007). A voice from the middle: Highlights of the 2007 NASSP/PDK middle school student poll. Bloomington, IN: NASSP/PDK.
Prime Minister Brown. (2009). PM’s speech on education for the new global age. Retrieved from the internet June 18, 2009 http://www.number10.gov.uk/Page19209
Rose, J. (2009). Independent review of the primary curriculum: Final report. Retrieved from the internet June 18, 2009 http://publications.teachernet.gov.uk/default.aspx?PageFunction=
productdetails&PageMode=publications&ProductId=DCSF-00499-2009& Yecke, C. P. (2005). Mayhem in the middle: How middle schools have failed America and
how to make them work. Washington, D.C.: Thomas B. Fordham Institute.
37373737
Thank you!Thank you!
Wendy GeigerWendy GeigerVirginia NAEP CoordinatorVirginia NAEP Coordinator
Carrie L. GiovannoneCarrie L. GiovannoneArizona NAEP CoordinatorArizona NAEP Coordinator
Pam A. SandovalPam A. SandovalColorado NAEP CoordinatorColorado NAEP Coordinator
Michele SonnenfeldMichele SonnenfeldFlorida NAEP CoordinatorFlorida NAEP Coordinator
Contact your State NAEP Coordinator for the latest NAEP results