The National Monitoring Study of Student Achievement (NMSSA) – Wānangatia te Putanga Tauira is an innovative new programme to assess and understand student achievement across the New Zealand Curriculum at the primary level in New Zealand.

PURPOSE

The purpose is to monitor change in achievement over time at the national level. The programme does not provide information about individual students, schools or teachers. It gathers data from representative samples of students in Years 4 and 8, over a rigorously designed four-year cycle that covers all areas of the New Zealand Curriculum, including the key competencies and the foundation areas of literacy and mathematics. Data collection for 2012 has just begun. The main focus is Science and English: writing.

In addition, the design of NMSSA recognises that students’ learning also takes place in and is influenced by, their family, whānau and community contexts and the school they attend. An important part of the programme involves gathering information on the contextual factors in students’ learning experiences that influence achievement, including attitudes, motivation and opportunities to learn.

Recognition of the crucial importance of students’ identity, language and culture is fundamental to the philosophy and practices of NMSSA The programme will pay specific attention to how Māori, Pasifika and students with special learning needs are succeeding and will seek to better understand the factors that influence student achievement for those key population groups. The high quality data coming from NMSSA will help the Government to monitor the impact of its strategies in Ka Hikitia; Pasifika Education Plan and Success for All - Every School, Every Child. NMSSA will also contribute to understanding the achievement of children from low socio-economic backgrounds.

National monitoring will provide schools, parents, whānau, policy makers and the wider public with robust data, along with a real sense of how well New Zealand primary level students are achieving over time and within key population groups. The data will complement the information available through international studies such as TIMSS and PIRLS.

THE NAME

The Māori title, Wānangatia te putanga tauira, is based on the karakia phrase Mānawatia te putanga tauira, which is about celebrating student success. Wānangatia te putanga tauira is about studying, considering, analysing student success and achievement.

PRINCIPLES

Key ideas and principles have guided the design and development of NMSSA.

Trustworthy informationNational monitoring will provide an on-going source of trusted information. A broad range of knowledge, skills, key competencies, literacy and mathematics across the curriculum are assessed to give a full picture of what students know and can do. The programme will provide high-quality data to inform policy and practice and deepen public understanding of New Zealand student progress. It uses a low-stakes, light-sampling approach which will result in information that is highly dependable.

Identity, language and culture are centralNational monitoring recognises the diverse nature of students and acknowledges the central role that students’ identity, language and culture play in their learning. A range of assessment tasks and approaches are designed to be meaningful and enjoyable for the students, to help to gain a rich picture of their capabilities.

W ā n a n g a t i a t e P u t a n g a T a u i r a

National Monitoring Study of Student Achievement

Understanding student achievementNational monitoring is designed to gather data not only on student achievement but also on the contextual factors in students’ experiences at school, and in the home, whānau and community that influence their education outcomes. Information will be gathered on motivation and attitudes, learning experiences and learning environments, drawing on the perspectives of students, teachers and principals.

Involving teachersTeachers are involved in the development, trialling and administration of tasks, and in the marking of student responses. This means that national monitoring is grounded in good teaching practice and that professional skills developed during the study are carried back into schools. In reporting the data, the study will use dissemination processes that will enhance the assessment capabilities of New Zealand educators at all levels.

Exemplifying excellence NMSSA will exemplify best practice in both formative and summative assessment practice across the curriculum. The programme aims to lead the way in the development of new approaches to assessment.

WHAT WILL BE ASSESSED: THE FOUR-YEAR CYCLE

WHO IS INVOLVED WITH NMSSA? NMSSA is a Ministry of Education initiative, run on contract by the Educational Assessment Research Unit (EARU) of Otago University in a collaboration with the New Zealand Council for Educational Research (NZCER). Associate Professor Alison Gilmore is the project director.

The study is overseen by a Ministry of Education steering group and study board. It is supported by Māori, Pasifika, Special Education and technical reference groups and curriculum/contextual advisory panels.

During the cycle, the study will also gather contextual information to allow a better understanding of student achievement. The information will come from question-naires and sometimes interviews with students, teachers and principals.

Some information is collected annually in relation to specific areas of the curriculum, e.g. questions about opportunities for learning and self-efficacy. Other areas are more generic, such as the climate for learning. Other contextual factors will be assessed only once during the cycle, such as questions about the role of information technology in learning, or to gain the perspectives of parents and whānau.

National monitoring also includes a programme of work called innovation and special studies. The focus of these is decided annually with the Ministry of Education. They include studies that enhance the design of NMSSA, explore national monitoring findings that merit further investigation, and enable us to collect in-depth monitoring information for sub-groups of the population.

There will be some flexibility about what is explored within a given cycle, while at the same time retaining consistency across cycles in order to examine trends over time. NMSSA will be a programme that responds in subsequent cycles to what has been learned, and that works collaboratively with the Ministry of Education and the education community at large, to ensure it produces outcomes that are of critical importance to New Zealand’s educational enterprise.

The programme will assess students in Year 4 and Year 8 of schooling. The four-year cycle has been designed so that it:

•Coversallthelearningareasincludingastrongfocusonthekey competencies

•Regularlyassessesthefoundationlearningareasofliteracyand mathematics

•Combinesthelearningareasandthetypesofassessmentsused to optimise the information gained each year

• Ismanageableintermsofthedemandsitplacesonschools

•Ensuresstudentstakingparthaveapositiveexperience

Year of cycle Assessment Programme

Year 1

2012

Science

English: writing *

Year 2

2013

Physical Education and Health

Mathematics and Statistics

Year 3

2014

Social Studies/Technology

English: reading, listening and viewing

Year 4

2015

The Arts

Re-analysis of data, special reporting, reflection and forward planning

* In 2016, English would include speaking and presenting

NMSSA builds on the strengths of the National Educational Monitoring Project - well-known to teachers as NEMP - which ended in 2010.

THE APPROACH: A RANGE OF ASSESSMENTS

There are two components: survey assessments and in-depth assessments. Together they will provide a comprehensive and rich description of student achievement, strengths and weaknesses, and progress observed at the system level.

Survey assessments will use approaches that provide a strong base for advanced psychometric and statistical analysis. Survey assessments (pen and paper) will be more appropriate for some areas of the curriculum, such as Mathematics.

The survey assessment approach will also be used for gathering contextual information such as attitude, self-efficacy, opportunities for learning, and support for learning.

In-depth assessments will be conducted in one-to-one interviews with trained assessors and in individualised or team performance assessments.

In-depth assessments will be used to assess achievement in all areas and are particularly important in Physical Education and the Arts, as well as aspects of the curriculum that cannot be thoroughly assessed using survey-only assessment approaches.

In-depth approaches enable us to probe more deeply into areas where children are having difficulty with particular concepts or skills and will lead to ideas about the sort of assistance that can be offered.

The role of teacher assessors

The assessments will be administered by trained teacher assessors who will be seconded from their teaching roles for one school term. This will ensure the programme is able to:

• Obtaindataofconsistentlyhighquality

• Avoid being too onerous on schools in terms ofadministration

• Strongly connect to schools through the teacherassessors

The teacher assessor role will also be a positive professional learning and development opportunity for teachers involved. The programme will select high-quality teachers who have a strong cultural awareness. They undertake a four-day training course; and work in pairs to visit each school. The assessment schedule in schools seeks to ensure that good rapport is established with the students, and that students have a positive experience.

Which schools and students?Approximately 5,000 students in 200 schools are randomly selected to take part in national monitoring. Up to 25 students at each school will be selected, with eight of those students going on to participate in the in-depth study.The support of the selected schools and the parents of selected students is sought.What is assessed?Writing and science in 2012 and over the next four years other areas will include mathematics and statistics, health and physical education, the arts, listening, reading, speaking, technology and social sciences, as well as key competencies and literacy across the curriculum.Who administers the tasks?The tasks are administered by teachers, who are seconded from schools for four days of training and then spend nine weeks administering the tasks in the selected schools.How are the assessments conducted?Assessment is carried out over 2½ days. With the support of trained teacher assessors, students work on tasks in three different ways:• Paper-and-pencil assessments and questionnaires about their

learning experiences

• Performancetasks(independentlyandingroupsoffour)

• Interviewtasks,video-tapedtoenabledetailedanalysis

FURTHER INFORMATION

For further information contact:

The Educational Assessment Research Unit

University of Otago

Box 56

Dunedin 9054

New Zealand

Free phone: 0800 808 561

Fax: 03 479 7550

Email: nmssa@otago.ac.nz

Web: http://earu.otago.ac.nz

July 2012

REPORTING NMSSA DATA

The psychometric approach that underpins NMSSA allows achievement for Year 4 and Year 8 students to be reported on the same scale and for progress in achievement from Year 4 to Year 8 across the curriculum to be analysed. It also enables analysis of student progress from one four-year cycle to another. Differences in achievement and attitudes between sub-groups (e.g., boys and girls, ethnic/cultural groupings, Year 4 and Year 8) will be examined with effect sizes which can be related to estimated levels of growth or difference.

The reporting formats will be targeted to a range of audiences and will provide an overall national picture of:

•Students’ achievement and what they bring to theirlearning (attitudes, beliefs and motivation)

•What the school provides for students’ learning fromteachers and principals of the selected students

•The performances of population sub-groups (suchas, Māori, Pasifika, boys and girls, and school decile or region)

•Thefactorsthatinfluencestudentachievement

The communication of results will provide concrete exemplars of what is meant by various levels of achievement. It will emphasise growth and/or change at the system level as well as the current performance level.

REPORTING TO THE EDUCATION SECTOR

As well as giving the sector access to the overall findings, the aim is to encourage use of the assessment resources and processes by classroom teachers, and to provide material and opportunities for teachers’ professional learning in assessment.

Reporting will include:

•Awebsitecontainingallreports.Thiswillincludepodcasts of key messages that can be used in schools, at board of trustee meetings, parent community meetings, and by the wider public

•AnannualpublicationinEducation Gazette that highlights key findings and implications for practice

•Presentationsatprofessionalmeetings

•Releaseofassessmenttasksforuseinschoolselectronically (website and DVD)

•Abiannualworkshopwithanationalandinternationalfocus

The results for science and writing will be available in Term 3, 2013.