Early Childhood Education StudyProgress Report

Juliane Hencke, Caroline Sharp, Tony Bertram, Chris PascalIEA Standing Committee – 5 October 2013

2

Content of Presentation• Study Overview• Study Management• Summary of 1st NRC Meeting• Conceptual Diagram• Sampling Design• Research Questions• Instruments• Study Milestones and Next Steps

3

Study Overview

• ECES is a cross-national study… – Exploring, describing, and analysing provision of early

childhood education (ECE) – Analyzing how ECE contributes to children’s outcomes

• Aims to…– Provide a framework for countries to benchmark their ECE

systems in an international context– Examine the relationships between policy aims, provision,

staffing, parental satisfaction and child outcomes– Investigate the relationships between characteristics of

ECE and children’s competencies

4

What do we provide with ECES?• Country profiles in terms of system characteristics• Descriptives showing what are the characteristics of

ECE in each country on different levels (system, ECE settings, practitioners, parents, children)

• Once we understand the characteristics, what are the relationships between the different levels within and across countries

• What are the outcome differences in terms of different provider types and how do they relate to the outlined policies

• Identify approaches to facilitate these variations

5

What makes ECES unique?• Diversity of countries• Covers different levels of ECE:

intended - implemented - achieved policies• Covers diverse consumer perspectives (settings,

practitioners, parents)• Provide various outcome measures based on an

international framework

• Examines key structural elements from provision to outcome including policy, providers, parents and children

6

Study Management• IEA Data Processing and Research Center (IEA DPC):

Study coordination, practitioner questionnaire, sampling, data processing, analysis

• National Foundation for Educational Research (NFER): Study framework, ECE setting and home questionnaire

• Centre for Research in Early Childhood (CREC): Policy questionnaire and child assessment

• IEA Secretariat: Translation verification, quality monitoring

7

Study Management• Joint Management Committee (JMC)– Regular meetings of international study consortium

• Project Advisory Committee (PAC)– Expert advice on framework refinement, instrument

development and reporting

• National Research Coordinators (NRCs)– Regular meetings to contribute to and to review key

elements of the study

8

Participants at the 1st NRC Meeting

Austria Denmark NorwayBahrain England PolandBelgium (Flemish) Estonia Russian FederationBrazil Germany SloveniaCanada Hong Kong SpainChile Italy ThailandCroatia Lithuania Turkey (did not show up)Czech Republic New Zealand USA (online)

9

Summary of 1st NRC Meeting• Child outcomes are the aspect of greatest interest to

country representatives• Framework needs to be re-organized, so that the

assessment module is the focus of the study • Research questions need to more directed to outcome

measures• There should be a greater focus on pedagogic practice• Interest was expressed in administering the

assessment to younger children (as national option?)

10

Conceptual Diagram

National/ Regional/ Local Policies

ECE Setting• Structural characteristics• Leadership and quality

processes

Practitioners• Practitioner

characteristics • Process and

engagement

Family and Child• Characteristics • Home Learning

Environment

Child Outcomes1. Social and emotional

2. Dispositional and Learning

3. Physical

4. Language and literacy

5. Numeracy

Parental Satisfaction

Phase 1

Phase 2

Sampling Design

ECE Settings of focused children (200)

Practitioners of focused children

(15 or all per setting; about 1,500)

Focus: Children attending center-based education and care in the year before entering ISCED 1

Child Assessment

Children (15 or all per setting;

minimum: 3,000)

Policy Questionnaire (1)

Parents/Carers of sampled children

Ideally: PSU, first sampling

stage

Second sampling

stage

12

ECES Research QuestionsStudy focus is on how ECE contributes to children’s outcomes 1. What variations exist between and within countries in

outcomes for children attending ECE?2. What aspects of ECE settings and education systems are

related to outcomes for children’s early learning and development?

a. System context, policy aims and QA for ECEb. ECE setting characteristics, aims and resourcesc. ECE staff qualifications and CPDd. ECE staff practices in working with childrene. ECE setting leadership and QA systemsf. Communication with and involvement of families

13

ECES Research Questions3. What aspects of children’s personal and social

background and family context are related to children's outcomes?

a. To what extent do these relationships differ within and between countries?

b. Are there particular aspects of the ECE system or setting that mediate these relationships?

14

ECES Research Questions4. What is the relationship between parental satisfaction

with ECE and key features of the setting?1. How is parental satisfaction with ECE related to family and

child characteristics?2. How is parental satisfaction with their child’s ECE setting

related to child outcomes?

5. To what extent do ECE systems differ in their availability, reach and inclusion of children in the period between birth and ISCED 1?

15

Instruments

Phase 1• Policy QuestionnairePhase 2• Assessment module evaluating different aspects of

children’s development and learning– Practitioner ratings (?)– Assessment tasks

• ECES includes comprehensive set of questionnaires– ECE Setting Questionnaire– Practitioner Questionnaire– Home Questionnaire

16

Policy Questionnaire

• To capture evidence on the wider policy context for ECE in each participating country from birth to age of compulsory schooling.

• Completed by NRC (or country designate) to provide robust national level data (qualitative and quantitative) which will address the study research questions.

17

Identification of Key Learning Domains, Sub-Domains and Indicators• Theoretical Grounding: Vygotskian socio-constructivist

approach to learning which accepts children‘s experience and environment shape learning outcomes;

• Empirical Grounding: thorough review of international evidence to identify those early learning outcomes which are asssociated significantly with later achievement and attainment and which are widely acknowledged as important at end of ISCED 0, and which can be assessed reliably, rigorously and efficiently.

Assessment Module

18

Learning Domains, Sub-Domains, Indicators

Social & Emotional

Competence

Dispositional and Learning

Competence

Physical Competence

Language & Literacy

Competence

NumeracyCompetence

a. Sociability

b. Emotional literacy and well being

a. Attitudes and mindsets b. Underpinning learning skills

a. Gross motor skills

b. Fine motor skills

a. Early language skills

b. Early literacy skills

a. Number concepts and operations

b. Spatial concepts

• Social competence• Self-regulation

• Self-concept

• Exploratory drive and curiosity• Involvement/ interest

• Persistence

• Locomotion• Coordination

• Manipulation and handling

• Speaking and communication• Story picture sequencing

• Comprehension

• Simple number computation • Problem solving

• Language of spatial objects

Domains Sub-Domains Indicators

19

• Psychometrically robust and reliable• Inclusive of 5 Learning Domains and able to

discriminate developmental differences• Graduated assessment (possible 3-7 year span, but

probably most 5-6 years)• Ethical, respectful of children’s rights and well being• Interactive, meaningful and attractive items• Easy to train and implement• Adaptable to different cultural contexts and low

resource settings

Assessment Design Considerations

20

Two assessment tools which will be used as a blended assessment approach for all 5 learning domains to ensure reliability and efficiency in the assessment process:

1. Direct Assessment Tasks (DAT) (by trained External Administrator)

2. Practitioner Rating (PR) (by practitioner who is familiar to the children)

Assessment Tools

DATPR

21

Social & Emotional

Dispositional and Learning

Physical

Language & Literacy

Numeracy

Sub-Domain 1a

Sub-Domain 1b

Sub-Domain 2a

Sub-Domain 2b

Sub-Domain 3a

Sub-Domain 3b

Sub-Domain 4a

Sub-Domain 4b

Sub-Domain 5a

Sub-Domain 5b

I 1a1

I 1a2

I 1b1

I 2a1

I 2a2

I 2b1

I 3a1

I 3a2

I 3b1

I 4a1

I 4a2

I 4b1

I 5a1

I 5a2

I 5b1

Assessment Design

22

• DAT administered by trained External Administrator: 30 mins per child

• PR administered by setting practitioner: 15 mins per child

• Clear and standardised instructions for assessment administration for DAT and PR

• 15 sampled children per setting assessed• Assessment undertaken in 2 week period in setting

during a three month window before the end of ISCED 0• IEA QA procedures applied

Administration of Assessment

23

• No ranking of countries but could for example, draw comparisons against countries with similar models of/approaches toECE

• Separate scales for 5 Learning Domains• Linking of outcomes evidence to in country and between

country systemic, structural and process evidence• Country profile using

radar diagrams with country mean scores,for example:

Reporting of Assessment

Social and Emotional Competence

Dispositional and Learning Competence

Physical CompetenceLanguage and Literacy Competence

Numeracy Competence

0

5

10

Country ACountry B

24

ECE Setting Questionnaire• To help gain a deeper understanding of key aspects of ECE

from the perspective of those responsible for leading/managing ECE settings.

• It will be completed by the leaders/managers of the sampled settings (approximately 200 per country).

• It will take approximately 45 minutes to complete and can be administered on paper or online.

• The questions will capture information on setting characteristics and all three study themes.

25

Practitioner Questionnaire• To help gain a deeper understanding of key aspects of

ECE from the perspective of practitioners• Focus on pedagogical action and the quality of the

settings• Target population: All staff directly responsible for the

education and care of children in the last year of ISCED Level 0

• It will take approximately 45 minutes to complete and can be administered on paper or online.

26

Home Questionnaire• The Home Questionnaire will enable the study to report

on the expectations, experiences and views of parents of children currently attending ECE in the year before ISCED 1.

• It will be completed by the parent/primary caregiver of the sampled children (approximately 3000 parents per country).

• It will take approximately 30 minutes to complete and can be administered on paper or online.

• The questions will capture information on family/child characteristics and will focus on all three study themes.

27

Study Publications• Study Framework– Conceptual underpinnings– To be published after Field Trial (Feb 2016)

• International Reports– Results from international data collection summarized in four

thematic reports– Published in 2015, 2017 and 2018

• Technical Report– Information on methods and procedures

• International Database and User Guide– To be released in 2018

28

Study Milestones – Phase 1

• 2013– Jun 10-11: 1st Project Advisory Committee meeting

(Birmingham/ UK)– Jul 29-Aug 1: 1st National Research Coordinator meeting

(Hamburg/ Germany)– Nov/Dec: Pilot Policy Questionnaire

• 2014– Apr/ May: Main Study Policy Questionnaire

• 2015– Feb: Publish Report 1:

Outcomes of Policy Questionnaire

29

Study Milestones – Phase 2

• 2014– Jan 9-10: PAC meeting (Hamburg, Germany)– Feb 24-28: 2nd NRC Meeting (Rome, Italy)– Sep 15-19: 3rd NRC Meeting (TBA)– Nov: Data Management Seminar (Hamburg, Germany)

• 2015– Apr/ May: Field Test

• 2016– Feb: Publish Study Framework– Apr/May: Main Study Northern Hemisphere Countries– Sep/Oct: Main Study Southern Hemisphere Countries

30

Study Milestones – Phase 2

• 2017– Oct: Publish Report 2: Outcomes Questionnaires

• 2018– Feb: Publish Report 3: Assessment Outcomes– Sep: Publish Report 4: Final Report– Nov: Release International Database