21st Century Skills, Pedagogy and Human Capital Formation in Singapore: Some Findings from theFormation in Singapore: Some Findings from the Longitudinal Life Pathways Study of Singaporean

Adolescents

Education QueenslandMay 28, 2008.y ,

Professor David Hogan gDean, Office of Educational Research

National Institute of EducationSingaporeSingapore

Quote for the Day

“Everywhere educationalists andEverywhere educationalists and statesmen are aware that the changes which have occurred in the structure of contemporary societies in theircontemporary societies, in their domestic economies as in their foreign affairs, require parallel transformations, no less profound in the special arena ofno less profound in the special arena of the school system.”

Emile Durkheim, The Evolution of Educational Thought in France (1904).

OUTLINE

Crisis of the East Asian Modernization ModelThe East Asian Modernization ModelMetaphysical Anxietiesp yCRPP

Longitudinal Life Choices & Pathways Project  (Panel 6)Research Objectives, Questions and FrameworkOutcome MeasuresResearch DesigngContemporary models of 21st century skillsHuman Capital Formation in Singapore

Classroom pedagogy in Singaporep g gy g pTowards a disciplinary pedagogy 1

The Postmodern Existential Condition and SubjectivityTowards a disciplinary pedagogy 2Towards a disciplinary pedagogy 2

The Singapore East Asian Education M d i i M d lModernization Model

The Singapore version of the East Asian Modernization Model (EAMM) t b d t t f ti b ildi b d th f ll i tmaps out  a broad strategy of nation building based on the following sets 

of claims: 

• Economic/human capital thesis

• Meritocratic thesis

• Pedagogical Thesis

• Cultural thesis

• Organizational thesis

• Nationhood thesis• Nationhood thesis 

Each of these theses under pressure, and have been so for some time, producing no little anxiety and, in time, a range of remedies…

44

TIMSS Results 2003, Mathematics, Grade 4 and 8Source: http://timss.bc.edu/timss2003i/mathD.htmlSource: http://timss.bc.edu/timss2003i/mathD.html

Grade 4 Average Score Grade 8 Average Score

Country Rank TIMSS2003

Country Rank TIMSS2003

Singapore 1 594 Singapore 1 605g p g p

Hong Kong SAR

2 575 Korea 2 589

Japan 3 565 Hong Kong SAR

3 586

Chinese Taipei 4 564 Chinese Taipei 4 585p p

Belgium‐Flemish

5 551 Japan 5 570

InternationalAverage

‐ 495 International Average

‐ 467

55

AnxietiesAnxieties

• Achievement CrisisAchievement Crisis

• Instructional and Cognitive Outcomes

i / l i• Innovation/Implementation

• Culture Trouble/Moral Panic

• Nationhood: A State but not a Nation?

66

1. Achievement Anxieties

• While Singapore does well on TIMSS, concerns about EA PISA results 

• Narrower E‐W gap than TIMSSg p

• Western students do better on more open ended tasks

• Singapore partaking in PISA in 2009: How well will it do?

• TIMSS assessments don’t measure 21st century skills or the kinds of skills and dispositions needed for successful management of work in KBE worksites 

• Very narrow measurement band: Almost no measurement of broader institutional and existential capacities  that are important for successful 

f 21st i i i l i “ f h lf ”management of 21st century institutional environments or “care of the self.”

• EAMM Ignores findings from “revisionist” human capital theory research, especially importance of non‐cognitive skills and attributes to productivity growth and life time income streams 

77

Instructional & Cognitive Outcomes• Anxiety that Singapore schools too focused on development of skills and 

understandings that have, increasingly, little relevance to 21st century institutional settings, especially “knowledge economy” worksites…

• Anxiety that pedagogical practices in Singapore are suboptimal with respect to development of 21st century skills

– Lots of drill and practice. But does repetition lead to understanding, as the East Asian learner model assumes?

– Do teachers know the difference between rote learning and repetition? g pWhen is repetition really repetition and not rote learning?

• Anxiety about the limited integration of technology into classroomAnxiety about the limited integration of technology into classroom pedagogy reflecting teacher judgments that new technologies are broadly inefficient with respect to the achievement of conventional assessment driven classroom objectivesj

88

Innovation /Implementation

• Anxiety that the close alignment of instructional practices with high stakes assessments has resulted in the over‐regulation of classroom pedagogy g p g gyand constrained the opportunity for, and interest in,  pedagogical innovation

• Anxiety that teachers lack the necessary capacities ‐‐ skills (eg. classroom inquiry) and dispositions (eg. reflexivity) ‐‐ for evidence‐based or reflective pedagogy necessary for effective p g gy y

• Anxiety that the dominant instructional pattern in Singapore (as elsewhere) are highly intractable and resistant to change [Teacher beliefs? Vernacularare highly intractable and resistant to change [Teacher beliefs? Vernacular Confucianism? Folk pedagogy?  Institutional rules? Teaching scripts? Interactional genres? Teacher habitus? Assessment system? Transaction costs of innovation? Teacher appraisal system? Risk avoidance?]pp y ]

99

4.  Culture Trouble/Moral Panic: Decline of East Asian/Confucian classroom moral economy

• Motivation Crisis

• Some evidence that traditional “East Asian” discourse of hard work / effort  among both students and teachers being replaced by ability/deficitamong both students and teachers being replaced by ability/deficit discourses of student achievement

S id f d li i l l f t d t li d t• Some evidence of declining levels of student compliance and engagement

• Anxiety that EA systems encourage a heavy reliance on extrinsic motivation and instrumental outcomes and that this leaves studemts disinterested in content that is not “in the exam” and renders the dominant pedagogical exchange vulnerable to a decline in its exogenous terms of trade (declining rates of return to educational investments declining social mobility ratesrates of return to educational investments, declining social mobility rates, etc)

• Anxiety that personal ego achievement goals becoming more widespread: motivated to learn in order to beat others rather than for its own sake... 

1010

Culture Trouble/Moral Panic – cont’d• Crisis of Meaning (making)/ Narrativity• Crisis of Meaning (making)/ Narrativity

• Anxiety (at least among some educators) about the absence of narrativity and focus on meaning‐making in EA classrooms in favor  of content coverage and exam preparation. p p

• Anxiety that classrooms all too often constructed as manufactories of drill and practice and/or competitive marketplaces rather than communities of learners engaged in distributed forms of knowledge building

• Values Crisis

• Anxiety about the creeping infiltration of Western, modernist and postmodern values d id i j i h l di l i di i l f i / iand identity projects into the classroom, displacing traditional Confucian/East Asian 

values and identity projects. 

• In short pervasive moral anxiety that modernization and globalization resulting in• In short, pervasive moral anxiety that modernization and globalization resulting in the “Westernization” of classroom, that Western values invading and colonizing Confucian sanctuary or “museums of virtue”. Yet EA countries, including Singapore, strongly committed, since the early 1990s, to becoming a global city 

1111

Centre for Research On Pedagogy and Practice (CRPP):and Practice (CRPP):

A V B i f I d iA Very Brief Introduction

12

Thinking Schools, Learning Nation, 19971997

“We will bring about a mindset change among Singaporeans. We must get away

from the idea that it is only the people at the top who should be thinking, and thefrom the idea that it is only the people at the top who should be thinking, and the

job of everyone else is to do as told. Instead we want to bring about a spirit of

innovation, of learning by doing, of everyone each at his own level all the time asking

how he can do his job better...” (Italics added).

Prime Minister, Goh Chok Tong, Speech at the Opening of the 7th International

Conference on Thinking in 1997, para. 31.

1313

CRPP

Established 2003Initial five year grant of $48m (SGD) ($37mInitial five year grant of $48m (SGD) ($37m AUD)K i i f h E A iKey context: crisis of the East Asian Modernization model

14

Thinking Schools, Learning Nation, 1997

“We will bring about a mindset change among Singaporeans. We must get 

away from the idea that it is only the people at the top who should be thinking,away from the idea that it is only the people at the top who should be thinking, 

and the job of everyone else is to do as told. Instead we want to bring about a 

spirit of innovation, of learning by doing, of everyone each at his own level all 

the time asking how he can do his job better...” (Italics added).

Prime Minister, Goh Chok Tong, Speech at the Opening of the 7th International 

Conference on Thinking in 1997, para. 31.

1515

CRPP: Objectives

CRPP established in 2003 with four key objectives:

to describe & measure patterns of classroom culture and pedagogy in Singaporean schools

to measure the impact of cultural orientations and pedagogical practices on student outcomes controllingpedagogical practices on student outcomes controlling for student characteristics

to identify opportunities for the improvement ofto identify opportunities for the improvement of pedagogical practice through a carefully designed & evidence-based intervention strategy

to support evidence-based policy formulation and instructional practice to meet the challenges of 21st

t i tit ti l i tcentury institutional environments

INSTITUTIONAL INSTITUTIONAL ENVIRONMENTENVIRONMENT

ORGANIZATION OF ORGANIZATION OF SCHOOLINGSCHOOLING

FIGURE 1. FIGURE 1. PEDAGOGICAL PEDAGOGICAL PRACTICESPRACTICES

MORAL ECONOMY MORAL ECONOMY OF THE CLASSROOMOF THE CLASSROOM

CONCEPTUAL MODEL CONCEPTUAL MODEL

OF CRRP RESEARCH OF CRRP RESEARCH

PROGRAM 2003PROGRAM 2003--20082008

PRACTICESPRACTICES

CLASSROOMCLASSROOM

OF THE CLASSROOMOF THE CLASSROOM

CLASSROOM CLASSROOM INTERACTIONSINTERACTIONS

STUDENT STUDENT OUTCOMESOUTCOMES

PEDAGOGICAL PEDAGOGICAL LIFE GOALS, LIFE GOALS, 

17

INTERVENTIONSINTERVENTIONSCHOICES AND CHOICES AND PATHWAYSPATHWAYS

Organization of CRPP Research Program, 2003-2008

Core ResearchCore ResearchCore Research Core Research ProgramProgram(2004(2004--07)07) EB EB

InterventionIntervention6 Panels6 Panels Intervention Intervention ProgramProgram(2006(2006--07)07)

Specific Focus Specific Focus Projects Projects (2004(2004--06)06)

20 projects20 projects

90+ Projects90+ Projects

CRPP Core Program

Panel 1: MOE student data system

Panel 2:Panel 2: S & T surveys of pedagogical practices and academic outcomes. S (n=18000);T (n=4000). 80 schools (40/40)

Panel 3:Panel 3:Classroom observation of 1200 lessons (450 units) in 60 schools (30 P, 30 S) across key subjects

Panel 4Electronic corpus of 1200 lessons above

Panel 5S and T assessment artifacts of 1200 lessons aboveS a d assess e t a t acts o 00 esso s abo e

Panel 6Multi-cohort longitudinal study of 30000 students of academicMulti cohort longitudinal study of 30000 students of academic and non-academic outcomes

LONGITUDINAL LIFE CHOICESLONGITUDINAL LIFE CHOICES AND PATHEWAYS PROJECT

(CRPP’s PANEL 6)

Panel 6: Key Research Objectives

Measure, map and model changes over an extended time period in the social and educational experiences of three cohorts of a large representative sample of Singaporean young people.

M d d l h ti i b d fMeasure, map and model changes over time in a broad range of academic and “non-academic” capacities, aspirations, attainments (outcomes) and pathways of Singaporean young people.

A h ff i f h li i d l i h i iAssess the effectiveness of schooling in developing those capacities

Challenge the current hiatus between student outcomes thatChallenge the current hiatus between student outcomes that parents and educators value and those that are currently assessed in school

Hiatus between education goals and assessmentassessment

One of the more interesting and revealing ironies of contemporary schooling is a peculiar hiatus between the breadth of theschooling is a peculiar hiatus between the breadth of the educational aspirations that systems proclaim on behalf of schools and the thinness of the measures of school outcomes that schools use to assess their performance in realizing their aspirations.

While s hool s stems h e in ested enormo s reso r es inWhile school systems have invested enormous resources in curriculum development, pedagogical reform and assessment of traditional academic outcomes, very little time, effort and money have been spent on developing assessment instruments, which allow for measuring the impact of schooling on broader “non-academic” outcomes that parents, communities, schools and p , ,governments value.

22

Panel 6: Research Questions - 1

What kind of academic and non-academic capacities, skills or competencies are young people likely to need to successfully manage rapidly changing 21st century institutional environments?What evidence is there that young people acquire theseWhat evidence is there that young people acquire these academic and non-academic capacities, skills or competencies over time as they go through school or participate in other social i i i ?institutions? What is the relationship between these academic and non-academic capacities, skills or competencies? To what extent doacademic capacities, skills or competencies? To what extent do they covary? Are they complementary or orthogonal?To what extent do these academic and non-academic capacities, skills or competencies vary across population groups and school experiences?

Panel 6: Research Questions - 2

How much of the variance in student capacities, skills or competencies is explained by differences between schools, between classrooms and streams within schools, and by differences between students within classrooms?How effective are schools in developing 21st century capacitiesHow effective are schools in developing 21 century capacities, skills or competencies? What schools have the strongest impact on growth rates in key relevant outcome measures controlling for the initial or intake characteristics of students? What are the characteristics – and characteristic practices – of these schools?characteristic practices of these schools? To what extent do current assessment practices in schools capture the development of 21st century capacities, skills or competencies?What are the implications of these findings for educational policy and practice, particularly with respect to assessment?

Panel 6: Outcome Measures1. Economic capacities and participation (“Human capital”)

Academic achievement / performance/ p“Knowledge economy” capacities / skillsLabor market participation and attainments

2. Social skills and participation (“Social capital”)Social NetworksFriendships Intimacy (family, friends, boy/girl friends)y ( y, , y g )TrustAttachment / MembershipLeadership skillsLeadership skillsCommunity group memberships

Outcome Measures

3. Civic capacities and participation (“civic capital”)Civic Beliefs (good citizenship justice national identity etc)Civic Beliefs (good citizenship, justice, national identity, etc)Civic Agency (participation in political groups, voting intentions, reading/ discussing the news) Civic Dispositions (interest/engagement trust respect forCivic Dispositions (interest/engagement, trust, respect for others…)Civic identity/national identities (attachment, membership)

4. Subjectivity (the organization of the self) Self beliefs (self efficacy/subjective agency, self concept, self

fid l f l)confidence, locus of control)ValuesGoals and aspirationspIdentity formations (social, personal)Reflexivity/self understanding (incl. meta-cognition)

Outcome Measures5. Subjective Wellbeing (“existential capital”)

Existential satisfactionMental health morbidities (anxiety depression exam anxiety)Mental health morbidities (anxiety, depression, exam anxiety)Positive and negative affectDomain specific life satisfaction Life satisfaction / happiness

6. Life Choices and PathwaysSelf formation (biography)School engagement and attainmentSchool engagement and attainmentWork, career and labor market participationSocial participation (networks, friendship, intimacy) (civil society)Family formation Participation in civic / political society

Research Designf h d f l d l d l d fLife Pathways Study is a four year accelerated longitudinal study of a

stratified random sample of three cohorts (Post-secondary 1, Secondary 1 and Primary 4) of 30,000 Singapore students from 40P, 40S, and 27PS

Survey administrationStudents from each cohort randomly allocated to one of four instruments that variously focus on one of four sets of outcome measures:

H i l f iHuman capital formationSocial and civic capacities and outcomesSubjectivity & subjective wellbeing Life goals choices and pathwaysLife goals, choices and pathways

Each instrument a common module of predictors plus dedicated predictors and key outcome measures from other three instruments for triangulation

Academic assessment in Mathematics and English to P and S cohorts

5 I tit ti l5 I tit ti l

Simplified Conceptual Model of Life Simplified Conceptual Model of Life Pathways Longitudinal ProjectPathways Longitudinal Project

1. Institutional Contexts1. Institutional Contexts

5. Institutional5. InstitutionalAttainments Attainments

[[SchoolSchool (academic (academic hi t) f ilhi t) f il

4. Institutional 4. Institutional ParticipationParticipation

( families, schools, peer ( families, schools, peer groups, cultural markets, groups, cultural markets, consumer markets, labor consumer markets, labor

markets communitymarkets community

achievement), family achievement), family (intimacy), peers (intimacy), peers

(friendship), work (friendship), work (income, promotion), (income, promotion), mm it (m mb r hip)mm it (m mb r hip)

Participation Participation (Social agency)(Social agency)

(schools, workplaces, peer (schools, workplaces, peer groups cultural marketsgroups cultural marketsmarkets, community markets, community

institutions and groups, institutions and groups, government policy, political government policy, political

society, etc)society, etc)

community (membership), community (membership), the selfthe self (subjective (subjective

wellbeing)]wellbeing)]

groups, cultural markets, groups, cultural markets, consumer markets, consumer markets,

community groups, political community groups, political society, the self).society, the self).

2. The Self 2. The Self

6. Life Pathways6. Life PathwaysBiography (“narrative”) Biography (“narrative”) of choices and pathways of choices and pathways 3. Capacities / “Capitals” 3. Capacities / “Capitals”

(Subjectivity)(Subjectivity)

(Self beliefs (self concept, self (Self beliefs (self concept, self efficacy/ subjective agency, efficacy/ subjective agency,

through time with through time with respect to respect to educationeducation, ,

work, marriage, work, marriage, citizenship, religion, citizenship, religion,

(Individual level capacities (Individual level capacities --academic, economic, social, academic, economic, social,

cultural, civic, subjective cultural, civic, subjective ----that that self confidence, self control), self confidence, self control),

values, goals & aspirations,values, goals & aspirations,identities, reflexivity )identities, reflexivity )

friendship, residence, friendship, residence, consumption, consumption,

community participation community participation and and self formationself formation

facilitate effective participation in facilitate effective participation in contemporary institutions)contemporary institutions)

Key theoretical frameworks

Sociology of modernization and globalizationGeneralized theory of skill/capacity/capital formation (human, social, civic, subjective)j )Life course/pathways researchSociology of stratification: who gets to get, be or do what?Identit theorIdentity theoryPostcolonial theorySocial agency theoryTheory of the selfSchool effects researchAssessment research Pedagogy/Curriculum theoryResearch on subjective wellbeing Research on 21st century skills and competenciesResearch on 21st century skills and competencies

30

Human Capital Theory and 21st

C nt SkillCentury Skills

5 I i i l5 I i i l

Simplified Conceptual Model of Life Simplified Conceptual Model of Life PathwaysPathways

Longitudinal ProjectLongitudinal Project

1. Institutional 1. Institutional ContextsContexts

5. Institutional5. InstitutionalAttainments Attainments

[[SchoolSchool (academic (academic d i l ) f ild i l ) f il

4. Institutional 4. Institutional ParticipationParticipation

Longitudinal ProjectLongitudinal Project

ContextsContexts

( families, schools, peer ( families, schools, peer groups, cultural markets, groups, cultural markets, consumer markets laborconsumer markets labor

credentials), family credentials), family (intimacy), peers (intimacy), peers

(friendship), work (friendship), work (income, promotion), (income, promotion),

i ( b hi )i ( b hi )

Participation Participation (Social agency)(Social agency)

(schools workplaces peer(schools workplaces peerconsumer markets, labor consumer markets, labor markets, community markets, community

institutions and groups, institutions and groups, government policy, political government policy, political

society etc)society etc)

community (membership), community (membership), the selfthe self (subjective (subjective

wellbeing)]wellbeing)]

(schools, workplaces, peer (schools, workplaces, peer groups, cultural markets, groups, cultural markets,

consumer markets, community consumer markets, community groups, political society, the self).groups, political society, the self).

society, etc)society, etc)

6. Life Pathways6. Life PathwaysBiography (“narrative”) Biography (“narrative”) of choices and pathways of choices and pathways 3. Capacities / “Capitals” 3. Capacities / “Capitals” 2. The Self 2. The Self p yp y

through time with through time with respect to respect to educationeducation, ,

work, marriage, work, marriage, citizenship, religion, citizenship, religion,

(Individual level capacities (Individual level capacities --academic, academic, economiceconomic, social, , social,

cultural, civic, subjective cultural, civic, subjective ----that that

(Subjectivity)(Subjectivity)

(Self beliefs (self concept, self (Self beliefs (self concept, self efficacy/ subjective agency, efficacy/ subjective agency, p, g ,p, g ,

friendship, residence, friendship, residence, consumption, consumption,

community participation community participation and and self formationself formation

facilitate effective participation in facilitate effective participation in institutional practices)institutional practices)

self confidence, self control), self confidence, self control), values, goals & aspirations,values, goals & aspirations,

identities, reflexivity, identities, reflexivity, personality (traits))personality (traits))

Human Capital Formation: Research Questions

How has the organization of work changed in recent times? How is it likely to change in the foreseeabletimes? How is it likely to change in the foreseeable future?

What skills and competencies are workers likely to need in knowledge economy worksites?

How should economic capacities and student outcomes be specified and measured?p

How strongly are they interrelated?

Human Capital Formation: Research Questions - 2

To what extent do these capacities vary across population and school groups?

What explains variations in the development of these capacities?

To what extent do current pedagogical practices in Singaporean classrooms focus on the development of 21st century skills and competencies? p

What impact to pedagogical practices have on the development of 21st century competencies?21 century competencies?

What kind of pedagogical practices are likely to nurture the kinds of understandings skills and dispositions needed to successfullyunderstandings, skills and dispositions needed to successfully negotiate 21st century institutions? What evidence is there that they are effective?

Human Capital Theory

Conventional human capital theory (aka Gary Becker) argues that cognitive skills are the primary source of labor market productivity gains and life time income streamsgains and life time income streams.

This view challenged by revisionist theories of human capital g y pthat find that “non-cognitive” capacities or characteristics –personality traits, motivation, locus of control beliefs, self-management skills and so on are as important if not moremanagement skills and so on — are as important if not more important than cognitive skills as sources of productivity gains and as predictors of life time income streams.*

This underscores the importance of a broad mapping of academic and “n n d mi ” p iti nd t m n p r l in tr m nt l“non-academic” capacities and outcomes on purely instrumental grounds alone …

1. The Organization of Work

Recent reports on contemporary workplaces in KBEs p p y pgenerally find that they are characterized by a new technical and social organization of work and by demands for new kinds of cognitive communication anddemands for new kinds of cognitive, communication and social skills

Conventional human capital models focus on development of a relatively narrow range of cognitive skills

Contemporary analysts of human capital have developed a much broader view of technical and social skills and

36

understandings necessary for effective participation in KBE worksites….

Technical organization of work

A new technical organisation of work, characterised by …significantly greater cognitive complexityexpanded information processing and knowledge requirementscross-functional, transdisciplinary, interactive task environments and project workreliance on high levels of tacit and uncodified knowledge“transdisciplinary” (Mode 2) knowledge production

Generating a demand for new kinds of cognitive and communication skills

analytical problem solvingknowledge applicationgeneration of new knowledge

37

understanding complex multi-level functional relationships and systems

Social organization of work

And a new social organisation of work characterized by greater autonomyflatter hierarchiesmore team workmore intensive interactionh d d i i kishared decision making

more risk takingmore extensive oral and written communication

Generating demand for new kinds of social understandings and skillsa sense of agencyinterpersonal problem solvingp p gindependencecollaborationtrust

38

adaptability

NRC, The Changing Nature of Work, 1999.

“Control and Discretion. The vertical division of labor is changing in organizations that have flattened their hierarchies, turned to team forms of work organization and adopted humanturned to team forms of work organization, and adopted human resource policies often described as “high-performance work systems.” Moreover, in these settings blue-collar and managerial jobs are undergoing significant changes thatmanagerial jobs are undergoing significant changes that are blurring the traditional lines of demarcation that separated these traditional categories. The autonomy of blue collar workers has increased and some of the responsibilities traditionally reserved for supervisors, such as quality control, scheduling, and other operational responsibilities, have been delegated to non-supervisory workers…”

39

NRC, The Changing Nature of Work, 1999.

“Task Scope. Because of the foregoing changes associated with i i l d t li d t d t l th fincreasingly decentralized autonomy and control, the scope of blue-collar jobs is expanding, particularly in settings that make use of teams or other high-performance features to improve quality innovation and customer responsiveness At the samequality, innovation, and customer responsiveness. At the same time, the growth of specialized scientific knowledge has increased demand for specialists with state-of-the-art technical knowledge. Furthermore, professionals and technicians with g pspecialized knowledge are more frequently working in interdependent, multidisciplinary, and cross-functional teams.”

40

“Cognitive Complexity. The cognitive complexity of work appears to be increasing for blue collar and service workers as aappears to be increasing for blue-collar and service workers as a result of the technical and the organizational changes discussed above. The dominant trend is toward work that mixes physical and sensory skills with higher-level cognitive skills required by information processing technologies. Technical, professional, and managerial work has always entailed high p , g y gcognitive content. Although there is no clear evidence to suggest that the cognitive levels of these jobs are changing in a significant way the types of cognitive skills that are required maysignificant way, the types of cognitive skills that are required may be changing ...”

“Interactive and Relationship Requirements. Interpersonal interactions are becoming more important in many jobs. g p y jInterdependence and more direct interactions with other employees, customers, clients, and patients increase the importance of both substantive and emotional interactionsimportance of both substantive and emotional interactions that require skills in communications, problem solving, and negotiations.”

National Research Council in the US, The Changing Nature of Work, 1999, 271-73

The New Commission on the Skills of the American Workforce America in the Global Economy 2006Workforce, America in the Global Economy, 2006

1. Academic CompetenciesBasic skills – fundamental academic skills needed to work with or acquire more specific skills. These skills include reading comprehension, active listening, writing, speaking, mathematics,comprehension, active listening, writing, speaking, mathematics, and science.English Language – knowledge of the structure and content of h h d h d f dthe English language including the meaning and spelling of words,

rules of composition, and grammar.Mathematics – knowledge of arithmetic algebra geometryMathematics knowledge of arithmetic, algebra, geometry, calculus, statistics, and their applicationsArts and Humanities – knowledge of facts and principles related to learning concerned with human thought, language, and the arts

43

2. Thinking CompetenciesCritical Thinking – using logic and reasoning to identify the t th d k f lt ti l ti l istrengths and weaknesses of alternative solutions, conclusions,

or approaches to problems.Originality – the ability to come up with unusual or clever ideas about a given topic or situation, or to develop creative ways to solve a problem.Innovation – creativity and alternative thinking to develop new y g pideas for and answers to work-related problemsDeductive reasoning – the ability to apply general rules to specific problems to produce answers that make sensespecific problems to produce answers that make sense.Inductive reasoning – the ability to combine pieces of information to form general rules or conclusions (includes fi di l ti hip i l l t d t )

44

finding a relationship among seemingly unrelated events).Mathematical reasoning – the ability to choose the right mathematical methods or formulas to solve a problem.

3. Workplace competenciesSocial skills – developed capacities used to work with people to achieve goalsto achieve goalsComplex problem solving – identifying complex problems, reviewing related information to develop or evaluate options; i l t l tiimplement solutionsThinking creatively – developing, designing, or creating new ideas, applications, relationships, systems, or products including artistic contributionsEngineering technology – knowledge of the practical application of engineering science and technology including pp g g gy gapplications to the design and production of various goods and services.Enterprising interests – these work environments involve

45

Enterprising interests – these work environments involve starting up and carrying out projects, often leading people and making decisions (20-21).

North Central Regional Laboratory’s Model of Digital Age Competencies

46

Transdisciplinary knowledge and skills

What is transdisciplinary knowledge and what does it require?Knowledge of complex systems (natural or social) that have no respect f di i li b d ifor disciplinary boundariesProblem solving skills/capacities as members of knowledge production teams. ‘Genuinely transdisciplinary curricula … involve teaching programmes that are orientated to understanding complex systems, are based on participation on problem solving teams, and draw h il n m d llin nd im l ti n t hniq ” (Mi h l Gibbheavily on modelling and simulation techniques.” (Michael Gibbons, 1998, p. 42).

d dTransdisciplinary knowledge is not …Interdisciplinary knowledge Multidisciplinary knowledge

47

OECD-PISA

“Underlying OECD/PISA is a dynamic model of lifelong learning in which new knowledge and skills necessary for successful adaptation to changing circumstances are continuously acquired over the lifeto changing circumstances are continuously acquired over the life cycle. Students cannot learn in school everything they will need to know in adult life. What they must acquire is the prerequisites f r f l l rnin in f t r lif Th pr r q i it r ffor successful learning in future life. These prerequisites are of both a cognitive and a motivational nature. Students must become able to organise and regulate their own learning, to learn independently and in groups, and to overcome difficulties in the learning process. This requires them to be aware of their own thinking processes and learning strategies and methods.own thinking processes and learning strategies and methods. Moreover, further learning and the acquisition of additional knowledge will increasingly occur in situations in which people work together and are dependent on one another ” OECD Me suringtogether and are dependent on one another. OECD, Measuring Student Knowledge and Skills: A New Framework for Assessment (OECD, 1999, pp. 9-10. 48

Conventional Human Capital F tiFormation:

English and Mathematics PerformancePerformance

494949

Mathematics Progress (2005 to 2006)S l dS l d

1000Scaled Scaled scorescore

904

950

900900

950

882

838816

855865

850

The 39 schools show The 39 schools show an an increase of 44increase of 44scaled score points.scaled score points.

760

816 815

750

800

700

750

505050

2005 2006

NoteNote. T. The scaled score are calculated using the Item Response Theory based equating process that he scaled score are calculated using the Item Response Theory based equating process that accommodates assessment differences in difficulty level across the year.accommodates assessment differences in difficulty level across the year.

UNSW English Progress (2005 to 2006)ScaledScaled

1268

1300Scaled Scaled scorescore

1196

1218

12381227

1200

1250

1125

1150

The 39 schools show The 39 schools show an an increase of 145increase of 145scaled score points.scaled score points.

1125

1093

10711082

1100

1038

1000

1050

515151

10002005 2006

NoteNote. T. The scaled score are calculated using the Item Response Theory based equating process that he scaled score are calculated using the Item Response Theory based equating process that accommodates assessment differences in difficulty level across the year.accommodates assessment differences in difficulty level across the year.

Research QuestionResearch Question

T h d i l l i lTo what extent do conventional neo-classical measures of human capital formation vary by

h l d l i ? Hschool stream and population group? How stratified are conventional human capital

?outcomes?

52

Effect Size Metric:

Omega square:Omega square:

ω2 ≤ .01 (small)

ω2 = 06 (moderate);ω =.06 (moderate);

ω2 >.16 (large);

ω2 ≥.03 threshold

53

ANOVAs of Math and English Achievement ANOVAs of Math and English Achievement b STREAMb STREAMby STREAMby STREAM

BY STREAM Gifted/Special Express Normal

(A)Normal

(T) F p ω2Special (A) (T)

Sec1

Math33.29

(n = 933)28.32

(n = 2799)21.51 (n =

1295)

16.69 (n = 677)

2778.81 <.01 .594

Sec1

English 53.68(n = 979)

44.34(n = 2760)

32.39(n =

1274)

22.79(n = 690)

2556.75 <.01 .573

13 56 9 80

Sec2

Math 27.26(n = 959)

21.01(n = 2882)

13.56(n =

1344)

9.80(n = 636)

2407.43 <.01 .554

48 70 37 26 24.47 17.92English 48.70

(n = 1021)37.26

(n = 2838) (n = 1317)

(n = 618)

2105.16 <.01 .521

54

Note. Effect sizes metric: ω2 ≤ .01 (small); ω2 =.06 (moderate); ω2 >.16 (large); ω2 ≥.03 threshold

ANOVAs of Math and English Achievement ANOVAs of Math and English Achievement b RESIDENCE TYPEb RESIDENCE TYPEby RESIDENCE TYPEby RESIDENCE TYPE

5-room/BY RESIDENCE 1–3 room

flat4-room

flat

5 room/executive

flat

Private housing F p ω2

Math 23.18(n 836)

24.71(n 1923)

26.87(n 1755)

29.94(n 1153) 246.81 < .01 .115

Sec1(n = 836) (n = 1923) (n = 1755) (n = 1153)

English 34.36(n = 805)

37.85(n = 1872)

42.36(n = 1775)

47.01(n = 1218) 245.35 < .01 .114

15 90 17 50 19 51 23 08

Sec2Math 15.90

(n = 821)17.50

(n = 1932)19.51

(n = 1824)23.08

(n = 1244) 235.67 < .01 .108

English 28.65(n = 767)

30.78(n = 1870)

35.58(n = 1887)

41.03(n = 1270) 230.30 < .01 .106

NoteNote. Effect sizes metric: . Effect sizes metric: ωω22 ≤ .01 (small); ≤ .01 (small); ωω22 =.06 (moderate); =.06 (moderate); ωω22 >.16 (large); >.16 (large); ωω22 ≥.03 threshold≥.03 threshold

55

Correlation between PSLE and UNSW AssessmentsCorrelation between PSLE and UNSW Assessments

N PSLE PSLE PSLEN PSLE(overall)

PSLE (math)

PSLE(English)

S1 Math (2005) 5368 .81** .74** .62**

S1 E li h (2005) 5380 80** 60** 71**S1 English (2005) 5380 .80** .60** .71**

S2 Math (2006) 5305 .77** .73** .61**

S2 English (2006) 5310 77** 67** 71**S2 English (2006) 5310 .77 .67 .71

NoteNote. Effect sizes metric: . Effect sizes metric: ωω22 ≤ .01 (small); ≤ .01 (small); ωω22 =.06 (moderate); =.06 (moderate); ωω22 >.16 (large); >.16 (large); ωω22 ≥.03 ≥.03 thresholdthresholdthresholdthreshold

56

The Stratification of Human Capital: Correlation between SES and Achievement (L1 only)*( y)

N SES(r)

Panel 2P5 Math (2004) 2848 .32**P5 English (2004) 1738 .36**S3 Math (2004) 3721 .29**S3 English (2004) 3394 .42**Panel 6Panel 6

S1 Math (2005) 5667 .34**

S1 English (2005) 5670 .34**

S2 Math (2006) 5821 .32**

S2 English (2006) 5794 .32**

57

NoteNote. Panel 6 uses only students’ residence type as the SES proxy.. Panel 6 uses only students’ residence type as the SES proxy.

In short …

High correlation between CRPP/UNSWHigh correlation between CRPP/UNSW measures of cognitive performance and social backgroundbackground

Hi h l i b CRPP/UNSWHigh correlation between CRPP/UNSW assessments of cognitive performance and

i l hi h k (PSLE)national high stakes assessment (PSLE)

58

“Knowledge Economy” Measures ofKnowledge Economy Measures of Human Capital Formation

595959

New Economy Competencies

Interpersonal skills

Interpersonal Problem-Solving Skills Group Engagement (in

Team WorkSocial & Leadership Skills

Group Engagement (in class)

Agentic Orientation to Risk-Taking

Problem Solving / Self Efficacy

Dispositions Adaptability Orientation to Decision Making

Work-related Orientation to Project Work

Orientation to CompetitionO i i L iSkills

WorkGeneric Agentic Skills

Orientation to Learning

Knowledge Multi-literacy SkillsInterdisciplinary Work SkillsManagement

SkillsInterdisciplinary Work SkillsInformation Management Skills

Time Management

606060

Self-Regulation Individual Engagement (in class)Effort Regulation

NEW ECONOMY COMPETENCIES(Secondary 2)

Cronbach’s Alpha

Means(1-6)

SD

Locus of Control (Internal) Scale (5 items) 0.82 5.00 0.79O S ( )Orientation to Risk Taking Scale (5 items) 0.85 4.55 0.83Information Management Scale (5 items) 0.85 4.51 0.84Confidence / Outcome Expectancies Scale (7) 0.89 4.40 0.86Locus of Control (External) Scale (6) (reversed) 0.80 4.32 1.00Engagement in Individual Work Scale (4 items) 0.90 4.29 0.97Adaptability Scale (6 items) 0.86 4.26 0.82Agentic Self Concept Scale (5 items) 0.79 4.22 0.90Effort Regulation Scale (reversed) (5 items) 0.88 4.22 1.10Self Efficacy Beliefs (Problem-Solving) Scale (6) 0.85 4.18 0.80Interpersonal Problem-Solving Skills Scale (5) 0.82 4.18 1.00Interdisciplinary Work Skills Scale (3 items) 0.84 4.17 0.83Multi-literacy Skills Scale (4 items) 0.73 4.06 0.87

61

Time Management Scale (3 items) 0.81 3.96 1.12Coping Skills Scale (3 items) 0.77 3.39 1.21

Risk Taking (Secondary 2)(Secondary 2)

ItemsMean Std

Skewness KurtosisItems(1-6) Dev

Skewness Kurtosis

To be successful you often need to take risks 4.71 .985 -.780 1.192

I'm willing to use different strategiesI m willing to use different strategies to see which one works best 4.51 .980 -.641 .970

I'm willing to try new things even though I might fail 4.48 1.001 -.649 .842

I don't mind trying something new even if I'm not sure of the outcome 4.42 1.027 -.673 .900

I like coming up with creative solutions to problems 4.41 1.051 -.573 .530solutions to problemsOrientation to Risk Taking Scale (5 items)

4.51 .854 -.665 1.556

C b h’ Al h (S d di d) 902

62

Cronbach’s Alpha (Standardized) .902

How have students’ self-reported new economy competencies y p

changed from…?

i.i. Secondary 1 to 2Secondary 1 to 2ii.ii. Primary 4 to 5Primary 4 to 5

636363

New Economy Competencies - Secondary

66

Sec 1Sec 1 Sec 2Sec 2

Mean ScoresMean ScoresGreatest increaseGreatest increase

Greatest decreaseGreatest decrease

44

55

33

11

22

64Note. Ranked order by Secondary 2 Note. Ranked order by Secondary 2 means, highest lowestmeans, highest lowest

Social Demography of Knowledge Economy Capacities.Capacities.

Research Question:

To what extent do key economic capacities vary across students from different y

population groups?

65

Key finding

Whereas strong correlation between SES andWhereas strong correlation between SES and cognitive performance, very low correlation between SES and self reported measures ofbetween SES and self reported measures of knowledge economy measures

66

Agentic Dispositions

5

6Gifted/Special Express Normal Academic Normal Technical

4

5

2

3

1

Adaptability Orientation todecision making

Orientation towardsrisk taking

Problem solvingefficacy

Differences in the means by Stream in Agentic Differences in the means by Stream in Agentic Dispositions are small. Similar results for other Dispositions are small. Similar results for other

67

ppknowledge economy capacities.knowledge economy capacities.

Work-related Dispositions

Comparisons by GenderPri4 Sec1

M F ω2 M F ω2

Orientation to Life Long Learning

4.91 5.06 .01 4.83 4.94 .00

Group Engagement 4 48 4 62 00 4 38 4 39 00Group Engagement 4.48 4.62 .00 4.38 4.39 .00

Individual Engagement 4.45 4.79 .03 4.23 4.35 .00

Persistence 4.82 5.00 .01 4.17 4.30 .00

Orientation towards Competition

3.88 3.68 .01 4.08 3.84 .01

N tN t ωω22 ≥ 03 thr h ld≥ 03 thr h ld

Differences in Gender means in WorkDifferences in Gender means in Work--related related Dispositions are statistically insignificant. Similar Dispositions are statistically insignificant. Similar

NoteNote. . ωω22 ≥ .03 threshold≥ .03 threshold

68

results for all other knowledge economy results for all other knowledge economy capacities. capacities.

Knowledge Management Skills

Comparisons by RaceSec1

Chinese Malay Indian ω2Chinese Malay Indian ω2

Interdisciplinary Skills 4.10 4.22 4.69 .04

Multiliteracies Skills 3.97 4.21 4.51 .04

Information Management 4.44 4.60 4.99 .03

NN 22 ≥ 03 h h ld Ab l d f≥ 03 h h ld Ab l d f P iP i 4 d4 d

Small to moderate differences by race. However, Small to moderate differences by race. However,

NoteNote. . ωω22 ≥ .03 threshold. Above scales were not measured for ≥ .03 threshold. Above scales were not measured for PriPri 4 students.4 students.

y ,y ,differences in other differences in other knowledge economy capacitiesknowledge economy capacitiesare small and insignificant.are small and insignificant.

69

Summary

Variations in knowledge economy capacities by population groups small or statistically insignificantp p g p y g

By contrast, variations by population group in cognitive y , y p p g p gachievement moderate to large

Suggests that a broader, more multi-dimensional assessment system that combined the two sets of measures would probably reduce group differencesmeasures would probably reduce group differences and inequality of educational achievement – in effect, promote greater equity.

70

Research Question:Q

How strong are the relationships between

(1) self-reported knowledge economy capacities and

(2) between knowledge economy capacitiesand cognitive achievement?and cognitive achievement?

717171

Correlation Matrix: Measures of Human Capital FormationCapital Formation

Agentic Di iti

Effective C i t’

Self R l t’

Knowledge M ’t

UNSW M th

UNSW E li hDispositions Communicat’

nRegulat’n Managem’t Math English

AgenticDisposit’ns .398*** .505*** .659*** .042 .093***

EffectiveCommunic 254*** 339*** 066* 082**Communic-ation

.254*** .339*** .066* .082**

Selfregulation .522*** .062* .080**g

KnowledgeManagment .066* .123***

72

In short…

Strong correlations between knowledge economy measures

Small or statistically insignificant relationships between measuresSmall or statistically insignificant relationships between measures of cognitive achievement and knowledge economy capacities

Assuming that new economy measures are better predictors ofAssuming that new economy measures are better predictors of workplace productivity (for which we have no direct data) the allocative and productive efficiency of the labor market would be improved by broader, more multi-dimensional p y ,assessments that directly measure knowledge economy capacities

In addition, improved equity: KE measures moderate highIn addition, improved equity: KE measures moderate high impact of SES on high stakes assessment

In sum multi-dimensional assessments can possibly increase

73

In sum, multi-dimensional assessments can possibly increase efficiency as well as equity of labor markets. No trade off between efficiency and equity. Have your cake and eat it too!

Caveat Emptor / What’s Next?

Caveat Emptor: our measures of knowledge economy measures likely to include measurement error because they are based on self reported measures of skills cfthey are based on self reported measures of skills cf authentic assessments. Ok for attitudes & dispositions

For the second phase of funding, we intend to focus on development of direct authentic performance assessments of knowledge economy skills as well asassessments of knowledge economy skills, as well as other forms of human capital (ie academic performance)

74

Classroom Pedagogy in Singapore

75

Teacher dominated pedagogy... (Classroom Observation Data)

Primary 5 Primary 5  % of % of  Secondary 3Secondary 3 % of % of (Rank Ordered)(Rank Ordered) PhasesPhases (Rank Ordered)(Rank Ordered) PhasesPhases

Whole Class Answer Checking / IRE

27.627.6 Whole Class Lecture/Whole Class Lecture/MonologueMonologue

32.032.0

Wh l Cl L t /Wh l Cl L t / 20 520 5 Wh l Cl AWh l Cl A 22 022 0Whole Class Lecture / Whole Class Lecture / MonologueMonologue

20.520.5 Whole Class Answer Whole Class Answer Checking/ IREChecking/ IRE

22.022.0

Whole Class Elicitation and Whole Class Elicitation and DiscussionDiscussion

6.36.3 Whole Class Elicitation and Whole Class Elicitation and DiscussionDiscussion

6.26.2

Whole Class Demonstration/Whole Class Demonstration/ActivityActivity

3.33.3 Whole Class Demonstration / Whole Class Demonstration / ActivityActivity

2.72.7

Choral RepetitionChoral Repetition 3.03.0 Choral RepetitionChoral Repetition 1.41.4Subtotal 60.7 Subtotal 62.3

Individual SeatworkIndividual Seatwork 18.318.3 Individual SeatworkIndividual Seatwork 17.617.6Small Group WorkSmall Group Work 12.112.1 Small Group WorkSmall Group Work 10.210.2Small Group WorkSmall Group Work 12.112.1 Small Group WorkSmall Group Work 10.210.2Student DemonstrationStudent Demonstration 5.25.2 Student DemonstrationStudent Demonstration 5.25.2TestTest‐‐TakingTaking 1.61.6 TestTest‐‐TakingTaking 1.61.6Laboratory/Experiment Laboratory/Experiment  2.22.2 Laboratory/ExperimentLaboratory/Experiment 1.11.1

76

Total 100 Total 100

N (No. of Lessons)N (No. of Lessons) 591591 N (No. of Lessons)N (No. of Lessons) 578578

Instructional Strategies: Mean Scores / SDInstructional Strategies: Mean Scores / SD

Secondary 3 Secondary 3 MMathematicsathematics

Secondary 3 Secondary 3 EnglishEnglishgg

MeanMean（（11‐‐55））

SDSD MeanMean（（11‐‐55））

SDSD ώ2

Direct Instruction 

3.79  .79  3.71  .74  ‐.004

Traditional teaching 

3.69 .64  3.38  .67  .355

A th ti 2 96 78 3 16 73 037Authentic pedagogy 

2.96  .78  3.16  .73  .037

ώώ22Omega Sq scale (ώOmega Sq scale (ώ22) ) <.01 <.01 –– Insignificant  ;  .011 to .06 Insignificant  ;  .011 to .06 –– Small to Small to moderate; .061 to .16 moderate; .061 to .16 –– Moderate to Large  Moderate to Large  

>.16 >.16 –– Large;  Large;   ≥.03 threshold value≥.03 threshold value77

Three Pedagogies: Secondary 3 Mathematics and English(Student Survey)g ( y)

S3 MathS3 

English F‐ratio p ώ2(1‐5)

English(1‐5)

F ratio p ώ

Traditional Teaching (a=.76, .75) 3.69 3.38 136.55 <.01 .355

F f H k 4 33 3 76 50 03 < 01 166Frequency of Homework 4.33 3.76 50.03 <.01 .166

Memorization 4.03 3.21 257.79 <.01 .511

Drill and Worksheets 3.84 3.48 69.24 <.01 .217

Lecture 3.61 3.78 10.80 <.01 .038

Textbook Focus 3.61 2.91 105.82 <.01 .299

Frequency of Exams 2.40 2.14 28.29 <.01 .100q y

Omega Sq scale (ώOmega Sq scale (ώ22) ) <.01 <.01 –– Insignificant  Insignificant  

.011 to .06 .011 to .06 –– Small to moderateSmall to moderate.061 to .16 .061 to .16 –– Moderate to Large  Moderate to Large  

>.16 >.16 –– LargeLarge≥.03 threshold value≥.03 threshold value 7878

Three Pedagogies: Secondary 3 Mathematics and Three Pedagogies: Secondary 3 Mathematics and English (Student Survey)English (Student Survey)English (Student Survey)English (Student Survey)

S3 S3 

English F ratio p ώ2Math(1‐5)

English(1‐5)

F‐ratio p ώ2

N (number of students) 4,690 4,547

Direct Teaching (a=.92, .91) 3.79 3.76 .05 .82 ‐.004

Maximizing Learning Time 4.02 4.05 .21 .65 ‐.003g g

Review 3.74 3.54 10.99 <.01 .039

Questioning 3.44 3.49 .73 .39 ‐.001

Structure and Clarity 3 73 3 73 00 98 004Structure and Clarity 3.73 3.73 .00 .98 ‐.004

Feedback 3.65 3.72 1.67 .20 .003

Omega Sq scale (ώOmega Sq scale (ώ22) ) <.01 <.01 –– Insignificant  ;  .011 to .06 Insignificant  ;  .011 to .06 –– Small to moderate; .061 to .16 Small to moderate; .061 to .16 –– Moderate to Large  Moderate to Large  

>.16 >.16 –– Large;  Large;   ≥.03 threshold value≥.03 threshold value 7979

Three Pedagogies: Secondary 3 Mathematics and Three Pedagogies: Secondary 3 Mathematics and E li h (S d S ) i dE li h (S d S ) i dEnglish (Student Survey)… continuedEnglish (Student Survey)… continued

S3 MathS3 

English F‐ratio p ώ2(1‐5)

English(1‐5)

F ratio p ώ

Authentic Pedagogy (a=.89, .90) 2.96 3.16 10.39 <.01 .037

D th f U d t di 3 38 3 32 1 70 19 003Depth of Understanding 3.38 3.32 1.70 .19 .003

Quality of Homework 3.34 3.19 10.98 <.01 .039

Classroom Discussion 3.26 3.48 14.70 <.01 .053

Meta‐language 3.11 3.23 4.35 <.05 .013

Criticality and Creativity 3.11 3.20 2.87 .09 .008

Integration of Knowledge across 2 99 3 15 10 23 < 01 036

g gDisciplines

2.99 3.15 10.23 <.01 .036

Connectedness to Real World 2.97 3.22 18.75 <.01 .067

Significance and Meaningfulness 2.94 3.18 17.51 <.01 .063Significance and Meaningfulness 2.94 3.18 17.51 .01 .063

Frequency of Project Work 1.14 1.54 73.77 <.01 .228

8080

• Intriguingly and critically the deployment ofIntriguingly, and critically, the deployment of these instructional strategies is (up to a point) complementary rather than orthogonalcomplementary rather than orthogonal …

8181

An Hybridic Pedagogy: Bivariate Correlations MatrixAn Hybridic Pedagogy: Bivariate Correlations Matrix

Secondary 3 MathSecondary 3 English

Direct Teaching

TraditionalTeaching

Authentic pedagogy

Direct Teaching

TraditionalTeaching

Authentic pedagogy

Directteaching

1 .62** .65 ** 1 .58** .68**

Traditionalteaching

.62** 1 .59** .58** 1 .69**

Authentic pedagogy

.65** .59** 1 .68** .69** 1

8282

Distribution of Epistemic Authority by Subject (Secondary 3) Distribution of Epistemic Authority by Subject (Secondary 3) (Classroom Observation Data)(Classroom Observation Data)

% of LessonsSubject All

Science Math English Social Studies

Student 6.2 2.7 25.3 9.8 11.3

Teacher 86.9 90.6 60.2 66.9 76.0

Test 0 7 8 9 6Test .0 .7 .8 .9 .6

Textbook  2.7 2.8 1.3 13.7 4.8

Internet .0 .3 .4 2.8 .8

Data 1.6 .0 .0 .0 .3

Mass Media .1 .2 .0 1.9 .5

Worksheet 2.5 2.1 8.0 1.8 3.8

Others .0 .6 4.1 2.1 1.8

N (Number of Lessons) 93 129 125 101 448N (Number of Lessons) 93 129 125 101 448

83

Student Voice (Secondary 3) Student Voice (Secondary 3) (Classroom Observation Data)(Classroom Observation Data)

Mean ScoreSubject

(Mean Scores; Range 0‐3)All F p w2

Sc Ma El SS

Primary 5Primary 5

Student V i

1.1 .4 1.0 1.6 .9 15.688 .000 .203Voice 

Secondary 3

Student Voice 

.9 .4 1.0 .9 .8 8.541 .000 .090

EncouragementEncouragement ScaleScale:: 00=Strongly=Strongly Discouraging,Discouraging, 11=Discouraging,=Discouraging, 22=Encouraging,=Encouraging, 33=Strongly=Strongly EncouragingEncouragingStudentStudent VoiceVoice ScaleScale:: 00=Nil,=Nil, 11=A=A Little,Little, 22=Sometimes,=Sometimes, 33=Almost=Almost AlwaysAlways

8484

Open and Closed Questions in Primary 5 SubjectsOpen and Closed Questions in Primary 5 Subjects (Transcribed Audiotape Corpus)

Open Closed Total  No. Question Types

P5 S bj t

of QuestionsNo. % No. %

P5  Subjects

English 72 12.3 511 87.7 583

Science 68 10.7 565 89.3 633Science 68 10.7 565 89.3 633

Mathematics  24 2.3 1044 97.7 1044

Social Studies 107 26.8 399 73.2 506

Total 271 12% 2,495 88% 2,766

85

Intellectual Quality of Knowledge Work in Primary 5 and Secondary 3Intellectual Quality of Knowledge Work in Primary 5 and Secondary 3(Classroom Observation Data)(Classroom Observation Data)

Knowledge ClassificationPrimary 5

(0 3)Secondary 3

(0 3)Knowledge Classification (0‐3) (0‐3)

Depth of Knowledge

Basic/Fact/Rote 2.4 2.3

Procedural/How To 0.7 1.1

Advanced Concept 0.1 0.2

Knowledge Manipulation & Productiong p

Reproduction 2.3 2.1

Interpretation 0.6 0.7

Application 0 4 0 5Application 0.4 0.5

Generation of Knowledge New to Students 0.1 0.2

Assessment of Knowledge Claims

Knowledge presented as Truth 2.6 2.5

Comparison 0.2 0.3

Critique 0.1 0.2

Number of lessons 591 578 86

Two Pedagogies: Exploratory Factor Analysis of Classroom Knowledge PracticesTwo Pedagogies: Exploratory Factor Analysis of Classroom Knowledge Practices

Factor LoadingsBasic Knowledge Transmission

Complex KnowledgeConstruction

Knowledge Presented as Truth  .982

Reproduction .945

Single Discipline 900Single Discipline .900

Basic/Factual/Rote .850

Specialized Language .580

Procedural Knowledge .425

Critique .817

Comparison .750

Advanced Concept .644

Generation of Knowledge New to Students .620

Interpretation 588Interpretation .588

Application .573

Cronbach’s Alpha .898 .742

Mean 2.0 0.4

Correlation  .378**

Explained Variance 58 % 87

Subject and level differences in enacted curriculum

Factor Mean Scores

3

2

2.5

3

ores P5-F1

1

1.5

Mea

n Sc

o P5-F2S3-F1S3-F2

0

0.5

S i M th E li h S i l

M

Science Math English SocialStudies

Subject

88F1: Basic Knowledge TransmissionF2: Complex Knowledge Construction

Intellectual Quality of Teachers’ Assessment Tasks

Criteria Grade 5 Grade 9n = 210 n = 136

Mean SD Mean SD

Depth of Knowledge

Factual Knowledge 3 36 80 3 30 82Factual Knowledge 3.36 .80 3.30 .82

Procedural Knowledge 2.61 1.03 2.54 .95

Advanced Concepts 1.69 .80 1.83 .93

Knowledge Manipulation & ProductionKnowledge Manipulation & Production

Reproduction 2.94 1.03 3.26 .92

Organization, Interpretation,or Evaluation of Information

2.27 .83 2.35 .87

Sustained Writing 2.18 1.16 2.49 1.06

Application/Problem‐Solving 1.94 .90 2.04 .90

Generation/Construction of Knowledge   New to Students 1.68 .91 1.50 .83

A t f K l d Cl iAssessment of Knowledge Claims

Presentation of  Knowledge as Truth  3.10 .99 3.48 .81

Compare and Contrast Knowledge 1.73 .83 1.82 .88

Critique of Knowledge 1 54 81 1 35 77Critique of Knowledge 1.54 .81 1.35 .77

Authentic Knowledge

Connections to the Real World beyond the Classroom 1.92 1.02 1.52 .8689

h ki d f d d 21What kind of pedagogy do 21st

century institutional contexts yrequire?

90

Knowledge ClassificationPrimary 5

(0 3)Secondary 3

(0 3)

Well, to begin....Well, to begin....

Knowledge Classification (0‐3) (0‐3)

Depth of Knowledge

Basic/Factual ?? ??Procedural/How To ?? ??Advanced Concept ?? ??Knowledge Manipulation & Productiong p

Reproduction ?? ??Interpretation ?? ??Application ?? ??Application ?? ??Generation of Knowledge New to Students ?? ??Assessment of Knowledge Claims

Knowledge presented as Truth ?? ??Comparison ?? ??Critique ?? ??

Number of lessons 591 578 91

Deep Understanding

“To understand is to make connections and bind together our knowledge into something that makes sense of things (whereas without understanding we might see only unclear, isolated, or unhelpful facts). But the word also implies doing, not just a mental act: a performative ability lies at the heart of p yunderstanding … To understand is to be able to wisely and effectively use – transfer – what we know, in context; to apply knowledge and skill effectively in realistic tasks and settingsknowledge and skill effectively, in realistic tasks and settings. To have understanding means that we show evidence of being able to transfer what we know. When we understand, we have a fluent and fluid grasp, not a rigid, formulaic grasp based only on recall and ‘plugging in.’”

Wiggins and McTighe, Understanding by Design, 2nd ed., pp. 7-8.92

Deep Understanding -2

a focus on conceptual relationships and frameworks (“connections”)a focus on conceptual relationships and frameworks ( connections ) rather than on isolated clumps or bits of information or on information processing.

involves significant levels of cognitive agency or “performances of understanding” reflecting the ability to employ knowledge / understanding in a variety of complex cognitive tasks rather than merely “know” isolated anda variety of complex cognitive tasks rather than merely know isolated and inert bits of information

E i i nin kin iti f th thEngaging in meaning making or cognitive performances rather than nouns denoting passive knowledge or inert information.

TTransfer of learning “to apply knowledge and skill effectively, in realistic tasks and settings. To have understanding means that we show evidence of being able to transfer what we know.”

93

These pedagogical features (in red) pointThese pedagogical features (in red) point towards a disciplinary pedagogy….

94

A Disciplinary Pedagogy?A Disciplinary Pedagogy?

A pedagogy that has strong disciplinary features is one that A pedagogy that has strong disciplinary features is one that

seeks to cultivate, in developmentally appropriate ways, student seeks to cultivate, in developmentally appropriate ways, student understandings, skills, norms, dispositions, identities and metaunderstandings, skills, norms, dispositions, identities and meta--cognitive capacitiescognitive capacities thatthatcognitive capacities cognitive capacities thatthat

permit students to participate effectively in distinctive domainpermit students to participate effectively in distinctive domain--specific disciplinary practices and conversations in classrooms specific disciplinary practices and conversations in classrooms

i d i t i iti f tii d i t i iti f ti f df dorganized as epistemic communities of practiceorganized as epistemic communities of practice focused onfocused on

the generation, validation, communication and application of the generation, validation, communication and application of domaindomain--specific disciplinary knowledge appropriate to academic andspecific disciplinary knowledge appropriate to academic anddomaindomain specific disciplinary knowledge appropriate to academic and specific disciplinary knowledge appropriate to academic and nonnon--academic institutional settings.academic institutional settings.

9595

A disciplinary pedagogy is one that isA disciplinary pedagogy is one that isA disciplinary pedagogy is one that isA disciplinary pedagogy is one that isLESS AND MORE

Academic / scholastic Practical

Arcane Institutionally relevant

Inauthentic Authentic

Abstract AgenticAbstract Agentic

Boring Engaging/motivating

Passive Active

Teaching Learning

Memorization (Deep) Understanding

I diff i i i N i f i i iIndifferent to epistemic virtues Nurturing of epistemic virtues

Indifferent to identity formation Identity formation

Transmission of knowledge to students Co-construction of knowledge with others g g

Knowledge consumption and display Knowledge production

Existentially irrelevant Existentially valuable9696

Fred Newmann: Authentic Knowledge and Fred Newmann: Authentic Knowledge and Di i li A hiDi i li A hiDisciplinary AchievementDisciplinary Achievement

“The term authentic achievement thus stands for intellectual “The term authentic achievement thus stands for intellectual li h t th tli h t th t th hilth hil i ifi ti ifi t dd i f li f l hhaccomplishments that are accomplishments that are worthwhileworthwhile, , significantsignificant, and , and meaningfulmeaningful, such as , such as

those undertaken by successful adults: scientists, musicians, business those undertaken by successful adults: scientists, musicians, business entrepreneurs, politicians, crafts people, attorneys, novelists, physicians and entrepreneurs, politicians, crafts people, attorneys, novelists, physicians and so on. With children we are concerned with a more restricted conception of so on. With children we are concerned with a more restricted conception of

hi t th t b li h d i h l F t d thi t th t b li h d i h l F t d tachievement, one that can be accomplished in schools. For students, we achievement, one that can be accomplished in schools. For students, we define authentic achievement through three criteria critical to significant define authentic achievement through three criteria critical to significant intellectual accomplishments: intellectual accomplishments: construction of knowledge, disciplined construction of knowledge, disciplined inquiry, and the value of achievement beyond the schoolinquiry, and the value of achievement beyond the school.” .”

Fred Newmann and Associates, Fred Newmann and Associates, Authentic Achievement: Restructuring Schools for Authentic Achievement: Restructuring Schools for Intellectual QualityIntellectual Quality. San Fransisco, Jossey Bass, 1996, pp. 22. San Fransisco, Jossey Bass, 1996, pp. 22--24.24.

9797

Knowledge Building: Bereiter & Scardamalia Knowledge Building: Bereiter & Scardamalia

“Knowledge building differs from the other approaches by emphasizing “Knowledge building differs from the other approaches by emphasizing conceptual artefacts (theories, designs, plans, histories, etc) as products, tools, conceptual artefacts (theories, designs, plans, histories, etc) as products, tools, and objects of inquiry... Activities such as model building, conducting and objects of inquiry... Activities such as model building, conducting

i d d i i d i i f b di d d i i d i i f b dexperiments, and producing reports are carried out in service of a broader experiments, and producing reports are carried out in service of a broader effort to produce some innovation or advance a knowledge frontier… effort to produce some innovation or advance a knowledge frontier…

In educational applications students are engaged in design in all phases and atIn educational applications students are engaged in design in all phases and atIn educational applications, students are engaged in design in all phases and at In educational applications, students are engaged in design in all phases and at all levels of the knowledgeall levels of the knowledge--building enterprise: defining problems, advancing building enterprise: defining problems, advancing initial ideas, using whatever resources and inquiry possibilities are available to initial ideas, using whatever resources and inquiry possibilities are available to improve those ideas, reformulating problems as the knowledge building improve those ideas, reformulating problems as the knowledge building advances, and presenting results ... Thus, it could be said that instead ofadvances, and presenting results ... Thus, it could be said that instead ofadvances, and presenting results ... Thus, it could be said that instead of advances, and presenting results ... Thus, it could be said that instead of assimilating designassimilating design--mode activity into the academic curriculum the academic mode activity into the academic curriculum the academic curriculum is assimilated into design mode…”curriculum is assimilated into design mode…”

B i & S d li “Ed i f h K l d A D iB i & S d li “Ed i f h K l d A D i ddBereiter & Scardamalia, “Education for the Knowledge Age: DesignBereiter & Scardamalia, “Education for the Knowledge Age: Design--centred centred Models of Teaching and Instructions.” In Alexander and Winne, eds., Models of Teaching and Instructions.” In Alexander and Winne, eds., Handbook of Educational PsychologyHandbook of Educational Psychology, 2006., 2006.

9898

Sociocultural learning theorySociocultural learning theory---- Ford and FormanFord and Forman

“What aspects of an academic discipline’s practices should be considered “What aspects of an academic discipline’s practices should be considered fundamental and therefore central to sociocultural learning objectives?fundamental and therefore central to sociocultural learning objectives?fundamental and therefore central to sociocultural learning objectives? fundamental and therefore central to sociocultural learning objectives? Participating in any practice means acting appropriately, and “appropriateness” Participating in any practice means acting appropriately, and “appropriateness” is judged as whether or not one’s actions “mesh” with those of other individuals is judged as whether or not one’s actions “mesh” with those of other individuals in the practice for forwarding the community’s aims. In any academic discipline, in the practice for forwarding the community’s aims. In any academic discipline, the aim of the practice is to build knowledge or, in other words, to decide whatthe aim of the practice is to build knowledge or, in other words, to decide whatthe aim of the practice is to build knowledge or, in other words, to decide what the aim of the practice is to build knowledge or, in other words, to decide what claims “count” as knowledge, distinguishing them from those that do not. claims “count” as knowledge, distinguishing them from those that do not. Deciding what counts as knowledge implies authority, and thus the raison d’etre Deciding what counts as knowledge implies authority, and thus the raison d’etre of academic practices is how theses practices ground disciplinary authority. of academic practices is how theses practices ground disciplinary authority. In In regard to educational concerns, therefore, these practices that ground authorityregard to educational concerns, therefore, these practices that ground authorityregard to educational concerns, therefore, these practices that ground authority regard to educational concerns, therefore, these practices that ground authority for deciding what counts as knowledge in disciplines are the ones that students for deciding what counts as knowledge in disciplines are the ones that students should engage in classrooms if their participation is to be authentic and they are should engage in classrooms if their participation is to be authentic and they are to learn fundamentally what practice is about.”to learn fundamentally what practice is about.”

9999

Ford and Forman Ford and Forman -- 22

For Ford and Forman, then, learning disciplinary knowledge requires For Ford and Forman, then, learning disciplinary knowledge requires participatingparticipatingin those authentic disciplinary practicesin those authentic disciplinary practices that generate, evaluate and report disciplinary that generate, evaluate and report disciplinary knowledge rather than just learning the results of these practicesknowledge rather than just learning the results of these practices “From a“From aknowledge rather than just learning the results of these practices. knowledge rather than just learning the results of these practices. From a From a sociocultural perspective, teaching students about any disciplinary practice sociocultural perspective, teaching students about any disciplinary practice requires that they be engaged authentically in the activities of individual agents requires that they be engaged authentically in the activities of individual agents and group members that are necessary for a practice to achieve its aims” and group members that are necessary for a practice to achieve its aims”

WhatWhat disciplinarydisciplinary practicespractices thenthen shouldshould studentsstudents participateparticipate in?in?PracticesPractices thatthat generategenerate knowledgeknowledge claimsclaimsPracticesPractices thatthat validatevalidate knowledgeknowledge claimsclaims (ie(ie..,, establishestablish thethe authoritativeauthoritative truthtruth valuevalue ofofgg (( ,,knowledgeknowledge claims)claims)PracticesPractices thatthat communicatecommunicate knowledgeknowledge claimsclaims intersubjectivelyintersubjectively

F d d F “R d fi i Di ipli L i i Cl C t t ”F d d F “R d fi i Di ipli L i i Cl C t t ”Ford and Forman, “Redefining Disciplinary Learning in Classroom Contexts,” Ford and Forman, “Redefining Disciplinary Learning in Classroom Contexts,” Review of Educational Research,Review of Educational Research, 2006, p. 3. 2006, p. 3.

100100

Disciplinary pedagogies…Disciplinary pedagogies…Disciplinary pedagogies…Disciplinary pedagogies…

Instrumentally valuable: Instrumentally valuable: helping young people to successfully helping young people to successfully yy p g y g p p yp g y g p p yparticipate in major 21participate in major 21stst century institutions (institutional century institutions (institutional practicespractices

h l i i di id l l d f l li i ifih l i i di id l l d f l li i ifihelping individuals to lead successful lives in specific helping individuals to lead successful lives in specific institutional settingsinstitutional settings

Intrinsically valuableIntrinsically valuable: valuable to people as persons rather than : valuable to people as persons rather than as role playersas role players

Helping persons as moral agents frame and lead a life that is Helping persons as moral agents frame and lead a life that is existentially meaningful and worthwhile to themselves and existentially meaningful and worthwhile to themselves and othersothersothers others But that’s another story for another dayBut that’s another story for another day

101101

THANK YOUTHANK YOUTHANK YOUTHANK YOU

102102