How many students are paying attention? (Senegal)

What can you learn from visits to schools and classrooms?

The revelation of the secret and mysterious causality chains

Helen AbadziSenior Education SpecialistFTI Secretariatc/o World BankMay 12, 2010Habadzi@worldbank.org

APEIE workshop

Inside inputs get converted into information

Project inputs

BuildingsTextbooksSalariesTrainingSchool grantsSchool plansCommunity management

classroomactivities

Information processing

Impact evidence

EnrollmentsDropoutRepetitionSkillsLearning outcomes

Data collection from the “black box” ?? !!

This session will present variables that can be observed

Useful for you personally As well as for school surveys

How well was a treatment implemented?Find out

• Physical inputs – Building, maintenance, furniture, consumables– Radios, lab equipment, maps?– Textbooks

• Training inputs– For teachers, principals, administrators

• Instructional delivery to students• Management committees• School budgets• School planning

Building condition may affect implementation of other components

Poor mixture of cement and sand

Structural problemsLack of maintenance

Structural defects in a school

Computers and labsHow often do students use them?

With what software?

Teacher and management training coursesAsk:Did teachers and principals receive the

training that had been financed?How many days? What did they do?

Lectures, activities?What can they remember from it?

Please ask for three items they recallDo they use the training information in

classroom or in school?Ask: How? Where?

School feedingNutritional supplements

• Do students in fact get fed what the government thinks?

• Do the schools get enough?– What kinds?

• How often are they delivered?

• When were they last delivered?

• Take a look at them, see expiration date

Management improvement componentsFind out:School grants & budgets

What did the schools buy? Did not buy? In what ways (if any) did the grant purchases increase the

amount of learning?Community management

How often do they meet? What % come? What is the added value of their presence?

Buildings, money, monitoring? Do they just approve what principal decides?

School planning What did school decide was missing? What did they do about it? Do you see deficiencies that the school staff did not find?

Do school management committees perform the expected functions?

Where are the textbooks?Look for themNowhere?In the students’ hands?In the classroom?In the library?How many students vs. textbooks in a class?

If textbooks are insufficient, they stay on the shelves

Class time spent in blackboard copying

Instructional delivery to students

How much actual teaching do you see?

Scheduling visits to schools

Sampling schools – important issuesOmitting certain schools from surveys

alters the sampleIf more than 5% of the schools are changed

Visit unannounced if possible !Perhaps not reveal to local authorities when

schools will be visited

Classroom observation experiencefor youTake a look inside classes if possible without

being seen (e.g. from window)What were they doing before you went in?

Sit down for 5-10 minutes Take video or audio evidenceTeachers may try to teach better in front of

you but may not know how!After a while everyone may forget about you.

Which variables are worth measuring from classrooms events?

Use as framework the rules of how people learn

Look at:Is time spent for instruction?Are all students engaged in learning tasks?Does teacher know the subject?Is teacher using methods that help

students remember?How many interact with the teacher

Look around the class and think:Can the students learn the expected subjects in this class?

How many of the enrolled students present?

How many interact with the teacher, participate?

How is the class time spent?Activities carried out?

Are students kept busy all the time? Are students processing information or just

waiting?

Problems identified in many classroomsInstruction may not be going onThey may be unable to see the blackboardMay spend much time copyingMay repeat without understandingThey may receive no feedbackBe unoccupied most of the time Grade 1-2 textbooks may not really teach the

poor how to read

How much of the time that governments pay for is used for instruction?

Class: When financing is converted to information

In a class hour, students must retain as much info as possible and recall it effortlessly when needed

Time use has been measured many timesStallings classroom snapshot

Only a small fraction of the time that a government budgets is converted into information

Class time as allotted by a government (e.g., 200 days, 1000 teaching hours)

Remaining after school closures (strikes, weather, teacher training, extra holidays)

Remaining after teacher absenteeism and tardiness

Remaining after student absenteeism

Class time devoted to any learning task

Learning time relevant to curriculum

Teacher off task (India)

Legal teacher absenteeism:Teacher’s book from rural HondurasHow many days did this class study?

Students unoccupied, waiting for the bell to ring (most of them illiterate)

Brazil – group work?

Brazil – group work?

Teacher absenteeism and time use in Maputo: Schools constantly in recess

To remember and use information, students must:Receive itPractice it, contemplate itHave prior knowledge on which to fit itRead fluently to learn from booksKnow the official language well!do math fluently to solve complex problems

High-quality schools: Offer activities that create complex cognitive networks

Students recite + read long texts+ manipulate + collect real-world samples + answer questions connecting various items + derive new conclusions from data + solve problems + practice for fluency + generalize into various circumstancesSpend time in “active learning”

Mexico – efforts to reclassify

Signs that the students will probably remember the informationGroup or even individual work with students

concentratedActivities of students reading, discussing

not merely verbally repeatingWriting material that is not simply copyingTeacher uses aids like flash cardsTeacher monitors individual student work

Gives feedback Teacher going towards the back,

addressing individual students who did not volunteer to answer a question

Good: Group work, engaged studentsbut time must be kept strictly

•

Poor-quality schools:may teach items in series

with few connections

The heroes of the revolution are…

The books of the bible are….

2x2=4, 2x3=6, 2x4=8, 2x5=10….

Students may just recite or listen…

The principles of constitutional law are…

How many are paying attention?One student recites, rest unoccupied

Low-quality schools High quality schools

Recife, Brazil: seatwork – teacher not monitoring

Student attendance and participation

Determine whether students will learn

Check:How many of the enrolled students are present?We all hear about large classes

If you go towards the end of the school year will you find them?

Students may enroll but attend rarely, particularly when they are illiterate or do not know the material.

Is there space in a classroom for everyone enrolled?There may only be enough space for the

regular studentsWhat excuses do you get?

Students falling behind are often absent• Bank documents say that classes are large…• 19 of 45 students in grade 4, outside Nampula,

Mozambique

Bangladesh – student absenteeism

Look at students’ notebooks

• How much is written?Too much: no textbooks

Just 2-3 pages? No classes or frequent absences

Check for “hidden dropout”:How many of the students are actually involved in a class?Ask the teacher: is s/he up to date with the

curriculum that must be taught?If so, observe how many students actually

follow teachers’ instructions, answer questions, seem involved

The teacher may say that s/he is up to date, but may be only teaching 3-4 studentsRest may be illiterate and uninvolved

“Hidden dropout?”:Teachers in Nepal interacting

mainly with the front of the class

Do teachers know sufficient material to teach?See material on the blackboard

Does it make sense?Example: Students who don’t know letter values cannot

learn them from others’ fast reading

Rural Mozambiquerura

What impresses you most about this scene?

Do these students discriminate among letters of the fuzzy blackboard from this distance?

Lack of textbooks translates into loss of time at all levels

• Due to a lack of knowledge and materials, teachers do very few activities – But they can follow a textbook

• Teacher boredom:How many of us would spend 20 years in blackboard transcription?

Dictation in Burkina FasoIt takes 30 seconds to read and 7+ minutes to

dictate this phrase:

“Les industries Francaises ont connu un développement considerable mais rencontrent des difficultés dans les ressources.. »

10tth grade math– private school

Can the students read fluently?

Reading often not taught specificallyMethods are often very inefficient

Informal way to find out how many are literateTake book, go 3-4 pages aheadAsk a student to readMark the start of 1 minute by your watch,

start a paragraph, mark end in one minuteTake photo or count words on the spot

Ask 2-3 simple questions about what they read

What is their approximate reading speed?

Can students read their notes to you?Illiterate students writing “art”

Short-term memoryCrucial for reading comprehension

Long-term memory

12 seconds at mostAbout 7 items4 pictures

Sound (OLE2)

35 words per minute

Quelle est la langue d’instruction?Les langues Africaines ont orthographie régulière Peuvent être automatisées dans quelques mois

La fluidité transfert aux autres langues qui ont la même écriture

Grade 1 textbooks – critical for subsequent years

How should efficient textbooks instruct in reading?

Burkina Faso grade 1 book

Are there dictionaries in local languages?

Activity at entry in class Brazil 2002

State Ceará Rio Grande de Norte

Alagoas Goias São Paulo

Copying 6 40%

6 33%

2 22%

8 (2 secondary) 17%

Text production 1 7%

3 17%

3 33%

4 28%

2 4%

Reading lesson in class

2 13%

1 6%

1 11%

1 2%

No activity 1 7%

5 (2 waiting for bell) 28%

1 11%

8 17% (some

teachers absent) Teacher busy with 2-3 students only

1 7%

1 11%

4 8%

Teacher teaching whole class

2 13%

1 11%

6 43%

10 22%

Group practice 1 7%

2 11%

3 21%

4 8%

Art and Play 1(scheduled activity)

7%

1 (play with letters) 6%

1 (play with letters)

11%

1 (drawing) 7%

8 (during subject matter)

17%

No. classes 15 18 9 14 45

No schools 8 6 7 5 8

Apparently instructional activities (some suboptimal)

34% 45% 48% 92% 40%

Sample form to record instructional data from school

visitsSchool Grade Student & teacher

attendance

What was the class doing at the moment of mission entry? (How likely is this activity to engage all students in the task?)

How many sampled students can read? How well? Comments

Teac

her

pres

ent?

No.

stud

ents

en

rolle

d

No.

stud

ents

pr

esen

t

No.

of t

extb

ooks

in

sigh

t

Teac

her

teac

hing

en

tire

clas

s

Indi

vidu

al o

r ch

oral

ans

wer

s no

. res

pond

ing

1-2

stud

ents

at

blac

kboa

rd

Stud

ents

doi

ng

seat

wor

k al

one

Stud

ents

wor

king

in

gro

ups

Cla

ss c

opyi

ng

from

bla

ck

Boar

d O

ther

teac

hing

ac

tivity

: art

, pla

y si

ngin

g, e

xam

, m

ultip

les

Teac

her

chat

ting

or d

iscip

linin

g st

uden

ts

N

o cl

ear

inst

ruct

iona

l ac

tivity

A

ppro

x %

of

stud

ents

off

task

Teac

her

estim

ate

of n

umbe

r or

%

1. C

ould

har

dly

read

lette

rs,

wor

ds, o

r no

t at

all

2.B

arel

y de

code

d,

flat v

oice

, no

com

preh

ensio

n

3. R

ead

abou

t a

wor

d pe

r se

c.,

som

e st

umbl

ing

4.

Rea

d flu

ently

w

ith a

ppro

pria

te

expr

essi

on

How to measure and quantify classroom variables?Need for summative scales, “low-inference”

instrumentsMany observation instruments exist

On the minute, Flanders, Virgilio, CLASSClassroom observation instruments

DifficultcomplexImperfect

Stallings modified classroom snapshot

Low inferenceEasier to communicate to less educated staffFor lower-level grades, deprived environments

Where time gets wasted

Stallings 5-minute interaction – to be updated?

Snapshot 10 I : I ndividual T: Teacher S: Small Gp

L: Large Gp E: Entire Gp

Number of Adults Present Number of Students Present MATERIALS

ACTIVITY None Books Notebook Chalk

board Manipul

ative Visual/ Computational Aids

Co-operative

Reading Aloud T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E

I nstruction/ Demonstration/ Lecture T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E Discussion T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E

Practice/ Drill T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E Kinesthetic/ Projects T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E

Written Assignments/ Seatwork T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E Copying T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E

Assignments T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E Other T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E Social I nteraction T 1 S L E I 1 S L E

Student Uninvolved I 1 S L E Being Disciplined T 1 S L E Classroom Management T 1 S L E I 1 S L E Adult Social I nteraction/ Adult Uninvolved T Adult Management T Teacher out of Room T

Training and future useClassroom snapshot takes 5 days trainingTrainees learn how to spot time and wastage

Instructional time loss as a monitoring indicatorWhat % of the time bought by governments for

students is used in learning activities?Ghana 39% Pernambuco (Brazil) 63%Dominican Republic 65%Tunisia 79%

What % of the curriculum covered each year?What % of objectives do students master?

What test scores should we expect?What % of students in a class interacted with a

teacher?

Instructional Time Use in the participating countries

Pernambuco (Brazil)

Ghana

Morocco

Tunisia

School Year (Days) 200 197 204 190 School Closures (Days) 4.79 3.17 1.38 5.15 Days after Closures 195.21 193.83 202.62 184.85 Teacher Absence (Days) 12.76 43.01 13.36 11.55 Teacher Delays 5.50 39.75 6.94 1.27 Early Class Dismissals 2.31 2.43 6.68 1.22 School Year (Days left after losses) 174.65 108.6 175.6 170.8 % Year available for teaching 87.3% 55.1% 86.1% 89.9% Engagement Rate in Interactive or Passive Classroom Tasks 72.1% 70.2% 82.6% 86.7% School Days Devoted to Learning 125.9 76.3 145.1 148.1 School Year % Spent Engaged in Learning Tasks 63.0% 38.7% 71.1% 77.9% Student off Task Rate 19.3% 21.1% 9.2% 9.9% Student Absence (Days) 7.82 9.04 4.30 3.35 Student Delays (Number of Times) 5.64 10.61 5.187 2.63

Percentage of Time Spent in Instructional Tasks

Instructional Strategy % of time

U.S Classroom Criteria

Brazil Ghana Morocco Tunisia

Interactive instruction 50% or more 52.4 59.9 62.8 61.2

Oral Reading 6.7 8.7 15.7 15.3

Teaching, Explanation 32.8 19.9 26.7 27.9

Discussion 6.3 24.1 6.6 6.2

Practice Drill 1.4 6.5 12.3 11.3

Passive instruction 35% or less 19.6 10.3 19.9 25.6

Seatwork 16.3 7.4 14.8 22.9

Copy 3.0 2.9 5.0 2.7

Total Instructional Time 72.1 70.2 82.6 86.7

Organizing/Management 15% or less 27.9 28.0 17.8 13.3

Student Off Task Rate 6% or less 19.3 21.1 9.2 9.9

73

Now how black is the box?

Thank you for your attention