An investigation into the use of assessment materials in schools for pupils with Social, Emotional and Behavioural Difficulties

An online survey of current practice

Gavin Weedy: Teacher at Goldwyn School, Ashford, Kent

Why bother with this survey? “Without the careful identification, classification, and

selection that should be a part of good assessment, social behaviour interventions are likely to be haphazard and disorganized at best and ineffective at worst. Clearly, practitioners and researchers who are serious about understanding and dealing effectively with children’s social skills and related social–behavioral issues must provide the proper foundation for their work through carefully planned assessment activities.” (Merrell, 2001)

My role As a practitioner I am directly involved in the

assessment of pupils with SEBD in a school

I am in contact with many colleagues in other settings

Driven by providing the best outcomes for the pupils and thence the school.

I am interested in spreading the word about best practice

Methodology of survey Using a Critical Realist paradigm:

Looks for causal mechanisms rather than social laws –the same causal powers can generate different outcomes

Allow for scope for change as research develops

Allows for both a Quantitative approach, such as “How many schools are using X, Y or Z assessment?”

And a Qualitative approach, such as “Why are they using X, Y or Z assessment?”

Design of the questionnaire used in the survey

Trialled with colleagues

10 simple questions – some with “yes” or “no” answers to get basic quantitative data

Others with scope for more in depth to generate qualitative answers

Quick to complete

Online for ease of completion

Potential for computer analysis

How was the survey carried out? Online survey of day schools, residential schools and

Pupil Referral Units.

Initial introductions via Kent Special School Heads group

Introductions via colleagues such Educational Psychologists

Word of mouth through attendance at SEBDA conferences

Known colleagues in other settings

Issues with data collection Replies needed on a timescale – perhaps missed out

some potential respondents ( though the research is ongoing after the thesis is submitted)

Small scale study of invited participants. Not truly representative.

Contentious wording of “assessment”

Use of alternative language - “behaviour profile”

Data analysis methods Online via

SurveyMonkeyTM

Using AtlasTI qualitative data analysis programme

Data generated A total of 20 establishments or individuals were asked

to complete the survey

11 respondents (in the timeframe)

Represents 55% compared to average of 31-34% for online survey requests

Tables of responses created and analysed

Qualitative responses coded and used in code families to establish any links such as

“is associated with”

“is a property of”

Questionnaire results Q1. When a new pupil is admitted to your provision, or soon

after, do you carry out some form of assessment? Could you please indicate if you use pre-intake testing.

Q2. If you do, do you use a named assessment tool(s)?

Q3. If you answered "Yes" to question 2, which assessment tool(s) do you use? Please outline for what purpose it is used.

Action outlined Frequency of action in

sample

Comment

Reading 3

Spelling 2

Sentence

comprehension

2

Literacy (generic) 1

Numeracy 2

National Curriculum

(NC) levels

2

Speech and language

screening

1

Behaviour specific 3 One developed

“in house”

One for

attachment

disorders

One for

behaviour

readiness for re-

integration

Pupil attitudes 2

Specific learning

difficulties

1 SnAP

Achievement 1

Ability 1

Resiliency 1 scale

Psychological reasons 2

Solution focussed

questions

1

Intelligence 1

respondent Named assessment purpose

1 WISC-IV Clinical tool for measurement

of learning disabilities and attention

disorders

1 WIAT-II Assessment of reading, language and

numerical attainment

1 Resiliency Scales Identify and measure core personal

qualities of resiliency in youth aged 9:0

years to 18:11 years.

1 Solution Focussed

Questions

Designed to help people explore their

strengths and resources rather than

concentrate on their problems and

deficits.

4 SNAP Special Needs

Assessment Profile

Diagnostic Assessment and Profiling for

Specific Learning Difficulties

4 WASI - Wechsler

Abbreviated Scale of

Intelligence

Test for different forms of intelligence

4 Beck Youth Inventory Testing for psychological profiling

4 WRAT 4 - The Wide

Range Achievement

Test

Measures an individual's ability

to read words, comprehend sentences, s

pell, and compute solutions

to maths problems.

6 PASS – pupils attitude

to self and school

Measurement of a pupil’s attitudes

towards themselves as learners and

their attitudes towards school,

7 WRAT 4 - The Wide Range

Achievement Test

Measures an individual's ability

to read words, comprehend sentences, spell, and

compute solutions to maths problems.

7 Num NFER National

Foundation for Educational

Research

Basic numeracy

7 PASS – pupils attitude to self

and school

Measurement of a pupil’s attitudes towards

themselves as learners and their attitudes towards

school,

7 Boxall Profile Aimed at looking at behaviour relating to

attachment issues.

8 The Muntham Wheel “ for monitoring and recording progress within the

areas of classroom conformity, task orientation and

the SEAL outcomes.”

8 McSherry behaviour

readiness to integrate scale

Determine readiness for pupils with SEBD to return

back into mainstream settings

9 STAR rating scale Not specified

12 CAT Cognitive Abilities Tests Test of reasoning ability

Named assessment tool and its function

Q4. If you named a particular assessment tool(s), can you briefly outline why you chose that one.

Reason given Frequency of action in

sample

Comment

Staff ability 1 Staff trained

Suggestion from

Educational

Psychologist (EP)

1

Offers best

information

1

Specific test reason 2 Resiliency scales – can

be built upon for

intervention

McSherry – good for

reintegration

“we built it” 1

Not named 1

“It’s very good” 1

“best we can find” 1

“mirrors school ethos

of care”

1

Q5. Do you have a behaviour monitoring system?

Q6. If you have a behaviour monitoring system, do you use it to directly inform your intervention strategies. For example, does the data generate "if this occurs"..."then this happens" type of intervention. Could you please give an example if it does.

Q7. If you have a behaviour monitoring system, can you briefly explain how you track intervention strategies.

Action outlined Frequency of action in

sample

Comment

Daily recording 6

Recorded weekly 3

Recorded other 1 Three weekly record

Recorded each half

term

2

Recorded each term 1

Recorded

electronically

2

Recorded on paper

system

1

Recorded method

unspecified

7

Use of credit / points

system

4

Recording of specific

intervention

2

Use of target setting 2

Specific use of term

“review”

5

Mention of specific

intervention measure

2 1 of IEP (individual

education programme

1 of IAP (individual

action plan)

Use of dedicated staff 2 1 of Learning mentor

team

1 of “associated staff”

Q9. If you have a behaviour monitoring system, can you briefly explain how you use the data it generates. For example, reporting to parents or showing progression, etc

Action outlined Frequency of action

in sample

Comment

Reporting to parents 5

End of term reports 2

Showing progress

specifically

2 Using coloured wheel

Annual reviews 3

Changing targets 4

Inform student directly 1

Inform key working 1

Pupil rewards 2

Contributes to whole

school data

1

Weekly case

management

1

Reports to governors 2

Reports to Ofsted /

DFE

1

Other 4 “informs diary writing”

“Inform CPD”

(continuing professional

development)

“Plan more

individualised

interventions”

“Graphs of types of

behaviours”

Q8. How do you know your behaviour intervention strategies are achieving the desired outcome?

Action outlined Frequency of

action in sample

Comment

Reduction in incidents /

physical interventions

5

Increase in appropriate

behaviours

1 Not specified what

“appropriate” are.

Raised achievement /

attainment / learning

4 Includes “qualifications”

Behaviour points 1

Behaviour review

meetings

1

Pupil progress meetings 1

Parental reports of

improvements

1

Improvement in scaling 1

Movement in targets 1

Attendance 2 Attendance is raised

Exclusions 1

Make progress 1 Unspecified – “children make

progress”

Other 2 “when we have understanding

and balance”

“unsure”

Q10. Please feel free to add any comments about SEBD provision and assessment.

ConclusionsIn this survey it is clear that SEBD schools do use assessment widely but

seemingly not for specific behavioural issues. This seems a little unusual

but in discussions with colleagues there would appear to be a simple lack

of knowledge or indeed the confidence to administer this type of testing.

If schools are able to rigorously demonstrate the pupils’ level of emotional

and behavioural difficulty, and subsequent impedance on learning thereof,

they will be able to go some way to defending or celebrating the academic

progress made by pupils.

I will be conducting further research specifically into the area of

behavioural assessment and hope that by posting links to the survey on

the websites of professional associations I will gain a more in depth

picture of what is currently being used.

References Ballantyne C (2005), Moving student evaluation of teaching online: reporting pilot outcomes and issues with afocus on how to

increase student response rate. Paper presented at the 2005 Australasian EvaluationsForum: University Learning and Reaching: Evaluating and Enhancing the Experience, UNSW,Sydney, 28–29 November.

Bhaskar, R. 1998a. "General introduction", in M. S. Archer, R. Bhaskar, A. Collier, T. Lawson, and A. Norrie, eds., Critical realism : essential readings. London: Routledge, pp. ix-xxiv

Creswell, J.W.& Plano Clark V.L.(2011) Designing and Conducting mixed methods research. Thousand Oaks, CA: Sage Publications, Inc.

Dommeyer, C.J., P. Baum, R.W. Hanna, and K.S. Chapman. (2004). Gathering faculty teaching evaluations by in-class and online surveys: their effects on response rates and evaluations .Assessment & Evaluation in Higher Education 29, no. 5: 611–623.

Elliott, S. N., Gresham, F. M., Frank, J. L., & Beddow, P. A. (2008). Intervention Validity of Social Behavior Rating Scales: Features of Assessments That Link Results to Treatment Plans. Assessment for Effective Intervention, 34(1), 15–24.

Krauss, S. E., & Putra, U. (2005). Research Paradigms and Meaning Making : A Primer, The Qualitative Report 10(4), 758–770.

Merrell, K. W. (2001). Assessment of Children’s Social Skills: Recent Developments, Best Practices, and New Directions. Exceptionality, 9(1-2), 3–18.

Nair, C.S., C. Wayland, and S. Soediro. (2005). Evaluating the student experience: a leap into the future.Paper presented at the 2005 Australasian Evaluations Forum: University Learning and Teaching:Evaluating and Enhancing the Experience, UNSW, Sydney, 28–29 November

Nulty D.D (2008): The adequacy of response rates to online and paper surveys: what can be done? Assessment & Evaluation in Higher Education Vol. 33, No. 3, June 2008, 301–314

Ogier, J. (2005). The response rates for online surveys—a hit and miss affair. Paper presented at the 2005. Australasian Evaluations Forum: University Learning and Teaching: Evaluating and Enhancing theExperience, UNSW, Sydney. 28–29 November

Watt, S., C. Simpson, C. McKillop, and V. Nunn (2002). Electronic course surveys: does automating feedback and reporting give better results? Assessment & Evaluation in Higher Education 27, no. 4:325–337