A REPORT CRITICALLY EVALUATING ASSESSMENT DEVELOPMENT CENTRES
WITHIN THE THE FIRE AND RESCUE SERVICE
Janine Frankland 26 April 2010
20393518
CONTENTS PAGE
Page
1.0 Introduction
2.0 Background 1
3.0 THE FIRE AND RESCUE SERVICE Workforce Development Strategy 2
4.0 Developing Leadership/Workforce Skills and Capacity 2
5.0 Assessment Development Centres 4
5.1 Definition of an Assessment Development Centre
5.2 THE FIRE AND RESCUE SERVICE Assessment Development Centre
5.3 Structure and Context of a Typical ADC
5.4 ADC Scoring Process
5.5 Interview Process
5.6 Progression through Development to Promotion
6.0 Evaluation of Assessment Development Centres 8
7.0 Bibliography 9
Appendix 1
Appendix 2
Appendix 3
1.0 INTRODUCTION
The following report will examine the use of Assessment Development Centres as a
selection tool for promotion within THE THE FIRE AND RESCUE SERVICE and
evaluate their contribution to meeting the organisation’s strategic needs.
2.0 BACKGROUND
The role of the Fire Service has changed in line with the modernisation agenda
outlined in the June 2003 White Paper “Our The Fire and Rescue Service” (Office of
the Deputy Prime Minister) and the The Fire and Rescue Service National
Framework 2008-11 (Department of Communities and Local Government, 2008).
The The Fire and Rescue Service National Framework sets out clear objectives for
all The Fire and Rescue Services. UK Fire Services are now proactive in preventing
fires and other risks, rather than being solely reactive and responding to incidents.
There is evidence that the Fire Service is moving away from a transactional to a
transformational leadership; in 2004, the Fire Services (Discipline) Regulations 1985
were abolished. In its place was a move to introduce new disciplinary procedures
based on the Advisory Conciliation and Arbitration Services best practice guidance,
which was brought inline within the aforementioned The Fire and Rescue Service
National Framework. In the past there has been an era of ‘command and control’,
where officers issued orders to control their subordinates and any questioning of
those orders would be met with disciplinary action. The subordinates expected to be
commanded by their superiors.
In recent years, there has been a slight move away from this command and control
ethos. In 2005, there was a restructure of the management composition and the ten
ranks were reduced down to seven roles. The word ‘officer’ was replaced by the
word ‘manager’ (see Appendix 1). A change in the way THE FIRE AND RESCUE
SERVICE promotes and develop individuals has seen a move towards coaching
rather than commanding.
In July 2006, new government legislation made significant changes to the way The
Fire and Rescue Services fulfil their new and existing statutory duties and The Chief
Fire Officers Association launched a national HR strategy for the development of
people management in the UK The Fire and Rescue Service.
Page 1 of 12
3.0 THE FIRE AND RESCUE SERVICE WORKFORCE DEVELOPMENT STRATEGY
In 2009, THE FIRE AND RESCUE SERVICE carried out a review of its Human
Resources Strategy (2005) and introduced The Workforce Development Strategy
2009/11. Within this document, THE FIRE AND RESCUE SERVICE recognise that
their employees are critical to the delivery of services to the community and in
achieving the organisation’s corporate aim of making Lancashire safer. The Strategy
sets out how THE FIRE AND RESCUE SERVICE work to have the right people in
the right place, with appropriate skills, competence and motivation to achieve this.
THE FIRE AND RESCUE SERVICE strategic objectives are outlined in the Risk
Management Plan (RMP). The RMP sets out THE FIRE AND RESCUE SERVICE’s
strategic direction (Lancashire Fire and Rescue, 2008). Through this plan, THE FIRE
AND RESCUE SERVICE assess local risk and identifies how it can deploy its
resources and deliver improved public safety.
The Workforce Development Strategy works in conjunction with the RMP in order for
the organisation to achieve those objectives, and “ensure long term effective service
delivery in a flexible and responsive way” (2009, p.1).
THE FIRE AND RESCUE SERVICE list six key priorities within the Strategy (2009,
p.5), which are a reflection of the The Fire and Rescue Service CFOA National HR
Strategy (2006, p.9).
1. Developing the organisation
2. Developing leadership capacity
3. Developing workforce skills and capacity
4. Recruiting and retaining employees
5. Pay and rewards
6. Developing the capacity of Human Resources
4.0 DEVELOPING LEADERSHIP/WORKFORCE SKILLS AND CAPACITY
The individuals that work for THE FIRE AND RESCUE SERVICE provide operational
or service delivery support.
Page 2 of 12
To further expand on the key priority listed within THE FIRE AND RESCUE
SERVICE’s Workforce Development Strategy of developing leadership/workforce
skills and capacity, this report looks at the process that THE FIRE AND RESCUE
SERVICE use to promote and develop future managers within the operational sector.
The operational sector refers to individuals working in one of the seven roles from
Firefighter through to Brigade Manager (see Appendix 1).
THE THE FIRE AND RESCUE SERVICE use The Integrated Personal Development
System as a framework for how it develops its workforce.
Two firefighters lost their lives at an incident on Gillender Street, London in 1991 and
as a result a Health and Safety Improvement notice was issued to brigades requiring
the identification of performance standards and a way of measuring firefighters
against those standards. National Training Strategy Group (TSG) was established to
analyse the selection, training and development needs of the Service and as a result,
in 1995, the Integrated Personal Development System (IPDS) was developed (Skills
for Justice, 2009). This system has now been embedded in the The Fire and Rescue
Service National Framework 2008-11.
The Skills for Justice Code of Practice document explains how the IPDS provides a
structure based on agreed standards of performance, within which organisations can
identify, assess and develop people to fulfil current and future roles.
Integral to the IPDS is meeting the needs of the aforementioned Service’s Risk
Management Plan (Skills for Justice, p.6). IPDS supports the Service’s objectives of
making communities safer and through THE FIRE AND RESCUE SERVICE’s plan to
constantly reduce risk in the community, assists the organisation in making sure that
their employees have the latest skill to contribute to public safety.
The statutory fire service examinations were abolished in 2004 (Fire Services
Examination Board, nd) and in their place, the The Fire and Rescue Service National
Framework 2004/5 stated that The Fire and Rescue Services should “use the
nationally developed personal qualities and attributes for assessing the potential of
candidates for a new operational role” (p.38).
Page 3 of 12
In order to facilitate this, THE FIRE AND RESCUE SERVICE carries out Assessment
Development Centres.
5.0 ASSESSMENT DEVELOPMENT CENTRES
5.1 Definition of an Assessment Development Centre
An Assessment Development Centre (ADC) could easily conjure up an image of an
actual place or location but many authors (Ballantyne and Povah 2004, Woodruffe
2000, Hay 1997 to name a few) state the term refers an event or as described by
Chartered Occupational Psychologists, Liam Healy & Associates (nd) as an
‘approach’.
Appelbaum et al (1998, p.11) describes assessment centres and “development
assessment centres” as having a common objective, which is to observe how
individuals handle a series of situations that might occur, however they do differ in
focus. Ballantyne and Povah (2004, p.142) suggest that development centres have
evolved from assessment centres and attempt to give the reader an explanation of
the main differences between the two. They suggest that assessment centres are
used for external recruitment and internal promotion; therefore the assessment
centre is the end of the process. Development centres are used for identifying fast-
track potential and diagnosing job related strengths and weaknesses; therefore the
development centre is the start of the process.
5.2 THE FIRE AND RESCUE SERVICE Assessment Development Centre
Within THE FIRE AND RESCUE SERVICE, ADCs are carried out at three levels –
Supervisory, Middle and Strategic Management. Appendix 1 shows how the seven
roles fit into each managerial level. Individuals are to be competent in their current
role before they can apply for attend the next level ADC.
Page 4 of 12
The DCLG describe the ADCs as national tests that are designed to put individuals
under a great deal of pressure in the same way that any risk critical situation would
do (2009, p.11).
Individuals are measured against generic management skills, known as Personal
Qualities and Attributes (PQAs). These PQAs can include subjects such as working
with others, effective communication, commitment to change, openness to diversity
and integrity etc (see Appendix 2). Further information on how individuals are
measured and scored against the PQAs will be provided later in this report.
5.2 Structure and Context of a Typical ADC
The process developed within THE FIRE AND RESCUE SERVICE includes Stage 1
ADC, Stage 2 ADC and a PQA interview as explained within the organisation’s
Training Service Order (THE THE FIRE AND RESCUE SERVICE, 2008).
The Stage 1 ADC is formerly known as an ITOP which is an Initial Test of Potential. It
is a preliminary sifting exercise carried out under supervised conditions, which
examines an individual’s performance of the some of the PQAs identified as being
most crucial for success as a manager within THE FIRE AND RESCUE SERVICE.
The ITOP is divided into the following two sections - the Situational Judgement Test
(SJT) and a Simulation Exercise. The SJT is a multi choice test – it details a scenario
and then asks which of the following choices is likely to be effective. The simulation
exercise consists of three written tasks. The exercise is set in a fictitious
organisation that is non-fire service related. The exercises are usually formed
around decision making, numerical data and writing a letter/article aimed at a specific
audience (UK Fire Service Resources, nd).
The tests are designed to assess how an individual’s performance matches up to the
standards outlined in the PQAs. All individuals are given feedback on their
performance by their line managers. Successful candidates will be invited to attend
the Stage 2 ADC and unsuccessful individuals have a development plan created for
them to address their development needs.
During the Stage 2 ADC, individuals will participate in a series of exercises that
simulate the kind of tasks that they might be expected to do as a manager. The
Page 5 of 12
scenarios are non fire service related, for example, in past ADCs, individuals have
been informed that they are a manager in an airport or at a leisure centre.
When individuals are invited to an ADC by letter, they are also provided with
background information which will give them information about the organisation they
are working for, their position within the company, who they are responsible for etc.
The series of exercises are carried out in one day and involve activities such as a
group discussion, role play, multi role play, inbox exercise and briefing exercise.
Each exercise is designed to measure individuals against certain PQAs, however,
the candidates are not informed as to which PQAs they will be measured against.
The exercises are observed by a team of assessors, who are usually internal senior
managers who have been specifically trained to carry out the PQA scoring.
5.3 ADC Scoring Process
THE FIRE AND RESCUE SERVICE (2008, p.14) state that each PQA has been
mapped to particular exercises within the ADC so feedback on each PQA can be
given from these sources. Each PQA is divided into constructs and each of these
constructs has a Behaviourally Anchored Rating Scale (BARS).
After observing and recording the behaviour of an individual in one of the ADC
exercises, assessors use the exercise’s specific BARS to assess the effectiveness of
the behaviour. Within each construct, the individual’s performance is compared to
four levels and assigned the rating of the nearest match. The ratings of each
construct are then added together to provide a score for the PQA being assessed on
that exercise. For an example of how this is structured, Appendix 3 shows how the
PQA of Problem Solving is divided into constructs measured against BARS along
with a description for the levels of performance from 1 to 4. THE FIRE AND
RESCUE SERVICE (2008, p.14)
After the ADC, the assessors rate each individual’s performance on the PQAs across
the various exercises using a pre-determined scoring process. Individuals will then
be given an overall score derived from their scores on the PQA.
Page 6 of 12
5.4 Interview Process
The PQA interview has been integrated as part of the selection process, where
individuals are to provide evidence against the PQAs from either their work or from
other activities outside work.
The questions fit in to the ‘STAR’ model, where answers will be expected to fit into
Situation, Task, Action and Result (UK Fire Service Resources, nd).
The evidence provided at the interview may increase an individual’s overall PQA
score.
Only when a firefighter is deemed “successful” at both the ADC and subsequent
interview, can the individual be placed on a development programme and considered
for promotion.
5.5 Progression through Development to Promotion
THE FIRE AND RESCUE SERVICE’s Human Resources Department (HR) carry out
succession planning by determining the number of positions that will become
available within its workforce in order to determine the number of managers that will
be required in the future. This is usually determined by the number of individuals that
are due for retirement, as the firefighter pension scheme enables individuals who
have completed 30 years service and reached a minimum age of 50 to retire.
A moderation meeting is held between the HR and Training Development
Departments to discuss each individual’s scores and determine the number of
individuals that will be considered successful and therefore be placed on a
management development programme.
The ADC is not a pass/fail, which Ballantyne and Povah describe as a distinguishing
feature of a development centre. Individuals are either successful and unsuccessful.
Again, as with the ITOP stage, unsuccessful applicants will receive feedback from
their line manager and have a plan created to address their development needs
(THE FIRE AND RESCUE SERVICE, 2008).
6.0 EVALUATION OF THE ASSESSMENT DEVELOPMENT CENTRE
Page 7 of 12
There has been little written about the effectiveness of assessment development
centres, despite development centres existing since AT&T introduced them in the
1950s (Thornton and Rupp, 2006; Ballantyne and Povah, 2004).
However, Taylor (2008, p.306) describes assessment centres as ‘the Rolls Royce’ of
selection methods and reports that validity studies have consistently found
assessment centre techniques to have good predictive ability.
If THE FIRE AND RESCUE SERVICE were to use the ADC on its own, it could be
argued that this would not be sufficient to guarantee good operational managers
within the Fire Service. It is difficult to substantiate this statement as there is little or
no written evidence to suggest this, other than that endorsed by the DCLG in the
published Integrated Personal Development System Code of Practice (2009, p.11)
where they recommend that ADCs form part of the selection process and should be
not be considered as a standalone process. The DCLG recommend that other
Human Resource Management activities should be utilised.
ADCs provide organisations with an indication that individuals already display some
of the managerial qualities prior to investment of a development programme, which
will equip the individual with further managerial skills. However, it does not take into
consideration any previous qualifications, technical knowledge or practical work
related experience, unless it is specifically tied in to evidence for the interview.
Prior to the introduction of ADCs, individuals seeking promotion were required to sit a
written and practical examination. Individuals were required to be ‘time-served’ in
their current role prior to seeking promotion. In the traditional promotion interviews
held , there was a culture of ‘who you know’ rather than what you know or the ‘halo
and horns’ effect. Anderson and Shackleton (1993) cited by CIPD (2010) highlight
these judgements along with other limitations of the traditional interview, highlight
why it is a poor predictor of an individual’s performance in the job.
However, according to statistics provided by the Fire Services Examination Board,
there was a relatively low pass rate. For example, individuals taking the Leading
Firefighter and Sub Officer* examinations, between the years of 1987 and 2004, an
average of 40% passed the written paper and 60% passed the practical (Fire
Services Examination Board, 2006).
Page 8 of 12
*Now referred to as Crew and Watch Managers
A solution to this issue was brought about with the introduction of The The Fire and
Rescue Service National Framework 2008-11. The caveat of being time served was
removed due to being deemed potentially discriminatory and to allow Fire Services
more flexibility with the promotion process by introducing ADCs.
There are shortfalls that should be highlighted within the current ADC process that
THE FIRE AND RESCUE SERVICE and other Fire Services use. There were only
three sets or ‘scenarios’ that were produced by the IPDS Hub which was originally
located at the Fire Service College. The sets have been in use since 2006 since
which time, only two of the sets are now used due to an issue with Set 1 (although
there is no available written evidence of this) and therefore the same set has been
used on more than one occasion which could lead to individuals becoming ‘familiar’
with the exercises as opposed to portraying a true picture of how they would behave.
This could also lead to individuals informing others of what is involved in each
exercise therefore giving them an unfair advantage or can an individual learn the
PQAs and BARS and therefore know how they need to behave in exercises in order
to score highly?
On carrying out research of PQAs measured in the ITOP against the PQAs
measured in the ADC, there appears to be minimal correlation. If an individual
scores high in a particular PQA in the ITOP, they would be expected to score high in
the PQA in the ADC but this was not always the case. This would bring in to
question the validity of ADCs or indeed some of the processes used within an ADC.
7.0 SUMMARY
It could be argued that the validity of ADCs meeting an organisation’s strategic
needs, would depend on what an organisation introduces as part of its ADC process.
In the past, THE FIRE AND RESCUE SERVICE only carried out interviews when
there were job positions for individuals on the development programme to apply for.
However, the organisation found that this was not effective. The organisation was
heavily investing in developing individuals that were deemed unsuitable for
promotion. As a result of this, THE FIRE AND RESCUE SERVICE introduced a PQA
Page 9 of 12
interview in to the ADC process prior to individuals being placed on a development
programme.
However, other stages such as a technical/practical assessment could be introduced
to confirm an individual’s suitability within the fire service operational field. At this
moment, technical and practical skills are only developed when an individual is
already placed on a development programme.
In conclusion, it difficult to directly quantify in this report if the ADC process currently
administered within THE FIRE AND RESCUE SERVICE directly contributes to
meeting the strategic needs over any other selection process. However, THE FIRE
AND RESCUE SERVICE should constantly monitor their selection methods and
contribute to their strategic needs by delivering best value and a safe, healthy,
competent and representative workforce.
WORD COUNT: 2945
Page 10 of 12
BIBLIOGRAPHY
Anderson, N and Shackleton, V. (1993) Successful selection interviewing. Oxford: Blackwell
Appelbaum, S H; Harel, V and Shapiro, B (1998). The Developmental Assessment Centre: The Next Generation. Career Development International pp.5-12
Chartered Institute for Professional Development (2010). ‘Selection Interviewing’. Available at http://www.cipd.co.uk/subjects/recruitmen/selectn/selnintvg.htm?WBCMODE=Present-ationUnpublishedfaculty.htm Accessed 26 April 2010
Department of Communities and Local Government (2008). ‘The The Fire and Rescue Service National Framework 2008-11’. Available at http://www.communities.gov.uk/documents/fire/ pdf/nationalframework200811.pdf Accessed 14 April 2010
Department of Communities and Local Government (2009). ‘Integrated Personal Development System Code of Practice’. Available at http://www.skillsforjustice-ipds.com/documents/IPDS%20Code%20of%20Practice.pdf Accessed 16 April 2010
Fire Services Examination Board (nd). ‘About the Examinations’. Available at http://www.fseb.gov.uk/exams/index.htm Accessed 18 April 2010
Hay, J (1997) Gower Assessment and Development Centre: Volume 1 (1st Edition) Surrey : Gower
THE THE FIRE AND RESCUE SERVICE (2008). ‘Risk Management Plan 2008-11’.
THE THE FIRE AND RESCUE SERVICE (2009). ‘The Workforce Development Strategy 2009/11’.
THE THE FIRE AND RESCUE SERVICE (2008). Service Order Training – 03. ‘Development and Promotion Procedures – Assessment and Development Centre Process’ Section 1 pp.1-18
Liam Healy & Associates (nd). ‘Assessment and Development Centres’. Available at http://www.psychometrics.co.uk/adc.htm Accessed 25 April 2010
Office of the Deputy Prime Minister (2003). ‘Our The Fire and Rescue Service’. Available at http://www.communities.gov.uk/documents/fire/pdf/128947.pdf Accessed 14 April 2010
Office of the Deputy Prime Minister (2004). ‘The The Fire and Rescue Service National Framework 2004/5’. Available at http://www.communities.gov.uk/documents/fire/pdf/ 128923.pdf Accessed 14 April 2010
Page 11 of 12
Office of Public Sector Information (2004). ‘The The Fire and Rescue Services Act 2004’.
Available at http://www.opsi.gov.uk/acts/acts2004/ukpga_20040021_en_1 Accessed 14
April 2010
Taylor, S (2008) People Resourcing (4th Edition) London : CIPD
Thornton, G C and Rupp, D E (2006). Assessment Centers in Human Resource Management (1st Edition) New Jersey : Erlbaum
UK Fire Service Resources (nd). ‘Situational Judgement Test - Multiple Choice’. Available at http://www.fireservice.co.uk/development/sjt.php Accessed 25 April 2010
Woodruffe, C (2000) Development and Assessment Centres (3rd Edition) London : CIPD
Page 12 of 12
THE THE FIRE AND RESCUE SERVICE – HIERARCHICAL STRUCTURE
Chief Fire Officer
Deputy Chief Fire Officer
Asst Chief Fire Officer
Area Manager
Group Manager
Station Manager
Watch Manager
Crew Manager
Firefighter
Appendix 1
Appendix 2
The Fire and Rescue Service Overview Personal Qualities and Attributes (PQAs) Framework – Operational Roles
* = PQA is relevant to operational aspects of managerial roles only; it is assumed that those already working in operational roles within the FRS already possess this PQA.
Skills for Justice (2009)http://www.skillsforjustice-ipds.com/documents/PQA%20Framework%20-%20all%20levels.pdf
Appendix 3
The Fire and Rescue Service Supervisory Level Generic Behaviourally Anchored Rating Scales (BARS)
4 Strong Performance – meets all of the positive behaviours expected of that role with no development needs being demonstrated.3 Acceptable Performance – meets most of the positive behaviours expected of that role with very few development needs being demonstrated.2 Development Need – meets some of the positive behaviours expected of that role but also displays some development needs or displays one significant
development need that counter balances the positive behaviours.1 Significant Development Need – meets none or very few of the positive behaviours expected of that role but displays mostly development needs.
Top Related