The Railway Training & Assessment Accreditation Scheme (RTAS) Baseline Inspection Results

Undertaken as part of the Inspection of Railway Engineering Accredited Training and Assessment Providers

Developed by the industry for the industry

ContentsForeword 3

Executive summary 4

Recommendations 5 - Overall quality of training providers 5 - Trainers’ upskilling 5 - Self-Assessment Reports and Quality Improvement Plans 5 - Equality and diversity 5 - Support for learners who have learning difficulties 5 - Feedback for learners and employers 5 - Sharing of best practice 5 - External funding of training provision with the rail industry 5

Background 6

NSARE accreditation process 7

Self Assessment Report (SAR) and Quality Improvement Plan (QIP) 9

Individual trainer/assessors 9

Desk top audit 9

Inspections 10

Inspection outcomes 11 - Trainer observations 11 - Assessment only providers 11 - Assessment – baseline survey 11 - Next inspections 11

NSARE report 12

Inspection findings 13 - Capacity to improve 13 - Learner outcomes 14 - Quality of provision 16 - Leadership and management 18 - Equality and diversity 21

Assessment of competence 23

Training skills 24

Training qualifications 26 - Continuing Professional Development (CPD) 26 - Age profile of trainers/assessors 28

Appendix 1 – Results of the Baseline Inspections in 2012 30

Foreword

In our ‘Skills Strategy for the Railway Engineering Sector’, which we published in February this year, we set out our plans to transform the education and training provision which serves the industry. This report is the first stake in the ground, setting the baseline from which we will drive continual improvement. Together with our skills forecasting work, which we will publish in the autumn, it brings into sharp focus the challenges facing our industry..

Whilst we have found high levels of motivation, pride and commitment, which give us confidence that we can meet the challenge of enhancing the industry’s capability to design, build and maintain this key component of the UK’s infrastructure, we are acutely aware that time is not on our side. Our industry is entering a period of investment unparalleled since the Victorian era; Crossrail, Thameslink, ambitious electrification programmes, upgrades to the London Underground and light rail schemes, the provision of new trains, the introduction of the European Rail traffic Management System and HS2 all add to the mounting pressure on our engineering resources. The education and training providers have a hugely important role to play in both training new recruits and up-skilling the existing workforce.

3

Gil Howarth Chief Executive & Company Secretary NSARE Limited

Executive summaryThis report summarises the findings of the first baseline inspection, carried out by NSARE, of training and assessment providers under the RTAS Concession from Network Rail. Although the league table, which is published for the first time, shows that 70% of training providers were judged ‘Good’ or ‘Outstanding’, the report highlights the need for an overarching strategic direction of the training provision if we are to develop the workforce of the future to meet the skills challenges. The introduction of the new accreditation and inspection framework, which will shift practices from a culture of compliance to one of continual improvement, is only the first step. We need to eliminate outdated training practices where they exist and up-skill those trainers who wish to play their part, but who do not have the qualifications to train at the highest level.

The baseline inspections have revealed, not surprisingly, that some of the training providers have found it difficult to move from the compliance-based audit system to a process of continuous improvement, driven by self-evaluation. Whist all training providers were able to demonstrate compliance with the requirements laid down by Network Rail (Sentinel Scheme Rules), there was limited evidence that feedback from learners and employers was regularly taken into account; most processes had been developed to meet organisational and statutory requirements rather than being focused on learner needs. Whilst training and assessment in the industry is seen as inclusive, there appears to be a lack of understanding of best practice around equality and diversity in some training establishments.

The inspectors found that the training delivered was competent, with trainers who were knowledgeable, enthusiastic and very safety conscious. However delivery concentrated mainly on knowledge transfer, rather than the development of a deeper understanding of the risks involved in railway engineering activities. Development of behaviours was not a key focus of the training and support for individuals, to help develop their literacy, numeracy and communication skills, was not a priority.

There was little evidence of widespread development for trainers, or rigorous Continuing Professional Development (CPD), with sharing of best practice between trainers only spasmodic. An area of concern is the low qualifications, in general, of the trainers within the sector – significantly lower than would be expected within an engineering sector. Demographics show that more than half the trainers are aged over 50 and less than 3% are under 30, highlighting a potential shortfall in the medium term and the need to both attract and recruit new high calibre trainers while up-skilling the existing trainers.

As well as accrediting and regularly inspecting education and training providers, NSARE will be continuing to promote careers in railway engineering at schools, colleges and universities; we will make national qualifications available in the competencies that our industry needs including innovations like Higher (Level4+) Apprenticeships. In our first full year of operation we have already seen a dramatic increase in the number of apprentices entering the sector. However we need to ensure that the education and training of these young people is of the highest quality. Thus at our National Conference, ‘Training Matters!’, we will be announcing plans to extend our training accreditation scheme to all education and training provision within the sector. This is what the employers within the sector want – we want everybody to play their part.

4

learner

RecommendationsOverall quality of training providers

The scheme should be expanded to include all Railway Infrastructure Engineering Training and Assessment Providers from 1st April 2013.

Only an overall judgement of “Good” or “Outstanding” should be acceptable for accreditation under the NSARE Inspection Framework for Railway Engineering Accredited Training and Assessment Providers. This would be effective for all inspections carried out after 1st January 2013.

In line with Ofsted guidelines, rename “Satisfactory” as “Requires Improvement” for all inspections carried out after 1st January 2013.

Trainers’ upskilling For any new trainer who requests to be

registered with NSARE after 1st October 2012, the minimum acceptable qualification in training will be the Certificate of Professional Development: Teaching in the Work Based Learning Sector (40 Credits at Level 4) or other programme with the same outcomes approved by NSARE.

For existing trainers who are currently registered with NSARE, who currently have training qualifications at Foundation level 3, to maintain their registration after 31st December 2013, they are required to undertake up-skilling training and achieve Certificate of Professional Development: Teaching in the Work Based Learning Sector (40 Credits at Level 4) or other programme with the same outcomes approved by NSARE.

Self-Assessment Reports and Quality Improvement Plans

By 1st December 2012, providers are required to upload their annual revision of the Self Assessment Report (SAR) and associated Quality Improvement Plan (QIP).

To ensure that the SARs and QIPs meet the requirements of the NSARE Accreditation scheme, the following actions are required by providers in developing the SAR and QIP:

• Evidence of participation in the development by directors, managers, trainers/assessors and administration staff

• Evidence of robust feedback from learners and customers• Focus on providers’ business needs• Alignment to the NSARE Inspection Framework• Critical, evaluative judgements, supported by evidence

Equality and diversity Providers should take steps to improve their overall performance

in Equality and Diversity. This should include raising the level of knowledge, understanding and application of Equality and Diversity throughout the training provision significantly, so that the trainers and assessors become role models for Equality and Diversity. Trainers should raise Equality and Diversity during the introduction to training courses.

Providers should review data management information within their organisations so that it is possible to differentiate between learners based on disability, gender and race. The data should be used to map learner achievements against each group, to ensure that the delivery of training and assessment is fair and equitable and provision is accessible for all learners.

Providers should review their documentation and publications including web sites to remove gender specific language.

Support for learners who have learning difficulties Providers should review their processes to ensure that they identify

learners who have learning difficulties or who have issues with literacy and numeracy. Providers should ensure that trainers are advised of these difficulties before courses commence.

Trainers should actively seek and implement opportunities to expand individuals’ ability in literacy, numeracy and communication skills throughout any training sessions.

Feedback for learners and employers Providers should review their processes to ensure that they receive

reflective and considered feedback from Learners and Employers on their delivery of training and assessments. This should be significantly more robust than the current post course evaluation.

Providers should review their processes to ensure that adequate data analysis of learner and employer feedback is made available and this should form part of regular management reviews.

Sharing of best practice Providers should review their processes for sharing of best practice

between trainers and assessors.

External funding of training provision with the rail industry NSARE, in association with the Skills Funding Agency, will review the

provision of funding for training via colleges, prison service and others to ensure that the provision meets the strategic needs of the rail industry and supports the development of sustainable employment for learners.

5

The National Skills Academy Railway Engineering (NSARE) was set up by the railway industry with the express aim of developing and implementing a unified railway engineering skills strategy for the whole industry, ensuring that training capability and capacity meets the future needs of Britain’s railways. NSARE was funded by the Department of Business Innovation and Skills, with a grant to cover the initial period of setup. NSARE is a company limited by guarantee whose members currently number over 200, are all engaged with railway engineering. NSARE is fully funded by the rail industry and derives this funding from undertaking activities which the rail industry identify as adding value and for which they are prepared to pay. Any surplus generated by NSARE will be re-invested into the industry.

One such activity is the accreditation of training and assessment delivery. Network Rail awarded NSARE the concession for training and assessment delivered under the Sentinel Scheme.

Sentinel is an integrated management system for some safety critical competencies on the railway introduced in April 1999 by Railtrack. Its purpose is to minimise the risk of untrained personnel carrying out safety critical work on the railway. The concession runs for an initial period of three years from January 2012, with the option to extend for two further years. Training and assessment covered by the Sentinel Scheme embraces the following key areas and applies only to training/ assessment carried out for workers who will be on Network Rail infrastructure.

Track Safety Training and Assessment Electrification Isolation Procedures Machine and Crane Controllers Auxiliary Operating Duties Track Induction Course

It is intended to extend the scope of the NSARE accreditation regime to cover all engineering activities in the near future.

As of the 1st September 2012 the following numbers apply to the NSARE accreditation scheme.

Training and Training/Assessment Providers 82

Assessment Only Providers 6

Individual Trainer/Assessors registered 395

Individual Trainer/Assessors registered (outside the scheme but registered voluntarily)

16

This report publishes findings from the first round of Rail Training and Assessment Accreditation Scheme (RTAS) inspections carried out of these training providers and assessment by NSARE.

The aim of the first inspection is to identify the current state of training and assessment as a baseline to measure future improvements. In addition to detailing the outcomes of the inspections, the report identifies areas for improvement and the strategic skills and succession-planning issues around training and assessment in general. It also, for the first time, produces a league table of training and assessment providers.

Note that while the data in this report specifically relates to Sentinel Training and Assessment the majority of the trainers/assessors covered by the report provide technical skills training and assessment to the wider industry.

Background

6

NSARE accreditation processNSARE has drawn extensively on the work of the Office of Standards in Education (Ofsted) to create an entirely new framework for training accreditation and inspection based on continual improvement. This contrasts with the previous training and accreditation licensing scheme, created by Railtrack in 1999 and inherited by Network Rail, which relied on compliance-based audits, with a simple pass or fail test; if the regulations remained unchanged, so too did the standards required from training and assessment providers.

The framework used by NSARE was derived from the Ofsted framework currently in general use for Work Based Learning provision. The objective of the NSARE approach is to inculcate a culture of self-sustaining, continual improvement within rail training and assessment providers so that, in the fullness of time, the rail engineering sector becomes the accepted exemplar for work-based training and assessment delivery.

The previous training and accreditation licensing scheme was introduced by Railtrack in 1999 and centred on annual compliance-based audits, supplemented by pseudo-random audits of 20% of training providers per annum. The audits were undertaken by a small dedicated team of auditors who worked primarily on a regional basis. The outcomes of the audit, was a pass/fail approach. Audit reports were not normally made available outside of the individual provider and Network Rail.

The Inspection Framework for Railway Engineering Accredited Training and Assessment Providers

detail

to create an entirely new framework for training accreditation and inspection based on continual improvement

7

Ofsted undertakes a variety of inspections, ranging from early years, primary and secondary education, HM Prison Education Service, further education colleges and work-based learning, which are directly supported by government funding. Each category of inspection has a dedicated inspection framework and inspectors. The NSARE Inspection Framework for Railway Engineering Accredited Training and Assessment Providers is derived from the Ofsted work-based learning framework and has been enhanced with some specific areas relating to railway engineering, and track safety.

There are five key sections within the framework, each of which is individually graded. The sections are:

Capacity to Improve Learner Outcomes Quality of Provision Leadership and Management Equality and Diversity

The grades in each of these are then combined into an overall effectiveness grade.

Overall Grade Capacity to Improve

Learner Outcomes Quality of Provision

Leadership & Management

Equality & Diversity

Outstanding Outstanding Outstanding Good Good Good

Good Good Good One category - Good Satisfactory

Satisfactory Satisfactory Satisfactory Satisfactory Satisfactory Satisfactory

Inadequate One or more categories inadequate

judgement

Each category of inspection has a dedicated inspection framework

8

Self Assessment Report (SAR) and Quality Improvement Plan (QIP)Each training and assessment provider was required to produce a SAR. This should be based on the NSARE inspection framework, with the provider making a critical self-judgement of their ability, supported by evidence. Additionally each provider is required to produce a QIP, again based on current ability and focused on how the provider will implement improvements to their service. Both of these documents are critical to the inspection process, with the SAR being the primary evidence against which the inspector evaluates the provider.

Individual trainer/assessorsAll trainers/assessors are individually registered with NSARE and were required to enter details of their qualifications, skills and experience on the SkillsBackbone™ - NSARE’s integrated, web-based, Training Accreditation and Skills Passport System. These details are reviewed and individual trainers/assessors are subsequently authorised to deliver specified training courses/assessment.

Desk top auditPrior to any inspection taking place, a desk top audit was undertaken by NSARE’s Training Accreditation Manager. The desk top audit ensured that each provider was fully compliant with the Sentinel Scheme Rules. Any feedback from the desk top audit was passed to the inspectors, who in addition to completing a sample check of training and assessment records and other key documentation, undertook further investigation as required.

reflective

9

quality

InspectionsThe inspection contract was awarded to Tribal Education Ltd, one of three companies that undertake inspections on behalf of Ofsted in England and Wales. Additionally Tribal Education works internationally with educational establishments to implement quality improvement schemes. Tribal Education currently delivers some 29,000 inspection events across the full range of activities undertaken by Ofsted. Tribal Education staff undertook the day-to-day planning for the inspections and additionally implemented an in-house quality assurance regime to ensure the reports produced by the inspectors were equitable across the range of provision and were comparable to other sectors.

The inspectors were selected from the Tribal Education resource pool. In addition to being qualified Ofsted inspectors, they have all held senior managerial positions within the education and training sector. All inspectors undertook a specific induction programme focused on the rail training sector. There were 19 inspectors who carried out inspections on the NSARE scheme. Additionally five of these inspectors were given full Personal Track Safety (PTS) training to enable them to go trackside and carry out a programme of inspections of assessments being undertaken against a range of railway competencies.

Inspection length was determined by the size of the provider:

Provider Category

Number of Trainer/Assessors

Inspection length

Small 0-4 1½ days

Medium 5-9 2½ days

Large 10+ 3½ days

data

10

Inspection outcomesThe inspectors undertook a wide range of activities aimed at gaining robust evidence on which to base their final professional judgement. These included:

Interviews with management teams Interviews with trainer and assessors Interviews with learners, both current and

previous Interviews with customers Confirmation of learners’ outcomes and

success rates Examination of training and assessment

records Evidence of progression for learners

The inspectors made copious notes during the inspections, which form a part of the inspection records held. On completion of the evidence gathering, the inspectors reflected on what they have seen and used their professional judgement to decide on the grade to be awarded. The inspectors compared evidence gathered with provision in the further education and work-based learning sector in England and Wales. The inspectors discussed the proposed grades with the providers’ senior managers, and agreed on the findings. The inspectors drafted a report, which was shared with the providers for factual checking. There were two further quality assurance stages that the individual reports undergo prior to publication.

Checking by an independent inspector, to ensure that the grades awarded are justified by the evidence

A final check by NSARE that the inspection report is aligned to the overall framework

Trainer observationsAs part of the inspection regime, the inspectors undertook observations of training being delivered. The observations were typically 40-60 minutes in length. After the observation the inspectors provided trainers with detailed feedback on their performance. 133 training sessions were directly observed during the baseline inspection.

Assessment only providersThe inspection of assessment only providers was undertaken using the NSARE accreditation framework. The inspectors spent one day with the provider, reviewing their management systems and records, and arranged to observe the delivery of an assessment in the work place. The observation was undertaken at night or weekends. There were six inspections of assessment-only providers.

Assessment – baseline surveyIn addition to the inspections undertaken for the assessment-only providers, an additional sample of 24 inspections were undertaken to look specifically at the assessment process. These assessments were undertaken by assessors who were part of the training and assessment providers.

Next inspectionsThe frequency of inspections is based on performance during the baseline inspections. The following criteria apply for follow on inspections:

Outstanding 24-36 months

Good 18–24 months

Satisfactory 12-15 months

Inadequate within 4 months

(Note if inadequate on second inspection provider will be removed from register)

consistency

11

Findings from the Baseline Inspection undertaken as part of the inspection of Railway Engineering Accredited Training and Assessment Providers.

This report has been distilled from the 88 individual provider reports and the 24 trackside inspections of assessments. The report identifies common trends and issues. Also contained in this report are collected data regarding the trainers and assessors themselves, from which all personal details and identifiers have been removed.

NSARE report

12

Inspection findingsThe key findings of this baseline inspection are categorised into the five key sections of the NSARE Inspection Framework:

Capacity to improve• Learner Outcomes• Quality of Provision• Leadership and Management• Equality and Diversity• Capacity to Improve

The criteria against which the providers were measured:

• The accredited training provider has a sound track record of sustained improvements

• The accredited training provider sets and meets ambitious targets to improve outcomes for all learners

• The accredited training provider has a clear vision and appropriate priorities that will sustain improvements and raise expectations for all users

• The accredited training provider’s process for self-assessment and quality improvements include appropriate use of the views of users to lead to demonstrable impact

• The accredited training provider has appropriate management structure and sufficient staff and other resources to carry through its plans for improvements

• Staff at all levels, within the accredited training provider contribute to securing sustained improvements

Due to the recent introduction of the framework it was difficult to assess the track record of providers accurately; however, judgements were made against evidence presented by the providers during the inspections. Many of the achievements identified were structured around the financial strengths and facilities offered by the providers. There was generally a lack of focus on improvements, which were directly applicable to the learners’ outcomes.

Many of the providers are very small companies often directly supported by family members; consequently, many lacked both the strategic vision to develop further to support the rail industry and the managerial resource to drive through quality improvements. The focus of many of the providers was on the day-to-day business, managing customers, delivering training and assessments, maintaining training and assessment records and invoicing clients.

Generally the Self-Assessment Plans (SARs) and associated Quality Improvement Plans (QIPs) were less robust than they may have been. This is mainly due to the fact that this is the first time many providers have undertaken this task. There were some notable exceptions to this, some from providers who have worked closely with the FE sector or who have staff from an educational management background, who have undertaken the development of SARs previously and others who fully appreciated the value of self-assessment as a managerial tool to improve quality.

“The provider has quickly grasped the impact of the recent changes that have moved the regular review process from an audit to a quality improvement model. They have made a confident start in using self-assessment, as a means of understanding the strengths and areas for improvement in training provision”.Extract from inspection report - Traxsydes

Many of the SARs were descriptive and lacked qualified judgement. The SAR is intended to be a critical self-evaluation of provision of training and assessment and should be judged against the NSARE inspection framework. The quantity of work undertaken varied extremely from half-page statements to a 36-page report. Some providers assessed their performance against the Sentinel Scheme rules and, as a consequence, did not make the cognitive shift from the previous compliance-based audit regime, to the NSARE accreditation process, which is based on a continuous improvement cycle.

The concept and structure of SARs and QIPs was explained at the road shows attended by all the providers. The Association of Rail Training Providers (ARTP) subsequently approached NSARE to provide additional workshops in SAR/QIP development; however they were unwilling to cover the cost of using inspectors to develop and deliver the workshops. The ARTP secured the support of a FE college, who provided the support pro bono.

From feedback during the inspections, it was noted that, in several instances, the SARs and QIPs were written in isolation by one manager without involving other training staff, or adequately reflecting feedback from learners and employers. Some providers relied on external consultants to produce their SAR and QIP, effectively delegating their managerial responsibility. There were several SARs produced within larger organisations, where the director or senior manager with overall responsibility for training and assessment functions, was not involved in the development of the SAR and QIP or in signing it off.

13

A limited number of providers did not grasp the importance of self-assessment.

“We have provided assessments for a wide variety of competencies for several years and have always met the criteria as laid down by Network Rail for such assessments; accordingly there can be no capacity to improve”.Extract from a provider’s SAR

The SAR is an annual submission to NSARE and the next round of inspections will expect to see a step change in the overall provision of SARs and QIPs. In determining a track record of sustained improvement, judgements will, in future, be based on progression from the previous inspection, rather than from the baseline inspection, where evidence provided by the provider was accepted on a historical basis.

Learner outcomesThe providers were measured against four subsets:

• Learners’ attainment of their learning goals• How well learners progress• How well do learners improve their economic

and social wellbeing through learning and training

• How safe does the learner feel

Learners’ attainment of their learning goals

• Learners achieve their learning goals including skills knowledge, understanding railway industry competence accreditation, nationally recognised qualifications and other challenging targets

• There are always significant variations in achievements in the attainment of different groups of learners

• Learners’ work meets or exceeds the requirements of the qualifications, learning goals or employment

• Learners participate and interact during their learning experience

The average pass rates for training delivered under the Sentinel Scheme are extremely high, to the extent that it is worth questioning the real value and accuracy of the assessment process. The current pass rates are reported to be:

Initial Training Outcomes

PTS (Personal Track Safety) 98.27%

Lookout 97.61%

IWA (Individual Working Alone) 97.12%

COSS (Controller of Site Safety) 93.17%

ES (Engineering Supervisor) 92.41%

PICOP (Person in Charge of Possession) 92.17%

SPICOP (Senior Person in Charge of Possession) 98.08%

Source NCCA Database

Pass rates at these levels raise questions as to whether the training and assessments are challenging and whether they have a positive impact on behaviours. Do they, for example, lead to a deep understanding of the risk associated with working within the railway environment?

The process for end-of-course assessments is common across all the track safety courses. There is a two-stage assessment process, where the learner firstly sits a question paper that is partly multi-choice and partly written. The pass rate is about 90%. If the learner achieves 90% or above, but has questions that have been incorrectly answered, these are put to the learner verbally and if the trainer is satisfied with the answer, the learner is awarded a pass mark. This process ensures that the learner has answered all the questions correctly.

It would be extremely unusual for educational establishments or the Further Education sector to achieve pass rates similar to rates achieved in the rail sector and this has to some extent skewed the learner-provision judgements.

There is no evidence of differentiation between groups of learners. Most of the providers do not collect data on ethnicity, gender or disadvantaged groups of learners consequently, with pass rates approaching 100% on a regular basis, the identification of differentiation is made more difficult. Without the ability to measure the effectiveness of training and any differentiation between groups of learners, it is not possible to say with any confidence that training delivery meets everyone’s’ needs and aspirations

“Many candidates currently attending the initial PTS courses are train cleaners often from ethnic backgrounds. Considerable care and guidance was provided to ensure that they felt comfortable in the training environment and that they understood the technicalities and content of the programme which was often outside their daily experience”. Extract from Inspection Report - EPPS Training Development Ltd

14

Several providers are working closely with prisons to provide job opportunities for prisoners upon release. While this is socially inclusive and supports many government initiatives, there does not appear to be any nationally developed strategy in this area. Nor is there is any evidence presented of longitudinal monitoring of the success of these initiatives.

Similarly some providers have partnerships with local FE colleges and provide training specifically aimed at young people who are Not in Employment, Education or Training (NEETs). Again these initiatives may be seen to be socially inclusive but there does not appear to be any overarching strategic direction on developing the workforce of the future. The sustainability of employment opportunities for these individuals should be questioned further. Recruitment into the industry may be better served by delivering a qualified workforce capable of attaining and progressing from a minimum level 2 qualification, thereby giving learners realistic expectations of long-term economic and social well-being.

All the learners interviewed during the inspections said they enjoyed their training; however it was the view of the inspectors that significantly more could be done to engage with learners, increase their interactivity and deepen their learning. The limited range of pedagogy used by trainers was a limiting factor in delivering outstanding training.

One particular problem highlighted was the very low number of learners on some courses – in some cases, just one. This significantly affects the ability of learners to benefit from interaction, discussion and group work. Maximum course numbers are mandated by Network Rail but not minimum.

How well learners progress• Learners develop personal skills• Learners enjoy learning and make progress

relative to their prior attainment and potential

• Learners demonstrate or develop the literacy, numeracy, language and functional skills required to complete their learning

There was limited evidence that learners make any significant progress in terms of developing their personal skills. There is an opportunity, especially around some of the higher level courses, to develop skills around team leadership, as many of the skills of a team leader are common with the skills required by a Controller of Site Safety (COSS). One limiting factor may be the emphasis on Instructional Techniques which is the basis of the qualification held by a high percentage of the current trainers.

It was clear from interviews and from analysis by inspectors of written feedback gathered by providers at the end of each course that all learners enjoyed the learning experience. Learners achieved their immediate goal of attaining the required certification, e.g. PTS, but there was no evidence that the attainment was relative to their previous experience or that the courses and training provided, allowed learners to fulfil their potential. The structure of the courses, the training and the delivery appear to be aimed at passing the test at the end of the course. Patently, more could be done to increase the learning outcomes.

There was only limited evidence presented throughout the inspections that providers make any significant efforts to increase the literacy and numeracy of learners on the range of track safety courses. Opportunities do exist within the current materials for trainers to identify and develop these skills, but these were not always taken.

How well do learners improve their economic and social well-being through learning and development?

• Learners develop relevant knowledge, understanding and skills which contribute to their economic and social well-being

• Learners increase their employability• Learners progress to further learning and/or gain promotion

Learners achieve their goal of attaining certification, which is a de facto acceptance that they have achieved the relevant knowledge through the course. Further mentoring takes place in the workplace to embed their understanding and skills. Attainment of industry wide certification furthers their economic well-being through opening up employment opportunities.

There was some evidence of progression, especially where the learners were in-house. However, there was little evidence that providers, who were not direct employers, monitored the progression of individuals, except when seeing them on further training, if and when they progressed.

“Learners enjoy and participate fully in the training they undertake. They acquire knowledge, understanding and skills effectively and successfully attain nationally recognised qualifications. The company has some notable examples of where some have progressed rapidly from initial training to become highly qualified, senior employees of the company”. Extract from Inspection Report - Stobart Rail

15

How safe do learners feel?• The extent to which health and safety is

embedded into the delivery of training• Learners use safe working practices in

learning and at work• Learners say they feel safe

All the learners questioned as part of the inspection process felt safe in the learning environment. There was a constant reference from all trainers regarding safety which inculcates safe working behaviours from the learners. The trainers used their own extensive experience to illustrate the importance of safe practices. The training centres visited supported a variety of safety campaigns led by their respective parent companies and the Confidential Incident Reporting and Analysis System for the rail industry (CIRAS) was well promoted.

“Learners feel safe, both in the learning environment and when on track. This is because provision puts a strong emphasis on safety procedures and the trainer uses his expert knowledge and experience of the rail industry, to exemplify and illustrate the importance of safety and the repercussions of not following safety protocols”. Extract from Inspection Report - West Coast Training PLC

Quality of provisionProviders were measured against three subsets:

• How effectively does training and assessment support learning and development?

• How effectively does the provision meet the needs and interests of the railway industry, employers, employees and individual learners?

• How effectively does the accredited training provider use partnerships to develop its provision to meet the needs of the railway industry, employers, employees and individual learners?

How effectively does training and assessment support learning and development?

• Interesting and appropriate learning methods and resources inspire and challenge all learners and enable them to extend their knowledge, skills and understanding

• Technology is used effectively to promote and support learning where appropriate

• Staff have the appropriate skills and expertise to provide good quality learning, assessment and information and support services for each learner and maintain and enhance their ability by Continuous Professional Development

• Assessment of learners’ performance and progress is timely, fair, consistent and reliable

• Learners receive constructive feedback on their progress and how they might improve

• Learners receive help to develop literacy, numeracy and language skills to support the attainment of their main learning goal

• Learning, training and assessment promotes equality and supports diversity

The quality of provision is generally good, but the question must be asked why it was not outstanding. Of the 133 observations of training that were undertaken as part of the inspection process, there was very limited evidence of inspirational training being delivered. Much of the training delivered, while considered competent, fell well short of being inspirational.

“Improve the quality of training, so that learners receive a frequent diet of inspirational training moments that make their learning strikingly memorable and long lasting”.Extract from Inspection Report - Weld-a-Rail

During each inspection, the inspector spent time in the classroom and occasional site visits, observing the delivery of training. This was normally a short session of between 40 minutes and one hour. Following the observation the inspector gave individual feedback to the relevant trainers. There were some common themes throughout the observations of training:

• There was effective use of PowerPoint presentations and videos provided by Network Rail

• Trainers made good use of personal experience, to supplement, contextualise and enliven course materials

• Trainers generally establish good rapport with learners early in the course and were generally effective in using humour

• Question and answer sessions were widely used and appreciated by learners, however, questioning techniques were underdeveloped. Broader use of different questioning styles would benefit learners further

16

• Use of varied learning resources were under-used e.g. whiteboards/flipcharts

• Use of group discussion and exercises could be more widely applied

• Many sessions were too long, with too much information in one session

• Much of the material was delivered by talking and listening

• Use of objects and artefacts that learners can handle was not routine

The range of classroom facilities is very wide, from what would be considered outstanding – bright, spacious, well-appointed and offering a good learning environment – to a selection of inappropriate, portacabin-style facilities that significantly hampered the delivery of effective learning. NSARE is about to embark on a separate nationwide programme to identify and classify all rail training facilities.

There was one significant incident where the trainer constantly used extreme and inappropriate language. When challenged by the inspector, the trainer referred to the Network Rail video containing similar, but less extreme language. While the language used in the Network Rail video may be acceptable for post watershed broadcasting, it could be considered inappropriate for use in the training environment, where watching the video is an essential part of the learning.

“The trainer promotes equality and diversity well by highlighting instances of conversation, which may cause offence in the videos”.Extract from Inspection Report - URS Scott Wilson

Trainers regularly use humour as an effective learning tool, however caution should be exercised to ensure that cultural and individual beliefs are not compromised by inappropriate language.

While a limited number of providers make good provision to develop learners’ literacy, numeracy and communication skills during training sessions, there is no evidence of this being a wide-spread practice

“Some described feedback as “excellent” because it helps them to achieve their goals. If they ever require additional help, the trainer provides effective one-to-one support in a sensitive manner, often outside of the usual hours of training”.Extract from Inspection Report - Distinction Training Ltd

Some providers work extensively with learners who have English as a second language. Where this is the case, the trainers are skilled at using appropriate learning styles for these learners.

Many of the difficulties reported by learners stem from a lack of understanding of the language used in the written assessments. There is little evidence of formal assessment of individuals’ literacy and numeracy ability prior to attending the course and only cursory assessment during the introduction stage. Learners are often invited to “let the trainer know” if they have problems.

Where an individual learner has declared dyslexia or other learning difficulty, adequate processes are adopted throughout the learning provision, however this is generally only triggered by the learner declaring a difficulty, rather than the employer advising the training provider in advance.

“The training provider, through working with the local further education college, has done much to equip its staff with the skills necessary to support learners who may have additional learning needs. These include those learners with dyslexia, autism and other needs. During the course of sessions attention is given to supporting learners’ numeracy needs in relation to some of the specialist terms and measurements that are peculiar to the rail industry. This attention could usefully be applied to potential literacy needs of learners through the use of identified ‘key words’”.Extract from Inspection Report - Keltbray Training

The assessment undertaken at the end of courses to confirm initial knowledge and understanding, were rigorously carried out by the providers. Where there were learners who required additional support and had previously declared learning difficulties, providers made adequate provision to support them in line with the Sentinel guidelines.

“Learners are given alternative verbal methods of assessment where it is previously known to the provider that they have literacy difficulties”.Extract of Inspection Report - J Murphy and Sons

17

All providers undertake a process of regular observation of the training they deliver, at a minimum interval of six months. These observations are carried out either by peer observations or, in larger organisations, by the training manager. The previous audit regime used a tick sheet approach to trainer observations which has been adopted by many of the providers. The feedback has, in many cases, been weak and has not identified area for continuous improvement.

How effectively does the provision meet the needs and interests of the railway industry, employers, employees and individual learners?

• The range and context of provision provides learners with the appropriate choice of subjects, levels and qualifications that are relevant to their medium and long-term personal career paths

• The accredited training provider considers employers views, identified training needs and information on performance, skills and labour demands

• Arrangements for training and assessment are flexible to suit employer and learner needs

The range of training offered by a significant proportion of the providers is limited to track safety related subjects. While the major contractors, and some of the other providers, offer a range of technical training in railway engineering, the emphasis on track safety means there is only limited opportunity for the development of long-term career paths and the providers are not best placed to expand the career choices of the learners.

Where training providers are part of a larger organisation, or work in close partnership with large employers, the strategic needs of the employers are taken into account when planning delivery. However the majority of training providers work reactively, either offering open courses, or providing one-off training events to customers

The delivery of training and assessments is very flexible, with training providers willing to deliver during nights and weekends as required.

“Training provision is highly flexible and is often provided at weekends and evenings. The provider has particularly detailed and structured progression routes for learners. The provider has outstanding partnerships. An established partnership with emergency services has increased awareness on working safely during emergencies on the railway. Links with partner organisations to train unemployed people are highly effective to improve participation of under- represented groups in the industry and provide employment in local communities”.Extract from Inspection Report - Scot-Train Ltd

While the providers generally offer a very flexible service to the industry, there were several cases where the inspectors observed inadequate planning and where the information exchanged between the provider and their customers caused confusion, with the consequence that learners either arrived at the wrong time, or did not arrive at all. This confusion led to training being delayed and low numbers of delegates, all of which had a detrimental effect on the learners.

How effectively does the accredited training provider use partnerships to develop provision to meet the needs of the Railway industry, employers, employees and individual learners?

• The provider develops partnerships with employers, local colleges and others that lead to demonstrable benefits to learners

The majority of providers do not have partnerships with local colleges or other learning organisations, however, where they do exist, they have been particularly well developed and benefited all partners and especially the learners.

“The provider has developed a range of outstanding partnerships to develop provision to meet the needs of employers and learners. Partnerships with colleges have enabled combined resources and expertise to provide programmes in prisons and across the north for apprentices. Highly effective links with railway museums have improved the range of resources and experiences learners gain through active track work. The response to lifelong learning is highly effective”.Extract from Inspection Report - Trackwork Ltd

Leadership and managementProviders were measured against five subsets:

• How effectively do leaders and managers raise expectations and promote ambition throughout the organisation?

• How effective are the measures for the governance of accredited training providers?

• How effectively does the accredited training provider engage with users to support and promote improvements?

• How effectively does self-assessment improve the quality of provision and outcomes for learners?

• How effectively does the accredited training provider use its resources to secure value for money?

18

Only two providers were judged to be outstanding in this category and of the “Good” providers a significant proportion were judged to be satisfactory for Leadership and Management. With the large number of small providers in the industry, it is understandable that more management effort is put into maintaining the day-to-day business, rather than thinking about the long-term future.

How effectively do leaders and managers raise expectations and promote ambition throughout the organisation?

• Leaders promote very high standards in a positive and supportive culture that aspires to excellence

• The approved training provider raises expectations through a clear and realistic strategy for planning and developing learning programmes and services

• Demanding targets are set and met throughout the organisation

• The provider uses data and information on learners and employers needs and local and national skills needs to plan and review the provision of service

• Resources including staff, accommodation, facilities and technologies, are developed and used to support learning effectively

Many providers do not aspire to excellence, but are content with achieving compliance. Leadership in the training field, although not universally weak, has an underlying tendency to maintain the status quo. There are pockets of good management, where the efforts of the business leaders has had a direct impact on the learners, however in many cases it was noted that directors and senior managers were focused on financial and resource issues.

There was no evidence presented to identify where strategic data was used to plan the training resources required. There was ample evidence of data being used tactically to plan for recertification courses and assessments. Some providers used the NCCA database as a key source of notifications on expiry dates for Sentinel-related competencies. This is done on a daily/weekly basis.

While there were some excellent learning facilities noted throughout the inspections, these were balanced by a significant number of sub-standard buildings with inadequate welfare facilities. NSARE are about to undertake a survey of all training facilities in the UK.

“The company is well staffed, has excellent resources as well as impressive purpose-built accommodation and training facilities, which are exceptionally well maintained and managed. Learners appreciate the high quality of facilities at the two training centres and they are a significant factor in learners completing their courses successfully”.Extract from Inspection Report - Babcock Rail

How effectively do directors provide leadership, direction and challenge?

• Directors set the mission and strategic direction of the accredited training provider

• Directors establish effective structures to monitor all aspects of the accredited training provider’s performance

• Directors ensure that their statutory duties, where applicable, are fulfilled

• Directors ensure that suitable and sufficient processes are in place to maintain documentation up to date and brief the contents and changes to relevant staff

• Directors ensure that effective processes are in place to prevent irregular or fraudulent issue of certificates of competence

A substantial proportion of the training provision is undertaken by small providers, many of whom are in the commercial training market. There is a lack of strategic vision especially among these providers. Where the training provider is part of a larger organisation, there is evidence that the delivery of training and assessments is delegated to the training managers, who, in some instances have lacked direction from senior managers and directors.

The Sentinel Scheme Rules and Network Rail standards identify all the statutory requirements for the delivery of training and assessment in the rail sector. To assure compliance with the requirements of the Sentinel Scheme Rules the providers are required to up-load copies of all their relevant procedures and other statutory documentation onto the NSARE SkillsBackbone™. These documents were reviewed as part of the desk top compliance audit carried out by NSARE. While there was a wide range of approaches to the procedures, ranging from the brief to the overly comprehensive, they were all fit for purpose.

It was noted that while many trainers did not have a comprehensive knowledge of all their procedures, they had sufficient working understanding of the salient issues to deliver the day-to-day training effectively.

The general control of Sentinel cards is fit for purpose, however, since taking responsibility for the accreditation of training and assessment providers in January 2012, there have been several incidents of irregularities in the issue of Sentinel cards. These have been fully investigated by NSARE, Network Rail and the British Transport Police. Appropriate action has been taken against individuals and providers.

19

How effectively does the accredited training provider engage with users to support and promote improvements?

• The accredited training provider implements and monitors an effective strategy to involve learners and employers in the decision making of the organisation

• The views of different user groups are canvassed and their views acted upon to plan, manage and improve provision

• External partnerships are promoted to ensure that the needs of learners at all levels are met

All training providers have processes for gathering feedback from learners at the end of each course. This system is colloquially referred to as “happy sheets”. These “happy sheets” are of variable quality and the questions asked do not always illicit reflective answers from the learners. While there is a considerable amount of data collected, and in some cases transcribed to spread sheets etc., there is limited evidence of effective

learner feedback. There was limited evidence of quality feedback from learners, collected in a reflective process rather than a reactive process, issued when learners are often ready to leave the training sessions.

There was limited evidence of feedback gained from sources other than learners post course “happy sheets”. Some providers were able to share employer endorsements with the inspectors and while this is an indication of customer satisfaction, it is not an accurate process for gathering feedback, with the aim of improving learner outcomes or customer engagement.

Where external partnerships have been implemented, they have generally been successful; however the majority of training providers have not yet embarked on this journey

transforming

20

Equality and diversityHow effectively does the accredited training provider actively support equality and diversity and tackle discrimination?

• Manages equality and diversity, particularly disability, gender and race and actively promotes equality and diversity among staff, learners, employers

“While providers are aware of the need for equality and diversity and policies are in place and have been briefed out to trainers and assessors, there is a general lack of focus on the issues. There was insufficient promotion of this on a day-to-day basis. Most providers do not collect or analyse data relating to the performance of different groups of learners. There was very little evidence of promotional material displayed in the training rooms or shared areas. There was little evidence in the training programmes that the published Equal Opportunities and Diversity policy is applicable to learners. Much of the promotional materials and in-house publications were observed to be particularly gender specific.

“A number of providers have invested in equality and diversity training for trainers and assessors; however, even where the training has been provided, many trainers and assessors are not applying this and acting as role models. There is by and large a fundamental lack of management attention on this key area. Many providers do not have a specific process for logging incidents or complaints in this field.

“VGC demonstrates an on-going commitment to development and provision for equalities and is an agenda item on all meetings including the Directors’ monthly report, regular “Be Safe Briefing” newsletters and Business Management Systems. The VGC Group staff handbook includes equalities polices and code of conduct polices including whistle-blowing procedures.

“A range of initiatives and training programmes are in place to recruit from a broad range of backgrounds that reflects the local community. Regular reports show positive outcomes in recruiting a diverse work force. EDI data show the percentage of workers by age-groups, gender, disability, pre-employment status, religion and sexual orientation”. Extract from Inspection Report - VGC

• Assesses the impact of its work in relation to equality and diversity and has taken appropriate action in response to the findings

• Makes sure that training in equality and diversity is effective so that directors, leaders, managers, staff and learners understand their roles in relation to equality and diversity

• Makes sure that all staff and learners are protected from harassment, bullying and discrimination

• Manages incidents and complaints, specifically about disability, gender and race equality

equality

21

22

As part of the baseline inspection regime it was agreed that a sample of assessments from training providers, who also provide assessment services, would be undertaken, with 24 inspections of on-site assessments carried out. These inspections of Competence Assessments were mainly carried out in a trackside environment, covering a range of skills including COSS, Engineering Supervisor, Machine and Crane Controller and Safe System of Work Planner.

The assessments were carried out in a professional manner, by appropriately qualified and experienced assessors, ensuring that all the requirements of the Sentinel Scheme Rules were observed. The assessments followed the protocol issued by Network Rail and made extensive use of the associated question banks.

The assessment decisions were seen to be fair and objective and reflected the knowledge and skills displayed by the candidates. The collection of evidence was rigorous. It was noted that during the observed assessments not all candidates were successful and where there were failings the assessor gave constructive feedback on the areas to improve on.

Where the assessments were carried out using real-time evidence in a live environment, the ability to use the full question bank was limited, due to resources available. The theory questions asked were done in a variety of locations, which may not be fully conducive to the task. Examples of locations where the questions were asked included, the roadside, sat in company vehicle parked at the work site, using a mobile welfare vehicle and in a Motorway service station.

There was one example where two candidates were being assessed by one assessor at the same time. This was identified as being a particularly disjointed approach and while, in the end, the judgement of competence was rigorous, this approach should be questioned.

In addition to the verbal feedback given to candidates, which was observed to be detailed, some assessors did not follow up with written feedback to the candidate or the employer.

There was a view expressed by several assessors that there are often additional pressures applied in planning assessments as a result of employers leaving it to the last minute before requesting assessments. This makes adequate planning more difficult.

Assessment of competence

23

During the inspections, the inspectors observed 133 individual trainers delivering training. The majority of the observations were of track safety courses. The direct observations were supplemented by interviews with both trainers and learners. The general feed-back from the inspectors was:

Trainers are enthusiastic about the rail industry

Trainers demonstrated in depth knowledge of their subject area

Safety is paramount and learners feel safe Trainers are inclusive and embrace a wide

range of learners from a wide cultural and ethnical background

The training delivery was competent, but lacked the inspirational element to make training really effective. The skills exhibited by the trainers, by and large reflected an instructional approach to delivery, focusing on knowledge transfer and failing to recognise the behavioural aspects of training.

Trainers use a limited range of training techniques Trainers speak for too long and impart too much information

in one session Trainers do not use sufficient group exercises to aid learning Trainers use a limited variety of pedagogy Limited evidence of trainers supporting literacy and numeracy

development Trainers are too rigid in following the Network Rail training packages

Training skills

excellence

24

25

Continuing Professional Development (CPD) The trainers are working in an environment where the majority of their peers have equivalent or possibly slightly higher

qualifications in training and development. Many of the trainers have extensive practical experience of training delivery and may be well placed to coach relatively new trainers, however their ability to coach training skills at a higher level is hampered by their lack of skills themselves. As 70% of providers are small businesses, employing four or less trainers, the support mechanisms that would normally be expected are not in place. Without the necessary expert guidance, it is therefore difficult for these organisations to engender a culture of continuous improvement.

Training qualificationsTraining qualifications held by each individual trainer/assessor were inputted to the NSARE SkillsBackbone™ during the initial registration process. Trainers additionally loaded an electronic copy of their certificates, which were validated against their data entry. There was a wide variety of training qualifications declared by the trainers. These qualifications have been evaluated against the Qualifications and Credit Framework (QCF) issued by the Office for Qualifications and Examinations Regulation (OFQUAL) for Teaching in the Life Long Learning Sector. Each qualification has been awarded an equivalent QCF using Standards Verification UK guidelines as a benchmark. Training qualifications were then divided into the following categories:

Entry Level 3 Intermediate Level 3 Intermediate Level 4 Advanced Level 4+

The qualifications held by trainers in the Rail Industry are summarised below:

Range of Trainer Qualifications

Trainers’ Level of Qualification

26

CPD, as a concept, is not generally well understood, or applied in line with the generally accepted approach. CPD, as a process, has been mandated on the rail trainers for a number of years but has focused on knowledge specifically around the delivery of the Sentinel suite of courses. Trainer examinations were introduced in 2006, using an on-line question bank managed by Question Tools. Network Rail set the questions used in the on-line assessments and the process gives some degree of confidence that the track safety trainers are maintaining their knowledge. However it could be argued that trainers maintain the knowledge base by the regular delivery of relevant training and that it should be the responsibility of the employing company to ensure that their knowledge is maintained up to date and that the trainers have the necessary skills to deliver. This approach is only used for specified track safety competencies and is not widely applied to engineering courses.

There was little evidence of CPD being used to enhance training delivery skills, or behavioural issues around training delivery. There are regular training observations carried out, but this generally done by a peer with the same level of experience as the trainer who is being evaluated and not by training managers with a significantly higher skill in training and development. Many observations use the trainer assessment check lists produced by the previous audit provider, which is fundamentally a tick sheet. The National Competence Control Agency (NCCA), which is the overarching data management system for all Sentinel training and assessment records, is advised at the conclusion of each six month CPD cycle that individual trainers have completed their CPD. There was little evidence presented during the inspection cycle that CPD is generally effective within the rail training sector.

There was limited evidence displayed during the inspections of peer to peer sharing of best practice. Where team meetings were commonly used the focus was on managerial, financial and operational matters and did not focus sufficiently on enhancing the learner experience.

An additional problem in the rail sector is that many of the trainers have a relatively low educational base from which to progress. This is an indication that many trainers have “risen through the ranks” and become trainers. This may be one reason that the support offered to learners with literacy and numeracy may fall short of what would be generally expected. It is expected in the Further Education (FE) sector that trainer would hold qualifications at least one level higher than they are teaching and ideally two levels. As the majority of training in the rail sector is at level 2 we would expect applying general FE Sector guidelines that most of trainers would have at least a level 4 qualification, in either academic or vocational areas.

Trainers’ Educational Qualifications

27

Age profile of trainers/assessorsThe average age of the trainers and assessors is 50 years old. Only 2.6% of trainer and assessors are currently aged 30 years or below, whereas 16% are 60 years old or above.

0 0 0 0 0

1 1

0

2

3

2

5

6 6

4 4 4

6

2 2

4

7

15

16

17

18

23

10

11

15

20

15

18

20

14 14

18

10

20

17

12

14

5

7

8

9

4

1

0

1

2

0 0

1 1

0

1

0 0 0 00

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80

No.

of T

rain

ers

Age of Trainers

An indication of the average length of time in a training and assessor position is identified by the elapse of time since the initial award of the training qualification. It ranges from a few months to 36 years, with an average of nine years.

Trainers’ Underpinning Educational Achievement

Level 1: General Secondary Education

Level 2: GCSE A-C or equivalent / NVQ Level 2

Level 3: A’ Level / BTec / ONC / NVQ Level 3

Level 4: HNC / NVQ Level 4

Level 5: HND / Foundation Degree

Level 6: Degree

Level 7: Higher Degree

relevant

Age Profile of Trainers

28

As can be identified by the graph below, there is a considerable fall after two years. This is in line with the practice in other industries, where trainers are drawn from front line positions and develop a range of skills within training, which then facilitate their return to front line operations at a higher level, where they apply both technical and managerial skills developed during their tenure in training. Following this model, it would be expected that the number of years since initial qualification would continue to fall, with the residue of trainers and assessors being the ones who have developed a career within the field and who have progressed to be the senior, experienced skill leaders. The current structure of the rail training sector may not be fully suited to this more general model due to the high percentage of small training providers.

skilled

No. of years elapsed since gaining qualifications

29

Appendix 1Results of the Baseline Inspections in 2012

standards

“The first, and most important, lesson is that no education system can be better than the quality of its teachers.”Rt Hon David Cameron MP

Extract from The Importance of TeachingThe Schools White Paper 2010Department of Education

30 31

www.nsare.org

NSARE Limited40 Bernard StreetLondon WC1N 1BYTel: 020 7841 8128Email: enquiries@nsare.org