1

Work Programme National Statistics Background Information Note

2

Contents 0. Introduction .................................................................................................. 4

1. Structure of the Work Programme ............................................................... 5

1.1. What is the Work Programme? .................................................................... 5

1.2. Joining the Work Programme ....................................................................... 5

1.2.1. Referrals ............................................................................................ 5

1.2.2. Attachments ...................................................................................... 6

1.2.3. Payment Groups ............................................................................ 6

1.3. Contracts and Providers .............................................................................. 7

2. Outcome Payments of the Work Programmes ............................................. 9

2.1 Job Outcome payments ........................................................................ 9

2.2 Sustainment payments ......................................................................... 9

2.3 Payment validation ....................................................................................... 9

2.3.1 Pre-Payment validation ................................................................... 10

2.3.2 Post-Payment validation .............................................................. 10

2.3.3 Sustainment payment validation ..................................................... 13

2.3.3.1 Pre-Payment validation ................................................................ 13

2.3.3.2 Post-Payment validation .............................................................. 13

2.4 Unclaimed job outcomes ..................................................................... 13

2.5 Completing the Work Programme .............................................................. 14

3 Performance measures of the Work Programme ................................... 15

3.1 Contractual measure ....................................................................... 15

3.1.1 Transparency measure ................................................................ 16

4 The Statistics ............................................................................................. 17

4.1 What is in the Work Programme National Statistics? ................................. 17

4.2 Where can I get the Statistics? .................................................................. 18

4.2.1 Statistical Summary......................................................................... 18

4.2.2 Stat-Xplore ...................................................................................... 18

4.2.3 Cohort and Time series data ........................................................... 19

4.3 Publication dates ........................................................................................ 19

5 Who uses Work Programme statistics? ..................................................... 21

5.1 User Engagement ............................................................................... 21

5.2 Known users and uses........................................................................ 21

3

5.3 User experiences ................................................................................ 23

6 Development of the statistics.................................................................. 24

6.2 Historical validation factor improvement .......................................... 25

7 Sources of the statistics ............................................................................. 27

8 Limitations of the statistics ......................................................................... 28

9 Quality Assurance ...................................................................................... 31

9.1 Initial quality assurance conducted during data development ............. 31

9.2 Routine Quality Assurance ................................................................. 31

Feedback .......................................................................................................... 33

Useful links ........................................................................................................ 33

4

0. Introduction

This note explains

1. The Work Programme: what it is and how it works

2. Work Programme payment systems and validation processes.

3. Work Programme National Statistics: what is included and why, how the statistics are put together, plus when and where they are published.

In addition to the Work Programme a number of other employment programme schemes, some of which are mandatory, have also been introduced to give Jobcentre Plus choice over what support to offer to claimants in their area based on claimant and local labour market characteristics.

To find out how the programmes fit together, view the full range of Employment Support Programme Official Statistics, find out when they will next be published and access the National Statistics website for the Work Programme visit the Employment programme official statistics page.

5

1. Structure of the Work Programme 1.1. What is the Work Programme?

Work Programme is one of a number of welfare to work reforms aimed at getting unemployed people into lasting work. It was launched throughout Great Britain in June 2011 and replaced much of the previously available employment support, including the New Deals, Employment Zones and Pathways to Work.

Work Programme is delivered by organisations called Providers, working under contract to DWP; see section 1.3 for more details. Providers work with claimants sent to them from Jobcentre Plus at specified points in their claim.

Providers are paid when an individual reaches six months in work, or three months for those expected to require more assistance. Providers are also paid for further work beyond this, up to a maximum of an additional one year or a little over a year and half for the harder to help groups; see section 1.2.3. Individuals generally remain on the Work Programme for a maximum of two years.

Work Programme Provider Guidance: Chapter 1 provides further introduction.

1.2. Joining the Work Programme

1.2.1. Referrals

Claimants are referred to the Work Programme by Jobcentre Plus.

Claimants have different entry points and different participation requirements, depending on their circumstances (such as age and benefit type).

Eligible claimants at the referral point will be claiming: Employment and Support Allowance (ESA). Jobseekers Allowance (JSA). Income Support (IS) (in England only). Incapacity benefits (in England only). Pension Credit Universal Credit

In releases up to June 2015 the statistics on the Work Programme did not include persons receiving Universal Credit as this benefit had not been running for sufficiently long. From the September 2015 release, figures on Universal Credit are included. People who are claiming Universal Credit at the time of their referral are assigned to legacy Work Programme Payment Groups.

6

For some claimants joining the Work Programme will be mandatory; some will have the option to volunteer. Work Programme Provider Guidance: Chapter 2 provides more detail. Participants receive 104 weeks of support on the Work Programme from the date of Referral and participants cannot be re-referred. See section 2.5 for more information on completing the Work Programme. New Referrals to the Work Programme ended in March 2017.

1.2.2. Attachments

Once a person has been referred to a provider, the provider is responsible for contacting them to discuss the programme and begin planning the steps needed to support them. Once this activity has taken place, the provider registers an attachment to the Work Programme. The statistics on Referrals show the number of people referred to the Work Programme by Jobcentre Plus. The statistics on Attachments show the number of people with whom the provider has completed initial engagement activity.

The number of Attachments at any point in time will always be slightly lower than the number of Referrals since an Attachment occurs after a Referral and some claimants may sign off benefit or move into work after Referral but before Attachment.

Over the lifetime of the Programme to date, the average (mean) time taken to attach following Referral is around 17 days. However there is evidence that this process is speeding up. The mean time taken for referrals to attach in the first (contractual) year of the programme was around 20 days whilst for the fifth year it is now around 11 days.

If a claimant who has not been attached to the Work Programme makes a further claim to benefit over the two years after they have been referred to the Work Programme, then the provider will engage with and attach the claimant at this point. Around than 1.5% of all Referrals never attach during the 104 weeks allotted time.

1.2.3. Payment Groups

Work Programme providers are paid primarily for the results they achieve and they are paid more for supporting people who are harder to help (Work Programme Provider Guidance: Chapter 9 provides more detail on payments). Upon referral, claimants are assigned to a Payment Group based on their circumstances. They are then randomly allocated by Jobcentre Plus to a provider in their area according to the market share of the providers.

7

A summary of payment groups is given below, Work Programme provider Guidance: Chapter 2 provides more detail.

1. JSA claimants aged 18-24 2. JSA claimants aged 25 and over 3. JSA Early Entrants 4. JSA ex-Incapacity Benefit(IB) claimants 5. ESA Volunteers 6. New ESA Customers

a. excluding 12 month prognosis claimants b. 12 month prognosis claimants only

7. ESA ex-IB claimants 8. IB/IS Volunteers 9. JSA Prison Leavers

Payment group 9 was introduced in March 2012 to provide immediate employment support to Jobseeker’s Allowance claimants who have made an advanced claim prior to leaving prison and for those prison leavers who make a claim to Jobseeker’s Allowance within 13 weeks of their release.

Referrals to payment group 9 started on 15th February 2012 due to prison leavers making advanced benefit claims. Attachments (see 1.2.2) to payment group 9 started from the 1st March 2012.

Referrals to payment group 6b (New ESA customers with 12 month prognosis) started on 12th November 2012. These Referrals are mandatory after the claimant’s Work Capability Assessment outcome.

Payment group 8 was funded through a European Social Fund allocation that has now closed. The payment group ceased taking Referrals in March 2015. This is reflected in the statistical releases from June 2015 onwards.

Payment Groups are assigned by Jobcentre Plus on the basis of a claimant’s circumstances, and benefit they receive. Figures will reflect the information recorded on the administrative systems, which includes some known minor inaccuracies. This is most notable in the JSA 18-24 and JSA 25+ payment groups, which contain some claimants who are not in the specified age groups. Universal Credit claimants are also assigned to the legacy payment groups recorded by the administrative systems.

1.3. Contracts and Providers

The Work Programme is currently delivered by 15 private, public and voluntary sector organisations, working under contract to DWP. These organisations are known as Prime providers.

8

For the purposes of Work Programme contracts, DWP divided the country into 18 contract package areas (CPAs). In each contract package area there are at least two prime providers and in some contract package areas there are three prime providers. This means that there are 40 individual contracts for the delivery of the Work Programme. The same prime provider may hold multiple contracts, provided that these are in different CPAs.

In late 2014 the Department renegotiated its contracts with prime contractors. As a result of these changes the Department has now brought in a new, performance measure based on the actual number of Referrals, against which providers’ performance is assessed. Formal performance interventions can now take place on a quarterly rather than annual basis.

The Department has also made a series of changes to the validation procedures. More details can be found in section 5.2 and Work Programme Provider Guidance: Chapter 9.

Providers working with DWP are expected to achieve the Merlin Standard of behaviour. Guidance for providers on how to administer the programme and the minimum service delivery standards providers have agreed to are published.

Prime providers have supply chains of organisations that provide services. Subcontractors in tier one of the supply chains are responsible for delivering the end to end process or a specific element of the service, such as job-broking. These partners are usually paid by results. Second tier organisations work on a call-off basis, as and when a prime provider judges a participant could benefit from that organisation’s help. The voluntary and community sector are represented at all levels of Work Programme delivery.

Administrative data is held against only the prime providers; as such the Work Programme National Statistics only report figures against the prime contracts.

9

2. Outcome Payments of the Work Programmes

Providers are paid when the claimants complete defined periods in work. Providers are expected to deliver 104 weeks of continuous support regardless of whether a participant changes benefits, or moves into employment.

Participants complete the Work Programme when they have either completed 104 weeks on the scheme or achieved the maximum number of Sustainment Payments permitted, unless a participant completes early.

Work Programme Provider Guidance: Chapter 9 provides more detail on payments. Work Programme Provider Guidance: Chapter 10 provides more detail on completing the Work Programme.

2.1 Job Outcome payments

A Job Outcome payment will be made after either three (for payment groups 3-8) or six (for payment groups 1, 2 and 9) months of continuous or cumulative spells in employment. Only one Job Outcome payment can be paid per participant.

2.2 Sustainment payments

Following a Job Outcome payment, Sustainment payments are paid for every subsequent four week period in continuous employment.

The provider will receive a maximum of 13 Sustainment payments for payment groups 1, 2 and 8, a maximum of 20 Sustainment payments for groups 3,4,5,6 and 9 and a maximum of 26 Sustainment payments for payment group 7.

A Job Outcome payment and subsequent Sustainment payments can be claimed beyond the two year point (up to the payment group maximum number of weeks) so long as the claimant began the employment during; and does not have a break in employment outside of, the 104 weeks maximum Work Programme period. Following a break in employment (2 calendar days or more) after the 104 week period a provider will no longer be eligible to receive further payments.

Payments to providers are subject to validation procedures. More details can be found in section 2.3 and Work Programme Provider Guidance: Chapter 9.

2.3 Payment validation

The outcome validation regime went live for the Work Programme in April 2012. The introduction coincided with an update to the Work Programme payment system on March 26th 2012 which supported the process.

10

This regime involves a number of validation procedures which are performed on Job Outcome payments and Sustainment payments paid to providers to ensure that only valid outcomes are paid for.

2.3.1 Pre-Payment validation

All Job Outcome payment claims are subject to an ‘off benefit check’ before payment. This involves an automated check to match participant information with the Customer Information System (CIS) to ensure that participants for whom Job Outcome payments are claimed were not claiming benefit during the reported period of employment.

Initially, Job Outcome payments failing the off-benefit check were subject to further validation and only paid when further in-work checks direct with employers/participants were successful. From October 2012 this off benefit process was refined to reduce the number of valid claims failing the initial automated check and requiring further validation before being released for payment.

In September 2012 around 8,000 Job Outcome payments had accumulated from the beginning of the programme, which had incorrectly failed the initial check and were awaiting further validation. These were released for payment in September 2012 and were fully validated through the post-payment process.

The automated off-benefit check now has an increased window, from the initial 2 days to 15, in which the check is applied, to allow for minor discrepancies between the details of the provider’s claim and the details on Departmental systems. Job Outcome payment claims that fail this automated check are removed from the system (unless they can be validated) and not paid.

Claims which pass the off-benefit check are released for payment, and are then subject to further post payment in-work checks. This validation was not performed on cases which failed the initial off-benefit check as they were already subjected to further in-work checks.

2.3.2 Post-Payment validation

A validation rate is calculated and used to strengthen the controls against fraud and error in the Job Outcome payments claimed by Work Programme providers. The rate helps extrapolate financial recoveries against all payments made to a contract in the extrapolation period, rather than for the sampled claims alone.

To calculate this rate, a sample of 17 claims per contract is randomly selected every month from the total population of Job Outcome payments that passed the automated off-benefit check and were subsequently paid in that month. For outcomes paid from April 2016, the HMRC check examined Real Time Information

11

(RTI) data to validate employment, rather than p45 data as previously used. This allows for better coverage and more accurate identification of employment spells. Failed HMRC checks are validated by confirming employment with either the employer or the individual.

Job Outcome payments that are found to be invalid are used to calculate the validation rate which is extrapolated across the total population (the sample has been chosen to be large enough to enable us to extrapolate). DWP reviews each extrapolation round to ensure that it operated as intended.

Once the percentage of invalid payments (from October 2014 a proportion of payments which could not be fully validated will be classified as invalid) has been calculated from the sample, the validation rate is applied to the total population minus the sample. The ‘fails’ in the sample are ‘backed out’ and money recouped.

Time is allowed for providers to challenge and for DWP to assess and arbitrate challenge. This process can take up to approximately 3 months, therefore the National Statistics on Job Outcomes for some providers could change slightly in the following release.

The validation rates are also used to derive adjustment factors which are then used to rate the National Statistics. These are derived using the number of the Job Outcomes which fail the post-payment validation process divided by the total number of Job Outcomes sampled. This ratio is applied to Job Outcomes (less the sample and those already validated) to adjust the National Statistics.

The 40 adjustment factors (one for each contract) are derived by the payment validation team every extrapolation period and sent to the National Statistics production team. These are then used to ‘rate’ the Job Outcome data from Provider Referrals and Payment System (PRaP) to produce the National Statistics. This is important for the accuracy of the statistics, consistency between financial recoveries from providers and counts/rates of Job Outcomes paid.

Prior to April 2013 monthly samples of 100 were brought together every two months. From April 2013, this moved to monthly samples of 33 brought together every three months. This was in response to the views of users of the statistics and supported a quarterly statistical publication (within three rather than four months of the reporting period and on a quarterly basis aligned to the financial year). From October 2014, the extrapolation period became every six months with monthly samples of 17. The results of 6 rounds of validation (one for each month being brought together every six months aligned to the financial year (April-September, October-March) to provide validation rates.

DWP have previously carried out analysis using a one off sample of HMRC RTI data to the end of December 2013; examining off benefit periods covered by employment for Work Programme participants.

12

Since the RTI system was not fully rolled out until October 2013 this sample data does not include information for all people who have taken part in the Work Programme. However by identifying all weeks off benefit containing employment from the period October-December 2013, an estimate for the full duration of the Work Programme for all individuals can be made by applying rating factors to weeks off benefit prior to October 2013.

Using this methodology it is estimated that in total to the end of December 2013 around 26 thousand individuals satisfy the conditions for a Job Outcome payment but no Job Outcome payment has been claimed (by the end of March 2014). Whilst alternative methodologies may result in different estimates, we expect at least 20 thousand individuals to satisfy these conditions.

The December 2014 National Statistics publication covering data to the end of September 2014 used the last of the quarterly samples (extracted from June-September 2014). Due to the operational move to six monthly financial recoveries from October 2014, validation rates will only be available every six months, covering the periods October-March and April-September.

In order to continue with a quarterly release schedule, the National Statistics will be adjusted using the latest available adjustment factor. This is in line with operational and management information procedures. March and September releases from 2015 used the previous adjustment factors derived from the most recent validation exercises at that time. So for example statistics published in March 2015 use validation rates from samples extracted to September 2014. The statistics published in June 2015 covering to the end of March 2015 use samples extracted from October 2014 - March 2015 and reflect the final payments made to providers. As noted in section 3.1.1, the full historic statistical series is refreshed each time the figures are released. The end to end post-payment validation process takes approximately 8 and a half months to complete. The routine sampling, checks and production of validation rates takes just over 1 month. These are performed monthly, one for each of the previous 6 months Job Outcomes payments.

The Job Outcome adjustment factors, for the relevant 6 months are sent to the National Statistics production team in the form of a dataset just over 3 weeks following the end of the period to be adjusted.

The first post payment validation exercise did not commence until June/July 2012 and was applied to all outcomes from the start of the Work Programme to May 2012. The associated adjustment factors used to rate the statistics relevant to this period were received by production team in August 2012. All Job Outcomes relating to this period were recorded clerically and uploaded to the Work Programme payment system in two stages during April and early May 2012.

13

If validation rates are amended at a later date, following the identification of an administrative error, or as a result of Provider Challenge the revisions will be applied to the Job Outcome National Statistics the next time they are released.

2.3.3 Sustainment payment validation

No sustainments can be paid until there is a valid Job Outcome payment to support the sustainment.

2.3.3.1 Pre-Payment validation

All Sustainment outcome payments are subject to a pre-payment automated off-benefit check. Claims are not paid where a legitimate claim to benefit has been identified in the relevant 4 week period.

2.3.3.2 Post-Payment validation

Until October 2014 the post-payment process of extrapolation was applied only to Job Outcomes, not to Sustainment payments. Instead, invalid Sustainment payments were removed from the PRaP system and therefore not included in the administrative data used for National Statistics. From October 2014 the process for Sustainment payments changed to the same regime as for Job Outcomes. The first National Statistics release which reflects the change is June 2015.

2.4 Unclaimed job outcomes

Unclaimed Outcomes are paid to reflect cases where a provider is unable to formally claim for an outcome achieved i.e. the participant refuses to share employment details or the providers loses contact after assisting the individual into employment. In September 2015, the first round of identification covering the first 15 referral cohorts (Jun11-Aug12) was completed and the volumes of Unclaimed Outcomes are reported in the December 2015 release for the first time. DWP analysis, which was reported in the March 2014 Work Programme Official Statistics publication, looked at participants’ off benefit periods covered by HMRC Real Time Information (RTI) employment evidence. The routine identification of these instances has been operationalized and providers have begun receiving respective outcome payments at a reduced rate. Unclaimed job outcomes are identified by the Department using HMRC Real Time employment Information and DWP benefit (National Benefits Database) data. Instances where time in employment and off-benefit meriting a Job Outcome

14

payment which has not been successfully claimed by the provider 32 months after referral date are identified. These are combined with a prescribed number of Sustainment Outcomes expected to have been achieved, where the provider was able to claim for the Job Outcome and was paid at a reduced rate as an Unclaimed Outcome. This recognises provider contributions whilst not overpaying for an outcome which does not meet the full requirements. It also retains the incentive for providers to maintain their current in-work support process in a bid to obtain the outcome at full-price.

2.5 Completing the Work Programme

People can stay on the Work Programme for a maximum of two years. From April 2014 Jobseeker’s Allowance (JSA) claimants who have not secured lasting employment after two years on the Work Programme are required to participate in the Help to Work programme. For further details see https://www.gov.uk/government/statistics/help-to-work-quarterly-statistics.

15

3 Performance measures of the Work Programme

Since providers are paid when individuals reach three or six months in work, an effective way to monitor the Work Programme is to track the progress of those joining in a particular month. This allows an assessment of progress by comparing monthly cohorts over time with the same duration of support.

3.1 Contractual measure

The Department calculates expectations of what providers should deliver. A summary is below. Work Programme Provider Guidance: Chapter 12 provides more detail.

All expectations are based on the analysis of historical data. Minimum expected levels for providers are set. The levels are set at ten percent above those that would be expected to occur in the absence of the Work Programme. For certain Payment Groups (1, 2 and 6a), these expectations are contractual minimum performance levels. This means that if providers do not achieve them, they may be subject to a formal performance improvement process.

The manner in which expected performance is measured has recently undergone a change. Under the new contracts from March 2015, minimum performance levels are now calculated for rolling measurement periods. The levels are calculated by profiling the number and makeup of persons actually joining in a particular month over the lifetime on the Work Programme. Work Programme Provider Guidance: Chapter 12 provides more detail on these profiles. At the end of each quarter the number of Job Outcome payments achieved in the prior twelve months are measured against minimum performance levels for each of the contract and payment groups 1, 2 and 6a.

The contracts run by Newcastle College Group in the North East Yorkshire and the Humber are not currently assessed by the minimum performance levels. NCG’s contract was terminated and Maximus has started to take their referrals. Due to this neither contract are included in the Minimum Performance Level statistics in the December 2015 release.

The new rolling 12 month measure replaced the former approach where, until March 2015, minimum performance levels were calculated for a fixed time interval, namely the financial year. Performance was assessed on the ratio of the number of claimants who did reach three/six months work compared to the expected number of claimants joining the Work Programme. At the end of each financial year, the ratio of Job Outcome payments paid in that year to Referrals made in that year (for contract and payment groups 1, 2 and 6a respectively) were compared against the minimum performance levels for these payment groups.

This method had limitations; notably that some of those joining would not have had long enough to reach three/six months in work and also the impact of changes

16

between expected and actual numbers joining. See also the review of Work Programme minimum performance levels.

3.1.1 Transparency measure

The indicator looks at performance by cohort once claimants have had 12 months on the Work Programme - time to receive a reasonable duration of support. This approach allows comparison of performance between cohorts. The indicator for June 2011, would be calculated by dividing the number of those who by the end of June 2012 had at least three/six months in work by the total number joining the Work Programme in June 2011. A higher indicator for later cohorts would demonstrate that the proportion of paid job outcomes for later cohorts had risen.

The indicator is published on the same release schedule and covers the same time period as the National Statistics. For example the statistics released in June 2014 covered to the end of March 2014. The latest indicator published at that time examined the proportion of the March 2013 cohort with at least three/six months in work by the end of March 2014. To reflect any updates to the figures the full historic statistical series is refreshed each time the figures are released.

The indicator is published as part of the National Statistics release via the Work Programme National Statistics homepage. More information on what cohort information is available via the Stat-Xplore (section 4.2.2) and the Open Data text (section 4.2.3).

Minimum expected levels are published alongside the indicator. These are based on the profiles for those joining in a particular month described in section 3.1. Work Programme Provider Guidance: Chapter 12 provides more detail on these profiles.

17

4 The Statistics The UK Statistics Authority has designated Work Programme statistics as National Statistics, in accordance with the Statistics and Registration Service Act 2007 and signifying compliance with the Code of Practice for Official Statistics. Designation can be broadly interpreted to mean that the statistics:

meet identified user needs; are well explained and readily accessible; are produced according to sound methods; and are managed impartially and objectively in the public interest.

Once statistics have been designated as National Statistics it is a statutory requirement that the Code of Practice shall continue to be observed.

4.1 What is in the Work Programme National Statistics?

The National Statistics show numbers joining the Work Programme (Referrals and Attachments), numbers achieving defined periods in work (measured via Job Outcomes and Sustainment Payments made to providers) and numbers completing the Programme. Statistics are also published on the main performance measures and the benefit status of participants.

Breakdowns are available by age, gender, ethnicity, disability flag, lone parent status, Payment Group, region, Local Authority, Parliamentary Constituency, primary health condition, contract and contract package area. Minor changes in Local Authority boundaries occurred before the September 2015 release. Numbers can be viewed on a monthly or cumulative basis or as monthly intakes tracked over time.

The Code of Practice for Official Statistics sets out the principles through which we protect the confidentiality of individuals within our statistics. We ensure confidentiality through the use of disclosure control, where random error is introduced into the statistics to ensure no data are released which could risk the identification of individuals in the statistics. From the June 2016 release onwards Work Programme Statistics have had a new disclosure control methodology applied to them to maintain consistency with Stat-Xplore. Due to this, some breakdowns may show very small differences between Stat-Xplore and published ODS tables. More details of the approach which has been copied from that used in Stat-Xplore can be found in the link below: https://sw.stat-xplore.dwp.gov.uk/webapi/metadata/Data%20Confidentiality.html?dbid=UC_Claims Statistics on payments reflect the final payments made to providers following payment validation procedures. More details can be found in section 2.3 and Work Programme Provider Guidance: Chapter 9.

18

Note that statistics describing the benefit circumstances of Work Programme participants were previously published via ad-hoc analyses. The methodology has now been aligned with other Work Programme cohort statistics. Participants are placed into monthly cohorts of Referrals and their benefit circumstances are tracked at the end of each of the subsequent months. This differs from the previous ad hoc analysis where the circumstances were tracked over subsequent weeks from their Referral or Attachment. As the cohorts statistics are created using data from the National Benefit Database (NBD), it will not include Universal Credit claimants as well.

4.2 Where can I get the Statistics?

The main mechanisms for viewing the National Statistics are the Statistical Summary and Stat-Xplore available via the Work Programme National Statistics homepage. The statistics can also be accessed via the Employment Programme Support Official Statistics webpage which provides access to other employment programme Official Statistics, the latest release timetable and other related information.

4.2.1 Statistical Summary

These documents present the main findings from the Work Programme National Statistics. A Statistical Summary is presented in a format developed to support understanding and interpretation. DWP welcome feedback on the format and also the users and uses of the statistics. A feedback questionnaire is available.

4.2.2 Stat-Xplore

Stat-Xplore is a statistical dissemination tool that allows users to create customised Work Programme tables on the numbers of Referrals, Attachments, Job Outcomes and Sustainment payments on a cumulative or monthly basis (cohort statistics are available via the Open Data, described in section 4.2.3):

The breakdowns available online are:

Month Geography Age Contract Work Programme Status Disability Indicator Ethnicity Gender Medical Condition Jobcentre Plus Discrict Length of Job Outcome

19

Lone Parent Indicator Mandatory or Voluntary Indicator Payment Group

These variables can be broken down even further; an example being Geography, which can be broken down into countries, regions, counties, parliamentary constituencies and local authorities. Some of these breakdowns are not available for Sustainment Payments.

4.2.3 Cohort and Time series data

The cohort and time series data for the Work Programme Statistics is also published, and is available in CSV format.

Work Programme cohort data includes breakdowns of people who have been referred to the programme by their month of referral:

Referral counts for each cohort For each month following referral for each cohort

o Count and percentage to have achieved a Job Outcome o Count and percentage to have spent some time off benefit (excluding

Universal Credit) o Count and percentage to have spent 13 weeks off benefit (excluding

Universal Credit) o Count and percentage to have spent 26 weeks off benefit (excluding

Universal Credit) o Count and percentage to be off benefit (excluding Universal Credit) at the end

of that month

The Time series data looks at referrals, attachments, job outcomes and sustainment payments over time.

4.3 Publication dates

Since June 2013, the Work Programme National Statistics have been published quarterly in March, June, September and December. The September 2017 statistical summary was the last publication to be released on a quarterly basis. From then on, the publication will be released every six months.

Publication dates are announced via the Employment Programme Support landing page, the Work Programme landing page and the UK Government Statistical Announcements page.

The statistics are published a little less than 3 months after the reference date. This is to allow sufficient time for the relevant data from the administrative systems to be recorded and extracted, and data from validation procedures to be compiled. Data is

20

then cleansed, combined and quality assured to create the Work Programme analytical dataset.

Whilst the statistics represent evidence of movement off benefit, they do not provide direct evidence of sustained employment as the analysis makes no assumptions towards the destinations of participants leaving benefit.

DWP published a report detailing findings from a study conducted to explore the destinations of a cohort of individuals who ended a claim for Jobseekers Allowance (JSA), Income Support (IS) and Employment and Support Allowance (ESA). This study aimed to provide the best possible estimate of the immediate and substantive destinations of leavers from JSA, IS and ESA benefit groups. The study explores:

movement into paid employment and the sustainability of this employment for each benefit group,

movement onto and between different out-of-work benefits,

other reasons for ending a benefit claim.

21

5 Who uses Work Programme statistics? 5.1 User Engagement

Prior to the first publication of Work Programme statistics, a note was placed on the Work Programme National Statistics homepage and internal stakeholders were consulted to ask for views on the types of statistics they would like to see published.

To make sure interested users can tell us what they think about our statistical products and to provide a forum to discuss statistical issues, throughout the continuing development, we have made our plans known to users in advance via the statistics web pages and invited user views via email (stats-consultation@dwp.gsi.gov.uk), Work Programme statistics questionnaire and the ‘Welfare and benefits’ community at www.statsusernet.org.uk. A public consultation on the content and presentation of Work Programme National Statistics after June 2013 was also conducted.

Internal user engagement has been important too. We continue to work closely with DWP colleagues on matters of methodology, classifications, definitions, coverage, timing and so on. These detailed issues can be very important, an apparently small change in a definition underpinning a statistic may be important in the context of certain uses.

Additionally, views are gathered via Departmental links to wider groups such as the (independently chaired) Work Programme: Building Best Practice group which includes prime contractors, subcontractors, voluntary and community sector organisations, the Employment Related Services Association (ERSA), the Association of the Chief Executives of Voluntary Organisations (ACEVO), the Confederation of British Industry (CBI), the Federation of Small Businesses (FSB), the Skills Funding Agency, Business in the Community and the Business Disability Forum.

DWP statisticians engage routinely with those responsible for answering Parliamentary Questions, Freedom of Information requests and request for wider briefing as well as press office colleagues to explain the statistics and guide their use and interpretation. We also monitor the media releases once the statistics have been released.

5.2 Known users and uses

Through the engagement detailed above, we have identified users

Within DWP

Work Programme policy, strategy, management information, operational support and associated analytical teams

DWP Private Office and Press Office

22

Parliamentary Question, Freedom of Information and Briefing teams

Who use the Work Programme statistical summary, Stat-Xplore, Open Data and underlying datasets to:

evaluate, develop and support policy, strategy and operational decisions, initiatives/options and business plans

answer Parliamentary Questions and Freedom of Information requests inform Departmental Responses to Commons Select Committees inform press office statements provide briefing lines including for devolved administrations

External Users

Outside DWP, the internet based publication mechanism means that understanding all the varied uses of the statistics and data is challenging.

Existing engagement with external user groups and organisations has been detailed above and we continue to seek external users to provide feedback on how and why they use our statistics through the mechanisms described.

Since the first release of Work Programme statistics in February 2012 DWP have heard from, or are aware of, a variety of users from:

Parliament: House of Commons Library, Parliamentary Committees Other government: Cabinet Office, HM Treasury, Scottish and Welsh

Government, local authorities/groups of local authorities Employment Services organisations: Prime providers, subcontractors, trade

bodies Charities and not-for-profit organisations Academics External commentators: leading news media and blogs United Kingdom Statistics Authority National Audit Office

As a result we know that the statistical products and data are used to:

provide general information on the Work Programme in Great Britain measure Government targets relating to DWP develop and evaluate policy within local authorities and other welfare to work

stakeholders and providers provide briefing inform academic research feed into analysis papers evidence National Audit Office and Parliamentary Committee reports

23

5.3 User experiences

The Statistical Summary outlines the valid uses of the statistics, i.e. to monitor the type and volume of individuals and outcomes achieved as defined in the payment model and contractual agreements.

Feedback via the engagement outlined above, has provided further information about how the statistics and data is ordinarily accessed and used and to what extent user needs are met .We have also gathered feedback on the information we provide on methodology, quality and via commentary in support of the statistics.

In summary, users

Welcome the large amount of statistics, breakdowns and underlying data that is released via Stat-Xplore

Find this information extremely beneficial in particular for local analysis Feel that the presentation of the statistics in the Statistical Summary is helpful

and user friendly Endorse the use of cohort statistics Previously, have requested more frequent and timely release of statistics – we

now publish quarterly with only a 3 month lag, compared to 6 monthly Are in general, satisfied with the level of Information provided on

methodology, quality and commentary to support the use of the statistics Feel that in general their needs are being met Identify some limitation with the tools

o the Tabulation Tool, in terms of the ability to download data to excel which has now been overcome with its replacement (Stat-Xplore)

o the Visualisation tool in terms of the structure of the underlying data Feel that we should add more context to the statistics in terms of rates of

benefit etc.

24

6 Development of the statistics

The March 2017 Work Programme publication includes changes from previous releases:

Delayed March 2016 Job Outcomes have been correctly assigned to March 2016.

There are approximately 200 cases with missing Contracts and CPA groups; this should be resolved in future releases.

There are two new JCP Districts; Greater Manchester and Cheshire.

In the September 2016 release, due to a technical issue a small number of Job Outcomes paid in June 2016 were not been recorded within performance data.

The December 2016 release shows the Maximus NE York Humber contract in the Minimum Performance Levels for the first time.

From December 2016 a new policy on rounding is applied to the Work Programme publication. The new policy can be seen in the table below:

Range of values Round to the nearest 0 to 1,000 10 1,001 to 10,000 100 10,001 to 100,000 1,000 100,001 to 1,000,000 10,000 1,000,001 to 10,000,000 100,000 10,000,001 to 100,000,000 1,000,000

From March 2017, data will include some demographic information on people that were claiming Universal Credit at the point of referral.

In February 2018, an error was detected in the number of job outcomes in September 2017’s release, covering the period July 2015 to March 2016. This was due to the wrong data file being incorporated into production of the September 2017 analytical dataset. This meant an over count of 0.2% for job outcomes at National level. The relevant tables have been amended.

25

6.1 Changes to Contracts and Providers

In March 2014 it was announced that the contract in North East Yorkshire and the Humber run by the Newcastle College Group would be terminated with 12 months’ notice. The replacement, following a procurement competition, Maximus Employment UK Ltd began receiving Referrals from 2nd February 2015. An existing provider G4S received all Referrals in North East Yorkshire and the Humber between 5th December 2014 to 30th January 2015.

With regard to the presentation of the statistics for the June 2015 release in North East Yorkshire and the Humber, the referrals already made to Newcastle College Group which had yet to have a Job Outcome were transferred to Maximus. Newcastle College Group has kept those who had already had a Job Outcome.

From December 2014, the Careers Development Group were known as Shaw Trust; their name is now Shaw Trust CDG. From February 2015, Interserve Working Futures acquired ESG Holding Ltd. These changes to the prime providers have been reflected in previous releases.

Changes to the prime providers which were not reflected in the June 2015 release but which are reflected in the September 2015 release are as follows:

Interserve Working Futures and ESG are both renamed Interserve Learning & Employment;

EOS-Works Ltd has been replaced by Avanta Enterprise Ltd; A4e has been acquired by Staffline Plc which also own Avanta Enterprise Ltd; A4e and Avanta have both been rebranded People Plus Inc; JHP Group Ltd are now renamed Learndirect Ltd; Newcastle College Group is abbreviated to NCG.

A further change occurs at the end of 2015: Pertemps becomes AMP. This change of name is not reflected in the December 2015 publication but is enacted in subsequent releases.

6.2 Historical validation factor improvement

Some of the people sampled have now had their details matched against HMRC’s Real Time Information data to check for periods in work, and this has slightly changed the validation factors which are applied. This impacts Job Outcomes from April 2014 to March 2015 and should result in a slight improvement in historic performance. The same process where RTI is used to complete the check was opted as the chosen method for Job Outcome and Sustainment Payments from April 2016 onwards. It replaced P45 data as it offered better coverage and more accurate identification of employment spells. The figures were retrospectively revised using this method.

26

The methodology of validation factors was reviewed in May 2017. Previously, all cases found to have a UC interest were fully validated at the pre-payment stage. It was discovered that UC cases were subtracted from the denominator of the validation factor. This meant that the procedure in which adjustment factors are applied to payments omitted Universal Credit claimant; the new approach adopted amends this issue and figures from April 2015 onwards have been corrected. This will have a small effect on outcomes within rounds 14 and 15 but will have a greater influence as the number of UC claimants with the programme increase.

6.3 Referrals

Note that some Referrals to Work Programme information sessions have been incorrectly entered onto the administrative computer system as Referrals to the Work Programme. By establishing the way in which these should be recorded, inaccurate Referrals have been identified and removed from the statistics; however any not recorded in this manner will not be accounted for.

6.4 Release dates

From September 2017, the Work Programme statistical summary will no longer be released quarterly but instead be released every six months.

27

7 Sources of the statistics Data is obtained from the Labour Market System (LMS). This is the system Jobcentre Plus uses to administer customer claims and refer customers to the Work Programme providers. The data contains information on the claimants’ individual characteristics and claim details.

Data on Attachments, Job Outcomes and Sustainment Payments are obtained from the Provider Referral and Payment system (PRaP). Providers use PRaP to attach customers and claim Job Outcomes and Sustainment payments. Data on payments are also used from payment validation procedures – further information on this can be found in section 2.3.

Statistics on those completing the Work Programme are derived from a combination of PRaP and the National Benefits Database (NBD).

Statistics on benefit status are derived by combining data from the Labour Market System (Jobcentre Plus administrative system) with the NBD.

The NBD (based on benefit administrative systems) is used to identify spells of Jobseekers Allowance (JSA), Employment and Support Allowance (ESA), Incapacity Benefits (IB), Income Support (IS) and Pension Credit (PC) by Work Programme participants.

The NBD is constructed by combining scans of live benefit claims from administrative systems. These scans are snapshot extracts taken at frequent intervals, for most systems this is fortnightly, consisting exclusively of claims live at the scan date. A claim that ends between extracts will not appear on the next one. Claim end dates are not included in the data extracts and are randomly allocated between the last scan in which the claim was live and the first scan in which the claim is no longer included. End dates are received for Jobseekers Allowance claims due to the nature of the administrative scans. Therefore end date estimation primarily affects non-JSA Work Programme Referrals. The volume and proportion of non JSA claimants varies from cohort to cohort. Whilst this may affect individual records, since estimated dates are random between the extract dates, there will be minimal impact on reported proportions off benefit at a point in time.

Data from these sources are combined and quality assured (see section 8) to create the Work Programme analytical dataset (WPAD). This dataset contains information relating to the published National Statistics, including dates of Referral, Attachment, Job Outcome payment and Sustainment payments. This enables derivation of the time taken for Referrals to Attach, plus comparison of information with previous datasets when the information is refreshed each quarter.

28

8 Limitations of the statistics Work Programme National statistics are sourced from data originally collected via systems which administer the Work Programme. As such, published figures reflect the information recorded both manually and automatically on these systems.

The statement of the administrative sources of DWP statistics describes the system assurances associated with DWP data sources.

Where manual or automatic processes are known to record information incorrectly, diligence is taken to ensure the published figures represent the Work Programme as accurately as possible. Quality assurance conducted both during development and routine production is covered in section 8. Information on known issues is recorded in this document.

Footnotes are applied where we are aware of anomalies and where possible data is corrected, although inevitably due to the nature of the data and systems, some inaccuracies will remain.

DWP have a policy for planned revisions describing how DWP will handle revisions and give confidence that all revisions will be handled in a transparent manner. For Work Programme National Statistics, to reflect any updates to the figures the full historic statistical series is refreshed each time the figures are released. We allow some time for additional information to be incorporated into DWP’s data systems. This time period is referred to as retrospection. Information may be submitted, corrected or resubmitted some time after the event occurred, for example payment information being received late or subsequently changed or removed. This means data presented in each release is subject to some retrospection and figures may be revised in subsequent releases. As well as revisions each quarter caused by general retrospection associated with data submission, further revisions result from retrospection associated with Work Programme processes, specifically the validation challenge process which can take approximately 3 months. Quality assurance procedures discussed in section 8.1 identify such changes between releases.

The biggest change between monthly figures occurs in the statistics on those completing the Programme (see Supplementary table 2.6 in the Statistical Summary). This is since benefit information is used to inform current Work Programme status, in particular statistics on those completing the allotted time and returning to Jobcentre Plus at 104 weeks. Due to the time taken to receive benefit information after the event, revisions will be higher (revisions will effectively reclassify those from other groups in the subsequent release) for statistics relating to the two most recent intakes to reach 104 weeks. The level of revision between the last two releases, for the last two intakes in both to reach 104 weeks, found a change of 15-20 percentage points with the level of revision for previous intakes less

29

than 3 percentage points. Therefore care must be taken when drawing conclusions involving the most recent two intakes to reach 104 weeks.

For benefit status statistics, due to the time scales of DWP being informed of changes in benefit circumstances, the last two months of benefit data is provisional. Analysis on previous months shows a 0.4% change in benefit status in those two months. There are minor differences between this publication and DWP’s operational MI caused by a small number of Gross Job outcomes being recorded in adjacent months, this leads to minor discrepancies in recorded Net Job outcomes in the two datasets

8.1 Measuring performance

Referral and Attachment statistics were first published in February 2012 and Job Outcome and Sustainment payments in November 2012. Outcome statistics were published later due to the time needed for outcomes to be achieved and for the Provider Referral and Payment system (PRaP) to be updated.

On 26th March 2012 the Provider Referral and Payment system (PRaP) was updated with functionality to record Work Programme Job Outcome payments electronically.

From April 2012, the functionality was used to allow all outcomes recorded clerically from September 2011 to be uploaded to the system. This took place in two stages during April and early May 2012.

The first stage related to outcomes to the end of March 2012 and stage two covered April outcomes. The new system functionality was opened to providers in mid-May 2012, from which point onwards all Job Outcome payments and Sustainment payments, including those from early May, were recorded directly on the Work Programme payment system.

A methodology was adopted for the National Statistics to allocate the clerical outcomes to specific months based on what would have been input had the system been live.

For stage one uploads relating to the end of March 2012, allocation was determined by the qualifying date of the outcome, stage two uploads relating to April were allocated to April and from mid-May onwards outcome statistics use the recorded payment date. When the new system functionality was opened to them in mid-May 2012, providers began to input outcomes relating to early-May to the system.

The information available to allocate early outcomes to a particular month coupled with failures, errors and delays during the upload and input process impacted on the

30

resultant time series to May 2012. For example no Sustainment payments are allocated to March 2012. The November 2012 release noted that a number of Job Outcomes were incorrectly assigned on the payment system to April 2012 rather than earlier months due to being allotted to stage two rather than stage one, in error, during the upload process. The issue was known to affect two contracts, Merseyside, Halton, Cumbria, Lancs (Ingeus UK Ltd.) and Devon, Cornwall, Dorset, Somerset (Prospect Services Ltd) with very few Job Outcome payments assigned pre April 2012. The cumulative total number of Job Outcome payments was not affected just the month to which they were attributed. It was also noted that other contracts may have been affected but to a lesser extent. DWP subsequently consulted all other providers and confirmed that no other contracts were affected. The originally incorrectly assigned payments have now been allocated to the correct months by amending the original methodology slightly for the affected contracts.

31

9 Quality Assurance Details of the initial quality assurance and routine quality assurance conducted ahead of each quarterly release are provided below.

9.1 Initial quality assurance conducted during data development

A large number of quality assurance checks were carried out during the early development of the statistics to ascertain:

reliability, completeness and level of disclosure of individual variables levels of duplicate, missing or contradictory information consistency across computer systems (LMS and PRaP) and with

management information, via cross checking monthly build up of figures in system data trends and variation in characteristic, time series and geographical

breakdowns a methodology to allocate early clerical outcomes to months consistent results during time series and cohort development via dual

methodology testing and manual checking consistent results during Stat-Xplore development through dual methodology

testing and manual checking trends and differences in post payment adjustment factors via secondary

quality assurance

This quality assurance found no issues; it showed the data was robust, consistent with management information and suitable for publishing. Individual variables were complete and consistent with existing sources, comparisons across systems were always within a 0.1% tolerance, including across key breakdowns and time series.

The above processes were subsequently used to develop data cleansing rules, assess retrospection and timeliness as well as form routine quality assurance procedures.

9.2 Routine Quality Assurance

A standard set of quality assurance procedures are conducted for each statistical release which consist of checking.

duplicate, missing or contradictory information accordance across computer systems (LMS and PRaP) and with

management information trends and variation in characteristic, time series and geographical

breakdowns trends and differences in post payment adjustment factors dual methodology testing of time series and cohort data

32

checks of Stat-Xplore variables and a sample of cross tabulations against perturbed Work Programme data and accompanying tables

Individual variables remain complete and consistent with existing sources, comparisons across systems remain predominantly within a 0.1% tolerance, including across key breakdowns and time series.

In addition to the quality assurance of the statistics, assurance of the underlying administrative data is also carried out. The National Audit Office published a report in August 2013 on their review of the data systems associated with the Work Programme Business Plan Transparency indicator (and therefore all key Work Programme National statistics). Section 2.3 contains details of the payment validation processes carried out in relation to financial recoveries against payments to providers.

Work Programme has also been used as a case study by the UK Statistics Authority in relation to Quality Assurance and Audit Arrangements for Administrative Data.

33

Feedback If you have any feedback, please contact Connor Byrne, Statistical Services, on 0203 267 5006 or connor.byrne@dwp.gsi.gov.uk.

Useful links The Work Programme Statistics are published quarterly on https://www.gov.uk/government/collections/work-programme-statistics--2.

Stat Xplore can be access at https://stat-xplore.dwp.gov.uk/.

The Work Programme Cohort and Timeseries data can be found at https://www.gov.uk/government/statistics/work-programme-cohort-and-time-series-data