ESCO (2015) SEC 037 FINAL

Creation date: 05/03/2015

Last update: 17/04/2015

Planning the Delivery

of ESCO v1

Purpose of this document

This document describes the project management aspects of the development and the delivery of

ESCO v1. It is written to support the Maintenance Committee (MAI) to understand the process of

the content development for the remaining 16 sectors (see the Sector Table in Annex A), and the

integration of the content of all sectors. The document describes as well how a dashboard will

soon provide transparency on progress of the work done on the content. (Detail on many of the

processes described in this document is provided in separate documents.)

Scope of this document

Included in this document is a description of the project management for the remaining

development of the ESCO v1 deliverable, including:

Content development for the remaining 16 sectors (see the sector table in annex A)

Content development across sectors

Tasks involved in integrating and aligning all sectors to ensure a consistent ESCO v1

deliverable for all sectors.

This document does not describe:

The development of the ESCO methodology

ESCO supporting processes like, for instance, the ESCO guideline management and ESCO

tool management (e.g. Collaborative Taxonomy Creator (CTC), and the online consulting

platform)

ESCO test processes (including fit-for-purpose testing and mapping pilot)

ESCO continuous improvement processes

ESCO communication processes

The publishing of demo versions of ESCO (ESCO V0.1 and V0.2)

Mapping of the final ESCO v1 to national classifications.

Final Deliverables

The deliverables of the ESCO v1 project are:

The ESCO v1 data model

The ESCO v1 classification containing occupations (OCC), knowledge, skills and

competences (KSC), as well as related concepts covering all sectors of economic activity

The ESCO v1 qualifications

Project Organization

The project organization relevant for the delivery of ESCO v1:

Figure 1. The ESCO Organisation.

Entity Responsibility for

European Commission Managing the overall project and the final deliverables

Secretariat (SEC) Project management, methodology development, quality

control, monitoring, and mediation for the online consultation

process

ESCO Board (BOA) Providing strategic guidance, and supporting the promotion of

developing, implementing and use of ESCO in the labour

market, and in the education and training sector.

ESCO Maintenance Committee

(MAI)

Providing guidance and support for the process to incorporate

the domain knowledge into the ESCO classification

Sector Reference Groups (SREFs) Responsible for domain knowledge and content development

for the first 11 sectors.

Online Community of Experts Providing domain knowledge, expert review and expert

validation for the remaining 16 sectors.

Cross Sector Reference Group

(CSREF)

Development of cross-sectoral knowledge, skill and

competences.

Software Development Partners Development of the software components relevant for

creating, maintaining and publishing the ESCO classification

Taxonomy Expert Group (TEG) Sector research, content development, monitoring and

mediation for the online consultation process

Content Development Partners Responsible for developing the traversal skills

Translators & Terminologists English language correction and the proper use terminology,

Translations

Milestones and High-Level Process

The ESCO project has two core deliverables relating to the ESCO classification itself, i.e. the ESCO

v1 data model, and the ESCO v1 classification. (At the next MAI meeting the SEC will present how

they will integrate qualifications in the classification.)

The work remaining for these core deliverables is:

The ESCO v1 data model: The work is complete. The data model is currently under life-

cycle management.

The ESCO v1 classification containing occupation, knowledge, skill, and competence

concepts relevant for the 27 defined sectors: The SREFs, TEG, and SEC have completed

most of the work for the first 11 sectors. Work for the remaining 16 sectors has started.

Additionally the work to integrate the sectors classifications into a balanced,

comprehensive ESCO classification will start soon.

The main process, i.e. the process that delivers the ESCO classification for the OCCs and KSCs for

the remaining 16 sectors is broken down into 3 phases:

1. Full content drafting (per sector)

2. Online consultation and refinement process

3. Sector finalisation process.

This process will deliver 16 additional subsets of the ESCO classification, one for each new sector.

In parallel to the work done on the content, a reference group is working on the creation of

transversal KSCs with a cross-sector relevancy for future versions of ESCO. (The TEG contextualises

the transversal KSCs already available while drafting of the full content.)

The SEC and the TEG submit the output of the sector finalisation process to the content

integration and clean-up process. This process will also spin-off the translation processes. The

integration and clean-up processes will, in steps, curate, enrich and improve the quality and the

richness of the ESCO classification. When the TEG and SEC conduct this process, the SEC will

regularly update the ESCO management bodies on the progress, the types of issues encountered,

and the improvements and enrichments being implemented.

When the European Commission decides to publish a formal release of the ESCO classification, the

SEC will prepare the release, and the MAI will be asked to formulate their final opinion on the

proposed release prior to a final decision by the European Commission to publish ESCO v1.

Technical Note:

With the formal publication of ESCO v1, the Commission will take responsibility for actively

managing the published URIs used for identifying the concepts of the classification. This means

that the semantics associated with a URI of a concept will stay intact between versions of ESCO. In

practice, this means for a user that a concept-mapping table from a national classification to ESCO

remains valid with new versions of ESCO.

Content development per sector (for the remaining 16 sectors)1

Figure 2. Content development per sector.

The SEC organises the content development for the remaining sectors in three major phases, i.e.:

1. Full content drafting, delivering the content in the Delta version (ready for online

consultation).

2. Online consultation and refinement, delivering the content in the Iota version (refined and

validated by the online community of experts)

3. Sector finalisation, delivering the content in the Nu version (Sector complete, ready for the

integration and clean-up process)

The individual process steps for these 3 phases are described in the following three sections.

1 Detail on the work associated with each individual step will be documented in the ESCO Guidelines v2.

Full content drafting

Figure 3. Full content drafting (per sector).

The ESCO sector content development work entails all the steps necessary to create a draft of the

content for a sector that is appropriate to be presented to the online community for online

consultation.

This first phase is broken down in:

1. A creation step in which the TEG delivers the sector breakdown with all relevant

occupations and occupation groups. The Alpha version is the intermediate deliverable

from this process step.

2. A validation step in which the SEC validates the Alpha version and helps to remediate

outstanding issues. Once validated, the content is set to be in Beta version.

3. A content extension step in which the TEG extends the OCCs captured in the sector

breakdown with all the essential and optional KSC and associated concepts. In this step,

the TEG also ensures that all KSC have all necessary properties defined. This step delivers

the Gamma version.

In parallel to work being done by the TEG, the MAI will be invited to provide comments in

CTC on the sectoral breakdown and list of occupations.

4. A validation step in which the SEC verifies if the Gamma version meets all the appropriate

quality criteria specified. Once validated, we declare the content for the sector to be in

the Delta version.

Online consultation and refinement2

Figure 4. The online consultation process and refinement process (per Sector).

The online consultation and refinement process defines the following five separate process steps:

5. The first online consultation. This is the process in which the online experts evaluate the

sector breakdown and the occupation profiles. The experts can provide suggestions and

comments and can engage in discussions with fellow experts as well as with TEG/SEC. The

TEG/SEC will monitor and mediate the process. In areas where insufficient activity takes

place, the SEC will take action to resolve the issue to ensure proper evaluation of the

content. In parallel to the work done by the Online Experts, the TEG and the SEC, the MAI

can monitor the activity on the platform. In case they observe issues, they can either

discuss that on the provided forum on CTC, or notify the SEC.

The annotated ESCO content that is the result of the first round of online consultation is

referred to as the Epsilon version.

6. On the basis of the first round of online consultation, the SEC will decide which sectors or

subsectors need a second round of online consultation. The SEC will only submit content

to the second round in case content was not properly discussed.

7. In the case, that the SEC decides on a second round of online consultation the TEG will

refine and create the Zeta version of the content on the basis of the comments of the first

online consultation. This process can be monitored by the MAI in CTC.

2 For details on this process see document ESCO (2015) SEC 020.

8. The online community of experts will review the Zeta content in the second round of

online consultation. In parallel to the work done by the Online Experts, the TEG and the

SEC, the MAI can monitor the activity on the platform. This process step will deliver the

Eta version.

9. The TEG refines the content that comes out of the online consultation process. During the

refinement process, the content is updated using the comments and suggestions from the

online consultation process. This delivers the Theta version. This process is monitored by

the SEC as well as by the MAI in CTC.

10. The online community will validate the Theta content and by doing so create the Iota

version. The TEG and SEC monitor this process. In parallel, the MAI can monitor this

process on the online consultation platform as well.

Finalisation of each sector

The Iota versions that are the result of the online consultation and refinement process enter the

process to finalise the content for a sector. During this phase persistent quality issues that are

identified during the last steps of the online consultation process or that surface during the quality

assessment are to be resolved in parallel to sharing the current state of the content for a sector

with the MAI for evaluation.

11. As a first step, the SEC will execute a quality assessment on the actual state of the Iota

content. The SEC will document possible issues and create a Kappa version.

Figure 5. The sector finalisation process.

12. The SEC will present the Kappa content for a sector, the Key Performance Indicators

(KPIs)3 and the associated report that describes the list of issues to the MAI for an opinion.

Possible feedback on the content will be captured to support the SEC and the TEG to

refine the content.

13. In parallel to presenting the Kappa content to the MAI, the TEG will start to address the

pending issues. The additional feedback from the MAI is added to the work as soon as it is

available. These activities will result in the Lambda version of the content.

14. To prepare for the alignment and rationalisation of the KSC concepts in a later stage, the

SEC needs to normalise the concepts. The knowledge structure requires the concepts to

be described as atomic concepts or inseparable concepts (e.g. “serve food and drinks” will

be reformulated as two concepts, i.e. “serve food” and “serve drinks”). Cases for which

normalization is not straightforward will be labeled to be “complex”. The SEC will resolve

these cases at a later stage.

15. The SEC does a final validation check on the Mu version and declares the content work for

the sector to be final with the Nu version. (Please note: The content is final when

evaluated within the scope of a single sector, re-evaluation in the scope of multiple sectors

3 Key Performance Indicators (KPIs): a list of measurable parameters are defined for the content to support Quality

assessment. E.g. average number of skills in a job profile, number of concepts that do not have a description (and links

to these concepts), balance between essential and optional skills.

will mean that the SEC needs to review and edit the content to achieve a balanced fit in

the whole ESCO classification.)

ESCO cross-sector development

The ESCO cross-sector development work for ESCO v1 consists of two major phases, i.e. phase 1 in

which the transversal KSC concepts (e.g. "measure") are identified and described, and phase 2 in

which the transversal KSC concepts are contextualized in context of the defined occupation

profiles.

For ESCO v1, phase 1 of the cross-sector development has been completed. The TEG executes

phase 2 in the context of the content development process.

In parallel, further work on the transversal KSCs will be done in preparation for a major future

ESCO releases (starting with ESCO v2).

ESCO integration and clean-up process4

The ESCO integration and clean-up process is a process that can be executed in an iterative

manner and can start as soon as the first sectors are available in their “Nu” version. The SEC will

test the process with the first three sectors that have been signed-off with their KSCs by the MAI

(AGRI, HOSP and VETE). The SEC uses the outcome of the test to determine how best to

implement it for the remaining sectors.

In order to structure the integration and clean-up process, we group the individual process steps

of the around three themes (see also figure 6), i.e.

1. clean-up of the integrated occupations pillar

2. clean-up of the integrated skills and competences pillar

3. adding the translations

Process step for the clean-up of the integrated occupations pillar:

16. This is the step in which the occupation concepts that have relevancy across sectors are

identified, aligned and rationalised.

4 For details on this process see document ESCO (2015) SEC 026 Final.

Figure 6. The ESCO integration and clean-up process.

Process Steps for the clean-up of the integrated skill and competences pillar:

17. The SEC and the TEG make the vast amount of knowledge, skills and competence concepts

manageable by grouping concepts around common themes. The SEC will decide which

themes to use on the basis of the actual knowledge in the data.

18. The SEC and the TEG will use the common themes to enrich the knowledge structure with

associations (“same as”, “broader than”, “narrower than”, or “related”)

19. During this step, the effect of merging the KSC concepts, which are declared to be the

same by the “same as” associations, will be evaluated. The SEC will merge the identified

concepts in case no undesirable side effects occur.

20. The SEC will investigate and resolve the cases that were labelled to be complex during

previous steps.

21. Correction of the English terminology used in the content. This step can happen as soon as

the collection of concepts is stable, i.e. once the occupational overlaps are addressed, and

the KSC concepts are split. Since there are no dependencies of the knowledge structure

itself, the implementation can occur in a parallel process. However, in order to have the

corrected terms available in the master content, the SEC will organise that the corrected

terms are copied into the core content as soon as they are available.

22. Once the corrected English terms are available there exists a stable basis for all the

translations. These translations can happen per language and independently of the main

process. The SEC will organize that the translations are integrated in the main content set

as soon as they are available.

Project Planning, Quality Control and Monitoring

Project Planning

Based on the process defined and the constraint to have to deliver the ESCO v1 classification in

December 2016, we have set the target dates for the milestone versions Delta, Iota, Pi, Sigma and

Omega. The SEC will use the target dates as a reference for testing if the process steps of the

newly defined integration and clean-up process can be executed in time. The SEC will also use the

target dates as a basis to negotiate the target dates for the other versions.

Target dates for the Key Milestone Versions:

Milestone

version

Description Target date

Delta Sector draft full content ready for online

consultation

September 2015

Iota Refined sector full content, validated by the online

community, and assessed by the SEC

February 2016

Nu Sector content finalised (Evaluated by the MAI,

with issues resolved and KSC concepts normalised)

March 2016

Pi Cleaned occupations, KSC collections created, KSC

concepts linked, English corrected, first set of

translations integrated

August 2016*

Sigma KSC concepts merged, KSC complex cases

addressed, translations integrated

November 2016*

Omega ESCO V1 created, approved and published December 2016**

* Preliminary target dates; the team has no experience with the underlying process; Tests with currently

available content will be conducted to validate assumptions made.

** December 2016: the deadline set by the European Commission.

Quality Control and Monitoring of the ESCO Content Development

The SEC has created the procedures for quality control and monitoring for the drafting for the

sector content development for the remaining 16 sectors using the experience with developing

the content for the first 11 sectors. The SEC has captured the experiences in the guidelines as well

as in the elaborate list of KPIs. The KPIs will soon be available on a dashboard for the different

defined versions of the content sets. This dashboard updates the values of the KPIs daily and will

provide through the KPIs transparent feedback on various aspects of the content. The SEC and the

TEG will use the KPIs as a first indicator for a wide range of issues. Issues like e.g. missing preferred

terms (PT), missing references to ISCO codes, and unbalanced distribution of essential and

optional skills will surface immediately. As a consequence of using the dashboard, the SEC will

save time on checking the statistical aspects of the classification of a sector, and thus have more

time to focus on the semantics and the classification itself. For instance, based on experience, the

SEC knows that it has to check consistency between the description of an OCC and the actual KSCs

associated.

The SEC will also use the same information to update the MAI at their meetings. The dashboard

with the KPIs and notes will provide a basic structure to present and discuss progress and

challenges.

Content development dashboard

The SEC has defined the milestones for the sector content development process around the

maturity of the content. Each version reflects a certain level of maturity of the content of a sector

or for the full ESCO content set. The SEC will describe the defined versions in detail in the updated

ESCO guidelines. The dashboard captures the associated list of KPIs and provides access to a

document in which issues can be captured. The dashboard has also incorporated the target dates

for the various versions. Based on the target dates, the dashboard shows if a milestone is met in

time or not.

The SEC will configure the dashboard around the versions defined for different slices of the

content. The definition for a slice, during the phase in which the sector content is developed, is

directly based on the NACE codes associated with the sectors. Using this definition the sector

content development work will be presented in the dashboard using the sectors as rows and the

versions as columns. In a similar fashion, we will represent the milestones for the integration and

clean-up process. However, during this phase, we will use a different definitions for the slices

represented in the rows. The rows will represent the OCC, the KSC and all the languages for the

group of sectors being integrated. This ensures that the SEC can monitor progress aligned with the

different streams of activity.

Through the dashboard, the team has at all times access to all the key information on the issues

and the progress being made on the content. Progress is reflected by showing in each cell the

target date for a version in a heat map5. (A mock-up of the dashboard is provided in figure 7)

This dashboard will be the leading instrument to guide the TEG and the SEC through the process. It

supports through the KPIs and the notes associated with versions a solid means for collaboration

between the SEC, the TEG and other parties that have a role in the process. It will also support the

weekly SEC-TEG alignment meeting that helps the SEC and TEG to manage and align focus, quality,

issues and priority for the delivery of the ESCO content.

5 Colours will indicate if milestones have been met.

Figure 7. A mock-up of the dashboard currently being developed.

Annex A

Table with the list of ESCO Sectors