Data, Metadata and Quality Management

Framework

(Quality and Information Management Framework at the Vietnamese Ministry of

Planning and Investment)

Michael Colledge and Bryan Fitzpatrick, Consultants

European Technical Assistance Programme for Vietnam (ETV2)

2

Content of Presentation

1. Context – Vietnamese Ministry of Planning and Investment

2. Problems

3. Solutions

4. Quality Concepts and Management

5. Information, Data and Metadata Concepts and Mgt

6. Quality and Information Management Framework

7. Conclusions

3

1. Context: Vietnamese Ministry of Planning and Investment (VMPI)

• Core business functions

• Organisation and working environment

• Inputs and outputs

4

Core business functions as defined in regulations• Developing strategies and plans for national socio-

economic development− includes analyzing and forecasting economic

performance

• Developing mechanisms and policies for economic management

• Issuing decisions, instructions, and circulars on planning and investment

• Managing official development assistance− including developing strategies for attracting,

coordinating and reporting

5

Core business functions as defined in regulations (continued)

• Managing procurement and tendering

• Managing enterprise and business registration

• Developing plans for renewal and development of state enterprises

• Promotion of small and medium size businesses

• Appraising and monitoring individual investments

6

Core business functions

• Briefly summarised−planning, monitoring, analysis, forecasting,

decision making

7

Organisation and Environment

• 28 Departments and Centres− fragmented, overlapping responsibilities

• Rapid evolution− Transition for market economy −No such ministry in western developed

countries−Move to role as Ministry of Economy

• General Statistics Office recently added

8

Information and Data Inputs and Outputs

• Characteristics of inputs−diverse range − relatively low volume

• Characteristics of outputs−mostly information rather than data

9

2. Quality Problems and Issues

• Information and data acquired are poor or unknown quality and/or lacking in timeliness

• Procedures and facilities for acquiring, processing, analysing and sharing information and data are poor

• Information and data often acquired and maintained on paper

• Data sources not fully exploited – duplication of data collected by GSO

• There are differing versions of what are nominally the same data

10

Quality Problems and Issues (continued)

• Limited metadata accompanying information and data− data are not well understood

• Departments/centres receive funding directly from donors and undertake uncoordinated developments− No organisational unit with mandate or

sufficient resources to take coordination or leadership role.

• There is no definitive repository of information and data

• MPI website is poorly populated and data are error prone

11

Underlying Problems• Division of functions amongst departments has

resulted in semi-autonomous fiefdoms with little motivation to collaborate

• No organizational unit responsible for developing, facilitating and monitoring general quality management policies, guidelines and procedures for the MPI as a whole

• No organizational unit responsible for developing, facilitating and monitoring general information, data and metadata management policies, guidelines and procedures

12

3. Solutions

• Introduction of the Quality and Information Management Framework

• Development principles−Simplicity−Use of Standards, Guidelines and

Recommended Practices−Harmonisation and Integration

13

Harmonisation and Integration

14

Transformation Process

Inputs Outputs

Figure 1:

15

Transformation ProcessInputs OutputsAcquisition

ProcessDistribution

Process

Repository

Figure 2

16

Figure 3

17

Transformation Processes

Inpu Acquisition

ProcessDistribution

Process

Repository

Figure 4

18

Transformation Processes

Inpu

Acquisition Processes

Distribution Processes

Repository 1

Figure 5

Repository 2

19

4. Quality Concepts and Management

• Evolution of quality concepts− Inspection−Quality control −Quality assurance− Total quality management

• International quality standards− ISO 9000 series−European Foundation for Quality

Management (EFQM) Excellence Model

20

ISO 9000 Series

• ISO 9000: 2005 Quality Management Systems - Fundamentals and Vocabulary− To provide concepts and terminology

• ISO 9001:2000 Quality Management Systems - Requirements − Basis for certification

• ISO 9004:2000 Quality Management Systems – Guidelines− for performance improvements in mature

system

21

ISO 9000 Quality Concepts

• Customer focus

• Leadership

• Involvement of people

• Process approach

• System approach to management

• Continuous improvement

• Fact based decision making

• Mutually beneficial supplier relationships

22

ISO 9001: 2000 Quality Management Systems

• Basis for certification

• Comprises five parts: − Quality Management System− Management Responsibility− Resource Management− Product Realization− Measurement, Analysis and Improvement

• Does not provide implementation guidelines

• Needs to be interpreted/adapted to particular circumstances

23

Interpretation/Adaptation to VMPI Context

• ISO Standards primarily designed in context of enterprises selling to market

• Need to be interpreted/adapted to particular situation

• Government organisation− not profit based − “user” rather than “customer”− mostly internal users− user cannot influence quality through purchase

decisions

• Whose products are information and data − with structural and reference metadata

• structural – needed to access product• reference – needed to understand its quality

24

Interpretation/Adaptation to VMPI Context (cont)

• Statistical offices are specialists in information/data

• Have developed several standards− Eurostat Quality Framework− OECD Quality Framework− IMF Data Quality Assessment Framework− Statistics Canada Quality Framework− Statistics Sweden…

• Frameworks differ only slightly

25

General Definition of Product Quality

• Starting point for quality management

• From customer/user perspective− “Totality of features and characteristics of a

product or service that bear on its ability to satisfy stated or implied needs” (ISO 8402 – 1986)

− “Degree to which set of inherent characteristics (of product) fulfill requirements (ISO 9000)

− “Fitness for Use”

• Need to specialise to government information agency − by identifying and defining characteristics of quality

26

Characteristics of Information/Data Product Quality

• Relevance− Degree to which products meet current and potential

user needs− Are required information/data available?− Are information/data available required?

• Accuracy/Reliability− Degree to which information/data correctly describe

event/phenomenon/situation they are designed to measure/represent

− Many aspects, no single overall measure

• Timeliness− Length of time between information/data availability and

events/ phenomena they describes− Measured relative to time period for which product is

likely to be useful

27

Characteristics of Information/Data Product Quality

• Accessibility− Extent to which users informed of information/data

availability− Suitability of format and medium by which product

accessed− Cost of access

• Interpretability− Ease with which user may understand and use

informatin/data− Definitions of concepts, coverage and data elements

• Coherence− Degree to which information/data products are

logically consistent and complete

28

Other Quality Considerations

• Quality dimensions are overlapping and interrelated

− cannot be combined into a single indicator

• Achieving acceptable level of quality is matter of trade-off

− for example accuracy against timeliness

• Cost must also be considered as constraint on quality

−quality cost trade-off

• Product quality achieved through process quality and performance management

29

Process Quality and Performance Management

• Two aspects to management of processes−Effectiveness – degree to which

processes generates products of high quality

−Efficiency – minimal use of resources in generating products

30

Figure 6. Process Quality and Performance Management

Quality and Performance Characteristics

Methodology Management

Information /Data

Management

Technology Management

Human Resource

Management

Effectiveness(Product Quality)

Relevance ** *

Accuracy ** *

Timeliness * * * *

Accessibility ** * *

Interpretability ** *

Coherence * * *

Efficiency(performance)

Resource Usage

** ** *

Key: ** primary responsibility; * shared responsibility

31

Quality Concepts - Conclusion

• In organisation whose primary inputs and outputs and information/data

information, data and metadata management is key to quality and performance management

32

5. Information, Data, Metadata Concepts and Management

33

Information and data management concepts and management

Terminology

• Data refers to numerically structured information− typically tables of numbers, graphs and charts

• Data has to be accompanied by metadata− describing to what the numbers refer and how

they have been produced

• Used in narrow sense information refers material that are not structured− reports, correspondence, pictures, images,

etc.

• Used in broad sense information includes data and metadata

34

Information Metadata Standards

• Dublin Core ISO 1588-2003

• Comprises set of descriptors for any information resource:− title, creator, subject−description, publisher, contributor−date, type, format, identifier− source, language, relation− coverage, and rights

• Others?

35

(Data) Metadata Standard

ISO/IEC 11179: Information Technology Metadata Registries

Standard aims to:

• provide a common understanding of data elements across organizational units and between organizations

• support re-use and integration of data over time, space, and applications

• harmonise and standardise data within an organization and across organizations

36

(Data) Metadata Standard

ISO/IEC 11179: Information Technology Metadata Registries (continued)

Comments:

• Part 3: Registry metamodel and basic attributes – generalised, complex difficult to understand

• Part 6: Registration – good starting point, will use simplified version

37

Data and Metadata Standards

ISO/TS 17369:2005 SDMX V1.0

• To support business practices by enabling efficient exchange and storage of data and metadata

• Provides format for structuring and reporting metadata, including methodology

• Provides an agreed structure fordata/metadata flows

SDMX V2.0 (not yet ISO standard)

• Being reviewed by WG2 of ISO TC 154

• Enhanced treatment of metadata

38

Figure 7: SDMX Top Level Model

39

SDMX Top Level ModelThis is a description of an data or metadata “flow” – an

abstracted data or metadata set that will potentially occur for many periods and from many providers (eg a regular table received by

MPI from various sources)

40

SDMX Top Level Model This is a description of an data or metadata “flow” – an

abstracted data or metadata set that will potentially occur for many periods and from many providers (eg a regular table received by

MPI from various sources)

This is an instance data or metadata set from a particular

provider at a particular time, eg, a particular table from Ninh Binh province, for a particular period

41

SDMX Top Level Model This is a description of an data or metadata “flow” – an

abstracted data or metadata set that will potentially occur for many periods and from many providers (eg a regular table received by

MPI from various sources)

This is an instance data or metadata set from a particular

provider at a particular time, eg, a particular table from Ninh Binh province, for a particular period

Provision Agreements indicate what Providers will provide what subset, when,

how often, and how

42

SDMX Top Level Model This is a description of an data or metadata “flow” – an

abstracted data or metadata set that will potentially occur for many periods and from many providers (eg a regular table received by

MPI from various sources)

This is an instance data or metadata set from a particular

provider at a particular time, eg, a particular table from Ninh Binh province, for a particular period

Provision Agreements indicate what Providers will provide what subset, when,

how often, and how

This identifies the Data Providers, giving indicative and contact information and linking to Provision Agreements and actual data and metadata sets

43

SDMX Top Level Model

This describes the structureof the data or metadata flow –

all the metadata needed to request and understand an instance of the flow (an actual data or metadata set). Links to all other structural

metadata.

This categorises all the defined data and metadata flows, providing

a structuring framework and a basis for searching. Links to

other structural metadata.

44

SDMX Top Level Model

45

SDMX Application at VMPI

Envisage SDMX Registry/Repository with:

• Code sets and classifications− environment for standardising and harmonising

• Data structure definitions− for all regular data sets, i.e., data flows

• Categorisation schemes− to index data flows

• Data storage environment for data sets− initially simple file store− possibly a database store− possibly a star-schema store

• with star schema, design generated automatically from structural metadata

• provides options for different “cuts” through data flows

46

6. Elements of Quality and Information Management Framework at VMPI

• A: More Strategic and Integrated Perspective of VMPI Role, Functions and Information:− A1: Review and Revision of Role and Functions− A2: Comprehensive Understanding of Inputs,

Processes and Outputs

• B: Fully Functional Quality and Information Management Programmes:− B1: Establish Quality Management System− B2: Establish Information, Data and Metadata

Management Programme

47

Elements of Quality and information management framework (continued)

• C: Improved Quality and Information Management Infrastructure:− C1: Develop Corporate Quality Management

Facilities− C2: Develop Corporate Metadata Management

Facilities− C3: Develop Corporate Facilities for

Acquisition, Capture, Storage, Access and Dissemination of Data

− C4: Develop Corporate Facilities for Acquisition, Storage, Access and Dissemination of Information

− C5: Enhance Corporate Planning, Monitoring, Analysis and Forecasting Facilities

48

Elements of Quality and information management framework (continued)

• D: Continuous Improvement and Reengineering of Business Processes:− D1: Continuous Improvement of Core Business

Processes− D2: Reengineering of Core Business Processes

• E: Comprehensive Quality and Information Management Training Programme:− E1: Develop and Conduct Quality Awareness and

Management Training− E2: Develop and Conduct Information, Data and

Metadata Management Training− E3. Develop and Conduct Training in Planning,

Monitoring, Analysis and Forecasting

49

Conclusions• In an organisation whose main inputs

and outputs are information and data− Quality management and information/data

management go hand in hand− Information/data management can be

viewed as component of quality management

− Quality management is good umbrella for achieving information/data management

• Need for recipe describing:− (parts of) international standards that are

readily applicable and useful− additional best practices that should be

considered