Data and Information Quality"Data quality remains a very overlooked issue in business intelligence, but a massive one. I continue to see failures due to a lack of attention to data quality. Data is the most fundamental component of any BI endeavor. It's the building blocks for insight. Companies have to get their data stores and data warehouses in good working order before they can begin extracting and acting on insights. If not, they'll be operating based on flawed information. “

Gartner

Some thoughts on measurement..

• Measurement needs to serve the deepest purposes of work. [...] we want to gather information that will help us be better contributors. But in too many organizations, just the reverse happens. The measures define what is meaningful rather than letting the greater meaning of the work define the measures. As the focus narrows, people disconnect from any larger purpose, and only do what is required of them. They become focused on meeting the petty requirements of measurement, and eventually, they die on the job. Margaret Wheatley[www.margaretwheatley.com ]

Information quality depends on ..

(i) Data quality(ii) How well it supports the work of different

individuals in the organisation to complete their tasks and help them make decisions

For example, poor customer information often leads to poor customer service.

Data• raw, unsummarized, unanalyzed facts.Elementary descriptions

of things, events, activities that are recorded, classified and stored, but not organised to convey any specific meaning.

Information• data processed into meaningful form. The recipient interprets

the meaning and draws conclusions or implications.

Knowledge• data or information that have been organized and processed to

convey understanding, experience, accumulated learning, expertise as they apply to a current problem or activity.

• the capacity to use and interpret information for use in decision making.

Quality Information characteristics

• Accuracy• Relevance• Completeness• Conciseness• Scope• Validity• Consistency

• Timeliness• Currency• Frequency• Right time

period

• coherence• clarity• detail• order• Presentation• Media• Accessibility• Security

Intrinsic

TimeForm/Accessibility

Data Qualitydetermines the usefulness of the data as well as the

quality of decisions based on them.• Contextual DQ – relevancy, value, timeliness

completeness, amount• Intrinsic DQ – accuracy, objectivity, believability,

reputation• Accessibility DQ – ease of access, security• Representation DQ – interpretability, ease of

understanding, concise, consistent representation.

• Context is very important as many metrics do not mean anything unless you know their context. Irrelevant information wastes time and may mislead. Data needs to be timely in many cases to ensure usefulness as things change fast.

• Intrinsic DQ is important as inaccurate data will lead to inaccurate and distorted results and inaccuracies can be magnified through a series of models.

• Data must be easy to access by those who need it but kept secure from intruders.

• Data needs to be represented in a usable manner. If data isn’t consistent it is difficult to integrate and use. e.g. difficulties/expense in setting up ERP systems and data warehouses due to inconsistent data.

Question: Pretend you are Mary HarneyHow can Sligo General achieve the NHS A&E

target: See 95% of patients within 4 hours and have either treated or referred them.

This problem requires actionHow would you gain the knowledge to solve this

problem and keep it solved?What information would you need?What would you measure and why?What data would you gather ?Who would input that data and how would you

ensure that it was accurate?

DIKARDATA

Quality

AcccurateCompleteValidConsistent

Information

Quality

Data qualityClarityRelevancePresentationTimelinessavailability

Knowledge

Quality

Information qualityArtefactsSkillsHeuristicsExperienceTalent

Action

Quality

AppropriateTimelyclear

Results

Quality

Consistent with objectivesEffectiveEfficient

RAKID – What data do we need to get these results?

DATA

Quality

AcccurateCompleteValidConsistent

Information

Quality

Data qualityClarityRelevancePresentationTimelinessavailability

Knowledge

Quality

Information qualityArtefactsSkillsHeuristicsExperienceTalent

Action

Quality

AppropriateTimelyclear

Results

Quality

Consistent with objectivesEffectiveEfficient

What are we trying to do? OBJECTIVES

• What do we need to know in order to do it?

• Who needs to know it?• What information do

they need?• What do we need to do

with that information?• Who needs to do it and

how?

• What do we actually know?

• Who knows it?• What information

do we have?• What are we

doing with it?• Who is doing it

and how?Information Audit Questions: Adapted from Orna(1999) in Chaffey and White(2011) fig 10.11 p4504

How can Data Quality be Ensured?

• Collect data automatically rather than relying on people to input it.

• Making sure that staff are well trained so that data is entered accurately

• Computer systems should validate data e.g. put data validation checks in Excel

• develop plans that outline what will be done with data when collected

• Have practices for preventing redundant data • Have methods for organizing data in a way that

makes sense to the business. • Choose systems which are easy to reconfigure to

reflect present/ future changes.• Have well designed databases and data

warehouses which update reasonably quickly (size of the update window = time taken to update).

How can we help ensure information quality?• Analysing decision situations so that it is clear

what information is needed, when, by whom, and in what format.

• Being clear about the data sources for that information and ensuring that quality data is collected.

• Regularly carry out information audits so that you know you’re collecting the right data and that your measures are meaningful.

How do we ensure meaningful measures?• How can we keep measures useful and current? • What will indicate that they are now obsolete? • How will we keep abreast of changes in context

that warrant new measures?• Who will look for the unintended consequences

that accompany any process and feed that information back to us?

• Measures are meaningful and important when generated by those doing the work.

• Any group can benefit from others' experience and from experts, but the final measures need to be their creation.

• Those closest to the work know a great deal about what is significant to measure.

1. Look at who creates measures

2. What measures will inform us about critical capacities?

• How will we measure these essential behaviors without destroying them through the assessment process?

• Do these measures honor and support the relationships and meaning-rich environments that give rise to these behaviors?

teamwork quality innovation

Commitment Learning

Questions to ask of measures.

• Do measures invite in newness and surprise? Do they encourage people to look in new places, or to see with new eyes?

•Will this particular information help individuals, teams, and the entire organization grow in the right direction?

• Will this information help us to deepen and expand the meaning of our work?

• Information richness depends on the delivery medium

face->face -> numeric document Rich information is needed to resolve ambiguity and

multiple conflicting interpretations of information

• Information Overload occurs when there is so much information that it is difficult for a manager to determine which information is relevant and which is not.

Information Issues

Information hardness - accuracy and verifiability of a piece of information.

• The accuracy of a piece of information depends on the degree to which the information is free from error.

• If the piece of information can be proved to be accurate, then it is verifiable.

• Accuracy depends on quality data

• Often computer-generated information is taken as accurate, when it is not. GIGO

• Completeness –is the information is free from omissions?

• Relevance – is the information appropriate input for a particular decision.

• Timeliness –Is the information up to date enough to be useful?

• Accessibility/format – is the information available in an understandable format to the people who need it?

Measurement and feedback...All life thrives on feedback and dies without it. In any living system, feedback

differs from measurement in several significant ways:1. Feedback is self-generated. An individual or system notices whatever they

determine is important for them. They ignore everything else.2. Feedback depends on context. The critical information is being generated

right now. Failing to notice the "now," or staying stuck in past assumptions, is very dangerous.

3. Feedback changes. What an individual or system chooses to notice will change depending on the past, the present, and the future. Looking for information only within rigid categories leads to blindness, which is also dangerous.

4. New and surprising information can get in. The boundaries are permeable.5. Feedback is life-sustaining. It provides essential information about how to

maintain one's existence. It also indicates when adaptation and growth are necessary.

6. Feedback supports movement toward fitness. Through the constant exchange of feedback, the individual and its environment coevolve towards mutual sustainability.

What do these distinctions tell us when looking for measures to aid in making decisions for

action?Feedback• Context dependent• Self-determined; the system

choose what to notice• Information accepted from

anywhere• System creates own meaning• Newness, surprise are

essential• Focus on adaptability and

growth• Meaning evolves• System co-adapts

Measurement• One size fits all• Imposed. Criteria are

established externally.• Information in fixed categories

only• Meaning is pre-determined• Prediction, routine are valued• Focus on stability and control• Meaning remains static• System adapts to the

measures