An empirical investigation of metadata issues in Business Intelligence environment for Higher...
Transcript of An empirical investigation of metadata issues in Business Intelligence environment for Higher...
An empirical investigation of metadata issues in Business Intelligence
environment for Higher Education Institutions
Yuriy Verbitskiy
Principal supervisor: William YeohAssociate supervisor: Andy Koronios
Minor Thesis final presentation
2
Outline
• Introduction: main principles of BI, BI environment, motivation and research question
• Literature review: BI issues, requirements and similar research
• Action research design• Reasons for providing metadata in BI and
requirements for metadata solution• Metadata solution: architecture, Metadata Framework
and metadata prototype• Metadata prototype implementation• Conclusions
3
Introduction – main principles of BI
• Business Intelligence (BI) is on the top of priority list for CIO worldwide during the last 3 years (Gartner)
• BI - is a set of concepts, methods, and technologies
• BI has a number of issues, such as:• Understanding of BI environment• Understanding of data it delivers
• Making decisions based on the results of BI tools is the biggest challenge for users (Lawton 2006)
4
Introduction – BI environment
SQL
Excel
XMLXML
DB2
… Data Warehouse
ETLETL
Data marts
OLAP
Reports
Business applications
Sales amounts
Metadata repository
Business rules
Business rules
Dashboards
5
Introduction – motivation and research question
• Australian universities use BI technology for different tasks
• To apply BI technology successfully, Australian universities require a metadata solution.
• Issues in the metadata implementation:• No standard approach for developing the metadata• Complexity of the metadata implementation
• This research investigates how to improve the metadata implementation for the BI environment in Higher Education Institutions.
6
Literature review – BI issues
• There are “three enterprise standards that are prerequisites to delivering a consistent single version of the truth” (Beyer 2007), which are: terminology, calculation and methodologies.
• “People and organization” is the most significant obstacle for supporting BI (Burton, Popkin et al. 2007).
• Staff members, who work on different layers in BI environment, tend to speak a different language (Chisholm 2008)
7
Literature review – requirements
• Indirect usage (Inmon, O'Neil et al. 2008)• Centralized metadata repository (Paolo Missier, Pinar
Alper et al. 2007; Inmon, O'Neil et al. 2008)• Interoperability or (API) for access by other software
(Vaduva and Dittrich 2001; Ward 2007)• Interchangeable metadata format
(Shankaranarayanan and Even 2006)• Browse, search, filters, facets (Vaduva and Dittrich
2001; Ward 2007; Foulonneau and Riley 2008)
8
Literature review – similar research
• A comprehensive repository model for managing the data warehouse metadata (Stöhr, Müller et al. 1999)
• A software architecture for metadata management that was developed for the data warehouse environment (Auth, Maur et al. 2002)
• A multi-dimensional metadata framework for the enterprise business intelligence (Stephens 2004)
• Metadata version and configuration management is extensively discussed in Friedrich (2005)
9
Action research design
10
Reasons for providing metadata in BI
• To provide consistency for descriptions and definitions of the data in BI environment
• To provide an overall enterprise view• To solve the problem of misinterpretation of some
terms which have different meanings for staff with different roles
• To provide translation between technical and business terms
11
Requirements for metadata implementationArea Requirement Priority
Presentation of metadata
Layered presentation of metadata MEDIUM
Providing the names of contact person, email HIGH
Browsing, Searching, Facets, Key words, Filters HIGH
Metadata repository
Easy customization of metadata structure in the future HIGH
Hierarchic metadata classification HIGH
Metadata structure is shown in metadata model to help users HIGH
Refreshing of metadata from various sources on a regular basis HIGH
Import/Export functionality with Excel HIGH
Metadata infrastructure
Accessibility from multiple places, uniform access mechanism MEDIUM
Integration with existing BI environment, context-sensitivity HIGH
Interchangeable metadata format MEDIUM
API for access by other software applications MEDIUM
Metadata management
Easy to support and change HIGH
Metadata stewardship HIGH
Access control HIGH
Metadata change technique HIGH
Metadata version management strategy LOW
Notification mechanism LOW
Metadata quality HIGH
12
Metadata solution – architecture
ASC
BAC
CAS
# Name Date
1 Fi sh 01/03/08
2 Bread 05/04/08
3 M eat 21 /03/08
Metadata repository
SQL Server 2005
Metadata interface
ASP.NET 2.0
BI environment
Cognos 8.4
ADO.NETWeb Services
Metadata Framework
Other Metadata sources
ASP Page
Excel files
Cognos JavaScript
pages
13
Metadata solution – Metadata Framework and prototype
Area Requirement Priority Framework Prototype
Presentation of metadata
Layered presentation of metadata MEDIUM Not discussed Not implemented
Providing the names of contact person, email HIGH Discussed Implemented
Browsing, Searching, Facets, Key words, Filters HIGH
Metadata repository
Easy customization of metadata structure in the future HIGH
Hierarchic metadata classification HIGH
Metadata structure is shown in metadata model to help users HIGH
Refreshing of metadata from various sources on a regular basis HIGH
Import/Export functionality with Excel HIGH
Metadata infrastructure
Accessibility from multiple places, uniform access mechanism MEDIUM
Integration with existing BI environment, context-sensitivity HIGH Interchangeable metadata format MEDIUM
API for access by other software applications MEDIUM
Metadata management
Easy to support and change HIGH
Metadata stewardship HIGH
Access control HIGH
Metadata change technique HIGH
Metadata version management strategy LOW
Notification mechanism LOW
Metadata quality HIGH
14
Metadata prototype implementation
• During the practical implementation two issues appeared:• metadata implementation solution very much depends on
the functionalities of the BI environment• a need to understand how the process of metadata change
management work in practice
• The benefits of metadata prototype :• Integration with BI environment• Basis for the powerful metadata interface• Standard solution for the metadata repository that allows
flexible customisation of metadata structure, and• Relatively simple support and improvement of the whole
application
15
Conclusions
• Constant business orientation of the metadata solution from the early stages
• Metadata solution should be based on• comprehensive metadata model and • implementation approach, which defines the main steps of
the metadata implementation process.
• Major findings:• Business metadata represents a major part of the
metadata and it is crucial for the Business Intelligence environment
• Majority of the metadata requirements are feasible to implement
16
Conclusions – strategic issues
General issues
Defining the scope for metadata project
Defining the metadata model
Technical issues
Integration with BI environment
Powerful metadata interface
Hierarchic metadata classification
Refreshing metadata from different sources on a regular basis
Metadata quality
Organisational issues
Understanding between business users and technical users
Metadata management and stewardship
17
Acknowledgements
• Supervisors: William Yeoh, Andy Koronios• UniSA Business Intelligence team members: Marc
Conboy, Duncan J Murray, Andrea Matulick and others
• My wife: Olga Ryabova
18
References• Beyer, M. A. (2007). Why Metadata Matters to Business Intelligence Initiatives, Gartner.• Burton, B., J. Popkin, et al. (2007). Workshop Results: Challenges Users Face in Supporting Business
Intelligence, Gartner.• Chisholm, M. (2008). "Business Intelligence Problems and the Abstraction-Translation Paradigm." Retrieved
28/12/2008, 2008.• Gartner. (2007). "Gartner EXP Survey of More than 1,400 CIOs Shows CIOs Must Create Leverage to Remain
Relevant to the Business." Retrieved 01/04/2009, from <http://www.gartner.com/it/page.jsp?id=501189> .• Gartner. (2008). "Gartner EXP Worldwide Survey of 1,500 CIOs Shows 85 Percent of CIOs Expect "Significant
Change" Over Next Three Years." Retrieved 01/04/2009, from <http://www.gartner.com/it/page.jsp?id=587309> .• Gartner. (2009). "Gartner EXP Worldwide Survey of More than 1,500 CIOs Shows IT Spending to Be Flat in
2009." Retrieved 01/04/2009, from <http://www.gartner.com/it/page.jsp?id=855612> .• Foulonneau, M. and J. Riley (2008). Metadata for Digital Resources. Implementation, System Design and
Interoperability. Oxford, Chandos Publishing.• Friedrich, J. R. (2005). Meta-data version and configuration management in multi-vendor environments. ACM
SIGMOD international conference on Management of data, Baltimore, Maryland, ACM.• Inmon, W., B. O'Neil, et al. (2008). Business Metadata, Capturing Enterprise Knowledge, Elsevier.• Lawton, G. (2006). Making Business Intelligence More Useful. Computer, IEEE Computer Society. 39: 14-16.• Paolo Missier, Pinar Alper, et al. (2007) "Requirements and Services for Metadata Management." Semantic
Knowledge Management.• Shankaranarayanan, G. and A. Even (2006). "The metadata enigma." Communications of the ACM 49(2): 88-94.• Vaduva, A. and K. R. Dittrich (2001). Metadata Management for Data Warehousing: Between Vision and Reality.
International Database Engineering & Applications Symposium.• Ward, D. (2007). Data and Metadata Reporting and Presentation Handbook, OECD.