Department of Health and Families is a Smoke Free Workplace

Business Intelligence Architecture and Strategy

Department of Health and Families is a Smoke Free Workplace

Authors

Stephen Moo Peter Kerr Francis De Silva Kristine Luke Jeff Robson Amanda Lanagan General Enquiries about this publication should be directed to: Data Warehouse Systems PO Box 40596 Casuarina NT 0811 Phone: (08) 8999 2630 Fax: (08) 8999 2618

© Department of Health and Families, Northern Territory 2010

This publication is copyright. The information in this report may be freely copied and distributed for non-profit purposes such as study, research, health service management and public information subject to the inclusion of an acknowledgement of the source. Reproduction for other purposes requires the written permission of the Chief Executive of the Department of Health and Families, Northern Territory.

Printed by the Government Printer of the Northern Territory, 2010



Table of Contents

Business Intelligence Architecture ...............................................................................1 Definition ......................................................................................................................1 Vision ...........................................................................................................................2 Systems Architecture ...................................................................................................3 Business Intelligence Strategy .....................................................................................4 Enterprise Data Governance........................................................................................5 Extraction and Transform Tools / Processes ...............................................................8 Data Repositories.......................................................................................................10 Analytical Tools / Applications....................................................................................11 Presentation Tools / Applications...............................................................................12 BI Operational Processes ..........................................................................................14 References.................................................................................................................15



- 1 -

Department of Health and Families Business Intelligence Architecture

Context The Department of Health and Families (DHF) is a diverse and complex organisation. The Business Intelligence Strategy recognises the complexity, volume and maturity of the Department’s information systems and client data collection. The Strategy advocates the need to develop and implement an appropriate architecture to support the Department’s business intelligence objectives and aspirations. The architectural approach acknowledges the complexity and dependencies that exist between the many components of business intelligence that range from timely and accurate collection of information at point of service, through the technical complexity of data transformation and end user reporting and analysis. The strategy also recognises the need to increase information use for a broad range of strategic, operational and research purposes. The DHF Corporate Plan 2009–2012 [1] and Strategic Information Plan 2009–2012 [2] support these aims such that the Business Intelligence (BI) Strategy is well aligned with overall corporate objectives. In particular, the Architecture and Strategy supports simplicity and transparency in the development and implementation of key performance indicators under the Council of Australian Governments (COAG) National Healthcare Reform Agreements. A key component of the Strategic Information Plan is to “develop an integrated Business Intelligence Architecture and Strategy”. This document is the articulation of that architecture and strategy and draws together the relevant themes in the Corporate and Strategic Information Plans. In developing the Strategy, consideration has been given to both the distinction and overlap between data used for operational and management purposes compared with that used to inform research questions. The Architecture and Strategy focus on data and information that is collected and used operationally, recognising the agency’s epidemiological role as a discrete but related function.

Definition The Business Intelligence Architecture can be defined as: An enterprise-wide set of systems, applications and governance processes that enable sophisticated analytics by allowing data, content and analyses to flow to those who need them, when they need them. [3] The Architecture mirrors the information lifecycle from entry and collection, through extraction, processing and transforming and finally reporting. The Architecture incorporates the following elements: [4]

Data Directory [5] Enterprise Data Governance [6] Extraction and Transform Tools / Processes Data Repositories Analytical Tools / Applications Presentation Tools / Applications Operational Processes.



- 2 -

Figure 1 DHF Information Lifecycle.

Vision An end-to-end business intelligence architecture – from point of data capture through to reporting, data analysis and application to service provision – which:

Instils confidence in data users Provides for consistency in data access across all data areas, systems and

service levels Is based upon a well-founded and practical policy framework and well-defined

standards Emphasises data quality Translates data into useful information Drives a ‘data smart’ workplace culture.



- 3 -

Figure 2 DHF Integrated Information Systems Architecture as of March 2010.



- 4 -

DHF Business Intelligence Strategy The implementation of the DHF Business Intelligence Strategy fulfils the core components of the Business Intelligence Architecture.

Element: Data Directory The Data Directory defines where in the organisation we can locate particular data. It is restricted to data regarding clients and client services and is applicable to corporate data collections only. The Data Directory is implemented as a map or index of data collections.

Current Status No such directory exists within DHF at the current time of publishing. Individual work units maintain their own directories to varying degrees of completeness. Future Direction / Needs Develop a Department-wide directory of data collections. Those who manage them are to provide a reference for whom to contact for particular data.



- 5 -

Element: Enterprise Data Governance The Enterprise Data Governance Framework [7] (Figure 3) defines data governance as the “Framework, policies and procedures governing the context, definition and validation of data in corporate information systems with the aim of improving business ownership, enhancing reproducibility, increasing accuracy and improving interpretation of data across the organisation.” Data governance defines explicit rules and standards which are to underpin client data collection, data management and reporting.

Figure 3 Enterprise Data Governance Framework.

Governance oversight belongs with the DHF Strategic Information Management Steering Committee (SIMSC) and associated Information Management Groups (Figure 4).



- 6 -

Figure 4 Information Management Group (IMG) structure. Core components

Business rules and standards Metadata Data Quality Framework, e.g. accuracy, completeness and consistency Increasing data analysis and interpretation skills in the Department.

Current Status The data governance framework is being implemented through the establishment of the Data Governance Unit together with the work of the Data Governance Working Group. A range of metadata and validation techniques are applied by reporting and data management units to varying levels of rigour. High variability in quality and timeliness in source information from client information systems has resulted from fragmented records management standards and practices. Future Direction / Needs

Develop rules around what information to collect and under what circumstances, as a core program responsibility rather than an IT/IM function.

Define, publish and implement standards and rules around currency, accuracy,

timeliness of client information collection and availability to achieve:

o data trustworthiness o timely decision-making and intervention o improved understanding of service utilisation o improved service management, e.g. compliance reporting o improved capacity to plan and improve services o consistency in data published across a range of reports.

Improve adherence to professional obligations for quality client record-keeping. Produce information to inform service planning, program evaluation, KPIs and

business intelligence. This is the ‘value-add’ derived from the organisation’s information assets.



- 7 -

The Data Governance Framework defines the following broad directions:

Determine technical and business metadata to be captured at each point along the Department’s information spectrum.

Determine validation principles and techniques to be applied to all reported data,

including data quality certification and statements. Develop a metadata and data quality status capture and reporting application. Develop a program to upskill managers and staff in information analysis and

interpretation.

Specific needs incorporated within or additional to those identified in the Data Governance Framework include:

Age and freeze data (cross-link to Architecture Element Data Repositories). Explore co-locating metadata, i.e. ‘Hover mouse over data element and a definition

appears’. Develop policies and guidelines for use of Excel and other analysis tools, including

listing business rules, describing manipulations, etc.

Document business rules, in particular those associated with operational system reports.

Transparency of data manipulation and mapping undertaken on source data.



- 8 -

Element: Extraction and Transform Tools / Processes Transformation tools and processes that describe how data is extracted, cleaned, transmitted and loaded to populate databases. These processes require input from both business and IT and establishing the processes requires considerable manual effort… standardising data definitions and managing missing data. [8] Core components

DataStage

Extraction processes from operational systems.

See Figure 5 for a diagram of the ETL process, incorporating source system and data repository.

Figure 5 Basic ETL Functionality. [9]

Current Status Extraction Data extraction from the Jade Operational Systems is via extract programs written in Jade Report Writer. When run, these programs create delimited or fixed width text files containing the extracted data which the Data Warehouse uses for the remainder of the ETL process. For CCIS and PCIS these extracts are created and maintained by the AMS team. Caresys requests for new, or updates to, extracts are submitted to the Caresys vendor Ascribe. Documentation levels and quality for these extracts varies. Comprehensive extract documentation exists for Caresys and PCIS Data Warehouse extracts.



- 9 -

Extraction from relational data sources (e.g. Patient Travel, Notifiable Disease System) is achieved by connecting the Data Warehouse ETL tool, IBM DataStage, directly to the source system databases to download the data. Transformation The functions of the transformation process include:

Data cleansing: application of pre-determined validation/business rules to rectify data quality issues and flag them for follow-up where they cannot be automatically resolved.

Data standardisation: ensuring that all date fields are stored in one format in the

warehouse, regardless of their format in their respective source systems. Data transformation: application of derivations, calculations and data

manipulation to meet reporting requirements which are unable to be efficiently implemented at the source system level.

Currently transformation functionality is implemented in both DataStage and various extraction programs (for Jade systems). Documentation of these transformation rules outside of DataStage has been virtually non-existent and is one of the biggest exposures currently within the warehouse. This is being actively addressed through the Acute and CCIS reporting environment rebuilds. Loading DataStage is used to populate data into the Data Warehouse and its associated datamarts. Future Direction / Needs The biggest weakness in this sphere is a lack of standardisation and documentation. A standard DataStage ETL methodology and associated templates are being developed as part of the Acute and CCIS management reporting environment rebuilds. Documentation standards/templates will also be produced. The Department currently uses DataStage version 7.5 (server edition). The latest version available is 8.x, with a different technical architecture. v7.5 is more than adequate for the Department’s ETL needs for the foreseeable future and IBM has indicated that it has no plans to withdraw support for it. However an upgrade will need to be considered given the risk in assuming IBM’s continued support of v7.5. There are also some functionality enhancements that will be useful in v8.x. Upgrading to v8.x (server edition) is free, however the process will not be straightforward as the technical architecture of v8 is significantly different, including substantially higher hardware infrastructure requirements. With regards to the Jade system extracts, we should also strive for a level of standardisation in terms of documentation and validation processes (e.g. utilisation of new CCIS validation process).



- 10 -

Element: Data Repositories Repositories organise data and metadata, storing them as secondary storage. Core Components

Data Warehouse Data Marts Metadata repository (data dictionary).

Figure 6 Data Repositories. Current Status The Data Warehouse and Data Marts currently exist. Re-modelling of the warehouse and data marts – to improve reporting capacity, reduce redundancy and improve process efficiency – is underway through the Acute (Emergency Department Data Re-development Proof of Concept) and CCIS management reporting environment rebuilds. There is no synthesised metadata repository. The repository is being developed under the aegis of Data Governance. Future Direction / Needs

Metadata Repository development (cross-link to Architecture Element Enterprise Data Governance).

Explore co-locating metadata (see Enterprise Data Governance). Ageing and freezing data (see Enterprise Data Governance). Define the role and scope of data warehouse services and develop a formal

corporate statement of purpose. Determine the Department’s policy regarding corporate data collections held

external to the data warehouse and reported to external agencies. Policy and procedures around development of databases against specified

standards.



- 11 -

Element: Analytical Tools / Applications Core Components

Excel, Access (>65000 rows) SAS, EpiInfo, MapInfo, GIS (Health Gains) Stata OLAP (On-line Analytical Processing).

Current Status

Excel prone to at least 20 per cent error Major gap between canned reports and power users in controlled analytics.

Future Direction / Needs

Excel (take up under Enterprise Data Governance) Pivot tables, multi-dimensional reports Statistical Data Linkage with NCRIS – SA, NT, Tas Agreement Statistical Data Linkage: WA – Health, Education, Community Services,

Ambulance etc.



- 12 -

Element: Presentation Tools / Applications The Department’s presentation tools and applications are used by information analysts and other users. Many users are without an information management or information technology background required to access, display, visualise and manipulate data. Such data reflects the real-time state of play of the current moment, for example Emergency Department bed occupancy at a given time. Core Components

BusinessObjects Crystal Reports Operational system reports Potentially real-time BI solutions Structured Query Language (SQL).

Current Status

Operational System reports: Jade applications – can design own reports Crystal Reports: exists Real-time BI: Operational System reports, Jade ReportWriter SQL: report directly out of MedChart via SQL Analyser An issue with proprietary systems regarding how we ensure we report and extract

to the data warehouse.



- 13 -

Future Direction / Needs:

BusinessObjects:

o Utilise to full capacity o Consistency in skill sets o Review system management responsibility o Take advantage of new versions more frequently o Determine whether the system meets our current and emerging needs.

Operational System Reports:

o Document business rules (see Enterprise Data Governance) o CareSys – develop flexibility to design or adapt existing reports.

Real-time BI: Explore capacity to view transformed data, not just data straight off

operational system.



- 14 -

Element: BI Operational Processes Standards, policies and procedures for maintaining security, auditing, archiving, privacy across secondary data storage. Core components

Security Auditing Archiving Privacy and data access controls.

Current Status

Security: Risks relating to access to client information and identifiers in secondary data sources. Risks associated with access to sensitive information, e.g. performance indicators; communicable disease incidence.

Auditing: Inability to audit access to BusinessObjects reports and Universes. Archiving: Unspecified point in time at which historical data ceases to be available in DW production environment.

A range of historical data collections need to be archived to the data warehouse (e.g. CCTIS decommissioning).

Privacy and data access controls: The existing Data Access Protocol (DAP) governs access to corporate data collections and underpins the resolution of staff member access queries. The data warehouse has the capacity to facilitate auditing.

Future Direction / Needs and Actions

Security: Document security procedures. Auditing: Clarify audit capacity in BusinessObjects V3.1 or alternative means (eg. Third party software). Archiving: Query Records Management on archiving standards for secondary data storage, including disposal policy (eg. tape backup policy). Determine strategy, priority and responsibility for historical data collection archiving from decommissioned client information systems.

Privacy and data access controls: Determine responsibility for managing queries on staff member system access. Re-visit data sponsor model adequacy. Community profiling, in particular of Health Status: A project based on small population datasets, requiring data linkage and challenging the existing Data Access Protocol (DAP).



- 15 -

References 1 Department of Health and Families. Corporate Plan 2009–2012. August 2009. p 22

Priority Action Area 5 “Safety, Quality and Accountability”, Key Focus Area 3 “Conduct research, knowledge exchange and performance monitoring that leads to best practice”, including: Strengthen the use of evidence and best practice in service delivery through

knowledge exchange; and

Promote knowledge exchange and evidence-based practice. 2 Department of Health and Families. Strategic Information Plan 2009-2012. September

2009. Strategy 1 in the Strategic Information Plan 2009–2012 promotes related business

outcomes including (p 4): [The] development of a performance-based culture that understands and

values the use of information;

Information and knowledge to support proactive management and service

reform.

The focus on information infrastructure management is further conveyed through the following business outcomes and core components across the Plan: [Develop the] ability to provide high quality information to support increased

service demands and service improvements and reform (p 7)

[Develop] audit ability and accountability (p 4)

Improve access, collection and use of client information at the point of service

to improve quality of service delivery and achieve compliance with policies and

procedures (p 7). 3 Accenture (Institute for High Performance Business). The Architecture of Business

Intelligence. Adapted from Davenport, T.H., Harris, J.G. “Competing on Analytics: The New Science of Winning”. March 2007. p 3.

4 Ibid. 5 In Davenport et al (p 3), this element is conceived as Data Management (“how the

right data is acquired and managed”). For DHF, Enterprise Data Governance encapsulates the concepts of data management and more, but excludes the notion of cataloguing data collections. Hence the separate, additional element ‘Data Directory’.

6 See note above. 7 Lanagan, A. A Framework for Enterprise Data Governance, v1.5. Endorsed by Chief

Information Officer October 2009. 8 Accenture (Institute for High Performance Business). The Architecture of Business

Intelligence. Adapted from Davenport, T.H., Harris, J.G. “Competing on Analytics: The New Science of Winning”. March 2007. p 7.

9 Sherman, R. Beyond ETL & Data Warehousing. In “Information Management Best Practices: Data Integration”. January 2009. Fig. 1, p 3.

Department of Health and Families is a Smoke Free Workplace

Documents

Transcript of Department of Health and Families is a Smoke Free Workplace