NCR#5599939 - v8

Final Report

Evaluation of Information for Decision Making Project Number: 1570/13069

September 2013

Evaluation, Performance Measurement, and Review Branch Audit and Evaluation Sector

NCR#5599939 - v8

Table of Content Executive Summary .................................................................................................................... iv 

1. Introduction ............................................................................................................................... 1 

1.1  Overview .......................................................................................................................... 1 

1.2  Background ...................................................................................................................... 1 

1.3  Scope and Activities ......................................................................................................... 2 

1.4  Governing Policies ........................................................................................................... 3 

1.5  Key Stakeholders and Beneficiaries ................................................................................. 3 

1.6  Previous Audits and Evaluations ...................................................................................... 4 

1.7  Resources ......................................................................................................................... 5 

1.8   Expected Outcomes .......................................................................................................... 5 

2. Evaluation Methodology ......................................................................................................... 6 

2.1   Evaluation Scope and Timing .......................................................................................... 6 

2.2   Evaluation Issues and Questions ...................................................................................... 6 

2.3   Evaluation Methodology .................................................................................................. 8 

3. Key Findings – Relevance .................................................................................................... 11 

3.1  Is there a continued need for the types of information available for decision making? 11 

3.2   Does the type of information being collected align with government priorities, federal roles, jurisdiction and responsibilities? ..................................................................................... 15 

4. Key Findings – Success/Effectiveness .............................................................................. 17 

4.1   How well does departmental information allow AANDC to achieve the outcomes identified in the Policy for Information Management and Policy on Management, Resources and Results Structure? ............................................................................................................... 17 

4.2   What are the key factors helping or impeding the availability, credibility and reliability of information for decision making? ......................................................................................... 23 

5. Key Findings – Efficiency/Economy .................................................................................... 28 

5.1   Does the Department have the capacity to analyze its data? To what extent is this capacity actualized? How is the analysis of data used? ............................................................ 28 

5.2  Is information disseminated sufficiently and appropriately throughout the Department? 30 

5.3   How effective and efficient is storage and access to information in the Department? .. 32 

5.4   To what degree does departmental governance facilitate open lines of communication? 33 

5.5  Is information for decision making within the Department cost-effective? ................... 36 

Conclusion ................................................................................................................................... 39 

NCR#5599939 - v8

Bibliography ................................................................................................................................ 40 

iv

Executive Summary The Evaluation, Performance Measurement and Review Branch at Aboriginal Affairs and Northern Development Canada (AANDC) has undertaken an evaluation of Information for Decision Making. Information for Decision Making was flagged as high risk in the AANDC 2012 Corporate Risk Profile. As such, the Information Management, Information Technology, Management and Oversight and Financial Management sub-activities under the Department’s Chart of Accounts have been evaluated in relation to this risk. This is the first evaluation of internal services conducted by the Department. The evaluation found that while information for decision making is strong at AANDC relative to other organizations, several key challenges exist:

· Challenges with Information Technology (IT) Systems: 78 IT systems exist at AANDC, which are not properly integrated with each other. The Department's main information database, CIDM, has been characterized as cumbersome and slow, with excessive document classification requirements.

· Information Management Culture (storage and classification): In part as a result of IT

systems that are not user-friendly, employees at AANDC generally do not follow proper guidelines for information management. Such guidelines include proper classification of meta-data to make information retrievable and digitization of paper records.

· Information Management Culture (privacy versus collaboration and dissemination):

There is hesitancy at AANDC to share information with others. As a result, information has been produced in 'silos', which can result in unnecessary and costly duplication efforts. Furthermore, a hesitancy to dispose of unnecessary information causes excess storage costs and difficulty searching for what is most pertinent.

· Lack of context for financial decisions: Financial reports are often not fully populated,

resulting in a lack of information for decision making. Additionally, due to a lack of systems integration, financial decisions are sometimes made without non-financial qualitative context for support. In Information Management (IM) and IT particularly, decisions are often made based on budgetary concerns without a complete understanding of the need for enhanced IM/IT capacity.

· Narrow timelines: The main constraint on AANDC's capacity to analyze its data is the

context of tight timelines in which accurate and comprehensive information is required by decision makers. Strategies such as time management training and securing a supportive IM/IT base could provide realistic measures to foster the advisory function that is vital to decision makers.

· Excessive reliance on informal networks: Given the challenges described above to

accessing needed information, senior management routinely finds informal networks of personal contacts more helpful for supporting decision making. Often personal

v

experience is relied upon more than departmental analysis. When accompanied with employee turnover, this presents a real concern to maintenance of corporate memory.

· Low prioritization of Information for Decision Making: These effects are exacerbated by

a structure in which information for decision making is not prioritized. Concerns have been expressed that information for decision making is viewed primarily through a financial lens, and that fostering the relationship between the Chief Information Officer and senior management, particularly the Deputy Minister, may help to support the IM/IT function at AANDC.

The issues above present potential risks to access, reliability, credibility, timeliness, cost-efficiency and other pertinent qualities of information for decision making. These concerns, as well as opportunities for mitigation, are outlined in this report.

1

1. Introduction 1.1 Overview The 2009 Treasury Board Secretariat Policy on Evaluation mandates that all direct program spending be evaluated every five years. The Policy gave departments until March 31, 2013, to fully implement Section 6.1.8, requiring that this include internal services. This evaluation is the first in the Department to evaluate internal services. For the purpose of this evaluation, these internal services are examined separately from the programs they support. Information for Decision Making was flagged as high risk in the Aboriginal Affairs and Northern Development Canada (AANDC) 2012 Corporate Risk profile and thus, it was decided that this would comprise the first evaluation of internal services. The evaluation was part of the Evaluation, Performance Measurement and Review Branch’s (EPMRB) summer student internship program for 2013. For scoping, it was decided that the evaluation would include a selection of sub-activities under the Department’s Chart of Accounts, which included Information Management/Information Technology (IM/IT), Management and Oversight and Financial Management. The performance of AANDC in ensuring appropriate information for decision making was available, specifically as it relates to the sub-activities mentioned, was measured against key outcomes in both the Treasury Board Policy on Information Management and the Management, Resources and Results Structures (MRRS) Policy. 1.2 Background The AANDC Corporate Risk Profile identified the Information for Decision Making Risk area as high risk. This risk area acknowledges the critical importance of having information that is reliable, timely, pertinent, consistent and accurate to support good decision making, and that skilled and experienced people need relevant and accurate information to be effective. In AANDC’s 2013–14 Report on Plans and Priorities, information for decision making was again identified as an area of high risk. This was echoed in the Executive Risk Report for AANDC, which warned that the Department will not make sufficient progress to improve access to timely, pertinent, consistent and accurate information to support planning/policy, resource allocation and programming decisions, monitoring/oversight, and to fulfill accountability, legal and statutory obligations unless the following major risk drivers are mitigated: • AANDC’s performance measures not fully defined and agreed upon. • Difficulty obtaining timely, accurate and complete data from outside sources. Recipients

faced with extensive reporting requirements and some lack capacity to report accurate and timely information.

• Many programs lack information and capacity to perform data analysis, including identifying opportunities for integration across programs.

• Absence of a centralized data warehouse to capture, share and mine program information. • Insufficient processes and systems contribute to issues around data integrity, duplicate data

entry and accessibility.

2

• Recipient reporting requirements not well aligned with program performance measures, leading to gaps in information or unnecessary information.

• AANDC monitoring and recipient auditing activities are insufficient to ensure integrity and reliability of recipient reporting.

• Systems of record are not used in a consistent manner to categorize and store information, making some records difficult to access.

1.3 Scope and Activities The purpose of this evaluation was to examine, insofar as it relates to the sub-activities listed above, the relevance of the information being collected, and the performance of the Department in ensuring information was available to support informed decision making. Internal services are defined as groups of related activities and resources that are administered to support the needs of programs and corporate obligations of an organization. Within the Government of Canada, they are divided into three activities: governance and management support, resource management services, and asset management services. The sub-activities being assessed fall under the governance and management support and resource management services. They include: Management and Oversight, Financial Management, and Information Management and Information Technology. A brief overview of these sub-activities is below: Management and Oversight According to Treasury Board Secretariat, Management and Oversight is a sub-activity of Governance and Management Support.1 Management and Oversight is defined as “those activities undertaken for determining strategic direction and allocating resources among services and processes, as well as those activities related to analyzing exposure to risk and determining appropriate countermeasures.2 Management and Oversight activities ensure that the service operations and programs of the federal government comply with applicable law, regulations, policies and/or plans.”3 The Management and Oversight sub-activity includes: Senior Management Operations; Strategic Policy; Policy Standards and Guidelines; Service Improvement; Audit and Evaluation Services; Business Continuity Planning; and Aboriginal Peoples Survey. Financial Management Financial Management is a sub-activity of resource management services and involves activities undertaken to ensure the prudent use of public resources. The Chief Financial Officer Sector’s roles and responsibilities mostly entail financial management. It provides strategic advice, oversight and support to the Deputy Minister and Senior Executive Team to ensure integrity and sound financial control and management in the planning, operations, and performance reporting of the Department. The Financial Management sub-activity includes: Financial Planning and Budgeting; Accounting Management; and Funding Services/Transfer Payment Management.

1 Treasury Board of Canada Secretariat, Presentation on Profile of GC Internal Services- Summary, 2008. 2 Library and Archives Canada, 2013, Generic Valuation Tool: Management and Oversight Services- DRAFT, 2013. Available at: http://www.gcpedia.gc.ca/gcwiki/images/6/6f/M%26O_Draft_GVT_May_2013.pdf. 3 Ibid

3

Information Management and Information Technology At AANDC, the Chief Information Officer oversees these resource management functions and ensures the Department has a framework and tools for the adequate management of information for the collection, storage, use and disposition of data. These functions are overseen by the Information Management Branch. Information Management Branch also sets out guidelines for and promotes awareness of proper information management practices. Furthermore, the Branch provides resources to assist programs in following these guidelines and also maintains a Records Office to store and classify departmental documents. Information Technology is a sub-activity of resources management services and is the enabler of effective information management. Information Technology at AANDC primarily involves the design, creation and maintenance of software and systems for information capture, storage and analysis. The efficient and effective use of information technology supports government priorities and program delivery, increases productivity, and enhances services to the public. The Information Technology sub-sub-activity includes: Information Technology Services; and Application 1.4 Governing Policies The following pieces of legislation, policies and directives are relevant to the evaluation of information for decision making:

Financial Administration Act Library and Archives Canada Act

Treasury Board Policy on Financial Management Governance Treasury Board Policy on Information Management Treasury Board Policy on Management, Resources and Results Structures

Treasury Board Directive on Information Management Roles and Responsibilities Treasury Board Directive on Recordkeeping

1.5 Key Stakeholders and Beneficiaries The following organization and groups may benefit from an Evaluation of Information for Decision Making:

AANDC departmental functions o Evaluation of information for decision making will impact decision makers within

the organization such as the Director, Director General, Assistant Deputy Minister, and Deputy Minister.

o May affect how branches within the Department collect, record and store data. o May also affect processes for information dissemination and application within

the branches of the Department.

4

Central agencies due to their regulatory and oversight function Other government departments, which may wish to evaluate information for decision

making. Aboriginal peoples as the recipients of policy and decision making at AANDC. The Canadian public, as broad recipients of policy implications and decision making at

AANDC.

1.6 Previous Audits and Evaluations In 2010, an audit of IM/IT Expenditures and Management Control Framework4 at AANDC was conducted. The findings revealed there was inconsistent and incomplete asset tracking, control deficiencies related to IM/IT asset procurement, inadequate cost-benefit analyses of third-party service agreements, and misalignment of IM/IT strategies employed across regions and Headquarters. It was recommended that a stronger IM/IT accountability framework be established to enforce greater consistency across the Department. It was also recommended that AANDC identify root causes embedded in overall IM/IT governance and address the issues in consultation with other government departments. In 2013, two reports were released highlighting a lack of information needed for performance measurement across government and within AANDC: The Horizontal Internal Audit of Compliance with the Policy on MRRS and the Annual Report on the State of Performance Measurement in Aboriginal Affairs and Northern Development Canada for 2011-12. The Horizontal Internal Audit of Compliance with the Policy on MRRS found departments have created Performance Measurement Frameworks and are collecting performance data, but departmental performance measurement processes do not provide the information needed to assess how programs are achieving their expected results. It was recommended that performance measures be improved to further support departmental planning and decision making. The findings were presented to each of the departments included in the scope of the audit, and management agreed to take action to address all applicable recommendations. The Annual Report on the State of Performance Measurement in Aboriginal Affairs and Northern Development Canada for 2011-12 found that much of the data collected by the Department remains unanalysed. As in previous years, the Department continues to encounter challenges in gathering and analysing existing information, accessing information by relevant users, and poor tracking tools. Its performance management is hindered by a lack of communication resulting in program silos, duplication of information and missed opportunities to learn from past mistakes.

4 Aboriginal Affairs and Northern Development, 2010, Audit of IM/IT Expenditures and Management Control Framework. Available at: http://www.aadnc-aandc.gc.ca/eng/1321552088414/1321552132191.

5

1.7 Resources The extent of coverage for this evaluation can be broken down as follows:

Areas of Coverage Percentage of Coverage Amount ($) Information Management 100% $16,711,627 Information Technology 100% $9,738,278 Management and Oversight 50% $18,368,049 Financial Management 10% $2,669,359

Total: $47,487,313 1.8 Expected Outcomes The AANDC Corporate Risk Profile 2012 highlighted a lack of performance measurement indicators as a risk to Information for Decision Making. Thus, the most reliable benchmark against which to measure performance of this function are the expected outcomes of the policies on Information Management and Management, Resources and Results Structures. From the Policy on Information Management:5

That programs provide convenient access to relevant, reliable, comprehensive and timely information.

That information and records are managed as valuable assets to support the outcomes of programs and services, as well as operational needs and accountabilities.

That governance structures, mechanisms and resources are in place to ensure the continuous and effective management of information.

From the Policy on Management, Resources and Results Structures:6

That departments collect sound financial and non-financial information, results and other key data on programs to support informed decisions on program management, and new program proposals.

That sound financial results and other key program data are generated to support improved allocation and reallocation decisions in individual departments and across the Government.

That public performance reporting to Parliament and Canadians is based on sound financial and non-financial performance information.

Based on those expected outcomes, the evaluation will generally examine the collection, storage, analysis, consistency and dissemination of financial and non-financial information as a means of measuring the quality of information for decision making. 5 Treasury Board Secretariat, 2007, Policy on Information Management. Available at: http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=12742&section=text#sec5.2. 6 Treasury Board Secretariat, 2012, Policy on Management, Resources and Results Structures. Available at: http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=18218&section=text.

6

2. Evaluation Methodology 2.1 Evaluation Scope and Timing The aim of this evaluation is to assess whether decisions makers have the necessary and required information available to them in order to make well-informed decisions. It should be noted that this evaluation does not attempt to assess the effectiveness or the outcomes of any given decision. As discussed in Section 1.8, information should be conveniently accessible, relevant, reliable, comprehensive and timely. Financial and non-financial information should be managed as valuable assets to inform decisions on program management, new program proposals, and improve allocation and reallocation decisions in individual departments and across the Government. There was no specific time frame for this evaluation. It is a real time assessment of internal services in relation to information for decision making within the Department. The evaluation was conducted between May to August 2013 by ten interns working in the EPMRB with the oversight of two senior analysts. The Terms of Reference were approved by AANDC's Evaluation, Performance Measurement and Review Committee (EPMRC) on June 21, 2013. 2.2 Evaluation Issues and Questions As per the Treasury Board Secretariat Directive on Evaluation,7 this evaluation examined the following core evaluation issues:

EVALUATION ISSUES EVALUATION QUESTIONSResearch Question To what extent do the Management Oversight; Financial Management; and Information Management and Information Technology sub-activities contribute to the availability of credible information for decision making?

Relevance

7 Treasury Board Secretariat, 2009, Directive on the Evaluation Function. Available at: http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15681&section=text. Government of Canada.

7

Continued Need for Information for Decision Making (assessment of the extent to which information continues to support decision making)

1. Is there continued need for the types of information available for decision making?

a. What types of information are being accessed by decision makers?

b. Do decision makers have an awareness of the types of information available and how to access them?

c. Are they able to access the types of information required?

Alignment with Government Priorities(assessment of the linkages between departmental information for decision making and federal responsibilities and roles)

2. Does the type of information being collected align with government priorities, federal roles, jurisdiction and responsibilities?

Performance - Effectiveness/Success

Achievement of Expected Outcomes(assessment of progress towards expected outcomes (includes immediate, intermediate and ultimate outcomes)

3. How well does departmental information allow AANDC to achieve the outcomes identified in the Policy for Information Management and Policy on Management, Resources and Results Structure?

4. What are the key factors helping or impeding the availability, credibility and reliability of information for decision making?

Performance – Efficiency/EconomyDemonstration of Efficiency and Economy(assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes)

5. Does the Department have the capacity to analyze its data? To what extent is this capacity actualized? How is the analysis of data used?

6. Is information disseminated sufficiently and appropriately throughout the Department?

7. How effective and efficient is storage and access to information in the Department?

a. To what degree is it supported by information technology?

8. To what degree does departmental governance facilitate open lines of communication?

9. Is information for decision making within the Department cost-effective?

8

2.3 Evaluation Methodology The following section describes the data collection methods used to perform the evaluation work, as well as the major considerations, strengths and limitations of the report and quality assurance. 2.3.1 Data Sources The evaluation’s findings and conclusions are based on the analysis and triangulation of three lines of evidence: document review, literature review, and key informant interviews. Document Review Approximately 95 documents pertaining to the four sub-activities of internal services under the scope of this evaluation were collected and reviewed. This included but was not limited to: Treasury Board Secretariat policies, directives and standards related to internal services; internal AANDC frameworks, business plans, mandates, and terms of reference; previous audits and evaluations of internal services within other departments. Documents were reviewed specifically from the Management Oversight, Financial Management, and IM/IT sub-activities. Relevant documents were scanned for content relating to key evaluation questions and the analysis was triangulated with other lines of evidence. Literature Review Approximately 72 pieces of literature were reviewed that were pertinent to both the sub-activities specifically and Information for Decision Making as a whole. The literature includes academic journals, book reviews, university publications, think-tank publications, and strategy documents from other governments. The literature review aimed to demonstrate the ways that internal services support information for decision making in different capacities. Specifically, the literature review, as one of the lines of evidence, pursued the following objectives:

1. To determine the history and development of all of the sub-activities and their respective methodologies;

2. To determine best practices for the flow of information through internal services within the context of informed decision making;

3. To determine general experiences with the interaction of internal services and information for decision making across departments and governments in Canada and abroad; and

4. To explore the role of internal services within the context of Information for Decision Making in a broad range of organizational structures from both theoretical and empirical levels.

9

The findings from the literature review complemented the document review, offering a theoretical and practical perspective of the Information for Decision Making process. The literature review also offered potential explanations as to the development and structure of the current functions of the sub-activities within AANDC, as well as suggested best practices. These lessons learned could provide alternatives and innovative solutions to resolve issues pertinent to information for decision making within AANDC. Key Informant Interviews A number of informal preliminary interviews were conducted with key stakeholders within internal services to construct comprehensive key informant interview guides. A total of 28 key informant interviews were conducted with AANDC officials in Headquarters. Interviews have been conducted mostly with the following officials: Office of the Chief Information Officer; Office of the Chief Financial Officer; Information Management Branch; Legislative, Parliamentary and Regulatory Affairs; Strategic Policy; Planning and Resource Management; and the Human Resources and Workplace Branch. Interviews were semi-structured and conducted in-person or by telephone. Detailed notes were taken during the interviews and then transcribed and analyzed according to research themes. Interview notes were shared with key informants for fact verification upon request. 2.3.2 Considerations, Strengths and Limitations Considerations Information and knowledge management as well as change management are recurrent themes within the academic literature, which is pertinent to both public and private sector organizations. This evaluation provides an opportunity to learn from best practices in other jurisdictions and within the private sector to garner effective solutions to improve information for decision making. Strengths As mentioned previously, this is an area that is garnering increasing attention. Central agencies, especially Treasury Board Secretariat, are attempting to review internal services reporting in order to measure or benchmark them effectively. Based on the findings from the reports above, information for decision making is a pressing issue, which has increased in importance over recent years. As such, this evaluation is timely in addressing this heightened interest. Limitations There were time constraints to the evaluation over a four month period, which did not allow a comprehensive and in-depth assessment of information for decision making. Additionally, key informants were only from internal services directorates within AANDC. Although some program assistant deputy ministers were interviewed, it would have been beneficial to interview front line staff within programs in varying sectors of the Department, as well as external to the Department.

10

2.3.3 Roles, Responsibilities and Quality Assurance EPMRB of AANDC's Audit and Evaluation Sector was the authority for the Information for Decision Making evaluation, and managed the evaluation in line with EPMRB's Engagement Policy, outlined in Section 1.3, and Quality Assurance Strategy. The Quality Assurance Strategy is applied at all stages of the Department's evaluation and review projects, and includes a full set of standards, measures, guides and templates intended to enhance the quality of EPMRB's work. Oversight of daily activities was the responsibility of the EPMRB student evaluation team, headed by two senior evaluators. The EPMRB student evaluation team was responsible for overseeing all aspects of data collection and validation, as well as composition of the final report and recommendations. In addition, status updates and key findings were regularly provided to EPMRB and the Centre for Excellence in Evaluation for strategic advice and direction.

11

3. Key Findings – Relevance 3.1 Is there a continued need for the types of information available for decision making? Finding: Decision makers access internal and external types of information, which are relevant to decision making within the Department. These internal and external sources include data-driven reports, advice and analysis provided at committees, and pertinent departmental or government documents. It is clear that decision makers are aware of the types of information available but may not readily make use of the guidance provided by directorates with expert knowledge and advice. Information accessibility can also be restricted by slow and partially integrated IT systems; poor information management; data entry omission; and privacy and security restrictions. 3.1.1 What types of information are being accessed by decision makers? According to key informants, decision makers are accessing a variety of internal and external information. These include: data driven reports, analysis and advice presented at committees, and pertinent departmental or government documents. Although many of these information sources are published and made available to the public, they are primarily taken from within the Department or the federal government. Data-driven Reports Many key informants noted that decision makers draw on data and reports from software such as GCIMS, OASIS, GLMS, OSMS, SNP8 or PeopleSoft. They provide a variety of financial and non-financial information required to make decisions. Raw data and numbers are entered into these programs, which then keep track of a variety of departmental information and activities (daily, weekly, monthly, semi-annually and annually). These reports can include: departmental financial statements (monthly, quarterly and annually), variance analysis on department financial statements, reports on liabilities, specific liabilities reports, accounts receivable aging reports, and accounting for direct loans and loan guarantees.

Analysis and Advice Presented in Committees Key informants, specifically those for the Management and Oversight sub-activity, commented that information for decision making is garnered through regularly scheduled committees such as the Financial Management Committee, Executive Committee, Directors' General Policy Review Committee, Integrated Planning Committee and Operations Committee. Presentations provide context to frame policy options and alternatives, which facilitate early decision making processes and provide direction to help a sector or program. Information for decision making can be gathered from committee discussions that are recorded in formal meeting minutes.

8 GCIMS: Grants and Contributions Information Management System

12

Pertinent departmental or government documents Numerous key informants stated that they draw on publicly available government documents to facilitate decision making. These include the Speech from the Throne and the Federal Budget. Other published departmental documents inform decision making such as the Reports on Plans and Priorities, Departmental Performance Reports, Public Accounts, and Main and Supplementary Estimates. 3.1.2 Do decision makers have an awareness of the types of information available and how to access them? Key informants have noted that decision makers may not be fully aware of the information contained in formal departmental storage. This is because the Comprehensive Integrated Document Management (CIDM) system, the main information repository, lacks a search engine that can search beyond the first layer of metadata to inform searchers of what documents are available on a given topic. Once proper naming conventions have been put in place and data is stored in one repository, a search engine can enable access to this information. There are a number of solutions already in place elsewhere within the Government of Canada that promote efficient data retrieval – such as geospatial labeling (data with implicit or explicit reference to a location relative to the Earth)9 and data tagging. Both of these facilitate data retrieval, and thus information access, through the categorization and systematic structuring of unstructured data. However, this capacity is lacking in CIDM. In some instances, decision makers are aware of the types of information available to them but do not readily access it. Some directorates in the Department are designed to have a facilitative and supportive function by providing advice and expertise. These directorates have direct linkages with the Deputy Minister, the Minister’s Office, Parliament and Central Agencies. According to key informant interviews, the expert advice provided by these directorates may not be fully taken into consideration when making decisions. For example, concern was expressed that some decision makers prefer to rely on their own knowledge and disregard the advice of analysts whose expertise align with Treasury Board and Privy Council Office guidelines. This creates a tension between varying programs and internal services within the Department. On the one hand, this could be construed as an issue of compliance as some decision makers may attempt to ignore departmental governance structures to meet the goals and objectives of their programs; on the other, it could be viewed as a problem inherent in some governance structures, as they may not always see a legitimate role for such analysts. Key informants have indicated that decision makers tend to rely on personal relationships and informal modes of communication to access the information needed to make decisions, including informal conversations, emails or telephone correspondence. These types of activities are observed at all levels of government and in the literature. For example, in his study on municipal governments, Katopol observed that employees made phone calls and used personal contacts to

9 Treasury Board Secretariat, 2009, Standard on Geospatial Data. Available at: http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=16553&section=text.

13

obtain or determine the location of required information.10 It should be noted that reliance on personal contacts is not indicative that AANDC’s information management systems are entirely insufficient to respond to the needs of decision makers. Rather, informal communication and personal relationships are viewed as expedient and reliable sources of information. 3.1.3 Are they able to access the types of information required? Key informants and literature sources pointed out several challenges that restrict access to the types of information required for decision making. These can be categorized into four issue areas: IT speed and integration; poor information management; data entry omissions; and privacy and security restrictions. Slow and Partially Integrated IT Systems User satisfaction is strongly correlated with the use and success of IT systems. Many key informants acknowledged they were not satisfied with CIDM and had mixed feelings about the use of this system to access pertinent information for decision making. Some key informants expressed no concerns with CIDM, noting that if they wanted information they simply extracted it from the database. This ease with extracting data from CIDM may be attributed to the fact that managers and senior executives do not use this database directly. Thus, they may not have any problems accessing data because it has been provided by their staff. These employees must search CIDM for the needed information being requested. Others, however, stressed that CIDM was inefficient and slow to access information. This could factor into why decision makers rely on personal relationships and informal modes of communication to access the information needed to make decisions. Kernaghan and Gunraj argue that IT is the driving force behind making information more usable and valuable.11 Key informants echoed this finding citing that a lack of a common IT platform was a challenge to accessibility of information. Some Financial Management informants suggested one of the primary elements impeding the access to information is the lack of a common dashboard to facilitate the extraction of complementary data from different systems. Management and Oversight key informants also noted challenges in accessing and synthesizing information for decision making when data is collected from different corporate systems that do not relate to one another. This can result in fragmentation where data may not be cross referenced in other systems, ultimately hindering access to comprehensive information. This will be expanded upon in Section 3.6 on dissemination of information.

10 Katopol, P.F, 2007, “Just enough” Information: Information behavior, organizational culture and decision making in municipal government. Canadian Journal of Information and Library Science. 31(3-4), p. 233. 11 Kernaghan, K., & Gunraj, J, 2004, “Integrating information technology into public administration: Conceptual and practical considerations.” Canadian Public Administration, pgs. 525-546.

14

Poor Information Management The classification of data makes it structured, which according to many key informants makes data more accessible. Numerous key informants noted there was difficulty accessing data due to employees not using appropriate classifications for metadata within repositories. Improper classification of metadata can exclude pertinent documents from file searches, thereby omitting key information and facts that could influence a decision. As a result, decision makers are not able to search for and retrieve information quickly and easily, thereby hindering their access to important and relevant data. Data Entry Omissions The literature on data-driven decision making emphasizes that information be inputted fully and accurately into a data warehouse or repository.12 Several key informants observed that data entry is often not complete as individuals do not fully populate the fields in the grants and contributions systems (First Nations and Inuit Transfer Payment / GCIMS). Circumstances and correspondence should be entered into the system; otherwise the numbers alone will lack appropriate context for decision makers. Powers13 argues managers can use systems that access these descriptive fields to provide current and historical data to support many decision tasks. If the tasks are performed regularly then a computerized decision support system can “potentially increase access to the data and help managers gain insights into organizational processes, stakeholder activities, employee performance and organization-wide performance metrics.”14 If information is not fully entered into a metadata repository, decision makers may have no clear and structured way of accessing it. As such, data warehouses and repositories should be frequently checked to ensure that important data is entered and maintained. Key informants corroborate this finding by suggesting that there needs to be a clean up of information within storage systems. If data entry errors exist or stagnant pools of data have been forgotten and not regularly updated, the data driven applications storage platforms could be worthless.15 Privacy Restrictions and Secured Information Many key informants acknowledged that they could not access information required for decision makers due to privacy restrictions. This was especially evident when attempting to make human resource decisions as certain privacy barriers are in place to protect the personal information of employees. The Business Decisions Support Team can build reporting systems to find the information needed but this takes time and requires training for employees. Privacy restrictions can also be put in place due to CIDM’s access control. This setting allows users to restrict access of their files to particular individuals, directorates, branches or sectors. Custom access control

12 Wheaton, Shorr, P, 2003, “10 Things You Always Wanted to Know About Data-Driven Decision Making.” Available at: http://www.scholastic.com/browse/article.jsp?id=423. Scholastic Administrator Magazine; Wilder Research, 2009, Data Entry and Analysis. Available at: http://www.evaluatod.org/resources/evaluation-guides/DataEntryAnalysis_2-09.pdf. 13 Power, Daniel, 2008, “Understanding Data Driven Decision Support Systems.” Information Systems Management 25(2), pgs.149-154. 14 Ibid. 15 Wheaton, “Data-Driven Decision Making.”

15

settings allow individuals to keep information private in CIDM that would otherwise be searchable and available. Other key informants commented that there was no secure location or repository for secret information. Management and Oversight informants especially struggle with this issue and expressed a need for a secure network to be created. This would allow individuals working with secret documents to access, share and collaborate on pertinent files. Due to security restrictions, access to key files entails inefficient sharing of physical or hard copies of documents. Again, this highlights the finding from above where decision makers communicate informally with employees as an expedient means to access needed secret information. This practice is mirrored in literature; one study suggests that obstacles originating from issues of privacy protection, complicated data management, including network security, and negative attitudes toward information sharing will be more critical barriers beyond the level of technical issues in regards to systems integration and consequently, information dissemination and decision making.16 Computer security is a balance between protecting information and enabling authorized access. The literature suggests that tightening security by making systems more inaccessible can hinder employee productivity.17 3.2 Does the type of information being collected align with government priorities, federal roles, jurisdiction and responsibilities? AANDC's mandate has a wide range of responsibilities that are shaped by centuries of history, and unique demographic and geographic challenges. Due to the dynamic role of the Department, AANDC collects a diversified range of data, both qualitative/quantitative and financial/non-financial, from a multitude of internal and external sources to meet its mandate. Literature suggests that continuously conducting environmental scans and updating information is important for effective decision making within an organization.18 AANDC conducts regular scanning. This is done in part through the Management and Oversight function, which plays a major role in ensuring that information provided to decision makers is in line with government priorities, federal roles and jurisdiction. For example, Cabinet Affairs provides a challenge and advisory role for programs to move their Memoranda to Cabinet for approval by decision makers.19[2] Cabinet Affairs assists programs in telling a comprehensive story to ensure that programs and projects meet the Department's mandated commitments, as well as those of the Government. The Management and Oversight function also facilitates information sharing, often through the coordination of various committees and presentations throughout the Department so that decisions can be made to address current issues in a comprehensive manner. Immediate priority

16 Ibid. 17 Post, G, 2007, “Evaluating information security tradeoffs: Restricting access can interfere with user tasks.” Computers & Security, pgs. 229-237. 18 Wei Choo, C., 2001, Environmental Scanning as Information Seeking and Organizational Learning. Available at: http://choo.fis.utoronto.ca/fis/respub/chooimreader.pdf. University of Toronto. 19 Cabinet Affairs Directorate, Aboriginal Affairs and Northern Development Canada, 2013, Overview of Critical Functions.

16

information needs frequently arise, resulting in ad hoc information generation. As such, the goal of streamlining information flow and engaging in more integrated strategic planning to inform decision making is slow. While no final decisions have been made yet, the discussion surrounding consolidation of Deputy Minister-chaired departmental committees for efficiency purposes is an example of progress toward this goal. As will be discussed at length in Section 3.4 of this report, much of the Department's expertise and knowledge arises from the use of informal networks. This leads to a wealth of tacit knowledge, which informants have identified as highly useful. However, this knowledge exists outside of formal networks, and there is therefore a risk that it may be lost as employee turnover occurs. As government priorities change, increased support for information management and technology systems can ensure the sustainability of AANDC's ability for change management. When priorities change, research inevitably occurs, and the more thoroughly information management has been done and the more user-friendly IT systems are, the more readily available previous research is to support future direction. The literature supports the notion that enhanced IM and IT can assist in change management and therefore should be integrated into departmental business strategies as a best practice. Alignment of IM and IT within broader priorities allows an organization to apply information resources to the most important business delivery tasks and operational activities.20 Specific details regarding the importance of IM and IT systems are discussed in sections 3.4, 3.6 and 3.7 of this report.

20Gregor, S., Hart, D., & Martin, N, 2007, “Enterprise Architectures: enablers of business strategy and IS/IT alignment in government. Information.” Technology & People, 20(2), pgs. 96-120.

17

4. Key Findings – Success/Effectiveness 4.1 How well does departmental information allow AANDC to achieve the outcomes identified in the Policy for Information Management and Policy on Management, Resources and Results Structure? Finding: Departmental information does not allow AANDC to fully achieve the outcomes identified in the Policy for Information Management. The Department faces risks to the relevance, reliability, accuracy and timeliness of information. Evidence shows the Department does not always manage its information and records as a valuable asset. While information technology platforms at AANDC are effective at collecting, storing and safeguarding information, this has all been done in silos. This presents a challenge for integrating information, and consistency of information. 4.1.1 Policy on Information Management The objectives of the Policy on Information Management are to achieve efficient and effective information management to support program and service delivery; foster informed decision making; facilitate accountability, transparency, and collaboration; and preserve and ensure access to information and records for the benefit of present and future generations.21 Its expected results are as follows: Government programs and services provide convenient access to relevant, reliable, comprehensive and timely information. Relevance The evaluation found that the Department often has too much information that is irrelevant to decision making. Key informants noted that the Department is generally unaware of the information it holds. This is not uncommon in many government departments in other jurisdictions. According to the Government of Hong Kong Efficiency Unit, the amount of information available to decision makers has increased exponentially,22 resulting in what Wurman23 describes as a “tsunami of data.” This can result in information overload because of the sheer volume and the unstructured, chaotic storage of data.24 As such, more information is not always useful and valuable to decision makers at AANDC.

21 Policy on Information Management. 22 Government of Hong Kong Efficiency Unit. (2010). Making Informed Decisions. 23 Wurman, R. S., 1997, Information Architecture. New York: Graphis Inc. 24 Government of Hong Kong Efficiency Unit. Making Informed Decisions.

18

There is an onus on decision makers to ensure that the information brought to bear on decision making is as relevant, complete, accurate and timely as possible.25 To mitigate this “tsunami of data”, key informants suggest employees engage in intelligent information gathering. This entails communicating with senior management and decision makers about what information they actually need rather than what information is available. This is especially important for AANDC, a department that is highly decentralized and is governed by a set of management practices that support a high degree of autonomy.26 With over 80 percent of its budget being allocated to Grants and Contributions (transfer payments),27 intelligent information gathering could significantly reduce reporting burden for numerous Aboriginal stakeholders, while simultaneously making departmental data mining and analysis more efficient and more relevant. Additionally, it should be noted that the Department employs statistical officers who work with program staff to ensure the information they gather using data collection instruments is in a format that is useable for statistical analysis. The evaluation found that there is a lack of awareness about their role in the Department and an inability to ensure that every data collection instrument is designed appropriately. Improving the data collection instruments and measuring what matters presents an opportunity to enhance the relevance of information collected. Reliability and Accuracy As was discussed in Section 3.1, the evaluation found information for decision making was sometimes hindered by data entry errors, which could make information unreliable or inaccurate. This was evidenced when several key informants pointed out instances where financial numbers were not consistent between the Chief Financial Officer Sector and programs. This can create disparate financial analyses and delays especially when writing important policy documents such as Memoranda to Cabinet and Treasury Board submissions. These accuracy problems are strongly linked to delays or errors in data entry. The literature emphasizes how crucial it is for data to be as accurate as possible so that statistical inferences are made correctly to indicate forecasting trends and other important knowledge.28 Inaccurate data limit the efficacy of Business Intelligence 29 platforms (software) to retrieve and manipulate information for reporting. Regardless of IT/IM innovation and investment, it cannot compensate for human error and inaccurate data entry. Data that are accessed by Business Intelligence applications must be accurate if managers of knowledge-based organizations are to make informed decisions.30

25 Ibid. 26 Aboriginal Affairs and Northern Development Canada. 2009. Chief Financial Officer Sector Management Practices Review. Available at: http://www.aadnc-aandc.gc.ca/eng/1333045321642/1333045366141. 27 Aboriginal Affairs and Northern Development Canada. 2013. Backgrounder – Aboriginal Affairs and Northern Development Canada’s Financial Picture (2013-2014). Available at: http://www.aadnc-aandc.gc.ca/eng/1363029324938/1363029382616. 28 Hedgebeth, Darius, 2007, “Data-driven decision making for the enterprise: an overview of business intelligence applications”. VINE, 37(4), pgs. 414-420. 29 Business Intelligence is defined as an IT application that helps organizations make decisions by using technology for reporting and data access, as well as analytical applications. 30 Hedgebeth, “Data-driven decision making for the enterprise.”

19

The literature suggests some best practices that can mitigate these errors such as increasing quality assurance mechanisms for data entry. These include: checking data (especially financial data) for discrepancies; encrypting information and ensuring it is not able to be easily changed or manipulated; automatic recovery and data saving mechanisms; and employee training on the importance of data entry accuracy.31 Improving data accuracy can allow the successful use of Business Intelligence applications, which aid knowledge-based organizations to provide valuable decision-making knowledge to minimize operating costs.32 Timeliness Many key informants expressed concern about the misclassification of files, which can contribute to a lack of timeliness of information. As noted previously, decision makers are often aware of the information required to make a decision. However, if they know certain information exists but cannot retrieve it easily, much time is spent trying to locate pertinent files. If information cannot be retrieved, the work may have to be duplicated to compensate for the loss. This was observed previously in the Annual Report on the State of Performance Measurement in Aboriginal Affairs and Northern Development Canada for 2011-2012. This report indicated that there was a duplication of information and/or multiple requests for the same information by various stakeholders in the Department.33 As such, this evaluation found that convenient access to timely information is hindered within the Department. Information and records are managed as valuable assets The Treasury Board Directive on Recordkeeping supports the Policy on Information Management by mandating that all departmental information must be stored in a repository.34 Repositories are usually filled with metadata, which is pertinent information about the specific files being stored. This can include: file numbers, file type (notes, reports, decisions, and correspondence), file title, file author, file date, and security classification. There are specific rules regarding the classification of metadata with the goal of improving accessibility and use of important information. 35 For example: file numbers in CIDM are linked to the Department’s Program Alignment Architecture, which can in theory provide decision makers with access to information on specific program areas pertinent to the Department’s mandate and objectives.

31 Wilder Research, 2009, Data entry and analysis: Evaluation resources from Wilder Research. Saint Paul: Minnesota: Wilder Foundation; MEASURE Evaluation, 2008, Data Demand and Information Use in the Health Sector: Case Study Series. Available at: http://www.cpc.unc.edu/measure/publications/sr-08-44. Chapel Hill, NC: MEASURE Evaluation. 32 Hedgebeth, “Data-driven decision making for the enterprise.” 33 Aboriginal Affairs and Northern Development Canada. 2013, Annual Report on the State of Performance Measurements in Aboriginal Affairs and Northern Development Canada for 2011-2012. 34 TBS Directive on Recordkeeping. 35 Treasury Board of Canada Secretariat, 2010, Standard on Metadata. Available at: http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=18909.

20

According to key informants, some employees avoid using proper information management practices all together. This avoidance could be the result of perceived difficulty or lack of understanding of IM/IT systems. Based on findings outlined in Section 3.1, CIDM is not considered to be an efficient database. Yang and Maxwell observe that within knowledge-based organizations, if information technology is not easy and efficient to use, employee usage of systems will be lower.36 As such, existing systems are not being used to their full operational capacity. The Department’s official IT/IM Strategy37 seeks to mitigate risk of improper management of information by aiming to instill a culture that values information management at AANDC. If such a culture is not fostered, data can be lost and of no use for decision making. Although key informants comment that information management is the responsibility of all employees, they have noted as well that instilling this culture must come from management. This is a requirement outlined in AANDC’s Enterprise Information Management Strategy 2010-2015, which stipulates that managers apply IM discipline in their own workplace as a good example to their employees.38 Governance structures, mechanisms and resources are in place to ensure continuous and effective management of information Current government-wide policies and directives regarding information technology, such as the policy on management of information technology39, emphasize the importance of horizontality. For example, documents mention the need for: information exchange through security practices, increased common and shared services within departments, a government-wide approach to the management of information technology, whole of government approaches to information integration and strategic directions.40 Findings from key informant interviews emphasize that horizontality is beneficial for numerous reasons, namely to: fill in gaps in information, generate new ideas, ensure consistency and transparency of information, and to streamline systems for efficiency. One key informant noted that once horizontality is achieved, the quality of information going up the line to decision makers is higher because everyone is using consistent information. The traditional “silo” design of departments runs strongly counter to the current emphasis found on whole-of-government approaches to information collection, storage, retrieval and sharing. A long history of working in silos under strict accountability requirements has created a culture of tunnel vision rather than the peripheral vision needed for horizontal government. This evaluation found that while information technology platforms at AANDC are effective at collecting, storing and safeguarding information, this has all been done in silos. This presents a challenge for integrating information, and consistency of information. 36 Yang, T., and T. Maxwell, 2011, “Information Sharing in Public Organizations: A Literature Review of Interpersonal, Intra-Organizational and Inter-Organizational Success Factors.” Government Information Quarterly, 28(2), pgs. 164-175. 37 Aboriginal Affairs and Northern Development Canada. Enterprise Information Management Strategy 2010-2015. 38 Ibid. 39 TBS Policy on Information Management. 40Ibid; TBS Standard on Geospatial Data; TBS Standard on Metadata.

21

Despite this obstacle, the evaluation also uncovered several strategies to promote horizontality; for one, information technology can help drive this transition. Within AANDC, two interviewees noted that the software Collaboration fosters internal horizontality by allowing individuals to share information and comment on joint files. Others commented that GCIMS (a new version of First Nations and Inuit Transfer Payment) improved information sharing by allowing stakeholders and recipients to input information. One notable potential tool for increased horizontality is GCdocs, which is expected to help bring down silos by creating a single library, auto-classification, a more powerful search engine and defaults to share information more widely. Additionally, the literature stresses the need for a strong enterprise architecture, which is a methodology for an information management strategy within an organization. Enterprise architecture is meant to facilitate communication between diverse groups and interests to provide a common ground for discussion and decision making. The literature also notes the need for an IM strategy and guidelines to facilitate unstructured information sharing through online community of interest.41 Consistency was an issue noted for the sub-activity of Management and Oversight in particular, as they are often responsible for ensuring that information, and in particular financial numbers, are consistent and correct when they are being sent to decision makers. For example, one interviewee wanted to know how much was spent on a particular program, and it took them a month to come up with a number. Another key informant commented there are habitual inconsistencies for budgets and expenditures between Chief Financial Officer and programs. 4.1.2 Policy on Management, Resources and Results Structures The objective of the Policy on MRRS is to ensure that the Government and Parliament receive integrated financial and non-financial program performance information for use to support improved allocation and reallocation decisions in individual departments and across the Government.42 This policy includes various elements such as: strategic outcomes, Program Alignment Architecture and performance measurement framework. These elements inform other results based accountability reports such as the Report on Plans and Priorities, Departmental Performance Report, and the Management Accountability Framework assessments. Notably, the policy mandates AANDC’s information systems and performance measurement be aligned with the MRRS.

41 Home Civil Service UK, 2008, Information Matters: Building Government’s Capability in Managing Knowledge and Information. Available at: http://www.nationalarchives.gov.uk/documents/information-management/information-matters-strategy.pdf. The National Archives- Government of the United Kingdom; Caverlee, James, Liu, Ling, and Steve Webb, 2010, “Social Trust Framework for Trusted Social Information Management: Architecture and Algorithms.” Information Sciences 180: 95-112. 42TBS Policy on MRRS.

22

Performance Alignment Architecture The Program Alignment Architecture is an architecture consisting of a structured inventory of all programs being delivered by a department.43 It allows the Department to identify and define its programs; structure its performance story; understand the relationships between programs and expected results to achieve the strategic outcomes of the Department; and structure internal analyses to allocate and reallocate resources within the organization. According to the Policy on MRRS, each program of the Program Alignment Architecture, at each level, must have planned and actual information on resources and results.44 This is not the case for internal services at AANDC, which are included under a general rubric of “Internal Services” within the Program Alignment Architecture.

AANDC has enumerated strategic outcomes (governance and management support, resource management services, and asset management services) for internal services in prior Program Alignment Architectures.45 These strategic outcomes include a variety of sub-activities such as management and oversight, financial management, information technology and information management. In AANDC’s Program Alignment Architecture 2013-2014, no strategic outcomes are enumerated for internal services. This will most likely be the case for the 2014-2015 Program Alignment Architecture as well. Nor are there any references made within previous or current performance measurement frameworks indicating performance indicators for internal services according to their strategic outcomes and activities. As such, employees may not be aware of the encompassing and important nature of internal services within AANDC and how to benchmark performance of these service activities to enhance departmental operational efficiency and decision making. This was corroborated by some key informants who expressed concern that IM and IT services did not operationalize or have clear measurement/benchmarks for successful and effective IT/IM systems. As an example, while CIDM is considered to be a major accomplishment by designers, as it allows for version control of documents and proper storage based on the Program Alignment Architecture, anecdotally users of CIDM disagree as to its effectiveness. The lack of a proper benchmark for the quality of internal services speaks to the importance of developing performance measurement in this area.

Integrated Financial and Non-Financial Information The MRRS states that the Government has the objective of increasingly integrated financial and non-financial program performance information for reporting to Parliament and other decision makers. 46 Furthermore, the Policy Framework for Information and Technology states that efficient integration is achieved when business, information, and technology are reliable and interoperable. This means standardizing information sources of common data, reusing information and technology solutions to eliminate duplication and redundancy, maximizing investments, and minimizing costs.47 Meeting these goals may require integration of AANDC’s 43 Treasury Board Secretariat, 2010, Program Alignment Architecture. 44 Ibid. 45 Aboriginal Affairs and Northern Development Canada. 2012. Appendix A: AANDC Program Activity Architecture. Available at: http://www.aadnc-aandc.gc.ca/eng/1345558981352/1345559176644. 46 TBS Policy on MRRS. 47 Treasury Board Secretariat, 2007, Policy Framework for Information and Technology. Available at: http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=12452.

23

IT systems, a discussion which is expanded upon in Section 3.7 of this report regarding dissemination of information. Review of Policy on MRRS and Management Accountability Framework Currently, the Policy on MRRS is undergoing a five year review. This is an attempt to address the state of the policy across departments, how it is being implemented government-wide, its lack of stability (many departments, including AANDC, have submitted major amendments to the Program Alignment Architecture structure for 2014-2015), and the management of appropriations or parliamentary voting through the MRRS structures.

Coinciding with the above review, Treasury Board Secretariat has been in consultation with the Public Service Management Advisory Committee and Director Generals from varying departments (including AANDC). The purpose of the consultations was to provide a status update on Management Accountability Framework 2.0 initiative; outline objectives for a new core area of management on integrated risk, planning and performance; seek input on current state of integrated risk, and seek suggestions for an approach to methodology for 2014-2015. The result is the creation of a new updated Management Accountability Framework that improves alignment with Treasury Board Secretariat policies and management expectations, as well as four approved core areas of management and five department-specific areas of management that will be assessed over the next three years. The new Management Accountability Framework includes a new key element integral to internal services and information for decision making, namely information management. With this in mind, it would be prudent to complete an analysis of AANDC’s adherence to the Policy on MRRS in regards to information for decision making at a future date once reviews and updates have been concluded and clear guidance has been provided by Treasury Board Secretariat. As such, an evaluation of AANDC’s adherence to the current Policy on MRRS in reference to information for decision making was not undertaken for this evaluation. 4.2 What are the key factors helping or impeding the availability, credibility and reliability of information for decision making? Finding: While AANDC’s document management solution and use of informal networks can help information for decision making, these informal networks, information management culture, human capacity and system capacity may all present threats to information being available, credible and reliable.

The current review of AANDC’s adherence to the MRRS provides an opportunity for future analysis of Information for Decision Making

24

Document Management Solution Relative to other departments, AANDC has a reliable document management system to ensure information is systematically stored and accessible. For example, CIDM ensures as much as possible through version control that documents are not duplicated; Groupwise and First Nations and Inuit Transfer Payment are integrated into CIDM to strengthen this process. This means that relative to other departments, AANDC is an effective and efficient manager of information. Despite this, challenges with CIDM exist such as slow response times, time-consuming metadata entry requirements, access control defaults that limit information sharing and lack of a proper search engine. Informal Networks The use of informal and personal networks are an important part of the discussion on key aiding factors as they pertain to information for decision making. Key informants suggest that the use of informal networks such as personal networks and informal meetings are heavily relied upon as a means to gather information for decisions, amongst senior management especially. There are a number of positive and negative implications of this finding. Use of informal networks can increase the availability of information in the short run, making access easier in the context of the challenges with formal information management structures discussed in sections 3.1 and 3.3. Key informants suggest that within the immediate context of a given unit at AANDC, informal networks can enhance the efficiency of decision making. Specifically, information seekers utilize informal networks because asking the right questions to the right people results in expedient access to the desired information. For time sensitive decisions, these information gathering techniques are perceived to be the most effective. The key informant interviews additionally suggest that individual expertise plays a large role in being able to use these information networks effectively. The literature indicates that the use of personal networks can result in more effective and cooperative sharing processes and communication across departments in a cross-functional matter as discussed previously, but also warns that it should not be relied upon exclusively.48 Informal networks can be a challenge to the availability of information in the long-run as a result of employee turnover. When networks break down over time, retaining knowledge is much more difficult than in a formal information system. Additionally, informal networks restrict other units in the Department from benefitting from the knowledge obtained. While informal networks can be efficient in the immediate term, the unintended impacts around the lack of standardization and classification of information can be a hindrance to decision making in the context of the broader department.

48 AANDC Program Activity Architecture.

25

The Treasury Board Directives on IM Roles and Responsibilities49 outline that governance structures are required to facilitate IM accountability. Further, the directive indicates that all employees have a responsibility to ensure proper IM procedures are followed. Key informant interviews indicate that there is a need for top-down pressure to sustain proper IM culture within the Department, and that senior level management appears to heavily utilize informal networks as a way to gather information for decision making, often circumventing the information storage systems that are in place and consequently shifting away from proper IM practices. As discussed in sections 3.1 and 3.3, key informants expressed concern about the extent to which proper IM procedures are followed and the resulting difficulty and time spent on finding stored information. The literature reviewed suggests that information system usage is often lower if the systems are not easy and efficient to use.50 Several key informants indicated that they opted to use informal networks rather than information systems due to both a lack of understanding the systems, and the perception that they are too difficult to use to gather the information being sought out. The literature states that “IT should not focus on creating a dream application, from a techie perspective. IT must focus on “user satisfaction”. All systems within an organization must focus on the customer, and for IT, their customer is the end user of their applications.”51 If a system is not designed from a user perspective, to be “user friendly”, it will result in individuals poorly managing data. Lower usages of information systems can negatively influence information retrieval and sharing, as it can result in limited amounts of structured data stored within the system, which can be detrimental to institutional memory in the long term.52 In summary, while the use of informal networks can foster communication and information in the short term, it is unlikely to be sustainable in the long term. Evidence from the literature reviewed suggests that accessing information via a personal network is not replicable by anyone outside of the network.53 Many key informants additionally indicated that communications are one-way and information seeking, rather than two-way open information sharing. Consequently, these communications are largely transactional in nature, which could be difficult to sustain over the long term. The literature also suggests that the reliance on specific personal networks to obtain information can reduce the chance of encountering new and potentially better information sources.54 This can result in a lack of comprehensive information, limiting the ability to make a fully-informed decision.

49 Treasury Board Secretariat, 2007, Directive on Information Management Roles and Responsibilities. Available at: http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=12754. 50 Yang and Maxwell, “Information Sharing in Public Organizations,” Pgs 164-175. 51 Plenert, G., 2013, Lean Management Principles for Information Technology . CRC Press. P.17. 52 Ibid., p.166. 53 Katopol, "Just enough" Information,” p.243. 54 Ibid., p. 233.

Formal systems for information capture are most effective when they are a result of collaboration between IM/IT experts and system users.

26

Tacit Knowledge Informal networks can lead to tacit knowledge, which is often not captured by formal IM and IT systems. Both the literature and key informants stress the importance of tacit knowledge as key to having the capacity to analyze data and support decision making. Tacit knowledge is described as the skills, experiences, insight, intuition and judgment held by individuals.55 It is also a collection of facts, concepts, experiences and knowledge held by a group of people and is strongly correlated with institutional memory. Both Bennett56 and Khatri and Ng57 observe that tacit knowledge is particularly valuable to upper-level managers, especially when they engage in strategic decision making. Key informant interviews confirm this finding, especially those engaged in the Management and Oversight sub-activity functions. Many Management and Oversight key informants noted that structural barriers to their capacity to analyze were overcome through the knowledge and expertise gained in their positions, which allowed them to acquire the appropriate information required to make informed decisions. Over time, tacit knowledge transcends individuals and requires the ongoing transmission of memories between members of the group.58 This ongoing transmission and exchange of knowledge results in a synergistic learning process that is difficult to capture.59 Literature pertinent to IM recognizes the tacit nature of knowledge and analysis. Davenport60 observes that analysis tends to come primarily from structured data but the content of the analysis itself is disseminated informally. Some key informants working within the Financial Management and Management and Oversight activities expressed concern about the risks associated with increased reliance on institutional memory for information for decision making, especially when individuals leave the Department. Information Management Culture As discussed in Section 3.3, the consistent creation, capture and use of metadata will likely contribute to the objectives of the Policy on Information Management and the Directive on Recordkeeping. However, at an individual level, IM guidelines are often not followed. This includes not transferring paper documents into electronic records and not using proper naming conventions and other metadata necessary to organize and retrieve information, resulting in lost time and information relevant to decision making.

55 Government of Hong Kong Efficiency Unit. Making Informed Decisions. 56 Bennett, R. H., 1998, “The Importance of Tacit Knowledge in Strategic Deliberations and Decisions”. Management Decision, 36(9), 589-597. 57 Khatri, Naresh and Ng, H. Alvin, 2000, “The Role of Intuition and Strategic Decision Making”. Human Relations, 53(1), 57-86. 58 Fornaro, J. G. (2008). “Institutional Memory: How good is your club’s memory? Really!!”. The Boardroom, January/February. 59 Bennett, “The Importance of Tacit Knowledge”; Hilbert, D.M., Billsus, D. and Denoue, L., 2006, Seamless Capture and Discovery for Corporate Memory. 60 Davenport, T. H., 1994, “Saving IT’s Soul: Human-Centered Information Management.” Harvard Business Review, March-April, pgs. 119-131.

27

A major operational obstacle is the lack of inter-operability, in the sense of differences across departments and governments in laws, policies, rules, regulations and standards.61 Unwillingness to share information among organizations may become a severe hindrance to those actively seeking information and resources to make policy.62 With performance-based reward systems, organizational members are more likely to share information and knowledge, unless the system is not specifically designed for encouraging information sharing.63

61 TBS Standard on Metadata. 62 Myeong, S., & Choi, Y., 2010, “Effects of Information Technology on Policy Decision-Making Processes: Some Evidences Beyond Rhetoric.” Administration & Society 42(4), pgs. 441-459. 63 Yang and Maxwell, “Information Sharing in Public Organizations,” pgs. 164-175.

28

5. Key Findings – Efficiency/Economy 5.1 Does the Department have the capacity to analyze its data? To what extent is this capacity actualized? How is the analysis of data used? Finding: The main constraint on AANDC's capacity to analyze its data is the context of tight timelines in which accurate and comprehensive information is required by decision makers. Strategies such as time management training and securing a supportive IM/IT base could provide realistic measures to foster the advisory function that is vital to decision makers. Time Constraints Key informants felt their capacity to analyze data was affected by narrow timelines. Individual employees often have a variety of competing priorities with inadequate opportunity to thoroughly complete required tasks. This is not uncommon in other public sector organizations.64 The literature as well as interviews suggest that a lack of time to analyze, synthesize, and interpret data hinders the credibility of decision making by reducing the usability of important information.65 Conversely, allocating valuable time (i.e., common planning time) or creating new structures (i.e., working groups) enables individuals to collectively interpret needs and decide what actions to pursue.66 Some key informants noted in this respect that they could not develop in-house expertise or build capacity because they had little time to invest in analyzing how to creatively meet their IT/IM needs. One key informant mentioned that there was an executive committee called the Forum on Strategic Issues, which enabled time and cost savings by having executives discuss the feasibility of programs/policies at a preliminary level prior to sectors spending time and resources on conducting multiple unnecessary reports and data. Particularly within the Management and Oversight function, key informants suggest that there is a greater need to allocate valuable time toward analysis and capacity-building with the aim of enabling decision makers to have a complete picture of the best course of action and consequently support credible decision making.

64 Katopol, “Just enough” Information,”: p. 233. 65 Feldman, J., & Tung, R., 2001, “Using Data-based Inquiry and Decision Making to Improve Instruction.” ERS Spectrum 19(3), 10–19; Ingram, D., Louis, K.S., & Schroeder, R.G., 2004, “Accountability policies and teacher decision making: Barriers to the use of data to improve practice.” Teachers College Record, 106(6), 1258–1287. 66 Lachat, M.A., 2001, Data-driven high school reform: The Breaking Ranks model. Providence, RI: LAB at Brown University.

29

Tight timelines can be exacerbated by strict regulatory context. The literature is explicit in arguing that tacit knowledge should be complimented by sound data analysis and formal insight.67 Several Financial Management key informants said they did not have sufficient capacity to analyze their data. This is not uncommon in most large organizations within the private sector. In their 2005 study, IBM Institute for Report for Business Value observed that almost half (45 percent) of chief financial officers interviewed saw creating high calibre insight and analysis for reporting as one of the top three critical issues facing chief financial officers and the finance function. Moreover, the IBM study posits limited financial analysis is partly due to regulatory compliance, transaction processing, and the rapid pace of change. There are an abundant number of Treasury Board Secretariat documents pertinent to Financial Management, which has strict compliance requirements regarding the collection and dissemination of accurate financial information. As such, some key informants noted they spend more time collecting information and ensuring accuracy in compliance, which resulted in limited analysis and use of the data generated. Strategies for Mitigation As some key informants expressed concern about having adequate employee-hours to analyze data, several strategies have been identified that may enhance the employee-hours currently available. For example, key informants discussed the importance of training to cope with timelines. Time management and stress management seminars may serve to mitigate the pressure placed on some analysts at AANDC by the nature of the timelines placed on analysis for decision making. These mitigation strategies are recommended in private sector business literature on managing with reduced resources.68 Additionally, the burden on analysts can be eased with effective IM processes and IT systems, which enable analysis by providing key information. Identifying risks and mitigation strategies in IM and IT has been addressed in sections 3.1, 3.3, 3.6, and 3.7 of this report. The greater the availability of accurate and relevant information, the easier the analysis can be. Finally, another potentially promising strategy for mitigating the increasing burden on analysts in the Department is a new practice known as ‘crowd sourcing.’ Crowd sourcing refers to disseminating an organization’s information on the internet to engage multiple groups with

67 Bennett, “The Importance of Tacit Knowledge,” 589-597; Khatri and Ng, “The Role of Intuition and Strategic Decision Making,” pgs. 57-86.; Pondy, L.R., 1983, “The union of rationality and intuition in management action. Introduction: common themes in executive thought and action.” In S. Srivastava and associates (Eds), The Executive Mind. San Francisco, CA: Jossey-Bass.; Simon, H.A., 1987, “Making Management Decisions: The Role of Intuition and Emotion.” Academy of Management Executive, February, pgs. 57–64. 68 Herbert, Mark, n.d., “Executing an Effective Workforce Reduction.” Available at http://www.newparadigmsllc.com/PDFs/WorkforceReduction.pdf. New Paradigms LLC, 8 p; Green, Peter and Denise Skinner, 2005. “Does time management training work? An evaluation,” International Journal of Training and Development 9:2, p. 124.

A combination of time and stress management support for in-house analysts, and crowd-sourcing to obtain outside perspectives can foster enhanced analysis capacity at AANDC without excessive additional resources.

30

varied expertise.69 Methods in this practice are still being refined to ensure the proper groups are targeted and the desired analysis is achieved, and commitment to it at AANDC would likely require moderate training and IT adjustments.70 However, with the potential to engage analysts from other jurisdictions, Aboriginal groups, universities and elsewhere at once and over the internet, it could prove promising for easing AANDC’s analysis burden. How Analysis is Used Ensuring barriers to analysis of information are reduced is vital to the decision-making process at AANDC. Analysts are required to sort through a vast body of information for streamlining purposes so that decision makers have the relevant information required. As discussed in the section detailing the types of information accessed by decision makers, tools such as GCIMS are available to provide data-driven reports for the purposes of financial decisions. However, a human analysis component is required to integrate financial analysis with qualitative context and relevance. Analysts in the Management and Oversight function are tasked with packaging this information in a way that is ultimately relevant as advice to decision makers. Therefore, the analysis function is vital to departmental decisions. 5.2 Is information disseminated sufficiently and appropriately throughout the Department? Finding: AANDC faces significant technological, human and regulation-driven barriers to dissemination of information. The accessibility of information for dissemination is hindered by information silos, which prevent efficient sharing throughout government. These structural obstacles are also referred to as the problem of departmentalism.71 The traditional “silo” design of departments runs strongly counter to a shift in government toward increased horizontality. Governance structures favour departmental projects, not interdepartmental partnerships. A lack of cooperation between departments has a high opportunity cost in wasted time and money as it can result in duplication of data that already exists.72 At AANDC, IT is effective at collecting, storing and safeguarding information, but it has all been done in silos. Data sharing agreements exist, but they are often labour intensive as they involve concerns over privacy, security, as well as accountability and the delineation of responsibilities between parties, ultimately creating few incentives toward information sharing and dissemination.

69 Pederson et. Al, 2013, “Conceptual Foundations of Crowdsourcing: A Review of IS Research.” 46th Hawaii International Conference on Systems Science, p. 579. 70 Psaier et. Al, 2011, “Resource and Agreement Management in Dynamic Crowdcomputing Environments.” 15th Annual Enterprise Distributed Object Computing Conference, 193. 71 Kernaghan and Gunraj, “Integrating information technology into public administration,”pgs. 525-546. 72 Plenert, Lean Management Principles for Information Technology, pgs. 123-139.

31

Information Technology Integration The literature suggests that one of the most efficient means of disseminating information for retrieval and analysis purposes is the use of information and communication technologies that promote efficiencies, improve productive and administrative processes, and assist in the generation of new information to be shared and integrated. For this reason, in the international information management and information technology community, the concept of centralized and integrated ‘open data warehouses’ has been strongly encouraged. 73 The AANDC Corporate Risk Profile identified the lack of a centralized or integrated data warehouse as a risk to duplication of data.74 Similarly, key informants suggest that there is actually sometimes too much information being disseminated and a lack of adequate integration of this information for reporting purposes. Another key informant discussed situations where resources are allocated toward obtaining legal opinions on questions that have already been answered by previous legal experts. If there was a means to obtain integrated legal advice, duplication and its associated costs would likely decrease. In other cases, it may be that there are systems like Oracle and First Nations and Inuit Transfer Payment, but no integrated database warehouse to ensure that the systems ‘speak to one another’. There are currently 78 applications in the Department and no common platform. Other key informants, particularly at the program level, also identified a need for greater IT integration to enable them to efficiently disseminate the right kinds of financial and non-financial information and ensure the consistency of this information. One key informant indicated that at present, the Department’s inability to ensure the consistency of some of its financial information in lieu of tight deadlines and day-to-day demands poses a large credibility risk to information for decision making. Increasingly, data held in data warehouses throughout government (such as GCdocs, currently being implemented across the Government) can be cross-matched with other data to produce valuable new information for policy making and program delivery. Nevertheless, there are significant barriers throughout government, and particularly throughout AANDC, in the integration of IT systems toward dissemination of sufficient and appropriate information for decision making. Hesitancy to Share Long experience working in silos under strict accountability requirements creates a culture of tunnel vision rather than the peripheral vision needed for horizontal government and effective dissemination of information.75 As a consequence, this climate within the Department has contributed toward building a reactive decision-making culture characterized by risk aversion,

73 Yang and Maxwell, “Information Sharing in Public Organizations,” pg. 1. 74 Aboriginal Affairs and Northern Development Canada, Corporate Risk profile 2012. 75 Ibid.

A department-wide discussion surrounding privacy, IT needs and employee habits for enhanced information sharing could result in reduced search time and duplication of research efforts.

32

where adoption of new technologies and comfort with organizational change develop slowly. Furthermore, the individual responsibility of ministers for the conduct of their departments encourages them to focus on the vertical dimension of government rather than horizontality,76 which could reduce opportunities to contribute to decision making around larger Government of Canada objectives. Key informants also suggest that within AANDC, a large factor contributing to a risk adverse culture and a lack of integration and systems capacity building is not only restricted time and resources to do so, but the simultaneous weighing of security and privacy concerns. 5.3 How effective and efficient is storage and access to information in the Department? Finding: Effective and efficient information storage and access is a fine balance within AANDC that is not currently conducive to timely, efficient, reliable and credible information for decision making. This results from both technologically - and human-driven causes. Disposition: Several key informants discussed the importance of document disposition after they are no longer needed; it was identified as a risk to proper information for decision making, as too much information makes relevant information difficult to identify. Part of the difficulty with disposition of information and documents is that the Library and Archives Canada Act and Treasury Board Directive on Record keeping place restrictions on the disposition of information deemed to be of business value.77 However, because of the historical and legal significance of certain documentation, as a precautionary measure AANDC is forced to hold onto its entire documentation inventory. "Intelligent information gathering" was mentioned as a possible solution to mitigate disposition issues. Essentially, this entails determining what is of business value, digitizing it, and disposing of the rest to make the repository lean and efficient. Informants suggest that the Department has to find a balance between maintaining important historical and legal records and agreements and being realistic about its storage needs. In the future, GCdocs may provide a solution to this problem so long as it procures an effective search engine which, as was discussed in Section 3.1, would inform on what information is already available. The transition to GCdocs will include compliance with government-wide standardized tools and technology in its data-warehouse repository. To preserve institutional memory, recording keeping must be in place to keep track of past decisions and choosing future priorities.

76 Kernaghan and Gunraj, “Integrating information technology into public administration,” pgs. 525-546. 77 Department of Justice, 2004; Library and Archives of Canada Act. Available at: http://laws-lois.justice.gc.ca/eng/acts/L-7.7/; Treasury Board of Canada Secretariat, 2009, Directive on Recordkeeping. Available at: http://www.tbs-sct.gc.ca/pol/index-eng.aspx?tree=directive.

33

Design and Delivery: Key informants consistently took issue with the design of CIDM and other information storage systems within the Department. Most key informants found CIDM to be cumbersome, slow, and difficult to use, and indicated many staff often rely more on informal networks and personal hard drives. Yet as the literature review suggests, successful information management strategies require due consultation with end-users.78 In fact, key informants outside of the Information Management Branch commented on the desire for increased consultation when creating data collection instruments, data repositories and data analysis platforms. This is reflective of the literature referenced in Section 3.6 discussing integration of IT systems. Interestingly, key informants from the IM/IT sectors explained that IM and IT specialists have expertise and best practices at hand that have the potential to make storage and collection tools more efficient and effective.79 In order to combat future storage and accessibility challenges, taking a more proactive and comprehensive approach to IM and IT was discussed, as opposed to patching problems with IT solutions exclusively. This would include comprehensive and integrated IT systems design that reflects both user needs and designer expertise. Key informants addressed two particular challenges in the storage and access of information, in effect pointing out the importance of turning unstructured data into structured, and thus, accessible, data. One key informant stressed the need for an Access to Information and Privacy (ATIP) database, to be used for ATIP requests. Two key informants commented on CIDM’s inability to store secret-level documents. However, the Enterprise Information and Records Management Sector have released step-by-step directions for the Department, called “Saving Protected C and Above Documents in CIDM.”80 The lack of knowledge of existing procedures and directives potentially highlights a disconnect between the awareness promoted by the Information Management Branch and its uptake elsewhere at AANDC. Nonetheless, both of these challenges address the need for document systems that allow information to be retrieved in an effective manner. 5.4 To what degree does departmental governance facilitate open lines of communication? Finding: The current approach to departmental governance has mixed results in facilitating open lines of communication. Committee Structure Some key informants were very satisfied with the function of committees and their role in facilitating open lines of communication, saying they provide an opportunity for questions to be answered, and for recorded follow-up to those questions. The Community of Practice on Indian and Northern Affairs Canada’s Deputy Minister-Chaired Decision-Making Committees (2011)

78 Plenert, Lean Management Principles for Information Technology, Chpt 3. 79 AANDC Enterprise Information Management Strategy 2010-2015. 80 Information Management Branch, Aboriginal Affairs and Northern Development Canada, 2012, Saving Protected C and Above Documents in CIDM.

34

notes that committees are meant to build working relationships and linkages between secretariats, share best practices and lessons learned among secretariats, and discuss improvements to the governance structure. Open lines of communication are important for AANDC because of its multifaceted responsibilities and mandate that is derived from multiple disciplines such as health, education and economics. Accordingly, the Community of Practice for AANDC's Deputy Minister Chaired Committees is designed to support AANDC internal decision making and corporate management by building relations and linkages between secretariats, sharing best practices and lessons learned to improve Deputy Minister-chaired committee operations, and discussing possible improvements to the committee governance structure such as increasing administrative efficiency.81 This design is encouraged by a body of literature, which suggests diverse teams generally offer multiple decision options, making the group more effective in regards to information for decision making.82 However, it should be noted that one author warns that “cross-functional communication requirements could result in a lengthy service durations, inaccurate information, and poor scheduling.83 Reflective of this, some key informants did draw attention to areas for improvement. For example, streamlining committees to provide an integrated approach and continuity of discussion was suggested, as was a stronger focus on strategic discussions at committees. This is addressed in part by the Community of Practice on AANDC’s Committee Governance Structure (2013), which states two key objectives on committee governance reform: 1) the integration and streamlining the current committee governance structure; and 2) better strategic use of executive presence at committee meetings. Reactive versus Proactive Governance Several key informants suggested that the governance of the Department appears to be largely reactive rather than proactive. A reactive approach results in much of the information accessed by decision makers being short-term in nature and sought out to answer a specific and often time-sensitive need.84 Several key informants corroborated this, and indicated that being reactionary limits the sectors’ ability for forward thinking in terms of information gathering and decision making. Several financial management key informants specifically mentioned that crisis and emergency management is prioritized, and as a result funding is diverted from potential investments in more proactive, strategic approaches.

81 Aboriginal Affairs and Northern Development Canada, 2013, Community of Practise on AANDC's Committee Governance Structure (2013). "The Way Forward: Options for Consideration.” 82 Athanasawa, Y., 2003, “Team Characteristics and Team Member Knowledge, Skills, and Ability Relationships to the Effectiveness of Cross-Functional Teams in the Public Sector.” International Journal of Public Administration, 26 (10/11), pgs. 1165-1203. 83Malayeff, John, 2006, “Exploration of internal service systems using lean principles.” Management Decision 44 (5), pgs. 674-689. 84 Katopol, “Just enough” Information,” p. 233.

35

Financial Authority over Information Management and Information Technology Within this context, key informants suggest that financial information is consistently prioritized by senior executives within AANDC. Decision makers consult with Finance through the Financial Management Committee and the Operations Committee, which are often the last point of contact prior to major strategic decisions regarding departmental priorities. Currently, the Financial Management Committee is active in decision making regarding IM and IT at AANDC. As a result of the Financial Management Committee making IM/IT decisions, concern was expressed that decisions are being based on availability of money rather than overall departmental priorities. There was a concern that because the main line of communication between the Chief Information Officer and senior management is the Chief Financial Officer and financial community, IM/IT is not being prioritized to the extent necessary for proper information for decision making. Without an open line of communication between the Chief Information Officer, senior management and particularly the Deputy Minister, informants noted decisions made were not always the most effective for aligning IM/IT needs with departmental priorities. According to the Harvard Business Review, fostering a better understanding between the Chief Executive Officer, other senior management and the Chief Information Officer is a common challenge in the private sector as well.85 The Chief Executive Officer can inform on best business practices and the Chief Information Officer can inform on the importance of information management and technology to the business as a whole.86 The review suggests that if the Chief Information Officer is involved in the structuring of an organization as a whole, it can operate significantly more effectively.87 These private sector best practices could prove useful to AANDC’s organizational structure.

85 Stikeleather, Jim, July 15, 2013, “The CIO In Crisis: What You Told Us.” Available at: http://blogs.hbr.org/cs/2013/07/the_cio_in_crisis_what_you_tol.html?utm. Harvard Business Review. 86 h Stikeleather, Jim , May 2, 2013, “The Metamorphosis of the CIO.” Available at: http://blogs.hbr.org/cs/2013/05/the_metamorphosis_of_the_cio.html. Harvard Business Review. 87 Stikeleather, Jim, May 6, 2013, “Three Ways CIOs Can Connect with the C-Suite.” Available at: http://blogs.hbr.org/cs/2013/05/three_ways_cios_can_connect_with_the_c-suite.html. Harvard Business Review.

Fostering the relationship between the Deputy Minister, the Chief Information Officer and senior management could enhance support for decision makers’ IM and IT needs.

36

5.5 Is information for decision making within the Department cost-effective? Finding: In order for information for decision making to be more cost –effective, AANDC can foster stronger IM culture with broader standardization and accessibility of data. The Value of Data In storing or in some way categorizing important unstructured data, the Department makes it retrievable, accessible and ultimately useful in informing decision makers. A number of literature sources have commented on the practice of assigning value to information. For example, the UK Knowledge Council, a council of senior leaders in knowledge and information management across the United Kingdom government, is currently developing a framework for mapping the location and value of knowledge and information88 in order to sort or prioritize available information in cases where there is too much information. The Canadian government’s own Policy on Information Management explains that integrating information management considerations into all aspects of government business enables information to be used and recognized as a valuable asset.89 One source suggested that the Department needs to have a better appreciation that data is its most valuable asset. If the value of information is maximized and turned into knowledge, decision making is better informed. If information is not managed well or combined with good knowledge management, “the value of information as an asset is undermined, and cost-effective, efficient service delivery is compromised.”90 Standardization and Accessibility A number of literature sources have stressed the importance of standardization across information platforms, and a movement away from operating within data silos. As mentioned previously, open data warehouses, with comprehensive naming conventions and categorization, have the potential to set IM culture standards across the Department, therefore, leading to more cost-effective storage and access practices. The cost-effectiveness of GC Docs’ more accessible approach stems from what was mentioned earlier – more accessible information is cheaper,91 because it decreases the duplication of data and helps identify irrelevant information.92 A shared approach will cost significantly less whereas having to recover, reconstruct or do without information, would cost much higher.93 One interviewee recalled a Privy

88 Home Civil Service UK, Information Matters, p. 8. 89 TBS Policy on Information Management. 90 Home Civil Service UK, Information Matters, p. 4. 91 AANDC Enterprise Information Management Strategy 2010-2015, pg. 25. 92 Canada School of Public Service, n.d. “Are You Ready for GCDOCS?” Available at: http://www.csps-efpc.gc.ca/BrowseCoursesBySubject/gcdocs-eng.aspx. 93 Home Civil Service UK, Information Matters, p. 8.

The United Kingdom government is currently developing a framework for mapping the location and value of knowledge and information; implementing this at AANDC could prove to be a useful information management practice.

37

Council Office cost-analysis of a typical contracted report at roughly $32,000, and explained that there was a very tangible cost associated with re-collecting or re-analyzing information. Thus, in using value-domains, as outlined by Treasury Board Secretariat policies, the Department can save money by not wasting time searching for information or undertaking the same work multiple times. The same interviewee noted that the opportunity cost of searching for information has been calculated at 32 full time employees per year per sector and AANDC likely faces similar costs. The Bottom Line Notably, a bottom-line driven private sector model may promote an efficient IM culture within their staff simply by connecting personal employee incentives (such as salaries) to the company’s success.94 Some key informants have suggested that a lack of incentives results in poor IM culture. This suggests that, though the incentives are different between the private sector and the public sector, cost-effectiveness does in part stem from an effective IM culture, which can lead to more comprehensive and timely information for decision making. Further, it should be noted that informed decision making often stands at odds with a minimal-cost approach to government. One literature source posits that data-driven decision making “demands a serious commitment to training, data cleanup, and maintenance that is often overlooked.95 Nevertheless, minimizing low value-added processes in an organization has the potential to minimize the cost to taxpayers without compromising the integrity and the rigor of information. Literature on the private sector advocates for ‘lean management principles,’ which involve identifying low value-added processes and changing them for greater efficiency. Where lean management has been practiced it has been credited with lower costs, increased capacity, and reduced cycle times resulting in an efficient decision making process.96 Within the information for decision making context, organizations reportedly realize through lean management that sometimes what is thought to be a valuable process is in fact “wasteful.”97 It should be noted that improved productivity in the Information for Decision Making process can be accomplished without any hiring or layoffs. It should also be noted that lean management principles recommend making cost-efficient decisions based on operational needs as opposed to purely based on financial metrics. As one scholar notes, “successful organizations make changes based on operational measures, and when these operational measures perform well, financial measures easily follow suit.”98

94 Muligan, Richard, 2000, “Comparing Accountability in the Public and Private Sectors.” Australian Journal of Public Administration, 59(1) 95 Wheaton, “10 Things You Always Wanted to Know About Data-Driven Decision Making.” 96 Plenert, Lean Management Principles for Information Technology, Pg 6. 97 Ibid., Pg. 134. 98 Ibid., Pg. 132.

Disseminating simple technique reminders to employees such as how to save a comprehensive Groupwise discussion thread into CIDM could result in increased time and cost efficiencies.

38

Minimizing low value-added processes is currently not an evident strategy in the Department. For example, the departmental email system, Groupwise, was never meant to be a storage system for email content. Groupwise directives and training sessions99 suggest that users save their emails into CIDM. Instead, key informants explain that employees tend to keep their emails in Groupwise and use the search function to find the necessary information, instead of saving categorized pieces. As a result, it was estimated that only about four to five percent of the average employee’s email is helpful. Thus, turning unstructured email content into structured, value-laden pieces of information, could ‘lean-down’ the burden on Groupwise, making it more efficient. This could decrease the amount of time spent searching for information, which may ultimately support decision makers. This is just one example of ways in which individual employees can contribute to more cost-efficient information for decision making at AANDC.

99 Information Management Branch, Aboriginal Affairs and Northern Development Canada, “IM Awareness – Using CIDM for Email Management.”

39

Conclusion This evaluation examined the Information Management, Information Technology, Management and Oversight, and Financial Management functions of the Department’s internal services in the context of the identified risk of Information for Decision Making. The evaluation found that while information for decision making is strong at AANDC relative to other organizations, several key challenges exist:

· Challenges with IT Systems: IT systems are not well integrated and could be better designed to incentivize proper use. 

· Information Management Culture: In part, as a result of IT systems that are not

user-friendly, employees at AANDC generally do not follow proper guidelines for information management. This includes proper digitization, classification, dissemination and disposition of information. 

· Lack of context for financial decisions: Financial reports are often not fully populated,

resulting in a lack of information for decision making. Additionally, due to a lack of systems integration, financial decisions are sometimes made without non-financial qualitative context for support.  

· Narrow timelines: The main constraint on AANDC's capacity to analyze its data is the

context of tight timelines in which accurate and comprehensive information is required by decision makers.  

· Excessive reliance on informal networks: Given the challenges described above to

accessing needed information, senior management routinely finds informal networks of personal contacts more helpful for supporting decision making. Often personal experience is relied upon more than departmental analysis. When accompanied with employee turnover, this presents a real concern to maintenance of corporate memory.  

· Low prioritization of Information for Decision Making: These effects are exacerbated by a structure in which information for decision making is not prioritized. Concerns have been expressed that information for decision making is viewed primarily through a financial lens, and that fostering the relationship between the Chief Information Officer and senior management, particularly the Deputy Minister, may help to support the IM/IT function at AANDC.

While the issues above present potential risks to access, reliability, credibility, timeliness, cost-efficiency and other pertinent qualities of information for decision making, opportunities exist for mitigation. Techniques such as collaboration between IT creators and users, increased IM awareness activities, time and stress management training and new practices, such as crowd sourcing could present low-cost opportunities to mitigate the risks outlined in the 2012 Corporate Risk Profile and corroborated by this evaluation.

40

Bibliography Aboriginal Affairs and Northern Development Canada. 2009. Chief Financial Officer Sector

Management Practices Review. Available at: http://www.aadnc-aandc.gc.ca/eng/1333045321642/1333045366141. Government of Canada.

Aboriginal Affairs and Northern Development Canada. 2012. Appendix A: AANDC Program Activity Architecture. Available at: http://www.aadnc-aandc.gc.ca/eng/1345558981352/1345559176644. Government of Canada.

Aboriginal Affairs and Northern Development Canada, 2012, Corporate Risk profile 2012. Government of Canada.

Aboriginal Affairs and Northern Development Canada. 2013, Annual Report on the State of Performance Measurements in Aboriginal Affairs and Northern Development Canada for 2011-2012. Government of Canada.

Aboriginal Affairs and Northern Development Canada. 2013. Backgrounder – Aboriginal Affairs and Northern Development Canada’s Financial Picture (2013-2014). Available at: http://www.aadnc-aandc.gc.ca/eng/1363029324938/1363029382616. Government of Canada.

Aboriginal Affairs and Northern Development Canada, 2013, Community of Practise on AANDC's Committee Governance Structure (2013). "The Way Forward: Options for Consideration” Government of Canada.

Aboriginal Affairs and Northern Development Canada. Enterprise Information Management Strategy 2010-2015. Government of Canada.

Athanasawa, Y., 2003, “Team Characteristics and Team Member Knowledge, Skills, and Ability Relationships to the Effectiveness of Cross-Functional Teams in the Public Sector.” International Journal of Public Administration, 26 (10/11), pgs. 1165-1203

Bennett, R. H., 1998, “The Importance of Tacit Knowledge in Strategic Deliberations and Decisions”. Management Decision, 36(9), 589-597.

Cabinet Affairs Directorate, Aboriginal Affairs and Northern Development Canada, 2013, Overview of Critical Functions. Government of Canada.

Canada School of Public Service, n.d. “Are You Ready for GCDOCS?” Available at: http://www.csps-efpc.gc.ca/BrowseCoursesBySubject/gcdocs-eng.aspx. Government of Canada.

Caverlee, James, Liu, Ling, and Steve Webb, 2010, “Social Trust Framework for Trusted Social Information Management: Architecture and Algorithms.” Information Sciences 180: 95-112.

Davenport, T. H., 1994, “Saving IT’s Soul: Human-Centered Information Management.” Harvard Business Review, March-April, pgs. 119-131.

Department of Justice, 2004, Library and Archives of Canada Act. Available at: http://laws-lois.justice.gc.ca/eng/acts/L-7.7/. Government of Canada

Feldman, J., & Tung, R., 2001, “Using Data-based Inquiry and Decision Making to Improve Instruction.” ERS Spectrum 19(3), 10–19; Ingram, D., Louis, K.S., & Schroeder, R.G., 2004, “Accountability policies and teacher decision making: Barriers to the use of data to improve practice.” Teachers College Record, 106(6), 1258–1287.

Fornaro, J. G. (2008). “Institutional Memory: How good is your club’s memory? Really!!”. The Boardroom, January/February.

41

Government of Hong Kong Efficiency Unit. 2010. Making Informed Decisions. Available at: http://www.eu.gov.hk/ english/publication/pub_bp/files/making_informed_decision.pdf. Government of Hong Kong.

Gregor, S., Hart, D., & Martin, N, 2007, “Enterprise Architectures: enablers of business strategy and IS/IT alignment in government. Information.” Technology & People, 20(2), pgs. 96-120.

Hedgebeth, Darius, 2007, “Data-driven decision making for the enterprise: an overview of business intelligence applications”. VINE, 37(4), pgs. 414-420.

Herbert, Mark, n.d., “Executing an Effective Workforce Reduction.” Available at http://www.newparadigmsllc.com/PDFs/WorkforceReduction.pdf. New Paradigms LLC, 8 p; Green, Peter and Denise Skinner, 2005. “Does time management training work? An evaluation,” International Journal of Training and Development 9:2, p. 124.

Hilbert, D.M., Billsus, D. and Denoue, L., 2006, Seamless Capture and Discovery for Corporate Memory. Available at: http://www.fxpal.com /publications/FXPAL-PR-06-370.pdf.

Home Civil Service UK, 2008, Information Matters: Building Government’s Capability in Managing Knowledge and Information. Available at: http://www.nationalarchives.gov.uk/documents/information-management/information-matters-strategy.pdf . The National Archives- Government of the United Kingdom;

Information Management Branch, Aboriginal Affairs and Northern Development Canada, 2012, Saving Protected C and Above Documents in CIDM. Available at: http://giti-imit/ppg/rpi/edm-eng.asp. Government of Canada.

Information Management Branch, Aboriginal Affairs and Northern Development Canada, “IM Awareness – Using CIDM for Email Management.” Government of Canada.

Ingram, D., Louis, K.S., & Schroeder, R.G., 2004, “Accountability policies and teacher decision making: Barriers to the use of data to improve practice.” Teachers College Record, 106(6), 1258–1287.

Jossey-Bass.; Simon, H.A., 1987, “Making Management Decisions: The Role of Intuition and Emotion.” Academy of Management Executive, February, pgs. 57–64.

Katopol, P.F, 2007, “Just enough” Information: Information behavior, organizational culture and decision making in municiple government. Canadian Journal of Information and Library Science. 31(3-4), p. 233

Kernaghan, K., & Gunraj, J, 2004, “Integrating information technology into public administration: Conceptual and practical considerations.” Canadian Public Administration 47(4), pgs. 525-546.

Kettley P, H., 2000, Learning from Cross-functional Teamwork. Available at: http://www.employment-studies.co.uk/pubs/summary.php?id=356. Institute for Employment Studies.

Khatri, Naresh and Ng, H. Alvin, 2000, “The Role of Intuition and Strategic Decision Making.” Human Relations, 53(1), pgs. 57-86.

Lachat, M.A., 2001, Data-driven high school reform: The Breaking Ranks model. Providence, RI: LAB at Brown University.

Library and Archives Canada, 2013, Generic Valuation Tool: Management and Oversight Services- DRAFT, 2013. Available at: http://www.gcpedia.gc.ca/gcwiki/images/6/6f/M%26O_Draft_GVT_May_2013.pdf. Government of Canada.

Malayeff, John, 2006, “Exploration of internal service systems using lean principles.” Management Decision 44 (5), pgs. 674-689.

42

MEASURE Evaluation, 2008, Data Demand and Information Use in the Health Sector: Case Study Series. Available at: http://www.cpc.unc.edu/measure/publications/sr-08-44. Chapel Hill, NC: MEASURE Evaluation.

Muligan, Richard, 2000, “Comparing Accountability in the Public and Private Sectors.” Australian Journal of Public Administration, 59(1)

Myeong, S., & Choi, Y., 2010, “Effects of Information Technology on Policy Decision-Making Processes: Some Evidences Beyond Rhetoric.” Administration & Society 42(4), pgs. 441-459.

Pederson et. Al, 2013, “Conceptual Foundations of Crowdsourcing: A Review of IS Research.” 46th Hawaii International Conference on Systems Science, p. 579.

Plenert, G., 2013, Lean Management Principles for Information Technology . CRC Press. P.17 Pondy, L.R., 1983, “The union of rationality and intuition in management action. Introduction:

common themes in executive thought and action.” In S. Srivastava and associates (Eds), The Executive Mind. San Francisco, CA

Post, G, 2007, “Evaluating information security tradeoffs: Restricting access can interfere with user tasks.” Computers & Security, pgs. 229-237.

Power, Daniel, 2008, “Understanding Data Driven Decision Support Systems.” Information Systems Management 25(2), pgs..149-154.

Psaier et. Al, 2011, “Resource and Agreement Management in Dynamic Crowdcomputing Environments.” 15th Annual Enterprise Distributed Object Computing Conference, 193.

Simon, H.A., 1987, “Making Management Decisions: The Role of Intuition and Emotion.” Academy of Management Executive, February, pgs. 57–64.

Stikeleather, Jim , May 2, 2013, “The Metamorphosis of the CIO.” Available at: http://blogs.hbr.org/cs/2013/05/the_metamorphosis_of_the_cio.html. Harvard Business Review.

Stikeleather, Jim, May May 6, 2013, “Three Ways CIOs Can Connect with the C-Suite.” Available at: http://blogs.hbr.org/cs/2013/05/three_ways_cios_can_connect_with_the_c-suite.html. Harvard Business Review.

Stikeleather, Jim, July 15, 2013, “The CIO In Crisis: What You Told Us.” Available at: http://blogs.hbr.org/cs/2013/07/the_cio_in_crisis_what_you_tol.html?utm. Harvard Business Review.

Treasury Board Secretariat, 2007, Directive on Information Management Roles and Responsibilities. Available at: http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=12754. Government of Canada.

Treasury Board Secretariat, 2007, Policy Framework for Information and Technology. Available at: http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=12452. Government of Canada.

Treasury Board of Canada Secretariat, 2007, Policy on Information Management. Available at: http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?section=text&id=12742. Government of Canada.

Treasury Board of Canada Secretariat, 2009, Directive on Recordkeeping. Available at: http://www.tbs-sct.gc.ca/pol/index-eng.aspx?tree=directive. Government of Canada.

Treasury Board Secretariat, 2009, Directive on the Evaluation Function. Available at: http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15681&section=text. Government of Canada.

Treasury Board Secretariat, 2009, Standard on Geospatial Data. Available at: http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=16553&section=text. Government of Canada.

Treasury Board Secretariat, 2010, Policy on Management, Resources and Results Structures. Available at: http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=18218. Government of Canada.

43

Treasury Board Secretariat, 2010, Program Alignment Architecture. Available at http://publiservice. tbs-sct.gc.ca/mrrs-sgrr/index-eng.asp. Government of Canada.

Treasury Board of Canada Secretariat, 2010, Standard on Metadata. Available at: http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=18909. Government of Canada.

Treasury Board of Canada Secretariat, Presentation on Profile of GC Internal Services- Summary, 2008. Available at: http://www.tbs- sct.gc.ca/presentations/mrrs-sgrr/pis-psi/page01-eng.asp. Government of Canada.

Wei Choo, C., 2001, Environmental Scanning as Information Seeking and Organizational Learning. Available at: http://choo.fis.utoronto.ca/fis/respub/chooimreader.pdf. University of Toronto.

Wheaton, Shorr, P, 2003, “10 Things You Always Wanted to Know About Data-Driven Decision Making.” Available at: http://www.scholastic.com/browse/article.jsp?id=423. Scholastic Administrator Magazine.

Wilder Research, 2009, Data entry and analysis: Evaluation resources from Wilder Research. Saint Paul: Minnesota: Wilder Foundation;

Wurman, R. S., 1997, Information Architecture. New York: Graphis Inc. Yang, T., and T. Maxwell, 2011, “Information Sharing in Public Organizations: A Literature

Review of Interpersonal, Intra-Organizational and Inter-Organizational Success Factors.” Government Information Quarterly, 28(2), pgs. 164-175.