Business Continuity Program/Disaster Recovery Program

Business Continuity Program/Disaster Recovery Program Update

Statewide BCP Project SDC Disaster Recovery Plan & Service Contract

Project eBRP Project

February 2007

Contributors: CNIC BCP Team CNIC Disaster Recovery Task Force State Data Center

CNIC Disaster Recovery Task Force 2

Background – Multiple Projects within a BCP/DR Program


Background - Projects

1) Statewide BCP Initiative This initiative was launched in March 2006 with the implementation of the

Statewide BCP Policy. It is managed by the DAS Enterprise BCP office and provides guidelines and direction for State agencies in their plan development.

2) eBRP Toolkit Project In Fall 2006, the eBRP Toolkit was selected for the storage and publication

of BCP and DR plans for state agencies. This project manages Phase 1 of the implementation, training and utilization of the toolkit by 12 state agencies. Subsequent phases will include the inclusion of additional agencies and testing.

3) BCP Tabletop Test Project At the conclusion of the eBRP Toolkit Project, this project will manage BCP

tabletop tests for 12 state agencies.


Background - Projects

4) SDC Recovery Service Contract Project This project establishes a disaster recovery service contract for

applications and systems hosted at the SDC.

5) SDC/Agency DR Test Project This project executes disaster recovery tests per the DR awarded service

contract for systems hosted at the SDC.

6) Agency DR Test Projects This project executes disaster recovery tests per the DR awarded service

contract for agency systems.

7) Ongoing BCP/DR Plan Maintenance Business Continuity Plans and Disaster Recovery Plans will be maintained

and routinely tested by each agency through their established maintenance programs.


Statewide BCP Initiative


Statewide BCP Initiative – Objectives

Designation of BCP Coordinators and Sponsors.

Identification of critical business functions.

Developed, tested, and maintained BCP.

Complete DR plans for the IT infrastructure that support the critical business functions.

Implementation of BCP maintenance programs.

Established awareness and training programs to create organizational awareness.


Ultimate Goal

All Agencies will have developed, maintained, and tested plans by June 2009 to restore and recover their business operations in the event of a disaster.


Phased Approached – Proposed Planning Goals

The Enterprise BCP Office has recommended that agencies develop their BCP utilizing a phased approach

This will focus agencies to work on those most critical business functions

Four Phases are recommended: Phase 1: CBF that must be restored within 2 days Phase 2: CBF that must be restored within 1 week Phase 3: CBF that must be restored within 2 weeks Phase 4: CBF that must be restored within 1 month


Statewide BCP Initiative – Tracking the Progress

The Enterprise BCP Program will use a “Scorecard” to track the progress of the implementation of the Statewide BCP Policy.

The Scorecard will use the standard Green – Yellow – Red Status indicators: Agencies that have met the planning goals will receive a

GREEN Status;

those that have partially met the goals will receive a YELLOW status; and

those who have not met the goal in any form will receive a RED Status.


Statewide BCP Initiative – The Scorecard

PartialPartialYes0033YesYesYellowH

YesYesYesYesYes0024YesYesGreenG

PendingReport

PendingReport1232YesYes

PendingReportF

N/AN/AYes0130YesYesGreenE

N/AN/AYes0000YesYesGreenD

NoYes0202YesYesRedC

PartialYesNo0018YesYesGreenB

PartialPartialYes18112019YesYesYellowA

TESTINGPhase 26-30-08

PLANSPhase 212-31-07

TESTINGPhase 112-31-07

PLANSPhase 112-31-06

Approveby Mgmt

PHASE 41 mo.RTO

PHASE 32 weekRTO

PHASE 21 weekRTO

PHASE 12 dayRTO

IdentifyCBF

6-30-06

AssignCoord1-31-06

AGENCYSTATUSAGENCY

BCP Development & TestingPhased Planning Methodology-CBF

Statewide BCP Implementation Scorecard


SDC Disaster Recovery Plan & Service Contract Project


SDC Disaster Recovery Plan & Contract Project Overview The CNIC CIO’s formed a CNIC Disaster Recovery

taskforce to examine and propose DR solutions for systems and services managed by the State Data Center.

The current Sungard DR service contract was extended through October 2007, while requirements for a new RFP and service contract can be gathered and awarded.


SDC Disaster Recovery Plan & Contract Project Objectives Gather requirements for critical business functions and applications

supported by SDC managed systems.

Issue an RFP for recovery service contract pricing.

Analyze pricing and select recovery options and a recovery vendor.

Issue a recovery service contract that will be executed with the SDC for SDC managed systems and

services may be executed by CNIC agencies for out-of-scope systems and

services

The DR component of this project will be tracked and managed by the use of a Scorecard which includes the DR tasks and milestones.

IT data gathered to support the acquisition of the DR service contract will be used to load SDC and Agency instances of eBRP.


Business Function RTO

0

2

4

6

8

10

DASDHS

DOR

ODOTOST

OSP

DCBSDOC

ODVAOED

OHCS

ODOF

Agency

Nu

mb

er o

f F

un

ctio

ns

<1 Day

1-2 Days

The Recovery Time Objective (RTO) is the least amount of time required by a business function and/or application to recover critical services, measured from time of disruption to resumption of critical business operations. As a part of the BIA, SunGard classified the RTOs for the critical functions identified by the State of Oregon agencies. The graph below denotes the number of functions that have an RTO of less than 1 day (blue) and between 1-2 days (burgundy).

SDC Disaster Recovery

Recovery Time Objective (RTO)

14


Business Function RPO

012345678

DAS

OHCS

ODOTOSP

DHSDOR

OEDOST

DCBS

ODVADOC

ODOF

Agency

Nu

mb

er

of

Fu

nc

tio

ns

<1 Day

1-2 Days

The Recovery Point Objective (RPO) is the amount of data that each business function is willing to lose if a disruption occurs. As a part of the BIA, SunGard classified the RPOs for the critical functions identified by the State of Oregon agencies. The graph below denotes the number of functions that have an RPO of less than 1 day (blue) and between 1-2 days (burgundy).

SDC Disaster Recovery

Recovery Point Objective (RPO)

Data


Critical Applications Agency Function Name RTO RPO

ODOT Manage Driver Safety Digital Photo Licensing - DPL 1 Day 1 Day

ODOT Manage Driver Safety Driver License Inquiry, including LEDS 1 Day 1 Day

ODOT Vehicle Registration Undercover Database 1 Day 1 Day

ODOT Vehicle Registration Vehicle Inquiry, including LEDS 1 Day 1 Day

OED Implied Consent Printing Function 1 Day 1 Day

OED Implied Consent Word 1 Day 1 Day

DOR Funds Distribution ITA 1 Day 2 Days

DAS Voice Communications Connectivity from GTD-5 to Salem Mall 1 Day > 1

Month

DAS Network Communications HP OpenView 1 Day

OED Receive UI Filings Telephone Initial Claims - IVR 1 Day

OHCS Payment Payroll Internet 2 Days n/a

DAS State Payroll Interface file from PPDB 2 Days 4 Hours

DAS State Payroll OSPA Application – production of payroll payments 2 Days 4 Hours

OHCS Collect Info STAN/ACH 2 Days 4 Hours

OHCS Payment Payroll STAN/ACH (automated clearing house) 2 Days 4 Hours

OST Banking Services Communication Application 2 Days 4 Hours

OST Banking Services Operations Application 2 Days 4 Hours

OST Banking Services State Treasury ACH Network 2 Days 4 Hours

DAS Print Mail BARR (check print) 2 Days 1 Day

DAS Personnel Mgmt CERT 2 Days 1 Day

DAS Ops & Maintenance Key Card Access Control (Mosler software – switching to Lenel)

2 Days 1 Day

OHCS Payment Payroll SFMA & Word (for forms) 2 Days 1 Day

OST Investmt Rptg Excel spreadsheets 2 Days 1 Day

DAS Print Mail Mail Management System 2 Days 2 Days

DCBS Bldg Code Inspctns BCD Elevator System 2 Days 2 Days

DCBS Enforce OSHA Regs Case Appeals Tracking 2 Days 2 Days

DCBS Enforce OSHA Regs Federal OSHA reporting system (IMIS) 2 Days 2 Days

DCBS Bldg Code Inspctns Tri-County Permits 2 Days 2 Days

DOR Payment Collections Agency Collection Tracking System (AS/400) 2 Days 2 Days

DOR Payment Collections BIS – Business Information System (AS/400) 2 Days 2 Days

DOR Payment Collections ITA – Accounting System (AS/400) 2 Days 2 Days

ODVA Veteran's Claims VetRex 2 Days 2 Days

OHCS Pay Debt AXYS system 2 Days 2 Days

OHCS Transfer Funds AXYS system 2 Days 2 Days

OHCS Mtg Pmt LIPS System 2 Days 2 Days

OHCS Payment Payroll OSPS (Personnel/Payroll) 2 Days 2 Days

OHCS Collect Info SFMA 2 Days 2 Days

OHCS Pay Debt Treasury statements 2 Days 2 Days

OHCS Transfer Funds Treasury statements 2 Days 2 Days

OHCS Pay Debt Trustee statements 2 Days 2 Days

OHCS Transfer Funds Trustee statements 2 Days 2 Days

OHCS Collect Info Vendor list 2 Days 2 Days

OSP ID Services *FOCUS/FORSECOM 2 Days 2 Days

OSP ID Services FICS 2 Days 2 Days

DAS Voice Communications Connectivity from GTD-5 to off-mall Salem via Qwest 2 Days > 1

Month

DAS Voice Communications Connectivity of Salem GTD-5 to Salem and Bend RSUs 2 Days > 1

Month

DAS Network Communications Remedy 2 Days > 1

month

SDCDisasterRecovery

16


SDC Disaster Recovery Plan & Service Project Scorecard

Note: The scorecard reflects phases for a) DR Plan Development, b) DR RFP and Service Contract, and c) Subsequent DR Planning and Tests.


Recovery Windows

Think about the risk we bear without High Availability of our business applications….. if infrastructure is not available

High Availability Recovery Window

Days MinsHrsWks Secs

Recovery PointRecovery Point

Mins DaysHrsSecs Wks

Recovery TimeRecovery Time

How Much of a Robust DR Capability is Needed?

When compared against Agency’s consensus on existing risk…..


eBRP Toolkit Project


eBRP Toolkit Project – Overview This project supports the “Statewide BCP Policy” by implementing the

eBRP Toolkit software within the 12 CNIC agencies.

The eBRP Toolkit bridges the gap between business and IT. This project encompasses two distinctive components, the development and maintenance of agency BCP and DR plans.

The software will allow state agencies to manage business continuity planning (BCP) and disaster recovery (DR) planning. The resulting plans published through the software will be used during an event to manage recovery strategies.

All 12 Agencies are expected to support and maintain the eBRP Toolkit by dedicating the resources necessary for the development of their business continuity plans and disaster recovery data within the tool. The implementation team consists of a minimum of a BCP and DR administrator from each agency.


eBRP Toolkit Project – Objectives Implementation of toolkit to the 12 CNIC agencies.

Build disaster recovery strategies and plans that are aligned with critical business functions and support agency business continuity plan development.

Data will be loaded and maintained by each of the agencies into their own eBRP instance, using data entry and toolkit import processes.

Each agency will determine the extent of data loaded based on their individual BCP requirements.

The 12 CNIC agencies will participate in the development and use of common terms and codes to support the long term goals of the statewide BCP initiative.


eBRP Toolkit Project – Bridging the Gap


eBRP Toolkit Project – Logistics All location data must be entered.

All technology data must be entered.

To populate the other sections of the Toolkit, each Agency will use their top 6 Critical Business Functions (CBF).

New milestones have been established.

To successfully meet the new milestones identified on the WBS, the complete location and technology data must be entered into the Toolkit as well as the information pertaining to the 6 CBF’s.

If an Agency has less than 6 CBF they will be required to input the data of those CBF they have identified.


eBRP Toolkit Project - WBS

eBRP Implementation Project

(9/14/06 – 12/31/07)eBRP Implementation Work Breakdown Structure

Enter Data(11/22/06 – 7/31/07)

5 Project Closeout(12/07)

9

Architect Metadata

(11/2/06 – 2/28/07)

4

Develop Phase 2 Recommendations

(12/31/07)

9.1

Train Administrators on

Enterprise(9/17/2006 )

3.2

Train BCP/DR Administrators

(9/27-9/28 2006 )

3.1

eBRP Software Training

(9/27/06 – 12/30/07)

3

Develop Response Plans (7/31/07–12/31/07)

6

Close Project1.4

Manage Project1.

3

Plan Project1.2

Initiate Project1.1

ProjectManagement

(9/14/06 –12/30/07)

1

eBRP Software Install

(9/25/06 – 12/4/07)

2

Install eBRP Software

(9/25-9/29)

2.1

Establish Security“https”

(12/4/07)

2.2

Load Organization(11/29/06-4/30/07)

5.1

Develop Business Plans for Top 6 CBF(10/31/07)

6.2

Load Technology

(4/30/07)

5.2

Develop IT Recovery

Plans(12/31/07)

6.3

Implement Enterprise Edition

(11/07)

8

Publish Information to

Enterprise(11-1-07)

8.1

Load All Locations (2/28/07)5

.1.1

Load IT Vendors(2/28/07)5

.1.3

Load Business Vendors(3/31/07)

5.1.

4

Load Teams(4/30/07)5

.1.5

Load Hardware(4/30/07)5

.2.1

Load Software(4/30/07)5

.2.2

Load Databases(4/30/07)5

.2.4

Load Applications

(4/30/07)5.2.

5

Load Network(4/30/07)5.

2.3

Load Processes(7/31/07)

5.3

Load Location Centric(7/31/07)5

.3.1

Load Process Centric(7/31/07)5

.3.2

Load Products(7/31/07)5.

3.3

Develop Contracts

(TBD)

7

Develop DR Kits

(TBD)

7.1

Develop DR Contracts

(TBD)

7.2

Develop DR Locations

(TBD)

7.3

Develop Plan Templates

(Core Team)(7/31/07)

6.1

Train Plans

3.1.

1

Train Process Modeling

11/16-11/17 20063.1

.2

Train Technology 1/17-1/18 2007

3.1.

3

Train Remaining

3.1

.4

SDC and Agencies will load all technology.Under Organization, Agencies will load all

data for Locations and Technology (in Green). IT Vendors are included in this

requirement.

For Employees, Teams, Processes, and Business Vendors, Agencies are required to load all data relating to their the top 6 critical business functions into the eBRP Toolkit to meet the required milestones.

Load Employees (4/30/07)5

.1.2

Develop Policies and Guidelines

(7/31/07)

4.5

Building Location Codes and

Building Naming Conventions

(12/31/06)

4.1

DR Metadata(2/28/07)4

.4

BCP Metadata(2/06/07)4

.2Glossary

(will continued to be updated(2/06/07)

4.3

Security Policy (Agency accountability and responsibility for

data)

4.5

.1

User Guide for Administrators4

.5.2


eBRP Project - Project Support & Toolkit Training

• Basic Training #1 – September 2006 (Plan Development)

• Basic Training #2 – November 2006 (Basic Entry & Organization Data)

• IT Training – January 17th & 18th 2007

• Basic Training #3 – TBD 2007 (Process Development)

Toolkit Assistance: for individual help, the Enterprise BCP Office, Enterprise Application Services (EAS), and the SDC are available to assist BCP and DR Administrators as needed. Call the Enterprise BCP Office to request help: 503-373-1067.

Business Continuity Program/Disaster Recovery Program

Business

Transcript of Business Continuity Program/Disaster Recovery Program