March 17, 2005© Gerald Isaacson 2005 Emergency Management Planning Business Continuity IT Partners.

Post on 21-Dec-2015

222 views 4 download

Tags:

Transcript of March 17, 2005© Gerald Isaacson 2005 Emergency Management Planning Business Continuity IT Partners.

March 17, 2005 © Gerald Isaacson 2005

Emergency Management Planning Business Continuity

IT Partners

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

A few definitions to get started:

… a “disaster”• The cake I was baking to bring to Xmas dinner• He lost a laptop with the only copy of his thesis• She lost her research and papers in the lab fire• Payroll system failed the day before payday• Asbestos released in a dorm renovation• The death of a student• The Northeast blackout• The recent tsunami

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

… a “disaster”

is an event, often unexpected, that seriously disrupts your usual operations or processes and can have long term impact on your normal way of life or that of your organization.

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

A few definitions to get started:

… RTO [Recovery Time Objective]

the point in time when you must have at least the critical aspects of your business operational again.

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

A few definitions to get started:

…RPO [Recovery Point Objective]

The last copy of your data that is out of harm’s

way – hopefully it is recently current.

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

A few definitions to get started:

… Business Continuity Planning

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

… it’s not rocket science

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

… it is:• a process to minimize the impact of a major

disruption to normal operations• a process to enable restoration of critical

assets• a process to restore normalcy to MIT as soon

as possible after a crisis.

… it is not just:• recovery of information technology resources

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

… and it is the phase of crisis management that follows the immediate actions taken to protect life and property and contain the event

… it begins when the situation has been stabilized.

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

… and it is now a national standard for both the public and private sectors

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

An ECAR report in March of this year, from Baylor Medical Center and the University of Houston, in the aftermath of hurricane

Allison, posed the following list of questions to ask to determine your

resilience to a disaster.

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

ECAR Research Bulletin Mar 1 2005

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

An ECAR report in March of this year, from Baylor Medical Center and the University of Houston, in the aftermath of hurricane

Allison, posed the following list of questions to ask to determine your

resilience to a disaster.

The answers constitute your Business Continuity Plan

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

The Risk Matrix

CHANGE

SOMETHING

IGNORENORMAL

PROCEDURES

PLAN

IMPACT

HIGH

LOW

PROBABILITYLOW HIGH

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

BOMB

MISC

ENVIRON

DATA

SOFTWARE

CIVIL

TELECOMM

FLOOD

HURR

EARTH

TORNADO

LIGHTNING

HARDWARE

Source: Gartner Group and Comdisco

Network Operations Disruptions

Power

Hardware

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

April 19, 2005 © Gerald Isaacson 2005

Mt. St. Helens – May 1980 – new threats arise

Business Continuity Planning

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

It’s different now….

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

© ECAR Research Bulletin Mar 1 2005

…but we still have the usual concerns

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

• When is it a Crisis?

Continuity Continuum

Minutes Hours Days Weeks

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

• When is it a Crisis?

Continuity Continuum

Minutes Hours Days Weeks

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

• When is it a Crisis?

Continuity Continuum

Minutes Hours Days Weeks

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

• When is it a Crisis?

Continuity Continuum

Minutes Hours Days Weeks

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

A crisis timeline --

April 19, 2005 © Gerald Isaacson 2005

Restoration of Critical Processing

Business Continuity Planning

Alarm Notification to First Responders

Data center fire

April 19, 2005 © Gerald Isaacson 2005

Restoration of Critical Processing

Business Continuity Planning

Activate the Emergency Operations Center

April 19, 2005 © Gerald Isaacson 2005

Restoration of Critical Processing

Business Continuity Planning

IT decision to move to a backup facility

April 19, 2005 © Gerald Isaacson 2005

Restoration of Critical Processing

Business Continuity Planning

Assemble IT recovery team at appropriate sites

April 19, 2005 © Gerald Isaacson 2005

Restoration of Critical Processing

Business Continuity Planning

Obtain backup tapes from off-premises storage

April 19, 2005 © Gerald Isaacson 2005

Restoration of Critical Processing

Business Continuity Planning

Acquire and install backup hardware and network connections

April 19, 2005 © Gerald Isaacson 2005

Restoration of Critical Processing

Business Continuity Planning

Restore Operating System and Network

April 19, 2005 © Gerald Isaacson 2005

Restoration of Critical Processing

Business Continuity Planning

Reload database and other data

April 19, 2005 © Gerald Isaacson 2005

Restoration of Critical Processing

Business Continuity Planning

Restore Critical Applications

April 19, 2005 © Gerald Isaacson 2005

Restoration of Critical Processing

Business Continuity Planning

Begin Critical Processing -

This is your Recovery Time Objective (RTO)

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Recovery Time Objective (RTO)

Do you measure it in hours or weeks?

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Recovery Time Objective (RTO)

Do you measure it in hours or weeks?

When do you need to start to reach it on time?

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Windows are Closing

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Increasing dependencies on electronic capabilities to do your job have shortened

the recovery timeframe.

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Increasing dependencies on electronic capabilities to do your job have shortened

the recovery timeframe.

This often leads to a disconnect between user’s expectations and the organization’s

ability to meet them.

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

© Lucent technologies

Move toAlternate Site

ReturnHome

ResumeBusiness

Data Synchronization

Restore Technology Capability

Restore Communications

Restore Business Functions

NotificationsVital Records

Lost Data

Data Recovery Objective

Recovery Time Objective

(If necessary)

High Level Look at a Recovery Effort

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Continuity Plan Development Phases• Understand the need

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Continuity Plan Development Phases• Understand the need

• Define the risk to the DLC and define the level of criticality

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Criticality Levels

• Category I Must be up in hours

• Category II Must be up in days

• Category III Must be up in weeks

• Category IV Can wait till operations are normal

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Continuity Plan Development Phases• Understand the need• Define the risk to the DLC and define the level

of criticality

• Give an individual responsibility and authority for overall planning

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Continuity Plan Development Phases• Understand the need• Define the risk to the DLC and define the level

of criticality• Give an individual responsibility and authority

for overall planning

• Ensure that organizational units – accounting, facilities, residential life, laboratories, etc. understand their individual responsibility for recovery of their operations.

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Continuity Plan Development Phases

• DLC’s develop their FARM (Functional Area Recovery Management) teams

• The Business Continuity Planning Team – (BCMT) consists of FARM Team Coordinators

• The BCMT is represented at the MIT Emergency Operations Center (EOC)

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Continuity Plan Development Phases

• Put together the recovery team

• Document the Plan

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

The Table of Contents from the TLO FARM Team Plan

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

• Key Plan Components– Contact list – up to date

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

• Key Plan Components– Contact list – up to date

– Resources needed – not just computing

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

• Key Plan Components– Contact list – up to date– Resources needed – not just computing

– Emergency procedures

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Continuity Plan Development Phases

• Document the plans for each department, lab or center

• Use a common template or framework• Plan review by Business Continuity

Management Team (BCMT) and Information Systems & Technology

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Continuity Plan Development Phases• Test, test, test

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Simulation Testing – MIT’s Central Utility Plant

• On an August afternoon, a fuel line rupture in the Central Utility Plant results in a fire.

• When the sprinkler system operates, the ensuing flood creates a hazardous waste issue due to the oil and ACM.

• The sprinkler operation also operates protective disconnects and the power is shut down.

• There have been 5 injuries of CUP workers because of this incident.

• After about 4 hours it is determined that the CUP’s clean up and return to service will take at least 24 hours, and the 80% of the campus that the CUP serves will remain without power.

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Continuity Plan Development Phases

• Test, test, test

• Maintain the plans – a very difficult process but critical to the long term viability of any plan

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

What’s next?

• Who should we be talking to in your area?

• Who should we be talking to who is not here?

• Any overall concerns about the project?

• Any specific concerns that we should be addressing?

April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning

Gerald Isaacson, CISSP William McShea, CFPS

gii@mit.edu wmcshea@mit.edu

617 253-1440 617 253-9491

32-013 32-013

web.mit.edu/bcmt http://web.mit.edu/environment/ehs/