HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC...

35
1 1 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services H. D. (Bud) Dowdy, Jr. This presentation is intended to provide client with: •information on 7R/E - the announcement in general & the specific 7R/E Cable Solution. •Reaffirm Lucent’s commitment to the Cable industry - tease regarding the Lucent Cable Business Unit and our involvement w/CableLabs & PacketCable. •While also stressing our eagerness & willingness to work w/them to provide the the client w/a solution that they find valuable, while gathering & soliciting additional requirements that they may have to ensure this happens.

Transcript of HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC...

Page 1: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

1

1 H. D. Dowdy, Jr.

HFC Network Reliability forToday’s & Tomorrow’s Services

H. D. (Bud) Dowdy, Jr.

This presentation is intended to provide client with:•information on 7R/E - the announcement in general & the specific 7R/E Cable Solution.•Reaffirm Lucent’s commitment to the Cable industry - tease regarding the Lucent Cable Business Unit and our involvement w/CableLabs & PacketCable.•While also stressing our eagerness & willingness to work w/them to provide the the client w/a solution that they find valuable, while gathering & soliciting additional requirements that they may have to ensure this happens.

Page 2: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

2

2 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesOverview

• Network Requirements of HFC Services

• Network Reliability and Availability History

• Causes of Network Outages and Weapons against

• Network Monitoring

• Summary

Page 3: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

3

3 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesNetwork Requirements of HFC Services

! Transmission quality! Effective bandwidth! Service reliability (the probability that a system will survive without

interruption for a defined period)! Outage Rate (the average rate at which service interruptions occur)! Availability (the percentage of time that service is available)

Network reliability should also take “degraded services into account and not focus on complete failure of a service.

Network reliability should should focus on delivering a quality service

Page 4: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

4

4 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesOverview

• Network Requirements of HFC Services

• Network Reliability and Availability History

• Causes of Network Outages and Weapons against

• Network Monitoring

• Summary

Page 5: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

5

5 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesNetwork Reliability and Availability HistoryDifferent industries use different measures of service integrity

Telephony Industry

Availability objective 99.99% (53 min/yr.)

! Bellcore “goal” chosen arbitrarily, not mandated by government regulation! Not the same as end user service availability!! Applies to network between local switch and network interface! Excludes local switch, customer premise equipment, in-home wiring and loss of

primary and backup power! Some customers may have consistently poor service but still meet objective! Clock starts when outage reported, not when it begins

In home wiring is largest single cause of service outage for catv industry

Page 6: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

6

6 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesNetwork Reliability and Availability HistoryDifferent industries use different measures of service integrity

Cable Industry/Broadcast Video

Availability objective 99.7% ( 26.3 hr/yr.)2 outages within 3 months for single customer

! CableLabs “goal” based on outages exceeding this rate become a major factor in customers perception of service quality

! 10 minute outage perceived as bad as an all day outage! “Outage” is 2 or more customers losing 1 or more channels, “Loss” is interruption not

degradation.! Includes power outages!! Does not count single outages, so drop and in-house wiring not included! Outage not count if repaired before being reported

In home wiring is largest single cause of service outage for catv industry

99.7=.6 outages/month and 4 hours MTTR

Page 7: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

7

7 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesNetwork Reliability and Availability History

Does Cable industry require 99.99% availability?

! Traditional HFC Plants 99+% (3.7 days/yr.)! Broadcast Video (Analog/Digital) 99.7% (26 hours/yr.)! Cable Modem 99.8% (13 hr/yr.)! Primary Line Telephony 99.99% (53 min/yr.) ! Secondary Line Telephony 99.7% (26 hours/yr.)

Note: 11/28/00 CableLabs document recommends an end-to-end availability objective of 99.94% (5.256 hrs.)

Depends on:What are customers willing to put up with for these servicesBusiness Model entering market

In home wiring is largest single cause of service outage for catv industry

Page 8: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

8

8 H. D. Dowdy, Jr.

Customer Expectations of Network Availability100% Available

99.99% (53 minutes available)

99.98% (106 minutes available)

99.97% (159 minutes available)

99.96% (212 minutes available)

Good Service

Requires Maintenance

Secondary Line• VoIP Set Top Box MTA• No Backup Power• No Carrier Grade CMTS, MTA, SW

Primary Line

Designed by BellcoreSpec

99.95% (265 minutes available)

99.94% (318 minutes available)

99.93% (371 minutes available)

Page 9: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

9

9 H. D. Dowdy, Jr.

Unavailability Related to Concentration of Revenue

Media Termination

Adapter

VoIP Customer

HFCAccessNetwork(DOCSIS)

Cable ModemTermination System

SONET Transport PSTN

SONET Transport

05

101520253035404550

Unavailability

Sub Lines Served 300 3k 100k Revenue per minute of Network Element drives reliability!

Page 10: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

10

10 H. D. Dowdy, Jr.

Network Availability “Primary Line”Engineering the End to End Telephony Solution

Media Termination

Adapter

VoIP Customer

HFCAccessNetwork(DOCSIS)

Cable ModemTermination System

SONET Transport PSTN

SONET Transport

Local Access (99.99% = 53 min) Switching (99.999% = 5.3 min)

10 1 136

5.353 minutes

5

Assumption is Power is always up

Inside Wire

Unavailability in Minutes

5.3

MTTR 4 4 4 4Redundant Redundant Redundant

4 4NOTE: Unavailability is calculated from MTBF historical data or FIT Rate/Part Count

Page 11: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

11

11 H. D. Dowdy, Jr.

Network Availability “Second Line”Engineering the End to End Telephony Solution

Media Termination

Adapter

VoIP Customer

HFCAccessNetwork(DOCSIS)

Cable ModemTermination System

SONET Transport PSTN

SONET Transport

Local Access (99.96-99.93% = 172-346 min) Switching (99.999% = 5.3 min)

60 6 45200

5.3172-346 minutes

20

Inside Wire

Unavailability in Minutes

5.3

MTTR 24 24 4 4Redundant

4 4NOTE: Unavailability is calculated from MTBF historical data or FIT Rate/Part Count

Page 12: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

12

12 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesOverview

• Network Requirements of HFC Services

• Network Reliability and Availability History

• Causes of Network Outages and Weapons Against

• Network Monitoring

• Summary

Page 13: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

13

13 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesCauses of Service Interruptions

! Signal Quality!Commercial power problems!Equipment failure! Interfering signals!Network Capacity!Customer misuse

Page 14: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

14

14 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesCauses of Service Interruptions - Signal Quality

!Signal Quality Effects on ServicesAnalog VideoDigital VideoCable ModemTelephony

Page 15: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

15

15 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesCauses of Service Interruptions-Signal Quality/Analog VideoExpectation grows with Consumer Product Improvements

! 10 years ago, consumer products not capable of displaying full resolution of NTSC signal

BETAVHS

! Today’s consumer equipment has greater resolution than broadcast NTSC Super VHSHi-8Laser DiscDVD

! FutureHDTVNext generation High Definition consumer products

! Trend to larger screen sizes also make video impairments more evident and customers less tolerant

Page 16: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

16

16 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesCauses of Service Interruptions-Signal Quality/Analog Video! Customers expectations are continually increasing

0

10

20

30

40

50

60

TASO 1958 CBS,NASA1983

Cable Labs1991

Perceptible/Annoying

Perceptible/SlightlyAnnoyingPerceptible/NotAnnoyingImperceptible

C/N

TASO-Television Allocations Study OrganizationCRC-Canadian Research Council

COCLUSION:Subscribers have become more critical and are likely to become even more so as they see more images delivered with digital video.

Annoying levels will continue to approach imperceptible

Page 17: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

17

17 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesCauses of Service Interruptions-Signal Quality! Digital Video

Blocking and freeze framesLoss of detail with heavy compressionAdvances like HDTV will also increase expectations of Standard Digital Video

! Cable ModemAvailability of cable modem service is already important to MANY CustomersHome users will demand higher availability as “reliable” service providers offer alternative technologies at similar prices (xDSL, Satellite)

! TelephonyHard for many people to leave existing reliable phone service Decide to market as Lifeline or Secondary ServicesHowever, proliferation of cell phones and Internet telephony (not VoIP!) have gotten people used to lesser quality phone service

Page 18: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

18

18 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesWeapons against Service Interruptions-Signal Quality

! Dealing with Signal QualityProactive plant maintenance

RF Monitoring Test Equipment

Make use of intelligent agents deployed in network (STB, CM’s)

Page 19: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

19

19 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesCauses of Service Interruptions - Commercial Power

! Commercial power problemsCommercial Power OutageLightning StrikesEquipment Failure

Page 20: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

20

20 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesWeapons against Service Interruptions- Commercial Power

! Dealing with Commercial power problemsPower Back-up (Battery, generators, mobile generators)

! Added maintenance issues (oil, batteries)

Network Powering, NIUs, MTA’s, EMTA’s & CPE

Page 21: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

21

21 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesWeapons against Service Interruptions- Commercial Power

050

100150200250300350400

ZeroBackup

4-HourBackup

8-hourbackup

12-hourbackup

Unavailability(min/year)

Average Commercial Power Unavailability in the United States(from Network Reliability Council)

CableLab’s task force showed 30% of all power supplies will see a service outage each year

Line powering a BIG issue. BELLCORE guidelines look for 53 minute total non-availability. Commercial power with 12 hour batteries will not achieve this.

Additionally, if NIUs or CM, MTA’s not line powered then they service will go down

Page 22: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

22

22 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesCauses of Service Interruptions- Equipment Failures! Power Supplies: 3%! Amplifiers: .15% to 10%! Optical Transmitters: 2.3%! Optical Receivers: .7% to 1.7%! Passive Devices: .07% to 1% ! Coaxial Connectors: .01% to .25%! Fiber-optic Cable: .3% to 3%! Coaxial Cable: .23% to 3%! Customer Premise Equipment: 7%?! Network Interface Devices: 5.4%! Headend Equipment: 5% to 30%Regional differences such as lightning strikes and underground construction laws have major effects on outages.from variety of sources: Network Reliability Council, Werner & Gatseos, Merk and Srode, Hamilton-Piercy and Balsdon, Bellcore

Page 23: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

23

23 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesWeapons against Service Interruptions - Equipment Failures

! Dealing with Equipment failureStart with high quality, reliable equipmentNetwork Architecture minimizing subs affected by single point of failureMonitoring equipment to isolate failure, decreasing repair time Redundant HardwareSelf healing ringsSpares

Page 24: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

24

24 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesCauses of Service Interruptions - Interfering Signals! Interfering Signal Problems

IngressCB RadiosAutomobile IgnitionConsumer Electronics

Internally generated signalsCSOCTBXMODCINCommon Path DistortionImproperly balanced plantEquipment failures (active and passive)

Page 25: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

25

25 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesWeapons against Service Interruptions- Interfering Signals

! Dealing with Interfering signalsCut-off switches (Located in taps, nodes or amplifiers)Ingress monitoring test equipmentUse intelligent agents (STB’s, CM’s) distributed in networkFrequency Agility Highly trained techsDevelop ingress mitigation techniquesNID rather than CPEPre-test of equipment and signal loading to verify margins in systemengineering calculations and manufacturers specs

Page 26: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

26

26 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesCauses of Service Interruptions - Network Capacity

! Network Capacity ProblemsInternet HogsSlow POP to InternetNo dial tone, Voice call blockingBack haul congestion

Page 27: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

27

27 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesWeapons against Service Interruptions- Network Capacity

! Dealing with Network CapacityTraffic engineering for HFC access portion on networkTraffic engineering for backhaul networkTraffic MonitoringHeadend lashups to facilitate redistributing serviceService contracts-Charge “Internet hogs” Network/Traffic Audits“Future Proof” Network minimize rebuilds

Page 28: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

28

28 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesCauses of Service Interruptions - Customer Misuse

! Customer misuse problemsPower off to CM or STBPC’s not configured properlyNot connected properly

Page 29: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

29

29 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesWeapons against Service Interruptions- Customer Misuse

! Dealing with Customer misuseImproved customer training Trained customer call center technical supportLiteratureOn-line self assistance

Page 30: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

30

30 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesOverview

• Network Requirements of HFC Services

• Network Reliability and Availability History

• Causes of Network Outages and Weapons against

• Network Monitoring

• Summary

Page 31: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

31

31 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesNetwork Monitoring

Video

High SpeedOnline

AccessTelephony

IP Telephony

LAN / WAN

Services being Integrated Network Operations Center

Multiservice Architecture

Master Headend

HubPSTN

Internet

FN

FN

Master Headend

Headend

Today’s Architecture

Page 32: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

32

32 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesNetwork Monitoring

Electronic Gateways

Call Agents

Policy Server

DHCP/ DNS

Server

Optical NetworkElements

Call AgentEMS

Optical EMS

CDR Collection

Billing Platform

TNAdmin

Order Entry Trouble Reporting

Order Management

Trouble Management

NetworkInventorySystem

CustomerRecordsDatabase

• Cable Modems & CMTS• DOCSIS/PacketCable NIU/MTA

• Video Servers• Set-top Boxes• Optical TX/RX

• PFG

CableConfig.

Manager

CircuitConfig

Dispatch/Work ForceMgmt

Optical Config.Mgmt.

RegistrationServer (Web)

IVR System(Voice)

Circuit Switches

ChurnManagement

Billing Inquiries(Web Based)

IDCM

CSR

EMS

CircuitFM

ATM EMS

ATMSwitches

Futu

re D

evel

opm

ent

DB

DB

Optical Fault

Mgmt.

CMTS / NIUEMS

Cable Network Elements

CableFault

Manager

Inter-DomainFault

ManagementDB

Page 33: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

33

33 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesNetwork Monitoring - Monitor all Intelligent Agents

OE

26

32

NID

PS

WW

OE

OE

Distribution Hub(s)

Fiber Transport & Distribution

High Speed Data CMTS

Channel Insertion

Telephony Termination

Opt

ical

Tra

nspo

rt

Video Channel

Equipment and Ad

Insertion

Circuit Switch / Trunks

High Speed Data

Routers, Servers,

and CMTS

Telephony over

Internet (VoIP)

Headend

Facility, Powering, and Environment

HFC Plant and End-user Terminals

EM

EM

EM

EM

EM

EM

EM

EM

Modem

Set-top

VoIP

Cab

le F

ault

Man

ager

Opt

ical

Fau

lt M

anag

erC

ircui

t Fau

lt M

anag

er

Page 34: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

34

34 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesOverview

• Network Requirements of HFC Services

• Network Reliability and Availability History

• Causes of Network Outages and Weapons against

• Network Monitoring

• Summary

Page 35: HFC Network Reliability for Today’s & Tomorrow’s … Reliability.pdf3 3 H. D. Dowdy, Jr. HFC Network Reliability for Today’s & Tomorrow’s Services Network Requirements of HFC

35

35 H. D. Dowdy, Jr.

HFC Network Reliability for Today’s & Tomorrow’s ServicesSummary! Decide on Services to be offered! Design/Build to needs of highest availability service offered! Decide on Business Plan - Do you need to be “highest quality service provider”! Develop appropriate architecture! Work with high quality equipment vendors! Perform availability/reliability/failure rate studies! Integration/Reliability Testing before deployment! Select high quality construction and Project Management firms! Partner with experts in Network Monitoring! Invest in Training!! Lost revenues when service unavailable or significantly degraded