Cyber Security Roadmap 08

8/2/2019 Cyber Security Roadmap 08

1/48

Roadmap SecureControl Systems

Water Sectorin the

toRoadmap SecureControl Systems

Water Sectorin the

to

Developed by

March 2008

Water Sector Coordinating Council Cyber Security Working Group

Sponsored by


2/48

Members of the Water Sector CoordinatingCouncil Cyber Security Working Group

Te Roadmap toSecure Control Systems in the Water Sectorwas developed by the

Water Sector Coordinating Council (WSCC) Cyber Security Working Group(CSWG) with support rom the Department o Homeland Security NationalCyber Security Division and American Water Works Association. Leadershipor this project was provided by Seth Johnson, WSCC-CSWG Representative;Bruce Larson, WSCC-CSWG Representative; Dave Edwards, Process ControlSystems Forum Water and Wastewater Representative; and Kevin Morley,WSCC Secretariat.


3/48

Roadmap to Secure Control Systems in the Water Sector

Acknowledgements

Te Water Sector Coordinating Council (WSCC)Cyber Security Working Group (CSWG) would

like to acknowledge everyone who contributed tothe development and nalization o the Roadmapto Secure Control Systems in the Water Sector. Inaccordance with the National In rastructureProtection Plan partnership model, the WSCCCyber Security Working Group worked in closecollaboration with the individuals identied belowand devoted signicant time, energy, eort, and

resources to develop a Roadmap to Secure ControlSystems in the Water Sector. Tis roadmap also

meets the cyber-related criteria identied by theGovernment Accountability Oce.1 Sponsorshipor roadmap activities came rom the American

Water Works Association (AWWA), which undedtwo roadmap development meetings, and theDepartment o Homeland Security, which undedthe meeting acilitation.

Paul Bennett, New York City Department oEnvironmental Protection

Amy Beth, Denver Water

Cli Bowen, Caliornia Department o Health Services

Jake Brodsky, Washington Suburban SanitaryCommission

Erica Brown, Association o Metropolitan WaterAgencies

Kim Bui, San Antonio Water System

Vic Burcheld, Columbus Water WorksRichard Castillon, Orange County Sanitation District

Rick DaPrato, Massachusetts Water Resources Authority

Kim Dyches, Utah Department o EnvironmentalProtection

Patrick Ellis, Broward County Water and WastewaterServices

Dave Edwards, Process Control Systems Forum/Metropolitan Water District o Southern Caliornia

Rod Graupmann, Pima County Waste Water

Management

Christina Grooby, Santa Clara Valley Water District

Darren Hollield, JEA

Seth Johnson, Water Sector Coordinating Council CyberSecurity Working Group Representative/ormerly oSanta Clara Valley Water District

WSCC Cyber Security Working Group (WSCC-CSWG)

Bruce Larson, Water Sector Coordinating Council CyberSecurity Working Group Representative/American Water

Carlon Latson, Denver Water

ony McConnell, Washington Suburban SanitaryCommission

Kevin Morley, Water Sector Coordinating CouncilSecretariat/American Water Works Association

Jerry Obrist, Lincoln Water

Elissa Ouyang, Caliornia Water Service Company

Kevin Quiggle, Detroit Water and Sewage DepartmentAlan Roberson, American Water Works Association

Candace Sands, EMA Inc.

Cheryl Santor, Metropolitan Water District o SouthernCaliornia

Birute Sonta, Metropolitan Water Reclamation District oGreater Chicago

Keith Smith, Metropolitan Water Reclamation District oGreater Chicago

Greg Spraul, Environmental Protection Agency Water

Security Division

Walt Wadlow, Santa Clara Valley Water District

Stan Williams, Santa Clara Valley Water District

Ray Yep, Santa Clara Valley Water District

Facilitators:

Jack Eisenhauer and Katie Jereza, Energetics Incorporated


4/48



5/48


Table of Contents

Executive Summary ..........................................................................................................................5 Te Industrial Control Systems Security Imperative .................................................................. 5 Industry Leadership ................................................................................................................... 5 Te Vision .................................................................................................................................. 5 A Strategic Framework ............................................................................................................... 6 Te Challenges Ahead ................................................................................................................ 6 A Call to Action ......................................................................................................................... 7 A Sustainable Approach ............................................................................................................. 7

I. Introduction ............................................................................................................................9 Roadmap Purpose ...................................................................................................................... 9 Roadmap Scope .......................................................................................................................... 9

Roadmap Organization .............................................................................................................. 9

II. Industrial Control Systems Use in the Water Sector ................................................................ 11 Supporting Missions and Business Functions ...........................................................................11 Evolution o ICS and odays Risks ..........................................................................................13 Cyber Security Treats...............................................................................................................13

III. Future rends and Drivers Infuencing ICS Security ............................................................... 17

IV. A Framework or Securing ICS in the Water Sector ................................................................ 21 Vision erms .............................................................................................................................21 ICS Security Goals ....................................................................................................................22 Strategies or Securing Industrial Control Systems ...................................................................22 Goal: Develop and Deploy ICS Security Programs ..................................................................24 Goal: Assess Risk......................................................................................................................27 Goal: Develop and Implement Risk Mitigation Measures .......................................................28 Goal: Partnership and Outreach ...............................................................................................31

V. Implementation ..................................................................................................................... 35

VI. For More Inormation ............................................................................................................ 37

Appendix A: Roadmap Process .....................................................................................................A-1

Appendix B: Reerences ...............................................................................................................B-1

Appendix C: Acronyms ................................................................................................................C-1


6/48



7/48


sae water supply could erode public condenceor, worse, produce signicant public health and

economic consequences.

Industry Leadership

Te urgent need to mitigate the risks associatedwith cyber systems has prompted industry andgovernment leaders to step orward and collaborate

on a unied securitystrategy. Teir eortshave produced thisRoadmap to SecureControl Systems in the

Water Sector, whichpresents a vision andsupporting rameworko goals and milestonesor reducing the risk oICS over the next ten

years. Tis strategic ramework enables industryand government to align their programs andinvestments, improving ICS security quickly andeciently. Te roadmap integrates the insights andideas o a broad cross-section o asset owners andoperators, industrial control systems experts, andgovernment leaders who met during workshopsheld in September and December 2007.

The Vision

By implementing this roadmap, water sectorindustry leaders believe that within 10 years,ICS throughout the water sector will be able tooperate with no loss o critical unction in vitalapplications during and ater a cyber event. Tis

vision conronts the ormidable technical, business,operational, and societal challenges that lie ahead

Vision for Securing Industrial ControlSystems in the Water Sector

In 10 years, industrial control systems or criticalapplications will be designed, installed, andmaintained to operate with no loss o criticalunction during and ater a cyber event.

Vision for Securing Industrial ControlSystems in the Water Sector

In 10 years, industrial control systems or criticalapplications will be designed, installed, andmaintained to operate with no loss o criticalunction during and ater a cyber event.

AnnualTreatAssessmentoftheIntelligenceCommunity(p.15)J.MichaelMcConnell,DirectorofNationalIntelligence,February7,2008

AnnualTreatAssessmentoftheIntelligenceCommunity(p.15)J.MichaelMcConnell,DirectorofNationalIntelligence,February7,2008

Executive Summary

odays industrial control systems (ICS)environments are incredibly complex assemblages

o technology, processes, and people that worktogether to successully carry out the missionsand business unctions o an organization. Tesesystems have improved water and wastewaterservice and increased reliability in thoseinrastructures. As ICS have become moreaordable and easier touse, most utilities havechosen to adopt themor process monitoringand/or control.2 Tis

reliance on ICS has letthe water sector andother dependent criticalinrastructuressuch as energy,transportation, andood and agriculturepotentially vulnerable totargeted cyber attack or accidental cyber events.

The Industrial Control Systems SecurityImperative

Cyber threats to ICS are changing and growing.

2

Computer attackers are seeking new targets andcriminal extortion is increasing. ICS security is nolonger simply about blocking hackers or updatinganti-virus sotware. A new underground digitaleconomy now provides a multi-billion dollarincentive or potential adversaries to exploit ICS

vulnerabilities.3

In todays highly dynamic and expanding digitaleconomy, much o the ICS that operate our current

water sector inrastructure are being used in ways

that were never intended. Many ICS were designeddecades ago with little or no consideration o cybersecurity. Increasing connectivity, the prolierationo access points, escalating system complexity,and wider use o common operating systems andplatorms have all contributed to heightenedsecurity risks. Any interruption o a clean and

Our information infrastructureincluding...embedded processors and controllers incritical industriesincreasingly is beingtargeted for exploitation and potentially for

disruption or destruction.

Our information infrastructureincluding...embedded processors and controllers incritical industriesincreasingly is beingtargeted for exploitation and potentially for

disruption or destruction.


8/48


in strengthening the resilience o critical systemsagainst increasingly sophisticated cyber attacks.

Organizations in the water sector have longrecognized that it is neither practical nor easibleto ully reduce the risk o all assets rom natural,

accidental, or intentional damage. However, thewater sectors track record o protecting publichealth and the environment reects an eectiveapproach to managing risk. Tis approachcombines the proper level o risk mitigationmeasures with the most appropriate response andrecovery to adequately achieve acceptable levels osecurity. Building on this approach, the industrys

vision or securing water sector ICS ocuses oncritical unctions o the most critical applicationsthose that, i lost, could result in human healthimpacts, loss o lie, public endangerment,

environmental damage, loss o public condence,or severe economic damage.

A Strategic Framework

Te water sector will pursue the ollowing strategic

goals in an eort to realize the vision o thisroadmap. Tese goals are the essential buildingblocks o an eective risk management strategy.

Develop and Deploy ICS Security Programs.Cross-unctional cyber security teams, includingexecutives, inormation technology (I) sta,ICS engineers and operators, ICS manuacturers,and security subject matter experts, will work

collaboratively to remove barriers and createpolicies that will reduce security vulnerabilities andaccelerate security advances.

Over the next 10 years, utilitiesthroughout the water sector will

have ICS security programs thatreect changes in technologies, operations, standards,regulations, and threat environments.

Assess Risk. Community water and wastewatersystems will have a thorough understanding otheir current security posture, helping them todetermine where ICS vulnerabilities exist andimplement timely remediation.

Within 10 years, the water sectorwill have a robust portolio o ICSrecommended security practice analysis

tools to efectively assess risk.

Develop and Implement Risk MitigationMeasures. When vulnerabilities are identied,risk will be assessed and mitigation measureswill be developed and applied to reduce risk, asappropriate.

Within 10 years, the water sector willhave cost-efective security solutions

or legacy systems, new architecturedesigns, and secured communication methods.

Partnership and Outreach. Close collaborationamong stakeholders and a strong and enduringcommitment o resources will accelerate andsustain widespread adoption o ICS securitypractices over the long term.

Over the next 10 years, water assetowners and operators will be workingcollaboratively with government and

sector stakeholders to accelerate security advances.

The Challenges Ahead

Signicant barriers exist to achieving the goalso the vision or securing ICS in the watersector. Because the requirements to mitigate

vulnerabilities and reduce risk are not ullyunderstood, many I sta and ICS engineersand operators have diculty collaborating onICS security improvements. Few executivesrecognize the reality o ICS security threats

Roadmap Scope

Tis roadmap considers all variables ormitigating vulnerabilities and reducing therisk o industrial control systems in the watersector, including:

Water and wastewater stakeholders andinrastructures

Partnerships

Critical unctions and applications

Near-, mid-, and long-term cyber securityactivities

10-year time rame


9/48


and their growing liabilities. Yet ICS risks arerapidly changing and growing. Te business caseor implementing ICS security has not beenestablished. Tus, the available resources or andocus on ICS security improvements and solutionsare limited. Managing change, such as installing

security patches, is dicult in operating systemsthat have little room or error.

A paradigm shit in management priorities isnecessary to achieve the goals outlined in thisdocument. Many o todays risk mitigationproducts are burdensome and dicult tounderstand.3 Coordination and inormationsharing between industry, government, and ICSmanuacturers is also dicult, primarily becausethe specic roles and responsibilities in thisemerging area are still being dened. Without

due consideration o these and other challenges, areliable, resilient water sector will not be possible inthe uture.

A Call to Action

Te Roadmap to Secure Control Systems in the WaterSectorwill continue to evolve as industry reactsto cyber threat environments, business pressures,

operational constraints, societal demands, andunanticipated events. While it does not cover allpathways to the uture, this roadmap does ocuson what its contributors believe to be a soundramework that addresses the most signicantICS challenges within the next 10 years. However,implementing the needed changes will involve themost dicult and complex steps toward achievingthe desired results. o that end, the industry hasoutlined an industry-managed process to create,launch, and manage ICS security initiatives thatare aligned with this roadmap.

Implementing this roadmap will require thecollective commitment, collaboration, resources,and eorts o key stakeholders (Exhibit 1)throughout the ICS liecycle. Strong leadership,action, and persistence is needed to ensure thatimportant issues receive adequate support andresources. In addition, achieving early successes isimportant to maintaining momentum generatedby the roadmap and convincing asset owners andstakeholders that the control systems securityramework can work. While the precise roles oorganizations in implementing this roadmap havenot yet been determined, they will take shape asthe roadmap is disseminated and reviewed bythose engaged. Te contributors o this roadmapencourage organizations and individuals to

participate in ways that will best capitalize ontheir distinct skills, capabilities, and resources ordeveloping the potential solutions described herein.

A Sustainable Approach

Te risk management planning process mustinclude constant exploration o emergingICS security capabilities, vulnerabilities,consequences and threats.4 Because the ICSsecurity concepts described in this roadmap areintentionally broad based, the specic details

o assessing risk and employing appropriaterisk mitigation strategies will be developed ina technical plan. As the water sector pursues the

strategies contained in the roadmap and technicalplan, it will continue to review, assess, and adjustthe mix o activities that will improve ICS securitytoday and in the uture.

Exhibit 1 Key Stakeholder Groups and Sample Members


10/48



11/48


I. Introduction

Leaders rom the drinking water and wastewaterindustries (water sector) and the government

have recognized the need to plan, coordinate, andocus ongoing eorts to improve industrial controlsystems (ICS) security. Tese leaders concur thatan actionable path orward is required to addresscritical needs and gaps and to prepare the sectoror a secure uture. Teir support helped to launcha public-private collaboration to develop thisRoadmap to Secure Control Systems in the WaterSector. Te roadmap ocuses on the goals andstrategic milestones or improving the security oICS in the water and wastewater inrastructures

over the next decade.Te roadmap content is the result o twomeetings held by members o the Water SectorCoordinating Council (WSCC). Te vision andstrategic ramework were designed by 30 expertsduring a workshop held on September 20, 2007, inSan Jose, Caliornia. Te WSCC Cyber Security

Working Group (CSWG) developed more specicdetails o the roadmap, including milestones,challenges to achieving them, and potentialsolutions, during a meeting held on December

20, 2007, in Washington, D.C. Te roadmapproject was developed by the WSCC-CSWG andjointly sponsored by the American Water WorksAssociation (AWWA) and the U.S. Departmento Homeland Security National Cyber SecurityDivision. For more inormation on the roadmapdevelopment process, please reer to Appendix A.

Roadmap Purpose

Te roadmap builds on existing government andindustry eorts to improve the security o ICS. Itis the culmination o two years o collaborationamong members o the water sector to examineproblems and solutions or ICS security. Tepurposes o this roadmap are as ollows:

Dene a consensus-based ramework thatarticulates strategies o owners and operatorsin the water sector to manage and reduce therisk o ICS.

Produce a broad-based plan or improvingsecurity preparedness, resilience, and response/

recovery o ICS over the next 10 years.

Guide eorts by industry, academia, andgovernment to plan, develop, and implementICS security solutions.

Promote extensive collaboration amongkey stakeholders to accelerate ICS securityadvances throughout the water sector.

Roadmap Scope

Te roadmapcombined with other initiatives

aims to provide a ramework to address the ullrange o needs or mitigating cyber security risk oICS across the water sector. For this roadmap, ICSare dened as the acilities, systems, equipment,services, and diagnostics that provide theunctional control and/or monitoring capabilitiesnecessary or the eective and reliable operation othe water sector in rastructure. While recognizingthe importance o physical protection, thisroadmap ocuses on the cyber security o ICS. Itdoes not specically address the security o other

business or cyber systems, except as they interacedirectly with the water sector ICS. Tis roadmapcovers goals, milestones, and activities over the near(0-1 year), mid (1-3 years), and long term (3-10

years). Security activities encompass recommendedpractices, outreach, training, certications,sotware patches, next-generation technologies,change management, inormation exchange, andimplementation.

Roadmap Organization

Te remainder o the roadmap is organized asollows:

Section II describes the undamental conceptsassociated with the current state o ICSsecurity in the water sector including: (i) themissions and business unctions ICS support;(ii) the major control components used in the

water sector; (iii) the unique attributes o ICSsystems and how they have changed over the


12/48

0 Roadmap to Secure Control Systems in the Water Sector

past decades to meet the sector needs; and (iv)an overview o ICS security risk, including

vulnerabilities, consequences, and reportedcyber events.

Section III discusses the undamental trends

driving ICS security that the water sectormust consider while preparing or the uture,including: (i) business environments; (ii) cybertechnologies; (iii) water operations; and (iv)societal needs.

Section IVdescribes a coherent strategy orachieving the vision and goals o the watersector or securing ICS over the next 10 years,

including: (i) develop and deploy ICS securityprograms; (ii) assess risk; (iii) develop andimplement risk mitigation measures; and (iv)partnership and outreach.

Section Vdescribes a process or turning ideas

into actions and proposes the main roadmapimplementation steps, including: (i) socializeroadmap; (ii) roadmap oversight and projectcoordination; (iii) initiate and implement newroadmap activities; and (iv) sustain eorts.

Section VI provides water sector contacts tond more inormation about this roadmap.


13/48


delivery o nished water. Te water sector usesICS to help manage treatment and distributionoperations and remotely monitor, and sometimescontrol, pressures and ows in water and

wastewater pipelines. In addition, ICS perormdata logging, alarming, and diagnostic unctionsso that large, complicated process systems can beoperated in a sae manner and maintained by acentrally located and relatively small sta.

ICS is a general term that encompasses severaltypes o control systems, including supervisorycontrol and data acquisition (SCADA)systems, distributed control systems (DCS),and Programmable Logic Controllers (PLC).5SCADA systems are highly distributed systemsused to control geographically dispersed assets,

where centralized data acquisition and control arecritical to system operation. In the water sector,they are used in water distribution and wastewatercollection systems. A DCS is a control architecturethat supervises multiple, integrated sub-systems

responsible or controlling the details o a localizedprocess, such as water and wastewater treatment.PLCs are computer-based solid-state devicesthat control industrial equipment and processes.Because the dierences in these control systemscan be considered subtle or the scope o thisdocumentwhich ocuses on the integrationo cyber security into these systemsSCADA

Industrial Control Systems (ICS)Monitor and/or Control a

Water System

Industrial control systems are computer-based acilities, systems, and equipment usedto remotely monitor and/or control criticalprocess and physical unctions. Tese systemscollect data rom the eld, process and displaythis inormation, and then, in some systems,relay control commands to local or remote

equipment.

Industrial Control Systems (ICS)Monitor and/or Control a

Water System

Industrial control systems are computer-based acilities, systems, and equipment usedto remotely monitor and/or control criticalprocess and physical unctions. Tese systemscollect data rom the eld, process and displaythis inormation, and then, in some systems,relay control commands to local or remote

equipment.

II. Industrial Control Systems Use inthe Water Sector

A clean, sae, and reliable water supplyandthe water system that delivers itis at the hearto everyday lie. Humans need water to survive.Businesses rely on water to operate and createproducts. Critical inrastructures, such as energy,transportation, and ood and agriculture, dependon the water inrastructure or sustaining the owo crucial goods and services. In addition, properlytreated wastewater is vital or preventing diseaseand protecting the environment. Saeguardingthe water sector against accidental impacts and

purposeul attack is paramount. Any prolongeddisruption o water supply could be devastating tothe American people and the U.S. economy.

Te water sector has a long and successul historyo protecting public health and the environment.Many o the measures necessary to saeguard the

water supply are in place to address unintentionalcontamination rom natural disasters. Over the lastew years, the water sector has also implementedadditional measures to protect its inrastructurerom deliberate attacks, such as physical assault,

intentional contamination, and cyber intrusion.Improving water service and sustainability, whilemaintaining aordability, has led to an increaseddependence on I. Nearly all eorts to enhanceoperations, reduce costs, and improve overall returnon investments rely on an I inrastructure, whichsupports a utilitys vital assets and unctions. Whilethe use o I systems within ICS architectures hascreated huge gains in reliability and productivity,they have also made the sector increasingly

vulnerable to malicious cyber attack.

Supporting Missions and BusinessFunctions

Water sector utilities depend on ICS tosuccessully carry out their missions and businessunctions. Tese systems allow or the monitoringo source water, continuous control o thetreatment processes, and the high quality and


14/48


systems, DCS, and PLC systems will be reerredto as ICS unless a specic reerence is made to one(e.g., eld device used in a SCADA system).

Major Control Components

ICS components comprise a central control stationwith one or more host computers, local processors,instruments, and operating equipment. Exhibit2.1 depicts the major components o a typical

ICS in a water treatment and distribution acility.6Te components operate under a prolieration ocontrol loops, human-machine interaces (HMIs),and remote diagnostics and maintenance toolsbuilt using an array o network protocols onlayered network architectures. Te componentscommunicate over short- and long-range channels,including the Internet and public-switchedtelephone networks using traditional cables or

wireless media.

Central Control StationTe brain o any ICS is the central control station.It acts as the master unit, while local processorslocated at remote eld sites usually act as slaveunits. Central control stations utilize one ormore host computers to provide the graphicaldisplays as well as the necessary computationaland networking horsepower. Tey also use datahistorians to log all process inormation within an

ICS. Input/output (I/O) servers are used to collect,buer, and provide access to process inormationrom the local processors. Te sophistication othe central control station varies with the size andlocation o the water system. For example, a largemetropolitan water and wastewater system mayuse modern process control systems to monitorand control their distribution network, the majortreatment plants, and the wastewater collection

systems. In small rural systems, a variety o basicand intermediate control systems technologies maybe in place because the utility does not have theeconomic base o a large system, nor the personnel

with the training to properly maintain advancedcontrol systems.

Human Machine Interface

Operators interact with the system or processthrough the HMI. It allows human operatorsto monitor the state o a process under control,

modiy control settings to change the controlobjective, and manually override automatic controloperations in the event o an emergency. Controlengineers use HMI to congure set points orcontrol algorithms and parameters in the controlsystem. Te HMI also displays process statusinormation, historical inormation, reports, andother inormation to operators, administrators,managers, business partners, and other authorizedusers. Te location, platorm, and interace may

Exhibit 2.1 Components of Typical Industrial Control System in the Water Sector

Source: GAO (07-1036)


15/48


vary a great deal. For example, an HMI could be adedicated platorm in the central control station, alaptop on a wireless local area network (LAN), or abrowser on any system connected to the Internet.

Local Processors

Local processors, such as PLCs, remote terminalunits (RUs), and intelligent electronic devices(IEDs), allow or automatic control o processinstruments and operating equipment. Tesedevices acquire data, communicate to other devices,and perorm local monitoring, processing, andcontrol. Some applications require monitoringdevices to be located at isolated equipment sites,pump stations and wells, or along a distant stretcho pipeline. Te processors are equipped withinput channels or sensing or metering; output

channels or control, indication, or alarms; anda communications port, such as wireless radiointeraces.

Instruments and Operating Equipment

Water and wastewater systems consist omeasurement points that need to be monitoredor optimal process control. Such measurementsare the basis or maintaining reliable storage,treatment, and distribution perormance.

Water sector instruments may provide online

measurements o chlorine, dissolved oxygen, color/turbidity, conductivity, pH, pressure, uid level,ow rate, and other critical elements. However,many tests continue to remain ofine. In moresophisticated systems, sensors communicate withlocal processors to control valve, pump, and mixeroperations. For example, maximum eciency canbe accomplished when pumps are instructed tooperate at o-peak times. Some systems work inconjunction with modeling sotware to instructthe local processor to start or stop pumps inanticipation o changes in demand.

Evolution of ICS and Todays Risks

In the United States, there are approximately160,000 public water systems serving about 250million people and more than 16,000 wastewaterutilities serving more than 225 million people.

Te Water Resources Foundation estimates thatrevenues in the U.S. water industry amount to

more than $150 billion a year. Tough ICS oeredwater utilities numerous benets when theyappeared on the market, ew utilities could aordthem; the systems required specic knowledgeo sotware, hardware, and communicationstechnology and had high capital costs. ICS have

since become more aordable and easier to use,and today most water utilities use ICS or processmonitoring and/or control.2 According to a 2007ARC Advisory Group study, the water sectorspent $214 million on ICS systems in 2006.7 Tatnumber is orecasted to reach more than $275million in 2011.

In todays highly dynamic and expanding digitaleconomy, much o the current water sectorinrastructure and the ICS that operate it arebeing used in ways that were never intended.

Many ICS were designed decades ago with littleor no consideration or cyber security. Increasingconnectivity, the prolieration o access points,escalating system complexity, and wider use ocommon operating systems and platorms have allcontributed to heightened security risks.

Cyber Security Threats

Troughout history, water systems have played aprominent role in political actions and militaryoperations. Te vital role o water in daily lie

and economic activity underscores its importanceto a secure and stable world. Consequently, anydisruption or contamination caused by a cyberevent would generate a great deal o publicity.Vast reservoirs and tens o thousands o mileso aqueducts and pipelines make the U.S.

water sector a challenge to secure. Te elevatedinterconnectivity, accessibility, and use o ICSurther expose these critical assets. As a result, the

water sector is vulnerable to potential cyber attackor natural disasters.

Evidence suggests that contamination o U.S.water supplies through cyber event ailures couldproduce signicant public health and economicconsequences. Experience with naturally occurringcontamination events has demonstrated thatcosts to the community may be considerable. Forexample, the 1993 Cryptosporidium outbreak inMilwaukee, Wisconsin, caused illness in more than


16/48


Water Sector Industrial Control Systems Risk Today

Some o the most serious constraints in design and changes in how ICS are currently used in thewater sector include:

Design Limitations. Historically, ICS have been designed or productivity and reliability;

as a result, cyber security was not considered. In addition, limited computing resources haveconstrained the control systems ability to perorm additional security unctions. Although olderlegacy systems may operate in more independent modes, they tend to have inadequate passwordpolicies and security administration, no data protection mechanisms, and inormation links thatare prone to snooping, interruption, and interception. Tese legacy ICS have very long servicelives (about 20 years), and could remain vulnerable.

More Open Environments. In the past, ICS systems operated in airly isolated environmentsand typically relied on proprietary sotware, hardware, and communications technology.Inltrating these systems oten required specic knowledge o individual system architecturesand physical access to system components. o enhance interoperability, architectures andsotware packages became more standardized using commercial o-the-shel technologies; this

elevates system accessibility to potential cyber attack. Increased Connectivity. odays operating needs have created a technology convergence o

physical and cyber inrastructures. Automation has increased due to the need to improveoperational eciencies and workorce shortages. ICS are increasingly connected to a companysenterprise system, rely on common operating platorms, and are accessible through theInternet. While these changes improve water system operability, they have also created serious

vulnerabilities because there has not been a concurrent improvement in security control systemseatures.

Increased Complexity.Te demand or real-time business inormation has increased systemcomplexity: access to ICS is being granted to more users, business and control systems areinterconnected, and the degree o interdependence among inrastructures has increased.

Dramatic dierences in the training and concerns o those in charge o I systems and thoseresponsible or control system operations have led to challenges in coordinating networksecurity between these two groups.

System Accessibility. Even limited use o the Internet exposes ICS to all o the inherentvulnerabilities o interconnected computer networks (e.g., viruses, worms, hackers, andterrorists). In addition, control channels use wireless or leased lines that pass throughcommercial telecommunications acilities, providing minimal protection against orgery o dataor control messages. Legacy systems oten allow back-door access via connections to third-party contractors and maintenance sta.

Supply Chain Limitations.Tere are ew manuacturers o sotware, hardware, and ICS or thewater sector. A disruption in the ICS supply chain could interere with a utilitys response to aailure in the ICS.

Inormation Availability. Manuals and training videos on ICS are publicly available, and manyhacker tools can now be downloaded or purchased on the Internet and applied with limitedsystem knowledge. Attackers do not have to be experts in ICS operations.


17/48


How Can Cyber Events Aect Water Systems?Cyber events can aect water system operations in a variety o ways, some with potentiallysignicant adverse eects in public health. Cyber events could do the ollowing:

Interere with the operation o water treatment equipment, which can cause chemical over-or under-dosing

Make unauthorized changes to programmed instruction in local processors to take controlo water distribution or wastewater collection systems, resulting in disabled service, reducedpressure fows o water into re hydrants, or overfow o untreated sewage into publicwaterways

Modiy the control systems sotware, producing unpredictable results

Block data or send alse inormation to operators to prevent them rom being aware oconditions or to initiate inappropriate actions

Change alarm thresholds or disable them

Prevent access to account inormation

Although many acilities have manual backup procedures in place, ailures o multiplesystems may overtax sta resourceseven i each ailure is manageable in itsel

Be used as ransomware

How Can Cyber Events Aect Water Systems?Cyber events can aect water system operations in a variety o ways, some with potentiallysignicant adverse eects in public health. Cyber events could do the ollowing:

Interere with the operation o water treatment equipment, which can cause chemical over-or under-dosing

Make unauthorized changes to programmed instruction in local processors to take controlo water distribution or wastewater collection systems, resulting in disabled service, reducedpressure fows o water into re hydrants, or overfow o untreated sewage into publicwaterways

Modiy the control systems sotware, producing unpredictable results

Block data or send alse inormation to operators to prevent them rom being aware oconditions or to initiate inappropriate actions

Change alarm thresholds or disable them

Prevent access to account inormation

Although many acilities have manual backup procedures in place, ailures o multiplesystems may overtax sta resourceseven i each ailure is manageable in itsel

Be used as ransomware

400,000 persons, and was estimated by the Centeror Disease Control (CDC) to cost a total o $96million, including over $31 million in lost wagesand productivity.8

Maintaining consumer condence is an ongoing

challenge or the water industry, even withouthaving experienced an attack. Despite the actthat U.S. water companies and utilities maintainsome o the highest quality public drinking waterin the world, a cyber attack on one portion o the

water supply could erode public condence in thesaety o drinking water across the country. Forexample, a disaster preparedness drill conductedin Caliornia in 2000 almost caused widespreadpanic throughout the state.9 Ater simulatingthe destruction o the Lake Nacimiento Dam,management had to quickly respond through mass

media to counter the subsequent panic.

Water supplies do not actually have to becontaminated or disruption to occur. Hoaxes orthreatened incidents o contamination can poseconsiderable management and response challengesor water utilities and political leaders. Forexample, when the village o Orwell, Ohio, received

a threat against its water supply in November2004, local leaders advised citizens not to use theirtap water or consumption while the incident wasbeing investigated.10 Village employees directlycontacted more than a thousand homes in theaected area via phone or by paper notice. Te

incident occurred over the Tanksgiving holidayand created huge demands on a small communitydespite being a hoax.

At the 2008 SANS SCADA Security Conerence,U.S. Central Intelligence Agency senior analyst

om Donahue announced that We haveinormation, rom multiple regions outside theUnited States, o cyber intrusions into utilities,ollowed by extortion demands. We suspect, butcannot conrm, that some o these attackershad the benet o inside knowledge. We have

inormation that cyber attacks have been usedto disrupt power equipment in several regionsoutside the United States. In at least one case,the disruption caused a power outage aectingmultiple cities. We do not know who executedthese attacks or why, but all involved intrusionsthrough the Internet.13


18/48


Real Cyber Events

Reported cyber attacks and unintentional incidents involving the water sector demonstrate thepotential impact o a cyber event. Te ollowing incidents illustrate the consequences o real cyberevents:

Insider hacks into sewage treatment plant (Australia, 2001)A ormer employee o thesotware developer repeatedly hacked (46 occasions) into the SCADA system that controlled aQueensland sewage treatment plant, releasing about 264,000 gallons o raw sewage into nearbyrivers and parks.6

Equipment malunction at water storage dam (St. Louis, MO, 2005)Te gauges at the SaukWater Storage Dam read dierently than the gauges at the dams remote monitoring station,causing a catastrophic ailure which released one billion gallons o water.6

Intruder plants malicious sotware in a water treatment system (Harrisburg, PA, 2006)Aoreign hacker penetrated security o a water ltering plant through the internet. Te intruderplanted malicious sotware that was capable o aecting the plants water treatment operations.6

Reported Vulnerability (Aurora 2007)CNN reported a control system vulnerability that coulddamage generators and motors. 11

Intruder sabotages a water canal SCADA system (Willows, CA, 2007)An intruder installedunauthorized sotware and damaged the computer used to divert water rom the SacramentoRiver. 12

CIA Conrms Cyber Attack Caused Multi-City Power Outage (New Orleans, 2008)CIAhas inormation that cyber intrusions into utilities (ollowed by extortion demands) have beenused to disrupt power equipment in several regions outside the United States.13


19/48


III. Future Trends and DriversInuencing Industrial ControlSystems Security

short-term horizon o elected ocials with thelong-term nature o utility management decisionscan be a daunting task. urnover o elected andother community leaders requires constant re-education eorts. Water sector utilities are ahidden inrastructure, which causes residentsto undervalue the service provided. Seasonal

weather patterns (e.g., summer and winter) andclimatic extremes (e.g., oods and droughts) createuncertainties in water demand patterns, making

both short- and long-term investment strategiescomplex and dicult. As such, the water sectormay continue to struggle with generating andsustaining support o a governing body, privateindustry, and the general community or cybersecurity.

Te dynamic cyber environment challenges theability o water utilities to combat new threats. Asbusiness environments, cyber technologies, wateroperations, and societal needs continue to reshapethe cyber security landscape, the security postureo the water sector will be increasingly challenged(see Exhibit 3.1). Without consideration o uturetrends and drivers, the water sector could beunprepared or the ormidable challenges ahead.

Business EnvironmentsSeveral dynamics within a water systemscommunity place signicant strategy demandson utility executives who must work eectively inthese contexts. For example, the cultural resistanceto change is dicult to overcome. Aligning the

Exhibit 3.1 Future Trends and Drivers Inuencing Industrial Control Systems (ICS) Security

BusinessEnvironment

Ability to change is slow due to scal constraints

Changing weather patterns create cyclic demand on water production

Increasing need or real-time business inormation Increasing convergence o inormation and operations technologies

Aging workorce, sta turnover, and reduction in experienced personnel

Cyberechnologies

Changing and growing ICS threats and accidents

Accelerating pace o change in threat sophistication and the resulting impact oattack rom these adversaries

Increasing use o electronic and wireless communications

Increasing use o open, non-proprietary systems

WaterOperations

Increasing need or aster operational response

Growing control and monitoring needs

Increasingly stringent water regulations increase instrumentation and monitoringrequirements

Competing capital investments, such as upgrading an aging inrastructure

Societal Needs Maintaining public condence in water quality

Growing population and expanding water scarcity


20/48


Te increasing need or real-time businessinormation, driven by the need to reduce costs,increase water distribution eciencies, and comply

with operational and nancial regulations, willrequire a new approach to I management, such assystem consolidation and integration (Exhibit 3.2).

For example, ICS will increasingly operate withdata and business systems to support emergingmanagement unctions. Eorts to seamlesslyintegrate these systems will also shape ICS securitypractices. In addition, the aging workorce israpidly reaching retirement. Many key positionsare not expected to be lled.2 o do more withless sta, utilities are installing additional controlsystems to operate the assets, take readings, andrecord condition-monitoring data. rainingprograms will need to increase to educate operators

on these newly installed technologies, and theymust occur more regularly to address turnover inexperienced sta.

Cyber Technologies

Te threat environment is changing and growing.2A 2003 American Water Works AssociationResearch Foundation (AwwaRF) report oundmore than 100 cases o actual, threatened,and disrupted plots to contaminate watersupplies. O those cases, 20 incidents involvedactual contamination events, more than hal o

which occurred in modern water supplies withpressurized pipe distribution.14 While ew o thesethreats are attributed to cyber attackers, mostexecutives, government ocials, and vendors donot ully appreciate the potential threat that existsto the water inrastructure due to the risks createdby vulnerabilities in control systems technologies.

Computer attackers are constantly looking ornew targets, and criminal extortion schemeshave already occurred.15 In December 2006, anautomated control systems vulnerability scanner

was released, allowing individuals outside theutility with relatively little experience in control

systems to quickly identiy vulnerabilities. In arecent computer industry paper, experts agreedthat attackers are orming a hacking industry, anunderground economy that exploits control system

vulnerabilities or economic gain. Raimund Genes,chie technical ocer o rend Micro, estimatesthis underground digital economy generated morerevenue than the $26 billion that legitimate securit

vendors generated in 2005.16 Te need or cybersecurity is real and is no longer about blockinghackers or updating anti-virus sotware to ensure

that systems are unctioning properly. Cyber threatare becoming more sophisticated and as new threatare introduced, the water sector must rapidly evolv

Central control stations are increasinglycommunicating with remote process controllers

via the Internet and wireless networks. Furtherintegration o shared telecommunicationstechnologies into normal business operations hasspawned increased levels o interconnectivity amoncorporate networks, control systems, and the outsid

world. Continued expansion o the U.S. water

sector has created still greater reliance on publictelecommunications networks to monitor andcommunicate with those growing assets. o achievhigher levels o interoperability among various Itechnologies, the water sector is shiting towardmore open, non-proprietary systems. Increasinginterconnectivity and openness exposes networkassets to potential cyber inltration and subsequenmanipulation o sensitive operations in the watersector.

Exhibit 3.2 Integration of Industrial Control Systems with Business Systems


21/48


Exhibit 3.3 Example of a Large Water Systems Future Sensor Needs

Water Operations

o minimize public exposure to contaminants orservice disruptions, while providing additional timeto evaluate the nature and severity o an abnormalevent, operational response time requirements are

increasing. New online contaminant monitoringsystems will target an average up-time o atleast 99.9 percent, or a mean time ofine o nomore than 10 minutes.17 Population growth,combined with an increasing number o regulatedcontaminants, will greatly expand and complicate

water systems. Tis complexity will lead to anexponential increase in the number o sensorsand telemetry sites that will be needed to supportadditional monitoring and control throughout thesystem. Exhibit 3.3 illustrates the uture sensorneeds o a large community water system, the

South Florida Water Management District, whichcurrently operates and maintains approximately1,800 miles o canals and levees, 25 majorpumping stations, and about 200 large and 2,000small water control structures.18 For the next 10

years and beyond, the water sector must make asubstantial reinvestment in inrastructure to replace

worn-out drinking water pipes and associatedstructures (valves, ttings, etc.). Te AWWAprojects expenditures o $250 billion over 30 yearsmay be required nationwide. Competing capitalinvestments will aggravate already overstretchedresources in the water sector and potentially limit

the implementation o ICS security solutions.

Societal Needs

Te public today wants a water utility, whetherpublic or private, to be aware and responsive totheir concerns. In the uture, successul waterutilities must also anticipate those concerns andbe prepared with accurate acts and inormation.

Tese issues are increasing customer serviceprograms, including community outreach,educational programs, and establishment o

innovative rate structures. In addition, populationgrowth combined with source water limitations areelevating public awareness o the need to conserve,reuse, and recycle water. Tis heightened awarenessis increasing the need or utilities to implementand promote both internal and external watereciency programs, which urther increases the useand complexity o ICS.

Source: South Florida Water Management District


22/48



23/48


In 10 years, industrial control systems orcritical applications will be designed, installed,and maintained to operate with no loss ocritical unction during and ater a cyber event.

In 10 years, industrial control systems orcritical applications will be designed, installed,and maintained to operate with no loss ocritical unction during and ater a cyber event.

Security measures that ensure the availability osae drinking water, wastewater treatment, andthe delivery o vital services, such as re ghting,continue to be a top priority or the water sector.

While much security work has ocused on physicalsecurityences, guards, intrusion detection, etc.eorts pertaining to the resiliency o industrialcontrol systems (ICS) have become more urgent.Advances in securing ICS must go ar beyondthe pressing security concerns o today by takinga comprehensive approach that prepares or theneeds o tomorrow.

Water sector utilities will need to understand andmanage ICS risks, secure their legacy systems,conduct vulnerability assessments, apply securitytools and practices, and consider next-generationsystemsall within a publicly transparent andcompetitive business environment. Governmenthas a large stake in the process because nearly allcritical inrastructures depend on a reliable, sae,and clean water supply. Any sustained disruptioncould endanger public health and saety. However,ICS security must compete with other investmentpriorities, such as in rastructure repairs, upgrades,and expansion. A coordinated strategy thatlinks and integrates the eorts o industry andgovernment is needed to achieve mission-criticalgoals. Tis concept maniests itsel in the watersectors vision statement and goals.

Vision

Based on sound risk management principles, thewater sector has developed the ollowing uniedvision or ICS security:

Te vision emphasizes critical applications,because it is neither practical nor easible toprotect all o the water sector assets rom cyber

IV. A Framework for Securing ICS in theWater Sector

events. Many o these assets are not threat targets,some are not vulnerable, and some would notcreate serious consequences i disabled. Watersystems in the U.S. vary according to size, source,treatment, and geography. Tese systems aretremendously diverse, ranging rom very small,privately owned systems (such as mobile homeparks) to huge, publicly owned systems servingmillions o people. Reservoirs may contain a ewmillion to several hundred billion gallons o water,

making it logistically dicult to contaminatethem with sucient quantities o toxins to cause

widespread illness. Across the U.S., there arealmost three million miles o distribution pipesand collection lines. While there is concern aboutthe vulnerability o distribution systems, thesenetworks were designed to withstand some losso capability without loss o critical unction. In

Vision Terms Dened

Critical Applications: ICS or criticalapplications include components and systemsthat are indispensable to the sae and reliableoperation o the water system. Criticality oan application is determined by the severityo consequences resulting rom its ailure orcompromise. Such components may includecontrols or operating pumps or managingpipeline pressure.

Cyber Event: A cyber event occurs when aterrorist attack, other intentional act, naturaldisaster, or other hazard destroys, incapacitates,or exploits all or part o a control systemnetwork with the potential to cause economicdamage, casualties, public harm, or loss o publiccondence.

Loss o Critical Function: A critical unctiono a water system is any operation, task, orservice that, were it to ail or be compromised,

would produce major saety, health, operation, or

economic consequences.


24/48


addition, water treatment can reduce the risko conventional microbial contamination. Byocusing on ICS systems or critical applications toprevent loss o crucial unctions, the water sectorcan develop strategic goals and milestones thateectively protect the public, customers, assets, and

shareholders.

ICS Security Goals

Realization o the vision requires concerted andocused eorts. Te water sector has developed and

will pursue a set o strategic goals articulating theseambitions. Tese goals will help ocus securityactivities to accelerate progress in achieving the

vision. As shown in Exhibit 4.1 and describedbelow, a ramework emphasizing a desired endstate and aggressive set o milestones will provide

a sound oundation or uture cyber securityinitiatives.

Develop and Deploy ICS Security Programs.Executives will recognize that ICS securityis critical to ullling mission-critical goals.Cross-unctional ICS security teams, includingexecutives, I sta, ICS engineers andoperators, ICS manuacturers, and securitysubject matter experts, will work collaborativelyto remove barriers and create policies that willreduce security vulnerabilities and accelerate

security advances. Over the next 10 years,utilities throughout the water sector will haveICS security programs that reect changes intechnologies, operations, standards, regulations,and threat environments.

Assess Risk. Community water and wastewatersystems will have a thorough understandingo their current security posture to determine

where ICS vulnerabilities exist and implementtimely remediation. Vulnerability assessments

will be integrated into cyber security plans.

Security improvement perormance will bemeasurable and consistent. By 2018, thesector will have a robust portolio o ICSrecommended security practice analysis toolsto eectively assess risk.

Develop and Implement Risk MitigationMeasures.When vulnerabilities are identied,protective measures will be developed and

applied, as appropriate. Risk can be urtherreduced by adding multiple layers o securityand redundant components. Control systems

will be capable o sel-diagnosis and real-timemonitoring and alerting, while being easy tomaintain and update. Within 10 years, the

sector will have cost-eective security solutionor legacy systems, new architecture designs,and secured communication methods.

Partnership and Outreach. Close collaborationamong stakeholders and a strong and enduringcommitment o resources will accelerateand sustain widespread adoption o ICSsecurity practices over the long term. Federalstakeholders will maintain ICS threat support.Inormation sharing will be adequate andtimely within the water sector, among critical

inrastructures, and between governmentagencies. Over the next 10 years, waterasset owners and operators will be workingcollaboratively with government and sectorstakeholders to accelerate security advances.

Tese goals provide a logical ramework ororganizing the collective eorts o industry,government, and other key stakeholders to achievethe vision. o be successul, however, specicmilestones must be accomplished in the 2008-2018period. Projects, activities, and initiatives that result

rom the roadmap should be tied to the milestonesshown in Exhibit 4.1.

Strategies for Securing ICS

Strategies or accomplishing the our goalspresented in Exhibit 4.1 are summarized inExhibits 4.2 through 4.7. Each goal presentsdistinct obstacles that must be overcome, requiresspecic achievements on an established timetable,and recommends potential solutions. Te rapidpace o change in cyber technologies combined

with uncertainties in markets, regulations, andrisk require that the water sector stay vigilant andresponsive to a variety o plausible utures. As the

water sector pursues the strategies contained in thisroadmap, it must review, assess, and adjust the mixo activities that will lead to success today and inthe uture.


25/48


Exhibit 4.1 Strategy for Securing Industrial Control Systems (ICS) in the Water Sector


26/48


Defense-In-Depth

Deense-in-depth is a technique o layeringsecurity mechanisms so that the impact o amechanism ailure is minimized.

Defense-In-Depth

Deense-in-depth is a technique o layeringsecurity mechanisms so that the impact o amechanism ailure is minimized.

Goal: Develop and Deploy ICS SecurityPrograms

Water sector organizations operate in a highlycomplex and interconnected world using Isystems and ICS. Organizations depend on both

o these systems to accomplish their missions andto carry out their business unctions and industrialoperations. Explicit management decisions arenecessary in order to balance the benets gainedrom the use o I and ICS systems with theoverall risk. Managing risk is not an exact science.

o secure I and ICS systems, risk managementmust bring together the best collective judgmentso the individuals responsible or the strategicplanning and day-to-day operations o the entireorganization.

Managing organizational risk related to Iand ICS systems begins with a undamentalcommitment by senior leadership in theorganization to make I and ICS security arst-order mission/business requirement. Tiscommitment ensures that sucient resourcesare available in the design, development,implementation, operation, and disposition oI and ICS systems to provide adequate levelso security, while meeting critical unctionexpectations. I and ICS security must beconsidered a strategic capability and an enabler

o missions and business unctions across theorganization. Cross-unctional ICS security teams,including executives, I sta, ICS engineers andoperators, and security subject matter experts, must

work collaboratively to remove barriers and createpolicies that will reduce security vulnerabilitiesand accelerate security advances. o adequatelyreect rapid changes in technologies, operations,standards, regulations, and threat environments,the utilities throughout the water sector musthave ICS security programs that are reviewed and

updated regularly.An overview o the challenges, milestones, andpotential solutions or developing and deployingICS security programs in the water sector is shownin Exhibit 4.2.

Challenges

Te complexity, diversity, and multitude omission, business, and operation unctions withinan organization require an organization-wideapproach to managing risk. However, obtaining an

organization-wide perspective by all authorizingocials and senior leaders is a complex task.Strong commitment, direct involvement, andongoing support do not exist rom senior leadersbecause they are unaware o the magnitude o ICSsecurity risk. Te lack o an established businesscase or implementing ICS security has alsokept executives rom developing security policiesthat integrate I with ICS security, and rominstitutionalizing these policies into the overallmanagement structure.

Tere is a long-standing I paradigm o oneapplication running one server, owned by one plantor division. Silosinternal divisions such as plants,I, distribution systems, and operationsexistbecause each o those disciplines has become

very complex. Each silo has dierent objectives,needs, and levels o expertise, which can hindercollaborationespecially between I and ICS,

which have dierent security requirements, such asdown time (i.e., periodic versus zero).

Legacy systems oten have constrained resourcesand lack security unctions. ICS componentsmay not have the computing resources neededto retrot these systems with current securitycapabilities. In addition, one single security productor technology cannot adequately protect an ICS.Doing so requires a combination o properlycongured security controls and eective securitypolicies. An eective cyber security strategy oran ICS should apply deense-in-depth, but thisstrategy is not well coordinated between vendorsand users in the water sector system. Also, the

water sector represents a small minority o the ICS

market, which provides little incentive or vendorsto pursue security activities specic to the watersector.


27/48


Potential Solutions

Eectively integrating security into an ICS willrequire the development and implementationo activities such as educating executives,dening and executing cyber security practices,

and ongoing assessment or improvement.Te industry must rst identiy recommendedpractices, such as connecting ICS and businessnetworks. Recommended practices can provideinsight on quick x solutionslow-cost, high-

value practices that can signicantly reduce riskin the short termand on long-term solutionsto sustain security improvements. One quickx or managing complexity has already beenidentied as a near-term milestone: the isolationo ICS rom public-switched networks, includingcable modems, direct dial modems, open 1s, and

Internet access. Tis will signicantly decreaseopportunities or exploitation and improve thesecurity posture o the inrastructure.

Security awareness is critical to obtaining buy-inrom executives, I personnel, and the vendorcommunity. Te industry needs an ICS securitymarketing strategy that includes socializing andcollaborating with executives, I, and operations.

Te state o Caliornia oers a model outreachprogram, which can be replicated in regionsacross the U.S. (Reer to Section VI or contact

inormation.) o establish a business case orICS security throughout the water sector, theindustry must develop a white paper that analyzesthe incentives and benets o implementing ICSsecurity. In addition, the Process Control SystemsForum is a venue or developing partnerships with

vendors.

Successully managing risk may necessitatereengineering the processes used to accomplishmissions and execute business unctions. Bypurposeully integrating I and ICS security into

the execution o missions and business unctions,system operators can signicantly reduce risk

without adversely aecting operation.

Perormance metrics are needed to help ensurethat ongoing ICS security eorts are conductedconsistently across the organization, as well as

the entire sector. By establishing perormancemetrics, organizations can determine the degreeto which security integration is occurring andmeasure progress. Automated collection o ICSsecurity inormation, including incident reportsand visualization tools or correlation purposes,

will help accelerate ICS security eorts nationwide.o sustain these eorts, the sector should evaluatethe needs to integrate this roadmap into the WaterSector Specic Plan.


28/48


Exhibit 4.2 Strategies for Developing and Deploying Industrial Control Systems (ICS) Security Programs


29/48


Goal: Assess Risk

Risk analysis is the process through which thethree components o riskthreat, vulnerability,and consequencewill be collectively analyzed todetermine the water sectors cyber security posture.

For ICS, an important aspect o risk assessment isdetermining the value o the data that is owingrom the control network to the corporate network.In some situations, the risk may be physical orsocial rather than purely economic. Te risk mayresult in an unrecoverable consequence ratherthan a temporary nancial setback. Eective riskassessments clearly delineate the mitigation costcompared to the eects o the consequence.

An accurate risk assessment o critical ICS assetsenables water sector stakeholders to prioritize

security needs and ocus limited resources onthe most urgent security issues. Risk assessmentdata are also necessary to build a sound businesscase or investment in creating, procuring, andimplementing ICS security measures. Conductingthese assessments or community water and

wastewater systems requires a robust portolio oICS security recommended practice analysis tools.

An overview o the challenges, milestones, andpotential solutions or assessing cyber risk in the

water sector is shown in Exhibit 4.3.

Challenges

Assessing ICS risk remains dicult due to alack o sucient analysis and measurementtools. Treats, when known, are oten hardto demonstrate and quantiy in terms thatare meaningul or decision makers. New

vulnerabilities can be introduced when businessor inrastructure networks increase connections

with ICS networks, and when new technologiesare integrated into the ICS. Te highly dynamicthreat environment, combined with the rapid paceo change in cyber technology, creates a signicantdilemma. In an industry amiliar with inertia,rapid identication o new vulnerabilities will bechallenging.

A cyber event in the water sector can produce acomplex web o consequences that spans manysectors o the economy and reaches well beyond

the individual or community experiencing theevent. Te consequences o a loss o publiccondence in a utility are oten overlooked;however, it is a real target or adversaries thatcould be accomplished through an ICS incident.

ogether, these actors present real challenges to a

managers ability to dene the magnitude o harmresulting rom a cyber event.

Inadequate assessment capabilities limit theability o companies to accurately dene ICSsecurity requirements. As a result, clear and up-ront requirements do not exist. Because someexecutives do not understand what is required,resources to initiate risk assessment activities arenot made available, leaving the development andimplementation o risk assessments at a stand still.

Potential Solutions

Near-term activities include developing a riskmatrix that reects consensus on how to rameand dene critical vulnerabilities and matchthem with appropriate mitigation strategies. Riskassessment toolssuch as end-to-end, threat-

vulnerability-consequence analysis and evaluationo cyber attack and response simulatorsshouldbe developed. Creating ICS risk assessment andreporting guidelines, as well as common metrics orbenchmarking ICS risks, is essential to acilitating

cost-eective and consistent assessments acrossthe water sector. Although sel-assessment toolsare currently available, there are uncertaintiesabout their use. Enhancing the risk assessmentmethodologies, rameworks or prioritizing controlmeasures, and cost justication tools will greatlyenhance the water sectors condence in theaccuracy and useulness o these tools.

Adoption o industry-approved incident reportingguidelines and recommended practices willincrease data on all aspects o risk and enable

the development o more accurate analysis andmodeling tools or assessing it. In addition,sharing lessons learned means that a companyis more likely to have the knowledge required torespond quickly to ICS emergencies, even whenappropriate security measures are not available.Sector-wide training on these tools will urtherenhance risk analysis capabilities across the sector.


30/48


In the long-term, establishing perormance metricsto benchmark within and across other sectors willbe essential to building an assurance case or cybersecurity programs.

Goal: Develop and Implement RiskMitigation Measures

It is impracticali not impossibleto ensurethat an ICS is 100 percent secure at any pointin time. Tereore, organizations seek to managerisk in order to achieve acceptable levels osecurity. Managing risk rom ICS systems tooperations, assets, individuals, other sectors, or thenation requires a holistic approach, such as theRisk Management Framework (Exhibit 4.4.).4

Exhibit 4.3 Strategies for Assessing Industrial Control Systems (ICS) Risk

Te ramework represents a cyber security liecycle that acilitates continuous monitoring andcontinuous improvement in the security state othe ICS systems as well as the overall resiliencyo the water sector organization. Tis approachcan provide the utility with enough exibility toquickly apply the proper level o risk mitigation

measures to the most appropriate ICS systemsto adequately protect the critical missions andbusiness unctions o the water sector organization

As ICS vulnerabilities are identied, known riskmitigation measures can be applied and newsolutions developed to meet emerging needs.For legacy systems, these measures oten includeapplying proven recommended practices and


31/48


security tools, implementing procedures andpatches or xing known security aws, creatingtraining programs or sta at all levels, andretrotting security technologies that do not

degrade system perormance. Communicationbetween remote devices and control centers, andbetween I systems and ICS, requires secure links,device-to-device authentication, and eectiveprotocols. However, the most comprehensivesecurity improvements are realized with thedevelopment and adoption o next-generation ICSarchitectures, which are inherently secure and oerenhanced unctionality and perormance. Tesesystems can provide deense-in-depth with built-in, end-to-end security.

Aordability o drinking water is one o the majormissions o a utility. With signicant investmenthurdles acing the water sector, risk mitigationmeasures will be dicult to ully implement atcurrent cost levels. As such, security solutionsor legacy and new architecture designs andcommunication methods must be cost-eective

within the next 10 years.

An overview o the challenges,milestones, and potential solutionsor developing and implementingrisk mitigation measures in the

water sector is shown in Exhibit 4.5.

Challenges

Eectively managing risk requiressignicant resources and eortsrom multiple organizations withinthe utility. For example, closecoordination and collaborationamong knowledgeable individuals(e.g., system architects, systems/security engineers, systemadministrators, physical securityexperts, personnel specialists,

etc.) will ensure that theappropriate personnel, processes,hardware, sotware, rmware,or environmental componentsprovide their designated securityunctionality (e.g., access control,identication and authentication,evaluating and accountability,

system and communications protection, physicalsecurity, personnel security, incident response,contingency planning, etc.). However, limitedresources make this process dicult to manage andcan hinder progress.

Legacy systems are especially vulnerable tocomputing resource availability and timingdisruptions. Many systems do not have desiredeatures including encryption capabilities, errorlogging, and password protection. Integratingnew technologies into these systems is especiallydicult, or even impossible. For example, older

versions o operating systems may no longer besupported by the vendor, making some patchesuseless. ypical next-generation ICS componentshave a lietime o ve years or more. For ICS,

where technology has been developed orvery specic use, the lietime o the deployedtechnology is oten 15-20 years longer.

Change management is paramount to maintainingboth I and ICS systems, yet signicant gaps inthe process remain. Unpatched systems representone o the greatest vulnerabilities. Sotware

Exhibit 4.4 Example of an Industrial Control Sytems Risk ManagementFramework


32/48


Exhibit 4.5 Strategies for Developing and Implementing Risk Mitigation Measures


33/48


updates on ICS cannot always be implementedon a timely basis because these updates need tobe thoroughly tested by the vendor and the utility.ICS outages oten must be planned and scheduleddays or weeks in advance. Water systems operate24/7, which means there is no room or error

when upgrading/patching an online ICS. Te ICSmay also require revalidation as part o the updateprocess. Actionable risk mitigation products thatowners and operators can easily understand anduse are not available. Te lack o collaborationbetween I and operations urther aggravates thechange management process.

Potential Solutions

Managing a cyber event includes preparation,detection and analysis, containment, eradication,

recovery, and outreach. Te industry rst mustcreate a cyber response protocol template (i.e.,guidance manual) that includes a predeterminedset o instructions or procedures to detect,respond to, and limit consequences o incidentsagainst an ICS in the water sector. Because o thecomplexity and multitude o systems, developinga decision-making tool will enable aster responsein balancing automation with manual controlsduring a cyber event. In the long term, automatedsecurity state and response support systems should

be developed. Also, training must be provided toensure security templates and management toolsare properly prepared and implemented.

Te water sector must dene baseline securityrequirements (i.e., undamental, intermediate, andadvanced levels) to establish security unctionality,quality, and assurance (i.e., grounds or condence)o ICS security activities. Tese activities shouldbe ully integrated into each phase o the systemlie cycleinitiation, development and acquisition,implementation, operations and maintenance, anddisposition.

In the near term, the sector must identiy, publish,and disseminate recommended practices, includingones or securing ICS network architectures andor providing physical and cyber security orremote acilities. Water sector vendors should beencouraged to conduct vulnerability assessments atthird party acilities. Tese assessments can expand

the understanding o potential ICS weaknessesand the most eective security practices to mitigatethem. Assessments can leverage cyber securityknowledge to resolve existing vulnerabilities andimprove the design and operation o more secure,more resilient, next-generation ICS.

Maintaining system and inormation integrityassures that sensitive data has not been modied ordeleted in an unauthorized or undetected manner.

While some security controls exist to addresssystem integrity concerns, they are not appropriateor all ICS applications. Cost-eective gatewaysecurity, including rewalls, intrusion detection,and anti-virus protection, must be developed or

water sector ICS. Tese controls should integratewell and have maximum host impacts.

As utilities implement security solutions intotheir cyber systems, they will need to understandthe level o security improvement to justiy andreinorce their value to senior leaders, investors, andcustomers. Te sector should establish metrics thatmeasure the security perormance o implementedsecurity solutions in order to establish a baseline operormance and measure uture progress.

Goal: Partnership and Outreach

Collaborative partnerships will leverages resources

and capabilities among utilities, associations,vendors, communities, government organizations,and others in improving the sectors ability toprepare and respond to cyber events. Combiningthe expertise and perspectives o all acets o thesector ensures that ICS security needs are beingmet and anticipated rom every angle. Additionally,inormation and cost sharing minimizes theduplication o technology development eorts andmaximizes resources to eciently achieve eectivesolutions.

Outreach activities are equally important, as theykeep industry groups across the nation inormedand up-to-date regarding eective strategiesand technologies to mitigate inrastructure risk.

Workshops, training courses, and recommendedpractices increase industry members awarenesso security risks to their own systems. Engagingthese groups through outreach encourages them toquickly implement new risk mitigation measuresand provide input rom the eld to help guide


34/48


uture technology development. A steady streamo communication with ederal entities and thegeneral public will sustain support or utureinvestments in cyber security.

Te uture o ICS security depends on government

and sector stakeholders (Exhibit 4.6) comingtogether to work toward common goals. Tisongoing collaboration will accelerate and sustainICS security advances in the individual utilities,the water sector, and the critical inrastructuresthat rely on a resilient water sector.

An overview o the challenges, milestones, andpotential solutions or conducting partnership andoutreach activities in the water sector is shown inExhibit 4.7.

ChallengesAs an emerging requirement or the water sector,ICS risk management is still somewhat isolated.

Tere is a sense that barriers are moved but notbroken down. Both industry and government arestruggling with how best to initiate ICS security

eorts and are still clariying their respectiveroles and responsibilities in this emerging area.Although multiple eorts are under way tomitigate ICS risk, eective security-orientedpartnerships have been dicult to establish, andpoor coordination and insucient inormationsharing among stakeholders has created conusion.

Outside o the ICS community, there is a poorunderstanding o cyber security issues, theirimplications, and needed actions. It was elt bymembers at the workshop that ederal and utilityresources are not adequately ocuesd on mitigatingICS risk in the water sector.2,3 Widespreadadoption o ICS security across the entire watersector is challenging due to the voluntary nature othe eort.

Potential Solutions

In the near-term, the water sector should conductnational workshops with government and sectorstakeholders. Te contacts and relationshipsdeveloped during these workshops will serve asa valuable resource in understanding the diverseperspectives on how to achieve common goals,

while promoting close cooperation andexchange o ICS security inormation.

Workshops ocused on recommendedpractices will provide a orum or

continuous improvement andacilitate widespread adoption

o these practices. Additionally,ICS security training shouldbe conducted or employeesand contractors.

Establishing and reinorcinga lie-cycle investmentand ramework or ICSsecurity requires an elevatedawareness o ICS security

risk within the water sector,across critical inrastructures,

and among vendor, commercialand government partners.

Eective Federal and stateincentives should be developed to

accelerate investment in secure ICStechnologies and practices. Industry

Exhibit 4.6 Key Stakeholder Groups and Sample Members


35/48


Exhibit 4.7 Strategies for Conducting Partnership and Outreach

associations, such as the American Water WorksAssociation and the Association o Metropolitan

Water Agencies, will need to update governmentand others on a regular basis to maintain ICSsecurity investments or the long term. Te sector

also needs to identiy, understand, and disseminatetimely ICS risk inormation within the sectorand among its partners. o simpliy and expeditethe sharing o ICS security threat inormation,the water sector should work with and leverage aederal program.

Although quantiying levels o awareness orcollaboration is not an easy task, the watersector should establish metrics that measureprogress in this important area. Metrics couldinclude the percent o utilities that have adopted

recommended practices, number o workshopsor training seminars held per year, the number ocommunication products disseminated throughoutthe sector, and the amount o investment in ICSsecurity.


36/48



37/48


V. Implementation

Te Roadmap to Secure Control Systems in the WaterSectorwill continue to evolve as industry reacts

to business pressures, cyber threats, operationalconstraints, societal demands, and unanticipatedevents. While it does not cover all pathways tothe uture, this roadmap does ocus on what itscontributors believe to be a sound ramework thataddresses the most signicant industrial controlsystems (ICS) challenges within the next ten years.As such, it is intended to guide the planning andimplementation o collaborative cyber securityprograms that will involve asset owners andoperators, industry associations, government,commercial entities, and researchers participatingin the national eort to improve security in watersector ICS.

Many water sector organizations have begun towork collaboratively with government agencies,other sectors, universities, and national laboratories,to coordinate eorts to address ICS securityconcerns. Yet, the current level o investment andresources typically alls short o critical needs.By working together to develop this roadmap,

the sector has taken its rst step in ICS riskmanagement transormation. Exhibit 5.1 outlines

the main roadmap implementation steps. Tesesteps are designed to catalyze buy-in with theroadmap, and subsequently launch and manageICS security projects. Strong leadership, action,and persistence is needed to ensure that importantneeds receive adequate support and resources. Inaddition, achieving early successes is important tomaintaining momentum generated by the roadmapand convincing asset owners and stakeholders thatthe ICS security ramework can work.

Socialize Roadmap

While the precise roles o organizations inimplementing this roadmap have not yet beendetermined, these roles will take shape as theroadmap is disseminated and reviewed by thoseengaged. Te roadmap socialization process shouldinclude motivating industry leaders to step orwardand initiate the most time-sensitive projects.

Roadmap Oversight and Project Coordination

Te contributors o this roadmap encourage

organizations and individuals to participate inways that will best capitalize on the

Cyber Security Roadmap 08

Documents

Transcript of Cyber Security Roadmap 08