PATCH MANAGEMENT: Issues and Practical

SolutionsPresented by:

ISSA Vancouver ChapterMarch 4, 2004

Code Red – July 2001 July 19, 2001 – 159 hosts infected

Code Red – July 2001 12 hours later – 4,920 hosts infected

Code Red – July 2001 12 hours later (24 total) – 341,015 hosts infected

SQL SLAMMER WORM

JANUARY 2003 same spread in TEN MINUTES

Slammer was nasty. In the first minute of its life, it doubled the number of machines it infected every 8.5 seconds.

(Just to put that in perspective, the Code Red virus concerned experts because it doubled its infections every 37 minutes. Slammer peaked in just three minutes, at which point it as scanning 55 million targets per second.) [thank goodness there are natural limits to this kind of growth and thank goodness Slammer didn't have a really nasty payload]

Early 2004 Status Update

• Automated attacks are successfully exploiting these software vulnerabilities, as increasingly sophisticated hacking tools become more readily available and easier to use.  

• Since 1995, over 15,000 security vulnerabilities in software products have been reported.  

• Attacks such as viruses and worms that once took weeks or months to propagate over the Internet now take only hours, or even minutes.  

• Patch Management is a critical strategic means of dealing with these increasing vulnerabilities.  

• Requires Management support, standardized policies, minimizing dedicated resources, risk assessment and testing.

Challenges

• What to patch first???• Two myths:

– The threat of attack from insiders is less likely and more tolerable than the threat of attack from outsiders.

– A high degree of technical skill is required to successfully exploit vulnerabilities, making the probability of attack unlikely.

• Threat profile and potential risks continue to increase

• Virus/Worm can now be delivered through common entry points, automatically executed, and then search for exploitable vulnerabilities on other platforms.

Challenges

• New vulnerabilities released daily

• Widespread publicity leads to releases of exploits

• Vendors must provide quick turnaround on patches

Business-Centric Approach

• Patch Management is a Process, not a Tool

• Link Business Objectives to Network Solutions– Quantify value of new initiatives

– Optimize existing infrastructure

– Identify best solutions

– Employ proven best practices and methodologies

– Foster collaborative culture

– Institute formal quality program from outset

Cost of Patching

Cost to Patch =

(Hours x Rate x Systems) + (Patch Failure% x (Hours x Rate x Systems))

So, if it takes an army of $70/hour technicians one hour to patch a system, and there are 2,000 systems, the cost is $140,000. If you estimate that 5 percent of the patches fail, and figure an average of two hours of recovery time (which includes help desk and IT support activities), that's 100 systems at $140 each -- another $14,000.

Another source quotes $234 per patch per desktop for a medium to large US organization

Cost of NOT Patching

• Lost productivity for the end user • Lost productivity for IT support

personnel • Loss of revenue (direct) • Legal/regulatory costs • Intellectual property losses • Loss of stored assets (financial)

What to do: Analysis

Baseline production systems• Gather comprehensive hardware and software inventory• Use the information to define standard software

baselines• Perform an audit to determine deviations from baseline• Install service packs and necessary software updates• An accurate software inventory is vital

• Base lining provides additional benefits that streamline patch management.

• Develop consistent standard software images• Perform risk assessment to identify and assign value to

assets to determine patching priorities

What to do: Analysis

Assess each computer for patches required– Scan for new vulnerabilities

• Automate as much as possible• Occur on a regular basis – daily, weekly

– Promptly notify administrators of new vulnerabilities• Enables faster response and proactive

remediation– Aggregate results across the environment

• Simplifies analysis

What to do: Keep Track

Patch Monitoring and Discovery – Build procedures for monitoring

patches as they are released.– Include monitoring of all appropriate

security intelligence sources required to identify any exposures or vulnerabilities that may impact the organization.

What to do: Test

Most important aspects of patch management• Bugs can occur in all software – patches are no exception• Patches may introduce unintended consequences and

break existing software

Structured Patch Evaluation testing methodology

• Define risks for testing servers and desktops– Usefulness may depend on security policies in place– Optimize based on complexity, resources and time

• Match system configurations of test computers to production computers

• Test vulnerability and system/application stability• Investigate, evaluate and test patches in accordance with

business objectives, security and IT operational goals.

What to do: Distribute

Policy based distribution– More efficient management

• Less administrative overhead• Faster remediation

– Ensures configuration for business continuity• In a 6-12 month period, 20% of computers become unpatched. • Reinstalls software if uninstalled

Targeted Distribution– Flexible targeting based on prioritization– Develop tools and templates to integrate with your change

management policy.– Develop procedures for the patch to go from testing, to

implementation, including updating standard builds as needed.

What to do: Monitoring

Ongoing monitoring• Detailed reporting covering the entire patch

process– Scan results

– Distribution process

– Installation status

Patch Maintenance • Develop tracking and reporting mechanisms• Develop security awareness processes

Benefits

• Proactively identify and remediate IT security vulnerabilities

• Focuses IT and security on the right set of problems to address

• Improved service performance and availability by optimizing business and systems processes

• Adds value to ongoing business initiatives, business continuity, reducing operating costs, and security mandates