THE TAXONOMY OF WISE INFRASTRUCTURE SPENDING€¦ · · 2013-10-30THE TAXONOMY OF WISE...

Network Monitoring, Security and Forensics

Process Query Systems, LLC

Lebanon, NH 03766

(603) 727-4477

www.FlowTraq.com

THE TAXONOMY OF

WISE INFRASTRUCTURE SPENDING

How to Manage, Protect and Control IT Assets for Smooth Infrastructure Operation

By Dr. Vincent Berk

CEO, Process Query Systems, LLC

http://www.linkedin.com/in/vincentberk

[email protected]

http://www.linkedin.com/in/vincentberk

mailto:[email protected]

Copyright © 2011 - Process Query Systems, LLC www.FlowTraq.com of 15


THE TAXONOMY OF WISE INFRASTRUCTURE SPENDING

Manage, Protect and Control IT Assets for Smooth Infrastructure Operation

By: Dr. Vincent Berk

The ultimate goal of any information infrastructure is to smoothly service the needs of the business and enable

the business to be successful and efficient. Information infrastructure plays a critical supporting role to the

business. Maintaining smooth infrastructure operation is an idealized end goal and, like light-speed, the closer

you get to reaching it, the more it will take to get there.

Spending your monetary resources arbitrarily will leave you vulnerable to failures and compromises that prevent

smooth business operation and put your business at risk. In fact, spending a little bit of time evaluating how to

put your limited resources to best use may get you a long way to smooth infrastructure operation, with a great

degree of confidence far into the future.

To make sure you cover all bases, I present a 3x3 taxonomy of infrastructure pain points that will lead to a

natural and comprehensive spending strategy. Depending on the size of your business, the resources available

to you, and importance of your infrastructure assets to your business, you may put more focus in some areas

than others. The three main areas to cover are:

Manage - your network (Monitoring)

Protect - your data (Security)

Control - your damage (Forensics)

In general I recommend spending your resources roughly equally between management, protection, and

damage control. Each of these areas breaks down into three more categories.



1. Manage Your Network

This is all about deployment and monitoring. The ultimate goal is uptime, the "nominal"

state. This means applying updates and patches, checking that your servers and services

are up and running, and monitoring the network for inefficiencies. This area breaks

down into the following 3 boxes:

1.1 Host Application Administration

The applications you implement on top of your infrastructure implement the

processes that make your business run. Whether these are databases,

webservers, specialized software packages or custom in-house programs doesn't

matter. Through the applications you implement your business. Keeping your

applications up-to-date, patched, supplied of sufficient disk space and memory,

and other administrative tasks is the easiest way to keep them running smoothly. This includes keeping

virus scanners and intrusion detection systems up-to-date (see box 2.2).

Frequently updating your operating systems to patch vulnerabilities and avoid problematic bugs is

included in this box. There are many software tools and manual strategies that can help you stay on top

of this task. For instance, a smaller office may simply request that every user turns on Microsoft

automatic updates, and regularly updates locally installed programs. Other solutions include remote

administration tools, which can range from free to very expensive.

If your infrastructure is UNIX-based, it might be easiest to write SSH-based scripts to manage hosts and

servers in the organization. Other options include the use of VNC® or Windows Remote Desktop.

1.2 Network Device Monitoring

The devices that move the packets between your hosts, servers, printers, and the Internet are actually

pretty powerful computers themselves. And monitoring how busy they are, and if they are experiencing

any faults or packet drops, is a key piece of the smooth infrastructure puzzle. Saturated switches or

routers with full CPU utilization will lead to dropped VOIP calls, choppy WebEx sessions, and failed

downloads.

Keeping an eye on these network devices can often be done through a simple and affordable SNMP

monitoring tool. SNMP, the Simple Network Management Protocol, is a widely supported way of

communicating with routers, switches, and other networked devices. Tools like InterMapper®, Orion, or

WhatsUp Gold are easy to deploy and offer affordable monitoring and alerting capability for your

network devices. Free alternatives are also available, and if you are a capable scripter, it may even make

sense to write your own SNMP monitoring tool to get just the data you need.



1.3 Network Traffic Monitoring

Box 1.2 helps you figure out how busy your network devices are. Box 1.3 helps you figure out what they

are busy with. For instance, if the uplink to your ISP is so saturated that you are experiencing poor VOIP

quality, it might make sense to take a look at the traffic traversing this link. Some people in your

organization may be hosting illegal movie content through peer-to-peer sharing programs!

Getting insight into the content of network traffic requires tools that can analyze the traffic. Many

options are available, from free to expensive. Most routers and switches support the exporting of traffic

"flow" records. Many protocols are out there; the most common are

NetFlow/IPFIX

sFlow®

JFlow

CFlow

For an overview of flow technologies, see my related white paper: “The NetFlow/sFlow/CFlow/ JFlow Flow Dilemma ”

Since these protocols are widely supported on most networking hardware, using one of them is often

the easiest path to understanding what your network traffic actually contains.

Flow analysis tools break down into two categories:

1. Aggregators that bunch the data together, and throw the incoming records away; and

2. Full-fidelity flow analyzers that keep every record and allow after-the-fact filtering of traffic.

In general, the aggregators are cheaper, while the full-fidelity analyzers are more powerful. For simply

analyzing top network users, and their top content, you may only need a flow aggregator.

Examples of aggregators are: ScrutinizerTM, ManageEngine, and the Solarwinds NetFlow Traffic Analyzer

(NTA). Full fidelity monitors include: FlowTraq®, InterMapper Flows, and SiLK. Each of these tools

supports a range of different protocols, and is available at a range of different price points.

Alternatively, you can use TCPdump, or Wireshark (or the commercial equivalent, Solera Networks

capture devices) to capture every packet that moves through a particular point in your network.

Analysis then becomes a manual process. Unfortunately, with the rise in digital media content (NetFlix®

movies, YouTube, Skype, VOIP), and the rise in encryption (HTTPS, SSL, TLS) the value of a full packet

capture has been sharply declining. Consider this: you are mostly storing a random collection of bits that

cannot be further analyzed!

http://www.flowtraq.com/corporate/resources/whitepapers/the-netflowsflowcflowjflow-flow-dilemma



2. Protect Your Data

What makes your business run is the information in your data, not the actual

computers on which the data resides. Protecting your data means developing proper

policies, enforcing those policies, keeping malware out of your organization, and

verifying that your protections are effective.

Unlike the other two categories (1 and 3), the money you spend in "protect your data"

will be very product-oriented. You can purchase many solutions, and all will

implement a piece of the data protection puzzle. On the other hand, categories one

and three will often be more expensive in terms of man-hours and expert time.

Again, you are protecting the information that supports and runs your business. You

do this by protecting the infrastructure that the information resides on. This breaks

down into 3 categories:

2.1 Policy Enforcement

The first step to take when protecting your data is to set sensible policies for keeping the data in, the

leakage to a minimum, and the offenders out. Policies generally include guidelines on where data may

reside (for instance, no social security numbers on individual laptops, no unencrypted credit card

information in emails), who has access to it (database and application software login accounts), and

some minimum standards on access control (firewalls) and user authentication (password policies, two-

factor authentication).

Developing a policy often comes down to using some common sense; however, it is easy to overlook

certain key items. Some SANS training or simply using Google for "network security policy" can help you

get a long way to covering the most important bases.

Implementing policy is where you build firewalls, limit user access through account management, and

enforce strong user passwords. In general, firewalls range from free (IPtables) to expensive (Firewall-1,

CISCO ASA), each with their own ease of configuration, management of multiple firewalls, and

complexity and power of filtering abilities.

Other systems in this category can help you limit what websites your users visit. Net Nanny® is the most

popular commercial solution for preventing kids (and co-workers) from visiting a range of websites with

potential offensive content. Depending on your aggressiveness in tweaking these systems, you might

accidently prevent your users from accessing important websites containing sporadic offensive words.

(For instance, what happens if an employee attempts to research the mating behavior of the European

vs. the African swallow??)



You might consider the use of a "DLP", or "Data Loss Prevention" system. These systems inspect passing

traffic for signatures of credit card numbers, social security numbers, or other, user-defined regular

expressions. If data is leaving your network through unmodified channels (i.e. non-encrypted, non-

compressed) these DLP devices may help prevent some data leakage. They cannot; however, catch USB

thumb drives walking out

of the building, or encrypted, or unknown formats being emailed or uploaded (this includes HTTPS

webmail!).

2.2 Malware Repulsion

Access control policies and firewalls only get you part of the way to protecting your data. Preventing

malware infestations is a second key piece that must be properly implemented at some level. Tools in

this category abound, mostly because the range of malware is increasing at a rapid pace. Every month

thousands of malware signatures are added to intrusion detection systems and virus scanners. It is

therefore easy to spend a lot of money in this category.

Unfortunately, malware detection is a difficult battle, and spending too much money on it might cause

you to overlook some of the 8 other important infrastructure spending boxes. Malware is often easily

produced, small (on average 125 lines of code), and can be hidden in many places. Files, emails, thumb

drives, websites, and even your database can all hold malware of various kinds. Detecting it is therefore

a multi-stage affair, and your detection ability is only as good as your signatures are. Failure to regularly

update your virus scanner or Intrusion Detection System will let the newest malware slip right by. What

is worse, at any one time, there are thousands pieces of yet-unknown malware that haven't been

classified.

Good estimates are hard to get, but malware detection vendors work around the clock to find these

new pieces of 'zero-day' badness.

On clients and servers a virus scanner can be deployed. McAfee® and NortonTM AntiVirus are the usual

candidates. Some high-valued servers may also benefit from a critical-file change detection system such

as Tripwire®, which notifies you if key operating system files were changed.

On the network level an email virus scanner and spam detector make sense. Barracuda Networks,

Sophos, and Symantec are all examples of vendors of tools to keep your email safe. ClamAV® and

SpamAssassin are free options for the cost-conscious, but may be harder to install correctly.

Intrusion Detection Systems are traffic inspectors that will notify you if a virus signature was seen in the

passing packets. An Intrusion Prevention System goes one step further and tries to block access to the

offending system once a signature has been matched. PaloAlto, TippingPoint, and StealthWatch are

pricey examples of commercial IDS/IPS systems. Snort® is a popular free variant.



Recently, IPS vendors have been promoting "behavioral detection" abilities, where patterns of access

are also considered in the detection scheme. Systems that suddenly show rapid outbound connection

patterns (that were previously absent) are blocked for "bad behavior." The user community is split in

their appreciation of these abilities, as "false positives" often prevent or delay necessary business

transactions. This often leads to required operator intervention, where the system must be configured

to ignore certain key systems. On the other hand, rapid viral spread can be prevented by these bad-

behavior intrusion prevention systems.

Finally, the malware category sees a number of active vulnerability scanners that scan your network for

any host or server that may have network-reachable vulnerable software running. These vulnerability

scanners contact each computer in your network and attempt a range of thousands of remote attacks to

see if any break through. After a scan, you are presented with a list of known vulnerabilities in your

network. Core Impact is a powerful commercial scanner, while OpenVAS (formerly Nessus) is a free

variant in the open-source community.

2.3 Security Audit

The third box in the "secure your data" category is the audit. In my experience this is often the most

neglected part of security. Dozens of times have I seen companies implement a firewall to block traffic,

only to realize months later that the traffic they were trying to block was traversing an alternate path in

the network. What good is implementing a policy if you neglect to check that your implementation is

working?

Audit means both visibility and understanding. You cannot audit what you cannot see, and you cannot

audit what you don't understand. This is especially important in the regulated industries. For example,

if you store medical records, you are bound by the HIPAA law. Being on top of who, when, where, and

what accesses the medical records in your organization is not a luxury, it is a legal requirement. You will

go to jail if you don't protect medical information sufficiently.

Similarly, handling credit card data (PCI compliance) and other financials (covered by the Sarbanes-Oxley

law) require a level of responsibility that transcends your organizations cash-flow abilities. People go to

jail if laws are broken. Understanding where these key pieces of data reside, where they flow to, and

how they are accessed is a necessity.

Auditing and compliance monitoring is at least as multi-faceted as malware control. Monitoring the logs

on key database servers is necessary, but by no means sufficient. For instance, suppose I store all the

medical records in a MySQL database, and I monitor the database logins through Splunk (a great tool). I

will notice when a particular user accesses an unusually large number of records. However, if an



attacker gains access through my database server through a weak SSH password and subsequently

downloads all raw MySQL database files, I will never know it! Still, all records are compromised.

As an absolute minimum, you ought to cover two bases in this this category:

1. Log collection and analysis

2. Traffic auditing

Your options for collecting and analyzing logs from hosts, servers, and applications are varied, and your

options for spending money range from free to very expensive for the most powerful tools.

In general, the more you spend, the easier the log analysis will become, so larger deployments justify

larger investments.

A free solution might entail collecting syslog in a central location and getting crafty with "sed," "awk,"

and "grep". This is probably the most flexible, and very powerful, yet most technical approach. At the

top end are the "Security Information Management Systems," or SIMS. Their primary function is

collection of log data, analysis, graphing, plotting, and alerting on special conditions. They can be a

handful to configure properly, but they add fantastic power to your audit and security monitoring

ability. SIMS (or sometimes SIEM systems) include ArcSightTM, NetForensics®, and NitroSecurity.

Monitoring of the traffic that traverses the network (especially to the key information stores) is at least

as important, and most SIMS simply don't cover this area effectively. The NetFlow monitoring abilities

of most SIMS are shameful. In case of meeting audit and compliance requirements you are constrained

to a full packet capture or a full-fidelity flow analyzer solution. This means TCPdump or WireShark for

networks with relatively low traffic load (full-cap is hard to manage in large networks). For busy

networks a full-fidelity flow solution is the only viable option. Note that aggregators are useless for this

task, as most data leakages will simply not be linked to the offending address or originating server. You

need all the records. FlowTraq provides this ability.

At this point it is necessary to point out that Data Loss Prevention solutions such as Symantec DLP

(formerly Vontu®) and those provided by other vendors only help with part of the problem. If data is

exfiltrated through encrypted or password protected compressed channels, their detection fails. Our

SSH exfiltration example would not be detected by any DLP solution.



3. Control Your Damage

No matter how tightly you control your resources, something will eventually go bump.

And it is better to get prepared in quiet times than to figure it all out in the turbulent

times right after an incident. Traffic forensics, log analysis, and backup recovery are

central to this area. This is also the area that is most often ignored, allowing the same

problem to affect an infrastructure many times over. You must find out what

happened, so you can prevent it from interrupting your business again in the future.

3.1 Backup (and Restore)

The first and most important priority when a system fails, is compromised, or

crashes is to get the service it provides back online. Whether this is a router, a

webserver, a database, or simply a host in use by an employee, all of these

provide a service to the business, and their lack of availability is going to impact

the business.

Being able to restore your backups is something best practiced when no recovery is actually needed.

Does it work? Do you know how to recover? Is everything there?

Redundancy also falls in this category. Having a spare switch, router, desktop, or laptop can make all the

difference in getting back up and running. Sometimes, having duplicate images of key servers such as

webservers or databases allows you to recover immediately, while still being able to investigate the

original failure. Make sure to use a sensible RAID configuration for storage systems and purchase the

spare disks when you buy the storage system. You would be surprised how difficult it might be to find a

replacement disk for your storage system when a component eventually does fail.

Backup solutions range from free (rsync, dd) to expensive (NetApp®). Convenience and ability to

recover quickly should factor into your decision. Online backup ("in the cloud") may be an enticing

alternative to hosting your own, but be warned: like anything in the cloud, how can you be sure nobody

else is making a restore of your backup? In this box, you may spend more on redundant systems and

equipment (spares) than on specific software products.

3.2 Hosts and Servers

Investigating compromises, failures, and data theft on a host or server starts with understanding the

failure condition. Logs and the files on disk will generally provide the quickest (and often only) clue to

the nature of the issue. Investigating file modifications, log entries, and virus scanner alerts can lead you

on the right path in preventing the issue from recurring.



In this case, SIMS may provide a helping hand, as well remote logging facilities such as syslog. Scanning

for rootkits, keyloggers, and undesired remote administration tools is another step in investigating the

issue at hand, although all of these might be easier found by investigating the network traffic (see box

3.3).

At the top end are investigative tools like EnCase that help you investigate disk images. Most money in

this category, however, will be spent on your time: find out what happened and how likely it is that it

will happen again.

3.3 Network

Investigating the traffic that traversed you network may often give a strong clue as to the nature of the

issue at hand. Botnet traffic, remote administration tools, large data transfers to external addresses will

all be clues to the nature of the compromise. When there is evidence in network traffic of the issue

under investigation, it often means that the issue is not contained within your organization, and that an

external party may be involved.

This box (3.3) is not very well suited to full packet capture. Although fullcap gives the ability to look

deep inside the packets, and find possible minute details hidden there, it often does not have a long

enough history to be useful. For example, a network with a saturated 100MBit uplink to an ISP will fill

12 megabytes per second of full-cap storage. That is 43 gigabytes per hour, and about 1 terabyte per

day. Answering the three basic traffic forensic questions:

1. Where did it come from?

2. When did it start?

3. Are any other systems affected?

These can only be answered if sufficient traffic history is captured. It is therefore useful to monitor

multiple points in the network, and keep full-fidelity data around for weeks or even months. Full-cap

cannot scale to this level; only full-fidelity flow analytics can.

Tools like FlowTraq are designed for this purpose, and will scale far into the future. Even though

network traffic is increasing the total number of flows per host on your network has remained relatively

steady over the years. The reason is that people have a limited ability to consume emails, watch videos,

and browse webpages. The content is constantly gaining in resolution, which is driving the rise in

required network bandwidth, but the number of communications (and thus network flows) rises only

slowly.



4. Now What Do I Do?

Implementing the 3x3 taxonomy in your network is a powerful and structured way of covering the most

important bases for ensuring a stable infrastructure.

Below I have outlined several examples that may serve as guidelines to get you started, including:

1. Sample Free Solution

2. Sample Low End Commercial Solution

3. Sample High End Commercial Solution

A SAMPLE FREE SOLUTION:

Free as in "no purchasing cost". Not all of the examples above are straightforward to install and run, and

support forums can sometimes be helpful, sometimes not. Updates and patches are sometimes sporadic. They

require a higher level of operator skill level.

In many cases it might make sense to augment commercial solutions with some of the free alternatives provided

in the example table.



A SAMPLE LOW END COMMERCIAL SOLUTION:

In many environments it makes a lot of sense to mix some of the freeware with some investments in solid

commercial solutions. FlowTraq LITE covers three boxes of the 9, while Splunk can help with two.

A commercial AV solution is recommended, just to ensure timely and comprehensive virus signatures.

Vulnerability scans can be done occasionally with a free tool like OpenVas. Using software RAID, online backup,

and automatic Windows Update will get you a long way to a smooth infrastructure on a shoestring budget.



A SAMPLE HIGH END COMMERCIAL SOLUTION

When money is no object, or professional support is required at every step, many options become available. It

may even make sense to deploy two different pieces of technology for the same task. Depending on specific

business requirements you may add any specialized piece of software or hardware to any of the 9 boxes.

Keep in mind that each technology that you deploy takes time and energy to understand and operate.

5. Thoughts on Application Software and Virtualization

Application software purchases are deliberately missing from this taxonomy. Databases, web portals,

accounting packages, and others are the programs that you use to implement your business processes with.

They are implemented on top of your infrastructure. You should consider the infrastructure as the foundation

for your business processes. If your foundation is shaky, your applications, and therefore your business, will be

at risk. Box 1.1, however, does deal with updating and patching your application software, which is a key

requirement of smooth infrastructure operation.

Virtualization is considered part of a hardware strategy. Although it provides redundancy and ease of backups in

some cases, it is primarily a way to drive down hardware costs while simplifying hardware maintenance. A



properly virtualized environment will run as smoothly and reliably as a properly implemented fully physical

environment. All 9 boxes apply the same way in both cases.

6. Conclusion

As the inventor and lead designer of the FlowTraq system, I am not impartial in my views of the infrastructure

field. However, in many years advising organizations on infrastructure stability and security, I have repeatedly

been amazed by the lack of attention to understanding, auditing, and investigating the contents of network

traffic.

Often very smart people don't have a good grip on the traffic in their network. Full-cap solutions are ubiquitous,

but offer a microscopic view where a macroscopic view is desired. SNMP and log monitoring tools are simply

too coarse, while SIMS are often considered an insulting abstraction from the important details.

When we designed FlowTraq, we specifically had these categories (1.3, 2.3, and 3.3) in mind. We want to

provide a tool that can cover ground in all three categories, at a minimal time and monetary investment. This

allows even smaller organizations to cover these three very important areas, with great accuracy, and without

breaking the bank.

___________________________________________________________________________________________

Dr. Vincent Berk, CEO of ProQueSys, has 15 years of IT security and network management

experience, and is the designer of the FlowTraq system. He is a member of the ACM, the IEEE, and

teaches computer architecture at Dartmouth College.

_________________________________________________________________________________________

FlowTraq®, from ProQueSys helps IT Administrators find data leaks in the network, investigate compromises,

and monitor network usage such as bandwidth consumption, applications in use, and changes in behavior or

network activity that may indicate a problem. FlowTraq is designed to complement and

improve existing network operations by providing traceability, statistics, and identification

of potential security events.

View FlowTraq Tutorial Videos

Interactive Online Demo

Download a 14-Day Trial of FlowTraq by ProQueSys

http://www.flowtraq.com/corporate/resources/video

http://www.flowtraq.com/corporate/product/flowtraq-demo

http://www.flowtraq.com/corporate/product/try-flowtraq



Copyright and Trademarks THE TAXONOMY OF WISE INFRASTRUCTURE SPENDING

Copyright © 2011 - Process Query Systems, LLC

FlowTraq® is a registered trademark of Process Query Systems

sFlow® is a registered trademark of InMon

VNC and RFB are registered trademarks of RealVNC Ltd.

InterMapper® is a registered trademark of Dartware, LLC

Orion NPM is a product of SolarWinds.

What's-Up Gold is a product of Ipswitch, Inc

Scrutinizer is a trademark of Plixer International, Inc.

ManageEngine is a trademark of ZOHO Corporation.

InterMapper Flows is a product of Dartware, LLC

SiLK, the System for Internet-Level Knowledge, is a collection of NetFlow tools developed by the CERT/NetSA (Network

Situational Awareness) Team

Wireshark is a registered trademark of the Wireshark Foundation

Solera and its related products are trademarks of Solera Networks Inc

NetFlix® is a registered trademark of Netflix, Inc

YouTube is a registered trademark of YouTube Google, Inc

The Skype name is a trademark of Skype Limited.

FireWall-1 is a registered trademark of Check Point Software Technologies, Inc.

Cisco ASA Series products are registered trademarks of Cisco Systems, Inc.

NetNanny® is a registered trademark of ContentWatch, Inc

McAfee, the McAfee logo, and SiteAdvisor are trademarks or registered trademarks of McAfee, Inc.

Norton™ is a registered trademark of Symantec Corporation

Tripwire® is a registered trademark of Tripwire, Inc.

ClamAV® is a registered trademark of Sourcefire Inc.

SpamAssassin is a registered trademark of Apache Software Foundation.

TippingPoint and Digital Vaccine are registered trademarks of 3Com Corporation or its subsidiaries.

StealthWatch is a registered trademark of Lancope, Inc

OpenVAS products are Free Software under GNU GPL and a fork of Nessus.

ArcSight™ is a registered trademark of ArcSight, LLC

netForensics® is a registered trademark of netForensics.com

NitroEDB, NitroICE, and NitroGuard are registered trademarks of NitroSecurity, Inc.

Vontu® is a registered trademark owned by Vontu, Inc

NetApp, the NetApp logo, Go further, faster, and Data ONTAP are trademarks or registered trademarks of NetApp, Inc.

EnCase is the registered trademark of Guidance Software Inc.

All other company and product names may be trademarks of their respective holders. While every effort is made to ensure the

information given is accurate, ProQueSys does not accept liability for any errors which may arise. Specifications and other information in

this document may be subject to change without notice.

THE TAXONOMY OF WISE INFRASTRUCTURE SPENDING€¦ · · 2013-10-30THE TAXONOMY OF WISE...

Documents

Transcript of THE TAXONOMY OF WISE INFRASTRUCTURE SPENDING€¦ · · 2013-10-30THE TAXONOMY OF WISE...