Open Source Network Monitoring Tools

Yasir Iqbal22-May-2010

In this presentation

IntroductionWhat are Network Monitoring ToolsBandwidth Monitoring Techniques/ServicesSetting up some monitoring ToolsConclusion

Introduction:- Why do we need to monitor and measure Bandwidth

Cost of Bandwidth is expensive for developing countries

Bandwidth in developing countries is expensive. In a report for the Partnership for Higher Education in Africa, Mike Jensen calculates that Makerere University pays about

$22,000/month for 1.5Mbps/768Kbps (in/out), Eduardo Mondlane pays $10,000/month for 1Mbps/384Kbps, while the University of Ghana pays $10,000/month for 1Mbps/512Kbps.

These figures indicate that African universities, outside of South Africa, are paying over $55,000/month for 4Mbps inbound and 2Mbps outbound. These figures are about 100 times more expensive than equivalent prices in North America or Europe.

Cont…

To Know if the ISP is providing us with the required bandwidth paid for.

To be able to optimize the available bandwidth◦ 59% of institutions do not monitor or manage bandwidth

at all (Belcher)

Ways to improve network performance

Upgrade infrastructure, to install faster, larger, and higher performing systems, lines and facilities.

Look for cheaper provider and Increase/upgrade your bandwidth.

Alternative approach◦ is to recognize that ‘bandwidth’ is a valuable institutional

resource or asset that needs to be managed, conserved, and shared as effectively as possible.

How do we measure Bandwidth?

Network Monitoring Tool

What are Network Monitoring Tools?

Allows the administrator to know the health status of the network.

It provides information about collected data and the analysis of such raw data with a view to using scarce or limited resources effectively.

Uses network probe. Probes let you isolate traffic problems and congestions slowing your network to a crawl.

What can we use the tools for?

Identifying unofficial services or servers Monitoring usage and traffic statisticsTroubleshooting your networkInvestigating a security incidentKeeping logs of users activities for

accountability

Who? What? Where? How? When?

Who is accessing your network?◦ students, academics, staff, visitors or others

What are they accessing your network for?◦ academic study, social use, business use, illegal use

Where are they accessing your network from?◦ internal, external

How are they accessing your network?◦ remote user, local Ethernet, WAN, dial-up, Wi-Fi, VPN

When did they access your network?◦ today, yesterday, last week, last month…

Network Monitoring Tools

Active tools◦ Ping – test connectivity to a host◦ Traceroute – show path to a host◦ MTR – combination of ping + traceroute◦ SNMP collectors (polling)

Passive Tools◦MRTG◦Nagios◦Cacti◦Ntop◦Webalizer

Passive Network Monitoring Tools

Multi-Router Traffic GrapherIs a tool for monitoring traffic loads on a network

link. MRTG generates HTML pages that provide a live, visual representation of the network traffic.

It can be used to monitor any SNMP MIB.Limitations◦ It cannot provide information that shows which host or

application may be causing a traffic bottleneck.◦ MRTG does not provide information about traffic type or

protocol statistics

CONT…

TCPdump◦Uses the packet capture library (libpcap).◦Prints the headers of packet on a network

interface, user analyses network status using this header manually

◦Has many option for capturing raw data, but it does not provide any analysis capability for the captured data.

CONT…..

IPTraf◦ IPTraf is a console-based network statistics utility for

Linux. It gathers a variety of figures such as TCP connection packet and byte counts, interface statistics and activity indicators, TCP/UDP traffic breakdowns, and LAN station packet and byte count

◦ Protocols Recognized◦ IP ◦ TCP ◦ UDP ◦ ICMP ◦ IGMP ◦ IGP ◦ IGRP ◦ OSPF ◦ ARP ◦ RARP

CONT…

Webalizer◦The Webalizer is a fast, free web server log

file analysis program. It produces highly detailed, easily configurable usage reports in HTML format, for viewing with a standard web browser.

◦http://seecs.nust.edu.pk/stats/apr_2010/usage_201004.html

Nagios

http://www.nagios.org/an enterprise-class network and server

monitoring system.Useful for:

◦Monitoring of network services.◦Monitoring of host resources (processor load, disk

usage, system logs)◦Contact notifications when service or host

problems occur and get resolved (via e-mail, SMS).

◦You can define event handlers that execute when triggered by certain events. (Proactive problem resolution)

OpenNMS

http://www.opennms.orgFunctionalities

◦High performance A single instance of OpenNMS supports monitoring of a

large number of nodes.◦Automation

OpenNMS minimizes the amount of manual configuration.

◦Rule-based configuration Flexible rules can be used to specify what services are

polled on what devices.

Cacti

http://www.cacti.netSimilar to MRTG.Based on RRDtool.Offers excellent graphing capabilities.Has extensive templates.

General Description of Cacti

1. Cacti is written as a group of PHP scripts.2. The key script is “poller.php”, which runs every 5 minutes

(by default). It resides in /usr/share/cacti/site.3. To work poller.php needs to be in /etc/cron.d/cacti like this:

MAILTO=root */5 * * * * www-data php /usr/share/cacti/site/poller.php >/dev/null 2>/var/log/cacti/poller-error.log

4. Cacti uses RRDtool to create graphs for each device and data that is collected about that device. You can adjust all of this from within the Cacti web interface.

5. The RRD data is stored in a MySQL database along with descriptions of each device that is monitored.

6. The RRD files are located in /var/lib/cacti/rra.

Advantagess

You can measure Availability, Load, Errors and more all with history.

– Cacti con view your router and switch interfaces and their traffic, including all error traffic as well.

– Cacti can measure drive capacity, CPU load (network h/w and servers) and much more. It can react to conditions and send notifications based on specified ranges.

Graphics – Allows you to use all the functionality of rrdgraph to

define graphics and automate how they are displayed. – Allows you to organize information in hierarchical tree

structures. Data Sources – Permits you to utilize all the functions of rrdcreate and

rrdupdate including defining several sources of information for each RRD file.

Advantages cont.

Data Collection – Supports SNMP including the use of php-snmp or net-snmp

– Data sources can be updated via SNMP o by defining scripts to do this.

– An optional component, cactid, implements SNMP routines in C with multi-threading. Important for very large installations, but not tested formally.

Templates – You can create templates to reutilize graphics definitions,

data and device sources User Management – You can manage users locally or via LDAP and you can

assign granular levels of authorization by user or groups of users.

Disadvantages

Configuration of Interfaces is Tedious – The first time you add an interfaces, add graphics for

each interface and place these graphics correctly on a hierarchical menu requires considerable time and effort.

– It’s very important that you keep your Cacti configuration up-to-date with your network. You must either assign someone to do this, or create appropriate scripts and data shares for this purpose.

– If you make a configuration error it can be tedious to correct it.

But, in reality, for continuous use or large installations it is likely that you will be using scripts and tools to automate the configuration of Cacti.

Setting up Cacti on CentOS 5

Cacti requires that the following software is installed on your system.◦ RRDTool 1.0.49 or 1.2.x or greater

◦ MySQL 4.1.x or 5.x or greater

◦ PHP 4.3.6 or greater, 5.x greater highly recommended for advanced features

◦ A Web Server e.g. Apache

◦ Net-Snmp

Mysql, PHP, Apache and SNMP packages are already installed

on your machine if not installed through yum utility. yum install mysql-server mysql php-mysql php-pear php-common php-gd

php-devel php php-mbstring php-cli php-snmp php-pear-Net-SMTP php-mysql httpd

rrdtool: Installation

Install rrdtool manually by downloading the latest version at the following

URL http://oss.oetiker.ch/rrdtool/ SCP the tarball into the /usr/src directory on your linux box. From a

command prompt, change into the /usr/src directory, and un-tar the tarball:

cd /usr/src tar -xzvf rrdtool-1.0.45.tar.gz

Change into the newly created directory:

cd rrdtool-1.0.45

Compile and install RRDTool:

./configure make make install

rrdtool: Installation

The default installation location is /usr/local/rrdtool-VERSION, so make some symbolic links to the executables:

ln -sf /usr/local/rrdtool-1.0.45/bin/rrdtool /usr/bin/rrdtool ln -sf /usr/local/rrdtool-1.0.45/bin/rrdupdate /usr/bin/rrdupdate ln -

sf /usr/local/rrdtool-1.0.45/bin/rrdcgi /usr/bin/rrdcgi

The RRDTool Perl library simplifies things when using RRDTool from a Perl script, so to compile and install the Perl library for RRDTool:

make site-perl-install

Create a directory for RRDTool databases, and a directory for the web images which it'll generate:

mkdir /var/lib/rrd mkdir /var/www/html/rrdtool

cacti: Installation

Extract the distribution tarball.

shell> tar xzvf cacti-version.tar.gz Create the MySQL database:

shell> mysqladmin --user=root create cacti Import the default cacti database:

shell> mysql cacti < cacti.sql

Optional: Create a MySQL username and password for Cacti. shell> mysql --user=root mysql mysql> GRANT ALL ON cacti.* TO cactiuser@localhost

IDENTIFIED BY 'somepassword'; mysql> flush privileges;

cacti: Installation

Edit include/config.php and specify the MySQL user, password and database for your Cacti configuration.

$database_default = "cacti"; $database_hostname = "localhost"; $database_username = "cactiuser"; $database_password = "cacti";

Set the appropriate permissions on cacti's directories for graph/log generation. You should execute these commands from inside cacti's directory to change the permissions.

shell> chown -R cactiuser rra/ log/ (Enter a valid username for cactiuser, this user will also be used in the next step for data gathering.)

Add a line to your /etc/crontab file similar to:

*/5 * * * * cactiuser php /var/www/html/cacti/poller.php > /dev/null 2>&1

cacti: Installation

Now use a web browser and open the following address:

http://localhost/cacti

You will see the following...

cacti: InstallationPress “Next >>”

cacti: Installation

Choose “New Install” and press “Next >>” again.

cacti: Installation

Your screen should look like this. If it does not ask your instructor for help.

Press “Finish”

Note!Be sure that “RRDTool 1.2.x” is chosen and not “1.0.x”.

cacti: First Login

First time login use:User Name: adminPassword: admin

cacti: Password Change

Now you must change the admin password. Please use the workshop password.

Add Devices: 1

• Management -> Devices -> Add• Specify device attributes

– Choose a device template and this will ask you for additional information about the device.

– You can add additional templates when, or if, you want.

Add Devices: 2

Add Devices: 3Choose SNMP version 2 for this workshop.At your own location you can use SNMP

version 3 if your devices support this.SNMP access is a security issue:

- Version 2 is not encrypted- Watch out for globally readable “public”

communities- Be careful about who can access r/w

communities.

Create Graphics

• Chose the “Create graphs for this host”• Under Graph Templates generally check

the top box that chooses all the available graphs to be displayed.

• Press Create.• You can change the default colors, but the

predefined definitions generally work well.

Create Graphics: Step 1

Create Graphics: Step 2

View the Graphics• Place the new device in its proper location

in your tree hierarchy. • Building your display hierarchy is your

decision. It might make sense to try drawing this out on paper first.–Under Management Graph Trees

select the Default Tree hierarchy (or, create one of your own).

Graphics Tree

First, press “Add” if you want a new graphing tree:

Second, name your tree, choose the sorting order (the author likes Natural Sorting and press “create”:

Graphics Trees

Third, add devices to your new tree:

Once you click “Add” you can add “Headers” (separators), graphs or hosts. Now we'll add Hosts to our newly created graph tree:

An Example…

Conclusions

• Cacti is very flexible due to its use of templates.• Once you understand the concepts behind RRDTool,

then how Cacti works should be (more or less) intuitive.• The visualization hierarchy of devices helps to organize

and discover new devices quickly.• There are very few to no statistics available about the

performance of cactid (volunteers are welcome!).• It is not easy to do a rediscover of devices. • To add lots of devices requires lots of time and effort.

Software such as Netdot, Netdisco, IPPlan, TIPP can help – as well as local scripts that update the Cacti back-end MySQL database directly.

NTop

http://ntop.orgNetwork probe that shows network

activity just like “top”.

Setting up Ntop

Download Ntop Using a tar ball

tar xpfz ntop-3.0-4.tar.gz./configuremakemake install

http://rpm.pbone.net Installing with RPM is also easy. The package name may vary, but

you simply use the command:rpm –uvh ntop-3.0-4mdk.i586.rpm

Run ntop (service ntop start) Go to a web browser type http://localhost:3000

Security Tools

Some security tools to consider:◦NetFilter IP Tables – Firewall◦WireShark – Protocol analyzer◦Snort – Intrusion detection◦Netcat – Feature rich tool. Great for debugging.◦Nessus – Vulnerability scanner◦Many many more…

?