Chapter 2

On the Job with a Network Manager

Activities performed by network managers Tools used by network managers How management tools and management

technology are used to achieve a given purpose

Support for network personnel The organization of tasks and workflows How network management personnel work

together Procedures that management personnel must

follow to complete a job

Overview - Topics

After reading this chapter you should gain a better understanding of the following: The types of operational tasks network

personnel face on a day-to-day basis How network management technology

supports network operators The different types of management tools

that network personnel use to do their jobs

Objectives

The three different scenarios provide an overview of the role that a network manager plays within an organization

Responsibilities differ widely depending on the type and size of the organization

The examples are illustrative and by no means comprehensive

A Day in the Life of a Network Manager

Pat is a network operator at the Network Operations Center (NOC) of GSP

Her group is responsible for monitoring the global backbone network and the access network

GSP is a multibillion-dollar business that provides global services with “five nines” service availability

Any disruption to service would cause revenue losses in the millions of dollars, expose GSP to penalties and liability claims, and put jobs in jeopardy

Pat: An Operator for Global Service Providers

A Command Central inside an NOC

Pat works in command central Big screens map the main sites of the

network and show statistics on network utilization

Map items appear in green when everything is operational

Occasionally entire nodes on the map turned yellow or red when connectivity is affected

A smaller screen on Pat’s desk shows a list of trouble tickets, which are used to track and resolve current network problems

Screen Views

The trouble tickets have two sources◦ customer-reported problems and ◦ Network problems

Reps provide “tier 1 support” for customers

If a service rep cannot help a customer, the customer is transferred to a rep with more troubleshooting experience

Pat’s group provides the third tier of support

Trouble Ticket Support

Trouble tickets typically contain◦ a description of the problem◦ who is affected◦ contact information

Sometimes this information is missing or in error

Trouble Ticket Contents

The second source of tickets Reported by systems that monitor alarm

messages sent by network equipment The problem with alarm messages is that

they rarely indicate the root cause of the problem

In most cases, they reflect symptoms that could be caused by any number of things

Network Related Trouble Tickets

Pat doesn’t see every single alarm in the network The alarm monitoring system tries to group

messages that appear to point to the same underlying problem

Viewing grouped messages is much more practical than addressing individual alarms

In the past, Pat has seen 20 tickets all related to the same problem

GSP has recently made significant progress in reducing redundancy

Despite the progress made many tickets still relate to the same underlying cause

Trouble Ticket Aggregation

Pat must first acknowledge that she has read each ticket

If she does not acknowledge the ticket, it is automatically escalated to her supervisor

Next Pat analyzes the ticket information She uses a monitoring application for the affected

part of the network to see what is going on She starts by verifying the symptoms that are

reported in the network In some cases, Pat decides that a piece of

equipment needs to be replaced

Trouble Ticket Resolution

If equipment needs replacement Pat opens a work order system to create an order for a new part

It is not Pat’s responsibility to dispatch a field technician

Sometimes technicians are dispatched to a remote location to inspect and/or replace the equipment

In other cases the rack-mounted equipment in the NOC is affected

Work Order System

Rack-Mounted Network Equipment

Cabling and Patch Panels

When the technicians exchange a part, they use a bar-code scanner that automatically updates the central inventory system

Pat enters what she did and when resolution is expected on the trouble ticket

For now, she is finished When the work order is fulfilled a notification from

the system will be sent to her in-box Pat used to close the tickets before receiving

notification from the work order system She now understands that procedures are essential

for GSP to control quality of the services it provides

Work Order Fulfillment

Chris and a colleague are responsible for the computer and networking infrastructure of the RC Stores headquarters and 40 branch locations

RC Stores’ network contains almost 100 routers◦ Typically, an access router and a wireless router in

the branch locations and◦ additional networking infrastructure in the

headquarters and at the warehouse

Chris: Network Administrator for a Medium-Size Business

RC Stores Network

A managed service provider (MSP) is used to interconnect the various locations of its network◦ The MSP has set up VPN tunnels between the access routers at

each site that connects the branch locations to the headquarters◦ The entire network can be managed as a single network

Chris runs a management platform from a workstation at his desk◦ The application provides a graphical view of the network that

displays the network topology◦ Chris has built a topology map that groups multiple routers into

“clusters”◦ A typical management application screen is shown in the next

slide Management applications support some statistical views

as well

RC Network Overview

A Typical Management Application Screen

Cisco Packet Telephony Center

Management Application Screen with Performance Graphs

Cisco Works IP Performance Monitor

Chris adds a phone for a new user and troubleshoots a problem that he suspects might be caused by problems on MSP’s network

Chris handles multiple responsibilities because of the smaller size of his organization and network

An Example Administrator Task

Sandy works in the Internet Data Center for a global Fortune 500 company, F500, Inc.

The data center◦ is the center of the company’s intranet, extranet, and Internet

presence◦ hosts the company’s external website and connects customers to

the online ordering system◦ is host to all the company’s crucial business data ◦ hosts the company’s internal website which provides access to

the data given the proper access privileges Since the functions provided by the data center are crucial

to its business, F500, Inc. decided not to outsource them The way F500, Inc. organizes and manages its processes

and supply chains differentiates it from its competitors

Sandy: Administrator and Planner in an Internet Data Center

Sandy is to develop a plan to accommodate a new supplier

It will involve configuring ◦ server and storage infrastructure critical for the business

relationship as well as◦ an extranet over which the shared data can be accessed

Sandy has a list of the databases that need to be shared and backed up and storage and network capacity must be assessed

Sandy’s main concern is security Sandy needs to set up a new VLAN to separate traffic on

the extranet from other network traffic A typical networks configuration screen is shown in the

next slide

Developing a Plan for a Supplier

A Management Application Screen for Port Configuration

Access control lists (ACLs) on the routers need to be updated to reflect the new extranet security policy

Finally, authentication, authorization, and accounting (AAA) servers need to be configured

Sandy must determine where the data will be hosted and how her decision will impact the internal data center topology

Sandy uses a performance-analysis application to make these determinations

Data are stored in multiple places. This will be transparent to applications.

A common uniform resource identifier (URI) will be used to address the data

Additional Plan Concerns

Pat, Chris, and Sandy handle their jobs in different ways The differences depend upon the size and purpose of the network and the

network functions that individual manages. Chris manages numerous aspects of his network.

Different tools are used by the managers to carry out the required management tasks

Pat’s job is determined by guidelines, procedures, and the way the work is divided at her organization

Both manually-executed and automated tasks exist Management tasks required abstract concepts that had to be broken down

into numerous subtasks Running a network involves monitoring operations, diagnosing failures,

configuring services, analyzing historical data, planning, security, and workforce management

Integration affects operator productivity◦ Pat used integrated applications to manage trouble tickets◦ Chris had to repeat some work because of a lack of integration in his phone system

applications

Observations

Device managers and craft terminals Network and element analyzers Management Platforms Collectors and Probes Intrusion Detection Systems Performance Analysis Systems Alarm Management Systems Trouble Ticket Systems Work Order Systems Workflow Management Systems and Workflow Engines Inventory Systems Service Provisioning Systems Service Order-Management Systems Billing Systems

Typical Management Tools

Craft terminals provide a graphical view of the physical configuration of network equipment

Craft terminals are used to◦ log into a device◦ view its current status◦ change configuration settings◦ perform diagnostic self-tests and◦ downloading software images

Managed equipment may provide a “built-in” craft interface that renders a view of the device

Field technicians typically load craft terminal software onto their notebooks so they can connect to a device and managed it via a USB or serial interface

Craft Terminals/Device Managers

A Graphical Device View

CiscoView for Catalyst 6500

Network or packet analyzers are used to view and analyze network traffic for diagnostic and and troubleshoot purposes

Network analyzers◦ capture or “sniff” packets that traverse the ports

of a router or switch and◦ Display packets in a human-readable format that

an experienced network operator can interpret

Network Analyzers

Similar to craft terminals but◦ contain a database for retaining device configuration

information◦ receive event messages from devices on the network

and◦ discover equipment deployed on the network

Element Managers may have an interface that allows applications to manage equipment through the element manager. This provides ◦ data synchronization between apps◦ a simpler user interface and◦ reduced management load

Element Managers

General-purpose applications used to monitor and manage the network, comparable to element managers

May be proprietary Often include toolkits that extend the

management platform Often compared to a PC operating system

because it provides a means for communicating with network devices and tracking communications via an internal database

Management Platforms

Collectors gather and store network data◦ Netflow is a common◦ It gathers data about the traffic traversing a

router or switch Probes are active collectors

◦ They can trigger events and collect the responses

Collectors and Probes

Provide information regarding suspicious activity on a network, such as◦ break-ins to routers or servers and◦ denial-of-service attacks

IDS listen◦ for alarms◦ inspect activity logs and◦ observe load patterns to determine if a network

has been compromised

Intrusion Detection Systems

Performance Analysis Systems Enable users to analyze traffic and

performance data to recognize trends and traffic patterns

Alarm Management Systems Collect, monitor, and group large volumes

of data that pertain to alarms that are triggered by different events on the network

May be provide an initial diagnosis of the cause of an alarm

Performance and Alarms

Trouble Ticket Systems Support network problem resolution Usually generated by users of the network who

experience a problem Assign tickets, escalate tickets that take too long to

resolve, and report statistics about the resolution process

Work Order Systems Assign and track maintenance jobs in a network Help organize workforce that performs jobs Track the maintenance process

Trouble Ticket and Work Oders

Help manage execution of predefined processes that consist of multiple steps and different owners

Not specific to networks Based upon concept of finite state machines Events are managed by a workflow engine

Workflow Management Systems and Workflow Engines

Inventory Systems Used to track assets which may be either

physical or services (such as a phone service)

Service Provisioning Systems Facilitate the deployment of services, such

as DSL, over a network Typically found in operational support

environments of large service providers

Inventory and Service Provisioning

Service Order-Management Systems◦ Used to manage orders for customers of large

service providers, generally not found in enterprise environments

◦ Manages workflows and processes of an organization

Billing Systems◦ Essential for collections of revenues◦ Analyze accounting and usage data to identify

which communications services were provided to whom

◦ Can be helpful in fraud detection

Order Management and Billing

Three scenarios illustrating how networks are managed in practice and the variety of tasks that are involved were discussed

The service provider scenario emphasized workflows, processes, and interactions

The medium-size enterprise scenario was characterized by a variety of tasks that had to be performed by one individual and a greater reliance on the individual expertise and intuition

The Internet Data Center scenario focused on the planning phase of a network’s life cycle as well as management practices for both the network and the devices and applications connected to it

The scenarios illustrate that network management involves management technology as well as organizational and human factors

In each case personnel were supported by a variety of tools A wide variety of different tools exist Running the largest, most complex networks usually involves

hundreds of management systems and applications

Chapter Summary