1 ©2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice

NNMi 9.0: Global Network Management

Larry Besaw

NNMi Architect

2

Introduction to Global Network Management

– Subtitle goes here

3

Terminology

–Global Network Management (GNM) – The feature name

–Global Manager – The central NNMi system which receives

data from remote NNMi systems

–Regional Manager – The remote NNMi system feeding

data back to the global manager

4

What Is Global Network Management?

– Consolidated topology gathered from many remote NNMi systems

• Single console containing entire global network

• Causal analysis of the entire network including inter-region connectivity

– Secure and fault tolerant communication to regional sites

• All communication can be encrypted

• No direct SNMP or ICMP from the global manager to devices

• Handles network outages between the global manager and the regional sites

– Massive scalability

• Global manager supports up to 65,000 remotely managed nodes

– Nodes are still locally managed by the regions

• Communication, discovery & polling configuration is managed on the regional managers

• Global manager automatically receives updates to topology and state changes

5

What It Is Not…

– It is not a redundancy solution

• There is no polling failover from a regional manager to a global manager or

between regional managers

• GNM supports application failover and HA for redundancy at each site

• The global manager will automatically reconnect to the new active node of an

application failover or HA cluster

– It is not an overlapping address or NAT solution

• A GNM deployment is similar to a standalone NNM in regards to nodes with

duplicate addresses (see the Monitoring Devices Located Behind a Static NAT

Gateway white paper for details)

• The management address must be unique on each node

• Address filtering may be used to remove private addresses

6

Global NNMi Cluster

SNMP Devices

Encrypted Message Bridge

Regional NNMi Cluster

NNMi NNMi

NNMi NNMi

Regional NNMi Cluster

NNMi NNMi

7

GNM Deployment

– GNM deployment models

• Single global manager with multiple regional managers

• Multiple global managers with multiple regional managers (each feeding more than one global manager)

• No loops allowed (e.g., two peers forwarding to each other)

• Only 2 levels currently supported

– Global manager can manage nodes locally as well as remotely

– Supports overlap in management domain of regional managers

– Any combination of operating systems is allowed for global and regional managers

– The global manager must be either the same version as or a later version than the regional managers

• Both global and regional manager must be at least version 9.0

• Always patch or upgrade the global manager before the regional managers

8

Requirements

– An iAdvanced license is required for a system to be a global manager

– Normal node count licenses apply on the global manager

– Time synchronization software must be running on each NNMi system

• Time zones may be different as long as all system have the same UTC time

– Each NNMi system must be able to resolve the Fully Qualified Domain

Name (FQDN) of the other systems

– The global manager needs TCP access to 3 ports on the regional

manager

• Sockets are always opened from the global to the regional

• Direct TCP access is required. Transparent firewalls are ok.

9

When to Deploy GNM

– Remote sites which have local network operators but where

there is also a requirement for a global network view

– Remote sites with no direct SNMP access to devices from

outside the local network

– Remote sites where the external network connection is

unreliable

– Extremely large environments where it is not possible to use

a single system solution

• Single system scalability: 25K nodes, 1 million interfaces, 200K polled

interfaces

10

GNM Architecture & Design

– Subtitle goes here

11

Architecture Overview

– A new concept of a System ID has been introduced which uniquely

identified each NNMi database so that it may participate in GNM

– All management data is forwarded from the regional to the global using

a secure store-and-forward message bridge

– Global managers perform discovery against the regional NNMi

database not the devices

– All polling is done by the regional manager with only changes

forwarded to the global

– Causal analysis runs independently on the global manager using the

global topology

– Traps may be forwarded to the global manager

– SNMP tools are proxied over the secure channel

12

System ID

– Each NNMi install now has a unique System ID associated with its database.

– Every node in the NNMi database is tagged with the System ID of where it

came from

– When GNM is configured the global manager first contact the new regional

and retrieves its System ID and stores that with the configuration

– A global manager checks this System ID every time it establishes a connection to

ensure it is communicating with the correct system. If the System ID doesn’t match

then the global manager will refuse to connect

– Resetting the database on the regional manager requires removal of the regional

manager configuration on the global manager. See Disconnect Communication

with a Regional Manager in the NNMi help for detailed steps.

Using backup-restore to clone a system will result in them both having the same

System ID and they will not be able to participate in GNM

13

Message Bridge

– Store and forward architecture gives resilience to network failures

between regions.

• State changes and traps are stored until they are delivered

• Other time-sensitive messages like discovery traffic and tunneled SNMP are kept for up to

10 minutes and then discarded because the clients will have timed out

– Supports industry standard SSL/TLS encryption

– Automatic failure detection and reconnection when network failures

happen

– New concept of a subscription ensures no state updates or traps are lost

even if the global manger is down when they occur

– Uses the System ID for message routing to the correct system

14

Message Subscription

– The global manager creates a subscription on each regional manager

– Messages for the global manager are associated with the subscription

and stored until they are successfully delivered

– The subscription is identified by the system ID of the global manager

• Resetting the database on a global manager would leave an open subscription on the

regional manager which needs to be cleaned up

• Removing the regional manager connection while the regional manager is up and

connected will automatically cleanup the subscription

• There is a tool, nnmsubscription.ovpl, to manually remove old subscriptions and the

associated messages if the global manager gets reset or is down and not coming back

15

Discovery

– No SNMP queries are made to devices from the global manager

• Discovery queries the regional NNMi database over the message bridge

– Connections, VLANs and Subnets are recalculated by the global manager based on the combined data from all the regions

– Rediscovery on the global happens when the regional sends a notification that it has finished its rediscovery for a node

• No separate rediscovery configuration on the global

• The global is always up to date because it rediscovers immediately after the regional

– Address filtering may be used on the global manager to filter out private addresses from each region

– The Configuration Poll action on the global manager will first execute on the regional system and then on the global

– nnmnoderediscover.ovpl can be used to force rediscovery of all nodes for a regional manager

16

State Polling

– The global manager does no direct polling of devices instead it receives all updates from the regional polling

– All configuration for communication and polling is done on the regional manager as normal

– The Monitoring Settings action will launch to the regional manager

– Only polls that detect a state change are forwarded to the global manager

• The exception to this is performance polling which if enabled on both the regional and global will cause each result to be forwarded

– The Status Poll action on a global manager will be forwarded over the bridge and be executed on the owning system for that node and state will be updated on both the regional and global

– Custom Poller does not currently support forwarding of custom polled instances or poll results to the global manager

17

Events

– Management events generated by the Causal Engine are not forwarded

because equivalent events will be generated on the global manager

– No traps are forwarded by default

• Traps on the regional manager that trigger discovery or polling affect the global

manager indirectly without the trap being forwarded

– Trap configuration contains a new option to forward to global managers

over the bridge

• Remote NNM 6.x/7.x events can also be configured for forwarding

– Traps from each regional are limited to a rate of 20 per second

(averaged over 5 minutes, spikes can be much higher) to prevent a

single regional flooding the global manager

18

SNMP

– The global manager will never perform direct SNMP communication

with remote nodes

– There is no need to configure communication settings on a global

manager unless it is also going to directly manage nodes

– Communication Settings action will launch to the regional system

– Anything on the global manager which requires SNMP access to

devices will be tunneled over the secure channel to the regional

manager and performed there.

• SNMP CLI tools

• Real-time SNMP Graphs

• Path View

• MIB Browser

• Custom Poller

19

UI

– Users can view the consolidated topology on the global manager

• Nodes have a Management Server attribute that indicates LOCAL or the regional

manager

• The Inventory → Nodes by Management Server view allows easy viewing of nodes from

different regional managers

• Node Group filters can be created based on the regional manager of a node

(nnmSystemName attribute)

– Some actions launch remote UIs from the regional manager

• Ping, Traceroute, Monitoring Settings, Communication Settings, Regional Manager

Console, Open node on regional manager

– Other actions launch local UIs that are proxied through the message

bridge

• Status Poll, Configuration Poll, Path View, Real-time SNMP Graphs, MIB Browser

20

User-modified Attributes

– Management Mode

• Changes to the management mode (e.g., managed, not managed, out of service) on the

regional manager are immediately sent to the global manager

• The management mode cannot be modified on the global manager for remotely

managed objects

– Interface Input Speed and Output Speed changes on the regional

manager are sent to the global manager during the next rediscovery

– Notes & Custom Attributes are not synchronized between global and

regional managers

21

GNM & SPIs

– Subtitle goes here

22

SPIs Supporting GNM

– SPIs that fully support GNM

• iSPI Performance for Metrics

• iSPI Performance for Traffic

• iSPI for MPLS

• iSPI for IP Multicast

– These can run on both the global manager and regional manager to

provide consolidated data

– Message Bridge

• The Metrics SPI uses the NNMi message bridge

• The Traffic SPI, MPLS SPI, and IP Multicast SPI each have their own message bridge

• Each SPI that has its own message bridge requires 3 additional ports to be opened in the

firewall

23

SPI Support

– iSPI Performance for Metrics

• Forwards all interface and node component performance poll results from the regional

manager to the global manager (depending on polling rate this can be significant

network traffic)

• Metrics SPI license required on both systems for the global manager to receive metrics

– iSPI Performance for Traffic

• Forwards summarized flow records from the regional master collector to the global master

collector

– iSPI for MPLS

• Forwards topology, state changes, and performance poll results from the regional

manager to the global manager

– iSPI for IP Multicast

• Forwards topology, state changes, and performance poll results from the regional

manager to the global manager

24

Other SPIs

– SPIs that do not fully support GNM

• iSPI NET

• iSPI Performance for QA

• iSPI for IP Telephony

– These can run on the global manager, the regional manager, or both,

but do not provide any consolidation of data

– When running on the global manager, they only operate on locally

managed nodes

25

Other SPI Support

– iSPI NET

• Recommended deployment is to run on the regional managers

• You can cross-launch from the global manager to a node on the regional manager to see flows for remotely managed nodes

• If running on the global manager, you need to change incident configuration to configure diagnostic flows to run only for locally managed nodes

• Run Diagnostics action is not allowed on remotely managed nodes

– iSPI Performance for QA

• Recommended deployment is to run only on the global manager with locally managed nodes

• If running on the regional manager, forwards management events to the global manager

– iSPI for IP Telephony

• Recommended deployment is to run only on the global manager with locally managed nodes (assuming single system scalability limits and low latency links)

• If running on the regional manager, forwards management events to the global manager

26

GNM Configuration

– Subtitle goes here

27

Configuration Areas

– Address filtering

– Node filtering

– Firewall

– Single Sign On

– Bridge Connection

28

Address Filtering Configuration

– Regions that use private address ranges internally may conflict on the

global manager

– It is recommended that private address ranges be filtered out to avoid

any problems on the global manager.

– Address filtering may be configured under Discovery Configuration →

Excluded IP Addresses form on either the regional manager or just on

the global manager

29

Node Filtering Configuration

– In some environments there will be nodes discovered on regional

managers which are not required on the global manager

– It is possible to configure a regional manager to only send nodes that

are in a certain node group up to the global manager

– Configuration is under Global Network Management → Forwarding

Filter and is configured on the regional manager.

Excluding nodes that are important for connectivity may affect causal

analysis on the global manager

30

Firewall Configuration

– All network sockets are opened from the global manager to the regional

manager

– If using non-encrypted communication the ports are (all ports are as

configured on the regional manager)

• jboss.http.port (default is 80)

• jboss.bisocket.port (default is 4457)

• jboss.jmsControl.port (default is 4458)

– If using encryption then the ports are

• jboss.https.port (default is 443)

• jboss.sslbisocket.port (default is 4459)

• jboss.ssljmsControl.port (default is 4460)

31

Single Sign-On Configuration

– It is recommended that Single Sign-On be configured on all NNMi

systems participating in GNM

– If single sign-on is not going to be used then it is recommended that it

be disabled as accessing two NNMi systems with enabled but not

correctly configured SSO can cause problems.

32

Message Bridge Configuration

– Final configuration step is to establish the bridge on the global manager

under Global Network Management → Regional Manager Connections

– Each regional manager has a user-friendly name and one or more

connections (multiple connections are used for application failover)

• Hostname should match the official FQDN of the regional manager

• If using encryption please see the deployment guide for detailed instructions on

configuring certificates

• Username is always “system”

• Password is the system password for the regional manager

– When both the forms are saved the global manager will contact the

regional to obtain its System ID and then save the configuration and

initiate the connection (refresh the table to see it connect)

– Once the connection is established the global will start discovery

33

NodeA.SS

NodeA.SS

NodeB.SS

NodeB.SS

nnm.keystore

nnm.truststore

NodeA and NodeB, using AppFailover:

1. Copy key+trust B to A

2. Merge key+trust on A

3. Copy merged file back to B

NodeX.SS

NodeX.SS

NodeY.SS

NodeY.SS

nnm.keystore

nnm.truststore

NodeZ.SS

NodeZ.SS

NodeX and NodeY, using HA;

virtual FQDN is NodeZ:

1. HA-configure creates NodeZ.SS

2. Copy the key+trust to node Y

NodeZ.SS

NodeZ.SS

NodeX.SS

NodeX.SS

nnm.truststore

NodeM and NodeN, GNM global manager

cluster using AppFailover (already configured)

NodeM.SS

NodeN.SS

nnm.truststore

NodeM.SS

NodeN.SS

NodeA.SS

NodeB.SS

NodeX.SS

NodeZ.SS

HP.COM HP.COMHP.COM HP.COM

HP.COM HP.COM

For corporate CA, get

your certs signed by

the CA, import into keystore,

import CA into truststore at

all sites.

NodeA.SS

NodeA.SS

Copy

NodeB.SS

NodeB.SS

Copy+merge

Node A Node B Node X Node YNodeA.SS

NodeB.SS

NodeX.SS

NodeZ.SS

Copy + merge

Certificate Configuration

34

Regional Manager Connections

35

Verifying Configuration

– Global Network Management tab on Help → System Information shows

GNM information

• Regional managers for global manager

• Global managers for regional manager

– Health tab on Help → System Information shows any problems with the

GNM configuration (e.g., connection failures)

36

Network Bandwidth Usage

– See the white paper NNMi Network Bandwidth Utilization (Standalone

and Global Network Management Environments) for measured network

bandwidth usage in sample GNM scenarios

• Available from http://support.openview.hp.com/selfsolve/manuals

37

Outcomes

–Don’t use GNM unless you really

need it. Single system deployment

is simpler and less expensive.

–Use GNM to effectively manage

your entire network from a single

console if you have any of the

following needs: • Hierarchical management due to organizational

structure

• Remote discovery and polling due to firewalls or

link bandwidth or link reliability

• Massive scalability

38

Q&A

39 ©2010 Hewlett-Packard Development Company, L.P.

To learn more on this topic, and to connect with your peers after

the conference, visit the HP Software Solutions Community:

www.hp.com/go/swcommunity

40