Installing and Configuring the Click to edit Master title ... · PDF fileModule 3 –...

Connect. Communicate. Collaborate

Click to edit Master title styleInstalling and Configuring the perfSONAR Services

2

COURSE OBJECTIVES

By the end of this course you will be able to:

• Describe key perfSONAR services.

• Install, configure and deploy the following perfSONAR services:

• The Lookup Service

• The BWCTL Measurement Point

• The SSH / Telnet Measurement Point

• The RRD Measurement Archive

• The SQL Measurement Archive

• Identify how to interact with the Service Desk.

3

THE PERFSONAR ADMINISTRATION GUIDE

Please note that these slides do not contain detailed instructions about how to install the perfSONAR services.

For detailed instructions, please refer to the perfSONAR Administration Guide.

4

COURSE OUTLINE

Module 1 – perfSONAR Technical Overview

Module 2 – General Installation Considerations

Module 3 – Installing and Configuring the Lookup Service

Module 4 – Installing and Configuring the BWCTL MP

Module 5 – Installing and Configuring the SSH / Telnet MP

Module 6 – The Service Desk

Module 7 – Installing and Configuring the RRD MA

Module 8 – Carrying out an Installation on Debian

Module 9 – Installing and Configuring the SQL MA

Module 10 – Feedback on perfSONAR Installation and Configuration

Connect. Communicate. Collaborate

Click to edit Master title styleMODULE 1: perfSONAR TECHNICAL

OVERVIEW

6

WHAT IS PERFSONAR?

perfSONAR is:

• A project consisting of a variety of organisations and individuals

• A set of protocols that:• Assume a set of services based on defined roles.

• Define their communication syntax and semantics.

• Allow anyone to develop an implementation of a service.

• A set of code• Service implementations

7

WHAT ARE THE PERFSONAR SERVICES?

The perfSONAR services form an interoperable, distributed performance measurement middleware framework.

perfSONAR stands for PERformance Service Oriented Network monitoring Architecture.

8

DESIGN OBJECTIVES

perfSONAR is designed to be:

• Decentralised and Scaleable• Large number of networks and services, large volume of data

• Each domain can set its own security policy

• Dynamic and ‘Self-Discovering’• Add and remove components during operation

• Components ‘automatically’ become aware of one another

• Secure• Will not put participating networks at risk of attack or congest them

• Modular• Allows discrete module development

9

THREE-TIER ARCHITECTURE

The perfSONAR framework:• Is middleware.

• Is distributed between domains.

• Facilitates inter-domain performance information sharing.

perfSONAR services ‘wrap’ existing measurement tools.

10

WHAT IS A SERVICE?

A Service is a tightly defined, independent entity that has a well defined interface and can be accessed directly.

11

ARCHITECTURAL PRINCIPLES

Postulate: all measurement systems contain a combination of:

• Measurement tools• Data Storage• Security and Policy implementation • Topology information• Visualization

Services have been identified that are:• Based on these ‘roles’• Based on requirement for other

functionality such as• Service discovery, resource protection,

data formatting

12

THE PERFSONAR SERVICES FRAMEWORK (1)

perfSONAR divides measurement system tools into generic ‘families’ or ‘services’. Each service has a protocol.

13


AuthenticationService


LookupService

LookupService

ResourceProtector Service


MeasurementArchive ServiceMeasurement

Archive ServiceMeasurementPoint Service

MeasurementPoint Service

Transformation Service


ClientClient

14

THE CLIENT



LookupService

LookupService








ClientClient

For example the perfSONAR Visualisation User Interface

15

THE LOOKUP SERVICE



LookupService

LookupService








ClientClient

Registers services including their capabilities.

Facilitates complex searches.

Participates in network of LookupServices

16

THE LOOKUP SERVICE (1)

Purpose: all other services must register with the lookup service in order to participate in the framework.

17


Other services (including measurement points) register their existence with a lookup service, by delivering ‘lookup information’:

• Location

• Type of Service

Each domain has an instance of the lookup service

• These instances (will) communicate with one another

Clients find other services by querying the lookup service.

• All the client needs to know is the URL of a Lookup Service

18

THE AUTHENTICATION SERVICE



LookupService

LookupService








ClientClient

Provides authentication for clients and protects privacy.

Can be federated.

19

THE AUTHENTICATION SERVICE

Purpose: ensures client-privacy and domain security by using role-based authentication and authorisation.

20

THE MEASUREMENT POINT SERVICE



LookupService

LookupService








ClientClient

Exposes measurement tools and publishes their data.

21

MEASUREMENT POINTS

Measurement Points:

• Belong to domains (domain = a network)

• Each measurement point implementation maps to a tool that provides one or several metrics

• Examples:– One-way-loss

– Jitter

– TCP throughput

– Show commands on routers

22

THE MEASUREMENT ARCHIVE SERVICE



LookupService

LookupService








ClientClient

Exposes measurement databases and file stores.

Publishes measurement data

Avoids queries to multiple Measurement Point Services

23

MEASUREMENT ARCHIVES

Purpose: expose measurement data held in databases or file systems.

• Wrapper for any type of storage mechanism (SQL Databases, RRD files, etc)

• Provides access to recent and stored data• Collects information from Measurement Points, Transformation

Services or other Measurement Archives.– i.e. it ‘subscribes’ to these other services

• Can also write to databases and file systems

24



LookupService

LookupService








ClientClientTransforms data in avariety of ways (e.g.,aggregation, filtering,correlation).

For future development. Precise role needs to be defined.

THE TRANSFORMATION SERVICE

25

THE RESOURCE PROTECTOR SERVICE



LookupService

LookupService








ClientClientControls the comsumption of limited resources (e.g. network bandwidth).

26


Each service has a specific function.

Each instance of a service belongs to an administrative domain.

27

PROTOCOLS

perfSONAR has developed a set of protocols for sharing performance data. These:

• Assume the services set out in the framework.

• Define their communication syntax (schema) and semantics (business logic).

• Allow anyone to develop an implementation of a defined service.

• Are compliant with the Global Grid Forum’s Network Measurement Working Group (NM-WG) schema specification.

• Are based on XML over SOAP.

You can see the protocols as ‘rules and tools’ for participating in the perfSONAR framework.

28

GENERIC SERVICES AND SERVICE IMPLEMENTATIONS

Implementation: tool-specific functionality

added

RRDMeasurement

Archive serviceimplementation

Round RobinDatabase files

SQLMeasurement

Archive serviceimplementation

SQL database

Implementation: tool-specific functionalityadded

MeasurementArchive Service

Common functionality,common protocol

Implementation: tool-specific functionality

added

BWCTLMeasurementPoint service

implementationBWCTL

measurementtool

SSH / TelnetMeasurementPoint service

implementationSSH / Telnetmeasurement

tool

Implementation: tool-specific functionalityadded

MeasurementPoint ServiceCommon functionality,

common protocol

29

SERVICE IMPLENTATIONS

The perfSONAR project has also developed a set of service implementations that use the defined protocols.

• Some of these have been developed by JRA1, within the GEANT2 project for the perfSONAR pilot:






30

DATA COLLECTION, NORMALISATION AND SHARING VIA THE FRAMEWORK

Standardised Syntax

Dat

a no

rmal

izat

ion

/co

nver

sion

to X

ML

Dat

a no

rmal

izat

ion

/co

nver

sion

to X

ML

Tool

-spe

cific

com

man

ds

Mea

sure

men

t Dat

a

Standa

rdise

d Syn

tax

Dat

a no

rmal

izat

ion

/co

nver

sion

to X

ML

Dat

a no

rmal

izat

ion

/co

nver

sion

to X

ML

Tool

-spe

cific

com

man

ds

Mea

sure

men

t Dat

a

31

FRAMEWORK FACILITATES NORMALISED END-TO-END PERFORMANCE DATA (SIMPLIFIED DEPICTION)

32

THE PERFSONAR PILOT: MEASUREMENT AND ARCHIVING SERVICES

33

THE PERFSONAR PILOT

34

THE ROADMAP

Enhanced Lookup Service:

• Lookup Services (ideally one per domain) peer with one another for increased ease of data discovery.

Authentication Service:

• Collaborating with JRA5 and Edugain to produce an implementation of the service.

• A data subscriber (client) and a data producer (server) can communicate directly across domains provided that they are authorised to do so.

Further service implementations using Netflow information or packet capture features

35

TECHNICAL OVERVIEW: SUMMARY

perfSONAR will provide an infrastructure to:

• Locate data sources

• Authenticate and authorise clients

• Protect resources and ration their usage

• Retrieve, normalise, transform and share data• Only a partial infrastructure is implemented in the pilot

perfSONAR is a flexible and open framework:

• perfSONAR services can ‘wrap around’ existing data collection tools

36

THE PERFSONAR SERVICES FRAMEWORK - RECAP

How do we expose measurement tools?

• Measurement point services

How do we expose measurement databases and file stores?

• Measurement archive services

How do we transform data (aggregate, correlate, filter etc.)?

• Transformation services

How do we locate all these services and their capabilities?

• Lookup Services

How do we protect resources?

• Resource Protection Services

How do we ensure a client is allowed to access a service?

• Authorisation and Authentication Services

MODULE 2 – GENERAL INSTALLATION CONSIDERATIONS

38

GENERAL INSTALLATION PREREQUISITES (1)

All of the perfSONAR services require the following:

• Recommended operating system: Redhat Linux / Fedora.• You can install on other platforms, but perfSONAR has not been tested

on these

• Installations on Windows are not supported

• ‘wget’ command must be available on the OS– Required by installers to download software

• Perl module: LWP• Perl version 5.6.1 or higher

These prerequisites must be manually installed before you begin installation of the perfSONAR services.

39

GENERAL INSTALLATION PREREQUISITES (2)

All of the perfSONAR services except for the BWCTL Measurement Point (a non-Java application) require the following:

• Java Developers’ Kit (JDK) version 1.5 or higher• Already installed for you on the training server

• Note that the RRD MA must have JDK version 1.5 (not any other)

• Apache Ant 1.6.x

• Tomcat application server – Jakarta Tomcat

These prerequisites must be manually installed before you begin installation of the perfSONAR services.

– Note: Tomcat can be automatically installed by the bundle installer, but it recommended that you download Tomcat from the Apache web-site and manually install it before running the bundle installer.

40

OTHER INSTALLATION PREREQUISITES

In addition to the General Installation prerequisites, each service has one or more prerequisites that are specific to it.

Some must be installed manually by you before you begin to install a service:

• These are listed in subsequent course modules and are also documented in the Installation Manual.

Some can be installed automatically by the perfSONAR Bundle Installer:

• These are documented in the perfSONAR Administration Guide.

41

THE ROLE OF THE ADMINISTRATION GUIDE

The perfSONAR Administration Guide:

• Lists all pre-requisite software.

• Provides step-by-step instructions explaining how to install the perfSONAR services.

• Will be used extensively during this course.

• Will be handed out to you in hard copy.

• Should be used as your guide when you are installing the services on your own servers.

During the exercises, please point out any inconsistencies or errors in the guide to your trainer.

42

THE ROLE OF THE SERVICE DESK

The Service Desk is being set up as a single point of contact for all issues relating to perfSONAR installation, configurationand use.

More information about the service desk is provided in module 6 of the course.

43

INSTALLATION STEPS

To install the perfSONAR services:

• Unzip and untar the bundle installer (perfSONAR-2.0.tar.gz)

• Empty the CLASSPATH variable.

• Execute the bundle installer

• Follow the on-screen dialogue

For detailed instructions, refer to the perfSONAR Administration Guide.

44

HOW DOES THE BUNDLE INSTALLER WORK?

• The bundle installer will ask you whether you want to:• Install a new service

• Modify or test an existing service

• Give feedback to the perfSONAR team

• If you choose to install a new service:• The installer will ask you to choose from the six available services

• It will then:– Automatically download the appropriate installation files and execute the installation

steps in sequence

– Ask you ‘interactive questions’ about the installation

45

CONFIGURING THE SERVICES – STITCHING

Before they can be used, several of the perfSONAR service implementations require a type of configuration known as ‘stitching’.

Stitching:

• Is the process of configuring metadata for your service.• Metadata is data that describes other data.

– E.g. data units, interface name, direction of traffic etc.

• Usually involves the creation of a ‘metadata configuration file’.

46

STITCHING IN CONTEXT (1)

Stitching allows a service to understand the data it is dealing with.

Examples: Which router? Inbound or outbound traffic? What data units? Etc.

Instance ofRRD


MRTG

MRTG

Cricket

Cricket

Performance data

Performance data

Performance dataPerformance data

RRD file InOut

InOut

POS-6/0 router

GEO/0 router

RRD file

47

STITCHING IN CONTEXT (2)


Examples: Which link? Which interfaces? Location of interfaces? Etc.

48

THE ADVANTAGES OF STITCHING

Ultimately, stitched metadata tells the perfSONAR framework precisely what kind of data your service sends or can receive.

Why this approach?

• The ability to perform stitching makes service implementations flexible.

• E.g. the RRD Measurement Archive can handle data from RRD files in different networks that are structured in different ways.

• Stitching allows service implementations to deal with multiple ‘flavours’ of the same kind of data.

• E.g. a single instance of the SQL Measurement Archive service can deal with both inbound and outbound traffic since you can mark each data source as providing information about either inbound or outbound traffic.

49

OTHER CONSIDERATIONS

If you are installing multiple services in the same Tomcat instance, using the same eXist XML database (as in training):

• Each service requiring XML database access must have a different username and password

• You must ensure that each service is allocated a unique name for its own collection

• Must be manually created for the Lookup service

Ensure that you only install one instance of ant on your machine

• Putting multiple ant installations in the same path causes problems

50

BACKUPS

Once you have successfully installed and configured a service, back it up.

• Backup the ‘webapps’ directory related to the service• Located by default within your perfSONAR bundle directory

• Can be used to restore the service

• Backup your metadata configuration files

51

ACTIVITIES (1)

Installing Tomcat

• Install

• Change ports in Tomcat’s .conf file (Two ports - must be unique for each participant)

• Start up

Installing Ant

• Install

• Change path variable in bash profile to include Ant location

• Start up

52

ACTIVITIES (2)

Notes:

• Java Developers’ Kit (JDK) is already installed on the server

• A UNIX account has been created for each of you

• In your UNIX account’s home directory you will find:• Tomcat installation files

• Ant installation files

• The perfSONAR bundle installer

MODULE 3 – INSTALLING AND CONFIGURING THE LOOKUP SERVICE

54


All other services must register with the lookup service in order to participate in the framework.

55


Other services (including measurement points) register their existence with the lookup service, by delivering ‘lookup information’:

• Location (URL)

• Type of Service

• Service-specific information• For example an Measurement Point will tell the Lookup Service what

kind of measurements it can take

Clients find other services by querying the lookup service.

• All the client needs to know is the URL of the Lookup Service

56


The LS keeps Lookup Information in Lookup Storage (LSSTORE), an XML database.

• Format of information described in NMWG schema.

57


58

LOOKUP MESSAGES

Other services can interact with the lookup service to:

• Register with it.

• De-register.

• Update registration details.

• Keep-alive registration details.

• Query the lookup store.

59

LOOKUP SERVICE-SPECIFIC PREREQUISITES

The following prerequisites are necessary for Lookup Service installation:

• Java Developer’s Kit (JDK) version 1.5 or later

• Apache Ant 1.6.x

• eXist XML Database version 1.0.1 or 1.1.1• Can re-use an existing eXist XML database

• Install new eXist XML database as a webapp via Tomcat

• Some configuration is required after installation

• You must set your JAVA_HOME environmental variable to your Java directory

60

THREE-STAGE INSTALLATION

The Installer will work through three stages:

• Pre-Install• Establishes information required for the rest of the process

– E.g. the application server port and service directory

• Configure• Set important parameters:

– E.g. Service type, name, URL, XML database username and password etc.

• Deploy• Deploys the service on the application server

61

HINTS AND TIPS

• Change the password for the eXist database admin user after the service installation is complete

• Prevents use of the eXist client application to alter data

• Not necessary in training, but important in a ‘live’ context

• If you change Tomcat’s default port, ensure you configure the Lookup Service to use the amended port number.

• For any installation on Linux:• Recommended that you download Tomcat from the Apache web-site

• Recommended that you do not use the version of Tomcat supplied with the distribution

62

TESTING

In order to find out whether the service has been successfully set up, perform the following test:

• ant client-echo• This contains an XML Database connectivity test

Subsequently, you can perform the following tests:

• ant client-register

• ant client-query

• ant client-deregister

Note that test results are not printed on the screen, but are put into an XML file.

63

INTERACTION WITH THE SERVICE DESK (1)

If you encounter problems during or after installation, contact the Service Desk with the following information:

• A description of the problem

• Software versions for the following:• Lookup Service

• eXist XML DB

• Java

• Ant– Continued on next slide…

64

INTERACTION WITH THE SERVICE DESK (2)

If you encounter problems during deployment of the service or runtime please give the service desk the following information:

• Log files (especially sonar.log)

• Configuration files• service.properties

• const.properties

• log4j.properties

• components.properties

• Request / response files (if run)

• The result of “ant client-echo” (if run)

65

LOOKUP SERVICE INSTALLATION OVERVIEW

Check that all of the required prerequisite software is installed.

Check that you have the right version of each prerequisite.

Download and install prerequisite software if necessary.

• Java

• Ant

• eXist XML database

• Tomcat application Server

Install the Lookup Service

• Follow the instructions in the perfSONAR Administration Guide

5) Test your installation

66

ACTIVITIES

Lookup service Installation and Configuration

• Demonstration

• Exercise• Dependency Checks – are all of the Required Prerequisites installed?

• Installing Manual Prerequisites – the eXist XML Database

• Preparing for the Installation

• Carrying out the Installation

• Testing the Installation

• Feedback

MODULE 4 – INSTALLING AND CONFIGURING THE BWCTL

MEASUREMENT POINT

68

MEASUREMENT POINTS (1)

Purpose: expose measurement tools to provide three types of performance measurement data:

• Active measurements

• Passive measurements

• Network state information

69

MEASUREMENT POINTS (2)

Measurement Points:

• Belong to domains (domain = a network)

• Each measurement point implementation maps to a tool that provides a specific metric

• Examples:– Active delay

– One-way-loss

– Jitter

– Available bandwidth

70

BWCTL MP SPECIFICS

The BWCTL MP:

• Is implemented as typical UNIX daemon.

• Is a wrapper for the BWCTL tool.

• Receives client requests to trigger BWCTL tests.

• Sends these requests to the BWCTL tool, which executes them.

• Returns test results to the client.

• Implementation could be adapted for use with other command line tools

• Change parsing of input and output parameters as necessary

71

THE BWCTL MP: USER-ADVANTAGES

Using the BWCTL Measurement point offers you two major advantages:

• You don’t have to be logged on to the machine where the BWCTL tool is installed

• You don’t have to configure BWCTL keys

Additionally, since the BWCTL MP ‘plugs-in’ to the perfSONAR framework, it makes measurements available to the perfSONAR community, subject to local security policies.

72

FOUR STAGE INSTALLATION

The Installer will work through four stages:


– E.g. the installation directory

• Configure• Confirms which necessary Perl modules already exist on the server

and which need to be installed.

• Deploy• Asks for the user and group ID the service will be started as

• Test• Offers the opportunity to test your installation

73

INSTALLATION PREREQUISITES

In addition to the general prerequisites required for all services, the BWCTL MP requires:

• BWCTL Tool version 1.1b or higher.

• Iperf Tool version 2.0.2 or higher.

74

HINTS AND TIPS

If required Perl Modules are not found in the local Perl installation, then they will be installed into the installation directory of the BWCTL MP.

• You could choose to use a system tool

Check your network connectivity!

75

TESTING

You can use the supplied test script to check whether the service daemon and the init script have been correctly installed.

• The test script will only work if the init script is installed properly.

• You need root privileges for this.

76

ISSUES WHEN INSTALLING UNDER DEBIAN

The included init script has been designed for installation on Fedora Linux, but should also work on other LSB compliant systems.

It is known NOT to work on Debian 3.1 and earlier. It was not tested on Debian 4.0.

The test script will not work out of the box on Debian, because it depends on the init script.

77

ADDING MEASUREMENTS WITH OTHER TOOLS

The BWCTL MP source code is structured in a modular fashion.

This approach makes it relatively easy to link further command-line measurement tools to the service.

• OWAMP functionality has already been added.

78

INTERACTION WITH THE SERVICE DESK

When things go wrong:

• Typically you will need to supply the service desk with the following information:

• Which operating system you are using, including the version

• Which version of Perl you are using

• Which version of BWCTL / IPERF you are using

• The installation stage at which you began to experience problems

• The log output of the installation scripts

• Whether there is anything unusual about the way in which your system is configured

79

BWCTL MP INSTALLATION OVERVIEW

1) Check that all of the required prerequisite software is installed.

2) Check that you have the right version of each prerequisite.

3) Download and install prerequisite software if necessary.

• BWCTL Tool

• IPERF

• Perl

4) Install the perfSONAR BWCTL Measurement Point



80

ACTIVITIES

BWCTL Measurement Point Installation and Configuration

• Demonstration

• Exercise:• Dependency Checks – are all of the Required Prerequisites Installed?




• Feedback

MODULE 5 – INSTALLING AND CONFIGURING THE SSH / TELNET

MEASUREMENT POINT

82

SSH / TELNET MP SPECIFICS (1)

The SSH / Telnet Measurement Point acts as a central contact point inside a network. It:

• Is able to retrieve information from routers • uses standard protocols such as SSH or Telnet

• Issues ‘show like’ commands– Discovers configuration information: routing tables, interface configuration etc.

• Can only issue pre-configured commands and parameters• Prevents usage for malicious attacks

83

SSH / TELNET MP SPECIFICS (2)

The SSH / Telnet MP is the back-end of the Looking Glass user interface

• Similar to existing Looking Glasses on the web (traceroute.org).

84

SSH / TELNET MP: AVAILABLE COMMANDS

A client can issue the SSH / Telnet MP two types of command:

• MetadataKeyRequest:• client asks the MP what its capabilities are

• SetupDataRequest:• Client issues a command with its parameters through the MP for

execution on a specific device

85

SSH / TELNET MEASUREMENT POINT USE CASES

Example use cases:

• Retrieval of a routing table entry for a specific network

• Traceroute command

• Ping

What you can do depends upon the commands and parameters that are configured for each instance of the measurement point.

86

SSH / TELNET MP PREREQUISITES

The SSH / Telnet MP is a Java application that is deployed using Axis and Apache Tomcat.

For communication with routers SSHTools [J2SSH], JSch[JSCH] or Telnet/SSH/Terminal for Java application [JTA] are required.

87

SUPPORTED NETWORK DEVICES

• Quagga (Telnet)

• Cisco (Telnet)

• Cisco (SSH)

• Juniper (Telnet)

• Juniper (SSH)

88

THREE STAGE INSTALLATION



– E.g. installation path, Tomcat path


– Give path of configuration (stitching) file or create the file interactively


89

STITCHING

For the SSH / TELNET Measurement Point, stitching is the process of defining the available devices and the commands and parameters that users can issue to them through the measurement point.

These settings are held in the service.properties file.

The service.properties file can be:

• Automatically created via a wizard-like script that runs during installation.

or:

• Manually created and then imported during configuration.

90

AN EXAMPLE OF STITCHING

Devices:

• Cisco1 (10.10.3.24)

• JuniperBerlin (10.10.1.14)

Commands:

• PING• Ping

• 1 parameter

• Syntax (reg exp): ^[0-9]+\\.[0-9]+\\.[0-9]+\\.[0-9]+(\\/[0-9]{1,2})?$

91

HINTS AND TIPS (1)

To complete the installation and configuration you will need to know:

• The routers and other devices that you want to make available via the SSH / Telnet MP.

• The commands that you want to make available for these routers and devices.

92

HINTS AND TIPS (2)

In order to protect the SSH / Telnet Webservice you should:

• Configure the regular expressions that can be used in requests

• I.e. configure acceptable commands and parameters etc.

• Configure a reasonable access rate for a device.• The access rate is the time between two requests in which no other

request can be handled.

93

HINTS AND TIPS (3)

For any installation on Linux:

• Recommended that you download Tomcat from the Apache web-site


94

TESTING

Check in browser at the MP’s URL if the service is running.

If so, run the ant test command.

• creates a SetupDataRequest for every command of every device, and sends it to the MP.

• Results coming back are automatically checked.

95


• No known issues when installing under Debian.

• Service has been deployed and tested completely on a Debian setup.

96


When problems arise during installation, please provide the following information:

• Java Version

• An URL which we can use to test remotely

• The service.properties file

97

SSH / TELNET MP INSTALLATION OVERVIEW




• Java

• Ant


4) Install the perfSONAR SSH / Telnet Measurement Point



98

ACTIVITIES

SSH / Telnet Measurement Point Installation and Configuration

• Demonstration

• Exercise• Dependency checks – are all of the Prerequisites Installed?



• Examining the service.configuration file


• Feedback

MODULE 6 –THE SERVICE DESK

100

THE ROLE OF THE SERVICE DESK

The Service Desk is a single Point of Contact for:

• The 5 MDM pilot deployers

• GEANT2 community NOC and PERT users

Through the service desk you can:

• Report incidents about the installation, configuration, operation and utilisation of the web services and visualisation tools.

• Ask questions:• About the MDM service

• About the installation, configuration and operation of perfSONAR web-services and visualisation tools

• Raise enhancement Requests

101

THE DUTIES OF THE SERVICE DESK (1)

The duties of the service desk will include:

• Logging all calls, events and requests.

• Acting as the ‘first layer’ of incident management.• Taking overall ownership of incidents

• Escalating and re-assigning them until they are resolved

• Monitoring the MDM service, the web-services and the visualisation tools.

102

INCIDENT MANAGEMENT EXAMPLE – KNOWN ERRORS

Several incidentmanagement processeshave been defined. Thisis one example.

103

THE DUTIES OF THE SERVICE DESK (2)

The duties of the service desk will include:

• Gathering feedback on the service provided.

• Generating regular reports on:• Incidents.

• Lessons learned.

• Missing pieces (continuous improvement).

• Offering a managed service for FCCN and GEANT2.

104

SUCCESS FACTORS

Critical success factors are:

• Well defined and efficient support processes

• Trouble ticketing system, CMDB, monitoring tools

• Documentation

• Training

105

SUPPORTED SOFTWARE

The Service Desk will support the:

• Installation, configuration and operation use of:• SQL MA, RRD MA, SSH / Telnet MP, BWCTL MP, L2 status MP, LS.

• Use and configuration of CNM and E2EMON.

• Installation and use of the perfSONAR UI and of the Looking Glass.

• Use of:• The Hades MA.

• The Hades Visualisation Tool.

• Use and operation of Hades monitoring tools.

106

L2 STATUS

Please note that the L2 status MA cannot be supported until released.

• E2EMon visualisation will be supported at the same time.

107

INFORMATION TO PROVIDE TO THE ASD

NREN MDM Contact detail

Deployed web-services IP addresses, URL, locations, the GPS installation quotes

When web-services are installed, so that the ASD can start monitoring it

For equipment shipment

When a planned maintenance will affect the MDM service

108

CONTACT DETAILS

E-mail: [email protected]

Phone: +44 1223 371 380 (available from 18th of June)• In the meanwhile, please call +44 1223 371 3xx

mailto:[email protected]

MODULE 7 – INSTALLING AND CONFIGURING THE RRD MEASUREMENT

ARCHIVE

110

MEASUREMENT ARCHIVES

Purpose: Measurement archives expose measurement data held in databases or file systems. They:

• Are wrappers for any type of storage mechanism (SQL Databases, RRD files, etc).

• Provide access to recent and stored data.

• Can also be used to write information to databases and file systems.

111

THE RRD MEASUREMENT ARCHIVE (1)

The Round Robin Database (RRD) Measurement Archive is a wrapper for binary files of the RRDTool format.

112

THE RRD MEASUREMENT ARCHIVE (2)

The RRD Measurement Archive has two main functions:

• Writing and storing measurement data in RRD files• E.g. information from perfSONAR Measurement Points collected as a

result of regularly scheduled or on-demand measurements

• Publishing measurement data held in RRD files to client applications

• E.g. Measurements that have been stored in RRD files by non-perfSONAR applications such as MRTG (Multi Router Traffic Grapher) or Cricket

113

THE RRD MEASUREMENT ARCHIVE IN CONTEXT

114

RRD MA INSTALLATION OVERVIEW

Follow these steps:

• Install the perfSONAR RRD Measurement Archive Service

• Generate and populate metadata configuration files

• Deploy the Web Service

• Test the deployed service to see if it is working

115

THE RRD MA: THREE STAGE INSTALLATION


• Pre-Install• Establishes information and performs tasks required for the rest of the

process– Collects information such as installation path, Tomcat port, eXist admin user

password

– Compiles the RRD J tool


– E.g. name and path of metadata configuration file, location of file-store, whether or not to automatically register with a lookup service


116

TESTING

You can execute a test script that sends a series of test requests to the service.

• The metadata configuration file used by the service must be the test metadata configuration file that is supplied with the installation files.

• The command to run is “ant test”

Once the test is complete, you can analyse the responses generated by the service.

• If there are problems, then clear error messages will be displayed in the responses.

• If there is no response, then there is a problem.

117

STITCHING FOR THE RRD MEASUREMENT ARCHIVE

In the context of a measurement archive, stitching:

• Is the process of configuring metadata that underlies the performance data handled by your archive.

• Metadata is data that describes other data– E.g. data units, interface name, direction of traffic etc.

• Involves the creation of a ‘metadata configuration file’.

Ultimately, the metadata configuration file tells the perfSONAR framework what kind of data the archive stores.

118

RRD MA STITCHING IN CONTEXT


Examples: Which router? Inbound or outbound traffic? What data units? Etc.

Instance ofRRD


MRTG

MRTG

Cricket

Cricket

Performance data

Performance data

Performance dataPerformance data

RRD file InOut

InOut

POS-6/0 router

GEO/0 router

RRD file

119

THE METADATA CONFIGURATION FILE

The Metadata Configuration File:

• Is an xml file that will expose information describing your network’s RRD files.

• Conforms to NMWG’s XML schema.

The default perfSONAR installation provides samples of:

• A metadata configuration file

• An RRD file

The sample metadata configuration file:

• Is provided as a template only and should be changed to correctly describe your own RRD archives.

120

STITCHING FOR THE RRD MA – THREE STEP PROCESS

Step 1 – Understand the metadata configuration file structure

Step 2 – Create your own metadata configuration file

Step 3 – Apply your metadata configuration file to your instance

121

STITCHING FOR THE RRD MA – STEP 1

To Understand the metadata configuration file structure you should:

• Refer to the metadata configuration file guide

• Study the sample metadata configuration file

• Gather information about each data source in your RRD files.

• A data source usually equates to a measurement of an interface’s traffic in a single direction.

• You should create a a metadata ‘chain’ for each data source.

• There is no limit to the number of metadata chains you configure.

122

STITCHING FOR THE RRD MA – CHAINS

A chain:

• Describes measurement data for a single interface, for one direction only

• Consists of two connected parts:• ‘Metadata’

– Host Name (DNS entry of the router containing the interface)

– IPV4 interface address

– Interface’s name

– Interface’s description

– Traffic direction (in or out)

– Authentication realm

– Capacity / interface speed

• ‘Data’– Name and path of the RRD file

– Data source within the RRD file

– Data storage unit (example: bps or Bps)

123

RRD MA METATDATA CHAINS – EXAMPLE OF METADATA

124

RRD MA METATDATA CHAINS – EXAMPLE OF DATA

125

STITCHING FOR THE RRD MA: STEP 2

Create your metadata configuration file

• Can be created by manual file editing.

• Can be generated using contribution scripts or your own scripts

• Contribution scripts are available for some tools– E.g. MRTG

• If scripts do not already exist for your tool, you can create your own script based on existing contribution scripts

• Manual file creation for a large number of interfaces would be labour-intensive and time-consuming

• When something changes in your network, update your file• Recommended that you automate this process

126

STITCHING FOR THE RRD MA: STEP 3

Apply your metadata configuration file to your instance

• Three possible methods:• Use the eXist XML database web-based User Interface

– Recommended method

• Use the perfSONAR installer– Not covered in this course; refer to the installation guide for details

• Use the service installation scripts– Not covered in this course; refer to the installation guide for details

127

MAKING THE MEASUREMENT ARCHIVE AVAILABLE TO THE VISUALISATION TOOLS

A user client application needs to know the address of the Measurement Archive.

• This information can be taken from the Lookup Service

• Contact Andreas Hanemann at DFN in order to get CNM to use your service.

• When the Lookup Service is installed, please notify the perfSONAR UI team.

128


In addition to the general prerequisites required for all services, the RRD MA requires:

• RRDTool version 1.2.x• The RRD MA can be installed on any Linux platform

129

HINTS AND TIPS

• During the installation, you will need to supply the location of the RRD Tool.

• It is better to store metadata configuration information in an XML database than in a text file as this improves performance.

• To run the RRD MA on a 64-bit machine, you must be consistent in your use of 32 bit or 64 bit software

• i.e. either all software (Java, rrdjtool, rrdtoo libs, rrd files) must be compiled for 32-bit or all software must be compiled for 64-bit.



130


The librrd2-dev package must be installed.

131


When problems arise during installation, please provide the following information as a minimum:

• Log files

• Configuration files

• Request / response messages

132

RRD MA INSTALLATION OVERVIEW




• Java

• Ant

• RRDTool



4) Install the perfSONAR RRD Measurement Archive



133

ACTIVITIES

RRD Measurement Archive Installation and Configuration

• Demonstration

• Exercise• Dependency Checks – are all of the Required Prerequisites Installed?




• Stitching – Editing the RRD Metadata Configuration File

• Testing the Results of Stitching

• Feedback

MODULE 8 – CARRYING OUT AN INSTALLATION ON DEBIAN

135

OVERVIEW OF DEBIAN

• Very popular OS among the Open-Source alternatives• Stability

• Powerful package management, Easy upgrade

• Huge Community

• Comes in 3 flavours• Stable: Integrate robust package. (Not always recent but… Security patched !!

AKA ETCH)

• Testing: The next Stable version (AKA LENNY)

• Unstable: Bleeding edge package (AKA SID)

• “Stable” is the way to go • Security

• Robust service due to package stability

136

RRD MA INSTALLATION: DEBIAN BASE INSTALLATION

• Get the ISO from www.debian.org• Either download the full distribution but …

• If possible prefer the NETINSTALL ISO (163 Meg)

• NETINSTALL provides up to date packages

• Install the minimum distribution (server/standard install)• A rule of thumb for production environment is …

• To always use the KISS method

• “If you don’t need it, don’t install it”

137

REMINDER: APT, DPKG, LOCATE ARE ALL FRIENDS

• apt• Package/Distribution management tool

• After “minimum install”, update source.list, chose your favorite mirror and add “non-free” repository

• Then … apt-get update; apt-get dist-upgrade

• apt-cache search , if you are lost !

• dpkg• -l <pkg_name>: List all packages installed

• -L<pkg_name>: List all files part of a package

• locate• Locate <file_name> “Where is <file_name>”

• Updatedb when to need to locate a file on the file system

138

PERFSONAR BUNDLE INSTALLER / RRD MA DEPENDENCIES

• ssh

• java

• rrdtool and … librrd2-dev

• wget

• perl

• libwww-perl (AKA LWP)

• gcc

• ant and don’t forget… ant-optional !

• Debian Tomcat or PerfSONAR Tomcat… It’s up to you !

139

TOMCAT

• Debian Tomcat• Disable security manager

• Enable TOMCAT but define a TOMCAT security policy

• Tomcat from bundle installer• Preferred method

• Easy upgrade

• Security manager disabled

140

POTENTIAL ISSUES

• Environment variable• LD_LIBRARY_PATH No RRD-GRAPH in PerfSONARUI

• JAVA_HOME not set TOMCAT won’t start

• Enable system wide environment variable• Put it in /etc/profile

• export LD_LIBRARY_PATH …

• Export JAVA_HOME …

• Tomcat refuse to install the services on “packaged Tomcat”

• Disable Tomcat security manager or …

• Add security policy

141

FINAL TOUCH

• Starting PerfSONAR service at system startup• /etc/init.d/rc.local script

• Run the service as perfsonar user ! (Avoid root …)

• Diagnostic tools• netstat –a | grep LISTEN check that Tomcat is listening at port

defined and also when the MA is interrogated

• ps –def | grep perfsonar check the process is running

142

LAST BUT NOT LEAST…. SECURITY

• PerfSONAR services provide access to sensitive infomation so …

• General rules• Permit « anyone» that wants to access the service ONLY

• Permit remote administration using SSH from your LAN

• Permit ICMP echo request from 194.141.0.9

• Available tools on RRD-MA host• TCPD (AKA TCP wrapper)

• IPTABLES

• Tripwire

• Available tools on the local LAN• Router access-list

• Switch VACL/Private VLAN

• Etc.

143

QUESTIONS?

Comments and suggestions are of course welcome !

MODULE 9 – INSTALLING AND CONFIGURING THE SQL MEASUREMENT

ARCHIVE

145

THE SQL MEASUREMENT ARCHIVE (1)

The SQL Measurement Archive is a wrapper that allows perfSONAR to access data stored in a database.

• utilisation and path status are currently supported

• Supports MySQL or PostgreSQL databases• Theoretically other databases can be used, but these have not been

tested

146

THE SQL MEASUREMENT ARCHIVE (2)

The SQL Measurement Archive has two main functions:

• Publishing measurement data from a database to client applications

• Writing and storing measurement data to a database

When installing, you can either:

• Setup a new database• Schema setup scripts included in installation

• Use an existing database• Configure the SQL MA to work with your existing database

147

STITCHING FOR THE SQL MEASUREMENT ARCHIVE

In the context of a measurement archive, stitching:

• Is the process of configuring metadata that underlies the performance data handled by your archive.

• Metadata is data that describes other data– E.g. data units, interface name, direction of traffic etc.

• Involves the creation of a ‘metadata configuration file’.

Ultimately, the metadata configuration file tells the perfSONAR framework what kind of data the archive stores.

148

SQL MA STITCHING IN CONTEXT


Examples: Which link? Which interfaces? Location of interfaces? Etc.

149

THE METADATA CONFIGURATION FILE

Your Metadata Configuration File will expose information describing your network’s:

• Interfaces and their utilisation data.

• Links and link status

The default perfSONAR installation provides samples of:

• A metadata configuration file

• A SQL database

The sample metadata configuration file:

• Is provided as a template only.

• Should be changed to correctly describe the your own SQL database.

150

STITCHING FOR THE SQL MA – THREE STEP PROCESS

Step 1 – Understand the metadata configuration file structure

Step 2 – Create your own metadata configuration file

Step 3 – Apply your metadata configuration file to your instance

151

STITCHING FOR THE SQL MA – STEP 1

To understand the metadata configuration file structure you should refer to:

• The metadata configuration file guide

• The sample metadata configuration file

There are two types of SQL MA metadata:

• Metadata describing interface utilisation• Similar to the RRD MA’s metadata

• Metadata describing path status

• Unique to the SQL MA

152

SQL MA ARCHITECTURE

153

UNDERSTANDING PATH STATUS METADATA

SQL MA path status metadata can be subdivided into:

• Node metadata

• Link metadata

First create metadata describing each node.

Then create metadata describing each link.

154

LINK METADATA AND NODE METADATA

When creating link and node metadata:

• You need to create a metadata chain for each link that you want to collect data about.

• Within the link’s metadata chain, associate two nodes with the link.– I.e. the start and demarcation points of the link

• The same node can be referred to in the metadata of multiple links.

155

STITCHING – METADATA FOR NODES AND LINKS

Node metadata includes:

• Node ID and name

• Node’s country, city and institution

• Node’s latitude and longitude

Link Metadata includes:

• Link name and global name

• Name of related nodes

• Roles of related nodes:• I.e. end point or demarcation point

156

STITCHING – PATH STATUS CHAINS

A path status chain describes measurement data for a single link and refers to multiple nodes.

Consists of two connected parts:

• ‘Metadata’• For Nodes

• For Links

• ‘Data’• Name of relational database configuration file

• Path ID

157

SIMPLIFIED LINK PATH METADATA EXAMPLE

158

NODE METADATA EXAMPLE

159

LINK CHAINS – AN EXAMPLE

160

LINK CHAIN DATA – AN EXAMPLE

161

SQL MA ARCHITECTURE

162

STITCHING FOR THE SQL MA– INTERFACE UTILISATION CHAINS

Interface utilisation chains:

• Describes measurement data for a single interface, for one direction only.

• Are the same as RRD Measurement Archive link chains• However the data is different because the storage type is different

• Consist of two connected parts:• ‘Metadata’

– Host Name (DNS entry of the router containing the interface)

– IPV4 interface address

– Interface’s name

– Interface’s description

– Traffic direction (in or out)

– Authentication realm

– Capacity / interface speed

• ‘Data’– Name of relational database configuration file

163

STITCHING FOR THE SQL MA: STEP 2

Create your own metadata configuration file.

• Can be created by manual file editing.

• Generate your file using contribution scripts or your own scripts.

• When something changes in your network, update your file.• Recommended that you automate the process

164

STITCHING FOR THE SQL MA: STEP 3

Apply your metadata configuration file to your instance

• Three methods:• Use the eXist XML database web-based User Interface

– Recommended method

• Use the perfSONAR installer– Not covered in this course; refer to the installation guide for details

• Use the service installation scripts– Not covered in this course; refer to the installation guide for details

165

THREE STAGE INSTALLATION OF THE SQL MA

The Installer will work through three stages for the SQL MA:


– E.g. installation path, Tomcat port, eXist admin user password


– E.g. setup the database

– username and password for database

– name and path of metadata configuration file

– location of file-store

– whether or not to automatically register with a lookup service


166

MAKING THE MEASUREMENT ARCHIVE AVAILABLE TO THE VISUALISATION TOOLS

A user client app needs to know the address of MA (can be taken from LS).

167

TESTING

You can execute a test script that sends a series of test requests to the service.

• The metadata configuration file used by the service must be the test metadata configuration file that is supplied with the installation files.

• The command to run is “ant test”

Once the test is complete, you should analyse the responses generated by the service.

168


In addition to the general prerequisites required for all services, the SQL MA requires:

• One of the following relational databases:• Mysql version 5.0

Or

• PostgreSQL version 8.x

169

HINTS AND TIPS

The SQL MA can be installed on any Linux platform.

Metadata Configuration must be held in XML database.



170


When problems arise during installation, please provide the following information as a minimum:

• Log files

• Configuration files

• Request / response messages

171

SQL MA INSTALLATION OVERVIEW




• Java

• Ant

• MySQL (or postgreSQL) database



4) Install the perfSONAR SQL Measurement Archive



172

ACTIVITIES

SQL Measurement Archive Installation and Configuration

• Demonstration

• Exercise• Dependency Checks – are all of the Required Prerequisites Installed?




• Stitching – Editing the SQL Metadata Configuration File

• Testing the Results of Stitching

• Feedback

MODULE 10 – FEEDBACK ON PERFSONAR INSTALLATION AND CONFIGURATION

174

ACTIVITIES

Please use the forms provided to give us feedback about the perfSONAR installation and configuration process.

• This asks for your feedback about the installation process and the associated software, not about the training course

• You will be asked for feedback about the training course separately

• For both the BWCTL MP and the JAVA Services• Provide three positive points and three things to enhance about the

installation and configuration process– No inter-personal issues

– No generalities. Be specific.

– Make suggestions about how to improve

• Provide answers to quantitative questions

175

FOR MORE INFORMATION

• www.geant2.net

• www.dante.net

• For latest news and factsheets http://www.geant2.net/media

• For research activities http://www.geant2.net/research

• The perfSONAR FAQ and mailing list can be found at www.perfsonar.net.

• The WIKI at wiki.perfsonar.net is also a valuable source of information.

http://www.geant2.net/

http://www.dante.net/

http://www.geant2.net/media

http://www.geant2.net/research

http://www.perfsonar.net/

176

RECAP OF COURSE OBJECTIVES

By the end of this course you will be able to:

• Describe key perfSONAR services.

• Install, configure and deploy the following perfSONAR services:






• Identify how to interact with the Service Desk.

Installing and Configuring the Click to edit Master title ... · PDF fileModule 3 –...

Documents

Transcript of Installing and Configuring the Click to edit Master title ... · PDF fileModule 3 –...