Solace Integration with Nagios Monitoring toolset...Solace Nagios Toolset Integration 8 3.3 Post...
Transcript of Solace Integration with Nagios Monitoring toolset...Solace Nagios Toolset Integration 8 3.3 Post...
Copyright copy Solace Systems Inc
httpwwwsolacesystemscom
Solace Integration with
Nagios Monitoring toolset Document Version 05
December 2015
This documents is an integration guide for monitoring Solace Message Routers with Nagios and
related tools such as Nagiograph and RRDTool
Nagios is an open source monitoring system widely deployed to monitor IT infrastructure including
computer systems network applications and services
Solace message routers unify many kinds of data movement so companies can efficiently and
cost-effectively move all of the information associated with better serving customers and making
smarter decisions The Solace 3560 message router is the highest performance data movement
technology available with the capacity and robustness to support the most demanding enterprise
messaging big data cloud computing and Internet of Things applications
Solace Nagios Toolset Integration
2
Table of Contents
Contents Table of Contents 2 1 Overview 4
11 Audience 5 12 Related Documentation 5 13 Prerequisites 5
131 Requirements 5 132 Assumptions 5
2 Why Solace 6 Superior Performance 6 Robustness 6 Simple Architecture 6 Simple Operations 6 Cost Savings 6
3 Nagios Setup 7 31 Prerequisites 7 32 Installation 7 33 Post install setup 8
331 HTTP Setup 8 332 Firewall setup 8 333 Nagios Admin User 8
34 Verification 9 4 Nagiosgraph Setup 10
41 Prerequisites 10 42 Installation 10 43 Post install setup 11
431 Restarting Services 11 44 Verification 12
5 Solace Nagios Plugin Setup 14 51 Prerequisites 14 52 Installation 14 53 Post Install Setup 16 54 Verification 16
6 NSCA Setup 18 61 Prerequisites 18 62 Installation 18 63 Post install setup 19 64 Verification 20
641 Sending Host Alert 20 642 Sending Service Alert 21
7 NRPE Setup 22 71 Prerequisites 22 72 Installation 22
721 Server (CentOS) 22 722 Client (Ubuntu) 22
73 Post Install Configuration 23 731 Client Configuration 23 732 Server Configuration 23
74 Verification 24 75 NCSA vs NRPE 24
8 Cacti Setup 25 81 Prerequisites 25
811 Cacti Dependencies 25 82 Installation 25 83 Post Install Setup 25
831 Add Custom Data Input method 26 832 Add Custom Data Template 27 833 Add a custom Graph Template 28
Solace Nagios Toolset Integration
3
834 Add Solace Appliance as Device 29 84 Verification 31
841 Basic Install 31 842 Solace host check 32 843 Solace VPN Metrics Sample 32
9 Additional Topics 34 91 Admin Tasks 34
911 Creating custom Maps 34 912 Adding additional routers VPNs 34 913 Putting Nagios on Verbose mode 34 914 Modifying iptables 35
92 Troubleshooting 36 921 Nagios not starting up 36 922 Canrsquot connect to HTTP (or other port) from outside 36 923 CGI Error in showcgi 36 924 CGI Error in Solace graphs 37 925 Canrsquot stat command file error 37
10 Setup Scripts and Config files 38 101 Install Info 38 102 Nagios 39
1021 setup_nagiossolace 39 1022 mk_nagiossolacecfg 45 1023 Solace Nagios Config (cfgsolacecfg) 47 1024 Template router (cfgsolace-templateroutercfg) 50 1025 Template and Sample router Config 53
103 Solace Nagios Plugin 53 104 NSCA 54
1041 setup_nsca 54 1042 send_ngalertpl 57 1043 nsca_init 61 1044 NSCA Config 63
105 NRPE 64 1051 Sample server config file 64 1052 Sample Client Config 65
Solace Nagios Toolset Integration
4
1 Overview Nagios (previously NetSaint) is an open-source monitoring and alerting system thatrsquos widely used to monitor systems
networks and infrastructure Objects monitored by Nagios are split into two categories hosts (physical machines) and
services (particular functionalities) Nagios does not perform any host or service checks on its own and relies on plugins
to do this This makes it a very modular and flexible solution
Extending this ldquomonitoring by pluginrdquo paradigm of Nagios Solace provides a set of plugin scripts that perform the
Appliance and VPN monitoring Following Nagios convention the plugins add host (router) and service (VPN
resources) categories
Nagiograph parses output and performance data from Nagios plugins and generates graphs and HTML pages for
reporting Nagiograph stores data in RRD (Round Robin Database) files as time series data RRDTool is an Open
source parser that integrates into Shell scripts Perl Python Ruby Tcl etc
The following picture illustrates different components and high level data flow
Solace Nagios Toolset Integration
5
11 Audience This document is primarily intended for Solace administrators and operations teams responsible for setting up and
monitoring infrastructure
12 Related Documentation These documents contain information related to the feature defined in this document
Document ID Document Title Document Source
[Solace-Portal] Solace Developer Portal httpdevsolacesystemscom
[Solace-FG] Solace Messaging Platform ndash Feature
Guide
httpdevsolacesystemscomdocsmessaging-
platform-feature-guide
[Solace-FP] Solace Messaging Platform ndash Feature
Provisioning
httpdevsolacesystemscomdocsmessaging-
platform-feature-provisioning
[Solace-CLI] Solace Message Router Command Line
Interface Reference
httpdevsolacesystemscomdocscli-reference
Table 1 - Related Documents
13 Prerequisites
131 Requirements
The installation and setup steps require that you have root access on the CentOS server
The installation and setup steps also require that you have CLI access to the Solace appliance
The Solace appliance should be reachable over network from the CentOS server running Nagios
132 Assumptions
This document is based on SolOS version 711 though earier versions (such as SolOS 62) should work
without any changes
Setup steps in this guide were tested on CentOS 63 CentOS 70 (both 64 Bit Minimal install) Command
syntax and file locations would vary on other flavors of Linux
Solace Nagios Toolset Integration
6
2 Why Solace Solace technology efficiently moves information between all kinds of applications users and devices anywhere in the
world over all kinds of networks Solace makes its state-of-the-art data movement capabilities available via hardware
and software ldquomessage routersrdquo that can meet the needs of any application or deployment environment Solacersquos
unique solution offers unmatched capacity performance robustness and TCO so our customers can focus on seizing
business opportunities instead of building and maintaining complex data distribution infrastructure
Superior Performance Solacersquos hardware and software messaging middleware products can cost-effectively meet the performance needs of
any application with feature parity and interoperability that lets companies start small and scale to support higher
volume or more demanding requirements over time and purpose-built appliances that offer 50-100x higher
performance than any other technology for customers or applications that require extremely high capacity or low
latency
Robustness Solace offers high availability (HA) and disaster recovery (DR) without the need for 3rd party products and fast failover
times no other solution can match Distributing data via dedicated TCP connections ensures an orderly well-behaved
system under load and patented techniques ensure that the performance of publishers and high-speed consumers is
never impacted by slow consumers
Simple Architecture Modern enterprises run applications that demand many kinds of data movement such as persistent messaging web
streaming WAN distribution and cloud-based communications By supporting all kinds of data movement with a unified
platform that can be deployed as a small-footprint software broker or high-capacity rack-mounted appliance Solace lets
architects design an end-to-end infrastructure thatrsquos easy to build applications for integrate with existing technologies
secure and scale
Simple Operations Solacersquos solution features a shared administration framework for all kinds of data movement deployment models and
network environments so itrsquos easy for IT staff to deploy monitor manage and upgrade their Solace-based messaging
environment
Cost Savings Solace reduces expenses with high-capacity hardware flexible software and the ability to deploy the right solution for
each problem Solacersquos support for many kinds of messaging lets you replace multiple messaging products with just
one built-in HA DR WAN and Web functionality eliminate the need for third-party products
Solace Nagios Toolset Integration
7
3 Nagios Setup
31 Prerequisites A working Nagios installatin requires the following packagestools
Httpd (Apache)
RRDTool
GD Grahics library and related Perl modules
NRPE (Nagios Remote Plugin Executor)
yum install -y httpd php gcc glibc glibc-common gd gd-devel make net-snmp wget
yum install rrdtool perl-rrdtool perl-GD
yum install perl-CGI perl-Time-HiRes
1 Enabling EPEL
The steps for enabling Extra Packge Repositor (EPEL) are release specific and please check for your install
CentOS 7
wget httpdlfedoraprojectorgpubepel7x86_64eepel-release-7-5noarchrpm
rpm -ivh epel-release-7-5noarchrpm
CentOS 63
rpm -Uvh httpdlfedoraprojectorgpubepel6x86_64epel-release-6-8noarchrpm
rpm -Uvh httprpmsfamillecolletcomenterpriseremi-release-6rpm
32 Installation
yum -y install nagios nagios-plugins-all nagios-plugins-nrpe nrpe
Solace Nagios Toolset Integration
8
33 Post install setup
331 HTTP Setup
Setting up an HTTP server (such as Apache) is beyond the scope of this document Pl refer to the documentation for
your HTTP server Nagios setup should have dropped the required config file to setup URL Aliases The file is typically
ltOptioalConfigDirgtnagioscfg (Eg etchttpdconfdnagioscfg)
If Checking Webadmin below doesnrsquot work check the httpd config and logs to see if this file was processed
332 Firewall setup
Firewall setup and making changes to existing firewall configuration is again beyond the scope of this document For
eg if you are using iptables (default firewall on CentOS) then you may need to explicitly allow incoming connection on
required ports (Refer Additional Topics Section)
333 Nagios Admin User
Create password for nagiosadmin user for web access
htpasswd ndashc etcnagiospasswd nagiosadmin
systemctl restart httpd
systemctl restart nagios
Solace Nagios Toolset Integration
9
34 Verification Point the webbrowser to httpyour-server-ip[port]nagios
Login with the username nagiosadmin and the password above step (htpassword command)
Sample page is shown below
Solace Nagios Toolset Integration
10
4 Nagiosgraph Setup
41 Prerequisites NagiosGraph installation requires the following packagestools
RRDTool and related Perl modules
GD Graphics Library and related Perl modules
yum install rrdtool perl-rrdtool perl-GD
42 Installation
cd optpkg or path of your choice
wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-
152targzwget
tar xvzf nagiosgraph-152targz
cd nagiosgraph-152
installpl --check-prereq
hellip
installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-
file=varlognagiosperfdatalog --var-dir=varnagios
hellip
hellip
Continue with this configuration [y] y
hellip
Solace Nagios Toolset Integration
11
43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if
necessary at the top of the script
Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific
tools are not in place or the Config files not setup corresponding steps will be skipped
431 Restarting Services
chkconfig httpd on ampamp chkconfig nagios on
systemctl restart httpd
systemctl restart nagios
setup_nagiossolace
Checking required files
hellip
Setting up config files
hellip
Restarting services
nagios
nagios (pid 4155) is running
httpd
hellip
Solace Nagios Toolset Integration
12
44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the
Nagiosgraph install Sample page is shown below
Solace Nagios Toolset Integration
13
Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph
for one of the services (HTTP) for a day is shown below
Solace Nagios Toolset Integration
14
5 Solace Nagios Plugin Setup
51 Prerequisites Solace Nagios plugin requirs the following tools packages
Perl modules to handle XML amp Nagios
Solace pluging package
yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin
perl -MCPAN -e shell
cpangt install NagiosPlugin
cpangt install NagiosPluginDieNicely
cpangt install LWPUserAgent
cpangt install NagiosConfig
cpangt exit
52 Installation
1 Create Sample file
Create a sample router Config This file used by the setup script (below) to create commands for this config This step
can also be repeated for other routersvpns using the command mk_nagiossolacecfg script
cat cfgsolace-vmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=19216856102
HOSTPORT=8080
USERNAME=admin
PASSWORD=
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
Solace Nagios Toolset Integration
15
2 Setup Plugin
Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already
setup the related steps will be skipped
mk_nagiossolacecfg cfgsolace-vmr2cfg
Generating Nagios config for Solace appliance
Using cfgfile cfgsolace-vmr2cfg
Using template file cfgsolace-templaterouter-no_bi_bridgecfg
Generating config
etcnagiosobjectssolace-solace-vmr2cfg
Setting up config files
etcnagiosnagioscfg
Solace Nagios Toolset Integration
16
53 Post Install Setup
54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will
be added to the web page
A sample entry is shown below
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
2
Table of Contents
Contents Table of Contents 2 1 Overview 4
11 Audience 5 12 Related Documentation 5 13 Prerequisites 5
131 Requirements 5 132 Assumptions 5
2 Why Solace 6 Superior Performance 6 Robustness 6 Simple Architecture 6 Simple Operations 6 Cost Savings 6
3 Nagios Setup 7 31 Prerequisites 7 32 Installation 7 33 Post install setup 8
331 HTTP Setup 8 332 Firewall setup 8 333 Nagios Admin User 8
34 Verification 9 4 Nagiosgraph Setup 10
41 Prerequisites 10 42 Installation 10 43 Post install setup 11
431 Restarting Services 11 44 Verification 12
5 Solace Nagios Plugin Setup 14 51 Prerequisites 14 52 Installation 14 53 Post Install Setup 16 54 Verification 16
6 NSCA Setup 18 61 Prerequisites 18 62 Installation 18 63 Post install setup 19 64 Verification 20
641 Sending Host Alert 20 642 Sending Service Alert 21
7 NRPE Setup 22 71 Prerequisites 22 72 Installation 22
721 Server (CentOS) 22 722 Client (Ubuntu) 22
73 Post Install Configuration 23 731 Client Configuration 23 732 Server Configuration 23
74 Verification 24 75 NCSA vs NRPE 24
8 Cacti Setup 25 81 Prerequisites 25
811 Cacti Dependencies 25 82 Installation 25 83 Post Install Setup 25
831 Add Custom Data Input method 26 832 Add Custom Data Template 27 833 Add a custom Graph Template 28
Solace Nagios Toolset Integration
3
834 Add Solace Appliance as Device 29 84 Verification 31
841 Basic Install 31 842 Solace host check 32 843 Solace VPN Metrics Sample 32
9 Additional Topics 34 91 Admin Tasks 34
911 Creating custom Maps 34 912 Adding additional routers VPNs 34 913 Putting Nagios on Verbose mode 34 914 Modifying iptables 35
92 Troubleshooting 36 921 Nagios not starting up 36 922 Canrsquot connect to HTTP (or other port) from outside 36 923 CGI Error in showcgi 36 924 CGI Error in Solace graphs 37 925 Canrsquot stat command file error 37
10 Setup Scripts and Config files 38 101 Install Info 38 102 Nagios 39
1021 setup_nagiossolace 39 1022 mk_nagiossolacecfg 45 1023 Solace Nagios Config (cfgsolacecfg) 47 1024 Template router (cfgsolace-templateroutercfg) 50 1025 Template and Sample router Config 53
103 Solace Nagios Plugin 53 104 NSCA 54
1041 setup_nsca 54 1042 send_ngalertpl 57 1043 nsca_init 61 1044 NSCA Config 63
105 NRPE 64 1051 Sample server config file 64 1052 Sample Client Config 65
Solace Nagios Toolset Integration
4
1 Overview Nagios (previously NetSaint) is an open-source monitoring and alerting system thatrsquos widely used to monitor systems
networks and infrastructure Objects monitored by Nagios are split into two categories hosts (physical machines) and
services (particular functionalities) Nagios does not perform any host or service checks on its own and relies on plugins
to do this This makes it a very modular and flexible solution
Extending this ldquomonitoring by pluginrdquo paradigm of Nagios Solace provides a set of plugin scripts that perform the
Appliance and VPN monitoring Following Nagios convention the plugins add host (router) and service (VPN
resources) categories
Nagiograph parses output and performance data from Nagios plugins and generates graphs and HTML pages for
reporting Nagiograph stores data in RRD (Round Robin Database) files as time series data RRDTool is an Open
source parser that integrates into Shell scripts Perl Python Ruby Tcl etc
The following picture illustrates different components and high level data flow
Solace Nagios Toolset Integration
5
11 Audience This document is primarily intended for Solace administrators and operations teams responsible for setting up and
monitoring infrastructure
12 Related Documentation These documents contain information related to the feature defined in this document
Document ID Document Title Document Source
[Solace-Portal] Solace Developer Portal httpdevsolacesystemscom
[Solace-FG] Solace Messaging Platform ndash Feature
Guide
httpdevsolacesystemscomdocsmessaging-
platform-feature-guide
[Solace-FP] Solace Messaging Platform ndash Feature
Provisioning
httpdevsolacesystemscomdocsmessaging-
platform-feature-provisioning
[Solace-CLI] Solace Message Router Command Line
Interface Reference
httpdevsolacesystemscomdocscli-reference
Table 1 - Related Documents
13 Prerequisites
131 Requirements
The installation and setup steps require that you have root access on the CentOS server
The installation and setup steps also require that you have CLI access to the Solace appliance
The Solace appliance should be reachable over network from the CentOS server running Nagios
132 Assumptions
This document is based on SolOS version 711 though earier versions (such as SolOS 62) should work
without any changes
Setup steps in this guide were tested on CentOS 63 CentOS 70 (both 64 Bit Minimal install) Command
syntax and file locations would vary on other flavors of Linux
Solace Nagios Toolset Integration
6
2 Why Solace Solace technology efficiently moves information between all kinds of applications users and devices anywhere in the
world over all kinds of networks Solace makes its state-of-the-art data movement capabilities available via hardware
and software ldquomessage routersrdquo that can meet the needs of any application or deployment environment Solacersquos
unique solution offers unmatched capacity performance robustness and TCO so our customers can focus on seizing
business opportunities instead of building and maintaining complex data distribution infrastructure
Superior Performance Solacersquos hardware and software messaging middleware products can cost-effectively meet the performance needs of
any application with feature parity and interoperability that lets companies start small and scale to support higher
volume or more demanding requirements over time and purpose-built appliances that offer 50-100x higher
performance than any other technology for customers or applications that require extremely high capacity or low
latency
Robustness Solace offers high availability (HA) and disaster recovery (DR) without the need for 3rd party products and fast failover
times no other solution can match Distributing data via dedicated TCP connections ensures an orderly well-behaved
system under load and patented techniques ensure that the performance of publishers and high-speed consumers is
never impacted by slow consumers
Simple Architecture Modern enterprises run applications that demand many kinds of data movement such as persistent messaging web
streaming WAN distribution and cloud-based communications By supporting all kinds of data movement with a unified
platform that can be deployed as a small-footprint software broker or high-capacity rack-mounted appliance Solace lets
architects design an end-to-end infrastructure thatrsquos easy to build applications for integrate with existing technologies
secure and scale
Simple Operations Solacersquos solution features a shared administration framework for all kinds of data movement deployment models and
network environments so itrsquos easy for IT staff to deploy monitor manage and upgrade their Solace-based messaging
environment
Cost Savings Solace reduces expenses with high-capacity hardware flexible software and the ability to deploy the right solution for
each problem Solacersquos support for many kinds of messaging lets you replace multiple messaging products with just
one built-in HA DR WAN and Web functionality eliminate the need for third-party products
Solace Nagios Toolset Integration
7
3 Nagios Setup
31 Prerequisites A working Nagios installatin requires the following packagestools
Httpd (Apache)
RRDTool
GD Grahics library and related Perl modules
NRPE (Nagios Remote Plugin Executor)
yum install -y httpd php gcc glibc glibc-common gd gd-devel make net-snmp wget
yum install rrdtool perl-rrdtool perl-GD
yum install perl-CGI perl-Time-HiRes
1 Enabling EPEL
The steps for enabling Extra Packge Repositor (EPEL) are release specific and please check for your install
CentOS 7
wget httpdlfedoraprojectorgpubepel7x86_64eepel-release-7-5noarchrpm
rpm -ivh epel-release-7-5noarchrpm
CentOS 63
rpm -Uvh httpdlfedoraprojectorgpubepel6x86_64epel-release-6-8noarchrpm
rpm -Uvh httprpmsfamillecolletcomenterpriseremi-release-6rpm
32 Installation
yum -y install nagios nagios-plugins-all nagios-plugins-nrpe nrpe
Solace Nagios Toolset Integration
8
33 Post install setup
331 HTTP Setup
Setting up an HTTP server (such as Apache) is beyond the scope of this document Pl refer to the documentation for
your HTTP server Nagios setup should have dropped the required config file to setup URL Aliases The file is typically
ltOptioalConfigDirgtnagioscfg (Eg etchttpdconfdnagioscfg)
If Checking Webadmin below doesnrsquot work check the httpd config and logs to see if this file was processed
332 Firewall setup
Firewall setup and making changes to existing firewall configuration is again beyond the scope of this document For
eg if you are using iptables (default firewall on CentOS) then you may need to explicitly allow incoming connection on
required ports (Refer Additional Topics Section)
333 Nagios Admin User
Create password for nagiosadmin user for web access
htpasswd ndashc etcnagiospasswd nagiosadmin
systemctl restart httpd
systemctl restart nagios
Solace Nagios Toolset Integration
9
34 Verification Point the webbrowser to httpyour-server-ip[port]nagios
Login with the username nagiosadmin and the password above step (htpassword command)
Sample page is shown below
Solace Nagios Toolset Integration
10
4 Nagiosgraph Setup
41 Prerequisites NagiosGraph installation requires the following packagestools
RRDTool and related Perl modules
GD Graphics Library and related Perl modules
yum install rrdtool perl-rrdtool perl-GD
42 Installation
cd optpkg or path of your choice
wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-
152targzwget
tar xvzf nagiosgraph-152targz
cd nagiosgraph-152
installpl --check-prereq
hellip
installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-
file=varlognagiosperfdatalog --var-dir=varnagios
hellip
hellip
Continue with this configuration [y] y
hellip
Solace Nagios Toolset Integration
11
43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if
necessary at the top of the script
Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific
tools are not in place or the Config files not setup corresponding steps will be skipped
431 Restarting Services
chkconfig httpd on ampamp chkconfig nagios on
systemctl restart httpd
systemctl restart nagios
setup_nagiossolace
Checking required files
hellip
Setting up config files
hellip
Restarting services
nagios
nagios (pid 4155) is running
httpd
hellip
Solace Nagios Toolset Integration
12
44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the
Nagiosgraph install Sample page is shown below
Solace Nagios Toolset Integration
13
Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph
for one of the services (HTTP) for a day is shown below
Solace Nagios Toolset Integration
14
5 Solace Nagios Plugin Setup
51 Prerequisites Solace Nagios plugin requirs the following tools packages
Perl modules to handle XML amp Nagios
Solace pluging package
yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin
perl -MCPAN -e shell
cpangt install NagiosPlugin
cpangt install NagiosPluginDieNicely
cpangt install LWPUserAgent
cpangt install NagiosConfig
cpangt exit
52 Installation
1 Create Sample file
Create a sample router Config This file used by the setup script (below) to create commands for this config This step
can also be repeated for other routersvpns using the command mk_nagiossolacecfg script
cat cfgsolace-vmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=19216856102
HOSTPORT=8080
USERNAME=admin
PASSWORD=
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
Solace Nagios Toolset Integration
15
2 Setup Plugin
Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already
setup the related steps will be skipped
mk_nagiossolacecfg cfgsolace-vmr2cfg
Generating Nagios config for Solace appliance
Using cfgfile cfgsolace-vmr2cfg
Using template file cfgsolace-templaterouter-no_bi_bridgecfg
Generating config
etcnagiosobjectssolace-solace-vmr2cfg
Setting up config files
etcnagiosnagioscfg
Solace Nagios Toolset Integration
16
53 Post Install Setup
54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will
be added to the web page
A sample entry is shown below
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
3
834 Add Solace Appliance as Device 29 84 Verification 31
841 Basic Install 31 842 Solace host check 32 843 Solace VPN Metrics Sample 32
9 Additional Topics 34 91 Admin Tasks 34
911 Creating custom Maps 34 912 Adding additional routers VPNs 34 913 Putting Nagios on Verbose mode 34 914 Modifying iptables 35
92 Troubleshooting 36 921 Nagios not starting up 36 922 Canrsquot connect to HTTP (or other port) from outside 36 923 CGI Error in showcgi 36 924 CGI Error in Solace graphs 37 925 Canrsquot stat command file error 37
10 Setup Scripts and Config files 38 101 Install Info 38 102 Nagios 39
1021 setup_nagiossolace 39 1022 mk_nagiossolacecfg 45 1023 Solace Nagios Config (cfgsolacecfg) 47 1024 Template router (cfgsolace-templateroutercfg) 50 1025 Template and Sample router Config 53
103 Solace Nagios Plugin 53 104 NSCA 54
1041 setup_nsca 54 1042 send_ngalertpl 57 1043 nsca_init 61 1044 NSCA Config 63
105 NRPE 64 1051 Sample server config file 64 1052 Sample Client Config 65
Solace Nagios Toolset Integration
4
1 Overview Nagios (previously NetSaint) is an open-source monitoring and alerting system thatrsquos widely used to monitor systems
networks and infrastructure Objects monitored by Nagios are split into two categories hosts (physical machines) and
services (particular functionalities) Nagios does not perform any host or service checks on its own and relies on plugins
to do this This makes it a very modular and flexible solution
Extending this ldquomonitoring by pluginrdquo paradigm of Nagios Solace provides a set of plugin scripts that perform the
Appliance and VPN monitoring Following Nagios convention the plugins add host (router) and service (VPN
resources) categories
Nagiograph parses output and performance data from Nagios plugins and generates graphs and HTML pages for
reporting Nagiograph stores data in RRD (Round Robin Database) files as time series data RRDTool is an Open
source parser that integrates into Shell scripts Perl Python Ruby Tcl etc
The following picture illustrates different components and high level data flow
Solace Nagios Toolset Integration
5
11 Audience This document is primarily intended for Solace administrators and operations teams responsible for setting up and
monitoring infrastructure
12 Related Documentation These documents contain information related to the feature defined in this document
Document ID Document Title Document Source
[Solace-Portal] Solace Developer Portal httpdevsolacesystemscom
[Solace-FG] Solace Messaging Platform ndash Feature
Guide
httpdevsolacesystemscomdocsmessaging-
platform-feature-guide
[Solace-FP] Solace Messaging Platform ndash Feature
Provisioning
httpdevsolacesystemscomdocsmessaging-
platform-feature-provisioning
[Solace-CLI] Solace Message Router Command Line
Interface Reference
httpdevsolacesystemscomdocscli-reference
Table 1 - Related Documents
13 Prerequisites
131 Requirements
The installation and setup steps require that you have root access on the CentOS server
The installation and setup steps also require that you have CLI access to the Solace appliance
The Solace appliance should be reachable over network from the CentOS server running Nagios
132 Assumptions
This document is based on SolOS version 711 though earier versions (such as SolOS 62) should work
without any changes
Setup steps in this guide were tested on CentOS 63 CentOS 70 (both 64 Bit Minimal install) Command
syntax and file locations would vary on other flavors of Linux
Solace Nagios Toolset Integration
6
2 Why Solace Solace technology efficiently moves information between all kinds of applications users and devices anywhere in the
world over all kinds of networks Solace makes its state-of-the-art data movement capabilities available via hardware
and software ldquomessage routersrdquo that can meet the needs of any application or deployment environment Solacersquos
unique solution offers unmatched capacity performance robustness and TCO so our customers can focus on seizing
business opportunities instead of building and maintaining complex data distribution infrastructure
Superior Performance Solacersquos hardware and software messaging middleware products can cost-effectively meet the performance needs of
any application with feature parity and interoperability that lets companies start small and scale to support higher
volume or more demanding requirements over time and purpose-built appliances that offer 50-100x higher
performance than any other technology for customers or applications that require extremely high capacity or low
latency
Robustness Solace offers high availability (HA) and disaster recovery (DR) without the need for 3rd party products and fast failover
times no other solution can match Distributing data via dedicated TCP connections ensures an orderly well-behaved
system under load and patented techniques ensure that the performance of publishers and high-speed consumers is
never impacted by slow consumers
Simple Architecture Modern enterprises run applications that demand many kinds of data movement such as persistent messaging web
streaming WAN distribution and cloud-based communications By supporting all kinds of data movement with a unified
platform that can be deployed as a small-footprint software broker or high-capacity rack-mounted appliance Solace lets
architects design an end-to-end infrastructure thatrsquos easy to build applications for integrate with existing technologies
secure and scale
Simple Operations Solacersquos solution features a shared administration framework for all kinds of data movement deployment models and
network environments so itrsquos easy for IT staff to deploy monitor manage and upgrade their Solace-based messaging
environment
Cost Savings Solace reduces expenses with high-capacity hardware flexible software and the ability to deploy the right solution for
each problem Solacersquos support for many kinds of messaging lets you replace multiple messaging products with just
one built-in HA DR WAN and Web functionality eliminate the need for third-party products
Solace Nagios Toolset Integration
7
3 Nagios Setup
31 Prerequisites A working Nagios installatin requires the following packagestools
Httpd (Apache)
RRDTool
GD Grahics library and related Perl modules
NRPE (Nagios Remote Plugin Executor)
yum install -y httpd php gcc glibc glibc-common gd gd-devel make net-snmp wget
yum install rrdtool perl-rrdtool perl-GD
yum install perl-CGI perl-Time-HiRes
1 Enabling EPEL
The steps for enabling Extra Packge Repositor (EPEL) are release specific and please check for your install
CentOS 7
wget httpdlfedoraprojectorgpubepel7x86_64eepel-release-7-5noarchrpm
rpm -ivh epel-release-7-5noarchrpm
CentOS 63
rpm -Uvh httpdlfedoraprojectorgpubepel6x86_64epel-release-6-8noarchrpm
rpm -Uvh httprpmsfamillecolletcomenterpriseremi-release-6rpm
32 Installation
yum -y install nagios nagios-plugins-all nagios-plugins-nrpe nrpe
Solace Nagios Toolset Integration
8
33 Post install setup
331 HTTP Setup
Setting up an HTTP server (such as Apache) is beyond the scope of this document Pl refer to the documentation for
your HTTP server Nagios setup should have dropped the required config file to setup URL Aliases The file is typically
ltOptioalConfigDirgtnagioscfg (Eg etchttpdconfdnagioscfg)
If Checking Webadmin below doesnrsquot work check the httpd config and logs to see if this file was processed
332 Firewall setup
Firewall setup and making changes to existing firewall configuration is again beyond the scope of this document For
eg if you are using iptables (default firewall on CentOS) then you may need to explicitly allow incoming connection on
required ports (Refer Additional Topics Section)
333 Nagios Admin User
Create password for nagiosadmin user for web access
htpasswd ndashc etcnagiospasswd nagiosadmin
systemctl restart httpd
systemctl restart nagios
Solace Nagios Toolset Integration
9
34 Verification Point the webbrowser to httpyour-server-ip[port]nagios
Login with the username nagiosadmin and the password above step (htpassword command)
Sample page is shown below
Solace Nagios Toolset Integration
10
4 Nagiosgraph Setup
41 Prerequisites NagiosGraph installation requires the following packagestools
RRDTool and related Perl modules
GD Graphics Library and related Perl modules
yum install rrdtool perl-rrdtool perl-GD
42 Installation
cd optpkg or path of your choice
wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-
152targzwget
tar xvzf nagiosgraph-152targz
cd nagiosgraph-152
installpl --check-prereq
hellip
installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-
file=varlognagiosperfdatalog --var-dir=varnagios
hellip
hellip
Continue with this configuration [y] y
hellip
Solace Nagios Toolset Integration
11
43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if
necessary at the top of the script
Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific
tools are not in place or the Config files not setup corresponding steps will be skipped
431 Restarting Services
chkconfig httpd on ampamp chkconfig nagios on
systemctl restart httpd
systemctl restart nagios
setup_nagiossolace
Checking required files
hellip
Setting up config files
hellip
Restarting services
nagios
nagios (pid 4155) is running
httpd
hellip
Solace Nagios Toolset Integration
12
44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the
Nagiosgraph install Sample page is shown below
Solace Nagios Toolset Integration
13
Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph
for one of the services (HTTP) for a day is shown below
Solace Nagios Toolset Integration
14
5 Solace Nagios Plugin Setup
51 Prerequisites Solace Nagios plugin requirs the following tools packages
Perl modules to handle XML amp Nagios
Solace pluging package
yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin
perl -MCPAN -e shell
cpangt install NagiosPlugin
cpangt install NagiosPluginDieNicely
cpangt install LWPUserAgent
cpangt install NagiosConfig
cpangt exit
52 Installation
1 Create Sample file
Create a sample router Config This file used by the setup script (below) to create commands for this config This step
can also be repeated for other routersvpns using the command mk_nagiossolacecfg script
cat cfgsolace-vmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=19216856102
HOSTPORT=8080
USERNAME=admin
PASSWORD=
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
Solace Nagios Toolset Integration
15
2 Setup Plugin
Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already
setup the related steps will be skipped
mk_nagiossolacecfg cfgsolace-vmr2cfg
Generating Nagios config for Solace appliance
Using cfgfile cfgsolace-vmr2cfg
Using template file cfgsolace-templaterouter-no_bi_bridgecfg
Generating config
etcnagiosobjectssolace-solace-vmr2cfg
Setting up config files
etcnagiosnagioscfg
Solace Nagios Toolset Integration
16
53 Post Install Setup
54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will
be added to the web page
A sample entry is shown below
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
4
1 Overview Nagios (previously NetSaint) is an open-source monitoring and alerting system thatrsquos widely used to monitor systems
networks and infrastructure Objects monitored by Nagios are split into two categories hosts (physical machines) and
services (particular functionalities) Nagios does not perform any host or service checks on its own and relies on plugins
to do this This makes it a very modular and flexible solution
Extending this ldquomonitoring by pluginrdquo paradigm of Nagios Solace provides a set of plugin scripts that perform the
Appliance and VPN monitoring Following Nagios convention the plugins add host (router) and service (VPN
resources) categories
Nagiograph parses output and performance data from Nagios plugins and generates graphs and HTML pages for
reporting Nagiograph stores data in RRD (Round Robin Database) files as time series data RRDTool is an Open
source parser that integrates into Shell scripts Perl Python Ruby Tcl etc
The following picture illustrates different components and high level data flow
Solace Nagios Toolset Integration
5
11 Audience This document is primarily intended for Solace administrators and operations teams responsible for setting up and
monitoring infrastructure
12 Related Documentation These documents contain information related to the feature defined in this document
Document ID Document Title Document Source
[Solace-Portal] Solace Developer Portal httpdevsolacesystemscom
[Solace-FG] Solace Messaging Platform ndash Feature
Guide
httpdevsolacesystemscomdocsmessaging-
platform-feature-guide
[Solace-FP] Solace Messaging Platform ndash Feature
Provisioning
httpdevsolacesystemscomdocsmessaging-
platform-feature-provisioning
[Solace-CLI] Solace Message Router Command Line
Interface Reference
httpdevsolacesystemscomdocscli-reference
Table 1 - Related Documents
13 Prerequisites
131 Requirements
The installation and setup steps require that you have root access on the CentOS server
The installation and setup steps also require that you have CLI access to the Solace appliance
The Solace appliance should be reachable over network from the CentOS server running Nagios
132 Assumptions
This document is based on SolOS version 711 though earier versions (such as SolOS 62) should work
without any changes
Setup steps in this guide were tested on CentOS 63 CentOS 70 (both 64 Bit Minimal install) Command
syntax and file locations would vary on other flavors of Linux
Solace Nagios Toolset Integration
6
2 Why Solace Solace technology efficiently moves information between all kinds of applications users and devices anywhere in the
world over all kinds of networks Solace makes its state-of-the-art data movement capabilities available via hardware
and software ldquomessage routersrdquo that can meet the needs of any application or deployment environment Solacersquos
unique solution offers unmatched capacity performance robustness and TCO so our customers can focus on seizing
business opportunities instead of building and maintaining complex data distribution infrastructure
Superior Performance Solacersquos hardware and software messaging middleware products can cost-effectively meet the performance needs of
any application with feature parity and interoperability that lets companies start small and scale to support higher
volume or more demanding requirements over time and purpose-built appliances that offer 50-100x higher
performance than any other technology for customers or applications that require extremely high capacity or low
latency
Robustness Solace offers high availability (HA) and disaster recovery (DR) without the need for 3rd party products and fast failover
times no other solution can match Distributing data via dedicated TCP connections ensures an orderly well-behaved
system under load and patented techniques ensure that the performance of publishers and high-speed consumers is
never impacted by slow consumers
Simple Architecture Modern enterprises run applications that demand many kinds of data movement such as persistent messaging web
streaming WAN distribution and cloud-based communications By supporting all kinds of data movement with a unified
platform that can be deployed as a small-footprint software broker or high-capacity rack-mounted appliance Solace lets
architects design an end-to-end infrastructure thatrsquos easy to build applications for integrate with existing technologies
secure and scale
Simple Operations Solacersquos solution features a shared administration framework for all kinds of data movement deployment models and
network environments so itrsquos easy for IT staff to deploy monitor manage and upgrade their Solace-based messaging
environment
Cost Savings Solace reduces expenses with high-capacity hardware flexible software and the ability to deploy the right solution for
each problem Solacersquos support for many kinds of messaging lets you replace multiple messaging products with just
one built-in HA DR WAN and Web functionality eliminate the need for third-party products
Solace Nagios Toolset Integration
7
3 Nagios Setup
31 Prerequisites A working Nagios installatin requires the following packagestools
Httpd (Apache)
RRDTool
GD Grahics library and related Perl modules
NRPE (Nagios Remote Plugin Executor)
yum install -y httpd php gcc glibc glibc-common gd gd-devel make net-snmp wget
yum install rrdtool perl-rrdtool perl-GD
yum install perl-CGI perl-Time-HiRes
1 Enabling EPEL
The steps for enabling Extra Packge Repositor (EPEL) are release specific and please check for your install
CentOS 7
wget httpdlfedoraprojectorgpubepel7x86_64eepel-release-7-5noarchrpm
rpm -ivh epel-release-7-5noarchrpm
CentOS 63
rpm -Uvh httpdlfedoraprojectorgpubepel6x86_64epel-release-6-8noarchrpm
rpm -Uvh httprpmsfamillecolletcomenterpriseremi-release-6rpm
32 Installation
yum -y install nagios nagios-plugins-all nagios-plugins-nrpe nrpe
Solace Nagios Toolset Integration
8
33 Post install setup
331 HTTP Setup
Setting up an HTTP server (such as Apache) is beyond the scope of this document Pl refer to the documentation for
your HTTP server Nagios setup should have dropped the required config file to setup URL Aliases The file is typically
ltOptioalConfigDirgtnagioscfg (Eg etchttpdconfdnagioscfg)
If Checking Webadmin below doesnrsquot work check the httpd config and logs to see if this file was processed
332 Firewall setup
Firewall setup and making changes to existing firewall configuration is again beyond the scope of this document For
eg if you are using iptables (default firewall on CentOS) then you may need to explicitly allow incoming connection on
required ports (Refer Additional Topics Section)
333 Nagios Admin User
Create password for nagiosadmin user for web access
htpasswd ndashc etcnagiospasswd nagiosadmin
systemctl restart httpd
systemctl restart nagios
Solace Nagios Toolset Integration
9
34 Verification Point the webbrowser to httpyour-server-ip[port]nagios
Login with the username nagiosadmin and the password above step (htpassword command)
Sample page is shown below
Solace Nagios Toolset Integration
10
4 Nagiosgraph Setup
41 Prerequisites NagiosGraph installation requires the following packagestools
RRDTool and related Perl modules
GD Graphics Library and related Perl modules
yum install rrdtool perl-rrdtool perl-GD
42 Installation
cd optpkg or path of your choice
wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-
152targzwget
tar xvzf nagiosgraph-152targz
cd nagiosgraph-152
installpl --check-prereq
hellip
installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-
file=varlognagiosperfdatalog --var-dir=varnagios
hellip
hellip
Continue with this configuration [y] y
hellip
Solace Nagios Toolset Integration
11
43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if
necessary at the top of the script
Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific
tools are not in place or the Config files not setup corresponding steps will be skipped
431 Restarting Services
chkconfig httpd on ampamp chkconfig nagios on
systemctl restart httpd
systemctl restart nagios
setup_nagiossolace
Checking required files
hellip
Setting up config files
hellip
Restarting services
nagios
nagios (pid 4155) is running
httpd
hellip
Solace Nagios Toolset Integration
12
44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the
Nagiosgraph install Sample page is shown below
Solace Nagios Toolset Integration
13
Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph
for one of the services (HTTP) for a day is shown below
Solace Nagios Toolset Integration
14
5 Solace Nagios Plugin Setup
51 Prerequisites Solace Nagios plugin requirs the following tools packages
Perl modules to handle XML amp Nagios
Solace pluging package
yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin
perl -MCPAN -e shell
cpangt install NagiosPlugin
cpangt install NagiosPluginDieNicely
cpangt install LWPUserAgent
cpangt install NagiosConfig
cpangt exit
52 Installation
1 Create Sample file
Create a sample router Config This file used by the setup script (below) to create commands for this config This step
can also be repeated for other routersvpns using the command mk_nagiossolacecfg script
cat cfgsolace-vmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=19216856102
HOSTPORT=8080
USERNAME=admin
PASSWORD=
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
Solace Nagios Toolset Integration
15
2 Setup Plugin
Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already
setup the related steps will be skipped
mk_nagiossolacecfg cfgsolace-vmr2cfg
Generating Nagios config for Solace appliance
Using cfgfile cfgsolace-vmr2cfg
Using template file cfgsolace-templaterouter-no_bi_bridgecfg
Generating config
etcnagiosobjectssolace-solace-vmr2cfg
Setting up config files
etcnagiosnagioscfg
Solace Nagios Toolset Integration
16
53 Post Install Setup
54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will
be added to the web page
A sample entry is shown below
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
5
11 Audience This document is primarily intended for Solace administrators and operations teams responsible for setting up and
monitoring infrastructure
12 Related Documentation These documents contain information related to the feature defined in this document
Document ID Document Title Document Source
[Solace-Portal] Solace Developer Portal httpdevsolacesystemscom
[Solace-FG] Solace Messaging Platform ndash Feature
Guide
httpdevsolacesystemscomdocsmessaging-
platform-feature-guide
[Solace-FP] Solace Messaging Platform ndash Feature
Provisioning
httpdevsolacesystemscomdocsmessaging-
platform-feature-provisioning
[Solace-CLI] Solace Message Router Command Line
Interface Reference
httpdevsolacesystemscomdocscli-reference
Table 1 - Related Documents
13 Prerequisites
131 Requirements
The installation and setup steps require that you have root access on the CentOS server
The installation and setup steps also require that you have CLI access to the Solace appliance
The Solace appliance should be reachable over network from the CentOS server running Nagios
132 Assumptions
This document is based on SolOS version 711 though earier versions (such as SolOS 62) should work
without any changes
Setup steps in this guide were tested on CentOS 63 CentOS 70 (both 64 Bit Minimal install) Command
syntax and file locations would vary on other flavors of Linux
Solace Nagios Toolset Integration
6
2 Why Solace Solace technology efficiently moves information between all kinds of applications users and devices anywhere in the
world over all kinds of networks Solace makes its state-of-the-art data movement capabilities available via hardware
and software ldquomessage routersrdquo that can meet the needs of any application or deployment environment Solacersquos
unique solution offers unmatched capacity performance robustness and TCO so our customers can focus on seizing
business opportunities instead of building and maintaining complex data distribution infrastructure
Superior Performance Solacersquos hardware and software messaging middleware products can cost-effectively meet the performance needs of
any application with feature parity and interoperability that lets companies start small and scale to support higher
volume or more demanding requirements over time and purpose-built appliances that offer 50-100x higher
performance than any other technology for customers or applications that require extremely high capacity or low
latency
Robustness Solace offers high availability (HA) and disaster recovery (DR) without the need for 3rd party products and fast failover
times no other solution can match Distributing data via dedicated TCP connections ensures an orderly well-behaved
system under load and patented techniques ensure that the performance of publishers and high-speed consumers is
never impacted by slow consumers
Simple Architecture Modern enterprises run applications that demand many kinds of data movement such as persistent messaging web
streaming WAN distribution and cloud-based communications By supporting all kinds of data movement with a unified
platform that can be deployed as a small-footprint software broker or high-capacity rack-mounted appliance Solace lets
architects design an end-to-end infrastructure thatrsquos easy to build applications for integrate with existing technologies
secure and scale
Simple Operations Solacersquos solution features a shared administration framework for all kinds of data movement deployment models and
network environments so itrsquos easy for IT staff to deploy monitor manage and upgrade their Solace-based messaging
environment
Cost Savings Solace reduces expenses with high-capacity hardware flexible software and the ability to deploy the right solution for
each problem Solacersquos support for many kinds of messaging lets you replace multiple messaging products with just
one built-in HA DR WAN and Web functionality eliminate the need for third-party products
Solace Nagios Toolset Integration
7
3 Nagios Setup
31 Prerequisites A working Nagios installatin requires the following packagestools
Httpd (Apache)
RRDTool
GD Grahics library and related Perl modules
NRPE (Nagios Remote Plugin Executor)
yum install -y httpd php gcc glibc glibc-common gd gd-devel make net-snmp wget
yum install rrdtool perl-rrdtool perl-GD
yum install perl-CGI perl-Time-HiRes
1 Enabling EPEL
The steps for enabling Extra Packge Repositor (EPEL) are release specific and please check for your install
CentOS 7
wget httpdlfedoraprojectorgpubepel7x86_64eepel-release-7-5noarchrpm
rpm -ivh epel-release-7-5noarchrpm
CentOS 63
rpm -Uvh httpdlfedoraprojectorgpubepel6x86_64epel-release-6-8noarchrpm
rpm -Uvh httprpmsfamillecolletcomenterpriseremi-release-6rpm
32 Installation
yum -y install nagios nagios-plugins-all nagios-plugins-nrpe nrpe
Solace Nagios Toolset Integration
8
33 Post install setup
331 HTTP Setup
Setting up an HTTP server (such as Apache) is beyond the scope of this document Pl refer to the documentation for
your HTTP server Nagios setup should have dropped the required config file to setup URL Aliases The file is typically
ltOptioalConfigDirgtnagioscfg (Eg etchttpdconfdnagioscfg)
If Checking Webadmin below doesnrsquot work check the httpd config and logs to see if this file was processed
332 Firewall setup
Firewall setup and making changes to existing firewall configuration is again beyond the scope of this document For
eg if you are using iptables (default firewall on CentOS) then you may need to explicitly allow incoming connection on
required ports (Refer Additional Topics Section)
333 Nagios Admin User
Create password for nagiosadmin user for web access
htpasswd ndashc etcnagiospasswd nagiosadmin
systemctl restart httpd
systemctl restart nagios
Solace Nagios Toolset Integration
9
34 Verification Point the webbrowser to httpyour-server-ip[port]nagios
Login with the username nagiosadmin and the password above step (htpassword command)
Sample page is shown below
Solace Nagios Toolset Integration
10
4 Nagiosgraph Setup
41 Prerequisites NagiosGraph installation requires the following packagestools
RRDTool and related Perl modules
GD Graphics Library and related Perl modules
yum install rrdtool perl-rrdtool perl-GD
42 Installation
cd optpkg or path of your choice
wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-
152targzwget
tar xvzf nagiosgraph-152targz
cd nagiosgraph-152
installpl --check-prereq
hellip
installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-
file=varlognagiosperfdatalog --var-dir=varnagios
hellip
hellip
Continue with this configuration [y] y
hellip
Solace Nagios Toolset Integration
11
43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if
necessary at the top of the script
Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific
tools are not in place or the Config files not setup corresponding steps will be skipped
431 Restarting Services
chkconfig httpd on ampamp chkconfig nagios on
systemctl restart httpd
systemctl restart nagios
setup_nagiossolace
Checking required files
hellip
Setting up config files
hellip
Restarting services
nagios
nagios (pid 4155) is running
httpd
hellip
Solace Nagios Toolset Integration
12
44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the
Nagiosgraph install Sample page is shown below
Solace Nagios Toolset Integration
13
Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph
for one of the services (HTTP) for a day is shown below
Solace Nagios Toolset Integration
14
5 Solace Nagios Plugin Setup
51 Prerequisites Solace Nagios plugin requirs the following tools packages
Perl modules to handle XML amp Nagios
Solace pluging package
yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin
perl -MCPAN -e shell
cpangt install NagiosPlugin
cpangt install NagiosPluginDieNicely
cpangt install LWPUserAgent
cpangt install NagiosConfig
cpangt exit
52 Installation
1 Create Sample file
Create a sample router Config This file used by the setup script (below) to create commands for this config This step
can also be repeated for other routersvpns using the command mk_nagiossolacecfg script
cat cfgsolace-vmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=19216856102
HOSTPORT=8080
USERNAME=admin
PASSWORD=
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
Solace Nagios Toolset Integration
15
2 Setup Plugin
Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already
setup the related steps will be skipped
mk_nagiossolacecfg cfgsolace-vmr2cfg
Generating Nagios config for Solace appliance
Using cfgfile cfgsolace-vmr2cfg
Using template file cfgsolace-templaterouter-no_bi_bridgecfg
Generating config
etcnagiosobjectssolace-solace-vmr2cfg
Setting up config files
etcnagiosnagioscfg
Solace Nagios Toolset Integration
16
53 Post Install Setup
54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will
be added to the web page
A sample entry is shown below
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
6
2 Why Solace Solace technology efficiently moves information between all kinds of applications users and devices anywhere in the
world over all kinds of networks Solace makes its state-of-the-art data movement capabilities available via hardware
and software ldquomessage routersrdquo that can meet the needs of any application or deployment environment Solacersquos
unique solution offers unmatched capacity performance robustness and TCO so our customers can focus on seizing
business opportunities instead of building and maintaining complex data distribution infrastructure
Superior Performance Solacersquos hardware and software messaging middleware products can cost-effectively meet the performance needs of
any application with feature parity and interoperability that lets companies start small and scale to support higher
volume or more demanding requirements over time and purpose-built appliances that offer 50-100x higher
performance than any other technology for customers or applications that require extremely high capacity or low
latency
Robustness Solace offers high availability (HA) and disaster recovery (DR) without the need for 3rd party products and fast failover
times no other solution can match Distributing data via dedicated TCP connections ensures an orderly well-behaved
system under load and patented techniques ensure that the performance of publishers and high-speed consumers is
never impacted by slow consumers
Simple Architecture Modern enterprises run applications that demand many kinds of data movement such as persistent messaging web
streaming WAN distribution and cloud-based communications By supporting all kinds of data movement with a unified
platform that can be deployed as a small-footprint software broker or high-capacity rack-mounted appliance Solace lets
architects design an end-to-end infrastructure thatrsquos easy to build applications for integrate with existing technologies
secure and scale
Simple Operations Solacersquos solution features a shared administration framework for all kinds of data movement deployment models and
network environments so itrsquos easy for IT staff to deploy monitor manage and upgrade their Solace-based messaging
environment
Cost Savings Solace reduces expenses with high-capacity hardware flexible software and the ability to deploy the right solution for
each problem Solacersquos support for many kinds of messaging lets you replace multiple messaging products with just
one built-in HA DR WAN and Web functionality eliminate the need for third-party products
Solace Nagios Toolset Integration
7
3 Nagios Setup
31 Prerequisites A working Nagios installatin requires the following packagestools
Httpd (Apache)
RRDTool
GD Grahics library and related Perl modules
NRPE (Nagios Remote Plugin Executor)
yum install -y httpd php gcc glibc glibc-common gd gd-devel make net-snmp wget
yum install rrdtool perl-rrdtool perl-GD
yum install perl-CGI perl-Time-HiRes
1 Enabling EPEL
The steps for enabling Extra Packge Repositor (EPEL) are release specific and please check for your install
CentOS 7
wget httpdlfedoraprojectorgpubepel7x86_64eepel-release-7-5noarchrpm
rpm -ivh epel-release-7-5noarchrpm
CentOS 63
rpm -Uvh httpdlfedoraprojectorgpubepel6x86_64epel-release-6-8noarchrpm
rpm -Uvh httprpmsfamillecolletcomenterpriseremi-release-6rpm
32 Installation
yum -y install nagios nagios-plugins-all nagios-plugins-nrpe nrpe
Solace Nagios Toolset Integration
8
33 Post install setup
331 HTTP Setup
Setting up an HTTP server (such as Apache) is beyond the scope of this document Pl refer to the documentation for
your HTTP server Nagios setup should have dropped the required config file to setup URL Aliases The file is typically
ltOptioalConfigDirgtnagioscfg (Eg etchttpdconfdnagioscfg)
If Checking Webadmin below doesnrsquot work check the httpd config and logs to see if this file was processed
332 Firewall setup
Firewall setup and making changes to existing firewall configuration is again beyond the scope of this document For
eg if you are using iptables (default firewall on CentOS) then you may need to explicitly allow incoming connection on
required ports (Refer Additional Topics Section)
333 Nagios Admin User
Create password for nagiosadmin user for web access
htpasswd ndashc etcnagiospasswd nagiosadmin
systemctl restart httpd
systemctl restart nagios
Solace Nagios Toolset Integration
9
34 Verification Point the webbrowser to httpyour-server-ip[port]nagios
Login with the username nagiosadmin and the password above step (htpassword command)
Sample page is shown below
Solace Nagios Toolset Integration
10
4 Nagiosgraph Setup
41 Prerequisites NagiosGraph installation requires the following packagestools
RRDTool and related Perl modules
GD Graphics Library and related Perl modules
yum install rrdtool perl-rrdtool perl-GD
42 Installation
cd optpkg or path of your choice
wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-
152targzwget
tar xvzf nagiosgraph-152targz
cd nagiosgraph-152
installpl --check-prereq
hellip
installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-
file=varlognagiosperfdatalog --var-dir=varnagios
hellip
hellip
Continue with this configuration [y] y
hellip
Solace Nagios Toolset Integration
11
43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if
necessary at the top of the script
Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific
tools are not in place or the Config files not setup corresponding steps will be skipped
431 Restarting Services
chkconfig httpd on ampamp chkconfig nagios on
systemctl restart httpd
systemctl restart nagios
setup_nagiossolace
Checking required files
hellip
Setting up config files
hellip
Restarting services
nagios
nagios (pid 4155) is running
httpd
hellip
Solace Nagios Toolset Integration
12
44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the
Nagiosgraph install Sample page is shown below
Solace Nagios Toolset Integration
13
Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph
for one of the services (HTTP) for a day is shown below
Solace Nagios Toolset Integration
14
5 Solace Nagios Plugin Setup
51 Prerequisites Solace Nagios plugin requirs the following tools packages
Perl modules to handle XML amp Nagios
Solace pluging package
yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin
perl -MCPAN -e shell
cpangt install NagiosPlugin
cpangt install NagiosPluginDieNicely
cpangt install LWPUserAgent
cpangt install NagiosConfig
cpangt exit
52 Installation
1 Create Sample file
Create a sample router Config This file used by the setup script (below) to create commands for this config This step
can also be repeated for other routersvpns using the command mk_nagiossolacecfg script
cat cfgsolace-vmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=19216856102
HOSTPORT=8080
USERNAME=admin
PASSWORD=
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
Solace Nagios Toolset Integration
15
2 Setup Plugin
Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already
setup the related steps will be skipped
mk_nagiossolacecfg cfgsolace-vmr2cfg
Generating Nagios config for Solace appliance
Using cfgfile cfgsolace-vmr2cfg
Using template file cfgsolace-templaterouter-no_bi_bridgecfg
Generating config
etcnagiosobjectssolace-solace-vmr2cfg
Setting up config files
etcnagiosnagioscfg
Solace Nagios Toolset Integration
16
53 Post Install Setup
54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will
be added to the web page
A sample entry is shown below
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
7
3 Nagios Setup
31 Prerequisites A working Nagios installatin requires the following packagestools
Httpd (Apache)
RRDTool
GD Grahics library and related Perl modules
NRPE (Nagios Remote Plugin Executor)
yum install -y httpd php gcc glibc glibc-common gd gd-devel make net-snmp wget
yum install rrdtool perl-rrdtool perl-GD
yum install perl-CGI perl-Time-HiRes
1 Enabling EPEL
The steps for enabling Extra Packge Repositor (EPEL) are release specific and please check for your install
CentOS 7
wget httpdlfedoraprojectorgpubepel7x86_64eepel-release-7-5noarchrpm
rpm -ivh epel-release-7-5noarchrpm
CentOS 63
rpm -Uvh httpdlfedoraprojectorgpubepel6x86_64epel-release-6-8noarchrpm
rpm -Uvh httprpmsfamillecolletcomenterpriseremi-release-6rpm
32 Installation
yum -y install nagios nagios-plugins-all nagios-plugins-nrpe nrpe
Solace Nagios Toolset Integration
8
33 Post install setup
331 HTTP Setup
Setting up an HTTP server (such as Apache) is beyond the scope of this document Pl refer to the documentation for
your HTTP server Nagios setup should have dropped the required config file to setup URL Aliases The file is typically
ltOptioalConfigDirgtnagioscfg (Eg etchttpdconfdnagioscfg)
If Checking Webadmin below doesnrsquot work check the httpd config and logs to see if this file was processed
332 Firewall setup
Firewall setup and making changes to existing firewall configuration is again beyond the scope of this document For
eg if you are using iptables (default firewall on CentOS) then you may need to explicitly allow incoming connection on
required ports (Refer Additional Topics Section)
333 Nagios Admin User
Create password for nagiosadmin user for web access
htpasswd ndashc etcnagiospasswd nagiosadmin
systemctl restart httpd
systemctl restart nagios
Solace Nagios Toolset Integration
9
34 Verification Point the webbrowser to httpyour-server-ip[port]nagios
Login with the username nagiosadmin and the password above step (htpassword command)
Sample page is shown below
Solace Nagios Toolset Integration
10
4 Nagiosgraph Setup
41 Prerequisites NagiosGraph installation requires the following packagestools
RRDTool and related Perl modules
GD Graphics Library and related Perl modules
yum install rrdtool perl-rrdtool perl-GD
42 Installation
cd optpkg or path of your choice
wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-
152targzwget
tar xvzf nagiosgraph-152targz
cd nagiosgraph-152
installpl --check-prereq
hellip
installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-
file=varlognagiosperfdatalog --var-dir=varnagios
hellip
hellip
Continue with this configuration [y] y
hellip
Solace Nagios Toolset Integration
11
43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if
necessary at the top of the script
Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific
tools are not in place or the Config files not setup corresponding steps will be skipped
431 Restarting Services
chkconfig httpd on ampamp chkconfig nagios on
systemctl restart httpd
systemctl restart nagios
setup_nagiossolace
Checking required files
hellip
Setting up config files
hellip
Restarting services
nagios
nagios (pid 4155) is running
httpd
hellip
Solace Nagios Toolset Integration
12
44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the
Nagiosgraph install Sample page is shown below
Solace Nagios Toolset Integration
13
Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph
for one of the services (HTTP) for a day is shown below
Solace Nagios Toolset Integration
14
5 Solace Nagios Plugin Setup
51 Prerequisites Solace Nagios plugin requirs the following tools packages
Perl modules to handle XML amp Nagios
Solace pluging package
yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin
perl -MCPAN -e shell
cpangt install NagiosPlugin
cpangt install NagiosPluginDieNicely
cpangt install LWPUserAgent
cpangt install NagiosConfig
cpangt exit
52 Installation
1 Create Sample file
Create a sample router Config This file used by the setup script (below) to create commands for this config This step
can also be repeated for other routersvpns using the command mk_nagiossolacecfg script
cat cfgsolace-vmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=19216856102
HOSTPORT=8080
USERNAME=admin
PASSWORD=
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
Solace Nagios Toolset Integration
15
2 Setup Plugin
Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already
setup the related steps will be skipped
mk_nagiossolacecfg cfgsolace-vmr2cfg
Generating Nagios config for Solace appliance
Using cfgfile cfgsolace-vmr2cfg
Using template file cfgsolace-templaterouter-no_bi_bridgecfg
Generating config
etcnagiosobjectssolace-solace-vmr2cfg
Setting up config files
etcnagiosnagioscfg
Solace Nagios Toolset Integration
16
53 Post Install Setup
54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will
be added to the web page
A sample entry is shown below
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
8
33 Post install setup
331 HTTP Setup
Setting up an HTTP server (such as Apache) is beyond the scope of this document Pl refer to the documentation for
your HTTP server Nagios setup should have dropped the required config file to setup URL Aliases The file is typically
ltOptioalConfigDirgtnagioscfg (Eg etchttpdconfdnagioscfg)
If Checking Webadmin below doesnrsquot work check the httpd config and logs to see if this file was processed
332 Firewall setup
Firewall setup and making changes to existing firewall configuration is again beyond the scope of this document For
eg if you are using iptables (default firewall on CentOS) then you may need to explicitly allow incoming connection on
required ports (Refer Additional Topics Section)
333 Nagios Admin User
Create password for nagiosadmin user for web access
htpasswd ndashc etcnagiospasswd nagiosadmin
systemctl restart httpd
systemctl restart nagios
Solace Nagios Toolset Integration
9
34 Verification Point the webbrowser to httpyour-server-ip[port]nagios
Login with the username nagiosadmin and the password above step (htpassword command)
Sample page is shown below
Solace Nagios Toolset Integration
10
4 Nagiosgraph Setup
41 Prerequisites NagiosGraph installation requires the following packagestools
RRDTool and related Perl modules
GD Graphics Library and related Perl modules
yum install rrdtool perl-rrdtool perl-GD
42 Installation
cd optpkg or path of your choice
wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-
152targzwget
tar xvzf nagiosgraph-152targz
cd nagiosgraph-152
installpl --check-prereq
hellip
installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-
file=varlognagiosperfdatalog --var-dir=varnagios
hellip
hellip
Continue with this configuration [y] y
hellip
Solace Nagios Toolset Integration
11
43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if
necessary at the top of the script
Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific
tools are not in place or the Config files not setup corresponding steps will be skipped
431 Restarting Services
chkconfig httpd on ampamp chkconfig nagios on
systemctl restart httpd
systemctl restart nagios
setup_nagiossolace
Checking required files
hellip
Setting up config files
hellip
Restarting services
nagios
nagios (pid 4155) is running
httpd
hellip
Solace Nagios Toolset Integration
12
44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the
Nagiosgraph install Sample page is shown below
Solace Nagios Toolset Integration
13
Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph
for one of the services (HTTP) for a day is shown below
Solace Nagios Toolset Integration
14
5 Solace Nagios Plugin Setup
51 Prerequisites Solace Nagios plugin requirs the following tools packages
Perl modules to handle XML amp Nagios
Solace pluging package
yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin
perl -MCPAN -e shell
cpangt install NagiosPlugin
cpangt install NagiosPluginDieNicely
cpangt install LWPUserAgent
cpangt install NagiosConfig
cpangt exit
52 Installation
1 Create Sample file
Create a sample router Config This file used by the setup script (below) to create commands for this config This step
can also be repeated for other routersvpns using the command mk_nagiossolacecfg script
cat cfgsolace-vmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=19216856102
HOSTPORT=8080
USERNAME=admin
PASSWORD=
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
Solace Nagios Toolset Integration
15
2 Setup Plugin
Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already
setup the related steps will be skipped
mk_nagiossolacecfg cfgsolace-vmr2cfg
Generating Nagios config for Solace appliance
Using cfgfile cfgsolace-vmr2cfg
Using template file cfgsolace-templaterouter-no_bi_bridgecfg
Generating config
etcnagiosobjectssolace-solace-vmr2cfg
Setting up config files
etcnagiosnagioscfg
Solace Nagios Toolset Integration
16
53 Post Install Setup
54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will
be added to the web page
A sample entry is shown below
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
9
34 Verification Point the webbrowser to httpyour-server-ip[port]nagios
Login with the username nagiosadmin and the password above step (htpassword command)
Sample page is shown below
Solace Nagios Toolset Integration
10
4 Nagiosgraph Setup
41 Prerequisites NagiosGraph installation requires the following packagestools
RRDTool and related Perl modules
GD Graphics Library and related Perl modules
yum install rrdtool perl-rrdtool perl-GD
42 Installation
cd optpkg or path of your choice
wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-
152targzwget
tar xvzf nagiosgraph-152targz
cd nagiosgraph-152
installpl --check-prereq
hellip
installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-
file=varlognagiosperfdatalog --var-dir=varnagios
hellip
hellip
Continue with this configuration [y] y
hellip
Solace Nagios Toolset Integration
11
43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if
necessary at the top of the script
Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific
tools are not in place or the Config files not setup corresponding steps will be skipped
431 Restarting Services
chkconfig httpd on ampamp chkconfig nagios on
systemctl restart httpd
systemctl restart nagios
setup_nagiossolace
Checking required files
hellip
Setting up config files
hellip
Restarting services
nagios
nagios (pid 4155) is running
httpd
hellip
Solace Nagios Toolset Integration
12
44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the
Nagiosgraph install Sample page is shown below
Solace Nagios Toolset Integration
13
Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph
for one of the services (HTTP) for a day is shown below
Solace Nagios Toolset Integration
14
5 Solace Nagios Plugin Setup
51 Prerequisites Solace Nagios plugin requirs the following tools packages
Perl modules to handle XML amp Nagios
Solace pluging package
yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin
perl -MCPAN -e shell
cpangt install NagiosPlugin
cpangt install NagiosPluginDieNicely
cpangt install LWPUserAgent
cpangt install NagiosConfig
cpangt exit
52 Installation
1 Create Sample file
Create a sample router Config This file used by the setup script (below) to create commands for this config This step
can also be repeated for other routersvpns using the command mk_nagiossolacecfg script
cat cfgsolace-vmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=19216856102
HOSTPORT=8080
USERNAME=admin
PASSWORD=
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
Solace Nagios Toolset Integration
15
2 Setup Plugin
Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already
setup the related steps will be skipped
mk_nagiossolacecfg cfgsolace-vmr2cfg
Generating Nagios config for Solace appliance
Using cfgfile cfgsolace-vmr2cfg
Using template file cfgsolace-templaterouter-no_bi_bridgecfg
Generating config
etcnagiosobjectssolace-solace-vmr2cfg
Setting up config files
etcnagiosnagioscfg
Solace Nagios Toolset Integration
16
53 Post Install Setup
54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will
be added to the web page
A sample entry is shown below
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
10
4 Nagiosgraph Setup
41 Prerequisites NagiosGraph installation requires the following packagestools
RRDTool and related Perl modules
GD Graphics Library and related Perl modules
yum install rrdtool perl-rrdtool perl-GD
42 Installation
cd optpkg or path of your choice
wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-
152targzwget
tar xvzf nagiosgraph-152targz
cd nagiosgraph-152
installpl --check-prereq
hellip
installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-
file=varlognagiosperfdatalog --var-dir=varnagios
hellip
hellip
Continue with this configuration [y] y
hellip
Solace Nagios Toolset Integration
11
43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if
necessary at the top of the script
Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific
tools are not in place or the Config files not setup corresponding steps will be skipped
431 Restarting Services
chkconfig httpd on ampamp chkconfig nagios on
systemctl restart httpd
systemctl restart nagios
setup_nagiossolace
Checking required files
hellip
Setting up config files
hellip
Restarting services
nagios
nagios (pid 4155) is running
httpd
hellip
Solace Nagios Toolset Integration
12
44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the
Nagiosgraph install Sample page is shown below
Solace Nagios Toolset Integration
13
Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph
for one of the services (HTTP) for a day is shown below
Solace Nagios Toolset Integration
14
5 Solace Nagios Plugin Setup
51 Prerequisites Solace Nagios plugin requirs the following tools packages
Perl modules to handle XML amp Nagios
Solace pluging package
yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin
perl -MCPAN -e shell
cpangt install NagiosPlugin
cpangt install NagiosPluginDieNicely
cpangt install LWPUserAgent
cpangt install NagiosConfig
cpangt exit
52 Installation
1 Create Sample file
Create a sample router Config This file used by the setup script (below) to create commands for this config This step
can also be repeated for other routersvpns using the command mk_nagiossolacecfg script
cat cfgsolace-vmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=19216856102
HOSTPORT=8080
USERNAME=admin
PASSWORD=
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
Solace Nagios Toolset Integration
15
2 Setup Plugin
Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already
setup the related steps will be skipped
mk_nagiossolacecfg cfgsolace-vmr2cfg
Generating Nagios config for Solace appliance
Using cfgfile cfgsolace-vmr2cfg
Using template file cfgsolace-templaterouter-no_bi_bridgecfg
Generating config
etcnagiosobjectssolace-solace-vmr2cfg
Setting up config files
etcnagiosnagioscfg
Solace Nagios Toolset Integration
16
53 Post Install Setup
54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will
be added to the web page
A sample entry is shown below
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
11
43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if
necessary at the top of the script
Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific
tools are not in place or the Config files not setup corresponding steps will be skipped
431 Restarting Services
chkconfig httpd on ampamp chkconfig nagios on
systemctl restart httpd
systemctl restart nagios
setup_nagiossolace
Checking required files
hellip
Setting up config files
hellip
Restarting services
nagios
nagios (pid 4155) is running
httpd
hellip
Solace Nagios Toolset Integration
12
44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the
Nagiosgraph install Sample page is shown below
Solace Nagios Toolset Integration
13
Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph
for one of the services (HTTP) for a day is shown below
Solace Nagios Toolset Integration
14
5 Solace Nagios Plugin Setup
51 Prerequisites Solace Nagios plugin requirs the following tools packages
Perl modules to handle XML amp Nagios
Solace pluging package
yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin
perl -MCPAN -e shell
cpangt install NagiosPlugin
cpangt install NagiosPluginDieNicely
cpangt install LWPUserAgent
cpangt install NagiosConfig
cpangt exit
52 Installation
1 Create Sample file
Create a sample router Config This file used by the setup script (below) to create commands for this config This step
can also be repeated for other routersvpns using the command mk_nagiossolacecfg script
cat cfgsolace-vmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=19216856102
HOSTPORT=8080
USERNAME=admin
PASSWORD=
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
Solace Nagios Toolset Integration
15
2 Setup Plugin
Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already
setup the related steps will be skipped
mk_nagiossolacecfg cfgsolace-vmr2cfg
Generating Nagios config for Solace appliance
Using cfgfile cfgsolace-vmr2cfg
Using template file cfgsolace-templaterouter-no_bi_bridgecfg
Generating config
etcnagiosobjectssolace-solace-vmr2cfg
Setting up config files
etcnagiosnagioscfg
Solace Nagios Toolset Integration
16
53 Post Install Setup
54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will
be added to the web page
A sample entry is shown below
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
12
44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the
Nagiosgraph install Sample page is shown below
Solace Nagios Toolset Integration
13
Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph
for one of the services (HTTP) for a day is shown below
Solace Nagios Toolset Integration
14
5 Solace Nagios Plugin Setup
51 Prerequisites Solace Nagios plugin requirs the following tools packages
Perl modules to handle XML amp Nagios
Solace pluging package
yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin
perl -MCPAN -e shell
cpangt install NagiosPlugin
cpangt install NagiosPluginDieNicely
cpangt install LWPUserAgent
cpangt install NagiosConfig
cpangt exit
52 Installation
1 Create Sample file
Create a sample router Config This file used by the setup script (below) to create commands for this config This step
can also be repeated for other routersvpns using the command mk_nagiossolacecfg script
cat cfgsolace-vmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=19216856102
HOSTPORT=8080
USERNAME=admin
PASSWORD=
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
Solace Nagios Toolset Integration
15
2 Setup Plugin
Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already
setup the related steps will be skipped
mk_nagiossolacecfg cfgsolace-vmr2cfg
Generating Nagios config for Solace appliance
Using cfgfile cfgsolace-vmr2cfg
Using template file cfgsolace-templaterouter-no_bi_bridgecfg
Generating config
etcnagiosobjectssolace-solace-vmr2cfg
Setting up config files
etcnagiosnagioscfg
Solace Nagios Toolset Integration
16
53 Post Install Setup
54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will
be added to the web page
A sample entry is shown below
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
13
Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph
for one of the services (HTTP) for a day is shown below
Solace Nagios Toolset Integration
14
5 Solace Nagios Plugin Setup
51 Prerequisites Solace Nagios plugin requirs the following tools packages
Perl modules to handle XML amp Nagios
Solace pluging package
yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin
perl -MCPAN -e shell
cpangt install NagiosPlugin
cpangt install NagiosPluginDieNicely
cpangt install LWPUserAgent
cpangt install NagiosConfig
cpangt exit
52 Installation
1 Create Sample file
Create a sample router Config This file used by the setup script (below) to create commands for this config This step
can also be repeated for other routersvpns using the command mk_nagiossolacecfg script
cat cfgsolace-vmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=19216856102
HOSTPORT=8080
USERNAME=admin
PASSWORD=
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
Solace Nagios Toolset Integration
15
2 Setup Plugin
Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already
setup the related steps will be skipped
mk_nagiossolacecfg cfgsolace-vmr2cfg
Generating Nagios config for Solace appliance
Using cfgfile cfgsolace-vmr2cfg
Using template file cfgsolace-templaterouter-no_bi_bridgecfg
Generating config
etcnagiosobjectssolace-solace-vmr2cfg
Setting up config files
etcnagiosnagioscfg
Solace Nagios Toolset Integration
16
53 Post Install Setup
54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will
be added to the web page
A sample entry is shown below
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
14
5 Solace Nagios Plugin Setup
51 Prerequisites Solace Nagios plugin requirs the following tools packages
Perl modules to handle XML amp Nagios
Solace pluging package
yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin
perl -MCPAN -e shell
cpangt install NagiosPlugin
cpangt install NagiosPluginDieNicely
cpangt install LWPUserAgent
cpangt install NagiosConfig
cpangt exit
52 Installation
1 Create Sample file
Create a sample router Config This file used by the setup script (below) to create commands for this config This step
can also be repeated for other routersvpns using the command mk_nagiossolacecfg script
cat cfgsolace-vmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=19216856102
HOSTPORT=8080
USERNAME=admin
PASSWORD=
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
Solace Nagios Toolset Integration
15
2 Setup Plugin
Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already
setup the related steps will be skipped
mk_nagiossolacecfg cfgsolace-vmr2cfg
Generating Nagios config for Solace appliance
Using cfgfile cfgsolace-vmr2cfg
Using template file cfgsolace-templaterouter-no_bi_bridgecfg
Generating config
etcnagiosobjectssolace-solace-vmr2cfg
Setting up config files
etcnagiosnagioscfg
Solace Nagios Toolset Integration
16
53 Post Install Setup
54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will
be added to the web page
A sample entry is shown below
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
15
2 Setup Plugin
Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already
setup the related steps will be skipped
mk_nagiossolacecfg cfgsolace-vmr2cfg
Generating Nagios config for Solace appliance
Using cfgfile cfgsolace-vmr2cfg
Using template file cfgsolace-templaterouter-no_bi_bridgecfg
Generating config
etcnagiosobjectssolace-solace-vmr2cfg
Setting up config files
etcnagiosnagioscfg
Solace Nagios Toolset Integration
16
53 Post Install Setup
54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will
be added to the web page
A sample entry is shown below
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
16
53 Post Install Setup
54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will
be added to the web page
A sample entry is shown below
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
17
Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown
below
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
18
6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the
external command file Nagios would periodically check the external command file and process them This ldquopassive
checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured
hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status
updates and alerting
61 Prerequisites
62 Installation
1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291
(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)
2 Untar and follow install instructions in the package
$ cd optpkg (or directory of your choice)
$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site
for latest package name and replace)
$ tar xvzf nsca-291targz (adjust version accordingly)
$ cd optpkgnsca-291 (adjust version accordingly)
$ configure
Configuration summary for nsca 291 01-27-2012
General Options
-------------------------
NSCA port 5667
NSCA user nagios
NSCA group nagios
$ make all
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
19
63 Post install setup
1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca
This will generate required Config file and setup nsca as service
NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP
setup_nsca -s optpkgnsca-291
srcdir optpkgnsca-291 ok
target dir usrlocalnsca ok
Checking
hellip
Installing
hellip
Creating service
Installation complete
2 Start service
Now that the component is installed start it up
systemctl restart nsca
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
20
64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl
ndashh for usage info
641 Sending Host Alert
send_ngalertpl -n vmr3 -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending host command vmr30 - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0
Output - [Status=UP]
Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe
[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection
Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -
[Status=UP]
Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]
Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
21
642 Sending Service Alert
send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP
Reading server config usrlocalnscacfgnsca_servercfg
starting send_nsca client
sending service command vmr3hardware0 - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Handling the connection
1 data packet(s) sent to host successfully
varlogmessages Logs
Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3
Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service
Description hardware Return Code 0 Output - [Status=UP]
Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe
Nov 9 154116 centos7-vm nsca[9888] End of connection
Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND
PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]
Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]
Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
22
7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing
monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an
active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be
configured on both the server and the client end though the client end changes are relatively light
In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server
as client
71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that
network connectivity exists between the server and client hosts and the ports are open both ways
72 Installation
721 Server (CentOS)
In addition to Nagios server packages we would need to install the following These were already covered in previous
sections and listed here for completeness and for any standalone installs
NRPE
Nagios Pluggins
OpenSSL
yum install -y nrpe nagios-plugins-all openssl
722 Client (Ubuntu)
On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component
apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-
standard
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
23
73 Post Install Configuration
731 Client Configuration
1 Add Nagios server IP to allowed_hosts in nrpecfg
grep ^allowed_hosts etcnagiosnrpecfg
allowed_hosts=127001192168562011921681223
2 Restart service
systemctl start nagios-nrpe-server
732 Server Configuration
1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in
ldquoSetup Scripts and Config Filesrdquo section
2 Update nagioscfg (etcnagios by default) and add the client config file
For eg
rootcentos7-vm confd tail etcnagiosnagioscfg
cfg_file=etcnagiosobjectsnrpe-test1cfg
3 Now verify the config and fix any errors
nagios -v etcnagiosnagioscfg
4 Restart services
systemctl restart nagios
systemctl restart nrpe
systemctl restart httpd
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
24
74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot
below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status
75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and
differences
1 NCSA does passive check This mean client choses when to send the status to the server This allowes
fine grain control over the update timing and useful for near real time updates
2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically
This leads to simpler install
3 NRPE does active check The server pulls the info from the client periodically This is better suited for
centrally adminsterored policies
4 NRPE requires a daemon be runnion on all the clients to service NRPE requests
5 NCSA is not in active development anymore (no updates since 2013)
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
25
8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD
(Round Robin Database) It can collect manage and display graphs of collected data
81 Prerequisites
811 Cacti Dependencies
Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use
MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration
unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not
required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files
82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are
instructions and materials available that can be readily used Here are some lnks
httpdocscactinetmanual0881_installation1_install_unix
httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-
fedora-17-12
httpwwwcactinetdownloadsdocshtmldebugginghtml
83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to
Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for
illustration which can be replaced with any monitoring using right scripts
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
26
831 Add Custom Data Input method
Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the
solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script
need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected
form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot
do any validation on Output Field
8311 Data Input
8312 Input
8313 Output
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
27
832 Add Custom Data Template
Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to
declare info for the specific queue (router name vpn name queue name and credentials)
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
28
833 Add a custom Graph Template
Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph
Template Items)
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
29
834 Add Solace Appliance as Device
Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to
run host specific services like ping test
8341 Add Data Source List
Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown
here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the
correspoinding RRD files for these resources
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
30
8342 Add Graph List
Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the
specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
31
84 Verification
841 Basic Install
Once you have basic Cacti setup up and running you should be able to access the install and do further configuration
thru web admin page at
httpltyour-server-ipgtcacti
After configuration you would be able to see basic system metrics for the localhost that would look similar to this
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
32
842 Solace host check
Once Solce router is added as a device from he Devices link you would be able view the router with basic health test
such as ping status
843 Solace VPN Metrics Sample
Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue
statistics showing queue depth for a single queue on a VPN
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
33
As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of
aggregation of queue stats over various window period from hour to year
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
34
9 Additional Topics
91 Admin Tasks
911 Creating custom Maps
Create custom maps for any outputs that need it for example turn the iface counter values into gauge
Add to usrlocalnagiosgraphetcnagiosgraphmap
Service type iface
outputIFACE OK - [Interface161EnabledyesLinkyes]
perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-
bytes=363571175913
perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)
and push s [ if_stats
[tx-pkts COUNTER int $1 ]
[rx-pkts COUNTER int $2 ]
[tx-bytes COUNTER int $3 ]
[rx-bytes COUNTER int $4 ] ]
912 Adding additional routers VPNs
Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run
mk_nagiossolacecfg to generate required solace plugin Config files
Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be
added into etcnagiosnagioscfg file
Note that the services need to be restarted for the changes to be effective
913 Putting Nagios on Verbose mode
Change debug level in Nagios config to 2048
grep ^debug_level etcnagiosnagioscfg
debug_level=2048
Now check the debug logs All commands executed and output will be logged over there
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
35
tail varlognagiosnagiosdebug
[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING
[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w
usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --
queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet
[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w
usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --
vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet
[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING
914 Modifying iptables
When you Cant access http from outside or when telnet to server on port 80 fail
Check Iptables routes
iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT
iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT
iptables-save | tee etcsysconfigiptables
service iptables save
service iptables restart
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
36
92 Troubleshooting
921 Nagios not starting up
Check Config file for any errors
usrsbinnagios -v etcnagiosnagioscfg
922 Canrsquot connect to HTTP (or other port) from outside
When access to port is working form within the box (for localhost) and not form outside check the iptables
923 CGI Error in showcgi
Check http errorlog
[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of
usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-
binstatuscgihost=all
[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers
showcgi referer http192168150137nagioscgi-binstatuscgihost=all
Check file and dir permissions They should be world readable and executable
Check SE (Security Enhanced Linux) Mode
chmod go+rx usrlocalnagiossbin
chmod go+rx usrlocalnagiossbinshowcgi
setenforce 0
service httpd restart
service nagios restart
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
37
924 CGI Error in Solace graphs
Check that the CGI scripts are runnable from command line
Check for missing required modules and Perl libraries
usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v
DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in
fully redundant state]
925 Canrsquot stat command file error
If you see error
Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo
chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
38
10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product
document for most recent syntax for config files
101 Install Info Default install paths and files
HTTP (Apache)
Config etchttpdconfhttpdconf
Logs varloghttpderror_log varloghttpdaccess_log
Nagios
Config file etcnagiosnagioscfg
Commands etcnagiosobjectscommandscfg
Log varlognagiosnagioslog
Perfdata log varlognagiosperfdatalog
Config Dir etcnagiosobjects
Command file varspoolnagioscmdnagioscmd
RRD
Data dir varnagiosrrd
Nagiosgraph
Config usrlocalnagiosgraphetc nagiosgraphconf
Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf
Log usrlocalnagiosgraphvarnagiosgraphlog
Nagios Solace
Config etcnagiossolacesolacecfg
Plugin dir usrlocalnagiosplugins
NSCA Daemon and Agent
Path usrlocalnsca
Config (server) usrlocalnscacfgnscacfg
Config (client) usrlocalnscacfgnsca_servercfg
usrlocalnscacfgnsca_clientcfg
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
39
102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball
package with the files and required plugin files
1021 setup_nagiossolace
binbash
setup_nagiossolace
httpcfg=etchttpdconfhttpdconf
nagioscfg=etcnagiosnagioscfg
nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf
nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf
nagioscmdfile=etcnagiosobjectscommandscfg
nagiostmplfile=etcnagiosobjectstemplatescfg
nagioslocalhostcfg=etcnagiosobjectslocalhostcfg
nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg
nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl
nagiossolaceplugindir=usrlocalnagiosplugins
nagiossolacecfgdir=etcnagiossolace
tag=BEGIN_SETUP-Nagiosgraph
echo
echo Checking required files
for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg
$nagiosgraphinsertscript do
if [ -f $f ] then
echo $f missing Exiting
exit
fi
echo $f
grep -q $tag $f ampamp echo already setup will be skipped
done
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
40
-------------------------------------------------------------------------
ts=$(date +ymd_HMS)
echo
echo Setting up config files
echo $nagioscfg
grep -q $tag $nagioscfg ||
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt $nagioscfg ltlt__END__
--- $tag
process nagios performance data using nagiosgraph
process_performance_data=1
service_perfdata_file=varlognagiosperfdatalog
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30
service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
host config with sample nagiosgraphs
cfg_file=etcnagiosobjectslocalhost-graphcfg
solace config
cfg_file=etcnagiossolacesolacecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo $nagiostmplfile
grep -q $tag $nagiostmplfile ||
cp -p $nagiostmplfile $nagiostmplfilesav-$ts
cat gtgt $nagiostmplfile ltlt__END__
-- $tag
define service
name graphed-service
action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
41
onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-
binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j
register 0
-- end setup
__END__
-------------------------------------------------------------------------
localhost-graphcfg generaed if not there already
outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg
echo $outfile
if [ -f $outfile ] then
sed -e slocal-servicelocal-servicegraphed-serviceg
-e s(localhost)1-grg
-e s(linux-servers)1-grg
$nagioslocalhostcfg gt tmplocalhost-graphcfg
cp -p tmplocalhost-graphcfg $outfile
fi
-------------------------------------------------------------------------
echo $nagioscmdfile
grep -q $tag $nagioscmdfile ||
cp -p $nagioscmdfile $nagioscmdfilesav-$ts
cat gtgt $nagioscmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata-for-nagiosgraph
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
42
echo $nagiosgraphcmdfile
grep -q $tag $nagiosgraphcmdfile ||
cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts
cat gtgt $nagiosgraphcmdfile ltlt__END__
-- $tag
command to process nagios performance data for nagiosgraph
define command
command_name process-service-perfdata
command_line $nagiosgraphinsertscript
-- end setup
__END__
-------------------------------------------------------------------------
note -- this file is overwritten (not append)
echo $nagiosgraphapachecfg
grep -q $tag $nagiosgraphapachecfg ||
cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts
cat gt $nagiosgraphapachecfg ltlt__END__
--- $tag
enable nagiosgraph CGI scripts
ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi
ltDirectory usrlocalnagiosgraphcgigt
Options ExecCGI
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
enable nagiosgraph CSS and JavaScript
Alias nagiosgraph usrlocalnagiosgraphshare
ltDirectory usrlocalnagiosgraphsharegt
Options None
AllowOverride None
Order allowdeny
Allow from all
ltDirectorygt
-- end setup
__END__
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
43
-------------------------------------------------------------------------
echo $nagiosgraphcfg
grep -q $tag $nagiosgraphcfg ||
cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts
sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng
$nagiosgraphcfg gt tmpnagiosgraphcfg
sed -e snagioscgi-binnagiosgraphcgi-bing
-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg
-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg
$nagiosgraphcfg gt tmpnagiosgraphcfg
cp -p tmpnagiosgraphcfg $nagiosgraphcfg
-------------------------------------------------------------------------
echo $httpcfg
grep -q $tag $httpcfg ||
cp -p $httpcfg $httpcfgsav-$ts
cat gtgt $httpcfg ltlt__END__
--- $tag
include $nagiosgraphapachecfg
-- end setup
__END__
-------------------------------------------------------------------------
echo
echo Setting Solace for Nagios
if [ -d plugin ] then
echo $nagiossolaceplugindir
[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir
cp -p plugin $nagiossolaceplugindir
chmod 555 $nagiossolaceplugindir
else
echo No plugin dir in package $nagiossolaceplugindir not setup
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
44
fi
if [ -f cfgsolacecfg ] then
echo $nagiossolacecfgdir
[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir
cp -p cfgsolacecfg $nagiossolacecfgdir
else
echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup
fi
-------------------------------------------------------------------------
try to gernerate sample cfg
mk_nagiossolacecfg cfgsampleroutercfg
-------------------------------------------------------------------------
echo
echo Restarting services
for s in nagios httpd do
echo $s
systemctl restart $s gt tmp$sout 2gtamp1
systemctl status $s
done
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
45
1022 mk_nagiossolacecfg
binbash
mk_nagiossolacecfg
generate nagios cfg file solace router from template
History
-------------------------------------------------------------
nram Nov 04 2015
- check and exit if cfgfile is added in nagioscfg
- handle cfg file with no or only unidirectional bridge
me=mk_nagiossolacecfg
nagioscfg=etcnagiosnagioscfg
nagioscmdfile=etcnagiosobjectscommandscfg
echo
echo Generating Nagios config for Solace appliance
tmplfile=cfgsolace-templateroutercfg
if [ $ -lt 1 ] then
echo Missing cfgfile argument exiting
exit
fi
cfgfile=$1
if [ -f $cfgfile ] then
echo Bad cfgfile argument $cfgfile exiting
exit
fi
echo Using cfgfile $cfgfile
$cfgfile
echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt
echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt
if [ -z $VPN_BRIDGENAME ] then
tmplfile=cfgsolace-templaterouter-no_bridgecfg
elif [ -z $VPN_BRIDGENAME_BI ] then
tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg
fi
if [ -f $tmplfile ] then
echo Template file not found $tmplfile exiting
exit
fi
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
46
echo Using template file $tmplfile
ext=$outfile
fname=$(basename $cfgfile)
outfile=etcnagiosobjectssolace-$fnamecfg
if [ -f $outfile ] then
echo File $outfile already exists exiting
exit
fi
grep -q ^cfg_file=$outfile $nagioscfg ampamp
echo File $outfile already in $nagioscfg exiting
exit
gt $outfile ||
echo Cant create output file $outfile exiting
exit
echo Generating config
echo $outfile
cat gt $outfile ltlt__END__
Nagios config for soalce appliance
generated from cfg file $cfgfile
template file used $tmplfile
$(date)
__END__
cat $tmplfile | while read line
do
eval echo $line
done gtgt $outfile
ts=$(date +ymd_HMS)
echo Setting up config files
echo $nagioscfg
cp -p $nagioscfg $nagioscfgsav-$ts
cat gtgt$nagioscfg ltlt__END__
-- created by $me $(date +D T)
cfg_file=$outfile
__END__
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
47
1023 Solace Nagios Config (cfgsolacecfg)
define command
command_name info_not_defined
command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not
defined in config
define command
command_name alarm
command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name bridge_uni
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q
define command
command_name bridge_bi
command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q
define command
command_name disk
command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$
-p $ARG3$
define command
command_name env
command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$
define command
command_name hardware
command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b
define command
command_name iface_eth
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
48
$ARG2$ -p $ARG3$ -i $ARG4$
define command
command_name iface_lag
command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$
define command
command_name memory
command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$
define command
command_name spool_vpn
command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r
$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$
define command
command_name message-spool
command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$
-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam
$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -
-wtm $ARG16$ --ctm $ARG17$ --quiet
define command
command_name partition
command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u
$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$
define command
command_name queue
command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --
checkAvail --quiet
define command
command_name redundancy
command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u
$ARG2$ -p $ARG3$
define command
command_name replication
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
49
command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -
u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet
define command
command_name vpn
command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --
user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --
warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
50
1024 Template router (cfgsolace-templateroutercfg)
define host
use generic-host
host_name $HOSTNAME
alias $HOSTNAME
address $HOSTIP
max_check_attempts 10
define service
use generic-service
host_name $HOSTNAME
service_description alarm
check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description bridge
check_command
bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description disk
check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description env
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
51
check_command
env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100
define service
use generic-service
host_name $HOSTNAME
service_description hardware
check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface
check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description iface_l
check_command
iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111
define service
use generic-service
host_name $HOSTNAME
service_description memory
check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090
define service
use generic-service
host_name $HOSTNAME
service_description spool_vpn
check_command
spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090
define service
use generic-service
host_name $HOSTNAME
service_description message-spool
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
52
check_command message-
spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000
00003000055000100000050000000
define service
use generic-service
host_name $HOSTNAME
service_description partition
check_command
partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6
define service
use generic-servicegraphed-service
host_name $HOSTNAME
service_description queue
check_command
queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000
define service
use generic-service
host_name $HOSTNAME
service_description redundancy
check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD
define service
use generic-service
host_name $HOSTNAME
service_description replication
check_command
replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME
define service
use generic-service
host_name $HOSTNAME
service_description vpn
check_command
vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
53
1025 Template and Sample router Config
$ cat cfgsampleroutercfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=your_hostname
HOSTIP=your_hostip
HOSTPORT=your_hostport
USERNAME=user_username
PASSWORD=user_password
VPNNAME=your_vpn
VPN_BRIDGENAME=your_bridgename-uni
VPN_BRIDGENAME_BI=your_bridgename-bi
$ cat cfgvmr2cfg
change the your_XXX in this file before running setup_nagiossolace script
HOSTNAME=solace-vmr2
HOSTIP=1921681140
HOSTPORT=8080
USERNAME=admin
PASSWORD=admin
VPNNAME=vpn1
VPN_BRIDGENAME=bridge_vpn1_to_2
VPN_BRIDGENAME_BI=
103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info
httpdevsolacesystemscomcontact-us
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
54
104 NSCA
1041 setup_nsca
binbash
setup_nsca setup nsca daemon
while getopts st arg
do
case $arg in
s) SRCDIR=$OPTARG
i) SVRIP=$OPTARG
t) TARGETDIR=$OPTARG
) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0
exit 2
esac
done
if [ -z $SRCDIR ] then
printf Missing -s ltnsca-src-dirgt Exiting n
exit 2
fi
printf srcdir $SRCDIR okn
if [ -z $SVRIP ] then
SVRIP=127001
printf Missing -i ltserver-ipgt Using $SVRIPn
fi
printf serverip $SVRIPn
[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca
if [ -d $TARGETDIR ] then
printf target dir $TARGETDIR already exists Exiting n
exit 3
fi
printf target dir $TARGETDIR okn
echo
host=$SVRIP
port=5667
gt cfgnsca_servercfg
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
55
printf Checking n
for file in srcnsca srcsend_nsca do
if [ -x $SRCDIR$file ] then
printf Required file $SRCDIR$file not found Exiting n
exit 3
fi
printf file $file okn
done
for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg
cfgnsca_clientcfg do
if [ -f $file ] then
printf Required file $file not found Exiting n
exit 3
fi
printf file $file okn
done
mkdir -p $TARGETDIRbincfgsbin
[ -d varrunnsca ] || mkdir varrunnsca
printf Installing n
printf $TARGETDIRsbinnsca n
cp -p $SRCDIRsrcnsca $TARGETDIRsbin
printf $TARGETDIRbinsend_nsca n
cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin
printf $TARGETDIRcfgnscacfg n
cp -p cfgnscacfg $TARGETDIRcfg
printf $TARGETDIRbinsend_ngalertpl n
cp -p scriptssend_ngalertpl $TARGETDIRbin
if [ -f etcinitdnsca ] then
printf etcinitdnsca n
cp -p scriptsnscainit etcinitdnsca
printf Creating service n
chkconfig --add nsca
else
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
56
echo etcinitdnsca exists Skipping service creation
fi
printf Installation completen
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
57
1042 send_ngalertpl
usrbinperl -w
------------------------------------------------------------------------
send_ngalertpl
send alert to nagios using NSCA agent
usage
see usage below
------------------------------------------------------------------------
use strict
use warnings
use SysHostname
use GetoptLong
my $me = send_ngalertpl
my ($help $verbose)
my $nscadir = usrlocalnsca
my $host = hostname()
my ($service $rc $info $data $msg)
my statusMap = (OK =gt 0 UP =gt 0
DOWN =gt 1 WARN =gt 1
UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)
-------------------------------------------------------------------
process args
GetOptions (nscadir|d=s =gt $nscadir
host|n=s =gt $host
status|c=s =gt $rc
service|s=s =gt $service
statusinfo|i=s =gt $info
perfdata|p=s =gt $data
message|m=s =gt $msg
help|h =gt $help
verbose|v =gt $verbose)
my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg
$nscadircfgnsca_clientcfg
$nscadirbinsend_nsca)
-------------------------------------------------------------------
usage
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
58
if ($help)
printf $me send alert message to Nagios server using NSCA agent
Usage
$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message
Options
[-nscadir|d ltnsca-dirgt] dir with nsca client installed
usrlocalnsca by default
[-host|n lthostgt] host name for alert (localhost by default)
[-service|s ltservicegt service for alert
sends a service alert if provided
sends a host alert if missing
-status|c ltstatusgt alert status code
Host Status
UP DOWN UNR (unreachable)
Service Status
OK WARN CRIT (critical)
-message|m ltmessagegt message to send
-statusinfo|i ltinfogt status info to send
-perfdata|d ltdatagt performance data to send
-h help (this message)
-v verbose
Payload
ltmessagegt - [ltstatusinfogt] perfdata
Pl refer Nagios documentation for format for the payload
Sample
Send host alert
$me -n demo-tr -c OK -i Status=UP -m remote host up
Send service alert
$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010
Errors=31010 -m remote service alert
exit 1
-------------------------------------------------------------------
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
59
validate args env
die Missing message arg -m unless $msg
die Missing status-code arg -c unless $rc
die Unknown status-code $rc unless exists $statusMap$rc
die Error reading server config $svrcfg unless -e $svrcfg
die Error reading client config $svrcfg unless -e $clcfg
-------------------------------------------------------------------
process server cfg
my svrcfgMap
open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg
print Reading server config $svrcfgn
while (ltSVRCFGgt)
next if
my ($tag $val) = split =
chop
chomp $val
$svrcfgMap$tag = $val
print svrcfgMap$tag = $valn if $verbose
close (SVRCFG)
my $status = $statusMap$rc
if ($verbose)
printf -- input --
nscadir $nscadir
server cfg $svrcfg
client cfg $clcfg
send_nsca $send_nsca_bin
hostname $host
service $service
status $rc ($status)
Message ---
Status Info $info
Perf Data $data
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
60
my $cmd
my $cmdtype = host
$msg = - [$info] if $info
$msg = | $data if $data
if ($service)
$cmdtype = service
$cmd = $host$service$status$msg
else
$cmd = $host$status$msg
print Command n$cmdn if $verbose
printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg
-d n if $verbose
printf starting send_nsca client n
open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||
die Error opening send_nsca ($send_nsca_bin)
print sending $cmdtype command $cmdn
print SEND_NSCA $cmdn
while (ltSEND_NSCAgt)
close (SEND_NSCA)
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
61
1043 nsca_init
binbash
nsca Startup script for the Nagios Service Check Acceptor
chkconfig - 85 15
description Nagios service check acceptor daemon
processname nsca
config usrlocalnscacfgnscacfg
pidfile varrunnscanscapid__NOT_USED
BEGIN INIT INFO
Provides nsca
Required-Start $local_fs $remote_fs $network $named
Required-Stop $local_fs $remote_fs $network
Should-Start distcache
Short-Description start and stop Nagios Service check Accaptor Server
Description Nagios extension
END INIT INFO
Start nsca in the C locale by default
NSCA_LANG=$NSCA_LANG-C
This will prevent initlog from swallowing up a pass-phrase prompt if
mod_ssl needs a pass-phrase from the user
INITLOG_ARGS=
Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server
with the thread-based worker MPM BE WARNED that some modules may not
work correctly with a thread-based MPM notably PHP will refuse to start
nsca=$NSCA-usrlocalnscasbinnsca
nscacfg=usrlocalnscacfgnscacfg
prog=nsca
pidfile=$PIDFILE-varrunnscanscapid
lockfile=$LOCKFILE-varlocksubsysnsca
RETVAL=0
STOP_TIMEOUT=$STOP_TIMEOUT-10
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
62
OPTIONS=-c $nscacfg --inetd
OPTIONS=-c $nscacfg
start()
echo -n $Starting $prog
LANG=$NSCA_LANG $nsca $OPTIONS
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp touch $lockfile
pgrep -f usrlocalnscasbinnscanscacfg$
return $RETVAL
When stopping nsca a delay (of default 10 second) is required
before SIGKILLing the nsca parent this gives enough time for the
nsca parent to SIGKILL any errant children
stop()
echo -n $Stopping $prog
pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f
usrlocalnscasbinnscanscacfg$
RETVAL=$
echo
[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile
See how we were called
case $1 in
start)
start
stop)
stop
status)
pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)
if [ -z $pid ] then
echo nsca is not running
else
echo nsca (pid $pid) is running
fi
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
63
restart)
stop
start
)
echo $Usage $prog start|stop|restart|condrestart|try-restart|force-
reload|reload|status|fullstatus|graceful|help|configtest
RETVAL=2
esac
exit $RETVAL
1044 NSCA Config
==gt cfgnscacfg lt==
log_facility=daemon
pid_file=varrunnscapid
server_port=5667
nsca_user=nagios
nsca_group=nagios
debug=0
command_file=varspoolnagioscmdnagioscmd
alternate_dump_file=usrlocalnagiosvarrwnscadump
aggregate_writes=0
append_to_file=0
max_packet_age=30
decryption_method=1
==gt cfgnsca_clientcfg lt==
encryption_method=1
==gt cfgnsca_servercfg lt==
host=127001
port=5667
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
64
105 NRPE
1051 Sample server config file
Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that
should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server
This config file does the following checks
regular host ping check (non NRPE)
check users (NRPE)
check load (NRPE)
cat etcnagiosobjectsnrpe-test1cfg
define host
use generic-host
host_name ubuntu-svr
alias ubuntu
address 1921681108
max_check_attempts 10
define service
use generic-service
host_name ubuntu-svr
service_description PING
check_command check_ping100020500060
define service
use generic-service
host_name ubuntu-svr
service_description Current Load
check_command check_nrpecheck_load
define service
use generic-service
host_name ubuntu-svr
service_description Total Processes
check_command check_nrpecheck_users
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped
Solace Nagios Toolset Integration
65
1052 Sample Client Config
cat etcnagiosnrpecfg
log_facility=daemon
pid_file=varrunnagiosnrpepid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127001192168562011921681223
dont_blame_nrpe=0
allow_bash_command_substitution=0
debug=0
command_timeout=60
connection_timeout=300
command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10
command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520
command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1
command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z
command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200
include=etcnagiosnrpe_localcfg
include_dir=etcnagiosnrped