Nagios Conference 2014 - Mike Weber - Nagios Rapid Deployment Options

PowerPoint Presentation

Rapid Deployment
Project

Mike Weber

[email protected]

Why: Rapid Deployment

Installation Size
5 years ago 1000 service checks
today 150,000 to 200,000

Admin Time Resources

Admin Skill Levels

Current Options

Core: Multiple Scripts
focus on importing not design

XI: Auto Discovery Wizard
discovery and import hosts and ports

XI: Bulk Host Import Wizard
clone current host with services
five field options which must be unique

Auto Discovery

Auto Discovery

Auto Discovery

Discovers Hosts

Discovers Ports

No Internal Discovery

Bulk Host Import Wizard




Clone Host Settings
host templates
host local settings

Import Hostgroups

Import Description

Import Host Parent

No Multiple Hostgroups

No Designated Host Template

No Exception Management

Rapid Deployment Project

Efficient Design
using object inheritance
service check selection
attention to triggers
notification
troubleshooting
solutions for exceptionsEfficient Check Selection
searchable by type, os, version
scripts for download
examples in text and imageEfficient Deployment
scripted implementation for 100s of hosts and 1000s of checks

Design Principles

Create Leverage
Managing from one service template per group

Maintain Simplicity
One template equals easier troubleshooting

Manage Exceptions
No Local Settings

Create Leverage

Skeleton Structure

Host Host Template (provides all necessary settings to all similar hosts)

Hostgroup Service Template Services

Create Leverage

#1: Create Host Templates: Core

define host { name lx_dev_ht alias Linux Dev Host Template check_command check_icmp!200.0,30%!300.0,40%!!!!!! max_check_attempts 5 check_interval 5 retry_interval 1 check_period 24x7 contact_groups lx_dev_cg notification_interval 300 notification_period workhours register 0

}

Create Leverage

#1: Create Host Templates: XI

Create Leverage

#2: Create Hostgroups: Core

define hostgroup { hostgroup_name lx_mysql_prod_hg alias Linux MySQL Prod Hostgroup members lx56prodmt,lx67prodmt }

define hostgroup { hostgroup_name lx_prod_hg alias Linux Prod Hostgroup members lx56prodmt,lx67prodmt }

Create Leverage

#2: Create Hostgroups: XI

Create Leverage

#3: Create Test Host: Core

define host { host_name lx56prodmt use lx_prod_ht address 192.168.5.156 register 1 }

Create Leverage

#3: Create Test Host

Create Leverage

#4: Create Service Template: Core

define service { name lin_prod_service service_description Linux Prod Service Template max_check_attempts 5 check_interval 4 retry_interval 1 check_period 24x7 notification_interval 60 notification_period 24x7 contacts nagiosadmin register 0

}

Create Leverage

#4: Create Service Template: XI

Create Leverage

#5: Create Services: Core

define service { service_description Linux CPU use lx_os_prod_st check_command check_nrpe!check_cpu_stats!-a '-w 85 -c 95'!!!!!! register 1 }

Create Leverage

#5: Create Services

Maintain Simplicity

5 Objects Model
Host Template
Hostgroup
Service Template
Services
Hosts

Troubleshooting
No Nesting

Training

Manage Exceptions

Local Settings
Since local settings are empty you can use those settings to override the template.

2 Objects Model
Service Template
Services

Local Exceptions

Exceptions

Summary

Focus on Design Leverage

Focus on Design Simplicity

Focus on Design Exceptions

Service Check Database


Searchable Database
Searchable by Type (WMI,NRPE,etc)
Searchable by Group (OS Metrics, SQL,Oracle,Apache, etc.)
Searchable by OS (Windows ,Linux,etc.)
Searchable by Version (Windows 2012,CentOS 6.x,etc.)

Explanation of Variables

Copy and Paste Scripts/Checks

Image Example: XI

Textual Example: Core








Current ProjectsWMI (check_wmi_plus v.1.59 = 70 checks)
goal discovery what is available
currently working on script to check availability of WMI classes
currently writing ini files for WMI classes
focus on 2012 servers

Oracle Database

SQL Database

Rapid Deployment: Core

Bash Shell ScriptMultiple Hostgroups

Parents

Switches/Routers with MRTG


Usage [IP_ADDR1= IP_ADDR2=...] ${1} [output directory] [community string]


The IP Addresses are the addresses for the router and switch in each store so MRTG can build config.

IP_ADDR1=10.10.1.1 IP_ADDR=10.10.1.2 ./create_stores.sh files/st00260.csv /usr/local/nagios/etc/objects/stores/st00260.cfg /usr/local/nagios/etc/objects/stores/store_hostgroups.cfg


CSV Filehostaddress,10.10.88.32,,ap01.st00778.st.example.com

hostaddress,10.10.88.33,,ap02.st00778.st.example.com

hostaddress,10.10.88.34,,reg01.st00778.st.example.com


##### Store Templates Used to Build Configs define host{ use kiosk host_name HOST_NAME alias HOST_NAME address IP_ADDRESS parents PARENT}


define hostgroup{ hostgroup_name hg_aps alias Store Access Pointsmembers }define hostgroup{ hostgroup_name hg_pcs alias Store PCsmembers }define hostgroup{ hostgroup_name hg_kiosks alias Store Kiosksmembers }


define service{use store-servicehostgroups hg_stsrv1_prodservice_description check_store_oracle_diskcheck_command check_nrpe!check_store_oracle_diskcheck_interval 15max_check_attempts 3normal_check_interval 5retry_check_interval 1check_period 24x7notification_period 24x7contact_groups linux_store_admins,store_support}


Each host must also be appended to the appropriate hostgroup. Hostgroups include:hg_aps - access points
hg_pcs- pcs, recv,deptmgr,muzak,train,wgb
hg_kiosks- kiosks (Windows 7 hosts)
hg_radiopc_prod
hg_stsrv1_prod- stsrv (store server)
hg_radiopc- radiopc (wireless)
hg_manager - managerpc (manager desktop)
hg_register- anything with "reg" (PCI issues)
hg_stprinter - anything with "prt" (printers)
hg_clock- clocks (time clocks for employees)
hg_winsrv- winsrv (Windows Server)


Goal is Full Deployment8 Fields for Deployment
address
host
host description
3 hostgroups
host template
parents

OS Metrics Hostgroup

Application Hostgroups = 2

Rapid Deployment Wizard



Rapid Deployment: XI

Python Script8 Fields for Deployment
address
host
host description
3 hostgroups
host template
parents

Resource Efficient for Deployment

XI: Script

XI: Script

Format of CSV File:ip_address,hostname,alias,'hostgroup1, hostgroup2, hostgroup3',host template,parents

74.125.239.40,940-60,ny_server,ny-windows-server-hg,windows-server-ht74.125.239.36,940-61,,ny-windows-server-hg,ny-oracle-hg,windows-server-ht74.125.239.34,940-32,ny-linux-server-hg,linux-server-ht

XI: Script

Format:ip_address,hostname,alias,'hostgroup,hostgroup,hostgroup',host template,host parentsOne HostgroupThe hostgroups should end in "hg".192.168.5.157,lx67prodmt,MySQL_Server,lx_prod_hg,lx_prod_ht,,

Two Hostgroups (Note the use of ' to enclose the hostgroups)192.168.5.157,lx67prodmt,MySQL_Server,'lx_prod_hg,lx_mysql_prod_hg',lx_prod_ht,

Two Hostgroups and Parents Setting192.168.5.157,lx67prodmt,MySQL_Server,'lx_prod_hg,lx_mysql_prod_hg',lx_prod_ht,cisco_sm300

XI: Script

Hostgroupslx_prod_hg Linux Production Hostgrouplx_dev_hg Linux Development Hostgroup

Host Templateslx_prod_ht Linux Production Host Templatelx_dev_ht Linux Development Host Template

Service Templateslx_os_prod_st Linux OS Metrics Production Service Template lx_mysql_prod_stLinux MySQL Production Service Templatelx_apache_prod_stLinux Apache Production Service Templatelx_defense_prod_stLinux Defensive Metrics Production Service Templatelx_os_dev_st Linux OS Metrics Development Service Template lx_mysql_dev_stLinux MySQL Development Service Templatelx_apache_dev_stLinux Apache Development Service Templatelx_defense_dev_stLinux Defensive Metrics Development Service Template

XI: Script

Serviceslx_cpu_os Linux CPUlx_mem_os Linux Memory lx_root_part_os Linux / Partitionlx_home_part_os Linux /boot Partitionlx_processes_os Linux Processeslx_ssh_os Linux SSH lx_load_os Linux Load lx_files_os Linux Fileslx_users_os Linux Users lx_cron_os Linux Cronlx_rsyslog_os Linux rsyslog

lx_connect_time_mysql MySQL Connect Time lx_open_conn_mysql MySQL Open Connections lx_thr_cache_hit_mysql MySQL Thread Cache Hit Ratiolx_tab_cache_hit_mysql MySQL Table Cache Hit Ratelx_slow_queries_mysql MySQL Slow Queries lx_long_run_proc_mysql MySQL Long Running Processeslx_uptime_mysql MySQL Uptime

XI: Script

Contact Groupslx_prod_cg Linux Production Contact Grouplx_dev_cg Linux Development Contact Group

lx_mysql_prod_cg Linux MySQL Production Contact Grouplx_mysql_dev_cg Linux MySQL Development Contact Grouplx_apache_prod_cg Linux Apache Production Contact Grouplx_apache_dev_cg Linux Apache Development Contact Grouplx_defense_cg Linux Defense Contact Group

XI: Script

Output:Running pre-flight check on configuration data...Checking objects...Checked 311 services....Checking obsessive compulsive processor commands...Checking misc settings...

Total Warnings: 0Total Errors: 0Things look okay - No serious problems were detected during the pre-flight checkRET: 0Running configuration check...done.Stopping nagios: .done.Starting nagios: done.

Script Downloads

Rapid Deployment Scripts:

Beginlinuxservers.com/nagiosconf

User: nagiosconference
Password: 54TBwh9

Only available during conference.

Conclusion

Mike Weber

[email protected]

Click to edit Master title style

Click to edit Master text styles

Second level

Third level

Fourth level

Fifth level

10/16/14


Click to edit Master subtitle style

10/16/14


Second level
Third level
Fourth level
Fifth level

10/16/14



10/16/14


Second level
Third level
Fourth level
Fifth level

Second level
Third level
Fourth level
Fifth level

10/16/14



Second level
Third level
Fourth level
Fifth level


Second level
Third level
Fourth level
Fifth level

10/16/14


10/16/14

10/16/14


Second level
Third level
Fourth level
Fifth level


10/16/14



10/16/14



Second level

Third level

Fourth level

Fifth level

10/16/14



Second level

Third level

Fourth level

Fifth level

10/16/14

PRESENTATION TITLE

Presenter Name

10/16/14

[email protected]



Second level

Third level

Fourth level

Fifth level

10/16/14

PRESENTATION TITLE

Presenter Name

10/16/14

[email protected]


Second level
Third level
Fourth level
Fifth level

10/16/14



10/16/14


Second level
Third level
Fourth level
Fifth level

Second level
Third level
Fourth level
Fifth level

10/16/14



Second level
Third level
Fourth level
Fifth level


Second level
Third level
Fourth level
Fifth level

10/16/14


10/16/14

10/16/14


Second level
Third level
Fourth level
Fifth level


10/16/14



10/16/14



Second level

Third level

Fourth level

Fifth level

10/16/14



Second level

Third level

Fourth level

Fifth level

10/16/14

Nagios Conference 2014 - Mike Weber - Nagios Rapid Deployment Options

Technology

Transcript of Nagios Conference 2014 - Mike Weber - Nagios Rapid Deployment Options