I B M Op e r a ti o n s An a l yti cs f o r z S y s t e m s IBM able 1. New features in IBM...

IBM Operations Analytics for z Systems

PDF file of IBM Knowledge Center topicsVersion 2 Release 2

IBM

ii Operations Analytics for z Systems: PDF file of IBM Knowledge Center topics

Contents

What's new in Version 2.2.0 . . . . . . 1

Conventions used in this information . . 7

Operations Analytics for z Systemsoverview . . . . . . . . . . . . . . 9z/OS Insight Packs . . . . . . . . . . . . 11IBM Operations Analytics - Log Analysis . . . . 12Extensions to Log Analysis for z/OS ProblemInsights and client-side Expert Advice . . . . . 13z/OS Log Forwarder . . . . . . . . . . . 15Installation and configuration checklists . . . . . 15

Log Analysis checklist . . . . . . . . . . 16z/OS Insight Packs and extensions checklist . . 17z/OS Log Forwarder checklist . . . . . . . 17

Planning for installation andconfiguration of Operations Analyticsfor z Systems . . . . . . . . . . . . 21System requirements . . . . . . . . . . . 21

z/OS Insight Pack requirements . . . . . . 21z/OS Log Forwarder requirements . . . . . 22

Planning for installation of Log Analysis . . . . 24Planning for Log Analysis to support LDAPinteraction with RACF for sign-on authentication . . 24Planning for installation of the z/OS Log Forwarder 25Planning for configuration of the z/OS LogForwarder . . . . . . . . . . . . . . . 26

z/OS Log Forwarder program files . . . . . 26Required authorities for z/OS Log Forwarderoperations . . . . . . . . . . . . . . 27z/OS Log Forwarder start procedure . . . . . 28Authorization with APF . . . . . . . . . 28Data sources . . . . . . . . . . . . . 29Scalable data collection architecture . . . . . 33Data collection . . . . . . . . . . . . 35Data forwarding . . . . . . . . . . . . 50

Installing Operations Analytics for zSystems . . . . . . . . . . . . . . 53Installing Log Analysis . . . . . . . . . . 53Installing the z/OS Insight Packs and extensions . . 55

Using the manual method of installation . . . 57Uninstalling the Insight Packs . . . . . . . . 58Removing data from Log Analysis. . . . . . . 59Uninstalling Log Analysis . . . . . . . . . 60

Upgrading Operations Analytics for zSystems . . . . . . . . . . . . . . 63Upgrading to V2.2.0.1 Interim Feature 2, includingupgrading to Log Analysis V1.3.3 Fix Pack 1 . . . 63Upgrading to V2.2.0.1 Interim Feature 2 withoutupgrading Log Analysis . . . . . . . . . . 67

Upgrading the z/OS Insight Packs . . . . . . 69Using the manual method of upgrading . . . . 71

Updating the z/OS Log Forwarder configuration . . 73

Configuring the z/OS Log Forwarder 75Preparing to configure the z/OS Log Forwarder . . 76

Creating the z/OS Log Forwarder startprocedure . . . . . . . . . . . . . . 76Preparing the z/OS NetView message providerfor use . . . . . . . . . . . . . . . 77Creating the z/OS SMF real-time data providerstart procedure . . . . . . . . . . . . 78

Preparing the Log Forwarder ConfigurationAssistant for use. . . . . . . . . . . . . 78

Preparing the Log Forwarder ConfigurationAssistant for sysplex-wide configuration. . . . 78Customizing the Log Forwarder ConfigurationAssistant start program . . . . . . . . . 79

Starting the Log Forwarder Configuration Assistant 80Defining the z/OS Log Forwarder environment . . 80Defining the sources of log data . . . . . . . 81

Correlation of the data to be analyzed with theassociated data gatherer types, data source types,and dashboards . . . . . . . . . . . . 82Data configuration file overview . . . . . . 85Manually configuring the z/OS Log Forwarderto forward data . . . . . . . . . . . . 89Encrypting passwords in the data configurationfile . . . . . . . . . . . . . . . . 94Verifying file path values for rolling logs . . . 95Defining multiple data gatherers in a single z/OSJob Log gatherer definition . . . . . . . . 96Loading historical data in batch mode foranalysis. . . . . . . . . . . . . . . 98

Verifying the identity of the target server forforwarding data . . . . . . . . . . . . . 99

Preparing to analyze z/OS log data 103Starting and stopping the z/OS Log Forwarder . . 103Starting and stopping the NetView messageprovider . . . . . . . . . . . . . . . 104Starting and stopping the SMF real-time dataprovider . . . . . . . . . . . . . . . 104Logging in to Log Analysis . . . . . . . . . 104

Use of cookies in the Log Analysis UI . . . . 105Grouping data sources to optimize troubleshootingin your IT environment . . . . . . . . . . 105Extending troubleshooting capability with CustomSearch Dashboard applications . . . . . . . 106

Customizing the dashboard applications . . . 106Running the dashboard applications . . . . . 111

Getting started with Problem Insights for z/OS . . 112Getting started with client-side Expert Advice . . 112

© Copyright IBM Corp. 2014, 2016 iii

Troubleshooting Operations Analyticsfor z Systems . . . . . . . . . . . 115Log files . . . . . . . . . . . . . . . 115Enabling tracing . . . . . . . . . . . . 115

Enabling static tracing for the z/OS LogForwarder . . . . . . . . . . . . . 116Enabling dynamic tracing for the z/OS LogForwarder . . . . . . . . . . . . . 116

z/OS Log Forwarder user ID has insufficientauthority . . . . . . . . . . . . . . . 118BPX messages precede GLA messages in the z/OSSYSLOG . . . . . . . . . . . . . . . 118Cannot shut down the z/OS Log Forwarder byusing the STOP command . . . . . . . . . 119When z/OS Log Forwarder starts, configurationchanges do not seem to be implemented . . . . 120APPLID values for CICS Transaction Server mightnot be correct in Log Analysis user interface . . . 121DB2 or MQ command prefix values might not becorrect in Log Analysis user interface . . . . . 121Log record skipped and not available in LogAnalysis . . . . . . . . . . . . . . . 122z/OS log data is missing in Log Analysis searchresults . . . . . . . . . . . . . . . . 122Cannot establish a secure connection between thez/OS Log Forwarder and the Logstash or loadbalancer server . . . . . . . . . . . . . 125NetView for z/OS problems . . . . . . . . 126

z/OS Log Forwarder message states that PPIissued return code 24. . . . . . . . . . 126NetView message provider GLANETV issuesmessage GLAL004E with return code 15 . . . 126NetView message provider GLANETV issuesmessage GLAL006E . . . . . . . . . . 127

SMF problems . . . . . . . . . . . . . 127SMF data is not showing in Log Analysis . . . 127SMF message explanations . . . . . . . . 128

Reference . . . . . . . . . . . . . 131Command reference for the z/OS Log Forwarder 131

DISPLAY command for data gatherers . . . . 131START command for data gatherers . . . . . 132STOP command for data gatherers . . . . . 133

Dashboards . . . . . . . . . . . . . . 135WebSphere Application Server for z/OS CustomSearch Dashboard applications . . . . . . 135z/OS Network Custom Search Dashboardapplications . . . . . . . . . . . . . 135z/OS SMF Custom Search Dashboardapplications . . . . . . . . . . . . . 136z/OS SYSLOG Custom Search Dashboardapplications . . . . . . . . . . . . . 140

WebSphere Application Server for z/OS datasource types . . . . . . . . . . . . . . 145

zOS-WAS-SYSOUT data source type . . . . . . 145zOS-WAS-SYSPRINT data source type . . . . . 148zOS-WAS-HPEL data source type . . . . . . 150

z/OS Network data source types . . . . . . . 153zOS-NetView data source type . . . . . . . 154

z/OS SMF data source types . . . . . . . . 157zOS-SMF30-Annotate annotations . . . . . . 157zOS-SMF80-Annotate annotations . . . . . . 158zOS-SMF110_E-Annotate annotations . . . . . 161zOS-SMF110_S_10-Annotate annotations . . . . 162zOS-SMF120-Annotate annotations. . . . . . 164

z/OS SYSLOG data source types . . . . . . . 168zOS-SYSLOG-Console data source type . . . . 169zOS-SYSLOG-SDSF data source type . . . . . 172zOS-syslogd data source type . . . . . . . 175zOS-CICS-MSGUSR data source type: threevariations . . . . . . . . . . . . . 177zOS-CICS-EYULOG data source type: threevariations . . . . . . . . . . . . . 178zOS-CICS-Annotate annotations . . . . . . 179

Property reference for the data configuration file 181System properties . . . . . . . . . . . 181Log receiver properties . . . . . . . . . 183Data gatherer properties . . . . . . . . . 186

Sample searches . . . . . . . . . . . . 200WebSphere Application Server for z/OS samplesearches . . . . . . . . . . . . . . 200z/OS Network sample searches . . . . . . 201z/OS SMF sample searches. . . . . . . . 202z/OS SYSLOG sample searches . . . . . . 204

SMF type 80-related records that are created by theSMF real-time data provider . . . . . . . . 208

SMF80_COMMAND record type . . . . . . . . 208SMF80_LOGON record type . . . . . . . . . 209SMF80_OMVS_RES record types . . . . . . . 210SMF80_OMVS_SEC record types . . . . . . . 210SMF80_OPERATION record type . . . . . . . 211SMF80_RESOURCE record type . . . . . . . 213

Variable reference for the environmentconfiguration file . . . . . . . . . . . . 214Variable reference for the z/OS Log Forwarderstart procedure . . . . . . . . . . . . . 216Variable reference for the z/OS SMF real-time dataprovider start procedure. . . . . . . . . . 216

Notices . . . . . . . . . . . . . . 219Terms and conditions for product documentation 220Trademarks . . . . . . . . . . . . . . 221

iv Operations Analytics for z Systems: PDF file of IBM Knowledge Center topics

What's new in Version 2.2.0

Review the new features that are available in Version 2.2.0 of IBM® OperationsAnalytics for z Systems to understand the potential benefits to your z/OS-based IToperations environment.v “V2.2.0.1 Interim Feature 2 (June 2016)”v “V2.2.0.1 Interim Feature 1 (March 2016)” on page 3v “V2.2.0.0 features (October 2015)” on page 3

V2.2.0.1 Interim Feature 2 (June 2016)

Table 1. New features in IBM Operations Analytics for z Systems V2.2.0.1 Interim Feature 2

Feature description More information

The Problem Insightsextension to IBM OperationsAnalytics - Log Analysisprovides real-time insight intoproblems in your ITenvironment, with suggestedactions to help resolve theproblems.

v “Extensions to Log Analysis for z/OS Problem Insightsand client-side Expert Advice” on page 13

v “z/OS Insight Pack requirements” on page 21

v “Installing Log Analysis” on page 53

v “Installing the z/OS Insight Packs and extensions” onpage 55

v “Getting started with Problem Insights for z/OS” onpage 112

The z/OS® Log Forwarder canbe configured to use thescalable data collectionarchitecture in IBMOperations Analytics - LogAnalysis to send data to theIBM Operations Analytics -Log Analysis server.

v “Operations Analytics for z Systems overview” on page9

v “Creation of data sources” on page 29

v “Scalable data collection architecture” on page 33

v “Data forwarding” on page 50

v “Log receiver properties” on page 183

v “Log Analysis receiver properties” on page 183

v “Logstash receiver properties” on page 185

v “Configuring the data destination” on page 93

Support for gathering datafrom a Virtual Storage AccessMethod (VSAM) cluster that isan entry-sequenced data set(ESDS)

v “Data gatherer properties” on page 186

v “z/OS Data Set gatherer properties” on page 190

v “Data collection from paired data sets” on page 50

© Copyright IBM Corp. 2014, 2016 1

Table 1. New features in IBM Operations Analytics for z Systems V2.2.0.1 Interim Feature2 (continued)


Support for gathering andanalyzing SMF record type120 subtype 9 data, which isrequest activity data that isproduced by WebSphere®

Application Server for z/OS

v “z/OS Insight Packs” on page 11

v “SMF real-time data provider” on page 39

v “Requirements for enabling the generation of SMF dataat the source” on page 41

v “Correlation of the data to be analyzed with theassociated data gatherer types, data source types, anddashboards” on page 82

v “z/OS SMF data source types” on page 157

v “zOS-SMF120-Annotate annotations” on page 164

v “z/OS SMF Custom Search Dashboard applications” onpage 136

v “z/OS UNIX Log File gatherer properties” on page 198

v “z/OS SMF sample searches” on page 202

The client-side Expert Adviceextension to IBM OperationsAnalytics - Log Analysis givesyou access to expert adviceeven if the IBM OperationsAnalytics - Log Analysisserver does not have access tothe Internet.

v “Extensions to Log Analysis for z/OS Problem Insightsand client-side Expert Advice” on page 13

v “z/OS Insight Pack requirements” on page 21

v “Installing Log Analysis” on page 53

v “Installing the z/OS Insight Packs and extensions” onpage 55

v “Getting started with client-side Expert Advice” on page112

New z/OS Log Forwardercommands for managing datagatherers without having tostop and restart the z/OS LogForwarder

v “DISPLAY command for data gatherers” on page 131

v “START command for data gatherers” on page 132

v “STOP command for data gatherers” on page 133

Capability to search for databased on sysplex or systemnames

v “zOS-WAS-SYSOUT-Annotate annotations” on page 146

v “zOS-WAS-SYSPRINT-Annotate annotations” on page 149

v “zOS-WAS-HPEL-Annotate annotations” on page 151

v “zOS-NetView-Annotate annotations” on page 155



v “zOS-SMF110_E-Annotate annotations” on page 161

v “zOS-SMF110_S_10-Annotate annotations” on page 162


v “zOS-SYSLOG-Console-Annotate annotations” on page170

v “zOS-SYSLOG-SDSF-Annotate annotations” on page 173

v “zOS-syslogd-Annotate annotations” on page 176

v “zOS-CICS-Annotate annotations” on page 179

2 Operations Analytics for z Systems: PDF file of IBM Knowledge Center topics

V2.2.0.1 Interim Feature 1 (March 2016)

Table 2. New features in IBM Operations Analytics for z Systems V2.2.0.1 Interim Feature 1


Support for gathering andanalyzing SMF record type 80data, which is producedduring Resource AccessControl Facility (RACF)processing

v “z/OS Insight Packs” on page 11


v “Requirements for enabling the generation of SMF dataat the source” on page 41







v “SMF type 80-related records that are created by theSMF real-time data provider” on page 208

V2.2.0.0 features (October 2015)

Table 3 indicates changes to some terminology.

Tip: If you are upgrading to IBM Operations Analytics for z Systems Version2.2.0.0, see “Updating the z/OS Log Forwarder configuration” on page 73 forinformation about how these terminology changes affect the z/OS Log Forwarderconfiguration.

Table 3. Terminology changes

Previous name New name

log gatherer data gatherer

log path file path

source type data source type

Table 4 on page 4 lists the new features and indicates where you can find moreinformation.

What's new in Version 2.2.0 3

Table 4. New features in IBM Operations Analytics for z Systems V2.2.0.0


Support for gathering andanalyzing NetView® for z/OSmessages


– “z/OS Insight Packs” on page 11

– “z/OS Log Forwarder” on page 15

v “z/OS Log Forwarder requirements” on page 22

v “Data sources” on page 29

– “Time zone information for z/OS log records” onpage 31

v “NetView message provider” on page 45

v “Defining the sources of log data” on page 81

v “z/OS Network data source types” on page 153

– “zOS-NetView data source type” on page 154

- “zOS-NetView-Annotate annotations” on page 155

v “z/OS Network Custom Search Dashboard applications”on page 135

v “z/OS Network sample searches” on page 201


– “z/OS NetView Message gatherer properties” onpage 195

Support for gathering andanalyzing the following SMFrecord type 110 data for CICS®

Transaction Server for z/OS:

v Global transaction managerstatistics data

v Monitoring exceptions data








– “zOS-SMF110_E-Annotate annotations” on page 161

– “zOS-SMF110_S_10-Annotate annotations” on page 162




Domain insight for z/OSsecurity

A new dashboard and samplesearches are available forResource Access ControlFacility (RACF®) messages inthe z/OS SYSLOG. The newdashboard can also highlightother intrusion-relatedindicators.


v “Security for z/OS Dashboard” on page 144

v “Security for z/OS sample searches” on page 206


Table 4. New features in IBM Operations Analytics for z Systems V2.2.0.0 (continued)


Support for gathering andanalyzing CICS TransactionServer for z/OS EYULOGdata

In the z/OS SYSLOG Insight Pack, three new data sourcetypes are provided for EYULOG data.

The log record annotator zOS-CICS-Annotate annotates thelog records for both EYULOG and MSGUSR log data. Inearlier releases of IBM Operations Analytics for z Systems,zOS-CICS-Annotate was named zOS-CICS-MSGUSR-Annotate.

The following topics provide more information:







v “z/OS SYSLOG data source types” on page 168

– “zOS-CICS-EYULOG data source type: three variations”on page 178


– “z/OS Job Log gatherer properties” on page 192

– “z/OS UNIX Log File gatherer properties” on page198

New DB2® for z/OS samplesearches

v “DB2 for z/OS sample searches” on page 204

If you collect data from az/OS job log, you can nowuse wildcard characters in thejob name to define multipledata gatherers in a singlez/OS Job Log gathererdefinition.

v “Defining multiple data gatherers in a single z/OS JobLog gatherer definition” on page 96

v “z/OS Job Log gatherer properties” on page 192

There is no longer arequirement to set the Java™

stack size on the IBMOperations Analytics - LogAnalysis server to 6144 KB.

Not applicable

What's new in Version 2.2.0 5

Conventions used in this information

This information describes conventions that are used in the IBM OperationsAnalytics for z Systems documentation.

LA_INSTALL_DIR directory

LA_INSTALL_DIR is the directory where IBM Operations Analytics - Log Analysis isinstalled.

The default location for installing Log Analysis is the /home/user/IBM/LogAnalysisdirectory, where user represents the user ID of the non-root user who installs LogAnalysis.


Operations Analytics for z Systems overview

IBM Operations Analytics for z Systems extends the capabilities of IBM OperationsAnalytics - Log Analysis to help you more quickly identify, isolate, and resolveproblems in a z/OS-based IT operations environment.

IBM Operations Analytics for z Systems provides the following capabilities:v Capability to collect z/OS logs across the System z® enterprise and to forward

them to the IBM Operations Analytics - Log Analysis server for analysisv Capability to index, search, and analyze application, middleware, and

infrastructure log data across the System z enterprisev Capability to quickly search and visualize errors across thousands of log recordsv Expert advice that is based on linking search results to available troubleshooting

information, such as best practices and previously documented solutionsv Continuous streaming of z/OS logs

Relationship between IBM Operations Analytics for z Systemsand IBM Operations Analytics - Log Analysis

IBM Operations Analytics - Log Analysis is a cross-platform solution foridentifying and resolving problems in an IT operations environment.

In IBM Operations Analytics - Log Analysis, an Insight Pack is software thatextends the capabilities of IBM Operations Analytics - Log Analysis to providesupport for loading and analyzing data from sources that share commoncharacteristics. Examples include log sources for a specific operating system or fora specific application, such as IBM WebSphere Application Server.

IBM Operations Analytics for z Systems includes z/OS Insight Packs that extendIBM Operations Analytics - Log Analysis function to analyze the various types ofz/OS log data.

IBM Operations Analytics for z Systems also includes a version of IBM OperationsAnalytics - Log Analysis.

Flow of source data

Figure 1 on page 10 illustrates the flow of data among the following primarycomponents of IBM Operations Analytics for z Systems:v “z/OS Insight Packs” on page 11v “IBM Operations Analytics - Log Analysis” on page 12v “z/OS Log Forwarder” on page 15


The following steps describe the data flow among components, which is indicatedby arrows in the illustration:1. In each z/OS logical partition (LPAR), the z/OS Log Forwarder retrieves the

data from the respective source and forwards it to the IBM OperationsAnalytics - Log Analysis server. A source can be any of the following entities:v Any job logv Any z/OS UNIX log file or entry-sequenced Virtual Storage Access Method

(VSAM) cluster that meets the following criteria:– Is encoded in Extended Binary Coded Decimal Interchange Code

(EBCDIC)– Is record-based– Has a time stamp in each record

v z/OS SYSLOG consolev NetView for z/OS messagesv WebSphere Application Server for z/OS High Performance Extensible

Logging (HPEL) logsThe z/OS Log Forwarder forwards data to the IBM Operations Analytics - LogAnalysis server either directly or by using the scalable data collectionarchitecture in IBM Operations Analytics - Log Analysis. If the scalable datacollection architecture is used, the z/OS Log Forwarder forwards the data to aLogstash server, and another Logstash server forwards the data to the IBMOperations Analytics - Log Analysis server.

2. The source data is processed by the respective z/OS Insight Pack. Insight isprovided for data from the following source types:v z/OS system log (SYSLOG)v CICS Transaction Server for z/OS EYULOG or MSGUSR log datav Network data, such as data from UNIX System Services system log (syslogd)

or z/OS Communications Serverv NetView for z/OS message data

UserInterface

GenericReceiver

Log Analysis server

z/OS LPAR 1

z/OS LogForwarder

z/OS LogForwarder

z/OS LPAR 2

321 z/OS

Insight Packs

Log

Log

Log

Log

Log

Log

Figure 1. Flow of source data among IBM Operations Analytics for z Systems components


v SMF datav WebSphere Application Server for z/OS logs that include SYSOUT,

SYSPRINT, or HPEL log data

Tip: Although IBM Operations Analytics for z Systems provides support forgathering data from a VSAM cluster that is an entry-sequenced data set (ESDS),no z/OS Insight Packs are provided for analyzing data from VSAM data sets.You can create your own insight pack for this purpose, or use a generic insightpack that is provided by IBM Operations Analytics - Log Analysis.

3. The user can analyze the data that is shown in the IBM Operations Analytics -Log Analysis user interface.

z/OS Insight PacksEach z/OS Insight Pack in IBM Operations Analytics for z Systems extends IBMOperations Analytics - Log Analysis function to analyze a unique type of z/OS logdata.

The following z/OS Insight Packs are available:

WebSphere Application Server for z/OS Insight PackThis Insight Pack enables Log Analysis to ingest, and perform searchesagainst, logging data that is retrieved from IBM WebSphere ApplicationServer for z/OS.

Summary: Install this Insight Pack if you want to analyze WebSphereApplication Server for z/OS log data.

z/OS Network Insight PackThis Insight Pack enables Log Analysis to perform searches against z/OSnetwork data. It also enables Log Analysis to ingest, and perform searchesagainst, IBM Tivoli® NetView for z/OS message data.

To analyze z/OS network data, you must use both the z/OS NetworkInsight Pack and the z/OS SYSLOG Insight Pack. The z/OS SYSLOGInsight Pack enables Log Analysis to ingest z/OS network data.

Summary: Install this Insight Pack if you want to analyze any of thefollowing data:v Network data, such as data from UNIX System Services system log

(syslogd) or z/OS Communications Serverv NetView for z/OS message data

z/OS SMF Insight PackThis Insight Pack enables Log Analysis to ingest, and perform searchesagainst, data that is retrieved from IBM z/OS System ManagementFacilities (SMF).

Summary: Install this Insight Pack if you want to analyze SMF data.

z/OS SYSLOG Insight PackThis Insight Pack enables Log Analysis to ingest, and perform searchesagainst, data that is retrieved from the following sources:v z/OS SYSLOG console, which includes data from the following software:

– IBM CICS Transaction Server for z/OS– IBM DB2 for z/OS

Operations Analytics for z Systems overview 11

– IBM IMS™ for z/OS– IBM MQ for z/OS– IBM Resource Access Control Facility (RACF)– z/OS Communications Server

v CICS Transaction Server for z/OS EYULOG and MSGUSR logv UNIX System Services system log (syslogd)

Summary: Install this Insight Pack if you want to analyze any of thefollowing data:v z/OS SYSLOG console datav CICS Transaction Server for z/OS EYULOG or MSGUSR log datav UNIX System Services system log (syslogd)v SMF data

Table 5 lists each z/OS Insight Pack with its associated installation package file.

Table 5. Installation package files for z/OS Insight Packs

Insight Pack Installation package

WebSphere Application Server for z/OS Insight Pack WASforzOSInsightPack_v2.2.0.2.zip

z/OS Network Insight PackImportant: To analyze network data, install both thez/OS SYSLOG Insight Pack and this insight pack.

zOSNetworkInsightPack_v1.1.0.2.zip

z/OS SMF Insight PackImportant: To analyze SMF data, install both the z/OSSYSLOG Insight Pack and this insight pack.

SMFforzOSInsightPack_v1.1.0.3.zip

z/OS SYSLOG Insight Pack SYSLOGforzOSInsightPack_v2.2.0.2.zip

IBM Operations Analytics - Log AnalysisIBM Operations Analytics - Log Analysis is used with the z/OS Log Forwarderand the respective z/OS Insight Pack to provide support for z/OS log data.

IBM Operations Analytics - Log Analysis includes the following functions that youcan configure in the Log Analysis UI:

Role-based access controlYou can create and modify users and roles to assign role-based accesscontrol to individual users.

For more information, see Users and roles in the Log Analysisdocumentation.

AlertingYou can create alerts that are based on events, and you can define theactions for the system to take when an alert is triggered. For example, youcan define an action to send an email notification to one or more peoplewhen an alert is triggered.

For more information, see Managing alerts in the Log Analysisdocumentation.


http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/config/iwa_config_pinstall_userrole_ovw_c.html

http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/config/iwa_config_alerting_c.html

Some components that are preinstalled with Log Analysis

Although many components are preinstalled with IBM Operations Analytics - LogAnalysis, the following components might be especially useful with your z/OSInsight Packs:

Expert advice custom search dashboardThe Expert Advice provides links to contextually relevant information tohelp you resolve problems quickly. Using this application, you can selectany column or cells in the Grid view and can launch a search of the IBMSupport Portal. The application searches for matches to unique terms thatare contained in the column that you select.

You can start the Expert Advice application by clicking Search Dashboardsin the navigation pane of the Search workspace.

For more information about this component, see Custom SearchDashboards in the Log Analysis documentation.

Tip: To use this Expert Advice, the Log Analysis server must have accessto the Internet. With the client-side Expert Advice extension that isprovided by IBM Operations Analytics for z Systems, you can accessexpert advice even if the Log Analysis server does not have access to theInternet. For more information about the client-side Expert Advice, see“Extensions to Log Analysis for z/OS Problem Insights and client-sideExpert Advice.”

WebSphere Application Server Insight PackThis Insight Pack is different from the WebSphere Application Server forz/OS Insight Pack. It is intended for use with WebSphere ApplicationServer on distributed platforms. It does not support native logging formatsfor WebSphere Application Server for z/OS. However, if you configureyour WebSphere Application Server for z/OS environment to use adistributed logging format, this Insight Pack provides you with therequired annotation and indexing capabilities.

For more information about this component, see WebSphere ApplicationServer Insight Pack in the Log Analysis documentation.

Extensions to Log Analysis for z/OS Problem Insights and client-sideExpert Advice

IBM Operations Analytics for z Systems V2.2.0.1 Interim Feature 2 providesextensions to IBM Operations Analytics - Log Analysis for z/OS Problem Insightsand client-side Expert Advice. To use these extensions, you must have IBMOperations Analytics - Log Analysis Version 1.3.3 Fix Pack 1 installed.

Problem Insights

The Problem Insights extension provides real-time insight into problems in your ITenvironment, with suggested actions to help resolve the problems.

If the Problem Insights component of IBM Operations Analytics for z Systems isinstalled, the Log Analysis UI includes a new tab that is titled Problem Insights.For each sysplex from which data is being forwarded to the Log Analysis server,the Problem Insights page includes insight about certain problems that areidentified in the ingested data.


http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/use/iwa_using_customapplications.html

http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/use/iwa_using_customapplications.html

http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/extend/iwa_extend_was_ovw.html

http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/extend/iwa_extend_was_ovw.html

You can select the time range for which you want to see insights. For example, ifyou want to know about certain problems that were identified in the last hour, youcan select the last hour as the time range. The default time range is the last 15minutes.

You can also reload the Problem Insights page and associated data by clicking theRefresh button that is located in the area of the UI where you select the timerange.

Each sysplex in the monitored environment is represented by a button thatindicates the number of problems that are found in that sysplex. Click a sysplexbutton to show or hide the associated sysplex data in the Problem Insights andSuggested Actions table. You can show the data for multiple sysplexes in the tableat the same time, and you can toggle the showing or hiding of data in the table byclicking the sysplex buttons.

The table includes the following information about each problem that isdiscovered:

SeverityThe indication of the severity of the problem.

SysplexThe sysplex where the problem occurred.

SystemThe system where the problem occurred.

SubsystemThe subsystem or system resources manager where the problem occurred.

Time The last time that the problem occurred in the selected time range. Forexample, if you select the last 15 minutes as the time range, this columnshows the last time that the problem occurred in the last 15 minutes.

Problem SummaryA summary of the problem that provides insight into the cause.

Count The total number of occurrences of the problem in the selected time range.

Suggested ActionsAn extension of the problem summary that provides more insight aboutthe problem, includes one or more actions that you can take to resolve theproblem, and links to other sources of information that might help youresolve the problem, such as relevant topics in the IBM Knowledge Center.

EvidenceThe message number of the message that identifies the problem, whichyou can click to view more information.

Client-side Expert Advice

After you install the IBM Operations Analytics for z Systems extensions, thefollowing applications are shown under Expert Advice in the Custom SearchDashboards panel of the left navigation pane of the Search workspace:v IBMSupportPortal-ExpertAdvice, which is the default Expert Advice in IBM

Operations Analytics - Log Analysis. When this Expert Advice is launched, theLog Analysis server sends search requests to the IBM Support Portal anddisplays the query search results in the Log Analysis UI.


v IBMSupportPortal-ExpertAdvice on Client, which is the extension forclient-side Expert Advice that is provided by IBM Operations Analytics for zSystems. When this Expert Advice is launched, the client browser sends searchrequests directly to the IBM Support Portal and opens a new browser tab todisplay the query search results.

z/OS Log ForwarderThe z/OS Log Forwarder can automatically monitor and forward z/OS data to theIBM Operations Analytics - Log Analysis server.

In this context, z/OS data includes any of the following data:v z/OS SYSLOG console datav NetView for z/OS message datav SMF datav Data from any job log or UNIX System Services log, including the following

logs:– CICS Transaction Server for z/OS EYULOG and MSGUSR log– UNIX System Services system log (syslogd)– WebSphere Application Server for z/OS logs

v Data from any entry-sequenced data set (ESDS) that is a Virtual Storage AccessMethod (VSAM) cluster.

Each logical partition (LPAR) must have its own instance of the z/OS LogForwarder.

In each LPAR, you must configure the z/OS Log Forwarder to specify which datais to be forwarded to the Log Analysis server.

In each LPAR where you want to collect the following data, you must also installor configure the respective mechanism for providing this data to the z/OS LogForwarder:

NetView for z/OS message dataYou must configure the NetView message provider.

For more information, see “NetView message provider” on page 45.

SMF dataYou must configure the SMF real-time data provider.

For more information, see “SMF real-time data provider” on page 39.

z/OS SYSLOG dataYou must install either the GLASYSG or GLAMDBG user exit.

For more information about the user exits, see “User exits for collectingz/OS SYSLOG data” on page 36.

Installation and configuration checklistsTo help you get started, these installation and configuration checklists summarizethe important steps in installing and configuring the Operations Analytics for zSystems components.

Before you can use Operations Analytics for z Systems to analyze log data, youmust complete the following tasks:


__ 1. Install IBM Operations Analytics - Log Analysis Version 1.3.0, 1.3.1, 1.3.2, or1.3.3.See the “Log Analysis checklist.”

__ 2. On the Log Analysis server, install the z/OS Insight Packs that you want touse.See the “z/OS Insight Packs and extensions checklist” on page 17.

__ 3. Install the z/OS Log Forwarder in each z/OS logical partition (LPAR) forwhich you want to process z/OS log messages.__ a. In each LPAR, configure the z/OS Log Forwarder to collect and

forward z/OS log data to the Log Analysis server.See the “z/OS Log Forwarder checklist” on page 17.

__ 4. Ensure that z/OS data sources are configured and grouped appropriately inLog Analysis.See “Preparing to analyze z/OS log data” on page 103.

Log Analysis checklistThis checklist summarizes the important steps in installing IBM OperationsAnalytics - Log Analysis.

Preparing to install Log Analysis

Complete the following steps before you install IBM Operations Analytics - LogAnalysis:__ 1. If you do not have a previously installed instance of Log Analysis Version

1.3.0, 1.3.1, 1.3.2, or 1.3.3, plan to install both Log Analysis Version 1.3.3Standard Edition and Fix Pack 1 of Version 1.3.3.

More information:

v “z/OS Insight Pack requirements” on page 21__ 2. Verify that the system requirements for Log Analysis, including the

installation of the prerequisite software, are met.

More information:

v Hardware and software requirements__ 3. Using the ulimit command, tune the Linux operating system so that the

number of concurrent files is 4096, and the virtual memory is unlimited.

More information:

v Hardware and software requirements__ 4. To install and run Log Analysis, you must be logged in to the Linux

computer system with a non-root user ID. Either create this non-root userID, or use an existing non-root user ID.

More information:

v “Planning for installation of Log Analysis” on page 24__ 5. Review the network port assignments for Log Analysis. If the default ports

are not available, determine alternative port assignments.

More information:

v Default ports__ 6. Decide whether you want to use the scalable data collection architecture.


https://developer.ibm.com/itoa/docs/log-analysis/documentation/hardware-software-requirements/


http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/install/iwa_defaultports_r.html

More information:

v “Scalable data collection architecture” on page 33__ 7. The IBM Tivoli Monitoring Log File Agent is an optional component that is

provided with Log Analysis and other IBM products. This component is notrequired for analyzing z/OS log data. Decide whether you want to installthis agent as part of the Log Analysis installation.

More information:

v Installing and configuring the IBM Tivoli Monitoring Log File Agent

Installing Log Analysis__ Verify that you are logged in to the Linux computer system with the

non-root user ID, and complete the steps in “Installing Log Analysis” onpage 53.

z/OS Insight Packs and extensions checklistThis checklist summarizes the important steps in installing z/OS Insight Packs andextensions.

Installing z/OS Insight Packs and extensions__ 1. Verify that Log Analysis is running.

More information:

v “Installing the z/OS Insight Packs and extensions” on page 55__ 2. Verify that you are logged in to the Linux computer system with the

non-root user ID that was used to install Log Analysis.

More information:

v “Installing the z/OS Insight Packs and extensions” on page 55__ 3. Install the z/OS Insight Packs with sample searches and the extensions for

Problem Insights and client-side Expert Advice by using the self-extractinginstaller file.

More information:

v “Installing the z/OS Insight Packs and extensions” on page 55

z/OS Log Forwarder checklistThis checklist summarizes the important steps in installing and configuring thez/OS Log Forwarder.

The following terms are used in the context of the z/OS Log Forwarderinstallation:

driving systemThe system on which you run SMP/E for z/OS (SMP/E) to install thez/OS Log Forwarder.

target systemThe system on which you configure and run the z/OS Log Forwarder.

Installing the z/OS Log Forwarder__ 1. Verify that the software requirements for the driving system are met.__ 2. To install the z/OS Log Forwarder, use SMP/E for z/OS (SMP/E).


http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/install/iwa_install_config_lfa_c.html

Assumption: The SMP/E relative files (RELFILES) for the z/OS Log Forwarderare available on the driving system.

For more information, see the IBM Operations Analytics for z Systems ProgramDirectory.

Preparing to configure the z/OS Log Forwarder

On each target system, complete the following steps before you configure the z/OSLog Forwarder:__ 1. Verify that the software requirements for each target system are met.

More information:

v “z/OS Log Forwarder requirements” on page 22__ 2. Verify that each target system has a properly configured TCP/IP stack.

More information:

v “Requirements for running the z/OS Log Forwarder” on page 23__ 3. Verify that each target system can communicate with the Log Analysis

server on the port that is defined during the Log Analysis installation.

More information:

v “Planning for installation of Log Analysis” on page 24__ 4. Determine the appropriate location for the z/OS Log Forwarder

configuration files (the environment configuration file and the dataconfiguration file).

More information:

v “z/OS Log Forwarder program files” on page 26__ 5. Determine which user ID to use for running the z/OS Log Forwarder, and

give that user ID the required authorities for file access and for issuingconsole messages.

More information:

v “Required authorities for z/OS Log Forwarder operations” on page 27__ 6. Determine the sources from which you want to collect log data.

More information:

v “Data sources” on page 29v “Data collection” on page 35

__ 7. If you want to collect z/OS SYSLOG data, install either the GLASYSG orGLAMDBG user exit for providing this data to the z/OS Log Forwarder.

More information:

v “User exits for collecting z/OS SYSLOG data” on page 36__ 8. If you want to collect NetView for z/OS message data, associate the

NetView message provider with a NetView autotask that has the requiredpermissions.

More information:

v “NetView message provider” on page 45


__ 9. If you want to collect System Management Facilities (SMF) data, completethe following steps:__ a. Verify the following items:

__ 1) On each target system, verify that SMF is active.__ 2) On each target system, verify that SMF is configured to use log

streams and to collect SMF data.__ 3) Verify that each target system has a hierarchical file system

(HFS) with enough available space to contain the z/OS SMFreal-time data provider temporary files.

__ b. Determine which user ID to use for running the z/OS SMF real-timedata provider, and give that user ID the required authorities for SMFlog stream and HFS file access.

More information:


Configuring the z/OS Log Forwarder to collect data__ 1. Create the z/OS Log Forwarder start procedure by updating the sample

procedure GLAPROC.

More information:

v “Creating the z/OS Log Forwarder start procedure” on page 76__ 2. If you are collecting NetView for z/OS message data, prepare the z/OS

NetView message provider for use.

More information:

v “Preparing the z/OS NetView message provider for use” on page 77__ 3. If you are collecting SMF data, create the z/OS SMF real-time data provider

start procedure by updating the sample procedure GLASMF.

More information:

v “Creating the z/OS SMF real-time data provider start procedure” on page78

__ 4. If necessary, customize the start program for the Log ForwarderConfiguration Assistant. For example, you might want to use thesysplex-wide configuration option or to customize variable values in theenvironment configuration file.

More information:

v “Preparing the Log Forwarder Configuration Assistant for use” on page78

__ 5. Start the Log Forwarder Configuration Assistant to define the z/OS LogForwarder environment (environment configuration file) and data gatherers(data configuration file).

Tips:

v Leave the value of the z/OS host name blank so that the z/OS LogForwarder can discover this host name.

v If you are not using the scalable data collection architecture, let the z/OSLog Forwarder automatically create data sources in Log Analysis.


v To prevent unauthorized use of the passwords that are stored in the z/OSLog Forwarder data configuration file, encrypt the passwords. See“Encrypting passwords in the data configuration file” on page 94.

More information:

v “Starting the Log Forwarder Configuration Assistant” on page 80__ 6. If you are not using the scalable data collection architecture, you might want

to secure communication between the z/OS Log Forwarder and the LogAnalysis server.

More information:

v “Verifying the identity of the target server for forwarding data” on page99


Planning for installation and configuration of OperationsAnalytics for z Systems

In planning for the installation of IBM Operations Analytics for z Systems, youmust also plan for the installation of IBM Operations Analytics - Log Analysis andthe installation and configuration of the z/OS Log Forwarder.

System requirementsEnsure that your environment meets the system requirements for IBM OperationsAnalytics for z Systems, which include the system requirements for IBMOperations Analytics - Log Analysis and the z/OS Log Forwarder.

Log Analysis requirements

For information about the system requirements for Log Analysis, see Hardwareand software requirements.

Restriction: If you deploy Log Analysis on Linux for System z, you can use theIBM InfoSphere® BigInsights® Hadoop Version 3.0 service, but at this time, theHadoop Distributed File System (HDFS) must be installed on an x86 Linux system.The support for integrating Log Analysis on Linux for System z with otherHadoop distributions is dependent on the platform support that is provided by therespective Hadoop distribution.

z/OS Insight Pack requirementsYou must install the z/OS Insight Packs on the IBM Operations Analytics - LogAnalysis server.

Hardware requirements

For the Log Analysis hardware requirements, see Hardware and softwarerequirements.

Software requirements

The z/OS Insight Packs must be installed on a Log Analysis server with LogAnalysis Version 1.3.0, 1.3.1, 1.3.2, or 1.3.3. The IBM Operations Analytics for zSystems Version 2.2.0.0 package includes IBM Operations Analytics - Log AnalysisVersion 1.3.1 Standard Edition. The IBM Operations Analytics for z SystemsVersion 2.2.0.1 Interim Feature 2 package includes IBM Operations Analytics - LogAnalysis Version 1.3.3 Standard Edition.

Tip: To use the Problem Insights and client-side Expert Advice extensions that areavailable in IBM Operations Analytics for z Systems Version 2.2.0.1 Interim Feature2, you must use Log Analysis Version 1.3.3 Fix Pack 1. To get Log Analysis Version1.3.3 Fix Pack 1, complete the following steps:1. Go to IBM Fix Central.2. In the Product selector field, start typing IBM Operations Analytics - Log

Analysis, and when the correct product name is shown in the resulting list,select it. More fields are then shown.






https://www.ibm.com/support/fixcentral/

3. In the Installed Version field, select 1.3.3.4. In the Platform field, select All.5. Click Continue.6. In the resulting “Identify fixes” window, select Browse for fixes, and click

Continue.7. In the “Select fixes” window, you should see the fix pack 1.3.3-TIV-IOALA-

FP001, which you can select and download. For installation instructions, see thereadme file.

Table 6 indicates the software versions that are supported by each z/OS InsightPack.

Table 6. Software versions that are supported by each z/OS Insight Pack

Insight Pack Supports data ingestion from this software

z/OS Network Insight Pack v z/OS 1.13, 2.1, and 2.2

v NetView for z/OS 6.1, 6.2, and 6.2.1

z/OS SMF Insight Pack v z/OS 1.13, 2.1, and 2.2

z/OS SYSLOG Insight Pack v z/OS 1.13, 2.1, and 2.2

v DB2 for z/OS 10.1 and 11.1

v CICS Transaction Server for z/OS 4.2, 5.1, and 5.2

v IMS for z/OS 11.1, 12.1, and 13.1

v MQ for z/OS 7.1 and 8.0

WebSphere Application Server forz/OS Insight Pack

WebSphere Application Server for z/OS 7.0, 8.0, and8.5.5

z/OS Log Forwarder requirementsYou must run the z/OS Log Forwarder in each z/OS logical partition (LPAR) forwhich you want to process z/OS log messages.

Hardware requirements

The z/OS Log Forwarder can run on any hardware that supports its operatingsystem and other software requirements.

Software requirements

The z/OS Log Forwarder requires the following software to be installed:v z/OS V1.13 or laterv z/OS Communications Serverv Resource Access Control Facility (RACF)v JES2 or JES3v IBM SDK for z/OS, Java Technology Edition, Version 6, 7, or 8. Both 31-bit and

64-bit versions are supported.

Important considerations:

– Use the latest available service release of the version of IBM SDK for z/OS,Java Technology Edition, that you choose, and apply fix packs as soon aspossible after they are released. To find the latest service release or fix pack,see IBM Java™ Standard Edition Products on z/OS.


http://www.ibm.com/systems/z/os/zos/tools/java/

– When the z/OS Log Forwarder runs on IBM SDK for z/OS, Java TechnologyEdition, Version 8, it can use the Transport Layer Security (TLS) Version 1.1and 1.2 protocols to secure communication with the IBM Operations Analytics- Log Analysis server. Log data traffic is then more secure than it is with theTLS Version 1 protocol. If the security of log data transmission is a concern,consider running the z/OS Log Forwarder on IBM SDK for z/OS, JavaTechnology Edition, Version 8.

– Although the z/OS Log Forwarder runs equally well on IBM SDK for z/OS,Java Technology Edition, Versions 6, 7 and 8, some tests indicate that CPUusage might increase when you run the z/OS Log Forwarder on Version 7. IfCPU usage of the z/OS Log Forwarder is a concern, consider running thez/OS Log Forwarder on IBM SDK for z/OS, Java Technology Edition, Version6 or Version 8.

Requirements for running the z/OS Log ForwarderBefore you start the z/OS Log Forwarder, you must verify that the z/OSenvironment is set up correctly so that the z/OS Log Forwarder can resolve hostnames and can access the TCP/IP resolver configuration file.

Host name resolution

To operate, the z/OS Log Forwarder must determine the fully qualified domainname (FQDN) of the system on which it is running. Therefore, the networking andname resolution services that are configured in the system for use by the z/OS LogForwarder should be activated before the z/OS Log Forwarder is started.

When networking and name resolution services are not activated: When thez/OS Log Forwarder is started automatically at IPL time, the initialization of theseservices might not be complete. In this situation, the z/OS Log Forwarder mightretrieve only a loopback address. The z/OS Log Forwarder then tries again every30 seconds to determine the FQDN until it successfully retrieves the local hostaddress.

Search order for the TCP/IP resolver configuration file

The z/OS Log Forwarder is a z/OS UNIX System Services program. It usesTCP/IP functions that require access to the TCP/IP resolver configuration file. Thisaccess is provided by using a resolver search order. The resolver search order forz/OS UNIX System Services programs is documented in the topic about resolverconfiguration files in the z/OS Communications Server: IP Configuration Guide.

The following list summarizes the resolver search order:1. GLOBALTCPIPDATA2. The RESOLVER_CONFIG environment variable in the z/OS Log Forwarder

environment configuration file3. The /etc/resolv.conf file4. The SYSTCPD DD statement in the z/OS Log Forwarder start procedure5. userid.TCPIP.DATA, where userid is the user ID that is associated with the z/OS

Log Forwarder start procedure6. SYS1.TCPPARMS(TCPDATA)7. DEFAULTTCPIPDATA8. TCPIP.TCPIP.DATA

Planning for installation and configuration 23

Verify that the resolver configuration file is available to the z/OS Log Forwarderby using one of the search order mechanisms.Related reference:“Variable reference for the environment configuration file” on page 214This reference lists the environment variables that you can update in the z/OS LogForwarder environment configuration file.

Planning for installation of Log AnalysisBefore you install IBM Operations Analytics - Log Analysis, review therequirements for the installation user ID, domain name resolution, and networkconnectivity between the z/OS Log Forwarder and the Log Analysis server. Also,decide whether you want to use the optional IBM Tivoli Monitoring Log FileAgent component.

Before you begin

Verify that your environment meets the system requirements that are described in“z/OS Insight Pack requirements” on page 21.

About this task

Installation user IDTo install and run Log Analysis, you must be logged in to the Linuxcomputer system with a non-root user ID.

Domain name resolutionOn the system where you plan to install Log Analysis, verify that thedetails for the Log Analysis server are maintained correctly in the/etc/hosts file.

Network connectivity between the z/OS Log Forwarder LPARs and the LogAnalysis server

The z/OS Log Forwarder LPARs must communicate with the Log Analysisserver on the port that is defined during Log Analysis installation.

The default value for this port is 9987. Ensure that communication on thisport is not blocked by a firewall or by other aspects of your networkconfiguration.

Optional Log File Agent componentThe IBM Tivoli Monitoring Log File Agent is an optional component that isprovided with Log Analysis and other IBM products. This component isnot required for analyzing z/OS log data.

For more information about installing the Log File Agent as part of the LogAnalysis installation, see Installing and configuring the IBM TivoliMonitoring Log File Agent.

Planning for Log Analysis to support LDAP interaction with RACF forsign-on authentication

You can configure IBM Operations Analytics - Log Analysis to support LDAPinteraction with RACF for sign-on authentication.

Procedure

You must plan for the following configuration steps:




1. In Log Analysis, verify that no data sources are created and that no permissionsare assigned to data sources.

2. In the Log Analysis UI, create at least one user ID for use in doingadministrative tasks in Log Analysis.

3. Define other user IDs for use in doing non-administrative tasks in LogAnalysis.

4. Because LDAP and RACF authenticate these user IDs during sign-on, ensurethat these user IDs are also defined to LDAP and RACF.

Tip: In this configuration with LDAP and RACF, Log Analysis cannot use thedefault user IDs unityadmin and unityuser due to RACF restrictions on thelength of a user ID (it must be 7 characters or less).

5. Configure Log Analysis to use LDAP for authentication.For instructions, see LDAP configuration.For z Systems LDAP support, choose the Tivoli Directory Server as the type ofLDAP server.

6. Save a backup copy of the following Log Analysis configuration files so thatyou can update these files.v LA_INSTALL_DIR/utilities/datacollector-client/

javaDatacollector.properties

v LA_INSTALL_DIR/remote_install_tool/config/rest-api.properties

v LA_INSTALL_DIR/UnityEIFReceiver/config/unity.conf

v LA_INSTALL_DIR/solr_install_tool/scripts/register_solr_instance.sh

7. Update the passwords in the four previously listed Log Analysis configurationfiles.For more information, see Updating passwords in the configuration files.

8. Map your LDAP groups to the Log Analysis security role so that the users canaccess Log Analysis.Individual users can also be granted user or administrative privileges.For more information, see LDAP configuration.

9. If you use the delete utility to prune data sources, provide a user ID andpassword that are known to RACF to this utility.You can encrypt the password if you choose to store it in the Log Analysisserver.

Planning for installation of the z/OS Log ForwarderYou install the z/OS Log Forwarder by using SMP/E for z/OS (SMP/E). SMP/E isan IBM licensed program that is used to install software and software changes onz/OS systems.

Before you begin

Verify that your environment meets the system requirements that are described in“z/OS Log Forwarder requirements” on page 22.

About this task

For the z/OS Log Forwarder installation instructions, see the Program Directoryfor IBM Operations Analytics for z Systems.


http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/install/iwa_install_ldapconfig_ovw.html

http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/install/iwa_pintsall_user_conf_update_t.html

http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/install/iwa_install_ldapconfig_ovw.html

Planning for configuration of the z/OS Log ForwarderThe configuration tasks for the z/OS Log Forwarder include defining the z/OSLog Forwarder environment and the sources of log data for data collection.

z/OS Log Forwarder program filesThe z/OS Log Forwarder program files include the installation files, theconfiguration files, and the files in the working directory.

Installation files

The z/OS Log Forwarder installation files are created during the installation of thez/OS Log Forwarder and are required for running the z/OS Log Forwarder.

Configuration files

The z/OS Log Forwarder configuration is defined in the following files:

environment configuration fileThis file specifies the operating environment for the z/OS Log Forwarder.

The base file name is zlf.conf.

data configuration fileThis file defines data gatherers, and the system that is to receive data fromthese gatherers, to the z/OS Log Forwarder.

The base file name is config.properties.

You customize these files as part of the z/OS Log Forwarder configuration. Sampleconfiguration files are stored in the following UNIX System Services directory:/usr/lpp/IBM/zscala/V2R2/samples

Determine where you want to store your customized configuration files. They mustbe stored in a UNIX System Services directory.

Files in the working directory

The z/OS Log Forwarder working directory contains the files that a z/OS LogForwarder instance uses in its operations. These files contain the followinginformation, for example:v The state of the z/OS Log Forwarder, or its progress in collecting log datav The key to use for decrypting passwords

For each z/OS Log Forwarder instance, you must assign a working directory forthe storage of these files. The working directory must be in a different physicallocation from the working directory for any other z/OS Log Forwarder instance.

To enable each z/OS Log Forwarder instance to access the correct workingdirectory, you must specify the appropriate value for the ZLF_WORK environmentvariable in the environment configuration file.

The following UNIX System Services directory is the default working directory:/etc/IBM/zscala/V2R2


Determine which directory to assign as the z/OS Log Forwarder working directoryfor your environment. The working directory must be a UNIX System Servicesdirectory.

Attention: Do not update, delete, or move the files in the working directory.

Sharing of program files among multiple z/OS Log Forwarderinstances

The UNIX System Services file system or systems that contain the z/OS LogForwarder program files (installation files, configuration files, and files in theworking directory) can be shared among multiple instances of the z/OS LogForwarder.

If a single directory contains the configuration files for more than one system, orlogical partition (LPAR), each configuration file name must include the names ofthe sysplex and the system (LPAR) to which the file applies. The file names mustuse the following conventions, where SYSNAME is the name of the system (LPAR)where the z/OS Log Forwarder runs, and SYSPLEX is the name of the sysplex (ormonoplex) in which that system is located. The values of both SYSPLEX andSYSNAME must be in all uppercase.

environment configuration fileSYSPLEX.SYSNAME.zlf.conf

data configuration fileSYSPLEX.SYSNAME.config.properties

If one file system contains the working directories for multiple instances of thez/OS Log Forwarder, the working directory for each instance must be uniquelynamed. Also, for each instance, the appropriate value must be specified for theZLF_WORK environment variable in the environment configuration file.

Required authorities for z/OS Log Forwarder operationsThe user ID that is associated with the z/OS Log Forwarder start procedure musthave the required authorities for file access and for issuing console messages.

Tip: The z/OS Log Forwarder user ID does not require any special MVS™

authority to run the z/OS Log Forwarder.

File access authority

The z/OS Log Forwarder user ID must have the appropriate authority to accessthe z/OS Log Forwarder program files, which include the installation files, theconfiguration files, and the files in the working directory.

Installation file accessThe z/OS Log Forwarder user ID must have read and execute permissionsto the z/OS Log Forwarder installation files in the UNIX System Servicesfile system.

Configuration file accessThe z/OS Log Forwarder user ID must have read permission to the z/OSLog Forwarder configuration files in the UNIX System Services file system.

Important: The user ID that configures the z/OS Log Forwarder musthave read/write permission to the configuration files.


Working directory accessThe z/OS Log Forwarder user ID must have read and write permissionsto the z/OS Log Forwarder working directory.

Authority to issue console messages

The z/OS Log Forwarder user ID must have the authority to issue consolemessages.

If you are using RACF as your SAF product, for example, either use the GLARACFsample that is provided in the SGLASAMP data set, or complete the followingsteps to assign this authority:1. In RACF, add the BPX.CONSOLE resource to the class FACILITY by using the

General Resource Profiles option in the RACF Services Option Menu.2. In the BPX.CONSOLE profile that was created (or updated) in the preceding step,

add the user ID that the z/OS Log Forwarder start procedure is associatedwith, and assign READ access to the user ID.

3. Issue the following command to activate your changes:SETROPTS RACLIST(FACILITY) REFRESH

z/OS Log Forwarder start procedureAfter you install the z/OS Log Forwarder, you must create the procedure to startthe z/OS Log Forwarder.

GLAPROC procedure in the sample library SGLASAMP

Create the z/OS Log Forwarder start procedure by using the GLAPROCprocedure, which is provided in the sample library SGLASAMP.

GLARACF procedure in the sample library SGLASAMP

For environments where the System Authorization Facility (SAF) product is theResource Access Control Facility (RACF), the sample library SGLASAMP includesthe GLARACF procedure. You can use the GLARACF procedure to create a userID for the GLAPROC procedure and to associate that user ID with the GLAPROCprocedure. The user ID that the GLARACF procedure creates is named GLALGF.

Tips:

v The GLAPROC procedure does not require the user ID to be GLALGF. This userID is provided only as a convenience.

v If the SAF product for your environment is not RACF, use the GLARACFsample procedure and the SAF product documentation to create the appropriatedefinitions in the SAF product.

Authorization with APFThe Operations Analytics for z Systems SGLALINK library must be authorizedwith the z/OS authorized program facility (APF).

To be authorized, a library name and volume ID must be in the list of authorizedlibraries in the PROGxx member of the SYS1.PARMLIB library.

To include the SGLALINK library in APF at system IPL, add the followingstatement to a PROGxx member:


APF ADD DSNAME(ZSCALA.V2R2M0.SGLALINK ) VOLUME(volname)

To dynamically add the SGLALINK library to APF after IPL, issue the followingMVS command:SETPROG APF,ADD,DSNAME=ZSCALA.V2R2M0.SGLALINK,VOLUME=volname

Data sourcesYou must determine the sources from which you want to collect data.

You can collect data from the following sources:v Any job logv Any z/OS UNIX log file or entry-sequenced Virtual Storage Access Method

(VSAM) cluster that meets the following criteria:– Is encoded in Extended Binary Coded Decimal Interchange Code (EBCDIC)– Is record-based– Has a time stamp in each record

v z/OS SYSLOG consolev NetView for z/OS messagesv WebSphere Application Server for z/OS High Performance Extensible Logging

(HPEL) logs

If you plan to analyze data from the following software, you must collect datafrom the z/OS SYSLOG:v IBM CICS Transaction Server for z/OSv IBM DB2 for z/OSv IBM IMS for z/OSv IBM MQ for z/OSv IBM Resource Access Control Facility (RACF)v z/OS Communications Server

In cooperation with IBM, third party organizations, such as IBM Business Partners,might extend the z/OS Log Forwarder to provide the capability to collect andanalyze log data from more sources. In that situation, the respective third partyorganization provides the documentation for configuring the z/OS Log Forwarderto collect data from those sources.

Creation of data sourcesIn the z/OS Log Forwarder data configuration file, you define a data gatherer foreach separate source of log data. A data gatherer is a component that monitors thelog data that is collected by the z/OS Log Forwarder.

For each data gatherer that you define in the z/OS Log Forwarder configuration, acorresponding data source must exist in IBM Operations Analytics - Log Analysis.

data sourceIn IBM Operations Analytics - Log Analysis, metadata about log data thatenables the log data to be ingested for analysis. The data source includes,for example, the log type, the origin of the log, and an annotation functionthat improves the searchability of the content.

To create a data source, you must have the following information:v A name that uniquely identifies the data source


v The data source type, as it is specified in the respective Insight Packconfiguration artifact

v Optionally, the name of an existing collection in Log Analysis with which youwant to associate data source. If you do not specify a collection name, the datasource is placed in a default collection that is determined by Log Analysis.

You can use either of the following two methods to create data sources. However,if you are using the scalable data collection architecture to forward data, you mustuse method 2 (manual creation of the data sources).

Method 1: Automatic creation of data sources as part of the z/OS LogForwarder configuration

If you are not using the scalable data collection architecture, the preferred methodfor creating data sources is to define them in the data gatherer definitions in thez/OS Log Forwarder data configuration file. Each data gatherer definition includesthe data source name and type.

If you are not using the scalable data collection architecture, when the z/OS LogForwarder starts, it verifies that the appropriate data sources exist in IBMOperations Analytics - Log Analysis. If a data source does not exist, the z/OS LogForwarder uses the data source name and type in the data gatherer definition tocreate the data source automatically.

If you are not using the scalable data collection architecture, and the followingconditions are true, the z/OS Log Forwarder exits its initialization with aconfiguration error and stops without transferring any data:v No data source exists in Log Analysis.v The z/OS Log Forwarder is not configured to create the data source.

Method 2: Manual creation of data sources by using the Log AnalysisData Sources workspace

An alternative method for creating data sources is to create them manually, but ifyou are using the scalable data collection architecture, manual creation is the onlymethod that is available.

If you are using the scalable data collection architecture, use the IP address of eachLPAR (rather than the fully qualified domain name) for the host name.

Before you start the z/OS Log Forwarder for the first time, manually configure thedata sources by using the Log Analysis Data Sources workspace.

Data source typesThe configuration artifacts that are provided with each z/OS Insight Pack includedata source types. A log file splitter, a log record annotator, and a collection areprovided for each type of data source.

Log file splittersThe log file splitters split data into records.

Log record annotators

The annotators annotate fields in the log records so that the records can bemore easily searched. Each field corresponds to part of a log record. Thefields are defined in the index configuration file, and each field is assignedindex configuration attributes.


The annotated fields are displayed in the IBM Operations Analytics - LogAnalysis Search workspace and can be used to filter or search the logrecords.

CollectionsIn IBM Operations Analytics - Log Analysis, a collection is a method fororganizing data sources of the same type. For example, you might want toassign all the data sources for a WebSphere Application Server cluster to asingle collection so that you can search them as a group.

If you want to organize z/OS data sources of the same type into onecollection, you can either create one or more collections of your own, oruse the collections that are predefined in the z/OS Insight Packs for eachdata source type.

Important: Data sources are not predefined. Before you can use an Insight Pack, atleast one data source must be defined.Related reference:“WebSphere Application Server for z/OS data source types” on page 145The WebSphere Application Server for z/OS Insight Pack includes support foringesting, and performing metadata searches against, the following types of datasources: zOS-WAS-SYSOUT, zOS-WAS-SYSPRINT, and zOS-WAS-HPEL.“z/OS Network data source types” on page 153The z/OS Network Insight Pack includes support for ingesting, and performingmetadata searches against, the zOS-NetView type of data source.“z/OS SMF data source types” on page 157The z/OS SMF Insight Pack includes support for ingesting, and performingmetadata searches against, the following types of data sources for SystemManagement Facilities (SMF) data: zOS-SMF30, zOS-SMF80, zOS-SMF110_E,zOS-SMF110_S_10, and zOS-SMF120.“z/OS SYSLOG data source types” on page 168The z/OS SYSLOG Insight Pack includes support for ingesting, and performingmetadata searches against, the following types of data sources:zOS-SYSLOG-Console, zOS-SYSLOG-SDSF, zOS-syslogd, the three variations ofzOS-CICS-MSGUSR, and the three variations of zOS-CICS-EYULOG.

Time zone information for z/OS log recordsIBM Operations Analytics - Log Analysis determines time zone information forz/OS log records. The time zone information varies depending on the source of thelog data.

If Log Analysis cannot determine the time zone information for each log record, itmight not identify the correct relative placement in time for log records fromdifferent sources.

The following information describes how time zone is determined for z/OS logrecords, depending on the source of the log data:

z/OS SYSLOG dataThe time stamps for z/OS SYSLOG messages include time zoneinformation.

CICS Transaction Server for z/OS log dataThe time stamps for CICS Transaction Server for z/OS EYULOG andMSGUSR log messages do not include time zone information. These timestamps are based on the local z/OS system time zone.


NetView for z/OS messagesThe time stamps for the NetView for z/OS messages that are provided bythe NetView message provider do not include time zone information.These time stamps are based on Coordinated Universal Time (UTC).

SMF dataThe time stamps for SMF messages do not include time zone information.These time stamps are based on the local z/OS system time zone.

UNIX System Services system log (syslogd) messagesThe time stamps for syslogd messages do not include time zoneinformation. These time stamps are based on the local z/OS system timezone.

WebSphere Application Server for z/OS log dataThe time stamps for the WebSphere Application Server for z/OS logmessages do not include time zone information, with the followingexceptions:v WebSphere Application Server for z/OS log messages data that is

produced in distributed format contains time stamps with time zoneinformation.

v WebSphere Application Server for z/OS log messages data that isretrieved from High Performance Extensible Logging (HPEL) containstime stamps with time zone information.

By default, time stamps in the WebSphere Application Server for z/OS logsare based on UTC. However, if the WebSphere Application Server for z/OSvariable ras_time_local is set to 1, time stamps are based on the local z/OSsystem time zone. WebSphere Application Server for z/OS variables can beset at the cell, cluster, node, or server scope level.

For each WebSphere Application Server for z/OS data set that is written toa JES job log or z/OS UNIX log file, determine whether the time stamps inthe log data are based on the local z/OS system time zone or on UTC.

Default time zone for logs that have no time zone information

When ingesting logs that have no time zone information, Log Analysis uses one ofthe following default time zone values:v If the logs were sent by the z/OS Log Forwarder, the time zone is the time zone

that is specified by the TZ environment variable on the z/OS Log Forwardersystem.

v If the logs were sent by another method, the time zone is the time zone of theLog Analysis server system.

Override of default time zone values

For all z/OS log data except the following data, you can override the default timezone value for each data gatherer by setting the value for a time zone property inthe z/OS Log Forwarder data configuration file. Again, you cannot override thetime zone for the following data:v NetView for z/OS message datav WebSphere Application Server for z/OS HPEL datav z/OS SYSLOG data


Scalable data collection architectureYou can use the scalable data collection architecture in IBM Operations Analytics -Log Analysis to send data to the Log Analysis server. If your LPARs periodicallygenerate excessive log volumes, or you want the z/OS Log Forwarder to continuecollecting log data even when the Log Analysis server is not available, you mightwant to consider using the scalable data architecture.

In this architecture, the z/OS Log Forwarder serves as a data collection agent. Datacollection agents forward data to intermediate components rather than directly tothe Log Analysis server.

The z/OS Log Forwarder forwards the data to a Logstash server, and anotherLogstash server forwards the data to the Log Analysis server.

Because the z/OS Log Forwarder does not send data directly to the Log Analysisserver, the z/OS Log Forwarder can keep up with the log data that it is readingeven if the Log Analysis server is busy ingesting other logs or is unavailablebecause of maintenance. Logstash can receive data from the z/OS Log Forwardereven if the target system is not available. If sufficient disk space is available,Logstash and Apache Kafka can also buffer that data until the target system isavailable, which reduces the risk that log data is missed because of log fileswrapping or being archived.

If you choose to use the scalable data collection architecture, you must also do thefollowing tasks:v Install and configure some other software components, such as Apache Kafka,

Logstash, and optionally, a load balancer such as HAProxy.v Configure the z/OS Log Forwarder to send data to the appropriate Logstash

server rather than directly to the Log Analysis server.

For your initial configuration, begin with a direct connection to the Log Analysisserver. You can later migrate your configuration to use the scalable data collectionarchitecture.

For information about planning for a scalable data collection deployment, seeDeploying the scalable data collection architecture. The following sections outlinethe configuration aspects that are unique to processing data from the z/OS LogForwarder.

Data source creation

If you use the scalable data collection architecture, the z/OS Log Forwarder doesnot create the data sources on the Log Analysis server. You must manually createdata sources.

For data to be associated with the appropriate data source, each data source musthave the correct host name and file path values. The host name must be the IPaddress of the LPAR where the data originates, and the file path must match thefile path value that is specified for the data gatherer in the z/OS Log Forwarderdata configuration file. If a file path value is not specified in the data configurationfile, see “Data gatherer properties” on page 186 to determine the default file pathvalue for the data gatherer.


http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/config/iwa_cnf_scldc_oview_c.html

Configuration of the Logstash instances in the sender cluster

In events that the z/OS Log Forwarder sends to Logstash, it sends the IP addressof the LPAR in the host field and sends the configured file path for the data in thepath field. Because the host and path fields are part of the Logstash event, they donot need to be specified in the Logstash configuration. The IBM OperationsAnalytics - Log Analysis output plugin uses these fields to determine which datasource to route data to.

The IBM Operations Analytics - Log Analysis output plugin must be configured tosend the fields systemName, sysplexName, and timeZone to the Log Analysis serveras metadata fields. If these fields are not included as metadata, time stamps mightnot be normalized properly, and some problem insights might not be available. Themetadata fields definition must include an entry for each data source that theLogstash instance must process, as shown in the following example:metadata_fields => {

"9.38.13.434@SYSLOG" => {"field_names" => "systemName,sysplexName,timeZone""field_paths" => "systemName,sysplexName,timeZone"

}"9.38.13.434@BBOSKDMS/SYSOUT" => {"field_names" => "systemName,sysplexName,timeZone""field_paths" => "systemName,sysplexName,timeZone"

}"9.38.13.323@SYSLOG" => {"field_names" => "systemName,sysplexName,timeZone""field_paths" => "systemName,sysplexName,timeZone"

}}

Kafka configuration

Define only one partition per topic. Kafka does not guarantee the order ofmessages across multiple partitions within a topic. Because a message mightcontain incomplete log records, all messages for each data source must be in thesame partition.

Configuration of the Logstash instances in the receiver cluster

Each Logstash instance in the receiver cluster must have an HTTP input and aKafka output that specifies the bootstrap servers and the topic to producemessages to. The z/OS Log Forwarder sends the IP address of the LPAR in thehost field and sends the configured file path for the data in the path field. Use%{host}_%{path} as the topic. Because there is only one partition per topic, you donot need to specify the message key.

The following example is a sample configuration for a Logstash instance in thereceiver cluster:input {

http {port => "8085"

}}output {

kafka {bootstrap_servers => "9.113.59.99:17991"topic_id => "%{host}_%{path}"

}}


Load balancer configuration

If two blocks of data that are destined for the same Log Analysis data source aresent to two different Logstash instances in the receiver cluster, they might arrive atthe Log Analysis server in the wrong order, and partial log records might beincorrectly combined. Therefore, if you are using a load balancer such as HAProxyto balance the load between several Logstash servers in the receiver cluster, theload balancer must be configured so that all requests for any host name and filepath combination are always routed to the same Logstash instance in the receivercluster.

The following example shows how the z/OS Log Forwarder constructs the path ofthe request Uniform Resource Identifier (URI):/hostname/filePath

The following information describes two options for configuring HAProxy:v Configure HAProxy such that any two requests with the same URI path value

are routed to the same Logstash instance.v Configure HAProxy such that any two requests from the same LPAR (the same

host name) are routed to the same Logstash instance, which is a simplersolution.

You can set the load balancing algorithm to uri by using the balance keyword, asshown in the following example:balance uri

This algorithm hashes the URI and divides the hash value by the total weight ofthe running servers to determine which server receives the request. This methodensures that the same URI is always directed to the same server as long as noserver starts or stops.

Remember: All data that is read by a data gatherer must arrive at the LogAnalysis server in the order in which it was read. Otherwise, partial records maybe combined incorrectly.

Data collectionThe z/OS Log Forwarder configuration steps vary depending on the sources thatyou collect data from and on whether you use the scalable data collectionarchitecture.

The following information summarizes steps to take, depending on which sourcesyou plan to collect data from:

z/OS SYSLOGIf you plan to collect z/OS SYSLOG data, you must install either theGLASYSG or GLAMDBG user exit. See “User exits for collecting z/OSSYSLOG data” on page 36.

CICS Transaction Server for z/OS logsIf you plan to collect CICS Transaction Server for z/OS log data, you mustcomplete the following steps:v Determine the CICS Transaction Server for z/OS regions from which

you want to collect EYULOG or MSGUSR log data.v Decide whether you want to collect EYULOG or MSGUSR log data from

the CICSPlex® SM application.


NetView for z/OS messagesIf you plan to collect NetView for z/OS message data, you must determinethe NetView for z/OS domains from which to collect data. You must alsoconfigure the NetView message provider to monitor and forward NetViewfor z/OS messages to the z/OS Log Forwarder. See “NetView messageprovider” on page 45.

System Management Facilities (SMF)If you plan to collect SMF data, you must configure the SMF real-time dataprovider to monitor and forward SMF data to the z/OS Log Forwarder.See “SMF real-time data provider” on page 39.

WebSphere Application Server for z/OS logsIf you plan to collect log data for WebSphere Application Server for z/OS,you must determine the application servers from which to collect data.Also, for each of these application servers, you must determine where toretrieve the log data. See “WebSphere Application Server for z/OS logdata” on page 45.

User exits for collecting z/OS SYSLOG dataIf you plan to collect z/OS SYSLOG data, you must install either the GLASYSG orGLAMDBG user exit. The GLASYSG and GLAMDBG user exits, and othermodules that are used by these user exits, are provided with Operations Analyticsfor z Systems and are in the SGLALPA product library.

All modules in the SGLALPA library must be added to the system link pack area(LPA). For more information about the LPA, see the z/OS MVS Initialization andTuning Guide.

The following modules are in the SGLALPA library:v GLADSRAW (a program call module)v GLAGDSDL (a program call module)v GLAGLMSG (a program call module)v GLAMDBGv GLASYSGv GLAUERQ (a program call module)

You must install the GLASYSG or GLAMDBG user exit on the appropriate MVSinstallation exit. Table 7 indicates the MVS installation exit on which to install eachuser exit and describes how to choose which user exit to install.

Both user exits allocate a data space with a minimum size of 100 MB and amaximum size of 500 MB. The data space is used to store z/OS SYSLOG data forretrieval by the z/OS Log Forwarder.

Table 7. User exits with associated MVS installation exits and usage notes

User exit

MVS installation exit onwhich to install the userexit Usage note for user exit

GLASYSG CNZ_MSGTOSYSLOG Install this user exit unless your z/OSsystem is running JES3 with the DLOGoption enabled.

GLAMDBG CNZ_WTOMDBEXIT If your z/OS system is running JES3 withthe DLOG option enabled, install this userexit.


Adding load modules to an LPA

To add the SGLALPA library to the pageable link pack area (PLPA) at system IPL,add the following statement to an LPALSTxx member. Replace zscala.v2r2m0 withthe target library high-level qualifier that is used to install Operations Analytics forz Systems, and replace volume with the volume identifier of the data set.zscala.v2r2m0.SGLALPA(volume)

To add the individual modules in the SGLALPA library to the dynamic LPA afterthe system IPL, issue the following MVS system commands:SETPROG LPA,ADD,MODNAME=GLASYSG,DSNAME=zscala.v2r2m0.SGLALPASETPROG LPA,ADD,MODNAME=GLAMDBG,DSNAME=zscala.v2r2m0.SGLALPASETPROG LPA,ADD,MODNAME=GLADSRAW,DSNAME=zscala.v2r2m0.SGLALPASETPROG LPA,ADD,MODNAME=GLAGDSDL,DSNAME=zscala.v2r2m0.SGLALPASETPROG LPA,ADD,MODNAME=GLAGLMSG,DSNAME=zscala.v2r2m0.SGLALPASETPROG LPA,ADD,MODNAME=GLAUERQ,DSNAME=zscala.v2r2m0.SGLALPA

Installing user exits

To install a user exit on an MVS installation exit at system IPL, add one of thefollowing statements to a PROGxx member:v

EXIT ADD EXITNAME(CNZ_MSGTOSYSLOG) MODNAME(GLASYSG)

vEXIT ADD EXITNAME(CNZ_WTOMDBEXIT) MODNAME(GLAMDBG)

To dynamically install a user exit after the system IPL, issue one of the followingMVS commands:v

SETPROG EXIT,ADD,EXITNAME=CNZ_MSGTOSYSLOG,MODNAME=GLASYSG

vSETPROG EXIT,ADD,EXITNAME=CNZ_WTOMDBEXIT,MODNAME=GLAMDBG

manageUserExit utility for managing the installed user exit:

The GLASYSG and GLAMDBG user exits create system resources that might needto be managed while they are in operation. The manageUserExit utility is a shellscript that can be used to manage the system resources. The utility is included inthe product samples directory in the hierarchical file system.

The following system resources might need to be managed:v A data space, which is used to store z/OS SYSLOG data for retrieval by the

z/OS Log Forwarder.v Program call modules, which are loaded by the user exit and made available to

other programs (such as the z/OS Log Forwarder and the manageUserExitutility) for interacting with the data space.

manageUserExit.sh description

This utility manages the data space and program call modules that are controlledby the user exit. For example, you can use the utility to complete the followingmanagement actions:v Refresh the data space.v Refresh the program call modules.


v Delete the data space, unload the program call modules, and uninstall the userexit from the MVS installation exit.

Important: Before you run the manageUserExit.sh utility, stop any instances of thez/OS Log Forwarder that are gathering z/OS SYSLOG data. This action preventsthe z/OS Log Forwarder from trying to access or call a system resource that isbeing deleted. An abend might occur if the z/OS Log Forwarder accesses anon-existent data space or calls a non-existent program call module.

manageUserExit.sh details

FormatmanageUserExit.sh -p[d] [environment_configuration_directory]

manageUserExit.sh -d[p] [environment_configuration_directory]

manageUserExit.sh -u [environment_configuration_directory]

Parameters

-d Refreshes the data space by deleting and re-creating it.

For normal operations, refreshing the data space is not needed.However, for example, if you are requested to refresh the dataspace by IBM Software Support, use this parameter to delete andre-create the data space. All z/OS SYSLOG data that is in the dataspace before deletion is lost.

-p Refreshes the program call modules by unloading and reloadingfrom the LPA.

Refreshing the program call modules might be necessary whenmaintenance is applied. Updates to the modules in the SGLALPAlibrary must be reloaded by the user exit. Use this parameter tounload the previously loaded program call modules and load thenew program call modules.

Tips:

1. Before you refresh the program call modules, the modules mustbe loaded dynamically into the system LPA. If the program callmodules are currently in the dynamic LPA, the user exit mustbe uninstalled, and the old program call modules must bedeleted from the dynamic LPA before the new modules can bereloaded. The user exit must then be reinstalled on the MVSinstallation exit.

2. If the application of maintenance requires a refresh of theprogram call modules, the maintenance information specifiesthat a refresh is necessary.

-u Deletes the data space, unloads the program call modules, anduninstalls the user exit.

Examples

manageUserExit.sh -pd /etc/IBM/zscala/V2R2This command refreshes both the data space and program callmodules. In this example, the directory /etc/IBM/zscala/V2R2contains the environment configuration file.

manageUserExit.sh -uThis command uses the ZLF_CONF environment variable to find


the directory that contains the environment configuration file. Italso deletes the data space, unloads the program call modules, anduninstalls the user exit.

Exit values

0 Successful completion

-1 Did not complete successfully

MessagesThe utility issues messages to standard output. The messages have theprefix GLAK.

manageUserExit.sh usage notes

The following information describes some tips for using the manageUserExit.shutility:v To run the manageUserExit.sh utility, you must specify at least one parameter.v Specification of the environment configuration directory is optional. However, if

this directory is not specified, the ZLF_CONF environment variable must be set,and its value must be the working directory that contains the zlf.conf file thatis used by the z/OS Log Forwarder.For example, if the zlf.conf file is in /etc/IBM/zscala/V2R2, either theenvironment configuration directory or the value of the ZLF_CONF environmentvariable must be this directory.

v The -p and -d parameters cannot be used with the -u parameter.v The utility requests operations by using a system common storage area. The

requested operation does not complete until the user exit is called by a systemconsole message. The requested operations are not run synchronously by theutility.

v The utility can be run even if the user exit is not active or installed. Therequested operations are completed when the user exit is activated and is calledby a system console message.

v When the utility completes successfully, it indicates only that it made a requestof the user exit. A system console message is issued by the user exit when itperforms the requested operations.

SMF real-time data providerIf you plan to collect System Management Facilities (SMF) data, you mustconfigure the SMF real-time data provider to monitor and forward SMF data to thez/OS Log Forwarder.

The SMF real-time data provider runs as a started task that collects SMF data froman SMF log stream and stores it in the UNIX System Services file system forprocessing by the z/OS Log Forwarder.

In configuring the SMF real-time data provider, you must specify the log stream tomonitor for SMF records and the location in which to store the SMF data. Youmust also configure the z/OS Log Forwarder to specify where the SMF real-timedata provider stores the SMF data.

The SMF real-time data provider program files include the installation files and theconfiguration files. These files are created during the installation of the z/OS LogForwarder and are required for running the SMF real-time data provider.


The SMF real-time data provider configuration is defined in the sample startprocedure GLASMF. This file defines the following items:v SMF log stream on which to listen for SMF datav Location of the data transfer files in the hierarchical file system (HFS)

Tip: You can use the same HFS directory to store all types of SMF data forprocessing by the z/OS Log Forwarder.

v Time interval that specifies how often the SMF real-time data provider collectsSMF data

Configuration for the SMF log stream

All SMF data that is processed by the SMF real-time data provider must originatefrom a single SMF log stream. For example, if you want to collect data for SMF 30,SMF 80, SMF 110, and SMF 120 record types, all four record types must beincluded in one SMF log stream. SMF can write records to the SMF log stream fora time interval that is defined by parameters that are specific to each record type.For example, an SMF 30 record type can be configured to write interval recordsevery 15 minutes, but it might write records in different 15-minute intervals foreach address space that is being monitored.

The z/OS SYS1.PARMLIB member SMFPRMxx (or its equivalent) must enable thecollection of each type of SMF record that you want to collect in the log stream.

At regular intervals, the SMF real-time data provider queries the SMF log streamfor new data. The default, and minimum, interval for querying the log stream is 1minute (for example, the provider queries the log stream every minute).Depending on your environment, you might want to increase this time interval.After each query interval, the SMF real-time data provider makes the new SMFrecords available to the z/OS Log Forwarder, which checks for these updates every10 seconds when it checks for other log file updates.

More information about collecting SMF data

For more information about collecting SMF data, see the information about theMVS System Management Facilities in the z/OS MVS documentation in the IBMKnowledge Center.Related tasks:“Creating the z/OS SMF real-time data provider start procedure” on page 78Create the start procedure for the z/OS SMF real-time data provider by updatingthe sample procedure GLASMF, which is provided in the sample librarySGLASAMP.“Configuring the z/OS Log Forwarder to forward SMF data” on page 90For SMF data to be sent from the z/OS SMF real-time data provider, you mustconfigure the z/OS Log Forwarder to forward the z/OS SMF real-time dataprovider data transfer files.“Starting and stopping the SMF real-time data provider” on page 104To collect SMF data and send it to the z/OS Log Forwarder, the SMF real-timedata provider must be active. The SMF real-time data provider is started as aStarted Task by using the z/OS SMF real-time data provider start procedure.


https://www.ibm.com/support/knowledgecenter/SSLTBW_1.13.0/com.ibm.zos.r13.iea/iea.htm

http://www.ibm.com/support/knowledgecenter


Requirements for enabling the generation of SMF data at the source:

In addition to configuring the SMF real-time data provider to monitor and forwardSMF data to the z/OS Log Forwarder, you must enable SMF data to be generatedat its source.

SMF 30 data generation

SMF record type 30 data is job performance data for z/OS software, which isbased on accounting data.

To enable the generation of SMF record type 30 data, the only requirement is toinclude the SMF 30 record type in the single SMF log stream that the SMFreal-time data provider processes, as described in “Configuration for the SMF logstream” on page 40.


SMF record type 80 data is produced during Resource Access Control Facility(RACF) processing.

To enable the generation of SMF record type 80 data, you must include the SMF 80record type in the single SMF log stream that the SMF real-time data providerprocesses, as described in “Configuration for the SMF log stream” on page 40.RACF must also be installed, active, and configured to protect resources.

For information about the subset of SMF record type 80 data that the SMFreal-time data provider collects, see SMF record type 80 records that the SMFreal-time data provider collects.

SMF also records information that is gathered by RACF auditing. By using variousRACF options, you can regulate the granularity of SMF record type 80 data that iscollected. In the IBM Knowledge Center, see the following information:v Information about the following options of the SETROPTS LOGOPTIONS

command, through which you can control auditing:– DIRSRCH

– DIRACC

– FSOBJ

– FSSEC

v Examples for setting audit controls by using SETROPTS

Before you enable RACF log options, consider the impact in your environment. Forexample, enabling RACF log options can result in the following consequences:v An increase in the amount of disk space that is used for loggingv An increase in the network activity that is required to transmit SMF data to the

IBM Operations Analytics - Log Analysis server


The SMF real-time data provider collects only a subset of the SMF data that isgenerated by CICS Transaction Server for z/OS. It collects the following data fromSMF record type 110:v Monitoring exceptions data for CICS Transaction Server for z/OS from SMF type

110 subtype 1 records, with a class where data = 4



https://www.ibm.com/support/knowledgecenter/SSLTBW_1.13.0/com.ibm.zos.r13.icha800/ichza8c024.htm

https://www.ibm.com/support/knowledgecenter/SSLTBW_1.13.0/com.ibm.zos.r13.icha800/audexmp.htm

v Global transaction manager statistics data for CICS Transaction Server for z/OSfrom SMF type 110 subtype 2 records, with a class where STID = 10

To enable the generation of SMF record type 110 data, you must include the SMF110 record type in the single SMF log stream that the SMF real-time data providerprocesses, as described in “Configuration for the SMF log stream” on page 40. Youmust also define the following CICS Transaction Server for z/OS initializationparameters in the SYSIN data set of the CICS startup job stream:STATRCD=ON, Interval statistics recordingSTATINT=001000, Interval definitionMN=ON, Turn monitoring on or offMNEXC=ON, Exceptions monitoringMNRES=ON, Resource monitoring

For more information about enabling the generation of SMF record type 110 data,see Specifying system initialization parameters before startup in the CICSTransaction Server for z/OS Version 5.2 documentation.

The SMF real-time data provider creates the following record types as it extractsthe relevant data from SMF type 110 records:v zOS-SMF110_E for monitoring exceptions datav zOS-SMF110_S_10 for global transaction manager statistics data

The monitoring exceptions records contain information about CICS TransactionServer for z/OS resource shortages that occur during a transaction, such asqueuing for file strings and waiting for temporary storage. This data highlightspossible problems in CICS system operation. It can help you identify systemconstraints that affect the performance of your transactions. CICS writes oneexception record for each exception condition that occurs.

The global transaction manager records contain transactions summary informationfor CICS Transaction Server for z/OS. This data can give you a more holistic viewof the CICS region, including a comparison among the current and peak numbersof transactions that are running in the region, and the maximum number ofallowed transactions.


The SMF real-time data provider collects only a subset of the SMF data that isgenerated by WebSphere Application Server for z/OS. It collects performance datafrom SMF record type 120 subtype 9. The default SMF type 120 subtype 9 recordcontains information for properly monitoring the performance of your EJBcomponents and web applications.

Tip: This data does not include data for the WebSphere Liberty server.

To enable the generation of SMF record type 120 data, you must include the SMF120 record type in the single SMF log stream that the SMF real-time data providerprocesses, as described in “Configuration for the SMF log stream” on page 40.Also, for each application server instance that you want to monitor, you mustspecify properties for SMF data collection by setting WebSphere Application Serverfor z/OS environment variables from the WebSphere Application ServerAdministrative Console. For more information about enabling the generation ofSMF record type 120 data, see Using the administrative console to enableproperties for specific SMF record types in the WebSphere Application Server forz/OS Version 8.5.5 documentation.


https://www.ibm.com/support/knowledgecenter/en/SSGMCP_5.2.0/com.ibm.cics.ts.doc/dfha6/topics/dfha602.html

http://www.ibm.com/support/knowledgecenter/SS7K4U_8.5.5/com.ibm.websphere.zseries.doc/ae/ttrb_SMFusingsmadmapp.html

http://www.ibm.com/support/knowledgecenter/SS7K4U_8.5.5/com.ibm.websphere.zseries.doc/ae/ttrb_SMFusingsmadmapp.html

The SMF real-time data provider creates the following record types as it extractsthe performance data from SMF type 120 subtype 9 records:v SMF120_REQAPPL for WebSphere application recordsv SMF120_REQCONT for WebSphere controller records

The SMF type 120 subtype 9 record contains information about the activity of theWebSphere server and the hosted applications. This record is produced whenever aserver receives a request. When you do capacity planning, consider the costs thatare involved in running requests and the number of requests that you processduring a specific time. You can use the SMF type 120 subtype 9 record to monitorwhich requests are associated with which applications, the number of requests thatoccur, and the amount of resource that each request uses. You can also use thisrecord to identify the applications that are involved and the amount of CPU timethat the requests use.

As part of planning to collect SMF 120 data, consider the disk space requirementsfor storing the data and the increase in network activity that is required to transmitSMF data to the IBM Operations Analytics - Log Analysis server.

To reduce any system performance degradation due to data collection and toimprove the usability of the data, the SMF real-time data provider aggregates theSMF activity records in 1-minute collection intervals by default. Ensure that thecollection interval is an integral factor of the SMF global recording interval, asmeasured in minutes, so that data collection is synchronized. For example, a 1-, 3-,or 5-minute collection interval is an integral factor of a typical 15-minute SMFglobal recording interval, but a 4-minute collection interval is not. The SMF globalrecording interval INTERVAL(nn) is defined in the SMFPRMxx member ofSYS1.PARMLIB (or its equivalent).

Working directory for the SMF real-time data provider:

The working directory for the SMF real-time data provider contains the datatransfer files that are used to exchange data between the SMF real-time dataprovider and the z/OS Log Forwarder.

Location of the working directory

This directory contains files that are different from those in the working directoryfor the z/OS Log Forwarder, and it must not share the same location as the z/OSLog Forwarder working directory.

For each SMF real-time data provider, you must assign a working directory for thestorage of these data transfer files. The working directory must also be in adifferent physical location from the working directory for any other SMF real-timedata provider.

The location of the SMF real-time data provider working directory is specified inthe start procedure GLASMF.

The following UNIX System Services directory is the default working directory:/etc/IBM/zscala/V2R2/smf

Determine which directory to assign as the SMF real-time data provider workingdirectory for your environment. The working directory must be a UNIX SystemServices directory.


File management in the working directory

In the working directory, the SMF real-time data provider creates a series of filesthat each contain the data for one day. The z/OS Log Forwarder keeps track of thedata that is sent and forwards only new data as it is entered in the file for therespective day.

To conserve disk space, regularly delete the data transfer files in the UNIX SystemServices directory. You can configure a cron job to delete files after a certainnumber of days. For example, to delete the files after 30 days, complete thefollowing steps:1. Use the following crontab command to edit the crontab list (assuming that you

are a user with the appropriate authority to delete the files):crontab -e

2. Enter the following line, for example:0 0 * * * find /etc/IBM/zscala/V2R2/smf -name "SMF*.csv" -mtime +30 -exec rm {} \;

Line explanation:

This line sets up a schedule at the 0 minute of the 0 hour every day of themonth (first *), month of the year (second *), and day of the week (third *).The find command is run, and it searches the directory /etc/IBM/zscala/V2R2/smf for all files with a name that matches the search string SMF*.csv and thatwas last modified 30 or more days ago. For all files with a name that matches,the rm (remove) command is run.

For more information about the crontab, find, and rm commands, see the z/OSUNIX System Services Command Reference.

Authorities that are required for collecting SMF data:

To collect SMF data, you must configure the appropriate authorities for the z/OSLog Forwarder user ID and for the user ID that is associated with the SMFreal-time data provider start procedure.

z/OS Log Forwarder user ID

The z/OS Log Forwarder user ID must have read authority to the SMF real-timedata provider working directory.

SMF real-time data provider user ID

The user ID that is associated with the SMF real-time data provider must have thefollowing authorities:v Read and write authority to the SMF real-time data provider files in the SMF

real-time data provider working directory

Tip: The GLARACF job configures appropriate authority so that the SMFreal-time data provider started task runs under the same user ID as the z/OSLog Forwarder. This configuration ensures that both the SMF real-time dataprovider and the z/OS Log Forwarder have appropriate access rights to the datatransfer files.

v Authority to read the SMF log stream. If you are using RACF as your SAFproduct, for example, you must give this user ID the following authority to theprofile that you set up to secure your SMF log streams, where IFASMF.stream1 isthe name of the SMF log stream that is being used to capture SMF records:


PERMIT IFASMF.stream1 CLASS(LOGSTRM) ACCESS(READ) ID(GLALGF)

v Update access to the RACF profile MVS.SWITCH.SMF in the OPERCMDS RACFclass so that the user ID can issue the MVS SWITCH SMF command.The SMF real-time data provider periodically issues the MVS SWITCH SMFcommand to ensure that it is accessing the most up-to-date data.To grant the user ID update access to this RACF profile, issue the followingcommands:PERMIT MVS.SWITCH.SMF CLASS(OPERCMDS) ACCESS(UPDATE) ID(GLALGF)SETROPTS RACLIST(OPERCMDS) REFRESH

.

NetView message providerIf you plan to collect NetView for z/OS message data, you must configure theNetView message provider to monitor and forward NetView for z/OS messages tothe z/OS Log Forwarder. You must also define the NetView Message gatherer inthe z/OS Log Forwarder data configuration file.

The NetView message provider must be associated with a NetView autotask andcan be run as a long-running command to get NetView for z/OS messages.

Permissions that the NetView autotask requires

The NetView autotask to which you associate the NetView message provider musthave the following permissions:v Permission to access and edit the configuration directory for the NetView

message provider by using the queued sequential access method (QSAM).v Permission to issue CZR messages by using the PIPE and CNMECZFS commandsv Permission to use the LISTVAR and PPI commandsRelated tasks:“Preparing the z/OS NetView message provider for use” on page 77If you plan to collect NetView for z/OS message data, you must complete severalsteps to make the NetView message provider ready for use. The NetView messageprovider is defined in the REXX module GLANETV in the SGLACLST data set.“Configuring the z/OS Log Forwarder to forward NetView for z/OS messagedata” on page 90For NetView for z/OS message data to be sent from the z/OS NetView messageprovider, you must configure the z/OS Log Forwarder to forward the NetView forz/OS message data.“Starting and stopping the NetView message provider” on page 104To collect NetView for z/OS message data and send it to the z/OS Log Forwarder,the NetView message provider must be active. The NetView message provider isstarted as a Started Task by using the REXX module GLANETV.

WebSphere Application Server for z/OS log dataIf you plan to collect log data for WebSphere Application Server for z/OS, youmust determine the application servers from which to collect log data.

Then, for each of these application servers, you must determine where to retrievethe log data.

If the application server is configured to use High Performance Extensible Logging(HPEL) mode, a best practice is to retrieve the log data by using the HPEL API.


If the application server is configured to use basic logging, the log data is retrievedfrom JES job logs, z/OS UNIX log files, or both, depending on how the server isconfigured.

If the application server is configured to use HPEL mode

For each application server that is configured to use HPEL mode, complete thefollowing steps:1. Use the WebSphere Integrated Solutions Console to determine the HPEL

logging and trace directories. Logging and trace information is typically in thesame directory, but it can be configured to be in different directories.

2. Determine whether logging data only, trace data only, or both, is collected.Logging data includes data from the java.util.logging package (the levelDETAIL and higher), the System.out stream, and the System.err stream.Trace data includes data from the java.util.logging package (the levelDETAIL and lower).

3. Ensure that the user ID that is associated with the z/OS Log Forwarderprocedure is authorized to read the HPEL logging and trace directories andfiles.

If the application server is configured to log to JES job logs

For each application server that is configured to log to JES job logs, complete thefollowing steps:1. Determine which regions of the application server to collect log data from.

Region Focus area

Controller region Inbound and outbound communication,security, and transaction control

Servant region Most of the application server components

Adjunct region Internal messaging

2. For each application server region, determine the job name.

Region Typical job name

Controller region Server short name

Servant region Job name for the controller region with an“S” appended

Adjunct region Job name for the controller region with an“A” appended

3. For each application server region, determine whether to collect SYSOUT data,SYSPRINT data, or both types of data.SYSOUT data includes Java logs (non-trace levels) and native message logs.SYSPRINT data includes Java logs (with trace levels) and native trace.

If the application server is configured to log to z/OS UNIX log files

For each application server that is configured to log to z/OS UNIX log files,complete the following steps:1. Determine which regions of the application server to collect log data from.


Region Focus area

Controller region Inbound and outbound communication,security, and transaction control

Servant region Most of the application server components

Adjunct region Internal messaging

2. For each application server region, determine whether to collect SYSOUT data,SYSPRINT data, or both types of data.SYSOUT data includes Java logs (non-trace levels) and native message logs.SYSPRINT data includes Java logs (with trace levels) and native trace.

3. Determine the file path of each z/OS UNIX log file to be collected.4. For each z/OS UNIX file to be collected, determine whether the path name is

constant or varies. The path name varies in the following situations:v When date and time substitution is used in the file name that is specified in

the data definition (DD) statement. The use of date and time substitutioncauses a new log file to be created for each server instance.

v When the WebSphere environment variable redirect_server_output_dir is usedto redirect output to files. The use of this variable causes a new log file to becreated for each server instance. It also gives you the capability to use theROLL_LOGS parameter of the modify command to create a new set of log files.

For more information about file paths for rolling logs, see “Data collection froma rolling z/OS UNIX log.”

5. Ensure that the user ID that is associated with the z/OS Log Forwarderprocedure is authorized to read the z/OS UNIX log files.

Data collection from a rolling z/OS UNIX logThe z/OS Log Forwarder can gather log data from rolling z/OS UNIX logs. Theuse of a rolling log prevents any one log file from getting too large and makes theprocess of pruning old log data from the system is much easier.

A rolling log is a dynamic, sequential set of files that contains a continuous streamof log data. A new file is added whenever a previous file exceeds some threshold(for example, the file surpasses a specified size, or a specified time interval passes).Sometimes, older files are pruned (automatically or manually) so that only adefined number of files is retained.

For example, with a rolling log, a new file might be created once a day, or atspecified times. The log is a set of logically grouped log files, rather than just onelog file. Individual files are differentiated by an index or a time stamp in the filename.

Important: The z/OS Log Forwarder does not gather log data from a rolling log ifthe following events occurred when the log was rolled:v A log file was renamed.v The contents of a log file were removed.

File path pattern for a rolling log:

The z/OS Log Forwarder uses a file path pattern with one or more wildcardcharacters to identify the log files that must be logically grouped into one logicallog (a rolling log) and mapped to one data source in IBM Operations Analytics -Log Analysis.


The following wildcard characters are valid in a file path pattern:

Wildcard character What the character represents

? Any single character

* Any sequence of characters

Assume that a rolling log uses the following file naming scheme, where the integern is incremented for each new log file:/u/myLogDir/myLogFile.n.log

For example, n is 1 for the first file, 2 for the second file, and 3 for the third file.

In this example, the following file path pattern matches all of the file path names:/u/myLogDir/myLogFile.*.log

You must determine the appropriate file path pattern for each set of log files thatare gathered and include it in the z/OS Log Forwarder configuration. The file pathpattern must be as specific as possible so that only the appropriate log files areincluded.

You can use the file path pattern utility to verify your patterns. It determineswhich files on the current system are included by each file path pattern.

Sample scenario that uses date and time substitution in the JCL cataloged procedure:

WebSphere Application Server for z/OS SYSOUT and SYSPRINT logs and CICSTransaction Server for z/OS EYULOG and MSGUSR logs can be redirected toz/OS UNIX files. They can then be rolled by using date and time substitution inthe JCL cataloged procedure that is used to start the server job or CICS region.Each time that the server job or CICS region is restarted, a new file is created.

Assume that the following SYSOUT DD statement from a WebSphere ApplicationServer for z/OS JCL cataloged procedure is used to start a servant job://SYSOUT DD PATH=’/u/myLogDir/myServer.servant.&LYYMMDD..&LHHMMSS..sysout.log’,// PATHOPTS=(OWRONLY,OCREAT),PATHMODE=SIRWXU

The variable &LYYMMDD. is replaced by the local date on which the serverinstance was started, and the date is in YYMMDD format. Similarly, the variable&LHHMMSS. is replaced by the local time in which the server instance wasstarted, and the time is in HHMMSS format.

To convert a path with date and time variables into a file path pattern for z/OSLog Forwarder configuration, replace the date and time variables with one or morewildcard characters.

For example, in this scenario, replace &LYYMMDD. with ?????? because the dateformat YYMMDD is always six characters. Similarly, replace &LHHMMSS. with?????? because the time format HHMMSS is always six characters.

File path pattern for this scenario

Use the following file path pattern for this scenario:/u/myLogDir/myServer.servant.??????.??????.sysout.log


Sample scenario that uses the redirect_server_output_dir environment variable:

WebSphere Application Server for z/OS SYSOUT and SYSPRINT logs can also beredirected to z/OS UNIX files and rolled by using the WebSphere environmentvariable redirect_server_output_dir .

A new set of files for SYSOUT and SYSPRINT is created for each server region atthe following times:v Each time that the server job is restarted.v Each time that the modify command is issued with the ROLL_LOGS parameter.

The new files are created in the directory that is specified by theredirect_server_output_dir environment variable.

The following file naming conventions are used for the redirected files:cellName.nodeName.serverName.jobName.jobId.asType.date.time.SYSOUT.txtcellName.nodeName.serverName.jobName.jobId.asType.date.time.SYSPRINT.txt

For each server region, the cell name, node name, server name, job name, andaddress space type are constant. Only the job ID, date, and time are variable.

To convert one of these file naming convention into a file path pattern for z/OSLog Forwarder configuration, complete the following steps:1. Add the absolute path, which is specified in the WebSphere environment

variable redirect_server_output_dir, to the beginning of the file path pattern.2. Replace cellName, nodeName, serverName, and jobName with the appropriate

values.3. Replace asType with CTL (for controller), SR (for servant), or CRA (for adjunct).4. If you are using JES2, replace jobId with ????????, which matches any eight

characters.If you are using JES3, replace jobId with *, which matches any sequence ofcharacters. In JES3, jobId is sometimes incorrectly populated with the job namerather than the job ID.

5. Replace date with ??????, which matches any six characters.6. Replace time with ??????, which matches any six characters.

File path pattern for this scenario

The following file path pattern is an example of the pattern to use for SYSPRINTfiles for the BBOSAPP server that is using JES2:/u/myLogDir/BBOCELL.BBONODE.BBOSAPP.BBOSAPPS.????????.SR.??????.??????.SYSPRINT.txt

Collection of log data for multiple WebSphere Application Server for z/OSservant processes:

WebSphere Application Server for z/OS can be configured to enable multipleservant processes, and the z/OS Log Forwarder supports the collection of log datafor multiple servant processes.

If you use multiple servant processes, and you use one of the following methods toroll logs, each servant process logs messages to a separate rolling log:v You use date and time substitution in a JCL cataloged procedure.v You use the WebSphere environment variable redirect_server_output_dir.


However, for a single server, the logs for all servant processes are mapped to asingle Log Analysis data source. These logs cannot be mapped to a unique datasource per servant process.

The z/OS Log Forwarder uses a file path pattern with one or more wildcardcharacters to identify the log files for all of the servant processes.

Data collection from paired data setsThe z/OS Log Forwarder can gather log data from a logical pair of data sets,called paired data sets. The use of paired data sets prevents an individual data setfrom getting too large and makes the process of pruning old log data from thesystem much easier.

With paired data sets, data is logged to only one data set in the pair at a time.When that data set exceeds some threshold (for example, the data set surpasses aspecified size, or a specified time interval passes), the data in the other data set isdeleted, and logging switches to that other data set. This switching between eachdata set in the pair is repeated continuously as each threshold is exceeded.

When you define a z/OS Data Set gatherer, you can specify either a single data setor two data sets that are logically paired. If you specify two data sets, the contentsof both data sets are associated with the same data source in IBM OperationsAnalytics - Log Analysis. Both data sets must be entry-sequenced Virtual StorageAccess Method (VSAM) clusters. At least one of the data sets must be allocatedbefore the z/OS Log Forwarder is started.Related reference:“z/OS Data Set gatherer properties” on page 190The z/OS Data Set gatherer properties specify where to retrieve data that is storedin a Virtual Storage Access Method (VSAM) cluster that is an entry-sequenced dataset (ESDS).

Data forwardingYou can configure the z/OS Log Forwarder to forward data to an IBM OperationsAnalytics - Log Analysis server directly or by using the scalable data collectionarchitecture.

Forwarding data directly

Before you configure the z/OS Log Forwarder to forward data directly to an IBMOperations Analytics - Log Analysis server, decide what Log Analysis user nameand password to use for ingesting data.

Forwarding data by using the scalable data collectionarchitecture

When fowarding data by using the scalable data collection architecture, the z/OSLog Forwarder sends data to a Logstash server in the receiver cluster. It sendsevents to the Logstash server with the fields that are specified in Table 8 on page51.


Table 8. Logstash fields for which the z/OS Log Forwarder provides values

Field name Field description

host The fully-qualified domain name of thez/OS Log Forwarder system that sent thedata.Tip: In some versions of Logstash, thisvalue is replaced with the IP address.

message A block of raw log data that can containone, many, or no log records. The first andlast log records in the block might be partialrecords (for example, the end of a log recordat the beginning of the block, or thebeginning of a log record at the end of theblock).

path The file path value that is associated withthe log data.

In the z/OS Log Forwarder dataconfiguration file, if a value is specified forthe dataGatherers.id.filePath property,that value is used as this value. Otherwise,the z/OS Log Forwarder creates a defaultvalue for each defined data gatherer. Formore information about the default value foreach type of data gatherer, see “Datagatherer properties” on page 186.

sourcetype For z/OS SYSLOG gatherers, this value iszOS-SYSLOG-Console.

For WAS HPEL gatherers, this value iszOS-WAS-HPEL.

For all other data gatherers, this value is thevalue that is specified for thedataGatherers.id.dataSourceType propertyin the z/OS Log Forwarder dataconfiguration file. If no value is specified fordataGatherers.id.dataSourceType, this fieldis omitted.

sysplexName The sysplex name of the z/OS LogForwarder system that sent the data.

systemName The system name of the z/OS LogForwarder system that sent the data.

timeZone The time zone that is associated with the logdata.

In the z/OS Log Forwarder dataconfiguration file, if a value is specified forthe dataGatherers.id.timeZone property,that value is used as this value. Otherwise,the value is determined based on the valueof the TZ variable in the z/OS LogForwarder environment configuration file.Sample values include +0000, -0400, and+0300.

Related reference:


“Log receiver properties” on page 183The log receiver properties specify the destination for the log data that is gatheredby the z/OS Log Forwarder instance.


Installing Operations Analytics for z Systems™

As part of installing Operations Analytics for z Systems, you install IBMOperations Analytics - Log Analysis, the z/OS Insight Packs that you want to use,and the z/OS Log Forwarder.

Before you begin

If you have an earlier version of Operations Analytics for z Systems, see“Upgrading Operations Analytics for z Systems” on page 63.

About this task

You must install the software in the following order:1. Log Analysis2. One or more of the following z/OS Insight Packs, depending on the type of log

data that you plan to analyze:v z/OS Network Insight Packv z/OS SMF Insight Packv z/OS SYSLOG Insight Packv WebSphere Application Server for z/OS Insight Pack

3. z/OS Log ForwarderFor information about installing the z/OS Log Forwarder, see “Planning forinstallation of the z/OS Log Forwarder” on page 25.

Tip: To verify that your version of the Operations Analytics for z Systems is thelatest available version, check for updates on the IBM Software Support site.

Installing Log AnalysisIBM Operations Analytics for z Systems can be used with an existing instance ofIBM Operations Analytics - Log Analysis Version 1.3.0, 1.3.1, 1.3.2, or 1.3.3. If oneof these Log Analysis versions is not installed, you must install it.

Before you begin




Continue.


http://www.ibm.com/software/support


7. In the “Select fixes” window, you should see the fix pack 1.3.3-TIV-IOALA-FP001, which you can select and download. For installation instructions, see thereadme file.

Complete the planning task that is described in “Planning for installation of LogAnalysis” on page 24.

The Log Analysis installation media is included in the Operations Analytics for zSystems offering on the DVDs with the following labels:

IBM Operations Analytics - Log Analysis 1.3 Linux 64 bit (LCD8-2724)IBM Operations Analytics - Log Analysis 1.3 Linux on System z 64 bit (LCD7-6059)IBM Operations Analytics - Log Analysis 1.3 Linux on Power 8 64 bit (LCD8-2737)

For more information about installing Log Analysis, see Installing in the LogAnalysis documentation.

Procedure1. Insert the DVD media into the DVD drive of (or mount the corresponding ISO

image on) the Linux system that you plan to use as the Log Analysis server. Ifthe DVD or image is not mounted automatically, mount it by using one of theutilities that are provided with the Linux operating system.

2. In a terminal window, change to the DVD or image directory by issuing thefollowing command, where media_mountpoint is the directory in which themedia is mounted:cd media_mountpoint

3. Run the installation process in either graphical or console mode, and providevalues as necessary.

Option Description

Graphical mode Run the install.sh script.

Console mode Run the install.sh script with the -coption, as shown in the following example:

./install.sh –c

4. Wait for installation to complete.When the installation is complete, the Log Analysis server starts automatically.

5. To verify the status of the Log Analysis server, issue the following command:LA_INSTALL_DIR/utilities/unity.sh -status

The following example shows sample output of this command:Mon April 27 09:43:13 EDT 2016IBM Operations Analytics - Log Analysis v1.3.3.1 STANDARD EDITION Application Services Status:---------------------------------------------------------No. Service Status Process ID---------------------------------------------------------1 Derby Network Server UP 262792 ZooKeeper UP 263173 Websphere Liberty Profile UP 264404 EIF Receiver UP 26579---------------------------------------------------------Getting status of Solr on myhost.mydomain.comStatus of Solr Nodes:-----------------------------------------------------------------No. Instance Name Host Status State-----------------------------------------------------------------1 SOLR_NODE_LOCAL myhost.mydomain.com UP ACTIVE-----------------------------------------------------------------All Application Services are in Running StateChecking server initialization status: Server has initialized!


http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/install/iwa_install_oview_c.html

http://www.ibm.com/support/knowledgecenter/SSPFMY


Depending on the components that are installed, different services aredisplayed in the output for this command. Verify that all services display an UPstatus. Also, verify that the last message in the output indicates that the serverhas initialized.

Installing the z/OS Insight Packs and extensionsFor IBM Operations Analytics - Log Analysis to provide insight about z/OSoperational data, you must install several IBM Operations Analytics for z Systemscomponents for Log Analysis, including the z/OS Insight Packs and the extensionsfor Problem Insights and client-side Expert Advice.

Before you begin

Log Analysis must be running, and you must be logged in to the Linux computersystem with the non-root user ID that was used to install Log Analysis.

For more information about each Insight Pack, including its installation packagename, review “z/OS Insight Packs” on page 11.

For more information about the extensions, review “Extensions to Log Analysis forz/OS Problem Insights and client-side Expert Advice” on page 13.

Important:

v If you plan to analyze network data, install both the z/OS Network Insight Packand the z/OS SYSLOG Insight Pack.

v If you plan to analyze SMF data, install both the z/OS SMF Insight Pack and thez/OS SYSLOG Insight Pack.

The installation packages are on the product DVD with the following label:IBM Operations Analytics for z Systems (LCD7-6544)

About this task

Each Insight Pack includes optional sample searches (sometimes called Quick Searchsamples) to help you find common errors in the software products that the InsightPack supports.

You can use either of the following installation methods:

Install by using the self-extracting installer fileThe self-extracting installer file ioaz_install.run simplifies the installationof the Insight Packs with sample searches and the extensions for ProblemInsights and client-side Expert Advice.

With this installation method, all four z/OS Insight Packs and their samplesearches are installed. You cannot install these individually with thismethod.

If the Log Analysis server version is not Version 1.3.3 Fix Pack 1, theextensions for Problem Insights and client-side Expert Advice are notinstalled. Only the four z/OS Insight Packs and their sample searches areinstalled.

Installing Operations Analytics for z Systems™ 55

To install the Insight Packs, the installer uses the pkg_mgmt.sh commandwith the -upgrade option. If previous versions of the z/OS Insight Packsare installed on the system, the installer upgrades those Insight Packs aspart of the installation process.

Important: During the installation of the Problem Insights and client-sideExpert Advice extensions with the installer file, the Log Analysis server isstopped and restarted.

Install manuallyIf you do not want to use the self-extracting installer file to install thez/OS Insight Packs, use the procedure that is summarized in “Using themanual method of installation” on page 57, and skip the followingprocedure.

With this installation method, the extensions for Problem Insights andclient-side Expert Advice are not installed. Only the four z/OS InsightPacks and their sample searches are installed.

The examples in this procedure use the following location as the Log Analysisinstallation directory (LA_INSTALL_DIR):/home/scala/IBM/LogAnalysis/

Procedure1. Insert the Insight Pack DVD media into the DVD drive of the Log Analysis

server. If the DVD is not mounted automatically, mount it by using one of theutilities that are provided with the Linux operating system.

Restriction: If you obtained the z/OS Insight Packs and extensions as a .tarfile from IBM Fix Central, unpack the .tar file into a temporary directory onthe target computer, and complete the remaining steps in this procedure.

2. To install Insight Packs, extensions, and sample searches, the installer musthave the directory where Log Analysis is installed (LA_INSTALL_DIR). Use oneof the following methods to specify this directory and to run the installer:

Option Instructions

Specify on thecommand linewhen you run theinstaller

To run the installer, and specify the directory on the command, runthe following command, for example:

sh ioaz_install.run option -- /home/scala/IBM/LogAnalysis/

You can also specify the following options on the command:

--noexecSpecifies that the installer script must not be run.

--target dirSpecifies an alternate location for extracting files.

The default directory is /tmp/ioaz_install.

Specify in theLA_INSTALL_DIRenvironmentvariable

To set the LA_INSTALL_DIR environment variable, and run theinstaller, run the following commands, for example:

export LA_INSTALL_DIR=/home/scala/IBM/LogAnalysis/sh ioaz_install.run

Specify when theinstaller runs andrequests thedirectory

When you run the installer, if the directory is not specified throughanother method, the installer prompts the user to enter the directory.

To run the installer, run the following command:

sh ioaz_install.run


If you try to install with an incorrect user ID: Remember that to install theInsight Packs and extensions, you must be logged in to the Linux computersystem with the non-root user ID that was used to install Log Analysis.

If you try to install these with some other user ID, and then rerun the installerwith the correct user ID, the installer might stop with the following message:Creating directory /tmp/install_ioazVerifying archive integrity... All good.Uncompressing Install Extensions and Insight Packs 100% Extraction failed.Terminated

To resolve this error, remove the directory and log file by running the followingcommands:rm -rf /tmp/install_ioazrm /tmp/ioaz_install.log

Related reference:“Sample searches” on page 200If you installed any of the optional sample searches (sometimes called Quick Searchsamples) in IBM Operations Analytics - Log Analysis, they are available in the zosfolder in the Saved Searches navigator of the Log Analysis user interface.

Using the manual method of installationIf you do not want to use the self-extracting installer file to install the z/OS InsightPacks and sample searches, you can instead use the manual method of installation.

Before you begin

Review the “Before you begin” and “About this task” sections in “Installing thez/OS Insight Packs and extensions” on page 55.

About this task

For more information about installing Insight Packs, see pkg_mgmt.sh command inthe Log Analysis documentation.




2. On the Log Analysis server, create a directory in which to store the InsightPack. For example, issue one of the following commands, depending on theInsight Pack that you are installing:mkdir LA_INSTALL_DIR/unity_content/SMFforzOS

mkdir LA_INSTALL_DIR/unity_content/SYSLOGforzOS

mkdir LA_INSTALL_DIR/unity_content/WASforzOS

mkdir LA_INSTALL_DIR/unity_content/zOSNetwork


http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/extend/iwa_extend_pkg_mgmt_cmd_c.html

3. Copy the compressed file for the Insight Pack from the DVD media into thedirectory that you created in step 2 on page 57. For example, issue one of thefollowing commands:cp dvd_mountpoint/SMFforzOS/SMFforzOSInsightPack_v1.1.0.3.zip

LA_INSTALL_DIR/unity_content/SMFforzOS

cp dvd_mountpoint/SYSLOGforzOS/SYSLOGforzOSInsightPack_v2.2.0.2.zipLA_INSTALL_DIR/unity_content/SYSLOGforzOS

cp dvd_mountpoint/WASforzOS/WASforzOSInsightPack_v2.2.0.2.zipLA_INSTALL_DIR/unity_content/WASforzOS

cp dvd_mountpoint/zOSNetwork/zOSNetworkInsightPack_v1.1.0.2.zipLA_INSTALL_DIR/unity_content/zOSNetwork

4. Install the Insight Pack by using the pkg_mgmt.sh command that is providedwith Log Analysis. For example, issue one of the following commands:LA_INSTALL_DIR/utilities/pkg_mgmt.sh -install

LA_INSTALL_DIR/unity_content/SMFforzOS/SMFforzOSInsightPack_v1.1.0.3.zip

LA_INSTALL_DIR/utilities/pkg_mgmt.sh -installLA_INSTALL_DIR/unity_content/SYSLOGforzOS/SYSLOGforzOSInsightPack_v2.2.0.2.zip

LA_INSTALL_DIR/utilities/pkg_mgmt.sh -installLA_INSTALL_DIR/unity_content/WASforzOS/WASforzOSInsightPack_v2.2.0.2.zip

LA_INSTALL_DIR/utilities/pkg_mgmt.sh -installLA_INSTALL_DIR/unity_content/zOSNetwork/zOSNetworkInsightPack_v1.1.0.2.zip

5. Optional: To install the optional sample searches, complete the following steps:a. Copy the samples directory from the Insight Pack installation media to the

Log Analysis installation directory, as shown in the following examples:cp -r dvd_mountpoint/SMFforzOS/samples LA_INSTALL_DIR/unity_content/SMFforzOS

cp -r dvd_mountpoint/SYSLOGforzOS/samples LA_INSTALL_DIR/unity_content/SYSLOGforzOS

cp -r dvd_mountpoint/WASforzOS/samples LA_INSTALL_DIR/unity_content/WASforzOS

cp -r dvd_mountpoint/zOSNetwork/samples LA_INSTALL_DIR/unity_content/zOSNetwork

b. Change to the samples directory in the Log Analysis installation directory,as shown in the following examples:cd LA_INSTALL_DIR/unity_content/SMFforzOS/samples

cd LA_INSTALL_DIR/unity_content/SYSLOGforzOS/samples

cd LA_INSTALL_DIR/unity_content/WASforzOS/samples

cd LA_INSTALL_DIR/unity_content/zOSNetwork/samples

c. To load the sample searches, ensure that Log Analysis is running, and runthe following program from the appropriate samples directories:./LoadSamples.sh

Uninstalling the Insight PacksWhen you uninstall an Insight Pack, all data sources and collections that areassociated with the Insight Pack are deleted. If ingested data exists for any datasource types that are defined by the Insight Pack, you must remove that databefore you can uninstall the Insight Pack.

Before you begin

For information about the data source types that are defined by each Insight Pack,see the following information:


Insight Pack Data source types

WebSphere Application Server for z/OSInsight Pack

“WebSphere Application Server for z/OSdata source types” on page 145

z/OS Network Insight Pack “z/OS Network data source types” on page153

z/OS SMF Insight Pack “z/OS SMF data source types” on page 157

z/OS SYSLOG Insight Pack “z/OS SYSLOG data source types” on page168

If you are uninstalling the WebSphere Application Server for z/OS InsightPack: The WASSystemOut data source type is defined by the WebSphere ApplicationServer Insight Pack that is provided with Log Analysis. Data sources that areassociated with this data source type are not deleted when you uninstall theWebSphere Application Server for z/OS Insight Pack. You do not have to removedata for the WASSystemOut data source type.

Procedure

To uninstall an Insight Pack, complete the following steps:1. If ingested data exists for any data source types that are defined by the Insight

Pack, remove that data from IBM Operations Analytics - Log Analysis byfollowing the instructions in “Removing data from Log Analysis.”

2. Uninstall the Insight Pack by using the pkg_mgmt.sh command that is providedwith Log Analysis. For example, issue one of the following commands,depending on the Insight Pack that you are uninstalling:LA_INSTALL_DIR/utilities/pkg_mgmt.sh -uninstall


LA_INSTALL_DIR/utilities/pkg_mgmt.sh -uninstallLA_INSTALL_DIR/unity_content/SYSLOGforzOS/SYSLOGforzOSInsightPack_v2.2.0.2.zip

LA_INSTALL_DIR/utilities/pkg_mgmt.sh -uninstallLA_INSTALL_DIR/unity_content/WASforzOS/WASforzOSInsightPack_v2.2.0.2.zip

LA_INSTALL_DIR/utilities/pkg_mgmt.sh -uninstallLA_INSTALL_DIR/unity_content/zOSNetwork/zOSNetworkInsightPack_v1.1.0.2.zip

Removing data from Log AnalysisTo remove data from IBM Operations Analytics - Log Analysis, use the deletiontool.

Before you begin

Before you remove the data, complete the following steps:1. Stop each instance of the z/OS Log Forwarder.2. Ensure that Log Analysis is running.

About this task

The deletion tool provides the following options (or use cases) for deleting data:1. Delete all data from a single data source.


2. Delete all data from a single collection.3. For a specified time period, delete data from all data sources.4. At regular intervals, delete data that is older than the specified retention

period.

For more information about removing data from Log Analysis, seedelete.properties.

Procedure

To remove data from Log Analysis by using the deletion tool, complete thefollowing steps:1. In the LA_INSTALL_DIR/utilities/deleteUtility directory, open the

delete.properties file.2. For the use case that you want to run, specify the use case number and the

variables that are associated with that use case.The use cases are summarized in About this task.

3. Save the delete.properties file.4. Run the following command, where Python_path represents the location where

Python is installed, and password represents the password that is defined in thedelete.properties file for the associated user name:Python_path deleteUtility.py password

Uninstalling Log AnalysisYou can uninstall IBM Operations Analytics - Log Analysis by using the IBMInstallation Manager.

Before you begin

Before you uninstall Log Analysis, uninstall any remote installations of ApacheSolr.

For more information about uninstalling Log Analysis, see Removing Log Analysis.

Procedure

Run the uninstallation process in either graphical or console mode.

Option Description

Graphical mode 1. Go to the following directory, whereim_install_dir represents the directorywhere the IBM Installation Manager isinstalled: im_install_dir/IBM/InstallationManager/eclipse.

2. Run the following command:

./launcher


http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/admin/iwa_delete.props_r.html

http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/install/iwa_uninstall_oview_c.html

Option Description

Console mode 1. Go to the following directory, whereim_install_dir represents the directorywhere the IBM Installation Manager isinstalled: im_install_dir/IBM/InstallationManager/eclipse/tools.

2. Run the following command:

./imcl -c


Upgrading Operations Analytics for z Systems

You can upgrade from IBM Operations Analytics for z Systems Version 2.1.0.0 toVersion 2.2.0.1 Interim Feature 2. IBM Operations Analytics for z Systems Version2.2.0.1 Interim Feature 2 includes IBM Operations Analytics - Log Analysis Version1.3.3 Standard Edition, but Log Analysis Versions 1.3.0, 1.3.1, and 1.3.2 are alsosupported.

Before you begin

At this time, a direct upgrade from IBM Operations Analytics - Log Analysis V1.3.0to V1.3.3 Fix Pack 1 is not supported.

If you have Log Analysis V1.3.0 installed in your environment, upgrading to LogAnalysis V1.3.3 Fix Pack 1 is optional.




Continue.7. In the “Select fixes” window, you should see the fix pack 1.3.3-TIV-IOALA-

FP001, which you can select and download. For installation instructions, see thereadme file.

About this task

Log data, data sources, and collections that are associated with the followingInsight Packs are retained during the upgrade:v WebSphere Application Server for z/OS Insight Packv z/OS SMF Insight Packv z/OS SYSLOG Insight Pack

Upgrading to V2.2.0.1 Interim Feature 2, including upgrading to LogAnalysis V1.3.3 Fix Pack 1

If you choose to upgrade to IBM Operations Analytics - Log Analysis V1.3.3 FixPack 1 as part of the upgrade to IBM Operations Analytics for z Systems Version2.2.0.0, you must uninstall IBM Operations Analytics - Log Analysis.



Procedure

To upgrade IBM Operations Analytics for z Systems, complete the following steps:1. Stop each instance of the z/OS Log Forwarder.2. In each LPAR from which you gather z/OS SMF data, stop the z/OS SMF

real-time data provider.3. Stop the IBM Operations Analytics - Log Analysis server.4. Back up the data in IBM Operations Analytics - Log Analysis.

For more information about backing up the data, see one of the followingtopics, depending on which one is relevant for the Log Analysis release:v Version 1.3.1 topic about upgrading, backing up, and migrating datav Version 1.3.2 topic about upgrading, backing up, and migrating datav Version 1.3.3 topic about upgrading, backing up, and migrating data

.5. Uninstall IBM Operations Analytics - Log Analysis.

See “Uninstalling Log Analysis” on page 60.6. Install IBM Operations Analytics - Log Analysis Version 1.3.1.

See “Installing Log Analysis” on page 53.7. Restore the backed up data to Log Analysis.

For more information about restoring the data, see one of the following topics,depending on the respective Log Analysis release:v Version 1.3.1 topic about upgrading, backing up, and migrating datav Version 1.3.2 topic about upgrading, backing up, and migrating datav Version 1.3.3 topic about upgrading, backing up, and migrating data

8. Repeat steps 4 - 7 to upgrade Log Analysis Version 1.3.1 to 1.3.2.9. Repeat steps 4 - 7 to upgrade Log Analysis Version 1.3.2 to 1.3.3.

10. Install Fix Pack 1 of Log Analysis Version 1.3.3.To get Log Analysis Version 1.3.3 Fix Pack 1, complete the following steps:a. Go to IBM Fix Central.b. In the Product selector field, start typing IBM Operations Analytics - Log

Analysis, and when the correct product name is shown in the resultinglist, select it. More fields are then shown.

c. In the Installed Version field, select 1.3.3.d. In the Platform field, select All.e. Click Continue.f. In the resulting “Identify fixes” window, select Browse for fixes, and click

Continue.g. In the “Select fixes” window, you should see the fix pack

1.3.3-TIV-IOALA-FP001, which you can select and download. Forinstallation instructions, see the readme file.

11. Because Custom Search Dashboard applications are reinstalled as part of thisupgrade, back up any IBM-provided Custom Search Dashboard applicationsthat you customized.These Custom Search Dashboard applications are in the following directories:


http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.1/com.ibm.scala_1.3.1.doc/admin/iwa_admin_backup_restore.html


http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/admin/iwa_admin_backup_restore.html



http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/admin/iwa_admin_backup_restore.html


WebSphere Application Server for z/OS Insight Pack directory fordashboard applications

LA_INSTALL_DIR/AppFramework/Apps/WASforzOSInsightPack_v2.1.0.0

z/OS Network Insight Pack directory for dashboard applicationsLA_INSTALL_DIR/AppFramework/Apps/zOSNetworkInsightPack_v1.0.0.0

z/OS SMF Insight Pack directory for dashboard applicationsLA_INSTALL_DIR/AppFramework/Apps/SMFforzOSInsightPack_v1.1.0.0

z/OS SYSLOG Insight Pack directory for dashboard applicationsLA_INSTALL_DIR/AppFramework/Apps/SYSLOGforzOSInsightPack_v2.1.0.0

12. Upgrade the z/OS Insight Packs that you use.See “Upgrading the z/OS Insight Packs” on page 69.

13. Update the IBM-provided Custom Search Dashboard applications in thefollowing directories to include any custom changes that you made to theprevious version of these Apps.






14. If you installed the optional sample searches for the z/OS Network InsightPack, z/OS SMF Insight Pack, or z/OS SYSLOG Insight Pack, upgrade thesesamples.See “Using the manual method of upgrading” on page 71.

15. Install the Version 2.2 z/OS Log Forwarder by using SMP/E for z/OS(SMP/E).For the installation instructions, see the Program Directory for IBM OperationsAnalytics for z Systems.

16. Update the configuration for each instance of the z/OS Log Forwarder.See “Updating the z/OS Log Forwarder configuration” on page 73.

17. In each LPAR from which you gather z/OS SYSLOG data, manually uninstallthe GLASYSG or GLAMDBG user exit from the CNZ_MSGTOSYSLOG orCNZ_WTOMDBEXIT MVS installation exit, and complete the following stepsto refresh the modules in the link pack area (LPA):a. Use the manageUserExit utility with the -p option to request a refresh of

the procedure call modules.This request is not processed until the user exit is reinstalled in the nextstep.

Upgrading Operations Analytics for z Systems 65

For information about the manageUserExit utility, see “manageUserExitutility for managing the installed user exit” on page 37.

b. If the modules are currently in the dynamic LPA, delete the modules.c. Add the modules to the dynamic LPA.

18. For gathering z/OS SYSLOG data, install one of the new user exits.For more information, see “User exits for collecting z/OS SYSLOG data” onpage 36.After the user exit processes the first message, it acts on the request (from theprevious step) to refresh the procedure call modules.

Tip: When the new user exit is installed on the MVS installation exit, allsystem resources that were created and used by the previous user exit areautomatically migrated (if migration is necessary). These system resources arenot deleted when the previous user exit is deleted from the installation exit.The system resources include the data space and program call modules.

19. In each LPAR from which you gather z/OS SMF data, complete the followingsteps:a. Copy the new SMF real-time data provider start procedure (GLASMF) to

your user procedure library.b. In the new version of the samples, include the changes that you added in

the previous version of the samples.

Important: The configuration file GLASMFCF is now part of the GLASMFsample. Incorporate any customizations from your GLASMFCF sampleinto the new GLASMF sample.

The DRLIN data definition includes new members that are required for thisversion of the data provider, and the the COLLECT SMF line now contains thephrase WITHOUT STATISTICS after the log stream name.For more information about creating the start procedure, see the followingtopics:v “Creating the z/OS SMF real-time data provider start procedure” on page

78v “Variable reference for the z/OS SMF real-time data provider start

procedure” on page 21620. Complete the following steps to upgrade the Log Forwarder Configuration

Assistant start program (GLACONFG REXX):a. Copy the new GLACONFG sample into your local CLIST library.b. Merge any changes that you made in your existing copy into the new

copy. For example, ensure that the variables CLISTLIB, PANELLIB, andMESSAGELIB point to the correct installation data sets, if you renamedthese data sets when you installed the z/OS Log Forwarder.

For more information about the Log Forwarder Configuration Assistant startprogram, see “Customizing the Log Forwarder Configuration Assistant startprogram” on page 79.

21. Restart each instance of the z/OS Log Forwarder.

Tip: Message GLAH007I is issued to indicate that the action to refresh theprocedure call modules (from the user exit for gathering z/OS SYSLOG data)is complete.

22. In each LPAR from which you gather z/OS SMF data, restart the z/OS SMFreal-time data provider.


Upgrading to V2.2.0.1 Interim Feature 2 without upgrading LogAnalysis

Upgrading to IBM Operations Analytics - Log Analysis V1.3.3 Fix Pack 1 isoptional. If you choose not to upgrade to Log Analysis V1.3.3 Fix Pack 1 as part ofthe upgrade to IBM Operations Analytics for z Systems Version 2.2.0.1 InterimFeature 2, you do not need to uninstall Log Analysis V1.3.0.

Procedure

To upgrade IBM Operations Analytics for z Systems, complete the following steps:1. Stop each instance of the z/OS Log Forwarder.2. In each LPAR from which you gather z/OS SMF data, stop the z/OS SMF

real-time data provider.3. Stop the IBM Operations Analytics - Log Analysis server.4. Because Custom Search Dashboard applications are reinstalled as part of this

upgrade, back up any IBM-provided Custom Search Dashboard applicationsthat you customized.These Custom Search Dashboard applications are in the following directories:






5. Upgrade the z/OS Insight Packs that you use.See “Upgrading the z/OS Insight Packs” on page 69.

6. Update the IBM-provided Custom Search Dashboard applications in thefollowing directories to include any custom changes that you made to theprevious version of these Apps.







7. If you installed the optional sample searches for the z/OS Network InsightPack, z/OS SMF Insight Pack, or z/OS SYSLOG Insight Pack, upgrade thesesamples.See “Using the manual method of upgrading” on page 71.

8. Install the Version 2.2 z/OS Log Forwarder by using SMP/E for z/OS(SMP/E).For the installation instructions, see the Program Directory for IBM OperationsAnalytics for z Systems.

9. Update the configuration for each instance of the z/OS Log Forwarder.See “Updating the z/OS Log Forwarder configuration” on page 73.

10. In each LPAR from which you gather z/OS SYSLOG data, manually uninstallthe GLASYSG or GLAMDBG user exit from the CNZ_MSGTOSYSLOG orCNZ_WTOMDBEXIT MVS installation exit, and complete the following stepsto refresh the modules in the link pack area (LPA):a. Use the manageUserExit utility with the -p option to request a refresh of

the procedure call modules.This request is not processed until the user exit is reinstalled in the nextstep.For information about the manageUserExit utility, see “manageUserExitutility for managing the installed user exit” on page 37.

b. If the modules are currently in the dynamic LPA, delete the modules.c. Add the modules to the dynamic LPA.

11. For gathering z/OS SYSLOG data, install one of the new user exits.For more information, see “User exits for collecting z/OS SYSLOG data” onpage 36.After the user exit processes the first message, it acts on the request (from theprevious step) to refresh the procedure call modules.

Tip: When the new user exit is installed on the MVS installation exit, allsystem resources that were created and used by the previous user exit areautomatically migrated (if migration is necessary). These system resources arenot deleted when the previous user exit is deleted from the installation exit.The system resources include the data space and program call modules.

12. In each LPAR from which you gather z/OS SMF data, complete the followingsteps:a. Copy the new SMF real-time data provider start procedure (GLASMF) to

your user procedure library.b. In the new version of the samples, include the changes that you added in

the previous version of the samples.

Important: The configuration file GLASMFCF is now part of the GLASMFsample. Incorporate any customizations from your GLASMFCF sampleinto the new GLASMF sample.

The DRLIN data definition includes new members that are required for thisversion of the data provider, and the the COLLECT SMF line now contains thephrase WITHOUT STATISTICS after the log stream name.For more information about creating the start procedure, see the followingtopics:


v “Creating the z/OS SMF real-time data provider start procedure” on page78

v “Variable reference for the z/OS SMF real-time data provider startprocedure” on page 216

13. Complete the following steps to upgrade the Log Forwarder ConfigurationAssistant start program (GLACONFG REXX):a. Copy the new GLACONFG sample into your local CLIST library.b. Merge any changes that you made in your existing copy into the new

copy. For example, ensure that the variables CLISTLIB, PANELLIB, andMESSAGELIB point to the correct installation data sets, if you renamedthese data sets when you installed the z/OS Log Forwarder.

For more information about the Log Forwarder Configuration Assistant startprogram, see “Customizing the Log Forwarder Configuration Assistant startprogram” on page 79.

14. Restart each instance of the z/OS Log Forwarder.

Tip: Message GLAH007I is issued to indicate that the action to refresh theprocedure call modules (from the user exit for gathering z/OS SYSLOG data)is complete.

15. In each LPAR from which you gather z/OS SMF data, restart the z/OS SMFreal-time data provider.

Upgrading the z/OS Insight PacksIf you installed the previous version of a z/OS Insight Pack, and you plan tocontinue analyzing the same log data, you must upgrade the Insight Packs thatyou use. If you previously installed the optional sample searches for the z/OSNetwork Insight Pack, z/OS SMF Insight Pack, or z/OS SYSLOG Insight Pack, youmust upgrade them.

Before you begin

As part of the upgrade procedure, you must delete some old sample searches. Ifyou edited any of the sample searches but did not change the file name, give yourcustomized searches unique file names so that they are not unintentionally deletedduring an upgrade.

Before you upgrade the z/OS Insight Packs, complete the following steps:1. Stop each instance of the z/OS Log Forwarder.2. Ensure that IBM Operations Analytics - Log Analysis is running.3. Ensure that you are logged in to the Linux computer system with the non-root

user ID that was used to install Log Analysis.

About this task

You can use either of the following methods to upgrade the Insight Packs and theoptional sample searches:

Upgrade by using the self-extracting installer fileThe self-extracting installer file ioaz_install.run simplifies the upgradingof the Insight Packs with sample searches and simplifies the installation ofthe extensions for Problem Insights and client-side Expert Advice.


With this upgrade method, all four z/OS Insight Packs and their samplesearches are upgraded. You cannot upgrade these individually with thismethod. Also, if the Log Analysis server version is Version 1.3.3 Fix Pack 1,the extensions for Problem Insights and client-side Expert Advice areinstalled as part of the upgrade.

Important: During the installation of the Problem Insights and client-sideExpert Advice extensions with the installer file, the Log Analysis server isstopped and restarted.

Install manuallyIf you do not want to use the self-extracting installer file to upgrade z/OSInsight Packs, use the procedure that is summarized in “Using the manualmethod of upgrading” on page 71, and skip the following procedure.However, with the manual procedure, the extensions for Problem Insightsand client-side Expert Advice are not installed.

Procedure1. Verify that Log Analysis is running.2. In the Log Analysis user interface, expand Saved Searches.3. Right-click each of the following search names, and click Delete to delete each

respective search sample:

WebSphere Application Server for z/OS Insight Pack sample searchesDelete the WAS Exceptions search from the was folder.

z/OS SMF Insight Pack sample searchesDelete each search that is listed under the smf folder.

z/OS SYSLOG Insight Pack sample searchesDelete the following searches from their respective folders:v db2

– DB2 Critical Data Set Messagesv mq

– MQ Interesting Informational4. Insert the Insight Pack DVD media into the DVD drive of the Log Analysis



5. To upgrade Insight Packs, extensions, and sample searches, the installer musthave the directory where Log Analysis is installed (LA_INSTALL_DIR). Use oneof the following methods to specify this directory and to run the installer:


Option Instructions

Specify on thecommand linewhen you run theinstaller

To run the installer, and specify the directory on the command, runthe following command, for example:

sh ioaz_install.run option -- /home/scala/IBM/LogAnalysis/

You can also specify the following options on the command:

--noexecSpecifies that the installer script must not be run.

--target dirSpecifies an alternate location for extracting files.

The default directory is /tmp/ioaz_install.

Specify in theLA_INSTALL_DIRenvironmentvariable

To set the LA_INSTALL_DIR environment variable, and run theinstaller, run the following commands, for example:

export LA_INSTALL_DIR=/home/scala/IBM/LogAnalysis/sh ioaz_install.run

Specify when theinstaller runs andrequests thedirectory

When you run the installer, if the directory is not specified throughanother method, the installer prompts the user to enter the directory.

To run the installer, run the following command:

sh ioaz_install.run

If you try to install with an incorrect user ID: Remember that to install theInsight Packs and extensions, you must be logged in to the Linux computersystem with the non-root user ID that was used to install Log Analysis.

If you try to install these with some other user ID, and then rerun the installerwith the correct user ID, the installer might stop with the following message:Creating directory /tmp/install_ioazVerifying archive integrity... All good.Uncompressing Install Extensions and Insight Packs 100% Extraction failed.Terminated

To resolve this error, remove the directory and log file by running the followingcommands:rm -rf /tmp/install_ioazrm /tmp/ioaz_install.log

Using the manual method of upgradingIf you do not want to use the self-extracting installer file to upgrade the z/OSInsight Packs and sample searches, you can instead use the manual method ofupgrading.

Before you begin

Before you upgrade the z/OS Insight Packs, complete the following steps:1. Stop each instance of the z/OS Log Forwarder.2. Ensure that IBM Operations Analytics - Log Analysis is running.3. Ensure that you are logged in to the Linux computer system with the non-root

user ID that was used to install Log Analysis.


About this task

You can install the new Insight Pack by using the in-place upgrade option. Withthis option, any data, collections, or data sources are preserved.

For more information about doing an in-place upgrade of an Insight Pack, seeUpgrading an Insight Pack.




2. Copy the compressed file for the Insight Pack from the DVD media into theappropriate directory. For example, issue one of the following commands:cp dvd_mountpoint/SMFforzOS/SMFforzOSInsightPack_v1.1.0.3.zip

LA_INSTALL_DIR/unity_content/SMFforzOS

cp dvd_mountpoint/SYSLOGforzOS/SYSLOGforzOSInsightPack_v2.2.0.2.zipLA_INSTALL_DIR/unity_content/SYSLOGforzOS

cp dvd_mountpoint/WASforzOS/WASforzOSInsightPack_v2.2.0.2.zipLA_INSTALL_DIR/unity_content/WASforzOS

cp dvd_mountpoint/zOSNetwork/zOSNetworkInsightPack_v1.1.0.2.zipLA_INSTALL_DIR/unity_content/zOSNetwork

3. Install the Insight Pack by using the upgrade option (rather than the installoption) on the pkg_mgmt.sh command that is provided with Log Analysis. Forexample, issue one of the following commands, depending on the Insight Packthat you are upgrading:LA_INSTALL_DIR/utilities/pkg_mgmt.sh -upgrade


LA_INSTALL_DIR/utilities/pkg_mgmt.sh -upgradeLA_INSTALL_DIR/unity_content/SYSLOGforzOS/SYSLOGforzOSInsightPack_v2.2.0.2.zip

LA_INSTALL_DIR/utilities/pkg_mgmt.sh -upgradeLA_INSTALL_DIR/unity_content/WASforzOS/WASforzOSInsightPack_v2.2.0.2.zip

LA_INSTALL_DIR/utilities/pkg_mgmt.sh -upgradeLA_INSTALL_DIR/unity_content/zOSNetwork/zOSNetworkInsightPack_v1.1.0.2.zip

4. In the Log Analysis user interface, expand Saved Searches.5. Right-click each of the following search names, and click Delete to delete each

respective search sample:

WebSphere Application Server for z/OS Insight Pack sample searchesDelete the WAS Exceptions search from the was folder.

z/OS SMF Insight Pack sample searchesDelete each search that is listed under the smf folder.

z/OS SYSLOG Insight Pack sample searchesDelete the following searches from their respective folders:v db2

– DB2 Critical Data Set Messages


http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/extend/iwa_extend_upgrade_pack.html

v mq

– MQ Interesting Informational6. Install the updated sample searches as described in “Using the manual method

of installation” on page 57.

Tip: You can ignore warning messages about the sample searches that were notdeleted. Those searches were not changed for this release, but the installationprocess tries to install them and issues a warning if they are already present.

Updating the z/OS Log Forwarder configurationAfter you install Version 2.2 of the z/OS Log Forwarder in the logical partitions(LPARs), you must update the configuration for each instance of the z/OS LogForwarder.

Before you begin

Ensure that Version 2.2 of the z/OS Log Forwarder is installed in the appropriateLPARs. Install the z/OS Log Forwarder by using SMP/E for z/OS (SMP/E). Forthe installation instructions, see the Program Directory for IBM OperationsAnalytics for z Systems.

Table 9 indicates the z/OS Log Forwarder default installation directory accordingto the version of the z/OS Insight Pack.

Table 9. z/OS Log Forwarder default installation directory

Version of Operations Analyticsfor z Systems z/OS Log Forwarder default installation directory

Version 2.1.0 /usr/lpp/IBM/zscala/V2R1

Version 2.2.0 /usr/lpp/IBM/zscala/V2R2

Procedure

Update the configuration for each instance of the z/OS Log Forwarder bycompleting the steps that are applicable to your environment.1. If you installed the z/OS Log Forwarder in a different directory than the

directory that was used for the previous version, complete the followingupdates to reflect the new directory:a. Edit the z/OS Log Forwarder start procedure, and set the GLABASE

procedure variable to the directory where the startup.sh script is located.The following directory is the default installation directory for thestartup.sh script:/usr/lpp/IBM/zscala/V2R2/samples

b. Edit the environment configuration file to update the ZLF_HOME andZLF_LOG environment variables.The following default values indicate the default installation directories:

Environment variable Default value

ZLF_HOME /usr/lpp/IBM/zscala/V2R2

ZLF_LOG /usr/lpp/IBM/zscala/V2R2/samples


2. In the z/OS Log Forwarder data configuration file, change the references to thefollowing Operations Analytics for z Systems V2.1.0 properties as indicated:

Change the reference to this V2.1.0property: To this reference:

logGatherers.id.key dataGatherers.id.key

logGatherers.id.sourceType dataGatherers.id.dataSourceType

logGatherers.id.logPath dataGatherers.id.filePath

Tips:

v id represents an identifier that uniquely identifies the data gatherer.v key represents the data gatherer characteristic that is specified by the

property.Use one of the following options to update the file:

Use the Log Forwarder Configuration AssistantWhen the Configuration Assistant reads the definitions in the dataconfiguration file, it automatically replaces the obsolete references withthe correct ones.

Edit the data configuration file manuallyComplete the following steps in this order:a. Find all lines with the code logGatherers.id.key.

On these lines, replace logGatherers with dataGatherers.b. Find all lines with the code dataGatherers.id.sourceType.

On these lines, replace sourceType with dataSourceType.c. Find all lines with the code dataGatherers.id.logPath.

On these lines, replace logPath with filePath.


Configuring the z/OS Log Forwarder

Configure the z/OS Log Forwarder to collect and forward z/OS log data to theIBM Operations Analytics - Log Analysis server.

Before you begin

Review “Planning for configuration of the z/OS Log Forwarder” on page 26.

Also, complete the tasks that are described in “Preparing to configure the z/OSLog Forwarder” on page 76.

The following steps summarize how log data is annotated so that you can analyzeit in the IBM Operations Analytics - Log Analysis user interface.1. The z/OS Log Forwarder forwards log data to the IBM Operations Analytics -

Log Analysis server for ingestion.2. Log file splitters determine how to split the log data into records.3. Log record annotators annotate the log records.

About this task

Before you start the z/OS Log Forwarder, you must customize the following z/OSLog Forwarder configuration files:v Environment configuration file to define the z/OS Log Forwarder environmentv Data configuration file to define the sources of log data

Use the Log Forwarder Configuration Assistant to customize the z/OS LogForwarder data configuration and environment configuration files.

Restriction: If you collect data by using a data gatherer type that is provided by athird party organization in cooperation with IBM, you cannot use theConfiguration Assistant to configure the data gatherers of that type. Rather, youmust manually configure those data gatherers.

The Log Forwarder Configuration Assistant is a set of Interactive SystemProductivity Facility (ISPF) panels and programs. You run the Log ForwarderConfiguration Assistant from the ISPF Command Shell.

You can configure the z/OS Log Forwarder in either of the following ways:

Local hostThis configuration option creates and maintains a single set ofconfiguration files for the logical partition (LPAR) that you are logged into.

In the Configuration Assistant session, you configure the z/OS LogForwarder for the local LPAR. The resulting data configuration file andenvironment configuration file are stored in the local hierarchical filesystem (HFS) in a UNIX System Services directory, which is called theconfiguration directory.


Sysplex-wideThis configuration option creates and manages configuration files for allLPARs in the local sysplex.

In a single session of the Configuration Assistant, you configure the z/OSLog Forwarder for all LPARs in a sysplex. When you start theConfiguration Assistant, it discovers the LPARs that are active in thesysplex and lists them in a panel. You can then select an LPAR andconfigure the z/OS Log Forwarder for that LPAR.

All configuration files for all LPARs are stored in a single UNIX SystemServices directory, which is called the configuration directory. You caninclude this configuration directory in a shared HFS that is accessed by allLPARs in the sysplex, or you can copy the configuration files to directoriesin each LPAR as needed.

Preparing to configure the z/OS Log ForwarderBefore you configure the z/OS Log Forwarder, create the z/OS Log Forwarderstart procedure. Also, if applicable, prepare the z/OS NetView message providerfor use, and create the z/OS SMF real-time data provider start procedure.

Creating the z/OS Log Forwarder start procedureCreate the z/OS Log Forwarder start procedure by updating the sample procedureGLAPROC.

Before you begin

Review the following topics:v “z/OS Log Forwarder program files” on page 26v “Required authorities for z/OS Log Forwarder operations” on page 27v “z/OS Log Forwarder start procedure” on page 28

.

About this task

For information about the variables that you can update in the z/OS LogForwarder start procedure, see “Variable reference for the z/OS Log Forwarderstart procedure” on page 216.

Procedure

To create the z/OS Log Forwarder start procedure, complete the following steps:1. Copy the sample procedure GLAPROC to a user procedure library.

Tip: You can rename this procedure according to your installation conventions.Throughout this documentation, the name GLAPROC is used to mean thez/OS Log Forwarder start procedure.

2. Copy the sample job GLARACF to a user job library.3. To define a user ID and associate it with the GLAPROC procedure, update the

user copy of the GLARACF job according to the comments in the sample andto the following instructions:v If the user ID exists, comment out the ADDUSER statement.


v If a user ID other than GLALGF is to be associated with the GLAPROCprocedure, change the USER on the STDATA parameter.

4. Submit the updated user copy of the GLARACF job.5. Update the user copy of the GLAPROC procedure according to the comments

in the sample.

Preparing the z/OS NetView message provider for useIf you plan to collect NetView for z/OS message data, you must complete severalsteps to make the NetView message provider ready for use. The NetView messageprovider is defined in the REXX module GLANETV in the SGLACLST data set.

Before you begin

Review “NetView message provider” on page 45.

Procedure

To prepare the NetView message provider for use, complete the following steps:1. Ensure that the GLANETV module is placed in the DSICLD data set that is

defined in the NetView procedure. Also, ensure that the NetView autotask hasaccess to the GLANETV module.

2. In the CNMSTYLE member, specify the following information by usingcommon variables:

Information to specify How to specifyExample entry in CNMSTYLEmember

Indication of whether to start theNetView message provider in cold orwarm start mode

Specify either of the following valuesfor the COMMON.GLANETV.STARTvariable:

v C for cold start mode

v W for warm start mode, which isthe default mode

COMMON.GLANETV.START = W

Configuration directory for theNetView message provider

For the configuration directory,specify a partitioned data set (PDS)where the z/OS Log Forwarder canstore some information to keep trackof its progress in reading log data.

Specify the data set as the value oftheCOMMON.GLANETV.CONFIG.DIRvariable.

The default value is USER.CLIST.

COMMON.GLANETV.CONFIG.DIR =USER.CLIST

What to do next

Define a z/OS NetView Message gatherer to the z/OS Log Forwarder, and ensurethat the gatherer starts before the NetView message provider is started. For moreinformation, see Configuring the z/OS Log Forwarder to forward NetView forz/OS message data and “Starting and stopping the NetView message provider” onpage 104.

Configuring the z/OS Log Forwarder 77

Creating the z/OS SMF real-time data provider start procedureCreate the start procedure for the z/OS SMF real-time data provider by updatingthe sample procedure GLASMF, which is provided in the sample librarySGLASAMP.

Before you begin

Review “SMF real-time data provider” on page 39.

About this task

For information about the variables that you can update in the z/OS SMF real-timedata provider start procedure, see “Variable reference for the z/OS SMF real-timedata provider start procedure” on page 216.

Procedure

To create the z/OS SMF real-time data provider start procedure, complete thefollowing steps:1. Copy the sample procedure GLASMF from the sample library SGLASAMP to a

user procedure library.

Tip: You can rename this procedure according to your installation conventions.Throughout this documentation, the name GLASMF is used to mean the z/OSSMF real-time data provider start procedure.

2. Update the user copy of the GLASMF procedure according to the comments inthe sample.

What to do next

Define a z/OS UNIX Log File gatherer to the z/OS Log Forwarder to retrieve theSMF data, and ensure that the gatherer starts before the SMF real-time dataprovider is started. For more information, see Configuring the z/OS LogForwarder to forward SMF data and “Starting and stopping the SMF real-time dataprovider” on page 104.

Preparing the Log Forwarder Configuration Assistant for useBefore you start the Log Forwarder Configuration Assistant, you must prepare theConfiguration Assistant for use in your environment.

Preparing the Log Forwarder Configuration Assistant forsysplex-wide configuration

If you want to use the sysplex-wide configuration option to configure the z/OSLog Forwarder, you must prepare the Log Forwarder Configuration Assistant todiscover the logical partitions (LPARs) in a sysplex.

Procedure

Before you use the sysplex-wide configuration option, complete the followingsteps:1. Add the SGLALINK library to the LINKLIST library.2. Update the PARMLIB member IKJTSOxx to authorize the command GLAIGXCF,

as shown in the following example:


AUTHCMD NAMES( /* AUTHORIZED COMMANDS */ +GLAIGXCF /* SCALAZ CONFIG ASSISTANT */ +

3. Refresh the library lookaside (LLA) directory to include the new load module.4. Issue the SET IKJTSO=xx command to include the changes to the PARMLIB

member.

Customizing the Log Forwarder Configuration Assistant startprogram

When you run the Log Forwarder Configuration Assistant from ISPF, it starts byrunning the GLACONFG REXX sample that is included in theZSCALA.V2R2M0.SGLASAMP library. Verify that the variable values in theGLACONFG REXX sample are sufficient for your environment, or customize theGLACONFG REXX program if necessary.

About this task

Some variable values in the GLACONFG REXX sample are set to the defaultvalues of environment variables from the z/OS Log Forwarder environmentconfiguration file. The Log Forwarder Configuration Assistant uses the variablevalues from the environment configuration file to populate its environmentconfiguration panel, from which you configure the z/OS Log Forwarder for eachlogical partition (LPAR) in a sysplex. You might want to update these values foryour environment.

For example, if, in your LPARs, you use a level of Java that is different from thelevel that is specified in the environment configuration file, you can change thisvalue in the environment configuration file. Then, when you go to the environmentconfiguration panel of the Log Forwarder Configuration Assistant, your level ofJava is shown as the default value.

For more information about the variables that you can update, see “Variablereference for the environment configuration file” on page 214.

Procedure

To customize the GLACONFG REXX program, complete the following steps:1. Copy the GLACONFG REXX program from the sample library

ZSCALA.V2R2M0.SGLASAMP to a local CLIST library.2. If IBM Operations Analytics - Log Analysis is not installed in the default

libraries, you must update the values of the following variables in theGLACONFG REXX program to point to your installation libraries:CLISTLIB = "ZSCALA.V2R2M0.SGLACLST"PANELLIB = "ZSCALA.V2R2M0.SGLAPNL"MESSAGELIB = "ZSCALA.V2R2M0.SGLAMSG"

3. Update the value of the PROPFILE variable to point to the directory where thez/OS Log Forwarder environment and data configuration files are stored, asshown in the following example:PROPFILE = "/etc/IBM/zscala/V2R2"

4. Verify that the environment variables in the environment configuration file areset to the correct values, and update them if necessary.

Debug function for troubleshootingThe Log Forwarder Configuration Assistant includes a debug function for use byIBM Software Support.


You can enable this function by using the following variables in the GLACONFGREXX program:DBUGPATH = "/usr/lpp/IBM/zscala/V2R2/samples"DBUGSPEC = "OFF"

If requested by IBM Software Support, change the value of the DBUGSPECvariable to ON before you run the start program (GLACONFG REXX) for theConfiguration Assistant.

The DBUGPATH variable points to a UNIX System Services directory where theConfiguration Assistant writes debug information to files with names that beginwith “debug.” Ensure that the user who runs the Configuration Assistant has writeauthority for this directory.

Starting the Log Forwarder Configuration AssistantUse the Log Forwarder Configuration Assistant to customize the environmentconfiguration file and the data configuration file for the z/OS Log Forwarder.

Before you begin

Complete the tasks that are described in “Preparing the Log ForwarderConfiguration Assistant for use” on page 78.

Procedure1. Use the ISPF Command Shell to start the Log Forwarder Configuration

Assistant.2. Run one of the following commands, depending on the configuration option

that you want:

Option Description

Local host configuration ex ’your_clist_library(GLACONFG)’

Sysplex-wide configuration ex ’your_clist_library(GLACONFG)’’-SYSPLEX’

What to do next

The Log Forwarder Configuration Assistant includes online help for each panel.

For more information about the fields and values in the panels, see “Propertyreference for the data configuration file” on page 181.

Defining the z/OS Log Forwarder environmentIn the z/OS Log Forwarder environment configuration file, you must updateseveral environment variables that are used by the z/OS Log Forwarder startprocedure (startup.sh script).

About this task

The following directory contains the sample environment configuration file:/usr/lpp/IBM/zscala/V2R2/samples


For more information about the environment variables, see “Variable reference forthe environment configuration file” on page 214.

Remember: Use the Log Forwarder Configuration Assistant to update theenvironment configuration file. The following instructions describe how tomanually update the file if you prefer that method.

Procedure

If you choose to update the file manually rather than by using the Log ForwarderConfiguration Assistant, complete the following steps.1. Copy the zlf.conf sample file to the directory that contains the z/OS Log

Forwarder configuration files.2. If the directory is to contain z/OS Log Forwarder configuration files for more

than one system, or logical partition (LPAR), rename the file toSYSPLEX.SYSNAME.zlf.conf, where SYSNAME is the name of the system (LPAR)where the z/OS Log Forwarder runs, and SYSPLEX is the name of the sysplex(or monoplex) in which that system is located.The values of both SYSPLEX and SYSNAME must be in all uppercase.

Tip: To determine the sysplex and system names of a logical partition (LPAR),issue the following z/OS UNIX System Services commands:sysvar SYSPLEXsysvar SYSNAME

3. Update the file content as appropriate according to “Variable reference for theenvironment configuration file” on page 214.

Defining the sources of log dataThe z/OS Log Forwarder data configuration file defines data sources, and thesystem that receives data from these sources, to the z/OS Log Forwarder.

Before you begin

Use the Log Forwarder Configuration Assistant to update the data configurationfile. The procedure in this topic describes how to manually update the file, if youprefer that method.

The following directory contains the sample data configuration file:/usr/lpp/IBM/zscala/V2R2/samples

For more information about the properties in the data configuration file, see“Property reference for the data configuration file” on page 181.

In the Log Forwarder Configuration Assistant, the Data Source File Path isequivalent to the dataGatherers.id.filePath property in the data configurationfile. The value is a unique identifier that represents the data origin.


Field name in Log ForwarderConfiguration Assistant Property in data configuration file More information about the value

Data Source File Path dataGatherers.id.filePath The value is dependent on the typeof data gatherer. See the descriptionof the dataGatherers.id.filePathproperty in the following topics:

v WAS_HPEL

v ZOS_DATA_SET

v ZOS_JOB_LOG

v ZOS_NETVIEW

v ZOS_SYS

v ZOS_UNIX_FILE

Procedure

If you choose to update the file manually rather than by using the Log ForwarderConfiguration Assistant, complete the following steps.1. Copy the config.properties sample file to the directory that contains the z/OS

Log Forwarder configuration files.2. If the directory is to contain z/OS Log Forwarder configuration files for more

than one system, or logical partition (LPAR), rename the file toSYSPLEX.SYSNAME.config.properties, where SYSNAME is the name of the system(LPAR) where the z/OS Log Forwarder runs, and SYSPLEX is the name of thesysplex (or monoplex) in which that system is located.The values of both SYSPLEX and SYSNAME must be in all uppercase.

Tip: To determine the sysplex and system names of a logical partition (LPAR),issue the following z/OS UNIX System Services commands:sysvar SYSPLEXsysvar SYSNAME

3. Update the file content as appropriate according to “Property reference for thedata configuration file” on page 181.

Correlation of the data to be analyzed with the associateddata gatherer types, data source types, and dashboards

For the data that you want to analyze, you must know which data gatherer typesto define during the configuration of the z/OS Log Forwarder. The data gatherertypes are associated with data source types that are defined in the Log Analysisserver and UI.

The data source types are also associated with IBM-provided Custom SearchDashboard applications. You can customize these dashboard applications, whichare available in the Log Analysis UI, to help you analyze the data and troubleshootproblems.

The following sections correlate each type of data that you can analyze with thefollowing information:v The Insight Packs that must be installed to analyze the datav The associated data gatherer types that you must define during the

configuration of the z/OS Log Forwarder


v The associated data source types that are defined in the Log Analysis server andUI

v The associated Custom Search Dashboard applications from the Insight Pack thatyou can customize and use in the Log Analysis UI

CICS Transaction Server for z/OS data

Table 10. Analyzing CICS Transaction Server for z/OS data: required Insight Packs andassociated data gatherer types, data source types, and dashboards

RequiredInsight Packs

Data gatherertypes Data source types Dashboard applications

z/OS SYSLOGInsight Pack

v ZOS_SYS v zOS-SYSLOG-Console

v zOS-SYSLOG-SDSF

“CICS Transaction Serverfor z/OS Dashboard” onpage 142

For EYULOG andMSGUSR log:

v ZOS_JOB_LOG

v ZOS_UNIX_FILE

For EYULOG:

v zOS-CICS-EYULOG

v zOS-CICS-EYULOGDMY

v zOS-CICS-EYULOGYMD

For MSGUSR log:

v zOS-CICS-MSGUSR

v zOS-CICS-MSGUSRDMY

v zOS-CICS-MSGUSRYMD

None

z/OS SMFInsight Pack

v ZOS_UNIX_FILE v zOS-SMF110_E

v zOS-SMF110_S_10

“z/OS SMF Custom SearchDashboard applications” onpage 136

DB2 for z/OS data

Table 11. Analyzing DB2 for z/OS data: required Insight Packs and associated data gatherertypes, data source types, and dashboards





v zOS-SYSLOG-SDSF

“DB2 for z/OS Dashboard”on page 143

IMS for z/OS data

Table 12. Analyzing IMS for z/OS data: required Insight Packs and associated data gatherertypes, data source types, and dashboards





v zOS-SYSLOG-SDSF

“IMS for z/OS Dashboard”on page 143


MQ for z/OS data

Table 13. Analyzing MQ for z/OS data: required Insight Packs and associated data gatherertypes, data source types, and dashboards





v zOS-SYSLOG-SDSF

“MQ for z/OS Dashboard”on page 144

NetView for z/OS messages

Table 14. Analyzing NetView for z/OS messages: required Insight Packs and associateddata gatherer types, data source types, and dashboards



z/OS NetworkInsight Pack

v ZOS_NETVIEW v zOS-NetView “NetView for z/OSDashboard” on page 136

Network data

Network data can include, for example, data from UNIX System Services systemlog (syslogd) or z/OS Communications Server.

Table 15. Analyzing network data: required Insight Packs and associated data gatherertypes, data source types, and dashboards


Data gatherertypes Data source types

Custom Search Dashboardapplication

z/OS NetworkInsight Pack

None None “z/OS NetworkingDashboard” on page 135



v zOS-SYSLOG-SDSF

v “SYSLOG for z/OS TimeComparison Dashboard”on page 141

v “SYSLOG for z/OSDashboard” on page 141

v ZOS_UNIX_FILE v zOS-syslogd None

Security data

Table 16. Analyzing security data: required Insight Packs and associated data gatherertypes, data source types, and dashboards


Data gatherertypes Data source types

Custom Search Dashboardapplication



v zOS-SYSLOG-SDSF

“Security for z/OSDashboard” on page 144


v ZOS_UNIX_FILE zOS-SMF80 (RACF) “z/OS SMF Custom SearchDashboard applications” onpage 136


SMF data

Table 17. Analyzing SMF data: required Insight Packs and associated data gatherer types,data source types, and dashboards




v ZOS_UNIX_FILE v zOS-SMF30 (z/OS jobperformance)

v zOS-SMF80 (RACF)

v zOS-SMF110_E (CICSTransaction Serverfor z/OS)

v zOS-SMF110_S_10 (CICS TransactionServer for z/OS)

v zOS-SMF120(WebSphereApplication Serverfor z/OS)

“z/OS SMF Custom SearchDashboard applications” onpage 136



v zOS-SYSLOG-SDSF

None

WebSphere Application Server for z/OS data

Table 18. Analyzing WebSphere Application Server for z/OS data: required Insight Packsand associated data gatherer types, data source types, and dashboards



WebSphereApplicationServer for z/OSInsight Pack

v ZOS_JOB_LOG

v ZOS_UNIX_FILE

v zOS-WAS-SYSOUT

v zOS-WAS-SYSPRINT

“WebSphere ApplicationServer for z/OS CustomSearch Dashboardapplications” on page 135v WAS_HPEL v zOS-WAS-HPEL


v ZOS_UNIX_FILE v zOS-SMF120 “z/OS SMF Custom SearchDashboard applications” onpage 136

z/OS system log (SYSLOG) data

Table 19. Analyzing z/OS SYSLOG data: required Insight Packs and associated datagatherer types, data source types, and dashboards





v zOS-SYSLOG-SDSF

v “SYSLOG for z/OS TimeComparison Dashboard”on page 141

v “SYSLOG for z/OSDashboard” on page 141

Data configuration file overviewEach line in the z/OS Log Forwarder data configuration file must be a key-valuepair, a blank line, or a comment line.


Remember: Use the Log Forwarder Configuration Assistant to update the dataconfiguration file. This information is provided only for your reference.

Key-value pairA line that contains a key and a value with the equal sign = between them.

The key contains all of the characters in the line (starting with the firstnon-white-space character) up to, but not including, either of the followingcharacters:v The first equal sign =v The first white-space character other than the line terminator

Any white-space characters after the key are ignored. The equal sign = isignored, and any white-space characters after the equal sign are alsoignored.

All remaining characters on the line become part of the associated valuestring. If no characters remain, the value is the empty string "".

Blank lineA line that contains only white-space characters. Space, tab, and form feedcharacters are considered to be white-space characters.

The z/OS Log Forwarder ignores blank lines.

Comment lineA line that has either the number sign # or exclamation mark ! as its firstnon-white-space character.

The z/OS Log Forwarder ignores comment lines.

Evaluation example of a data configuration fileThis evaluation example of a z/OS Log Forwarder data configuration file is a basicexample for a test or proof-of-concept environment. Each line of the example isexplained after the example.

Assumptions that apply to this configuration example

The following assumptions apply to this configuration:v You do not need to verify the identity of the IBM Operations Analytics - Log

Analysis host by validating its certificate.v The default user unityadmin is used to access IBM Operations Analytics - Log

Analysis.v The data sources are created in default collections that are determined by the

IBM Operations Analytics - Log Analysis server.

ExampleLine----1 # Define the data destination that will receive the logs2 logReceiver.host = myhost.mydomain.com34 # Define a z/OS SYSLOG gatherer5 dataGatherers.1.type = ZOS_SYS6 dataGatherers.1.dataSourceName = myhost_syslog78 # Define a job data gatherer for DAYTRADS server9 dataGatherers.2.type = ZOS_JOB_LOG10 dataGatherers.2.filePath = DAYTRADS/SYSPRINT


11 dataGatherers.2.dataSourceName = myhost_daytrads_sysprint12 dataGatherers.2.dataSourceType = zOS-WAS-SYSPRINT13 dataGatherers.2.jobName = DAYTRADS14 dataGatherers.2.ddName = SYSPRINT

Explanation of each line

Line 1 A comment line. It is ignored by the z/OS Log Forwarder.

Line 2 Contains a property with a key of logReceiver.host and a value ofmyhost.mydomain.com, which specifies the host name for the datadestination. Because no logReceiver.type is specified, the z/OS LogForwarder assumes that myhost.mydomain.com is an IBM OperationsAnalytics - Log Analysis server.

Line 3 A blank line. It is ignored by the z/OS Log Forwarder.


Line 5 Contains a property with a key of dataGatherers.1.type and a value ofZOS_SYS.

This property specifies that log data must be gathered from the localSYSLOG by using the user exits.

Line 6 Contains a property with a key of dataGatherers.1.dataSourceName and avalue of myhost_syslog.

This property specifies the following information:v That the z/OS Log Forwarder must create the data source in IBM

Operations Analytics - Log Analysis if the data source is not alreadyconfigured

v The name to use for the data source

Line 7 A blank line. It is ignored by the z/OS Log Forwarder.


Line 9 Contains a property with a key of dataGatherers.2.type and a value ofZOS_JOB_LOG.

The identifier value 2 indicates that this property applies to a different datagatherer element than the property on line 5, which has the identifier value1.

This property specifies that WebSphere Application Server for z/OS logdata must be gathered from a job log.

Line 10Contains a property with a key of dataGatherers.2.filePath and a value ofDAYTRADS/SYSPRINT.

The identifier value 2 indicates that this property applies to the same datagatherer element as the property on line 9.

This property specifies the file path that the z/OS Log Forwarder uses tosend log data from this job log to IBM Operations Analytics - Log Analysis.

Line 11Contains a property with a key of dataGatherers.2.dataSourceName and avalue of myhost_daytrads_sysprint.

This property specifies the following information:


v That the z/OS Log Forwarder must create the data source in IBMOperations Analytics - Log Analysis if the data source is not alreadyconfigured

v The name to use for the data source

Line 12Contains a property with a key of dataGatherers.2.dataSourceType and avalue of zOS-WAS-SYSPRINT.

This property specifies the type of data source that must be created in IBMOperations Analytics - Log Analysis if the data source is not alreadyconfigured.

Line 13Contains a property with a key of dataGatherers.2.jobName and a value ofDAYTRADS.

This property specifies the name of the job from which to gather job logdata. In this example, the job is a servant job for the Day Traderapplication.

Line 14Contains a property with a key of dataGatherers.2.ddName and a value ofSYSPRINT.

This property specifies the name of the data set from which to gather joblog data.

Production example of a data configuration fileThis example of a z/OS Log Forwarder data configuration file is a more realisticexample for a secure, production environment.

Configuration differences from the evaluation example

This production configuration differs from the evaluation configuration in thefollowing primary ways:v Collections are specified for each data source, as indicated in lines 16 and 23. In

the evaluation sample, the data sources are created in default collections.v This configuration is more secure for the following reasons:

Lines 2 - 5

– The identity of the IBM Operations Analytics - Log Analysis host isverified by comparing the security certificate that is provided by thissystem with the security certificate that is stored in the specifiedtruststore.

– The truststore password is encrypted.

Lines 9 - 11

– A unique and more secure user name and password are used toaccess IBM Operations Analytics - Log Analysis.

– The IBM Operations Analytics - Log Analysis password is encrypted.

ExampleLine----1 # Validate the Log Analysis server security certificate2 systemInfo.trustAllCertificates = false,3 systemInfo.truststore = /my/truststore.jks4 systemInfo.truststorePassword = fa938bc92936bcbd9f4934ae9aa5a5be5 systemInfo.truststorePasswordInitializationVector = 32787ec89af848195c509c539d93f1d9


67 # Define the Log Analytics server that will receive the logs8 logReceiver.host = myhost.mydomain.com9 logReceiver.username = zlfuser10 logReceiver.password = 40aafda1ddba7d7ae08ebbe06aabfd6f11 logReceiver.passwordInitializationVector = b53ef6c25d61b87a6335e484d437e3b91213 # Define a z/OS SYSLOG gatherer14 dataGatherers.1.type = ZOS_SYS15 dataGatherers.1.dataSourceName = myhost_syslog16 dataGatherers.1.collectionName = zOS-SYSLOG-Console-Collection1718 # Define a z/OS Job Log gatherer for DAYTRADS server19 dataGatherers.2.type = ZOS_JOB_LOG20 dataGatherers.2.filePath = DAYTRADS/SYSPRINT21 dataGatherers.2.dataSourceName = myhost_daytrads_sysprint22 dataGatherers.2.dataSourceType = zOS-WAS-SYSPRINT23 dataGatherers.2.collectionName = zOS-WAS-SYSPRINT-Collection24 dataGatherers.2.jobName = DAYTRADS25 dataGatherers.2.ddName = SYSPRINT

Manually configuring the z/OS Log Forwarder to forward dataA best practice is to use the Log Forwarder Configuration Assistant to configurethe z/OS Log Forwarder. However, you can manually configure the z/OS LogForwarder to forward data, if you prefer that method.

Before you begin

The procedures in this section describe how to manually configure the z/OS LogForwarder to forward data from various sources. To use the Log ForwarderConfiguration Assistant instead, see “Starting the Log Forwarder ConfigurationAssistant” on page 80.

For more information about the properties in the data configuration file, see“Property reference for the data configuration file” on page 181.

Configuring the z/OS Log Forwarder to forward CICS TransactionServer for z/OS log dataFor CICS Transaction Server for z/OS log data to be sent from CICS TransactionServer for z/OS, you must configure the z/OS Log Forwarder to forward the CICSTransaction Server for z/OS job logs or UNIX files.

Before you begin

By default, the logs are stored in JES spool. However, they can also be configuredto be stored in UNIX files.

About this task

For information about the relevant data gatherer properties for CICS TransactionServer for z/OS data, see the following topics:v “z/OS Job Log gatherer properties” on page 192v “z/OS UNIX Log File gatherer properties” on page 198

Procedure

To update the data configuration file manually, rather than by using the LogForwarder Configuration Assistant, use the following example as a guide. In thisexample, the MSGUSR and EYULOG logs must be forwarded from the CICSregion that is named CMAS51.


dataGatherers.1.type = ZOS_JOB_LOGdataGatherers.1.dataSourceType = zOS-CICS-MSGUSRdataGatherers.1.dataSourceName = SYSNAME_CMAS51_MSGUSRdataGatherers.1.jobName = CMAS51dataGatherers.1.ddName = MSGUSRdataGatherers.2.type = ZOS_JOB_LOGdataGatherers.2.dataSourceType = zOS-CICS-EYULOGdataGatherers.2.dataSourceName = SYSNAME_CMAS51_EYULOGdataGatherers.2.jobName = CMAS51dataGatherers.2.ddName = EYULOG

Configuring the z/OS Log Forwarder to forward NetView for z/OSmessage dataFor NetView for z/OS message data to be sent from the z/OS NetView messageprovider, you must configure the z/OS Log Forwarder to forward the NetView forz/OS message data.

Before you begin

Review “NetView message provider” on page 45 and “Preparing the z/OSNetView message provider for use” on page 77.

About this task

For information about the relevant data gatherer properties for NetView for z/OSmessage data, see “z/OS NetView Message gatherer properties” on page 195.

Procedure

To update the data configuration file manually, rather than by using the LogForwarder Configuration Assistant, use the following example as a guide. In thisexample, NetView for z/OS message data must be forwarded for the NetViewdomain CNM01.dataGatherers.1.type = ZOS_NETVIEWdataGatherers.1.dataSourceName = SYSNAME_CNM01dataGatherers.1.domainName = CNM01

Configuring the z/OS Log Forwarder to forward SMF dataFor SMF data to be sent from the z/OS SMF real-time data provider, you mustconfigure the z/OS Log Forwarder to forward the z/OS SMF real-time dataprovider data transfer files.

Before you begin

Review “SMF real-time data provider” on page 39.

By default, the data transfer files are stored in the following UNIX System Servicesdirectory:/etc/IBM/zscala/V2R2/smf

About this task

For information about the relevant data gatherer properties for SMF data, see“z/OS UNIX Log File gatherer properties” on page 198.


Procedure

To update the data configuration file manually, rather than by using the LogForwarder Configuration Assistant, use the following example as a guide. In thisexample, the data transfer files are in the default directory /etc/IBM/zscala/V2R2/smf.dataGatherers.1.type = ZOS_UNIX_FILEdataGatherers.1.dataSourceType = zOS_SMF30dataGatherers.1.dataSourceName = SYSNAME_SMF30dataGatherers.1.filePath = /etc/IBM/zscala/V2R2/smf/SMF30_*.csv

Configuring the z/OS Log Forwarder to forward z/OS SYSLOGdataFor z/OS SYSLOG data to be sent, you must configure the z/OS Log Forwarder toforward the z/OS SYSLOG data.

Before you begin

Review “User exits for collecting z/OS SYSLOG data” on page 36.

About this task

For information about the relevant data gatherer properties for z/OS SYSLOGdata, see “z/OS SYSLOG gatherer properties” on page 196.

Procedure

To update the data configuration file manually, rather than by using the LogForwarder Configuration Assistant, use the following example as a guide:dataGatherers.1.type = ZOS_SYSdataGatherers.1.dataSourceName = SYSNAME_SYSLOG

Configuring the z/OS Log Forwarder to forward syslogd dataFor UNIX System Services system log (syslogd) data to be sent, you mustconfigure the z/OS Log Forwarder to forward the syslogd UNIX files.

Before you begin

By default, the syslogd files are stored in the following UNIX System Servicesdirectory:/tmp/syslogd

About this task

For information about the relevant data gatherer properties for syslogd data, see“z/OS UNIX Log File gatherer properties” on page 198.

Procedure

To update the data configuration file manually, rather than by using the LogForwarder Configuration Assistant, use the following example as a guide. In thisexample, the syslogd files are in the default directory /tmp/syslogd, and the threefiles auth.log, debug.log, and error.log must be forwarded.dataGatherers.1.type = ZOS_UNIX_FILEdataGatherers.1.dataSourceType = zOS-syslogddataGatherers.1.dataSourceName = SYSNAME_syslogd_authdataGatherers.1.filePath = /tmp/syslogd/auth.log


dataGatherers.2.type = ZOS_UNIX_FILEdataGatherers.2.dataSourceType = zOS-syslogddataGatherers.2.dataSourceName = SYSNAME_syslogd_debugdataGatherers.2.filePath = /tmp/syslogd/debug.logdataGatherers.3.type = ZOS_UNIX_FILEdataGatherers.3.dataSourceType = zOS-syslogddataGatherers.3.dataSourceName = SYSNAME_syslogd_errordataGatherers.3.filePath = /tmp/syslogd/error.log

Configuring the z/OS Log Forwarder to forward WebSphereApplication Server for z/OS log dataFor WebSphere Application Server for z/OS log data to be sent from WebSphereApplication Server for z/OS, you must configure the z/OS Log Forwarder toforward the WebSphere Application Server for z/OS job logs or UNIX files.

Before you begin

Review “WebSphere Application Server for z/OS log data” on page 45.

By default, the logs are stored in JES spool. However, they can also be configuredeither to be stored in UNIX files or to be retrieved by using the High PerformanceExtensible Logging (HPEL) API.

About this task

For information about the relevant data gatherer properties for WebSphereApplication Server for z/OS data, see the following topics:v “WAS HPEL gatherer properties” on page 187v “z/OS Job Log gatherer properties” on page 192v “z/OS UNIX Log File gatherer properties” on page 198

Procedure

To update the data configuration file manually, rather than by using the LogForwarder Configuration Assistant, use one of the following examples as a guide:

Example 1In this example, the SYSPRINT logs must be forwarded for the controller,adjunct, and servant regions of the server with short name BBOSABC.dataGatherers.1.type = ZOS_JOB_LOGdataGatherers.1.dataSourceType = zOS-WAS-SYSPRINTdataGatherers.1.dataSourceName = SYSNAME_BBOSABC_SYSPRINTdataGatherers.1.jobName = BBOSABCdataGatherers.1.ddName = SYSPRINTdataGatherers.2.type = ZOS_JOB_LOGdataGatherers.2.dataSourceType = zOS-WAS-SYSPRINTdataGatherers.2.dataSourceName = SYSNAME_BBOSABCA_SYSPRINTdataGatherers.2.jobName = BBOSABCAdataGatherers.2.ddName = SYSPRINTdataGatherers.3.type = ZOS_JOB_LOGdataGatherers.3.dataSourceType = zOS-WAS-SYSPRINTdataGatherers.3.dataSourceName = SYSNAME_BBOSABCS_SYSPRINTdataGatherers.3.jobName = BBOSABCSdataGatherers.3.ddName = SYSPRINT

Example 2In this example, the SYSPRINT logs must be forwarded for the controllerand servant regions of the server with short name BBOSABC in cell BBOCELLand node BBONODE. The server output is redirected to the /u/myLogDirdirectory.


dataGatherers.1.type = ZOS_UNIX_FILEdataGatherers.1.dataSourceType = zOS-WAS-SYSPRINTdataGatherers.1.dataSourceName = SYSNAME_BBOSABC_SYSPRINTdataGatherers.1.filePath =

/u/myLogDir/BBOCELL.BBONODE.BBOSABC.BBOSABC.????????.CTL.??????.??????.SYSPRINT.txtdataGatherers.2.type = ZOS_UNIX_FILEdataGatherers.2.dataSourceType = zOS-WAS-SYSPRINTdataGatherers.2.dataSourceName = SYSNAME_BBOSABCS_SYSPRINTdataGatherers.2.filePath =

/u/myLogDir/BBOCELL.BBONODE.BBOSABC.BBOSABCS.????????.SR.??????.??????.SYSPRINT.txt

Example 3In this example, the HPEL log and trace logs must be forwarded for serverabcServer in cell BBOCELL and node BBONODE.dataGatherers.1.type = WAS_HPELdataGatherers.1.dataSourceType = zOS-WAS-HPELdataGatherers.1.dataSourceName = SYSNAME_BBOSABCdataGatherers.1.logDirectory =

/u/WebSphere/V8R5/bbocell/bbonode/AppServer/profiles/default/logs/abcServerdataGatherers.1.traceDirectory =

/u/WebSphere/V8R5/bbocell/bbonode/AppServer/profiles/default/logs/abcServer

Configuring the data destinationConfigure the z/OS Log Forwarder to forward data to an IBM OperationsAnalytics - Log Analysis server directly or by using the scalable data collectionarchitecture.

Before you begin

Review “Data forwarding” on page 50.

About this task

For information about log receiver properties, see “Log receiver properties” onpage 183.

Procedure

To update the data configuration file manually, rather than by using the LogForwarder Configuration Assistant, use the following examples as a guide.

To forward data directlylogReceiver.host = myhost.mydomain.comlogReceiver.username = zlfuserlogReceiver.password = 40aafda1ddba7d7ae08ebbe06aabfd6flogReceiver.passwordInitializationVector = b53ef6c25d61b87a6335e484d437e3b9

The logReceiver.host property specifies that myhost.mydomain.com is thehost name of the Log Analysis server.

The logReceiver.username property specifies that zlfuser is the user namefor accessing Log Analysis to ingest data. The logReceiver.password andlogReceiver.passwordInitializationVector properties specify theencrypted password. For more information about encrypting passwords,see “Encrypting passwords in the data configuration file” on page 94.

To forward data by using the scalable data collection architecturelogReceiver.type = LOGSTASHlogReceiver.host = myhost.mydomain.comlogReceiver.port = 8080logReceiver.secureTransport = false

The logReceiver.type property specifies that a Logstash server in thereceiver cluster receives the data.


The logReceiver.host property specifies that myhost.mydomain.com is thehost name of a Logstash server in the receiver cluster. However, this hostname might be the host name of a load balancer server, such as HAProxy,that forwards the data to the appropriate Logstash server in the receivercluster.

The logReceiver.port property specifies that 8080 is the port on which theLogstash or load balancer server is listening.

The logReceiver.secureTransport property specifies whether to useTransport Layer Security (TLS). Specifying a value of true for this propertyindicates that HTTPS (secure), rather than HTTP, is to be used forcommunication. The following conditions must also be met for securecommunication:v The Logstash or load balancer server to which data is forwarded must

also be configured to use TLS.v The z/OS Log Forwarder must be configured to either trust all

certificates or verify the identity of the Logstash or load balancer server.For more information, see “Verifying the identity of the target server forforwarding data” on page 99.

Encrypting passwords in the data configuration fileTo prevent unauthorized use of the passwords that are stored in the z/OS LogForwarder data configuration file, you must encrypt the passwords. To encrypt thepasswords, you can use either the Log Forwarder Configuration Assistant or apassword encryption utility that is provided with the z/OS Log Forwarder.

About this task

Passwords in the z/OS Log Forwarder data configuration file can be stored in theclear or as encrypted passwords.

The following example shows that the passwords for the IBM Operations Analytics- Log Analysis server and for the z/OS Log Forwarder truststore are stored in theclear:systemInfo.truststorePassword = hUP2eweflogReceiver.password = ba5anAch

These passwords are not secure, even if access to the data configuration file issecured through access control software that is part of the operating system.

To prevent unauthorized use of the passwords that are stored in the dataconfiguration file, encrypt the passwords. When an encrypted password shows inthe data configuration file, the initialization vector that is used to seed theencryption must also be present. The presence of an initialization vector indicatesto the z/OS Log Forwarder that the corresponding password is encrypted, asshown in the following example:systemInfo.truststorePassword = fa938bc92936bcbd9f4934ae9aa5a5besystemInfo.truststorePasswordInitializationVector = 32787ec89af848195c509c539d93f1d9logReceiver.password = 40aafda1ddba7d7ae08ebbe06aabfd6flogReceiver.passwordInitializationVector = b53ef6c25d61b87a6335e484d437e3b9

Encrypted passwords and initialization vectors show in the data configuration fileas hex string encoded values. In these values, each pair of characters specifies thehex value of a byte in the password or initialization vector.


Procedure

To encrypt passwords in the z/OS Log Forwarder data configuration file, useeither of the following methods.v Use the Log Forwarder Configuration Assistant, which encrypts passwords if

they are provided.v Use the password encryption utility, which is the encrypt.sh script that is in the

/usr/lpp/IBM/zscala/V2R2/samples directory.

To encrypt a password by using the encrypt.sh script, issue the followingcommand, where environment_config_directory is the directory that contains theenvironment configuration file, and password_to_encrypt is the password that youwant to encrypt:encrypt.sh environment_config_directory password_to_encrypt

The following example further illustrates how to issue the command and includessample values:/usr/lpp/IBM/zscala/V2R2/samples/encrypt.sh /usr/lpp/IBM/zscala/V2R2 u3c0UJmo

The command response is written to standard output (STDOUT). As shown in thefollowing example, it contains the encrypted password and the initialization vectorvalues for you to copy to the z/OS Log Forwarder data configuration file:encryptedPassword=a9014898a67d83a21c4a734ab6e52853initializationVector=e857540f66881463cc0d5192caadfcd1

If you are running the encryption utility for the first time, some informationalmessages are also written to standard error (STDERR). These messages indicatethat a keystore was created and that a secret key was created in the keystore.The keystore file, which is named keystore, is placed in the z/OS Log Forwarderworking directory. Because the z/OS Log Forwarder uses the secret key in thekeystore to decrypt the passwords, do not delete the keystore file. If the keystorefile is deleted, any encrypted passwords must be re-encrypted with a new secretkey.

Verifying file path values for rolling logsThe z/OS Log Forwarder includes a file path pattern utility to help you verify thefile path values for any rolling logs. The utility can determine which files on asystem are to be processed by each configured z/OS UNIX Log File gatherer.

Before you begin

For information about rolling logs or file path patterns, see “Data collection from arolling z/OS UNIX log” on page 47.

Procedure

To verify the file path values for z/OS UNIX Log File gatherers that are specifiedin the data configuration file, complete the following step:

Issue the following command in the logical partition (LPAR) where the z/OS LogForwarder runs:checkFilePattern.sh configuration_directory


The variable configuration_directory represents the directory that contains both thedata configuration file and the environment configuration file.The following example further illustrates how to issue the command and includessample values:/usr/lpp/IBM/zscala/V2R2/samples/checkFilepattern.sh /usr/lpp/IBM/zscala/V2R2

Optionally, a data gatherer identifier can be specified so that the file path for onlythe specified data gatherer is checked. The following example shows that the datagatherer identifier 9 is specified:/usr/lpp/IBM/zscala/V2R2/samples/checkFilepattern.sh /usr/lpp/IBM/zscala/V2R2 9

The command response is written to standard output (STDOUT). As shown in thefollowing example, it contains a list of all files that match each file path value:INFO: GLAB021I The file path pattern/u/myLogDir/BBOCELL.BBONODE.BBOSAPP.BBOSAPPS.????????.SR.??????.??????.SYSPRINT.txtfor data gatherer identifier 5 resolves to the following files:/u/myLogDir/BBOCELL.BBONODE.BBOSAPP.BBOSAPPS.STC00036.SR.140929.170703.SYSPRINT.txt/u/myLogDir/BBOCELL.BBONODE.BBOSAPP.BBOSAPPS.STC00158.SR.140929.193451.SYSPRINT.txt/u/myLogDir/BBOCELL.BBONODE.BBOSAPP.BBOSAPPS.STC00252.SR.141006.134949.SYSPRINT.txtINFO: GLAB021I The file path pattern/u/myLogDir/BBOCELL.BBONODE.BBOSAPP.BBOSAPPS.????????.SR.??????.??????.SYSOUT.txtfor data gatherer identifier 7 resolves to the following files:/u/myLogDir/BBOCELL.BBONODE.BBOSAPP.BBOSAPPS.STC00036.SR.140929.170703.SYSOUT.txt/u/myLogDir/BBOCELL.BBONODE.BBOSAPP.BBOSAPPS.STC00158.SR.140929.193451.SYSOUT.txt/u/myLogDir/BBOCELL.BBONODE.BBOSAPP.BBOSAPPS.STC00252.SR.141006.134949.SYSOUT.txt

The following example shows the command response that is written for the datagatherer if no files match a pattern:WARNING: GLAB022W The file path pattern/u/myLogDir/BBOCELL.BBONODE.BBOSAPP.BBOSAPPS.????????.SR.??????.??????.SYSPRINT.txtfor data gatherer identifier 6 resolves to no files.

Defining multiple data gatherers in a single z/OS Job Loggatherer definition

In defining a z/OS Job Log gatherer, you can use wildcard characters in the jobname to define multiple data gatherers in a single definition.

About this task

The following wildcard characters are valid in the job name value, which is thevalue of the dataGatherers.id.jobName property in the data configuration file:

Wildcard character What the character represents

? Any single character

* Any sequence of characters, including an emptysequence

If you use wildcard characters in the job name, the job name value becomes apattern, and the z/OS Job Log gatherer definition becomes a template. When thez/OS Log Forwarder is started, it searches the JES spool for job names that matchthe pattern, and it creates a separate data gatherer for each unique job name that itdiscovers. After the z/OS Log Forwarder initialization is complete, the z/OS LogForwarder continues to monitor the job names on the JES spool. As it discoversnew job names that match the pattern, it creates more data gatherers by using thesame z/OS Job Log gatherer template.


For example, if the job name value is BBOS???S, and the JES spool contains thefollowing jobs, two data gatherers are created, one for job name BBOSABCS and onefor job name BBOSDEFS:JOBNAME JobIDBBODMGR STC00586BBODMGRS STC00588BBODMNC STC00587BBON001 STC00589BBOSABC STC00590BBOSABC STC00600BBOSABCS STC00592BBOSABCS STC00602BBOSDEF STC00594BBOSDEFS STC00596BBOSDEFS STC00598GLAPROC STC00661SYSLOG STC00552

Tip: To avoid gathering data from job logs that you do not intend to gather from,ensure that the job name pattern is not too broad.

Each resulting data gatherer is based on the template and is assigned the followingvalues:

Template property Data gatherer value

dataGatherers.id.collectionName(collection name)

The template value for the collection name

dataGatherers.id.dataSourceName(data source name)

One of the following values:

v If the template contains a value for thedata source name, _jobName_ddName isappended to that value for each resultingdata gatherer. The jobName is thediscovered job name, and the ddName isthe value of the dataGatherers.id.ddNameproperty.

v If the template does not contain a valuefor the data source name, the resultingdata gatherers do not have a data sourcename.

Important: The best practice is to provide atemplate value for the data source name. Ifyou do not provide this value, you mustcreate data sources in IBM OperationsAnalytics - Log Analysis for all possiblematching job names before you start thez/OS Log Forwarder.

dataGatherers.id.dataSourceType(data source type)

The template value for the data source type

dataGatherers.id.ddName(ddname)

The template value for the data definitionname (ddname)


Template property Data gatherer value

dataGatherers.id.filePath(file path)

One of the following values:

v If the template contains a value for the filepath, /jobName/ddName is appended to thatvalue for each resulting data gatherer. ThejobName is the discovered job name, andthe ddName is the value of thedataGatherers.id.ddName property.

v If the template does not contain a valuefor the file path, the resulting datagatherers have a file path value ofjobName/ddName.

dataGatherers.id.jobName(job name)

The discovered job name, without wildcardcharacters

dataGatherers.id.timeZone(time zone)

The template value for the time zone

dataGatherers.id.type(type)

The template value for the type

For more information about the z/OS Job Log gatherer definition, see “z/OS JobLog gatherer properties” on page 192.

Loading historical data in batch mode for analysisYou can use the IBM Operations Analytics - Log Analysis Data Collector client toload historical data in batch mode for analysis.

Before you begin

Restriction:

Because NetView for z/OS message data is received directly from the NetView forz/OS program, NetView for z/OS historical data cannot be loaded in batch mode.

You must transfer the data to be loaded either to the Log Analysis server or to anon-z/OS host that has the Log Analysis Data Collector configured. The data mustalso be converted to ASCII.

If the data is in the SYSLOG log or on the JES spool, it must first be printed to adata set.

About this task

If it does not exist, you must create a data source with a data source type thatcorresponds to the type of data that is to be loaded. The following table indicatesthe data source types for static data that you might want to load in batch mode.

Static data Data source type

z/OS SYSLOG Open Print Data Set zOS-SYSLOG-SDSF

z/OS syslogd log zOS-syslogd


Static data Data source type

CICS MSGUSR log with the default dateformat of month-day-year

zOS-CICS-MSGUSR

CICS MSGUSR log with a date format ofday-month-year

zOS-CICS-MSGUSRDMY

CICS MSGUSR log with a date format ofyear-month-day

zOS-CICS-MSGUSRYMD

CICS EYULOG log with the default dateformat of month-day-year

zOS-CICS-EYULOG

CICS EYULOG log with a date format ofday-month-year

zOS-CICS-EYULOGDMY

CICS EYULOG log with a date format ofyear-month-day

zOS-CICS-EYULOGYMD

WebSphere Application Server for z/OSSYSOUT log

zOS-WAS-SYSOUT

WebSphere Application Server for z/OSSYSPRINT log

zOS-WAS-SYSPRINT

For information about how to use the Data Collector client to load the data, seeData Collector client.

Verifying the identity of the target server for forwarding dataFor communication to be secure, the identity of the target server must be verified,regardless of whether the z/OS Log Forwarder is forwarding data directly to anIBM Operations Analytics - Log Analysis server or by using the scalable datacollection architecture. Because, by default, the identity of the target server forforwarding data is trusted without verification, you must configure the z/OS LogForwarder for secure communication.

Before you begin

Review the following prerequisite information about how communication is, or canbe, configured, depending on your configuration for forwarding data:

Communication if forwarding data directly to an IBM Operations Analytics -Log Analysis server

Communication between the z/OS Log Forwarder and the Log Analysisserver occurs over the Hypertext Transfer Protocol Secure (HTTPS).

Communication if using the scalable data collection architectureIf you use the scalable collection architecture, the z/OS Log Forwarderforwards data to a Logstash server, or to a load balancer server thatbalances the load among multiple Logstash servers. By default,communication between the z/OS Log Forwarder and the Logstash or loadbalancer server occurs over HTTP, but it can be configured to occur overHTTPS.

If you use both Logstash and a load balancer, the server certificate thatmust be verified depends on whether the load balancer is configured toterminate Transport Layer Security (TLS) sessions or to pass them toLogstash.v If the load balancer is configured to terminate TLS sessions, the load

balancer server certificate must be verified.


http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/config/iwa_config_lfa_dt_clltr_c.html

v If the load balancer is configured to pass TLS sessions to Logstash, theLogstash server certificate must be verified.

For information about how to configure Logstash for secure transport,review the Logstash documentation.

About this task

Specifying systemInfo.trustAllCertificates = false in the z/OS Log Forwarderdata configuration file instructs the z/OS Log Forwarder to compare the securitycertificates from the target server with the certificates that are stored in a localtruststore. Therefore, you must manually import a security certificate for the targetserver into the local z/OS Log Forwarder truststore.

Location of z/OS Log Forwarder sample truststoreA sample truststore for the z/OS Log Forwarder is in the followinglocation:/usr/lpp/IBM/zscala/V2R2/samples/truststore.jks

The passphrase for this sample is z1manager.

Default location of Log Analysis server certificate fileThe default location of the Log Analysis server certificate file is thefollowing path:LA_INSTALL_DIR/wlp/usr/servers/Unity/resources/security/client.crt

Procedure

To secure communication, complete the following steps:1. Copy the default truststore for the z/OS Log Forwarder to the directory that

contains the z/OS Log Forwarder configuration files or to a location of yourchoice.

2. Update the z/OS Log Forwarder data configuration file to specify the newtruststore location.To use an existing truststore rather than the sample that is provided by default,update the z/OS Log Forwarder data configuration file to specify the values forsystemInfo.truststore and systemInfo.truststorePassword.

Tip: In the data configuration file, if you configure a relative path to thetruststore file, that path is relative to the z/OS Log Forwarder workingdirectory.

3. Complete the following steps to import the certificate for the target server intothe truststore.To import the certificate, use the keytool Java key and certificate managementutility. Any computer system that has Java installed can be used to do this.a. Copy the certificate file from the target server to the system on which you

plan to run the keytool utility.Attention: To prevent file corruption, file transfer must occur in binarymode.

b. Issue the following command (all on one line) to import the certificate intothe truststore:JRE_path/bin/keytool -import -alias key_alias -file

certificate_path/client.crt -keystorekeystore_path/truststore.jks -storepass z1manager


Tip: This command creates the truststore file if it does not exist.4. Issue the following command to change the passphrase for the truststore, and

update the z/OS Log Forwarder data configuration file to specify the newpassphrase:JRE_path/bin/keytool -storepasswd -new new_password

-keystore keystore_path/truststore.jks

Remember: To prevent unauthorized use of the passwords that are stored inthe data configuration file, encrypt the passwords.


Preparing to analyze z/OS log data

To begin collecting and analyzing z/OS log data, you must know how to start andstop the z/OS Log Forwarder and log in to IBM Operations Analytics - LogAnalysis. To optimize troubleshooting in your IT operations environment, assigndata sources to groups within the Log Analysis user interface, and customize theCustom Search Dashboards that are provided in the z/OS Insight Packs.

Starting and stopping the z/OS Log ForwarderThe z/OS Log Forwarder is started as a Started Task by using the z/OS LogForwarder start procedure.

About this task

The z/OS Log Forwarder records its progress in reading log data so that if it isstopped and restarted, it can resume log data collection at the point where itpreviously ended. This process ensures that no log data is missed.

persistent data fileWhen the z/OS Log Forwarder is stopped, it saves information for eachdata gatherer into a persistent data file. When it starts again, it resumes logdata collection, which is based on information in the persistent data file.

The persistent data file is written to the z/OS Log Forwarder workingdirectory.

Example of when to use cold start rather than warm startIf you start the z/OS Log Forwarder after it is stopped for an extendedperiod, the z/OS Log Forwarder might take a long time to collect the logdata that accumulated while it was shut down. The gathering of currentlog records does not begin until this process is complete. Therefore, if thez/OS Log Forwarder is shut down for more than a few hours, you mightwant to bypass the persistent data file by starting the z/OS Log Forwarderin cold start mode.

Procedurev Start the z/OS Log Forwarder by using one of the following methods.

– Warm start:Issue the following START system command:S GLAPROC

– Cold start:To bypass the persistent data file, start the z/OS Log Forwarder in cold startmode by using the -C option, as shown in the following example:S GLAPROC,OPT=-C

v To stop the z/OS Log Forwarder, issue the following STOP system command:P GLAPROC


Starting and stopping the NetView message providerTo collect NetView for z/OS message data and send it to the z/OS Log Forwarder,the NetView message provider must be active. The NetView message provider isstarted as a Started Task by using the REXX module GLANETV.

Before you begin

Review “NetView message provider” on page 45.

You can start the REXX module GLANETV from the command line of an existingNetView user ID, or create a new NetView user ID to support the running of thisREXX module.

Procedurev To start the NetView message provider, specify either C (cold start) or W (warm

start) for the COMMON.GLANETV.START variable in the CNMSTYLE member,as shown in the following example:COMMON.GLANETV.START = W

v To stop the NetView message provider, change the value of the GLANETV.STOPvariable in the CGED panel to YES.

Starting and stopping the SMF real-time data providerTo collect SMF data and send it to the z/OS Log Forwarder, the SMF real-timedata provider must be active. The SMF real-time data provider is started as aStarted Task by using the z/OS SMF real-time data provider start procedure.

Procedurev To start the SMF real-time data provider, issue the following command:

S GLASMF

v To stop the SMF real-time data provider, issue the following command:P GLASMF

Logging in to Log AnalysisTo create or update data sources (including to group data sources), you must log into IBM Operations Analytics - Log Analysis with a user name that hasadministrator authority. If you are only analyzing z/OS log data, the user namethat you use does not require administrator authority.

Before you begin

For information about administering, using, and troubleshooting IBM OperationsAnalytics - Log Analysis, see the Log Analysis documentation.

Procedure

To log in to IBM Operations Analytics - Log Analysis for the first time, use thefollowing URL:https://fully_qualified_scala_hostname:secure_port/Unity

where fully_qualified_scala_hostname is the fully qualified domain name of theIBM Operations Analytics - Log Analysis server, and secure_port is the Web



Console secure port that is defined during the installation of IBM OperationsAnalytics - Log Analysis. The default value for this port is 9987.

Use of cookies in the Log Analysis UIIn the IBM Operations Analytics - Log Analysis UI, you can enable the searchhistory. Log Analysis then uses a cookie that is saved in the temporary directory ofthe browser to remember the last 10 terms that are entered in the search field.

By default, the cookie that saves the search terms expires every 30 days.

For information about how to enable or clear this search history, see theinformation about enabling the GUI search history in the Enabling the GUI searchhistory.

Privacy policy considerations

IBM Software products, including software as a service solutions, (“SoftwareOfferings”) may use cookies or other technologies to collect product usageinformation, to help improve the end user experience, to tailor interactions withthe end user, or for other purposes. In many cases, no personally identifiableinformation is collected by the Software Offerings. Some of our Software Offeringscan help enable you to collect personally identifiable information. If this SoftwareOffering uses cookies to collect personally identifiable information, specificinformation about this offering’s use of cookies is set forth below.

This Software Offering does not use cookies or other technologies to collectpersonally identifiable information.

For more information about the use of various technologies, including cookies, forthese purposes, see IBM's Privacy Policy at http://www.ibm.com/privacy andIBM's Online Privacy Statement at http://www.ibm.com/privacy/details in thesection entitled “Cookies, Web Beacons and Other Technologies,” and the “IBMSoftware Products and Software-as-a-Service Privacy Statement” athttp://www.ibm.com/software/info/product-privacy.

Grouping data sources to optimize troubleshooting in your ITenvironment

By defining logical groups of data sources within IBM Operations Analytics - LogAnalysis, you can more easily apply searches to related sets of data sources tooptimize troubleshooting in your IT environment.

About this task

Of the data sources that you defined in the z/OS Log Forwarder configuration,determine whether any can be organized into meaningful logical groups.

For example, assume that your IT environment contains an online bankingapplication with a WebSphere Application Server for z/OS front end and a DB2 forz/OS back end. Organizing the associated data sources into a group can help youto quickly focus your troubleshooting efforts on the online banking application.

Although you can configure the z/OS Log Forwarder to create the necessary datasources if they do not exist, the z/OS Log Forwarder does not group the data

Preparing to analyze z/OS log data 105

http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/config/iwa_config_gui_search_enable.html

http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/config/iwa_config_gui_search_enable.html

http://www.ibm.com/privacy

http://www.ibm.com/privacy/details

http://www.ibm.com/software/info/product-privacy

sources. To group the data sources that are created by the z/OS Log Forwarder,you must manually assign the data sources to groups by using Log Analysis userinterface.

For more information about grouping data sources, see Editing service topologyinformation in Group JSON.

Extending troubleshooting capability with Custom Search Dashboardapplications

The Custom Search Dashboard applications that are provided in the z/OS InsightPacks are samples of custom logic that you can use to extend the troubleshootingcapability of IBM Operations Analytics - Log Analysis. You can use these samplesto create dashboards for presenting the generated data in a useful visual format,such as a chart or graph, and for presenting HTML content.

Before you begin

For information about the content of each dashboard, see “Dashboards” on page135.

For information about the data source types that are used in populating thedashboards, see “Correlation of the data to be analyzed with the associated datagatherer types, data source types, and dashboards” on page 82.

For more information about Custom Search Dashboards, see Custom SearchDashboards.

About this task

Before you run these applications, you must customize them for your environment.

Each dashboard application is defined in a JavaScript Object Notation (JSON) filethat specifies the following information:v The script that runs the custom logicv The parameters to pass to the scriptv The output charts to display in the dashboard

To update the parameters for a dashboard application, edit the appropriate *.appfile in the following directories:

WebSphere Application Server for z/OS Custom Search Dashboard applicationsLA_INSTALL_DIR/AppFramework/Apps/WASforzOSInsightPack_2.2.0.0

z/OS Network Custom Search Dashboard applicationsLA_INSTALL_DIR/AppFramework/Apps/zOSNetworkInsightPack_v1.1.0.0

z/OS SMF Custom Search Dashboard applicationsLA_INSTALL_DIR/AppFramework/Apps/SMFforzOSInsightPack_v1.1.0.3

z/OS SYSLOG Custom Search Dashboard applicationsLA_INSTALL_DIR/AppFramework/Apps/SYSLOGforzOSInsightPack_2.2.0.0

Customizing the dashboard applicationsBefore you run the Custom Search Dashboard applications, customize them foryour environment.


http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/admin/iwa_service_top_json.html

http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/admin/iwa_service_top_json.html

http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/extend/iwa_extend_customapps_ovw.html

http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/extend/iwa_extend_customapps_ovw.html

Before you begin

In this topic, the DB2 for z/OS Troubleshooting dashboard is used as an exampleto show you how to customize a dashboard application.

If you are customizing the SYSLOG for z/OS Time Comparison dashboard, alsosee “Customizing the SYSLOG for z/OS Time Comparison dashboard” on page111.

About this task

To customize a Troubleshooting application, you must update the parameters forthe following items:

data sourceThe data source, or group of data sources, to use as input for theTroubleshooting dashboard

time intervalThe time interval for which to extract data to present in theTroubleshooting dashboard

host name (optional)The host name for which to group log data in the Troubleshootingdashboard. Specifying this parameter is optional, but grouping log data byhost name can help you identify where problems are occurring.

Procedure1. To update the parameters, edit the following file:

LA_INSTALL_DIR/AppFramework/Apps/SYSLOGforzOSInsightPack_2.2.0.0/DB2_for_zOS_Troubleshooting.app

In the search parameter, you specify the data source, or group of data sources,to be used as input for the DB2 for z/OS Troubleshooting dashboard.

Tip: The name of the data source or group of data sources that you specifymust match the name that was defined in the Data Sources tab of theAdministrative Settings workspace in IBM Operations Analytics - Log Analysis.

2. To specify the data sources to be used for the dashboard data, use one of thefollowing formats, depending on whether you want to specify data sourcesindividually or as a group:v To specify individual data sources, use the following format:

"parameters": [{"name": "search","type": "SearchQuery","value": {

"logsources": [{

"type": "logSource","name": "SYSLOG1"

},{

"type": "logSource","name": "SYSLOG2"

}]

}},

Specify the following values for the search parameter:


type logSource

name The name of the data source that was defined in the Data Sourcestab of the Administrative Settings workspace in IBM OperationsAnalytics - Log Analysis. An example value is SYSLOG.

v To specify a group of data sources, use the following format:"parameters": [

{"name": "search","type": "SearchQuery","value": {

"logsources": [{

"type": "tag","name": "/day trader/trading application1/"

},{

"type": "tag","name": "/day trader/trading application2/"

}]

}},

Specify the following values for the search parameter:

type tag

name The tag path that was defined in the Data Sources tab of theAdministrative Settings workspace in IBM Operations Analytics -Log Analysis. An example value is /day trader/tradingapplication/.

3. To specify the time interval for which to extract data to present in thedashboard, define either a relative time interval or a custom time interval.v To specify a relative time interval, use the relativeTimeInterval parameter,

as shown in the following format:"parameters": [


"logsources": [{

"type": "logSource","name": "SYSLOG"

}]

}},{

"name": "relativeTimeInterval","type": "string","value": "LastDay"

},{

"name": "timeFormat","type": "data","value": {

"timeUnit": "hour","timeUnitFormat": "MM-dd HH:mm"

}},{

"name": "hostnameField",


"type": "string","value": "SystemName"

}],

The following values are valid for the relativeTimeInterval parameter:– LastQuarterHour

– LastHour

– LastDay

– LastWeek

– LastMonth

– LastYear

v To specify a custom time interval, use the timestamp value in the searchparameter, as shown in the following format:"parameters": [


"filter":{"range":{

"timestamp":{"from":"01/01/2012 00:00:00.000 -0400","to":"04/25/2013 00:00:00.000 -0400","dateFormat":"MM/dd/yyyy HH:mm:ss.SSS Z"

}}

},"logsources": [

{"type": "logSource","name": "SYSLOG"

}]

}},{


"timeUnit": "hour","timeUnitFormat": "yyyy-MM-dd HH:mm:ss"

}},{

"name": "hostnameField","type": "string","value": "SystemName"

}],

Specify the following values:

from The start of the time interval for which data is extracted to present inthe dashboard

to The end of the time interval for which data is extracted to present inthe dashboard

dateFormatThe date format string that is used in the from and to fields


4. The time format defines how the time interval (either relative or custom) isdisplayed on the x-axis of a chart in the dashboard. To specify the time format,use the timeFormat parameter, as shown in the following format:{


"timeUnit": "hour","timeUnitFormat": "MM-dd HH:mm"

}},

Specify the following values:

timeUnitThe discrete time unit that is displayed on the x-axis of a chart in thedashboard. The following values are valid:v minute

v hour

v day

v week

v month

v year

The time unit should be a smaller unit than the time interval. Forexample, if the chart displays a time interval of LastWeek, a reasonabletime unit is day.

timeUnitFormatThe date format of the time unit that is displayed on the x-axis of achart in the dashboard, as specified by the Java SimpleDateFormat class.For example, a date format might be yyyy-MM-dd HH:mm:ssZ.

5. Optional: To specify the host name, use the hostnameField parameter, as shownin the following format:{

"name": "hostnameField","type": "string","value": "SystemName"

}

Specify one of the following values for the host name, or accept the defaultvalue, which is SystemName:

hostnameThe host name that is specified in the Service Topology that isassociated with the data source.

SystemNameThe system name.

6. Ensure that the port on which IBM Operations Analytics - Log Analysis listensis set to the default value of 9987 in the following file:LA_INSTALL_DIR/AppFramework/Apps/SYSLOGforzOSInsightPack_2.2.0.0/CommonAppMod.py

For example, ensure that the following line of the file contains the correct portnumber:baseurl = ’https://localhost:9987/Unity’


Customizing the SYSLOG for z/OS Time Comparison dashboardTo customize the SYSLOG for z/OS Time Comparison dashboard, you must firstcomplete the basic customization for the SYSLOG for z/OS dashboard. Then,update the parameter for the time filter in the *.app file.

Before you begin

For more information about the basic customization, see “Customizing thedashboard applications” on page 106.

About this task

To specify a time filter, use the timefilters parameter, as shown in the followingformat. The lastnum value specifies the number of days between the two timeperiods for the comparison."filter": {

"timefilters":{

"lastnum": 1,"granularity": "days","type": "relative"

}}

You can use the SYSLOG for z/OS Time Comparison dashboard in either of thefollowing ways:v If you have an active set of search results in the Log Analysis user interface, this

application uses the time filter, the data set filter, and the query from that searchresult as the base for time period 1. Time period 2 is then based on what youspecify for the time filter in the *.app file. In the preceding example, time period2 is one day.

v If you have no active search results in the Log Analysis user interface, timeperiod 1 is based on the values for the logsources and therelativeTimeInterval parameters.

Procedure

To update the parameters, edit the appropriate *.app file in the followingdirectory:LA_INSTALL_DIR/AppFramework/Apps/SYSLOGforzOSInsightPack_2.2.0.0

Running the dashboard applicationsYou run the Custom Search Dashboard applications from the IBM OperationsAnalytics - Log Analysis user interface.

Procedure

To run the applications, complete the following steps:1. Log in to the IBM Operations Analytics - Log Analysis Search workspace.2. In the Search Dashboards section, expand one of the following folders,

depending on which applications you want to run:v SMFforzOSInsightPack_v1.1.0.3

v SYSLOGforzOSInsightPack_v2.2.0.0

v WASforzOSInsightPack_v2.2.0.0


v zOSNetworkInsightPack_v1.1.0.0

Tip: If you do not see the folder, refresh the contents of this section.3. Double-click the name of the application.

Tip: If you hover over the application tab in the workspace, the tooltipdisplays the time that the application was run, which is an indication of thetime that the data for the charts was generated.

Getting started with Problem Insights for z/OSIf the Problem Insights component of IBM Operations Analytics for z Systems isinstalled, the IBM Operations Analytics - Log Analysis UI includes a new tab thatis titled Problem Insights. For each sysplex from which data is being forwarded tothe Log Analysis server, the Problem Insights page includes insight about certainproblems that are identified in the ingested data.

Before you begin

Ensure that the Problem Insights component is installed. Also, ensure that you areusing Log Analysis Version 1.3.3 Fix Pack 1. For more information about theProblem Insights component and the system requirements and installationinformation for this component, see the following topics:v “Extensions to Log Analysis for z/OS Problem Insights and client-side Expert

Advice” on page 13v “z/OS Insight Pack requirements” on page 21v “Installing Log Analysis” on page 53v “Installing the z/OS Insight Packs and extensions” on page 55

Procedure

To use the Problem Insights page in the Log Analysis UI, click the tab that is titledProblem Insights.

Getting started with client-side Expert AdviceIf the client-side Expert Advice component of IBM Operations Analytics for zSystems is installed, IBMSupportPortal-ExpertAdvice on Client is a choice underExpert Advice in the Search workspace of the IBM Operations Analytics - LogAnalysis UI. With this extension, you can access Expert Advice even if the LogAnalysis server does not have access to the Internet.

Before you begin

Ensure that the client-side Expert Advice component is installed. Also, ensure thatyou are using Log Analysis Version 1.3.3 Fix Pack 1. For more information aboutthe client-side Expert Advice component and the system requirements andinstallation information for this component, see the following topics:v “Extensions to Log Analysis for z/OS Problem Insights and client-side Expert

Advice” on page 13v “z/OS Insight Pack requirements” on page 21v “Installing Log Analysis” on page 53v “Installing the z/OS Insight Packs and extensions” on page 55


About this task

The only field that you can update in the IBMSupportPortal-ExpertAdvice onClient.app file is _TERM_LIMIT, which defines the maximum number of keywordsthat a user can search for at one time.

The following data is removed from the search keywords by the client-side ExpertAdvice (IBMSupportPortal-ExpertAdvice on Client.app) to increase the chance ofa successful search on the client:v URLsv File namesv File pathsv IP addressesv Numbersv Double spaces

Tips for using the client-side Expert Advice:

1. Verify that the client browser settings allow pop-up windows from the LogAnalysis server.

2. If the Log Analysis server cannot be reached, clear cookies for that domain, andagain, try to connect.

Procedure

To use the client-side Expert Advice, complete the following steps:1. In the _TERM_LIMIT field of the IBMSupportPortal-ExpertAdvice on Client.app

file, define the maximum number of keywords that a user can search for at onetime.

2. To launch the client-side Expert Advice, click IBMSupportPortal-ExpertAdviceon Client under Expert Advice in the Custom Search Dashboards panel of theleft navigation pane of the Search workspace.The client browser sends search requests directly to the IBM Support Portal andopens a new browser tab to display the query search results.


Troubleshooting Operations Analytics for z Systems

This reference lists known problems that you might experience in using the IBMOperations Analytics for z Systems and describes known solutions.

About this task

You might also want to review log files or enable tracing for the z/OS LogForwarder or for IBM Operations Analytics - Log Analysis.

For IBM Operations Analytics - Log Analysis troubleshooting information, seeTroubleshooting.

Log filesTroubleshooting information is available in log files that are generated by the z/OSLog Forwarder and by IBM Operations Analytics - Log Analysis.

z/OS Log Forwarder log files

z/OS Log Forwarder logging information (and tracing information, if tracing isenabled) is sent to the STDERR data set on the GLAPROC job.

Significant z/OS Log Forwarder messages, such as the following messages, are alsowritten to the console:v Startup and shutdown messages are written as information messages.v Certain errors are written as action messages. These messages are cleared when

the error condition is resolved or when the z/OS Log Forwarder is stopped.

The level of message information that is provided on the console and in theSTDERR data set is the same. However, if stack trace data is available, it isincluded in only the STDERR data set.

IBM Operations Analytics - Log Analysis log files

The following Log Analysis log files provide troubleshooting information that isrelated to the processing of z/OS log data:v Logging that is related to ingestion of log records and Insight Pack processing is

contained in the LA_INSTALL_DIR/logs/GenericReceiver.log file on the LogAnalysis server.

v Logging that is related to searches is contained in the LA_INSTALL_DIR/logs/UnityApplication.log file on the Log Analysis server.

Enabling tracingFor certain problems, IBM Software Support might request that you enable tracingfor the z/OS Log Forwarder or for IBM Operations Analytics - Log Analysis.

About this task

For the z/OS Log Forwarder, you can enable tracing in either of the followingways:


http://www.ibm.com/support/knowledgecenter/SSPFMY_1.3.3/com.ibm.scala.doc/tshoot/iwa_getsc.html

v Enable static tracing by using the logging configuration filev Enable dynamic tracing by using the MVS MODIFY command

For information about logging and tracing for IBM Operations Analytics - LogAnalysis, see Troubleshooting.

Enabling static tracing for the z/OS Log ForwarderFor the z/OS Log Forwarder, you can enable static tracing by using the loggingconfiguration file.

About this task

The trace settings in the logging configuration file are applied each time that thez/OS Log Forwarder is started.

If you do not want to restart the z/OS Log Forwarder to enable tracing, use theMVS MODIFY command to enable dynamic tracing.

Procedure

To enable static tracing, complete the following steps:1. Copy the logging.properties file from the samples directory to a read/write

directory.2. Edit the logging.properties file as instructed by IBM Software Support.3. Update the environment configuration file to set the value of the ZLF_LOG

environment variable to the directory where you copied thelogging.properties file in step 1.

4. Restart the z/OS Log Forwarder.

What to do next

When the trace settings are no longer needed, return the logging configuration fileto its original contents.

Enabling dynamic tracing for the z/OS Log ForwarderFor the z/OS Log Forwarder, you can enable dynamic tracing by using the MVSMODIFY command.

About this task

Trace settings that are changed by using the MVS MODIFY command are notpersisted and therefore have no effect when the z/OS Log Forwarder is restarted.

To configure trace settings that persist each time that the z/OS Log Forwarder isstarted, use the logging configuration file to enable static tracing.

Procedure

To enable dynamic tracing, IBM Software Support might require you to issue oneor more of the following commands:



Option Description

Set trace To set the trace level for a specificcomponent of the z/OS Log Forwarder,issue the following system command:

F GLAPROC,APPL=SET,TRACE,logger,level

The values for logger and level are providedby IBM Software Support. Typically, level isone of the following three values:

EVENT The default tracing level. This levelprovides limited tracing that showsdetailed error and warningresponses from other applications.

DEBUG This level provides moderatetracing that shows values ofsignificant variables at key points inthe code path.

TRACE This level provides extensivetracing that shows detailed pathsthrough the code, including methodentry and exit.

Display trace 1. To display the trace levels for allcomponents of the z/OS Log Forwarder,issue the following system command:

F GLAPROC,APPL=DISPLAY,TRACE

The loggers with level values that areexplicitly set are the only ones that aredisplayed. Typically, only the root loggerhas a level value set, and that value istypically EVENT (the default level). Bydefault, all other loggers inherit theirlevel from the root logger.

2. To display the trace level for a specificcomponent of the z/OS Log Forwarder,issue the following system command:

F GLAPROC,APPL=DISPLAY,TRACE,logger

The value for logger is provided by IBMSoftware Support.

Troubleshooting Operations Analytics for z Systems 117

Option Description

Clear trace 1. To clear the trace levels for allcomponents of the z/OS Log Forwarder,issue the following system command:

F GLAPROC,APPL=CLEAR,TRACE

This command sets the value of the rootlogger level to EVENT. It also clears thevalues for all other loggers so that theyinherit their level from the root logger.After the enabled trace settings are nolonger needed, use this command toreturn to the default tracing state.

2. To clear the trace level for a specificcomponent of the z/OS Log Forwarder,issue the following system command:

F GLAPROC,APPL=CLEAR,TRACE,logger

The value for logger is provided by IBMSoftware Support. After a logger level iscleared, the logger inherits its level fromanother component.

z/OS Log Forwarder user ID has insufficient authorityThe z/OS Log Forwarder does not operate correctly due to insufficient authority.

Symptom

The following symptoms are possible:v Messages in the procedure STDERR data set indicate that the startup.sh script

cannot be found.v System Authorization Facility (SAF) messages indicate that the user has

insufficient authority to complete an operation. For example, the ICH408Imessage is issued by the Resource Access Control Facility (RACF) for authorityissues.

Solution

Verify that the user ID that is associated with the z/OS Log Forwarder has theappropriate authority to access the z/OS Log Forwarder files and directories.

BPX messages precede GLA messages in the z/OS SYSLOGIn the z/OS SYSLOG, messages with the prefix BPX precede the z/OS LogForwarder messages, which are messages with the prefix GLA.

Symptom

When the z/OS Log Forwarder starts, the z/OS SYSLOG includes the followingmessages:S GLAPROCBPXM023I (GLALGF) GLAA001I The z/OS Log Forwarder started successfullyBPXM023I (GLALGF) GLAA002I The z/OS Log Forwarder initialization is complete


Cause

The user ID that is associated with the z/OS Log Forwarder start procedure hasinsufficient authority.

If the appropriate authority is granted to the z/OS Log Forwarder start procedure,the BPX messages do not precede the GLA messages.

Because the BPX messages precede the GLA messages, the z/OS SYSLOG InsightPack does not recognize the GLA messages.

Solution

If you are using the Resource Access Control Facility (RACF) as your SystemAuthorization Facility (SAF) product, for example, either use the GLARACFsample that is provided in the SGLASAMP data set, or complete the followingsteps to resolve this problem:1. In RACF, add the BPX.CONSOLE resource to the class FACILITY by using the

General Resource Profiles option in the RACF Services Option Menu.2. In the BPX.CONSOLE profile that was created (or updated) in the preceding step,

add the user ID that the z/OS Log Forwarder start procedure is associatedwith, and assign READ access to the user ID.

3. Issue the following command to activate your changes:SETROPTS RACLIST(FACILITY) REFRESH

After the changes are made in RACF, the z/OS SYSLOG includes the followingmessages when the z/OS Log Forwarder starts:S GLAPROCGLAA001I The z/OS Log Forwarder started successfullyGLAA002I The z/OS Log Forwarder initialization is complete

Cannot shut down the z/OS Log Forwarder by using the STOPcommand

The z/OS Log Forwarder procedure cannot be shut down by using the STOPsystem command.

Symptom

In rare situations, you cannot shut down the z/OS Log Forwarder procedure byusing the STOP system command (P GLAPROC).

Cause

The cause is an infrequent failure of the Java virtual machine that is used by thez/OS Log Forwarder to shut down.

Solution

Use the CANCEL system command (C GLAPROC) to cancel the z/OS Log Forwarderprocedure.


Attention: When the z/OS Log Forwarder restarts after it is canceled in thismanner, it might restart in cold start mode. In cold start mode, the z/OS LogForwarder does not resume log data collection at the point where it previouslyended. Therefore, the last log record that was previously processed might bemissed.

When z/OS Log Forwarder starts, configuration changes do not seemto be implemented

The z/OS Log Forwarder data configuration file was updated, but theconfiguration changes do not seem to be implemented when the z/OS LogForwarder starts.

Cause

The following causes are the most probable:v The wrong data configuration file was edited.v The data configuration file that was edited is not in a directory where the z/OS

Log Forwarder can find and read it.

You can configure the z/OS Log Forwarder in either of the following ways:

Local hostThis configuration option creates and maintains a single set ofconfiguration files for the logical partition (LPAR) that you are logged into.

Sysplex-wideThis configuration option creates and manages configuration files for allLPARs in the local sysplex.

If a single directory contains the configuration files for more than one system, orlogical partition (LPAR), each configuration file name must include the names ofthe sysplex and the system (LPAR) to which the file applies. The file names mustuse the following conventions, where SYSNAME is the name of the system (LPAR)where the z/OS Log Forwarder runs, and SYSPLEX is the name of the sysplex (ormonoplex) in which that system is located. The values of both SYSPLEX andSYSNAME must be in all uppercase.

environment configuration fileSYSPLEX.SYSNAME.zlf.conf

data configuration fileSYSPLEX.SYSNAME.config.properties

Tip: When the z/OS Log Forwarder starts, it reads the data configuration file. If adata configuration file for an LPAR is in the same directory as a data configurationfile that has only the base file name config.properties, the z/OS Log Forwarderuses the file with the fully qualified name.

Solution

Verify that the configuration changes were made in the appropriate z/OS LogForwarder data configuration file. Also, verify that the updated configuration file isin the appropriate directory so that the z/OS Log Forwarder can find it.

For more information, see the following topics:


v “z/OS Log Forwarder program files” on page 26v “Configuring the z/OS Log Forwarder” on page 75v “Variable reference for the z/OS Log Forwarder start procedure” on page 216

APPLID values for CICS Transaction Server might not be correct inLog Analysis user interface

For CICS Transaction Server for z/OS, the application identifier (APPLID) valuesmight not be correct in the user interface of IBM Operations Analytics - LogAnalysis.

Symptom

APPLID values for CICS Transaction Server for z/OS are expected to be providedin grid views and patterns in the Log Analysis user interface. However, if theAPPLID value is not present in the CICS Transaction Server for z/OS message text,the first word of the message text is incorrectly used as the APPLID.

Cause

CICS Transaction Server for z/OS typically includes the APPLID as the first wordof the message. However, when CICS Transaction Server for z/OS messages do notinclude the APPLID as the first word in the message, the z/OS SYSLOG InsightPack incorrectly assumes that the first word of the message is an APPLID.

Solution

No workaround is available.

DB2 or MQ command prefix values might not be correct in LogAnalysis user interface

For DB2 for z/OS and MQ for z/OS, the command prefix values might not becorrect in the user interface of IBM Operations Analytics - Log Analysis.

Symptom

Command prefix values for DB2 for z/OS and MQ for z/OS are expected to beprovided in grid views and patterns in the Log Analysis user interface. However, ifthe command prefix value is not present in the DB2 for z/OS or MQ for z/OSmessage text, the first word of the message text is incorrectly used as thecommand prefix.

Cause

DB2 for z/OS and MQ for z/OS typically include the command prefix as the firstword of the message. However, when DB2 for z/OS or MQ for z/OS messages donot include the command prefix as the first word in the message, the z/OSSYSLOG Insight Pack incorrectly assumes that the first word of the message is acommand prefix.

Solution



Log record skipped and not available in Log AnalysisIndividual log records are not retained by the IBM Operations Analytics - LogAnalysis server.

Symptom

If the Log Analysis server is shut down, the server might not retain the lasttransmitted log record for each data source.

Cause

When the Log Analysis server shuts down, the log records that the server isprocessing might be lost because the server does not cache in-process log records.

Solution


z/OS log data is missing in Log Analysis search resultsThe IBM Operations Analytics - Log Analysis search results do not include theexpected z/OS log data.

If log data that is issued in a z/OS logical partition (LPAR) is not displayed in IBMOperations Analytics - Log Analysis, the following steps can help you determinepossible causes.

If NetView or SMF data is not displayed:

v If NetView for z/OS messages are not displayed, also see “NetView for z/OSproblems” on page 126.

v If SMF data is not displayed, also see “SMF data is not showing in LogAnalysis” on page 127.

Step 1: Check the z/OS Log Forwarder STDERR data set for erroror warning messages

Determine whether the z/OS Log Forwarder logged any error or warningmessages to its STDERR data set. Most error and warning messages are alsowritten to the console, but any available exception and stack trace information iswritten only to STDERR.

Using the message ID, look up the message in the message referencedocumentation. Review the explanation, and take appropriate action.

Step 2: Check the GenericReceiver.log file for error or warningmessages

If no z/OS Log Forwarder error or warning messages are logged, review theLA_INSTALL_DIR/logs/GenericReceiver.log file on the IBM Operations Analytics -Log Analysis server.

Look for messages with severity ERROR or WARN. For more information about anyerror or warning messages that you find, see Troubleshooting.



Step 3: Check the GenericReceiver.log file for ingestion status

The LA_INSTALL_DIR/logs/GenericReceiver.log file also specifies the number oflog records that are processed in each batch of log data, as shown in the followingexample:03/06/14 15:58:25:660 EST [Default Executor-thread-4] INFO - DataCollectorRestServlet :Batch of Size 9 processed and encountered 0 failures

In this example, nine log records were successfully ingested, and no log recordsfailed to be ingested.

You can determine the data source name by looking for a data source ingestionrecord before the batch status record in the GenericReceiver.log. In the followingexample, the data source name is my_sysprint:03/06/14 15:58:25:650 EST [Default Executor-thread-4] INFO - UnityFlowController :Adding data source under ingestion, logsource = my_sysprint threadId = 96

Determine whether the status information in the GenericReceiver.log indicatesone of the following three situations:

All batch status records for a data source have a size value of 0Log data is being ingested into IBM Operations Analytics - Log Analysis,but one of the following issues is preventing any log records from beingidentified:v The data source type that is specified for the data source that is

configured in IBM Operations Analytics - Log Analysis is incorrect.

Examples:

– The data source type is zOS-CICS-MSGUSR, but the log data is beingsent from an EYULOG data set. Therefore, the data source type mustbe specified as zOS-CICS-EYULOG.

– The data source type is zOS-WAS-SYSPRINT, and the log data is beingsent from a SYSPRINT data set, but the SYSPRINT data set containslog data in the distributed format. Therefore, the data source typemust be specified as WASSystemOut.

v Although the data source type is correct, the log data does not containany records in the format that is expected by the data source type.

Examples:

– The SYSPRINT data contains only WebSphere Application Serverinternal trace records. It does not contain records that are producedby the Java RAS component. For SYSPRINT, only records that areproduced by the Java RAS component are ingested.

– The SYSPRINT data contains only unstructured data, such as datathat is produced by System.out.println() calls from Java code. ForSYSPRINT, only records that are produced by the Java RAScomponent are ingested.

– The SYSOUT data contains Java garbage collector trace data. Javagarbage collector data is not supported by the zOS-WAS-SYSOUT datasource type.

Batch status records are present for the data source, the batch size is nonzero,and the number of failures is zero

Log records are being ingested successfully. Determine whether you haveone of the following situations:


v The search criteria is incorrect or not broad enough to include theexpected log data.

Example: If the search time filter is set from 2:00 PM until 2:05 PM, arecord that was logged at 2:05:01 PM is not flagged.Verify the search filters. You might want to start with a broad search,and refine it as needed.

v The time zone information is incorrectly configured in the z/OS LogForwarder. This issue can cause the time stamps of the ingested recordsto be 1 - 12 hours earlier or later than they should be.Verify the values of the following variables:– TZ environment variable in the z/OS Log Forwarder start procedure– timeZone values in the z/OS Log Forwarder data configuration file

v If only the most recent log record is missing, the record might be held inbuffer by IBM Operations Analytics - Log Analysis. IBM OperationsAnalytics - Log Analysis cannot confirm whether a log record iscomplete until the next log record is received for that data source.Therefore, for each data source, the last log record that is sent to IBMOperations Analytics - Log Analysis is typically held (and not ingested)until the next log record is received.

No batch status records exist for the data sourceDetermine whether you have one of the following situations:v The z/OS Log Forwarder is started, but no log records are generated for

the data source. No log records are ingested because no log data existsto ingest.

v The user ID that is associated with the z/OS Log Forwarder startprocedure does not have the appropriate access to files or directories.

Example: For example, the user ID might not have read access to one ofthe following items:– A z/OS UNIX log file– The High Performance Extensible Logging (HPEL) log or trace

directory.For more information about the HPEL log or trace directory, see“WAS HPEL gatherer properties” on page 187.

Change the file permissions to give the user ID the appropriate access.For more information about the z/OS Log Forwarder user ID, see“Creating the z/OS Log Forwarder start procedure” on page 76.

v Because the data gatherer is incorrectly configured in the z/OS LogForwarder data configuration file, the z/OS Log Forwarder cannot findthe log data.

Example: Verify the values of the following property keys:

dataGatherers.id.dataSetName for a z/OS Data Set gathererIf the dataGatherers.id.dataSetName value is incorrect, the z/OSLog Forwarder cannot find the data set.

For more information, see “z/OS Data Set gatherer properties”on page 190.


dataGatherers.id.ddName for a z/OS Job Log gathererIf the dataGatherers.id.ddName value is incorrect, the z/OS LogForwarder cannot find any data sets.

For more information, see “z/OS Job Log gatherer properties”on page 192.

dataGatherers.id.filePath for a z/OS UNIX Log File gathererIf the dataGatherers.id.filePath value is incorrect, the z/OSLog Forwarder cannot find the log file.

For more information, see “z/OS UNIX Log File gathererproperties” on page 198.

Remember: The dataGatherers.id.filePath value must be anabsolute path.

dataGatherers.id.jobName for a z/OS Job Log gathererIf the dataGatherers.id.jobName value is incorrect, the z/OSLog Forwarder cannot find any jobs.

For more information, see “z/OS Job Log gatherer properties”on page 192.

Cannot establish a secure connection between the z/OS LogForwarder and the Logstash or load balancer server

The z/OS Log Forwarder is configured to forward data to a Logstash server and touse secure transport, but it cannot connect to the Logstash server.

Symptom

The z/OS Log Forwarder logs one or more GLAD0032E messages. These messagesare written to console, and they indicate that a request to forward data to theLogstash server was unsuccessful due to a problem with the security transport.

Cause

The Logstash server might not be correctly configured to use the Transport LayerSecurity (TLS) protocol. If a load balancer is being used to balance the load amongmultiple Logstash servers, the load balancer also might not be correctly configured.Also, a Logstash server cannot accept secure connections if it is running with anIBM Java Runtime Environment (JRE).

Solution

If the Logstash server is running with an IBM JRE, use another JRE instead, suchas OpenJDK.

Otherwise, run the following command from a Linux system that has networkconnectivity to the Logstash or load balancer server to which you are trying toconnect:openssl s_client -connect server:port

In the command, use the following values for server and port:

server The host name or IP address of the Logstash or load balancer server to


which you are trying to connect. This name or address is the value for thelogReceiver.host property in the z/OS Log Forwarder data configurationfile.

port The Logstash or load balancer port. This port is the value for thelogReceiver.port property in the z/OS Log Forwarder data configurationfile.

In the response from the command, look for the following lines of information:––-BEGIN CERTIFICATE––-

––-END CERTIFICATE––-

If the response does not include a server certificate, the target system is notcorrectly configured for TLS. Find and fix this Logstash or load balancerconfiguration problem.

If the response does include a server certificate, and the z/OS Log Forwarder isnot configured to trust all certificates, set the value of thesystemInfo.trustAllCertificates property in the data configuration file to true,and restart the z/OS Log Forwarder. If the z/OS Log Forwarder connects to theLogstash or load balancer server, the Logstash or load balancer server certificatewas not imported into the z/OS Log Forwarder truststore. Use the CANCEL systemcommand (C GLAPROC) to cancel the z/OS Log Forwarder procedure, and importthe certificate into the truststore.

NetView for z/OS problemsThis reference lists known problems that you might experience in using the z/OSNetwork Insight Pack and the NetView message provider and describes knownsolutions.

z/OS Log Forwarder message states that PPI issued returncode 24

A z/OS Log Forwarder message states that the program-to-program interface (PPI)issued return code 24.

Cause

The z/OS Log Forwarder PPI receiver failed with return code 24, which canindicate that the NetView for z/OS PPI is not active.

Solution

To start the PPI, start the NetView for z/OS subsystem.

NetView message provider GLANETV issues messageGLAL004E with return code 15

The NetView message provider, which is the REXX module GLANETV, issuesmessage GLAL004E with return code 15.

Cause

The GLANETV module received return code 15, which indicates that the z/OS LogForwarder program-to-program interface (PPI) receiver buffer is not yet defined.


The GLANETV module stops because the PPI receiver buffer is not defined.

Solution

Start the z/OS Log Forwarder before you start the NetView message provider.

NetView message provider GLANETV issues messageGLAL006E

The NetView message provider, which is the REXX module GLANETV, wasstarted, but it logged message GLAL006E and did not gather data.

Cause

Possibly, the NetView message provider tried to gather messages by using the CZRpipeline stage. Based on settings in the DEFAULTS CZBRWAIT command and on theamount of data in the NetView program, the CZR pipeline stage might time outand therefore, not return any data, which causes the GLANETV module to issuemessage GLAL006E.

Solution

Use one of the following solutions:v Restart the NetView message provider (the GLANETV module) in cold start

mode by specifying C for the COMMON.GLANETV.START variable in theCNMSTYLE member, as shown in the following example:COMMON.GLANETV.START = C

This action forces a cold start, which causes the provider to try to gather themost recent data.

v Specify a high value for the CZBRWAIT command by using the NetView DEFAULTSor OVERRIDE command, and restart the GLANETV module.

If neither solution resolves the problem, contact IBM Software Support.

SMF problemsThis reference lists known problems that you might experience in using the z/OSSMF Insight Pack and the SMF real-time data provider and describes knownsolutions.

SMF data is not showing in Log AnalysisSMF data is not showing in the IBM Operations Analytics - Log Analysis userinterface.

Cause

The flow of data from SMF records to Log Analysis is a multiple step process, andit can be impacted at any step. For example, the SMF real-time data provider mustbe able to read SMF records from the SMF log stream and to save that data to aUNIX System Services directory. The z/OS Log Forwarder must be able to readthat UNIX System Services directory and send the files to SMF data sources in theLog Analysis server.


Solution

To verify the SMF real-time data provider processing, complete the following steps:1. Verify that the SMF real-time data provider start procedure GLASMF is

updated according to the comments in the GLASMF file.2. Verify that the SMF real-time data provider is started and active.3. Edit the SMF real-time data provider start procedure GLASMF, and follow the

instructions to enable statistical recording. After you save this file, restart theSMF real-time data provider.

Remember:When debugging is complete, disable statistical recording.Browse the SMF real-time data provider job log DRLOUT. Each time that the SMFreal-time data provider gathers data, it writes statistics to this job log. Thestatistics include details about the number of records that are read andprocessed.If no records are being read or processed, verify that the SMF log stream that isspecified in the start procedure GLASMF is correct. Also, verify that this logstream is active and is collecting SMF data. The SMF real-time data providercannot gather data if SMF is using data sets rather than log streams.Verify that the SMF log stream collects interval data as specified by the INTVALparameter in the SMFPRMxx configuration file.

4. Verify that data is being populated in the UNIX System Services workingdirectory as specified by the HFSDIRECTORY parameter in the start procedureGLASMF. Each day, a new file that contains SMF data should be created. Ifthese files are not being created, verify that the user ID that is running the SMFreal-time data provider has read/write access to the UNIX System Servicesworking directory.

5. If the UNIX System Services working directory contains files, and the filesinclude data, verify that the z/OS Log Forwarder is configured correctly toreference this working directory. Also, verify that the z/OS Log Forwarder hasthe appropriate definition to read the rolling log file names.

SMF message explanationsThis reference provides more information about some SMF messages.

When you start the SMF real-time data provider, you receivemessage IKJ56893I PATH /u/smftest/SMF30_D20150325.csv NOTALLOCATED+IGD17501I ATTEMPT TO OPEN A HFS FILE FAILED, 827

You receive this message in either of the following contexts:v The specified directory does not exist in UNIX System Services.v The permissions on the specified directory do not give read/write access to the

user ID that is running the SMF real-time data provider.

Complete the following steps to resolve the problem, depending on the cause:v Use the mkdir command to create the specified directory.v Use the chmod or chown command to give read/write access for the directory to

the SMF real-time data provider user ID.


Message DRL0185I No product registration performed is loggedin the DRLOUT job log

Ignore this message.

In the graphs of some SMF Search Dashboards, the message Nota supported chart type is shown

The z/OS SMF Insight Pack is dependent on the z/OS SYSLOG Insight Pack forgraph definitions. The z/OS SYSLOG Insight Pack must be installed for the SMFSearch Dashboards to display correctly.


Reference

Reference information is available for z/OS Log Forwarder commands, data sourcetypes, dashboards, messages, properties in the z/OS Log Forwarder dataconfiguration file, sample searches, and variables that you might need to update aspart of the z/OS Log Forwarder configuration.

Command reference for the z/OS Log ForwarderThis reference lists the z/OS Log Forwarder commands.

DISPLAY command for data gatherersTo view status and configuration information about the data gatherers that areknown to the z/OS Log Forwarder, use the z/OS Log Forwarder DISPLAYcommand.

Syntax for displaying status of all known data gatherers

To view the status of all known data gatherers, use the DISPLAY command with thissyntax.

►► F procname , APPL=DISPLAY , GATHERER , LIST ►◄

Sample output from commandGLAA032I The following list shows the known data gatherersin the format ’SIGNATURE STATUS’:ZOS_SYS:SYSLOG STARTED

Syntax for displaying information about a single data gatherer

To view the status or configuration information for a single data gatherer, use theDISPLAY command with this syntax.

►► F procname , APPL=DISPLAY , GATHERER , option , ►

► DATASET=datasetDOMAIN=domainHPELDIRECTORY='hpeldirectory'JOBNAME=jobname,DDNAME=ddnameSYSLOGUNIXFILEPATH='UNIXfilepath'SIGNATURE='signature'

►◄

Sample status output from commandGLAA034I The status for the data gatherer ZOS_SYS:SYSLOGis STOPPED

Sample configuration output from commandGLAA035I Configuration parameters for thedata gatherer ZOS_NETVIEW:NTV61 are:DATASOURCENAME: NMPIPL63_NETVIEWDOMAIN: NTV61TYPE: ZOS_NETVIEW


Parameter Descriptions

procnameThe name of the z/OS Log Forwarder procedure.

APPL=DISPLAYThe command to display information about a data gatherer.

optionUse one of the following option values:

STATUS Displays the status of a data gatherer that is known to the z/OS LogForwarder

CONFIG Displays the configuration statements for a data gatherer that is knownto the z/OS Log Forwarder

JOBNAME=jobname,DDNAME=ddnameFor a z/OS Job Log gatherer, the job name. To completely identify the job, youmust also include the data definition name (ddname).

Usage note: In defining a z/OS Job Log gatherer, you can use wildcardcharacters in the job name to define multiple data gatherers in a singledefinition. If a z/OS Job Log gatherer was defined by using wildcardcharacters, the values of the JOBNAME and DDNAME parameters mustreference the specific instance of a z/OS Job Log gatherer. For example, youmust specify JOB0011 or JOB0021 rather than JOB*1.

DATASET=datasetFor a z/OS Data Set gatherer, the name of the data set.

DOMAIN=domainFor a z/OS NetView Message gatherer, the name of the NetView domain.

HPELDIRECTORY='hpeldirectory'For a WebSphere Application Server High Performance Extensible Logging(HPEL) gatherer, the HPEL log directory.

Usage note: To prevent an error message, the value must be enclosed inquotation marks.

SYSLOGFor a z/OS SYSLOG gatherer.

UNIXFILEPATH='UNIXfilepath'For a z/OS UNIX Log File gatherer, the UNIX file path.


SIGNATURE='signature'For any data gatherer.

Each data gatherer has a unique signature for its identification.

START command for data gatherersTo start a data gatherer, use the z/OS Log Forwarder START command.

Syntax

►► F procname , APPL=START , GATHERER , ►



►◄



APPL=STARTThe command to start a data gatherer.












STOP command for data gatherersTo stop a data gatherer that is running, use the z/OS Log Forwarder STOPcommand.

Reference 133

Syntax

►► F procname , APPL=STOP , GATHERER , ►


►◄



APPL=STOPThe command to stop a data gatherer.













DashboardsThe Custom Search Dashboard applications that are provided in the z/OS InsightPacks can help you troubleshoot problems in your IT operations environment.Each main troubleshooting dashboard contains visual representations of the datathat is generated by the dashboard application. “Information links” dashboardscontain links to troubleshooting information in the respective softwaredocumentation, including message explanations.

WebSphere Application Server for z/OS Custom SearchDashboard applications

The WebSphere Application Server for z/OS Custom Search Dashboardapplications include dashboards for WebSphere Application Server for z/OS.

WebSphere Application Server for z/OS Dashboard

This dashboard contains the following content:

Message Counts - Top 5 over Last DayShows counts of the top five WebSphere Application Server for z/OSmessages, based on the msgClassifier value, that are found in a timeinterval.

Messages by Hostname - Top 5 over Last DayShows counts of the WebSphere Application Server for z/OS messages,based on the msgClassifier value, that are found for a host name in a timeinterval.

Java Exception Counts - Top 5 over Last DayShows counts of Java exceptions, based on the javaException value, that arefound in a time interval.

Java Exception by Hostname - Top 5 over Last DayShows counts of Java exceptions, based on the javaException value, that arefound for a host name in a time interval.

z/OS Network Custom Search Dashboard applicationsThe z/OS Network Custom Search Dashboard applications include network-relateddashboards.

Information Links dashboards for the following software are included:v NetView for z/OSv TCP/IPv UNIX System Services system log (syslogd)

z/OS Networking Dashboard


syslogd Message Counts - Top 5 Applications per hour over Last DayShows the message count for each of the five applications that logged themost data in the syslogd file.

syslogd Messages by Hostname - Top 5 per hour over Last DayShows the message count for each of the five hosts that logged the mostdata in the syslogd file.

Reference 135

Total syslogd Message Counts per hour over Last DayShows the total message count for all applications in a time interval.

Total syslogd Messages by Hostname per hour over Last DayShows the total message count for all hosts in a time interval.

NetView for z/OS Dashboard


Message Counts - Top 5 per hour over Last DayShows counts of the top five NetView for z/OS messages per hour, basedon the MessageID value, that are found in a time interval.

Message Type Counts - Top 5 per hour over Last DayShows counts of the top five NetView for z/OS message types per hour,based on the MessageType value, that are found in a time interval.

Total Message Counts - Top 5 per hour over Last DayShows the total number of NetView for z/OS messages per hour that arefound in a time interval.

Messages by Hostname - Top 5 per hour over Last DayShows counts of the top five NetView for z/OS messages per hour, basedon the MessageID value, that are found for a host name in a time interval.

Message Types by Hostname - Top 5 per hour over Last DayShows counts of the top five NetView for z/OS message types, based onthe MessageType value, that are found for a host name in a time interval.

Total Messages by Hostname - Top 5 per hour over Last DayShows the total number of NetView for z/OS messages per hour that arefound for a host name in a time interval.

z/OS SMF Custom Search Dashboard applicationsThe z/OS SMF Custom Search Dashboard applications include dashboards thatrepresent SMF data for CICS Transaction Server for z/OS, DB2 for z/OS, IMS forz/OS, MQ for z/OS, security for z/OS, and WebSphere Application Server forz/OS.

The following dashboards are included:v CICS TS for z/OS Performance Dashboards:

– CICS Regions Dashboard– CICS Regions Transaction Dashboard– CICS Jobs Dashboard

v DB2 for z/OS Performance Dashboardv IMS for z/OS Performance Dashboardv MQ for z/OS Performance Dashboardv z/OS Jobs Performance Dashboardv z/OS SMF 80 Security Events Dashboardv z/OS Top 10 RACF SMF 80 Events Dashboardv WebSphere Application Server for z/OS System Activity Dashboardv WebSphere Application Server Activity Summary by Controller Dashboard


Content that is common to many dashboards from the z/OS SMFCustom Search Dashboard

The following dashboards contain the content that is described in this section:v CICS Jobs Dashboard (under CICS TS for z/OS Performance Dashboards)v DB2 for z/OS Performance Dashboardv IMS for z/OS Performance Dashboardv MQ for z/OS Performance Dashboardv z/OS Jobs Performance Dashboard

CPU Utilization - Top 5 Jobs over Last DayShows a line chart of the CPU usage of the top five tasks, based onaverage CPU usage during the last day.

CPU Utilization - Max and Average per Job over Last DayFor all tasks on each host, shows the maximum and average CPU usageduring the last day.

I/O Rate - Top 5 Jobs over Last DayShows a line chart of the I/O rate of the top five tasks, based on averageI/O rates during the last day.

I/O Rate - Max and Average per Job over Last Day For all tasks on each host, shows the maximum and average I/O ratesduring the last day.

Working Set - Top 5 Jobs over Last DayShows a line chart of the working set size of the top five tasks, based onaverage working set size during the last day.

Working Set - Max and Average per Job over Last DayFor all tasks on each host, shows the maximum and average working setsize during the last day.

Paging Rate - Top 5 Jobs over Last DayShows a line chart of the paging rate of the top five tasks, based onaverage paging rates during the last day.

Paging Rate - Max and Average per Job over Last DayFor all tasks on each host, shows the maximum and average paging rateduring the last day.

CICS Regions Dashboard

The CICS Regions Dashboard contains the following content:

Wait on Storage Exceptions per Region over Last DayFor each CICS region, shows the number of wait-on-storage events thatoccurred during the last day.

Exceptions by Resource ID over Last DayShows the number of exceptions that occurred for each resource ID duringthe last day.

Short on Storage per Region over Last DayFor each CICS region, shows the number of low-on-storage messages thatoccurred during the last day.

Tasks at Maximum Threshold per Region over Last DayFor each CICS region, shows the number of times that the number of

Reference 137

active user transactions equaled the specified maximum number of usertransactions (MXT) during the last day.

Storage Violations per Region over Last DayFor each CICS region, shows the number of storage violations thatoccurred during the last day.

CICS Regions Transaction Dashboard

The CICS Regions Transaction Dashboard contains the following content:

Transactions - Top 5 Regions over Last DayShows the number of transactions for the five CICS regions with thehighest number of transactions, based on the average number oftransactions during the last day.

Transactions - Max and Average per Region over Last DayFor each CICS region, shows the maximum and average numbers oftransactions that occurred during the last day.

Security events dashboards

The dashboards for security events focus on access attempts that fail due toincorrect credentials or insufficient authority levels. Failed access attempts mightindicate that external hackers are guessing at credentials or that a company insideris trying to use personal information to get a higher level of access thanauthorized. Violations due to insufficient authority might indicate either probingfor access to a resource from an existing user or a compromised credential.

These dashboards contain the following content:

z/OS SMF 80 Security Events DashboardThis dashboard provides an overview of the system access violations thatoccurred during the last week. It includes five pie chart views. Each viewis limited to a maximum of 1000 data points by count from the searchresults.

Failed z/OS Logon Attempts by UserIDBy user ID, shows the number of z/OS authentication violationsdue to incorrect or revoked credentials.

Failed z/OS Logon Attempts by TermIDBy terminal ID, shows the number of z/OS authenticationviolations due to incorrect or revoked credentials.

Failed z/OS UNIX File System Access AttemptsBy file name, shows the number of UNIX file access violations. Thestring /CWD represents the user’s current working directory.

Failed z/OS Resource Access AttemptsBy resource name, shows the number of access violations due toinsufficient RACF authority to access a z/OS resource.

Failed z/OS RACF CommandsBy user ID, shows the number of access violations due toinsufficient authority to issue a RACF command. Only securityadministrators are authorized to change the RACF settings ofanother user.


z/OS Top 10 RACF SMF 80 Events DashboardThis dashboard includes four graphical views. Each view is limited to amaximum of the 10 largest data points by count from the search results.

Top 10 Failed Logons by SystemName and UserIDBy user ID, shows the number of z/OS authentication violationsdue to incorrect or revoked credentials.

Top 10 Failed Logons by SystemName and UserID (Current day,off-shift)

By user ID, shows the number of z/OS authentication violationsdue to incorrect or revoked credentials that do not occur duringthe traditional prime shift work hours of 9:00 AM to 5:00 PMCoordinated Universal Time (UTC) on the current day.

Top 10 OMVS File Access Failures by UserID and FilenameBy user ID and file name, shows the number of UNIX file accessviolations due to insufficient RACF authority to access a UNIXresource.

Top 10 z/OS Resource Accesses by UserID and ResourceNameBy user ID and resource name, shows the number of accessviolations due to insufficient RACF authority to access a z/OSresource. Examples of z/OS resources include data sets, diskvolumes, and tape volumes.

WebSphere Application Server for z/OS dashboards

These dashboards contain the following content:

WebSphere Application Server for z/OS System Activity DashboardThis dashboard provides insight on the amount of processing that isperformed by WebSphere Application Server for z/OS for each z/OSsystem. The dashboard views are from a system-wide perspective andindicate when a problem might be developing with a web server orapplication. Problems are often shown initially because of spikes inperformance due to higher than normal activity. Performance data,together with synchronized WebSphere Application Server for z/OS logs,can provide a context for diagnosing application issues.

Total System-wide WebSphere Controller Request CountShows the number of requests that are processed by all theWebSphere Application Server for z/OS servers on the system.

Total System-wide WebSphere Controller CPU UtilizationAggregates the CPU utilization for all the WebSphere ApplicationServer for z/OS servers on the system. This view is a high-levelperspective of the amount of CPU resources that are used tohandle web requests.

Total System-wide WebSphere Controller Specialty CPU UtilizationShows the amount of WebSphere Application Server for z/OSprocessing that was offloaded to specialty processors on thissystem to free general purpose processors for other processing.

Total System-wide WebSphere Controller Eligible Specialty CPU Depending on the available capacity of specialty processors, workcan be moved there from the general purpose processors. Thisvalue shows the amount of WebSphere Application Server forz/OS processing that was eligible for dispatch on a specialty

Reference 139

processor, but because no specialty processor was available, wasinstead processed on a general purpose processor.

WebSphere Application Server Activity Summary by Controller DashboardThis dashboard provides insight on the amount of processing that isperformed by WebSphere Application Server for z/OS for each WebSpherecontroller address space across z/OS systems. The dashboard views arefrom a controller perspective and indicate when a problem might bedeveloping with an individual controller address space. Because manyWebSphere controllers might be running on one z/OS system, or multiplesystems might be providing data to the same IBM Operations Analytics -Log Analysis server, graph content is limited to views of the 10 controllerswith the most processing.

Top 10 Total WebSphere Controller Request CountShows the number of requests that are processed by individualWebSphere Application Server for z/OS controller address spaceson the system. The view is filtered for the top 10 controllers.

Top 10 Total WebSphere Controller CPU Utilization Aggregates the CPU utilization by controller name for theWebSphere Application Server for z/OS servers on the system.This view is a high-level perspective of the amount of CPUresources that are used by controller address spaces to handle webrequests. The view is filtered for the top 10 controllers.

Top 10 Total WebSphere Controller Specialty CPU UtilizationShows the amount of WebSphere controller processing that wasoffloaded to specialty processors on this system to free generalpurpose processors for other processing. The view is filtered forthe top 10 controllers.

Top 10 Total WebSphere Controller Eligible Specialty CPU UtilizationDepending on the available capacity of specialty processors, workcan be moved there from the general purpose processors. Thisvalue shows the amount of WebSphere controller processing thatwas eligible for dispatch on a specialty processor, but because nospecialty processor was available, was instead processed on ageneral purpose processor. The view is filtered for the top 10controllers.

z/OS SYSLOG Custom Search Dashboard applicationsThe z/OS SYSLOG Custom Search Dashboard applications include dashboards forSYSLOG for z/OS, CICS Transaction Server for z/OS, DB2 for z/OS, IMS forz/OS, MQ for z/OS, and security for z/OS.

The following troubleshooting dashboards are included:v SYSLOG for z/OS Dashboardv SYSLOG for z/OS Time Comparison Dashboardv “CICS Transaction Server for z/OS Dashboard” on page 142v “DB2 for z/OS Dashboard” on page 143v “IMS for z/OS Dashboard” on page 143v “MQ for z/OS Dashboard” on page 144v “Security for z/OS Dashboard” on page 144

Information Links dashboards for the following software are included:


v CICS Transaction Server for z/OSv DB2 for z/OSv IMS for z/OSv MQ for z/OSv Resource Access Control Facility (RACF)

SYSLOG for z/OS Dashboard


Message Counts - Top 5 per hour over Last DayShows counts of the top five z/OS SYSLOG messages per hour, based onthe MessageID value, that are found in the last day.

Message Type Counts - Top 5 per hour over Last DayShows counts of the top five z/OS SYSLOG message types per hour, basedon the MessageType value, that are found in the last day.

Total Message Counts per hour over Last DayShows the total number of z/OS SYSLOG messages per hour that arefound in the last day.

Messages by Hostname - Top 5 per hour over Last DayShows counts of the top five z/OS SYSLOG messages per hour, based onthe MessageID value, that are found for a host name in the last day.

Message Types by Hostname - Top 5 over Last DayShows counts of the top five z/OS SYSLOG message types, based on theMessageType value, that are found for a host name in the last day.

Total Messages by Hostname per hour over Last DayShows the total number of z/OS SYSLOG messages per hour that arefound for a host name in the last day.

SYSLOG for z/OS Time Comparison Dashboard

Known issue: Python Version 2.6 or later must be installed on the system wherethis dashboard is run. The dashboard fails when it is run on a system with anearlier version of Python.


Time Range - Time Period 1Shows the “from” and “to” dates and times that are used for the first timeperiod of the comparison.

Time Range - Time Period 2Shows the “from” and “to” dates and times that are used for the secondtime period of the comparison.

Message Type Counts - Top 5 per hour over Time Period 1Shows counts of the top five message types per hour, based on theMessageType value, that are found in the first time period.

Message Type Counts - Top 5 per hour over Time Period 2Shows counts of the top five message types per hour, based on theMessageType value, that are found in the second time period.

Reference 141

Message Type Counts - Top 5 from Time Periods 1 and 2Shows the top five message types from both time periods, based on theMessageType value.

Message Type - Percentage ChangeShows, as a percentage value, the amount of change between the top 5message type counts in the first time period and the second time period.

Message ID Counts - Top 5 per hour over Time Period 1Shows counts of the top five messages per hour, based on the MessageIDvalue, that are found in the first time period.

Message ID Counts - Top 5 per hour over Time Period 2Shows counts of the top five messages per hour, based on the MessageIDvalue, that are found in the second time period.

Message ID Counts - Top 5 from Time Periods 1 and 2Shows the top five messages from both time periods, based on theMessageID value.

Message ID - Percentage ChangeShows, as a percentage value, the amount of change between the top 5message ID counts in the first time period and the second time period.

Message Prefix Counts - Top 5 per hour over Time Period 1Shows counts of the top five message prefixes per hour, based on theMessagePrefix value, that are found in the first time period.

Message Prefix Counts - Top 5 per hour over Time Period 2Shows counts of the top five message prefixes per hour, based on theMessagePrefix value, that are found in the second time period.

Message Prefix Counts - Top 5 from Time Periods 1 and 2Shows the top five message prefixes from both time periods, based on theMessagePrefix value.

Message Prefix - Percentage ChangeShows, as a percentage value, the amount of change between the top 5message prefix counts in the first time period and the second time period.

CICS Transaction Server for z/OS Dashboard


CICS Message Counts - Top 5 per hour over Last DayShows counts of the top five CICS Transaction Server for z/OS messagesper hour, based on the MessageID value, that are found in the last day.

CICS Message Type Counts - Top 5 per hour over Last DayShows counts of the top five CICS Transaction Server for z/OS messagetypes per hour, based on the MessageType value, that are found in the lastday.

Total CICS Message Counts per hour over Last DayShows the total number of CICS Transaction Server for z/OS messages perhour that are found in the last day.

CICS Messages by Hostname - Top 5 per hour over Last DayShows counts of the top five CICS Transaction Server for z/OS messagesper hour, based on the MessageID value, that are found for a host name inthe last day.


CICS Message Types by Hostname - Top 5 over Last DayShows counts of the top five CICS Transaction Server for z/OS messagetypes, based on the MessageType value, that are found for a host name inthe last day.

Total CICS Messages by Hostname per hour over Last DayShows the total number of CICS Transaction Server for z/OS messages perhour that are found for a host name in the last day.

DB2 for z/OS Dashboard


DB2 Message Counts - Top 5 per hour over Last DayShows counts of the top five DB2 for z/OS messages per hour, based onthe MessageID value, that are found in the last day.

DB2 Message Type Counts - Top 5 per hour over Last DayShows counts of the top five DB2 for z/OS message types per hour, basedon the MessageType value, that are found in the last day.

Total DB2 Message Counts per hour over Last DayShows the total number of DB2 for z/OS messages per hour that are foundin the last day.

DB2 Message Count by Hostname - Top 5 per hour over Last DayShows counts of the top five DB2 for z/OS messages per hour, based onthe MessageID value, that are found for a host name in the last day.

DB2 Message Types by Hostname - Top 5 over Last DayShows counts of the top five DB2 for z/OS message types, based on theMessageType value, that are found for a host name in the last day.

Total DB2 Messages by Hostname per hour over Last DayShows the total number of DB2 for z/OS messages per hour that are foundfor a host name in the last day.

IMS for z/OS Dashboard


IMS Message Counts - Top 5 per hour over Last DayShows counts of the top five IMS for z/OS messages per hour, based onthe MessageID value, that are found in the last day.

IMS Message Type Counts - Top 5 per hour over Last DayShows counts of the top five IMS for z/OS message types per hour, basedon the MessageType value, that are found in the last day.

Total IMS Message Counts per hour over Last DayShows the total number of IMS for z/OS messages per hour that are foundin the last day.

IMS Messages by Hostname - Top 5 per hour over Last DayShows counts of the top five IMS for z/OS messages per hour, based onthe MessageID value, that are found for a host name in the last day.

IMS Message Types by Hostname - Top 5 over Last DayShows counts of the top five IMS for z/OS message types, based on theMessageType value, that are found for a host name in the last day.

Reference 143

Total IMS Messages by Hostname per hour over Last DayShows the total number of IMS for z/OS messages per hour that are foundfor a host name in the last day.

MQ for z/OS Dashboard


MQ Message Counts - Top 5 per hour over Last DayShows counts of the top five MQ for z/OS messages per hour, based onthe MessageID value, that are found in the last day.

MQ Message Type Counts - Top 5 per hour over Last DayShows counts of the top five MQ for z/OS message types per hour, basedon the MessageType value, that are found in the last day.

Total MQ Message Counts per hour over Last DayShows the total number of MQ for z/OS messages per hour that are foundin the last day.

MQ Messages by Hostname - Top 5 per hour over Last DayShows counts of the top five MQ for z/OS messages per hour, based onthe MessageID value, that are found for a host name in the last day.

MQ Message Types by Hostname - Top 5 over Last DayShows counts of the top five MQ for z/OS message types, based on theMessageType value, that are found for a host name in the last day.

Total MQ Messages by Hostname per hour over Last DayShows the total number of MQ for z/OS messages per hour that are foundfor a host name in the last day.

Security for z/OS Dashboard


Security Message Counts - Top 5 per hour over Last DayShows counts of the top five RACF messages per hour, based on theMessageID value, that are found in the last day.

Security Message Type Counts - Top 5 per hour over Last DayShows counts of the top five RACF message types per hour, based on theMessageType value, that are found in the last day.

Total Security Message Counts per hour over Last DayShows the total number of RACF messages per hour that are found in thelast day.

Security Messages by Hostname - Top 5 per hour over Last DayShows counts of the top five RACF messages per hour, based on theMessageID value, that are found for a host name in the last day.

Security Message Types by Hostname - Top 5 over Last DayShows counts of the top five RACF message types, based on theMessageType value, that are found for a host name in the last day.

Total Security Messages by Hostname per hour over Last DayShows the total number of RACF messages per hour that are found for ahost name in the last day.


Intrusion DetectionShows messages that are flagged as intrusion-related indicators by someapplications.

WebSphere Application Server for z/OS data source typesThe WebSphere Application Server for z/OS Insight Pack includes support foringesting, and performing metadata searches against, the following types of datasources: zOS-WAS-SYSOUT, zOS-WAS-SYSPRINT, and zOS-WAS-HPEL.

zOS-WAS-SYSOUTWebSphere Application Server for z/OS SYSOUT job log data

zOS-WAS-SYSPRINTWebSphere Application Server for z/OS SYSPRINT job log data

zOS-WAS-HPELWebSphere Application Server for z/OS High Performance ExtensibleLogging (HPEL) data

Table 20 lists the configuration artifacts that are provided with the Insight Pack foreach type of WebSphere Application Server for z/OS data source.

Table 20. Configuration artifacts that are provided with the WebSphere Application Server for z/OS Insight Pack

Data source type Splitter Annotator Collection

zOS-WAS-SYSOUT zOS-WAS-SYSOUT-Split zOS-WAS-SYSOUT-Annotate zOS-WAS-SYSOUT-Collection

zOS-WAS-SYSPRINT zOS-WAS-SYSPRINT-Split zOS-WAS-SYSPRINT-Annotate zOS-WAS-SYSPRINT-Collection

zOS-WAS-HPEL zOS-WAS-HPEL-Split zOS-WAS-HPEL-Annotate zOS-WAS-HPEL-Collection

zOS-WAS-SYSOUT data source typeFor WebSphere Application Server for z/OS data sources of type zOS-WAS-SYSOUT,the log file splitter zOS-WAS-SYSOUT-Split breaks up the data into log records. Thelog record annotator zOS-WAS-SYSOUT-Annotate annotates the log records.

For data sources of type zOS-WAS-SYSOUT, SYSOUT contains the WebSphereApplication Server for z/OS log stream. IBM Operations Analytics - Log Analysisingests all log records in SYSOUT.

File format

The following output is sample output from the error log. The line numbers areadded for illustrative purposes and are not in the actual output.1| BossLog: { 0017} 2008/10/01 15:58:25.557 03 SYSTEM=SY1 CELL=BBOCELL2| NODE=BBONODE CLUSTER=BBOC001 SERVER=BBOS001 PID=0X0100003C3| TID=0X24F82920 00000000 t=6C8B58 c=3.C5D02 ./bboiroot.cpp+11954| tag=classlvl... BBOU0012W The function IRootHomeImpl::findHome(const char*)5| +1195 received CORBA system exception CORBA::INTERNAL. Error code is6| C9C21200.

Table 21 on page 146 describes the fields in the sample output. Some of these fieldsare not annotated. For information about which fields are annotated, see“zOS-WAS-SYSOUT-Annotate annotations” on page 146.

Reference 145

Tip: If WebSphere Application Server for z/OS is configured to use distributedlogging, SYSOUT is either empty or contains Java garbage collection data. Javagarbage collection data is not processed by the WebSphere Application Server forz/OS Insight Pack.

Table 21. Fields in sample output from error log for zOS-WAS-SYSOUT data source

Line Component value Value description

1 0017 Entry number

1 2000/06/01 15:58:25.557 03 Date, time stamp, and two-digit recordversion number

1 SY1 System name

1 BBOCELL Cell name

2 BBONODE Node name

2 BBOC001 Cluster name

2 BBOS001 Server name

2 0X0100003C Process ID

3 0X24F82920 00000000 Thread ID

3 6C8B58 Thread address

3 3.C5D02 Request correlation information

3 ./bboiroot.cpp+1195 File name and line number

4 classlvl Message tag that is defined in theclassification file

4 BBOU0012W Log message number

4 The functionIRootHomeImpl::findHome(const char*)

Log message

5 - 6 +1195 received CORBA system exceptionCORBA::INTERNAL. Error code isC9C21200.

Continuation lines of the CERR jobmessage

zOS-WAS-SYSOUT-Split log record splitter

To start a new record, the zOS-WAS-SYSOUT-Split splitter uses the presence of thestring "BossLog: ".

Records that are generated by calls to the System.out.println() orSystem.err.println() methods are ingested, but are not treated as separate logrecords.

zOS-WAS-SYSOUT-Annotate annotationsThis reference describes the fields that are annotated by zOS-WAS-SYSOUT-Annotate.

Table 22 on page 147 includes the following information about each field:v The field name, which corresponds to a field in the IBM Operations Analytics -

Log Analysis Search workspacev The description of what the annotation representsv The primary index configuration attributes that are assigned to the field, such as

the data type and the indication of whether the field can be sorted, filtered, orsearched.


Table 22. Log record fields that are annotated by zOS-WAS-SYSOUT-Annotate

Field DescriptionDatatype Sortable Filterable Searchable

application The application name that is populatedby the group data source field

Text U

datasourceHostname The host name that is specified in thedata source

Text U U

entryNumber Entry number Long U

exceptionClassName If this record was generated due to anexception, this name is the class name inthe top stack trace entry.

Text U U

exceptionFileName If this record was generated due to anexception, this name is the file name inthe top stack trace entry.

Text U

exceptionLineNumber If this record was generated due to anexception, this number is the linenumber in the top stack trace entry.

Long

exceptionMethodName If this record was generated due to anexception, this name is the method namein the top stack trace entry.

Text U U

exceptionPackageName If this record was generated due to anexception, this name is the packagename in the top stack trace entry.

Text U U

hostname The host name that is populated by thegroup data source field

Text U U U

javaException The first Java exception name thatmatches the following pattern:

*.*Exception

Text U U

logRecord The log record Text U

message The log message text. If a value isdetected for msgClassifier, messagecontains the msgClassifier also.

Text U

messageTag The message tag that is defined in theclassification file

Text U U

middleware The middleware name that is populatedby the group data source field

Text U

msgClassifier The log message number Text U U

processID The process identifier Text U U

service The service name that is populated bythe group data source field

Text U

SysplexName The sysplex name Text U U

SystemName The system name Text U U U

threadAddress The thread address Text U U

threadID An eight-character hexadecimal threadidentifier.

Text U U

timestamp The time stamp of the log record Date U U U

Reference 147

zOS-WAS-SYSPRINT data source typeFor WebSphere Application Server for z/OS data sources of typezOS-WAS-SYSPRINT, the log file splitter zOS-WAS-SYSPRINT-Split breaks up the datainto log records. The log record annotator zOS-WAS-SYSPRINT-Annotate annotatesthe log records.

For data sources of type zOS-WAS-SYSPRINT, SYSPRINT contains many tracestreams, including Java trace, System.out trace, and native trace. IBM OperationsAnalytics - Log Analysis ingests Java trace log records that are reported by the JavaReliability and Serviceability (RAS) component. Trace records from othercomponents and trace streams are not ingested.

The following items are ingested:v Any message that is logged by a web application that is using Java core logging

facilitiesv Any message that is logged by any log utility (such as log4j) that is built on the

Java core logging facilities

Trace records that are generated by calls to the System.out.println() orSystem.err.println() methods are ingested, but are not treated as separate logrecords.

File format

The following output is sample output from a trace log that is ingested. The linenumbers are added for illustrative purposes and are not in the actual output.1| Trace: 2009/07/14 17:26:19.577 02 t=6C8B58 c=UNK key=P8 tag=jperf (13007004)2| SourceId: PingServlet3| ExtendedMessage: BBOO0222I: Audit Message from PingServlet

Table 23 describes the fields in the sample output. Some of these fields are notannotated. For information about which fields are annotated, see“zOS-WAS-SYSPRINT-Annotate annotations” on page 149.

Tip: If WebSphere Application Server for z/OS is configured to use distributedlogging, SYSPRINT contains log data in distributed format. Therefore, to annotatethat log data, you must use the WebSphere Application Server Insight Pack, whichis preinstalled as part of the IBM Operations Analytics - Log Analysis installation.

Table 23. Fields in sample output from trace log for zOS-WAS-SYSPRINT data source


1 2009/07/14 17:26:19.577 02 Date, time stamp, and two-digit recordversion number

1 6C8B58 Thread address

1 UNK Request correlation information

1 P8 System protection key

1 jperf Message tag from classification file

1 13007004 Trace specific value for this trace point

2 PingServlet Source ID

3 BBOO0222I: Audit Message fromPingServlet

Extended Message


zOS-WAS-SYSPRINT-Split log record splitter

To start a new record, the zOS-WAS-SYSPRINT-Split splitter uses the presence of thestring "Trace: ".

Records that are generated by calls to the System.out.println() orSystem.err.println() methods are ingested, but are not treated as separate logrecords.

zOS-WAS-SYSPRINT-Annotate annotationsThis reference describes the fields that are annotated by zOS-WAS-SYSPRINT-Annotate.

Table 24 includes the following information about each field:v The field name, which corresponds to a field in the IBM Operations Analytics -



Table 24. Log record fields that are annotated by zOS-WAS-SYSPRINT-Annotate


application The application name that is populated bythe group data source field

Text U

datasourceHostname The host name that is specified in the datasource

Text U U

exceptionClassName If this record was generated due to anexception, this name is the class name in thetop stack trace entry.

Text U U

exceptionFileName If this record was generated due to anexception, this name is the file name in thetop stack trace entry.

Text U

exceptionLineNumber If this record was generated due to anexception, this number is the line number inthe top stack trace entry.

Long

exceptionMethodName If this record was generated due to anexception, this name is the method name inthe top stack trace entry.

Text U U

exceptionPackageName If this record was generated due to anexception, this name is the package name inthe top stack trace entry.

Text U U

hostname The host name that is populated by the groupdata source field

Text U U U

javaException The first Java exception name that matchesthe following pattern:

*.*Exception

Text U U


message The extended message. If a value is detectedfor msgClassifier, message contains themsgClassifier also.

Text U

Reference 149

Table 24. Log record fields that are annotated by zOS-WAS-SYSPRINT-Annotate (continued)


messageTag The message tag that is defined in theclassification file

Text U U

middleware The middleware name that is populated bythe group data source field

Text U

msgClassifier The extended message number Text U U

service The service name that is populated by thegroup data source field

Text U

sourceID The source identifier Text U U



threadAddress The hexadecimal thread address Text U U


zOS-WAS-HPEL data source typeFor WebSphere Application Server for z/OS data sources of type zOS-WAS-HPEL, thelog file splitter zOS-WAS-HPEL-Split breaks up the data into log records. The logrecord annotator zOS-WAS-HPEL-Annotate annotates the log records.

File format

For data sources of type zOS-WAS-HPEL, the z/OS Log Forwarder retrieves datafrom the WebSphere Application Server for z/OS High Performance ExtensibleLogging (HPEL) API. The z/OS Log Forwarder converts the data to a customcomma-separated value (CSV) format and sends it to IBM Operations Analytics -Log Analysis.

The following format illustrates a sample WebSphere Application Server for z/OSmessage:"08/03/15 21:51:28:935UTC","00000001","com.ibm.ws390.orb.CommonBridge","printProperties","BBOJ0077I",,,,,,,"AUDIT","com.ibm.ws390.orb.CommonBridge","10",,"STC00968","BBOSKDM","BBOJ0077I:osgi.framework = file:/u/WebSphere/V8R5/bbocell/bbonode/AppServer/plugins/org.eclipse.osgi_.jar",

Table 25 describes the fields in the sample output. Some of these fields are notannotated. For information about which fields are annotated, see“zOS-WAS-HPEL-Annotate annotations” on page 151.

Table 25. Fields in sample output for zOS-WAS-HPEL data source

Component value Value description

08/03/15 21:51:28:935 UTC Time stamp of the log record

00000001 The ID of the thread on which this requestwas logged.

This ID is based on the java.util.loggingrepresentation of the thread ID and is notequivalent to the operating systemrepresentation of the thread ID.


Table 25. Fields in sample output for zOS-WAS-HPEL data source (continued)


com.ibm.ws390.orb.CommonBridge The name of the Java class that made thecall to the logger. This name might be thename of the source class that is supplied inthe call to the logger, or it might be aninferred source class name. The name mightnot be accurate.

printProperties The name of the method that made the callto the logger. This name might be the nameof the source method that is supplied in thecall to the logger, or it might be an inferredsource method name. The name might notbe accurate.

BBOJ0077I The message ID of the log record message.This message ID is the same regardless ofthe locale in which the message is rendered.For non-message records, and for othermessages that do not begin with a messageID, this field is empty.

AUDIT The message level, which is an indication ofthe severity of the message

com.ibm.ws390.orb.CommonBridge The name of the logger that created thisrecord

10 The sequence index of the message that isgenerated by the logger

STC00968 The ID of the JES job that created this record

BBOSKDM The name of the JES job that created thisrecord

BBOJ0077I: osgi.framework =file:/u/WebSphere/V8R5/bbocell/bbonode/AppServer/plugins/org.eclipse.osgi_.jar

The formatted version of the log record,with values substituted for any placeholderparameters

zOS-WAS-HPEL-Split log record splitter

The zOS-WAS-HPEL-Split splitter considers each line of the log data to be a record.However, if a line break is contained within a quoted field, the line break isconsidered to be part of the field rather than an indication of the end of a record.

zOS-WAS-HPEL-Annotate annotationsThis reference describes the fields that are annotated by zOS-WAS-HPEL-Annotate.




Reference 151

Table 26. Log record fields that are annotated by zOS-WAS-HPEL-Annotate


application The application name that is populated by thegroup data source field

Text U

appName The name of the Java Platform, EnterpriseEdition (Java EE) application that the log ortrace record relates to, if any.

Text U U

className The name of the class that made the call to thelogger. This name might be the name of thesource class that is supplied in the call to thelogger, or it might be an inferred source classname. The name might not be accurate.

Text U U

datasourceHostname The host name that is specified in the datasource

Text U U

exceptionClassName If this record was generated due to anexception, this name is the class name in thetop stack trace entry.

Text U U

exceptionFileName If this record was generated due to anexception, this name is the file name in the topstack trace entry.

Text U U

exceptionLineNumber If this record was generated due to anexception, this number is the line number inthe top stack trace entry.

Long

exceptionMethodName If this record was generated due to anexception, this name is the method name inthe top stack trace entry.

Text U U

exceptionPackageName If this record was generated due to anexception, this name is the package name inthe top stack trace entry.

Text U U

hostname The host name that is populated by the groupdata source field

Text U U U

javaException The first Java exception name that matches thefollowing pattern:

*.*Exception

Text U U

jobId The identifier of the Job Entry Subsystem (JES)job that created this record

Text U U

jobName The name of the JES job that created thisrecord

Text U U

level The message level, which is an indication ofthe severity of the message

Text U U

loggerName The name of the logger that created this record Text U U


message The formatted version of the log record, withvalues substituted for any placeholderparameters. If a value is detected formsgClassifier, message contains the msgClassifieralso.

Text U


Table 26. Log record fields that are annotated by zOS-WAS-HPEL-Annotate (continued)


methodName The name of the method that made the call tothe logger. This name might be the name ofthe source method that is supplied in the callto the logger, or it might be an inferred sourcemethod name. This name might not beaccurate.

Text U U

middleware The middleware name that is populated by thegroup data source field

Text U

msgClassifier The message identifier of the log recordmessage. This message ID is the sameregardless of the locale in which the messageis rendered. For non-message records, and forother messages that do not begin with amessage ID, this field is empty.

Text U U

sequence The sequence index of the message asgenerated by the logger

Long U

service The service name that is populated by thegroup data source field

Text U



threadID The identifier of the thread on which thisrequest was logged. This ID is based on thejava.util.logging representation of the threadID, and is not equivalent to the operatingsystem representation of the thread ID.

Text U U


traceBlockAll If this record was generated due to anexception, this is the stack trace. The stacktrace is computed only for records where athrowable exception is explicitly supplied bythe caller.

Text U

z/OS Network data source typesThe z/OS Network Insight Pack includes support for ingesting, and performingmetadata searches against, the zOS-NetView type of data source.

zOS-NetViewNetView for z/OS message data

Table 27 lists the configuration artifacts that are provided with the Insight Pack foreach type of z/OS Network data source.

Table 27. Configuration artifacts that are provided with the z/OS Network Insight Pack


zOS-NetView zOS-NetView-Split zOS-NetView-Annotate zOS-NetView-Collection

Reference 153

zOS-NetView data source typeFor z/OS Network data sources of type zOS-NetView, the log file splitterzOS-NetView-Split breaks up the data into log records. The log record annotatorzOS-NetView-Annotate annotates the log records.

File format

For data sources of type zOS-NetView, the z/OS NetView Message gatherer sendsdata directly to the z/OS Log Forwarder by using the NetViewprogram-to-program interface (PPI). The z/OS Log Forwarder forwards theNetView for z/OS data to IBM Operations Analytics - Log Analysis.

The following format illustrates a sample NetView message as formatted by thez/OS NetView Message gatherer:06/23/15 19:48:09,NMP190 ,CNM01 ,NETOP1 ,-,BNH365E AUTOTBL MEMBER SPECIFIEDIS NOT UNIQUE WITHIN THE LIST OF ACTIVE AUTOMATION TABLES

Table 28 describes the fields in the sample output.

Table 28. Fields in sample output from HFS file for zOS-NetView data source


06/23/15 19:48:09 Time stamp, which is formatted as MM/DD/YYHH:mm:ss

NMP190 System name, with blank characters addedas needed to form the maximum of 8characters

CNM01 Domain, with blank characters added asneeded to form the maximum of 8characters

NETOP1 Operator ID, with blank characters added asneeded to form the maximum of 8characters

- HDRMTYPE, which is only 1 character

BNH365E Message ID

BNH Message prefix

E Message type

BNH365E AUTOTBL MEMBER SPECIFIED IS NOTUNIQUE WITHIN THE LIST OF ACTIVEAUTOMATION TABLES

Message text

NetView commands can be echoed in the message, and they are also included inthe HFS file, as shown in the following example:06/23/15 19:49:56,NMP190 ,CNM01 ,NETOP1 ,*,LIST DEFAULTS


Table 29. Fields in sample output from HFS file with echoed NetView command


06/23/15 19:49:56 Time stamp

NMP190 System name

CNM01 Domain


Table 29. Fields in sample output from HFS file with echoed NetView command (continued)


NETOP1 Operator ID

* HDRMTYPE

Message ID

Message prefix

Message type

LIST DEFAULTS Message text

zOS-NetView-Split log record splitter

The NetView data that is provided by the z/OS NetView Message gatherer isformatted so that each of the fields in a log record are separated by a comma, asshown in the following example:06/23/15 19:48:09,NMP190 ,CNM55 ,NETOP1 ,-,BNH365E AUTOTBL MEMBER SPECIFIEDIS NOT UNIQUE WITHIN THE LIST OF ACTIVE AUTOMATION TABLES

To start a new record, the zOS-NetView-Split splitter searches for the followingfirst five fields in the record:v Time stampv System namev Domainv Operator IDv HDRMTYPE

zOS-NetView-Annotate annotationsThis reference describes the fields that are annotated by zOS-NetView-Annotate.




Table 30. Log record fields that are annotated by zOS-NetView-Annotate


datasourceHostname The host name that is specified in the IBMOperations Analytics - Log Analysis datasource

Text U U

Domain The NetView domain Text U U U

HDRMTYPE The NetView message type Text U


MessageID The message identifier

Also, see “Message IDs” on page 156.

Text U U U

Reference 155

Table 30. Log record fields that are annotated by zOS-NetView-Annotate (continued)


MessagePrefix The first 3 characters of the messageidentifier. If no value is detected forMessageID, MessagePrefix has no value.

Text U U

MessageText The message text. If a value is detected forMessageID, MessageText contains theMessageID also.

Text U

MessageType The 1-character message type that isspecified in the MessageID value. Validvalues are A, D, E, I, S, U, or W.

If no value is detected for MessageID, or ifthe MessageID value does not contain amessage type, MessageType has no value.

Text U U

OperatorID The NetView operator ID Text U

SubsystemID The identifier of the software product orsubsystem that generated the message.

Text U U U




Message IDs

A string is detected as a message ID if it matches one of the following formats:aaannnaaannntaaaannnaaaannntaaaaannnaaaaannntaaannnnaaannnntaaaannnnaaaannnntaaaaannnnaaaaannnntaaannnnnaaannnnntaaaannnnnaaaannnnntaaaaannnnnaaaaannnnnt

where:v a represents an uppercase alphabetic character (A - Z).

The string can have 3 to 5 uppercase alphabetic characters but only the first 3characters are considered the message prefix.

v n represents a numeric character (0 - 9).v t represents a type character (A, D, E, I, S, U, or W).

Sometimes, a string that is not a message ID, but matches one of the precedingformats, might show in the MessageID field.


z/OS SMF data source typesThe z/OS SMF Insight Pack includes support for ingesting, and performingmetadata searches against, the following types of data sources for SystemManagement Facilities (SMF) data: zOS-SMF30, zOS-SMF80, zOS-SMF110_E,zOS-SMF110_S_10, and zOS-SMF120.

zOS-SMF30Job performance data (based on accounting data) for z/OS software

zOS-SMF80Data that is produced during Resource Access Control Facility (RACF)processing

zOS-SMF110_EMonitoring exceptions data for CICS Transaction Server for z/OS

zOS-SMF110_S_10Global transaction manager statistics data for CICS Transaction Server forz/OS

zOS-SMF120Performance data for WebSphere Application Server for z/OS from SMFrecord type 120 subtype 9.

Tip: This data does not include data for the WebSphere Liberty server.

Table 31 lists the configuration artifacts that are provided with the Insight Pack foreach type of SMF data source.

Table 31. Configuration artifacts that are provided with the z/OS SMF Insight Pack


zOS-SMF30 zOS-SMF-Split zOS-SMF30-Annotate zOS-SMF30-Collection


zOS-SMF110_E zOS-SMF-Split zOS-SMF110_E-Annotate zOS-SMF110_E-Collection

zOS-SMF110_S_10 zOS-SMF-Split zOS-SMF110_S_10-Annotate zOS-SMF110_S_10-Collection


File format

For SMF data source types, the SMF data gatherer stores SMF data in a UNIXSystem Services file in a proprietary format that is similar to a log file format.

zOS-SMF-Split log record splitter

The zOS-SMF-Split splitter breaks up the SMF data that is sent by the z/OS LogForwarder into individual records. The SMF data file is formatted so that eachrecord is contained on a single line.

zOS-SMF30-Annotate annotationsThis reference describes the fields that are annotated by zOS-SMF30-Annotate.


Log Analysis Search workspace

Reference 157

v The description of what the annotation representsv The primary index configuration attributes that are assigned to the field, such as


Table 32. Log record fields that are annotated by zOS-SMF30-Annotate


CPU The CPU usage for the monitored task Double U U U

datasourceHostname The host name that is specified in theIBM Operations Analytics - LogAnalysis data source

Text U U

IORate The I/O rate for the monitored task Double U U U

JobName The 8-character name of the job on thez/OS system

Text U U U


PagingRate The paging rate for the monitored task Double U U U

ProgName The name of the program that isrunning under the monitored task

Text U U U

RecordType The type of SMF record Text U U U


SystemID The system identifier Text U U U


Task The job name for the task that issuedthe message

Text U


WorkingSet The working set size for the monitoredtask

Double U U U





The column titled “Corresponding SMF field” indicates the SMF field name thatcorresponds to the field name in the annotation.


Field DescriptionCorrespondingSMF field Data type Sortable Filterable Searchable

AccessAllow Access authority is allowed SMF80DTA Text

AccessReq Access authority is requested SMF80DTA Text


Table 33. Log record fields that are annotated by zOS-SMF80-Annotate (continued)


AccessType Setting that is used in granting access.The following values are possible:

v None

v Owner

v Group

v Other

SMF80DA2 Text

Application Application name that is specified onthe RACROUTE request

SMF80DTA Text U U U

AuditDesc Descriptive name of the operation thatis audited

SMF80DA2 Text U

AuditName Name of the operation that is audited SMF80DA2 Text U

Auditor AUDITOR attribute (Y/N) SMF80ATH Text

AuditorExec Auditor execute/search audit options SMF80DA2 Text

AuditorRead Auditor read access audit options SMF80DA2 Text

AuditorUserExec User execute/search audit options SMF80DA2 Text

AuditorUserRead User read access audit options SMF80DA2 Text

AuditorUserWrite User write access audit options SMF80DA2 Text

AuditorWrite Auditor write access audit options SMF80DA2 Text

AuthorityFlags Flags that indicate the authority checksthat are made for the user whorequested the action

SMF80ATH Text U

CHOWNGroupID z/OS UNIX group identifier (GID)input parameter

SMF80DA2 Text

CHOWNUserID z/OS UNIX user identifier (UID) inputparameter

SMF80DA2 Text

Class The class entries that are supplied byIBM in the class descriptor table(ICHRRCDX)

SMF80DTA Text U

Command A string that is derived by using theSMF80EVT and SMF80EVQ values

SMF80EVT,SMF80EVQ

Text U

EffectiveGroup User's effective GID setting SMF80DA2 Text

EffectiveUser User's effective UID setting SMF80DA2 Text

Event Short description of the event codeand qualifier

SMF80EVT,SMF80EVQ

Text U U U

EventCode Event code SMF80EVT Text U

EventDesc Verbose description of the event codeand qualifier

SMF80EVT Text U U U

EventQual Event code qualifier SMF80EVQ Text U

Failed Event code qualifier is nonzero, whichindicates a failed request (Y/N)

SMF80EVQ Text

Filename File name of the file that is beingchecked

SMF80DA2 Text U U U

FileOwnerGroup File owner's GID SMF80DA2 Text

FileOwnerUser File owner's UID SMF80DA2 Text

Generic Generic profile used (Y/N) SMF80DTP Text

GroupExec Group permissions bit: execute SMF80DA2 Text

GroupRead Group permissions bit: read SMF80DA2 Text

GroupWrite Group permissions bit: write SMF80DA2 Text

ISGID Requested file mode: S_ISGID bit SMF80DA2 Text

Reference 159



ISUID Requested file mode: S_ISUID bit SMF80DA2 Text

ISVTX Requested file mode: S_ISVTX bit SMF80DA2 Text

logRecord The log record Set by the dataprovider

Text U

OtherExec Other permissions bit: execute SMF80DA2 Text

OtherRead Other permissions bit: read SMF80DA2 Text

OtherWrite Other permissions bit: write SMF80DA2 Text

OwnerExec Owner permissions bit: execute SMF80DA2 Text

OwnerRead Owner permissions bit: read SMF80DA2 Text

OwnerWrite Owner permissions bit: write SMF80DA2 Text

Pathname Full path name of the file that is beingchecked

SMF80DA2 Text U

ProfileName Name of the Resource Access ControlFacility (RACF) profile that is used toaccess the resource

SMF80DTA Text

RealGroup User's real GID setting SMF80DA2 Text

RealUser User's real UID setting SMF80DA2 Text

RecordType Internal record type. The followingvalues are possible:

v SMF80_COMMAND

v SMF80_LOGON

v SMF80_OMVS_RES_1

v SMF80_OMVS_RES_2

v SMF80_OMVS_SEC_1

v SMF80_OMVS_SEC_2

v SMF80_OPERATION

v SMF80_RESOURCE

For information about these values, seeSMF record type 80 records that theSMF real-time data provider collects.

Set by the dataprovider

Text U U U

ResourceName Resource name SMF80DTA Text U U U

SavedGroup User's saved GID setting SMF80DA2 Text

SavedUser User's saved UID setting SMF80DA2 Text

Special SPECIAL attribute (Y/N) SMF80ATH Text

SuperUser z/OS UNIX superuser (Y/N) SMF80AU2 Text

SysplexName The sysplex name Set by the dataprovider

Text U U

SystemID The system identifier from the SIDparameter in the SMFPRMnn member

SMF80SID Text U U U

SystemName The system name Set by the dataprovider

Text U U U

TermID Terminal ID of the foreground user(zero if not available)

SMF80TRM Text U

timestamp Date that the record was moved to theSMF buffer

SMF80DTE,SMF80TME

Date U U U

UserID Identifier of the user that is associatedwith this event. The value of JobNameis used if the user is not defined toRACF.

SMF80USR Text U U U


zOS-SMF110_E-Annotate annotationsThis reference describes the fields that are annotated by zOS-SMF110_E-Annotate.




Table 34. Log record fields that are annotated by zOS-SMF110_E-Annotate

Field Description Data type Sortable Filterable Searchable

ApplID The product name Text U U U

CICSTrans The transactionidentification

Text U U U

datasourceHostname The host name that isspecified in the IBMOperations Analytics - LogAnalysis data source

Text U U

ExceptionID The exception ID Text U

ExceptionID2 The extended exception ID Text U

ExceptionNumber The exception sequencenumber for the task

Text U U

ExceptionType The exception type Text U U

JobName The 8-character name ofthe job on the z/OSsystem

Text U U U


LU The logical unit on thez/OS system

Text U

NetID The NETID if a networkqualified name is receivedfrom z/OSCommunications Server.For a z/OSCommunications Serverresource where thenetwork qualified name isnot yet received, NETID iseight blanks. In all othercases, this field is null.

Text U

ProgName The name of the currentlyrunning program for theuser task when theexception conditionoccurred

Text U U U


ResourceID The exception resourceidentification

Text U U

Reference 161

Table 34. Log record fields that are annotated by zOS-SMF110_E-Annotate (continued)


ResourceType The exception resourcetype

Text U U

SubsystemID The subsystemidentification

Text U U U




timestamp The time stamp of the logrecord

Date U U U

TransNum The transactionidentification number

Text U

UserID The user identification attask creation. Thisidentifier can also be theremote user identifier for atask that is created as theresult of receiving anATTACH request across amultiregion operation (MRO) or AdvancedProgram-to-ProgramCommunication (APPC)link with attach-timesecurity enabled.

Text U U U

zOS-SMF110_S_10-Annotate annotationsThis reference describes the fields that are annotated by zOS-SMF110_S_10-Annotate.




Table 35. Log record fields that are annotated by zOS-SMF110_S_10-Annotate


ApplID The application identifier Text U U U

AtsMxt An indicator of the limitfor the number ofconcurrent tasks

Text U U

datasourceHostname The host name that isspecified in the IBMOperations Analytics - LogAnalysis data source

Text U U


Table 35. Log record fields that are annotated by zOS-SMF110_S_10-Annotate (continued)


GmtsMxtReached According to Greenwichmean time (GMT), the timewhen the task limit (thevalue of MAXTASKS) is met

Date U

IntervalDuration For a status type(StatsType) of INT, theinterval duration, which isrepresented in the timeformat HHMMSS

Text


MAXTASKS The limit for the numberof concurrent tasks

Long


StatsArea The status area Text U

StatsType The status type. Forexample, one of thefollowing types:

v EOD

v INT

v REQ

v RRT

v USS

Text U




timestamp The time stamp of the logrecord

Date U U U

TransCount The number of user andsystem transactions thatare attached

Double U

TransCurrentActiveUser At the present time, thenumber of active usertransactions in the system

Long U U U

TransCurrent_QSec At the present time, thenumber of seconds thattransactions have beenqueued because the tasklimit (the value ofMAXTASKS) is met

Double U

TransPeakActiveUser The highest number ofactive user transactions

Long U

TransQueuedUser The number of queueduser transactions in thesystem

Long U

TransTimesAtMAXTASKS The number of times thatthe task limit (the value ofMAXTASKS) was met

Long U U

Reference 163

Table 35. Log record fields that are annotated by zOS-SMF110_S_10-Annotate (continued)


TransTotalActive For a specified timeinterval, the number ofactive user transactions inthe system

Long U

TransTotalDelayed For a specified timeinterval, the number ofuser transactions that weredelayed because the tasklimit (the value ofMAXTASKS) was met

Long U

TransTotal_QSec For a specified timeinterval, the number ofseconds that transactionswere queued because thetask limit (the value ofMAXTASKS) was met

Double U

TransTotalTasks At the time of the lastreset, the number oftransactions in the system

Double U U





The column titled “Corresponding SMF field” indicates the SMF field name thatcorresponds to the field name in the annotation.


Field DescriptionCorrespondingSMF field

Datatype Sortable Filterable Searchable

Application The application name SM1209EO Text U U U

ControllerJobname The job name for thecontroller

SM1209BT Text U U U

DeleteServiceCPUActiveCount The count of samples whenthe enclave delete CPUservice time was non-zero.Time is accumulated by theenclave as reported by theCPUSERVICE parameter ofthe IWM4EDEL API. Avalue of 0 indicates that theenclave was not deleted.

SM1209DNcount

Long

DispatchCPU The amount of CPU time,in microseconds, that isused by dispatch TCB.

SM1209CI Double U U U





EnclaveCPU The amount of CPU timethat was used by theenclave as reported by theCPUTIME parameter of theIWM4EDEL API.

SM1209DH Double

EnclaveServiceDeleteCPU The enclave delete CPUservice that is accumulatedby the enclave as reportedby the CPUSERVICEparameter of theIWM4EDEL API. A value of0 indicates that the enclavewas not deleted.

SM1209DN Double

logRecord The log record Set by the dataprovider

Text U

RecordType Internal record type. Thefollowing values arepossible:

v SMF120_REQAPPL, whichindicates a WebSphereapplication record

v SMF120_REQCONT, whichindicates a WebSpherecontroller record

Set by the dataprovider

Text U U U

RequestCount Request count Set by the dataprovider

Long U U U

RequestEnclaveCPU The enclave CPU time atthe end of the dispatch ofthis request, as reported bythe CPUTIME parameter ofthe IWMEQTME API. Theunits are in TOD format.

SM1209DA Double U U U

RequestTime The time that the requestwas received, or the timethat the WebSphereapplication or controllercompleted processing of therequest response.

SM1209CM,SM1209CQ

Double U U U

RequestType The type of request thatwas processed. Thefollowing values arepossible:

v HTTP

v HTTPS

v IIOP

v INTERNAL

v MBEAN

v MDB-A

v MDB-B

v MDB-C

v NOTKNOWN

v OTS

v SIP

v SIPS

v UNKNOWN

SM1209CK Text U U U

Reference 165




SpecialtyCPU The amount of CPU timethat was spent onnon-standard CPs, such asthe z Systems ApplicationAssist Processor (zAAP)and z Systems IntegratedInformation Processor(zIIP). This value isobtained from theTIMEUSED API.

SM1209CX Double U U U

SpecialtyCPUActiveCount The count of samples whenthe amount of CPU timethat was spent onnon-standard CPs, such asthe zAAP and zIIP, wasnon-zero. The CPUutilization value is obtainedfrom the TIMEUSED API.

SM1209CXcount

Long

SysplexName The sysplex name Set by the dataprovider

Text U U

SystemID The system identifier SM120SID Text U U U

SystemName The system name Set by the dataprovider

Text U U U

timestamp The date that the recordwas moved to the SMFbuffer

SM120DTE,SM120TME

Date U U U

zAAPCPUActiveCount The count of samples whenthe delete zAAP CPUenclave time was non-zero.A value of 0 indicates thatthe enclave was not deletedor not normalized. ThisCPU time is obtained fromthe ZAAPTIME field in theIWM4EDEL macro.

SM1209DIcount

Long

zAAPEligibleCPU The amount of CPU time atthe end of the dispatch ofthis request that is spent ona regular CP that couldhave been run on a zAAP,but the zAAP was notavailable. This value isobtained from theZAAPONCPTIME field inthe IWMEQTME macro.

SM1209DC Double U U U

zAAPEnclaveCPUNormalized The enclave zAAP CPUtime at the end of thedispatch of this request, asreported by the ZAAPTIMEparameter of theIWMEQTME API. Thisutilization is adjusted bythe zAAP normalizationfactor at the end of thedispatch of this request. Thenormalization factor isobtained from theZAAPNFACTOR parameter ofthe IWMEQTME API.

SM1209DG,SM1209DB

Double





zAAPEnclaveDeleteCPU The delete zAAP CPUenclave. A value of 0indicates that the enclavewas not deleted or notnormalized. This value isobtained from theZAAPTIME field in theIWM4EDEL macro. Thisvalue is normalized by theenclave delete zAAPnormalization factor asreported by theZAAPNFACTOR parameter ofthe IWM4EDEL API.

SM1209DJ,SM1209DI

Double

zAAPEnclaveServiceDeleteCPU The enclave delete zAAPService that is accumulatedby the enclave as reportedby the ZAAPSERVICEparameter of theIWM4EDEL API. A value of0 indicates that the enclavewas not deleted.

SM1209DM Double

zAAPServiceCPUActiveCount The count of samples whenthe enclave delete zAAPservice time was non-zero.Time is accumulated by theenclave as reported by theZAAPSERVICE parameter ofthe IWM4EDEL API. Avalue of 0 indicates that theenclave was not deleted.

SM1209DMcount

Long

zIIPCPUActiveCount The count of samples whenthe enclave delete zIIP timewas non-zero. Time isaccumulated by the enclaveas reported by the ZIIPTIMEparameter of theIWM4EDEL API. A value of0 indicates that the enclavewas not deleted.

SM1209DKcount

Long

zIIPEligibleCPUEnclave The eligible zIIP enclavethat is on the CPU at theend of the dispatch of thisrequest. This value isobtained from theZIIPTIME field in theIWMEQTME macro.

SM1209DF Double U U U

zIIPEnclaveCPU The zIIP enclave that is onthe CPU at the end of thedispatch of this request.This value is obtained fromthe ZIIPONCPTIME field inthe IWMEQTME macro.

SM1209DD Double

zIIPEnclaveDeleteCPU The enclave delete zIIP timethat is accumulated by theenclave as reported by theZIIPTIME parameter of theIWM4EDEL API. A value of0 indicates that the enclavewas not deleted.

SM1209DK Double

Reference 167




zIIPEnclaveQualityCPU The zIIP Quality Timeenclave that was on theCPU at the end of thedispatch of this request.This value is obtained fromthe ZIIPQUALTIME field inthe IWMEQTME macro.

SM1209DE Double

zIIPEnclaveServiceDeleteCPU The enclave delete zIIPservice that is accumulatedby the enclave as reportedby the ZIIPSERVICEparameter of theIWM4EDEL API. A value of0 indicates that the enclavewas not deleted or notnormalized.

SM1209DL Double

zIIPServiceCPUActiveCount The count of samples whenthe enclave delete zIIPservice time was non-zero.Time is accumulated by theenclave as reported by theZIIPSERVICE parameter ofthe IWM4EDEL API. Avalue of 0 indicates that theenclave was not deleted ornot normalized.

SM1209DLcount

Long

z/OS SYSLOG data source typesThe z/OS SYSLOG Insight Pack includes support for ingesting, and performingmetadata searches against, the following types of data sources:zOS-SYSLOG-Console, zOS-SYSLOG-SDSF, zOS-syslogd, the three variations ofzOS-CICS-MSGUSR, and the three variations of zOS-CICS-EYULOG.

zOS-SYSLOG-Consolez/OS SYSLOG data that is formatted by the z/OS Log Forwarder

zOS-SYSLOG-SDSFz/OS SYSLOG data that is formatted by the System Display and SearchFacility (SDSF)

zOS-syslogdSyslogd data in UNIX System Services

zOS-CICS-MSGUSR, zOS-CICS-MSGUSRDMY, and zOS-CICS-MSGUSRYMDCICS Transaction Server for z/OS MSGUSR log data.

Use one of the three variations of this data source type, depending on thedate format in the time stamp for the data source.v zOS-CICS-MSGUSR uses the default date format MDY.v zOS-CICS-MSGUSRDMY uses the format DMY.v zOS-CICS-MSGUSRYMD uses the format YMD.

zOS-CICS-EYULOG, zOS-CICS-EYULOGDMY, and zOS-CICS-EYULOGYMDCICS Transaction Server for z/OS EYULOG data.

Use one of the three variations of this data source type, depending on thedate format in the time stamp for the data source.


v zOS-CICS-EYULOG uses the default date format MDY.v zOS-CICS-EYULOGDMY uses the format DMY.v zOS-CICS-EYULOGYMD uses the format YMD.

Table 37 lists the configuration artifacts that are provided with the Insight Pack foreach type of z/OS SYSLOG data source.

Table 37. Configuration artifacts that are provided with the z/OS SYSLOG Insight Pack


zOS-SYSLOG-Console zOS-SYSLOG-Console-Split zOS-SYSLOG-Console-Annotate

zOS-SYSLOG-Console-Collection

zOS-SYSLOG-SDSF zOS-SYSLOG-SDSF-Split zOS-SYSLOG-SDSF-Annotate zOS-SYSLOG-SDSF-Collection

zOS-syslogd zOS-syslogd-Split zOS-syslogd-Annotate zOS-syslogd-Collection

zOS-CICS-MSGUSR zOS-CICS-MSGUSR-Split zOS-CICS-Annotate zOS-CICS-MSGUSR-Collection

zOS-CICS-MSGUSRDMY zOS-CICS-MSGUSRDMY-Split zOS-CICS-DMY-Annotate zOS-CICS-MSGUSRDMY-Collection

zOS-CICS-MSGUSRYMD zOS-CICS-MSGUSRYMD-Split zOS-CICS-YMD-Annotate zOS-CICS-MSGUSRYMD-Collection

zOS-CICS-EYULOG zOS-CICS-EYULOG-Split zOS-CICS-Annotate zOS-CICS-EYULOG-Collection

zOS-CICS-EYULOGDMY zOS-CICS-EYULOGDMY-Split zOS-CICS-DMY-Annotate zOS-CICS-EYULOGDMY-Collection

zOS-CICS-EYULOGYMD zOS-CICS-EYULOGYMD-Split zOS-CICS-YMD-Annotate zOS-CICS-EYULOGYMD-Collection

zOS-SYSLOG-Console data source typeFor z/OS SYSLOG data sources of type zOS-SYSLOG-Console, the log file splitterzOS-SYSLOG-Console-Split breaks up the data into log records. The log recordannotator zOS-SYSLOG-Console-Annotate annotates the log records.

File format

For data sources of type zOS-SYSLOG-Console, the data is converted to a customcomma-separated value (CSV) format.

The following sample illustrates the data in CSV format:NE,001E,15203 08.55.11.340 -0400,TVT7007,STC00722,INSTREAM,00000000000000000000000000000000,00000290,TSO ," LOGON"

Table 38 describes the fields in the sample output. Some of these fields are notannotated. For information about which fields are annotated, see“zOS-SYSLOG-Console-Annotate annotations” on page 170.

Table 38. Fields in sample output from SYSLOG for zOS-SYSLOG-Console data source


NE 2-character indicator that specifies whetherthe message is a single line message (NC orNE) or a multi-line message (MC)

001E Address space identifier

15203 08.55.11.340 -0400 Time stamp of the log record

TVT7007 System name

STC00722 Job identifier for the task that issued themessage

INSTREAM Console name

Reference 169

Table 38. Fields in sample output from SYSLOG for zOS-SYSLOG-Console datasource (continued)


00000000000000000000000000000000 Route codes

00000290 User exit flags

TSO 8-character name of the job on the z/OSsystem

LOGON Message text

zOS-SYSLOG-Console-Split log record splitter

The zOS-SYSLOG-Console-Split splitter considers each line of the log data to be arecord. However, if a line break is contained within a quoted field, the line break isconsidered to be part of the field rather than an indication of the end of a record.

zOS-SYSLOG-Console-Annotate annotationsThis reference describes the fields that are annotated by zOS-SYSLOG-Console-Annotate.




Table 39. Log record fields that are annotated by zOS-SYSLOG-Console-Annotate



ASID The address space identifier Text U U U

CommandPrefix The command prefix Text U U U

Component The component identifier, whichshows the domain or componentthat issues the message

Text U U U

ConsoleName The console name Text U U U

datasourceHostname The host name that is specified inthe IBM Operations Analytics - LogAnalysis data source

Text U U

JobName The 8-character name of the job onthe z/OS system

Text U U U



Also, see “Message IDs” on page171.

Text U U U

MessagePrefix The first 3 characters of the messageidentifier. If no value is detected forMessageID, MessagePrefix has novalue.

Text U U


Table 39. Log record fields that are annotated by zOS-SYSLOG-Console-Annotate (continued)


MessageText The message text. If a value isdetected for MessageID, MessageTextcontains the MessageID also.

Text U

MessageType The one-character message type thatis specified in the MessageID value.Valid values are A, I, E, W, D or S.

If no value is detected for MessageID,or if the MessageID value does notcontain a message type, MessageTypehas no value.

Text U U

RouteCodes The route codes Text U

SubsystemID The identifier of the softwareproduct or subsystem that generatedthe message.

Text U U U



Task The job identifier for the task thatissued the message

Text U


UserExitFlags The user exit flags Text U

Message IDs

A string is detected as a message ID if it matches one of the following formats:aaxxxnaaxxxntaaxxxxnaaxxxxntaaxxxxxnaaxxxxxntaaxxxxxxnaaxxxxxxnt$HASPnnn$HASPnnnnDFHaannDFHaannnDFHaannnnDFHnnDFHnntDFHnnnDFHnnnnEYUaannEYUaannnEYUaannnnEYUnnEYUnntEYUnnnEYUnnnn

where:v a represents an uppercase alphabetic character (A - Z).v n represents a numeric character (0 - 9).v x represents an uppercase alphabetic character or a numeric character.

Reference 171

v t represents a type character (A, I, E, W, D, or S). If the first 3 characters of themessage ID are DFH or EYU, U is also a valid type character.


zOS-SYSLOG-SDSF data source typeFor z/OS SYSLOG data sources of type zOS-SYSLOG-SDSF, the log file splitterzOS-SYSLOG-SDSF-Split breaks up the data into log records. The log recordannotator zOS-SYSLOG-SDSF-Annotate annotates the log records.

For data sources of type zOS-SYSLOG-SDSF, the log data is rendered by the SystemDisplay and Search Facility (SDSF) and can be ingested in batch mode by using theIBM Operations Analytics - Log Analysis Data Collector client.

File format

The following output is sample output from the SYSLOG. The line numbers areadded for illustrative purposes and are not in the actual output.1| MRC000000 TVT7007 13234 10:33:19.20 INTERNAL 00000290

IXC467I STOPPING PATH STRUCTURE IXCSTR1 1512| ER 151 00000290

RSN: SYSPLEX PARTITIONING OF LOCAL SYSTEM

Table 40 describes the fields in the sample output. Some of these fields are notannotated. For information about which fields are annotated, see“zOS-SYSLOG-SDSF-Annotate annotations” on page 173.

Table 40. Fields in sample output from SYSLOG for zOS-SYSLOG-SDSF data source


1 M Record type

1 R An indication of whether the line wasgenerated because of a command

1 C000000 Hexadecimal representation of routingcodes 1 - 28

1 TVT7007 System name

1 13234 Julian date

1 10:33:19.20 Time stamp

1 INTERNAL Job identifier for the task that issued themessage

1 00000290 Installation exit and message suppressionflags

1 IXC467I Message identifier

1 STOPPING PATH STRUCTURE ICSTR1 First line of message text

1 151 Multi-line identifier

2 E Record type

2 R An indication of whether the line wasgenerated because of a command

2 151 Multi-line identifier

2 00000290 Installation exit and message suppressionflags


Table 40. Fields in sample output from SYSLOG for zOS-SYSLOG-SDSF datasource (continued)


2 RSN: SYSPLEX PARTITIONING OF LOCALSYSTEM

Last line of message text

If the message is a DB2 or MQ message, a command prefix might be present, asshown in the following example:M 8000000 SYSL 13237 14:00:20.23 STC21767 00000294 DSNR031I :D91A...

|DB2 CommandPrefix

If the message is a CICS message, an Application ID might be present, as shown inthe following example:N FFFF000 MV20 14071 20:16:14.27 JOB21691 00000090 DFHPA1101 CMAS51 ...

|CICSApplID

zOS-SYSLOG-SDSF-Split log record splitter

To start a new record, the zOS-SYSLOG-SDSF-Split splitter uses the time stamp of aline. Records are split at the beginning of the line that contains the time stamp ratherthan at the beginning of the time stamp itself. The time stamp is a combination ofthe Julian date, the time, and the time zone values.

Any subsequent lines that do not contain a time stamp are considered to be part ofthe same record. In the following example, the first line has a time stamp. Becausethe second and third lines do not have a time stamp, they are considered to bepart of the same record. The fourth line has a time stamp and starts a new record.M 0000000 TVT7007 13314 13:25:15.31 00000290 IEA009I SYMBOLIC DEFINITIONS...D 007 00000290 IEASYM00E 007 00000290 IEASYM01N 4000000 TVT7007 13314 13:25:15.32 00000290 IEE252I MEMBER IEASYM00...

zOS-SYSLOG-SDSF-Annotate annotationsThis reference describes the fields that are annotated by zOS-SYSLOG-SDSF-Annotate.




Table 41. Log record fields that are annotated by zOS-SYSLOG-SDSF-Annotate



CommandPrefix The command prefix Text U U U

Reference 173

Table 41. Log record fields that are annotated by zOS-SYSLOG-SDSF-Annotate (continued)


Component The component identifier, which showsthe domain or component that issues themessage

Text U U U

ConsoleName The console name Text U U U


Text U U



Also, see “Message IDs.”

Text U U U


Text U U

MessageText The message text. If a value is detected forMessageID, MessageText contains theMessageID also.

Text U

MessageType The one-character message type that isspecified in the MessageID value. Validvalues are A, I, E, W, D or S.


Text U U

RouteCodes Routing codes 1 - 28 Text U


Text U U U



Task The job identifier for the task that issuedthe message. If the ConsoleName fieldcontains a value, this field does notcontain a value.

Text U


UserExitFlags The user exit flags Text U

Message IDs

A string is detected as a message ID if it matches one of the following formats:aaxxxnaaxxxntaaxxxxnaaxxxxntaaxxxxxnaaxxxxxntaaxxxxxxnaaxxxxxxnt$HASPnnn$HASPnnnnDFHaann


DFHaannnDFHaannnnDFHnnDFHnntDFHnnnDFHnnnnEYUaannEYUaannnEYUaannnnEYUnnEYUnntEYUnnnEYUnnnn

where:v a represents an uppercase alphabetic character (A - Z).v n represents a numeric character (0 - 9).v x represents an uppercase alphabetic character or a numeric character.v t represents a type character (A, I, E, W, D, or S). If the first 3 characters of the

message ID are DFH or EYU, U is also a valid type character.


zOS-syslogd data source typeFor z/OS SYSLOG data sources of type zOS-syslogd, the log file splitterzOS-syslogd-Split breaks up the data into log records. The log record annotatorzOS-syslogd-Annotate annotates the log records.

File format

The following output is sample output from the error log:Dec 15 14:29:23 TVT7008 sshd[33554458]: debug3: mm_request_send entering: type 25


Table 42. Fields in sample output from error log for zOS-syslogd data source


Dec 15 14:29:23 Time stamp

TVT7008 System name

sshd Application

33554458 Process ID

debug3: mm_request_send entering: type25

Message

zOS-syslogd-Split log record splitter

To start a new record, the zOS-syslogd-Split splitter uses the time stamp that isshown at the beginning of each line of the log data. Any subsequent lines that donot begin with a time stamp are considered to be part of the same record, althoughtypically each record is only one line. The time stamp is a combination of themonth, the day of the month, and the time.

Reference 175

The year is determined by assuming that the record is no more than one year old.Records that are more than one year old are not supported.

In the following example, each of the two lines begins a new record:Mar 24 07:29:30 localhost syslogd: FSUM1220 syslogd: restartMar 24 07:29:30 localhost syslogd: FSUM1232 syslogd: running non-swappable

Based on the preceding example, the following two scenarios illustrate how theyear is determined, depending on the present time:

Assume that today is January 2015The year that is associated with these records is 2014 because the month inthe records is March, and in January 2015, March 2015 is in the future.

Assume that today is June 2015The year that is associated with these records is 2015 because March, themonth in the records, is three months earlier than the present time of June2015.

zOS-syslogd-Annotate annotationsThis reference describes the fields that are annotated by zOS-syslogd-Annotate.




Table 43. Log record fields that are annotated by zOS-syslogd-Annotate


Application The application identifier Text U U U


Text U U


MessageID The message identifier Text U U U


Text U U

MessageText The message text Text U



Text U U

processID The process identifier Text U U U


Text U U U



Table 43. Log record fields that are annotated by zOS-syslogd-Annotate (continued)




zOS-CICS-MSGUSR data source type: three variationsThe three variations of the z/OS SYSLOG zOS-CICS-MSGUSR data source type arezOS-CICS-MSGUSR, zOS-CICS-MSGUSRDMY, and zOS-CICS-MSGUSRYMD. Each variationcorresponds to a different date format in the time stamp for the data source.

For z/OS SYSLOG data sources of type zOS-CICS-MSGUSR, the log file splitterzOS-CICS-MSGUSR-Split breaks up the data into log records. The log recordannotator zOS-CICS-Annotate annotates the log records.

For z/OS SYSLOG data sources of type zOS-CICS-MSGUSRDMY, the log file splitterzOS-CICS-MSGUSRDMY-Split breaks up the data into log records. The log recordannotator zOS-CICS-DMY-Annotate annotates the log records.

For z/OS SYSLOG data sources of type zOS-CICS-MSGUSRYMD, the log file splitterzOS-CICS-MSGUSRYMD-Split breaks up the data into log records. The log recordannotator zOS-CICS-YMD-Annotate annotates the log records.

File format

For data sources of type zOS-CICS-MSGUSR, zOS-CICS-MSGUSRDMY, orzOS-CICS-MSGUSRYMD, the following format illustrates a sample CICS message in theCICS Transaction Server for z/OS MSGUSR log:DFHAP1901 04/04/2014 17:03:25 CMAS01 SPI audit log is available.


Table 44. Fields in sample CICS message in the MSGUSR log for zOS-CICS-MSGUSR datasource


DFHAP1901 Message ID

04/04/2014 17:03:25 (for thezOS-CICS-MSGUSR data source type, whichuses a default date format of MDY)

Time stamp

CMAS01 Application identifier (APPLID)

SPI audit log is available. Message text

The CICS messages in the CICS MSGUSR log can have different formats, and thelog might include custom application messages.

Example: Assume that your custom application generates messages to be added tothe CICS MSGUSR log. When the CICS annotator processes the messages, it hasthe following behavior:

If the messages follow the preceding format with the time stamps, the CICSannotator uses the time stamp for each message.

Reference 177

If the messages do not follow the preceding format and do not have timestamps, the CICS annotator assigns the current time as the time stamp for eachmessage.

zOS-CICS-MSGUSR-Split log record splitter

The MSGUSR log file starts each new record in the first column of the file.Subsequent lines of the same record are indented a number of spaces.

To start a new record, the zOS-CICS-MSGUSR-Split splitter uses the absence ofspaces in the first few columns. Also, the records that have a time stamp, but donot include a date, are assigned to the prior record, as shown in the followingexample:DFHZC3437 I 04/07/2014 17:43:09 CMAS01 N701 CSNE Node N701 action taken: NOCREATE

CLSDST ABTASK ABSEND ABRECV ((1) Module name: DFHZNAC)DFHZC3462 I 04/07/2014 17:43:09 CMAS01 N701 CSNE Node N701 session terminated.

((2) Module name: DFHZCLS)NQNAME N701,CSNE,17:43:09,NET1 N701DFHZC5966 I 04/07/2014 17:43:09 CMAS01 DELETE started for TERMINAL ( N701)

(Module name: DFHBSTZ).DFHZC6966 I 04/07/2014 17:43:09 CMAS01 Autoinstall delete for terminal N701

with netname N701 was successful.

In this example, the line

NQNAME N701,CSNE,17:43:09,NET1 N701

is assigned to the record for message DFHZC3462.

zOS-CICS-EYULOG data source type: three variationsThe three variations of the z/OS SYSLOG zOS-CICS-EYULOG data source type arezOS-CICS-EYULOG, zOS-CICS-EYULOGDMY, and zOS-CICS-EYULOGYMD. Each variationcorresponds to a different date format in the time stamp for the data source.

For z/OS SYSLOG data sources of type zOS-CICS-EYULOG, the log file splitterzOS-CICS-EYULOG-Split breaks up the data into log records. The log recordannotator zOS-CICS-Annotate annotates the log records.

For z/OS SYSLOG data sources of type zOS-CICS-EYULOGDMY, the log file splitterzOS-CICS-EYULOGDMY-Split breaks up the data into log records. The log recordannotator zOS-CICS-DMY-Annotate annotates the log records.

For z/OS SYSLOG data sources of type zOS-CICS-EYULOGYMD, the log file splitterzOS-CICS-EYULOGYMD-Split breaks up the data into log records. The log recordannotator zOS-CICS-YMD-Annotate annotates the log records.

File format

For data sources of type zOS-CICS-EYULOG, zOS-CICS-EYULOGDMY, orzOS-CICS-EYULOGYMD, the following format illustrates a sample CICS message in theCICS Transaction Server for z/OS EYULOG:06/29/2015 19:41:26 EYUVS0001I WUINCM03 CICSPLEX SM WEB USER INTERFACE INITIALIZATION STARTED

Table 45 on page 179 describes the fields in the sample output.


Table 45. Fields in sample CICS message in the EYULOG for zOS-CICS-EYULOG data source


EYUVS0001I Message ID

06/29/2015 19:41:26 (for thezOS-CICS-EYULOG data source type, whichuses a default date format of MDY)

Time stamp

WUINCM03 Application identifier (APPLID)

CICSPLEX SM WEB USER INTERFACEINITIALIZATION STARTED

Message text

zOS-CICS-EYULOG-Split log record splitter

The EYULOG file starts each new record in the first column of the file.

To start a new record, the zOS-CICS-EYULOG-Split splitter uses the time stamp of aline. Any subsequent lines that do not contain a time stamp are considered to bepart of the same record.

In the following example, each line begins a new record:06/10/2015 19:41:26 EYUVS0001I CMAS01 CICSPLEX SM WEB USER INTERFACE INITIALIZATION STARTED.06/10/2015 19:41:26 EYUVS0107I CMAS01 READING STARTUP PARAMETERS.06/10/2015 19:41:26 EYUVS0109I CMAS01 TCPIPHOSTNAME(VM30059.IBM.COM)06/10/2015 19:41:26 EYUVS0109I CMAS01 TCPIPPORT(12345)06/10/2015 19:41:26 EYUVS0109I CMAS01 CMCIPORT(12346)06/10/2015 19:41:26 EYUVS0109I CMAS01 DEFAULTCMASCTXT(CMAS03)06/10/2015 19:41:26 EYUVS0109I CMAS01 DEFAULTCONTEXT(PLEX3)06/10/2015 19:41:26 EYUVS0109I CMAS01 DEFAULTSCOPE(PLEX3)06/10/2015 19:41:26 EYUVS0109I CMAS01 AUTOIMPORTDSN(DFH.V5R2M0.CPSM.SEYUVIEW)06/10/2015 19:41:26 EYUVS0109I CMAS01 AUTOIMPORTMEM(EYUEA*)06/10/2015 19:41:26 EYUVS0108I CMAS01 STARTUP PARAMETERS READ.06/10/2015 19:41:26 EYUVS0101I CMAS01 Parameter service initialization complete.06/10/2015 19:41:28 EYUVS1063I CMAS01 Import ’ALL (OVERWRITE)’ initiated for user (IBMUSER) from data06/10/2015 19:41:28 EYUVS1063I CMAS01 set(DFH.V5R2M0.CPSM.SEYUVIEW), member (EYUEA*).

zOS-CICS-Annotate annotationsThis reference describes the fields that are annotated by zOS-CICS-Annotate,zOS-CICS-DMY-Annotate, and zOS-CICS-YMD-Annotate.




Table 46. Log record fields that are annotated by zOS-CICS-Annotate, zOS-CICS-DMY-Annotate, andzOS-CICS-YMD-Annotate



Component The component identifier, which shows thedomain or component that issues themessage

Text U U U

Reference 179

Table 46. Log record fields that are annotated by zOS-CICS-Annotate, zOS-CICS-DMY-Annotate, andzOS-CICS-YMD-Annotate (continued)



Text U U



Also, see “Message IDs.”

Text U U U


Text U U

MessageText The message text Text U



Text U U


Text U U U




Message IDs

A string is detected as a message ID if it matches one of the following formats:DFHnnDFHnntDFHnnnDFHnnntDFHnnnnDFHnnnntDFHaannDFHaanntDFHaannnDFHaannntDFHaannnnDFHaannnntEYUnnEYUnntEYUnnnEYUnnntEYUnnnnEYUnnnntEYUaannEYUaanntEYUaannnEYUaannntEYUaannnnEYUaannnnt


where:v a represents an uppercase alphabetic character (A - Z).v n represents a numeric character (0 - 9).v t represents a type character (A, I, E, W, D, S, or U).


Property reference for the data configuration fileThis reference lists all key-value pairs for the properties in the z/OS LogForwarder data configuration file.

System propertiesThe system properties specify system information that is global to the z/OS LogForwarder instance.

The system properties have the following key-value pairs.

Each key is listed with its value description, its value type, an indication ofwhether its value is required, and its default value.

systemInfo.localHostNameA name, typically a fully qualified domain name, that represents the localhost. This name must match the host name value of the data source that isconfigured in IBM Operations Analytics - Log Analysis and must beunique for each LPAR.

Value typeString

Required value?No

Default valueThe fully qualified domain name of the local host.

systemInfo.timeZoneThe time zone that is assumed for any logs that do not specify the timezone as part of the time stamp. The format for this value is defined inANSI X3.51-1975.

Value typeString

Required value?No

Default valueThe value of the TZ environment variable.

systemInfo.trustAllCertificatesA true or false indication of whether the z/OS Log Forwarder shouldtrust all security certificates when it communicates with an IBM OperationsAnalytics - Log Analysis server by using the Transport Layer Security(TLS) cryptographic protocol.

A value of true indicates that the z/OS Log Forwarder does not verify theidentity of the IBM Operations Analytics - Log Analysis system. Inproduction environments, you might want to set this value to false.

Reference 181

Value typeTrue or false

Required value?No

Default valuetrue

systemInfo.truststoreThe keystore that contains the security certificates for the z/OS LogForwarder to trust. This value is applicable only if the value of thesystemInfo.trustAllCertificates key is false.

Tip: If a relative path to this file is configured, that path is relative to thevalue of the ZLF_WORK environment variable.

Value typeString

Required value?No

Default value"truststore.jks"

systemInfo.truststorePasswordThe password for the truststore. This value is applicable only if the valueof the systemInfo.trustAllCertificates key is false.

If a value is specified for thesystemInfo.truststorePasswordInitializationVector key, thissystemInfo.truststorePassword value must be a hex string encoding of anencrypted password. Otherwise, it must be a cleartext password.

Value typeString

Required value?If a value is specified for thesystemInfo.truststorePasswordInitializationVector key, thesystemInfo.truststorePassword value is required. Otherwise, thesystemInfo.truststorePassword value is not required.

Default value"z1manager"

systemInfo.truststorePasswordInitializationVectorA hex string encoding of the initialization vector that is used to seed theencryption of the encrypted password that is specified by thesystemInfo.truststorePassword key. This value is applicable only if thevalue of the systemInfo.trustAllCertificates key is false.

Value typeString

Required value?No

Default valueNone


Log receiver propertiesThe log receiver properties specify the destination for the log data that is gatheredby the z/OS Log Forwarder instance.

Table 47 lists the types of log receiver that you can define in the z/OS LogForwarder data configuration file.

Table 47. Types of log receiver

Type of data receiver Interface description

IBM Operations Analytics - Log Analysis server Sends data to the DataCollector RepresentationalState Transfer (REST) API

Logstash Sends data to a Logstashserver in the receiver cluster ofthe scalable data collectionarchitecture

Log Analysis receiver propertiesThe Log Analysis receiver properties specify the IBM Operations Analytics - LogAnalysis server that receives the log data that is gathered by the z/OS LogForwarder instance.

The log receiver properties have the following key-value pairs.


logReceiver.hostThe fully qualified domain name of the IBM Operations Analytics - LogAnalysis server that receives the log data.

Value typeString

Required value?Yes

Default valueNone

logReceiver.portThe IBM Operations Analytics - Log Analysis server secure port number.Specify this value only if the default port number was changed on theserver.

Value typeNumber

Required value?No

Default value9987

logReceiver.passwordThe password that is associated with the user name that is defined by thelogReceiver.username key. If a value is provided for thelogReceiver.passwordInitializationVector key, this

Reference 183

logReceiver.password value must be a hex string encoding of anencrypted password. Otherwise, it must be a cleartext password.

Value typeString

Required value?If a value is specified for thelogReceiver.passwordInitializationVector key, thelogReceiver.password value is required. Otherwise, thelogReceiver.password value is not required.

Default value"unityadmin"

logReceiver.passwordInitializationVectorA hex string encoding of the initialization vector that is used to seed theencryption of the encrypted password that is specified by thelogReceiver.password key.

Value typeString

Required value?No

Default valueNone

logReceiver.typeThe type of log receiver.

For the IBM Operations Analytics - Log Analysis server, this value must beIOALA. Because the default value is IOALA, you can omit the property.

Value typeString

Required value?No

Default valueIOALA

logReceiver.usernameThe name of an IBM Operations Analytics - Log Analysis user that meetsthe following characteristics:v For IBM Operations Analytics - Log Analysis V1.3.0, the name of an IBM

Operations Analytics - Log Analysis user in the UnityAdmins group.v For IBM Operations Analytics - Log Analysis V1.3.1 and V1.3.2, the

name must be unityadmin, which is the default value.v For IBM Operations Analytics - Log Analysis V1.3.3, the name of any

IBM Operations Analytics - Log Analysis user.

Value typeString

Required value?No



Logstash receiver propertiesThe Logstash log receiver properties specify the Logstash server that receives thelog data that is gathered by the z/OS Log Forwarder instance. The Logstash servermust have an HTTP input.

The log receiver properties have the following key-value pairs.


logReceiver.hostThe fully qualified domain name of the Logstash server that receives thelog data.

Value typeString

Required value?Yes

Default valueNone

logReceiver.passwordThe password that is associated with the user name that is defined by thelogReceiver.username key. If a value is provided for thelogReceiver.passwordInitializationVector key, thislogReceiver.password value must be a hex string encoding of anencrypted password. Otherwise, it must be a cleartext password.

The z/OS Log Forwarder provides the user name and password to theLogstash server only if the Logstash HTTP input is configured with a userand password.

Value typeString

Required value?If a value is specified for thelogReceiver.passwordInitializationVector key, thelogReceiver.password value is required. Otherwise, thelogReceiver.password value is not required.


logReceiver.passwordInitializationVectorA hex string encoding of the initialization vector that is used to seed theencryption of the encrypted password that is specified by thelogReceiver.password key.

Value typeString

Required value?No

Default valueNone

logReceiver.portThe port number. The value must match the port number that is specifiedin the HTTP input for the Logstash server.

Reference 185

Value typeNumber

Required value?No

Default value8080

logReceiver.secureTransportA true or false indication of whether the z/OS Log Forwarder must use theTransport Layer Security (TLS) cryptographic protocol to encryptcommunication. A value of true indicates that the z/OS Log Forwarderuses TLS.

In production environments, you might want to set this value to true. Ifthe value is set to true, the z/OS Log Forwarder must be configured toeither trust all certificates or verify the identity of the Logstash or loadbalancer server.

Value typeTrue or false

Required value?No

Default valuefalse

logReceiver.typeThe type of log receiver.

For Logstash, this value must be LOGSTASH.

Value typeString

Required value?Yes

Default valueIOALA

logReceiver.usernameThe user that is defined for basic access authentication with Logstash.

The z/OS Log Forwarder provides the user name and password to theLogstash server only if the Logstash HTTP input is configured with a userand password.

Value typeString

Required value?No


Data gatherer propertiesThe data gatherer properties specify the sources of the log data that this z/OS LogForwarder instance gathers.


Table 48 lists the types of data gatherer that you can define in the z/OS LogForwarder data configuration file.

Table 48. Types of data gatherer

Type of data gatherer Data that the gatherer can retrieve

WAS HPEL gatherer v WebSphere Application Server for z/OSlog data from a local High PerformanceExtensible Logging (HPEL) repository

z/OS Data Set gatherer v Data from a Virtual Storage AccessMethod (VSAM) cluster that is anentry-sequenced data set (ESDS)

z/OS Job Log gatherer v CICS Transaction Server for z/OS log datafrom the EYULOG or MSGUSR data set ofthe job log for a specified job name

v WebSphere Application Server for z/OSlog data from the SYSOUT or SYSPRINTdata set of the job log for a specified jobname

z/OS NetView Message gatherer v NetView for z/OS messages from aNetView message provider by using theNetView program-to-program interface(PPI)

z/OS SYSLOG gatherer v Log data from the z/OS system log(SYSLOG) for the system

z/OS UNIX Log File gatherer v CICS Transaction Server for z/OS log datafrom a z/OS UNIX file that containsEYULOG or MSGUSR log data

v SMF data

v UNIX System Services system log(syslogd) data

v WebSphere Application Server for z/OSlog data from a z/OS UNIX file thatcontains SYSOUT or SYSPRINT log data

Other data gatherer types might be provided by third party organizations, such asIBM Business Partners, in cooperation with IBM. For information about configuringthose data gatherer types, see the documentation that is provided by the respectivethird party organization.

The property key for each data gatherer has an identifier (represented by the idvariable in this properties reference) that correlates the key with the other propertykeys that are associated with the same data gatherer. The identifier can be anystring, but a good option is to use incrementing numbers (starting with 1 for thefirst data gatherer), as illustrated in “Evaluation example of a data configurationfile” on page 86.

WAS HPEL gatherer propertiesThe WAS HPEL gatherer properties specify that WebSphere Application Server forz/OS log data is retrieved from a local High Performance Extensible Logging(HPEL) repository.

The WAS HPEL gatherer properties have the following key-value pairs.

Reference 187


Tip: Data sources that the z/OS Log Forwarder creates for HPEL data use the datasource type zOS-WAS-HPEL. Therefore, you do not specify a value fordataGatherers.id.dataSourceType even if a value is provided fordataGatherers.id.dataSourceName.

dataGatherers.id.collectionNameTo associate the data source with a collection of your choice, specify acollection name. If you specify a collection name, it must be the name of acollection that is defined in IBM Operations Analytics - Log Analysis. Acollection name is valid only if you specify a value for thedataGatherers.id.dataSourceName key.

If you do not specify a collection name, the data source is placed in adefault collection that is determined by IBM Operations Analytics - LogAnalysis.

Value typeString

Required value?No

Default valueNone

dataGatherers.id.dataSourceNameThe name that is assigned to the data source in IBM Operations Analytics -Log Analysis, if the z/OS Log Forwarder creates the data source.

The z/OS Log Forwarder creates the data source if both of the followingconditions are true:v A value is specified for the dataGatherers.id.dataSourceName key.v At initialization time for the z/OS Log Forwarder, the data source does

not exist.

Value typeString

Required value?No

Default valueNone

dataGatherers.id.filePathA unique identifier that represents the data origin.

The identifier must be a virtual or physical path that represents the HPELlog data.

Typically, this path must be specified only if the log and trace data for theserver is gathered separately. Because the HPEL log directory and HPELtrace directory are typically the same directory, one of the data gatherersmust specify a virtual path to ensure that the value of thedataGatherers.id.filePath key is always unique.

Value typeString


Required value?No

Default valueThe value of dataGatherers.id.logDirectory key, if that value isspecified. Otherwise, the value ofdataGatherers.id.traceDirectory key.

dataGatherers.id.logDirectoryThe HPEL log directory for a specific application server, if this datagatherer should gather log data for this server. The subdirectory logdatamust exist in this directory and must contain the HPEL log files.

If only trace data is gathered, do not specify a value fordataGatherers.id.logDirectory.

Value typeString

Required value?If no value is specified for the dataGatherers.id.traceDirectorykey, the dataGatherers.id.logDirectory value is required.Otherwise, the dataGatherers.id.logDirectory value is notrequired.

Default valueNone

dataGatherers.id.traceDirectoryThe HPEL trace directory for a specific application server, if this datagatherer should gather trace data for this server. The subdirectorytracedata must exist in this directory and must contain the HPEL tracefiles.

If only log data is gathered, do not specify a value for thedataGatherers.id.traceDirectory key.

Value typeString

Required value?If no value is specified for the dataGatherers.id.logDirectory key,the dataGatherers.id.traceDirectory value is required. Otherwise,the dataGatherers.id.traceDirectory value is not required.

Default valueNone

dataGatherers.id.typeThe type of data gatherer.

For the WAS HPEL gatherer, the value must be WAS_HPEL.

Value typeString

Required value?Yes

Default valueNone

Reference 189

z/OS Data Set gatherer propertiesThe z/OS Data Set gatherer properties specify where to retrieve data that is storedin a Virtual Storage Access Method (VSAM) cluster that is an entry-sequenced dataset (ESDS).

The z/OS Data Set gatherer properties have the following key-value pairs.




Value typeString

Required value?No

Default valueNone

dataGatherers.id.dataSetNameThe name of the VSAM cluster (ESDS) that contains the data to retrieve, inthe following format:x.y.z

Value typeString

Required value?Yes

Default valueNone



not exist.

Value typeString

Required value?No

Default valueNone


dataGatherers.id.dataSourceTypeThe type that is assigned to the data source in IBM Operations Analytics -Log Analysis.

Value typeString

Required value?If a value is specified for the dataGatherers.id.dataSourceNamekey, the dataGatherers.id.dataSourceType value is required.Otherwise, the dataGatherers.id.dataSourceType value is notrequired.

Default valueNone


If no value is specified for the dataGatherers.id.filePath key, the z/OSLog Forwarder sets this value to the value that is specified for thedataGatherers.id.dataSetName key.

Value typeString

Required value?No

Default valueThe value that is specified for the dataGatherers.id.dataSetNamekey

dataGatherers.id.pairedDataSetNameThe name of the VSAM cluster (ESDS) that, together with the cluster thatis specified by dataGatherers.id.dataSetName, contains the data toretrieve, in the following format:x.y.z

If this second data set is specified, and at least one of the paired data setsexists when the z/OS Log Forwarder starts, the z/OS Log Forwarderassumes that data is sequentially written from one data set to the other,and that only one data set contains the most current data at any point intime.

Value typeString

Required value?No

Default valueNone

dataGatherers.id.timeZoneThe time zone that is assumed for time stamps in the gathered data. Theformat for this value is defined in ANSI X3.51-1975.

Specify this key only if time stamps are not included in the gathered data.

Value typeString

Reference 191

Required value?No

Default valueThe value that is specified for the systemInfo.timeZone key. If thevalue for that key is not configured, the default value is the valueof the TZ environment variable.


For the z/OS Data Set gatherer, the value must be ZOS_DATA_SET.

Value typeString

Required value?Yes

Default valueNone

z/OS Job Log gatherer propertiesThe z/OS Job Log gatherer properties specify where to retrieve CICS TransactionServer for z/OS or WebSphere Application Server for z/OS log data.

CICS Transaction Server for z/OS log dataIs retrieved from the EYULOG or MSGUSR data set of the job log for aspecified job name.

WebSphere Application Server for z/OS log dataIs retrieved from the SYSOUT or SYSPRINT data set of the job log for aspecified job name.

The z/OS Job Log gatherer properties have the following key-value pairs.




Value typeString

Required value?No

Default valueNone




not exist.

If the value of the dataGatherers.id.jobName key contains one or more ofthe wildcard characters * or ?, the z/OS Log Forwarder appends_jobName_ddName to this dataGatherers.id.dataSourceName value for eachdata gatherer that is created. The jobName is the discovered job name, andthe ddName is the value of the dataGatherers.id.ddName key.

Value typeString

Required value?No

Default valueNone


The following values are valid:v WASSystemOut

v zOS-CICS-EYULOG



v zOS-CICS-MSGUSR



v zOS-WAS-SYSOUT

v zOS-WAS-SYSPRINT

Value typeString


Default valueNone

dataGatherers.id.ddNameOne of the following data definition names (ddnames):v For CICS Transaction Server for z/OS, EYULOG or MSGUSRv For WebSphere Application Server for z/OS, SYSPRINT or SYSOUT

Value typeString

Required value?Yes

Reference 193

Default valueNone


The following identifier is an example:jobName/ddName

If the value of the dataGatherers.id.jobName key contains one or more ofthe wildcard characters * or ?, the z/OS Log Forwarder appends/jobName/ddName to this dataGatherers.id.filePath value for each datagatherer that is created. The jobName is the discovered job name, and theddName is the value of the dataGatherers.id.ddName key.

If no dataGatherers.id.filePath value is provided, the z/OS LogForwarder sets the value to jobName/ddName.

Value typeString

Required value?Yes

Default valueNone

dataGatherers.id.jobNameOne of the following job names:v For CICS Transaction Server for z/OS, the name of the server job that is

used to start a CICS region or to start the CICSPlex System Manager.v For WebSphere Application Server for z/OS, the name of the server job

to retrieve log data from.

The job name value must meet the following criteria:v Must not begin with a numberv Must be no more than eight charactersv Must contain only the following characters:

– The letters A - Z– The numbers 0 - 9– Any of the following characters: $ # @ -– The wildcard characters ? or *, if you are defining multiple data

gatherers in a single definition

Remember: To define multiple data gatherers in a single definition,you can use wildcard characters in the job name. For moreinformation about using wildcard characters, see “Defining multipledata gatherers in a single z/OS Job Log gatherer definition” on page96.

Value typeString

Required value?Yes

Default valueNone




Value typeString

Required value?No



For the z/OS Job Log gatherer, the value must be ZOS_JOB_LOG.

Value typeString

Required value?Yes

Default valueNone

z/OS NetView Message gatherer propertiesThe z/OS NetView Message gatherer properties specify that NetView for z/OSmessages are retrieved from a NetView message provider by using the NetViewprogram-to-program interface (PPI).

The z/OS NetView Message gatherer properties have the following key-valuepairs.




Value typeString

Required value?No

Default valueNone

Reference 195



not exist.

Value typeString

Required value?No

Default valueNone

dataGatherers.id.domainNameThe name of the NetView domain from which to gather messages.

Important: Do not define multiple z/OS NetView Message gatherers withthe same NetView domain name. Each gatherer must reference a uniquedomain name.

Value typeString

Required value?Yes

Default valueNone


Value typeString

Required value?No

Default valueThe value of dataGatherers.id.domainName key.


For the z/OS NetView Message gatherer, the value must be ZOS_NETVIEW.

Value typeString

Required value?Yes

Default valueNone

z/OS SYSLOG gatherer propertiesThe z/OS SYSLOG gatherer properties specify that log data is retrieved from thez/OS system log (SYSLOG) for the system.


The z/OS SYSLOG gatherer properties have the following key-value pairs.


Tips:

v Data sources that the z/OS Log Forwarder creates for the ZOS_SYS type of logdata use the data source type zOS-SYSLOG-Console. Therefore, you do not specifya value for dataGatherers.id.dataSourceType even if a value is provided fordataGatherers.id.dataSourceName.

v The dataGatherers.id.timeZone key-value pair is not valid for z/OS SYSLOGgatherer properties.



Value typeString

Required value?No

Default valueNone



not exist.

Value typeString

Required value?No

Default valueNone

dataGatherers.id.filePath

A unique identifier that represents the data origin.

Value typeString

Required value?No

Reference 197

Default valueSYSLOG

dataGatherers.id.typeThe type of data gatherer. For the z/OS SYSLOG gatherer, the value mustbe ZOS_SYS.

Value typeString

Required value?Yes

Default valueNone

z/OS UNIX Log File gatherer propertiesThe z/OS UNIX Log File gatherer properties specify where to retrieve CICSTransaction Server for z/OS log data, network data, SMF data, or WebSphereApplication Server for z/OS log data.

CICS Transaction Server for z/OS log dataIs retrieved from a z/OS UNIX file that contains EYULOG or MSGUSR logdata.

Network dataIs retrieved from a z/OS UNIX file that contains UNIX System Servicessystem log (syslogd) log data.

SMF dataIs retrieved from a z/OS UNIX file that contains SMF data.

WebSphere Application Server for z/OS log dataIs retrieved from a z/OS UNIX file that contains SYSOUT or SYSPRINTlog data.

The z/OS UNIX Log File gatherer properties have the following key-value pairs.




Value typeString

Required value?No

Default valueNone




not exist.

Value typeString

Required value?No

Default valueNone


The following values are valid:v WASSystemOut

v zOS-CICS-EYULOG



v zOS-CICS-MSGUSR



v zOS-SMF30

v zOS-SMF80

v zOS-SMF110_E

v zOS-SMF110_S_10

v zOS-SMF120

v zOS-syslogd

v zOS-WAS-SYSOUT

v zOS-WAS-SYSPRINT

Value typeString


Default valueNone


Reference 199

The identifier must be the absolute path, including the file name, of a logfile that contains EYULOG, MSGUSR, SMF, syslogd, SYSOUT, orSYSPRINT data.

For a rolling log, use wildcard characters to specify the absolute path ofthe set of logically associated log files. For more information about the filepaths for rolling logs, see “File path pattern for a rolling log” on page 47.

Value typeString

Required value?Yes

Default valueNone



Value typeString

Required value?No



For the z/OS UNIX Log File gatherer, the value must be ZOS_UNIX_FILE.

Value typeString

Required value?Yes

Default valueNone

Sample searchesIf you installed any of the optional sample searches (sometimes called Quick Searchsamples) in IBM Operations Analytics - Log Analysis, they are available in the zosfolder in the Saved Searches navigator of the Log Analysis user interface.

Some sample searches are organized into subfolders under the zos folder.

To use a sample search to search logs, double-click the title of the search in theSaved Searches navigator.

WebSphere Application Server for z/OS sample searchesYou can install these optional WebSphere Application Server for z/OS samplesearches for use with the WebSphere Application Server for z/OS Insight Pack.


WAS Error MessagesSearches for WebSphere Application Server for z/OS messages thatoccurred in the last day and that indicate an error occurred.

WAS ExceptionsSearches for occurrences of Java exceptions in the WebSphere ApplicationLogs during the last day.

z/OS Network sample searchesYou can install these optional z/OS Network sample searches for use with thez/OS Network Insight Pack.

Searches for common network errors

The following samples search for common network error messages:v ATTLS Error Messagesv CSSMTP Error Messagesv Device Error Messagesv FTP Error Messagesv IKED Error Messagesv IPSEC Error Messagesv OMPROUTE Error Messagesv PAGENT Error Messagesv Storage Error Messagesv syslogd FTPD Messagesv syslogd Messagesv syslgod SSHD Messagesv syslogd TELNETD Messagesv TCP/IP Error Messagesv TN3270 Telnet Error Messagesv VTAM® Connection Error Messagesv VTAM CSM Error Messagesv VTAM Storage Error Messages

NetView for z/OS sample searches

NetView MessagesSearches for NetView for z/OS messages that occurred during the last day.

NetView Action, Decision, ErrSearches for NetView for z/OS messages that occurred during the last dayand that indicate any of the following situations:v Immediate action is required.v A decision is required.v An error occurred.

NetView AutomationSearches for a set of predefined NetView for z/OS messages that indicatethat automation table violations possibly occurred during the last day.

Reference 201

NetView Command AuthorizationSearches for a set of predefined NetView for z/OS messages that indicatethat command authorization table violations possibly occurred during thelast day.

NetView Resource LimitsSearches for a set of predefined NetView for z/OS messages that indicatethat resource limits or storage thresholds were possibly exceeded duringthe last day.

NetView SecuritySearches for a set of predefined NetView for z/OS messages that indicatethat insufficient access authority or security environment violationspossibly occurred during the last day.

z/OS SMF sample searchesYou can install these optional z/OS SMF sample searches for use with the z/OSSMF Insight Pack.

CICS Transaction Server for z/OS sample searches:

All ExceptionsSearches for CICS Transaction Server for z/OS exceptions thatoccurred in the last day.

Job PerformanceSearches for records that occurred in the last day and have aprogram name of DFHSIP or EYU9XECS.

Policy ExceptionsSearches for CICS Transaction Server for z/OS SMF policy-basedexceptions that occurred in the last day.

Transaction SummarySearches for CICS Transaction Server for z/OS transactionsummary interval records that occurred in the last day.

Transaction Summary - WeekSearches for CICS Transaction Server for z/OS end-of-daytransaction summary records that occurred in the last week.

Transaction Times at MAXTASKSSearches for CICS Transaction Server for z/OS transaction recordswhere the number of active user transactions equaled the specifiedmaximum allowed number of user transactions.

Wait on Storage ExceptionsSearches for CICS storage manager messages and CICS TransactionServer for z/OS SMF Wait on Storage exceptions that occurred inthe last day.

DB2 for z/OS PerformanceSearches for records that occurred in the last day and have a programname of DSNYASCP or DSNADMT0.

IMS for z/OS PerformanceSearches for records that occurred in the last day and have a programname of DFSAMVRC0, DFSRRC00, or DXRRLM00.

MQ for z/OS PerformanceSearches for records that occurred in the last day and have a programname of CSQXJST or CSQYASCP.


z/OS Job PerformanceSearches for records that occurred in the last day and have an assignedprogram name.

Security sample searches:To obtain results from these searches, the Resource Access Control Facility(RACF) must be active and protecting the resources or commands that arethe subject of each search.

RACF SETROPTS Commands IssuedSearches for SETROPTS commands that were issued during the lastday.

RACF Accesses of config filesSearches for any file with the extension .config that was accessedby RACF during the last day.

RACF Activity for OperationsSearches for any events that were caused by a user with the RACFOperation attribute during the last day.

RACF Logons and CommandsSearches for logons and commands that were issued from aspecific terminal ID (TermID field) during the last day. The defaultvalue for the TermID field is *.

RACF CHOWN, CHGRP, CHMOD CmdsSearches for occurrences of the UNIX commands CHOWN,CHGRP, and CHMOD that were issued during the last day.

RACF Data Set Access SuccessesSearches for successful attempts by RACF to access data setsduring the last day.

RACF Failed Access AttemptsSearches for unsuccessful attempts by RACF to access data setsduring the last day.

WebSphere Application Server for z/OS sample searches:To obtain results from these searches, WebSphere Application Server forz/OS must be active and must be configured to create SMF 120 subtype 9records.

Controller Managed JavaBeansSearches for the managed JavaBeans requests that are processed bythe WebSphere Application Server Controller during the last day.

Cont Requests Not InternalsSearches for the requests for controller processing that are notattributed to internal WebSphere processing during the last day.

Activity for All AppsSearches for the requests for processing that are attributed toWebSphere Application Server for z/OS applications.

Apps with Nonzero Dispatch TCBSearches for the requests for processing that are attributed toWebSphere Application Server for z/OS applications with non-zerodispatch Task Control Block (TCB) time.

Reference 203

z/OS SYSLOG sample searchesYou can install these optional z/OS SYSLOG sample searches for use with thez/OS SYSLOG Insight Pack.v “CICS Transaction Server for z/OS sample searches”v “DB2 for z/OS sample searches”v “IMS for z/OS sample searches” on page 205v “Security for z/OS sample searches” on page 206v “MQ for z/OS sample searches” on page 207

CICS Transaction Server for z/OS sample searches

CICS TS MessagesSearches for CICS Transaction Server messages that occurred during thelast day. CICS Transaction Server messages start with the prefix DFH or EYU.

CICS Action, Decision, or ErrorSearches for CICS messages that occurred during the last day and thatindicate any of the following situations:v Immediate action is required.v A decision is required.v An error occurred.

The search is based on the CICS message IDs and on an action code of A,D, E, S, or U during the last day.

CICS TS Abend or SevereSearches for CICS Transaction Server messages that have both of thefollowing characteristics:v The messages occurred during the last day.v The messages have the format DFHccxxxx, where cc represents a

component identifier (such as SM for Storage Manager), and xxxx iseither 0001 or 0002 (which indicates an abend or severe error in thespecified component).

For example: This sample would search for DFHSM0001 but not forDFH0001.

CICS TS Key MessagesSearches for a set of predefined message numbers to determine whetherany of the corresponding messages occurred during the last day.

CICS TS Short on StorageSearches for CICS Transaction Server for z/OS messages that indicate thata storage shortage occurred during the last day.

CICS TS Storage ViolationsSearches for CICS Transaction Server for z/OS messages that indicate thata storage violation occurred during the last day.

DB2 for z/OS sample searches

DB2 MessagesSearches for DB2 messages that occurred during the last day. DB2messages start with the prefix DSN.


DB2 Action, Decision, or ErrorsSearches for DB2 messages that occurred during the last day and thatindicate any of the following situations:v Immediate action is required.v A decision is required.v An error occurred.

DB2 Data Set MessagesSearches for DB2 messages that occurred during the last day and thatindicate any of the following situations:v Failure of a data set definitionv Failure of a data set extendv Impending space shortage

DB2 Data Sharing MessagesSearches for internal resource lock manager (IRLM) messages that wereissued to DB2 during the last day and that indicate at least one of thefollowing situations:v The percentage of available lock structure capacity is low.v An error occurred when IRLM used the specified z/OS automatic restart

manager (ARM) function.

DB2 Lock Conflict MessagesSearches for DB2 messages that occurred during the last day and thatindicate that a plan was denied an IRLM lock due to a detected deadlockor timeout.

DB2 Log Data Set MessagesSearches for messages that indicate that DB2 log data sets are full, arebecoming full, or could not be allocated during the last day.

DB2 Log Frequency MessagesSearches for DB2 messages that occurred during the last day and thatindicate that log archives were offloaded or are waiting to be offloaded.

DB2 Pool Shortage MessagesSearches for DB2 messages that occurred during the last day and thatindicate that the amount of storage in the group buffer pool (GBP)coupling facility structure that is available for writing new pages is low orcritically low.

IMS for z/OS sample searches

IMS MessagesSearches for IMS messages that occurred during the last day. IMS messagesstart with any of the following prefixes:BPE, CQS, CSL, DFS, DSP, DXR, ELX, FRP, HWS, MDA, PCB, PGE, SEG, or SFL

IMS Action, Decision, or ErrorSearches for IMS messages that occurred during the last day and thatindicate any of the following situations:v Immediate action is required.v A decision is required.v An error occurred.

The search is based on the IMS message IDs and on an action code of A, E,W, or X during the last day.

Reference 205

IMS Security ViolationsSearches for error messages that indicate security violations that weredetected during the last day.

IMS Abend MessagesSearches for messages that indicate abends that were detected during thelast day.

IMS Common Queue Server MsgsSearches for IMS Common Queue Server component messages thatoccurred during the last day. IMS Common Queue Server messages startwith the prefix CQS.

IMS Resources in Waiting ErrorSearches for error messages that occurred during the last day and thatindicate that a resource is waiting on other resources to become available.

IMS DB Recovery Control ErrorsSearches for error messages that occurred for the DB Recovery Controlcomponent during the last day. DB Recovery Control messages start withthe prefix DSP.

IMS Connect MessagesSearches for IMS Connect component messages that occurred during thelast day. IMS Connect messages start with the prefix HWS.

IMS Log MessagesSearches for messages that occurred during the last day and that indicatehow often IMS logs are rolled.

IMS Stopped ResourcesSearches for messages that occurred during the last day and that indicatethe IMS and related components that stopped running.

IMS Pool IssuesSearches for messages that occurred during the last day and that indicateIMS pool-related issues.

IMS Terminal Related MsgsSearches for messages that occurred during the last day and that indicateIMS terminal-related issues or terminals that stopped receiving messages.

IMS Locking MessagesSearches for messages that occurred during the last day and that indicatethe IMS resources that are locked.

Security for z/OS sample searches

RACF MessagesSearches for RACF messages that occurred during the last day. ResourceAccess Control Facility (RACF) messages start with either of the followingprefixes:v ICHv IRR

RACF Action, Decision, or ErrorSearches for RACF messages that occurred during the last day and thatindicate any of the following situations:v Immediate action is required.v A decision is required.v An error occurred.


RACF Insufficient AccessSearches for RACF messages that occurred during the last day and thatindicate insufficient access authority.

RACF Insufficient AuthoritySearches for RACF messages that occurred during the last day and thatindicate insufficient authority.

RACF Invalid Logon AttemptsSearches for RACF messages that occurred during the last day and thatindicate invalid logon attempts.

MQ for z/OS sample searches

MQ MessagesSearches for MQ messages that occurred during the last day. MQ messagesstart with the prefix CSQ.

MQ Action, Decision, or ErrorSearches for MQ messages that occurred during the last day and thatindicate any of the following situations:v Immediate action is required.v A decision is required.v An error occurred.

The search is based on the MQ message IDs and on an action code of A, D,or E during the last day.

MQ Queue Manager StorageSearches for messages that indicate whether MQ queue manager requiredmore storage at any time during the last day.

MQ Logs Start and StopSearches for messages that are related to the starting, stopping, andflushing of the MQ log data sets during the last day.

MQ Key MessagesSearches for a set of predefined message numbers to determine whetherany of the corresponding messages occurred in the last day.

MQ Interesting InformationalSearches for a set of predefined message numbers for informationalmessages that can be analyzed to determine whether any of thecorresponding messages occurred during the last day.

MQ Channel Initiator ErrorsSearches for error messages that indicate the occurrence of MQ channelinitiator errors during the last day.

MQ Channel ErrorsSearches for error messages that indicate the occurrence of MQ channelerrors during the last day.

MQ Buffer Pool ErrorsSearches for error messages that indicate the occurrence of MQ buffer poolerrors during the last day.

Reference 207

SMF type 80-related records that are created by the SMF real-time dataprovider

The SMF real-time data provider collects a subset of the SMF data that is generatedby RACF. This reference describes the types of records that the SMF real-time dataprovider creates as it extracts relevant data from SMF type 80 records.

The SMF real-time data provider creates the following record types:v SMF80_COMMAND

v SMF80_LOGON

v SMF80_OMVS_RES_1

v SMF80_OMVS_RES_2

v SMF80_OMVS_SEC_1

v SMF80_OMVS_SEC_2

v SMF80_OPERATION

v SMF80_RESOURCE

From each SMF type 80 record that it collects, the SMF real-time data provider usesthe following information to determine what data to extract:v SMF event in the SMF80EVT fieldv RACF event code qualifier in the SMF80EVQ field

The data provider excludes SMF events that occur for hierarchical storagemanagement (HSM), for example, where the user ID SMF80USR = HSM.

For more information about SMF record type 80 records, see the following topics inthe IBM Knowledge Center:v SMF record type 80: RACF processing recordv Format of SMF record type 80 recordsv SMF record type 80 event codes and event code qualifiers

SMF80_COMMAND record typeSMF record type 80 records for events 8 - 25 are created when RACF commandsfail because the user who ran them does not have sufficient authority. Relevantfields from these SMF event records are stored in the SMF80_COMMAND records thatare created by the SMF real-time data provider.

Table 49 describes the event code qualifiers for events 8 - 25, which provide moreinformation about why the command failed.

Table 49. SMF80_COMMAND record type: event code qualifiers for events 8 - 25

Event code qualifier Description

1 Insufficient authority

2 Keyword violations detected

3 Successful listing of data sets

4 System error in listing of data sets



https://www.ibm.com/support/knowledgecenter/SSLTBW_1.13.0/com.ibm.zos.r13.icha300/type80.htm

https://www.ibm.com/support/knowledgecenter/SSLTBW_1.13.0/com.ibm.zos.r13.icha300/type80fmt.htm

https://www.ibm.com/support/knowledgecenter/SSLTBW_1.13.0/com.ibm.zos.r13.icha300/eventt.htm

SMF80_LOGON record typeSMF record type 80 records for event 1 are created when RACF authentication failsbecause of incorrect user credentials, which prevents the user from accessing thesystem. Relevant fields from this SMF event record are stored in the SMF80_LOGONrecords that are created by the SMF real-time data provider.

Table 50 describes the event code qualifiers for event 1, which provide moreinformation about why the logon failed.

Table 50. SMF80_LOGON record type: event code qualifiers for event 1


1 Invalid password

2 Invalid group

3 Invalid object identifier (OID) card

4 Invalid terminal/console

5 Invalid application

6 Revoked user ID attempting access

7 User ID automatically revoked

9 Undefined user ID

10 Insufficient security label authority

11 Not authorized to security label

14 System now requires more authority

15 Remote job entry—job not authorized

16 Surrogate class is inactive

17 Submitter is not authorized by user

18 Submitter is not authorized to security label

19 User is not authorized to job

20 Warning—insufficient security labelauthority

21 Warning—security label missing from job,user, or profile

22 Warning—not authorized to security label

23 Security labels not compatible

24 Warning—security labels not compatible

25 Current password has expired

26 Invalid new password

27 Verification failed by installation

28 Group access has been revoked

29 Object identifier (OID) card is required

30 Network job entry—job not authorized

31 Warning—unknown user from trusted nodepropagated

32 Successful initiation using PassTicket

33 Attempted replay of PassTicket

Reference 209

Table 50. SMF80_LOGON record type: event code qualifiers for event 1 (continued)


34 Client security label not equivalent toservers

35 User automatically revoked due to inactivity

36 Passphrase is not valid

37 New passphrase is not valid

38 Current passphrase has expired

39 No RACF user ID found for distributedidentity

SMF80_OMVS_RES record typesSMF record type 80 records for events 28 - 30 are created when the following z/OSUNIX operations occur: directory search, check access to directory, or check accessto file. Relevant fields from these SMF event records are stored in theSMF80_OMVS_RES_1 and SMF80_OMVS_RES_2 records that are created by the SMFreal-time data provider.

Table 51 describes the event code qualifiers for events 28 - 30, which provide moreinformation about the operation results.

Table 51. SMF80_OMVS_RES_1 and SMF80_OMVS_RES_2 record types: event code qualifiers forevents 28 - 30


0 Access allowed

1 Not authorized to search directory

2 Security label failure

SMF80_OMVS_SEC record typesSMF record type 80 records for events 31 and 33 - 35 are created when the z/OSUNIX commands CHAUDIT, CHMOD, or CHOWN are entered, or when the SETID bits fora file are cleared. Relevant fields from these SMF event records are stored in theSMF80_OMVS_SEC_1 and SMF80_OMVS_SEC_2 records that are created by the SMFreal-time data provider.

Table 52, Table 53 on page 211, Table 54 on page 211, and Table 55 on page 211describe the event code qualifiers for events 31 and 33 - 35, which provide moreinformation about the operation results.

Table 52. SMF80_OMVS_SEC_1 and SMF80_OMVS_SEC_2 record types: event code qualifiers forevent 31


0 File's audit options changed

1 Caller does not have authority to changeuser audit options of specified file

2 Caller does not have authority to changeauditor audit options





0 File's mode changed

1 Caller does not have authority to changemode of specified file




0 File's owner or group owner changed

1 Caller does not have authority to changeowner or group owner of specified file




0 S_ISUID, S_ISGID, and S_ISVTX bits changedto zero (write).

SMF80_OPERATION record typeSMF record type 80 records for events 2 - 7 are created when a z/OS resource thatis protected by RACF is updated, deleted, or accessed by a user that is defined toRACF with the SPECIAL attribute. Relevant fields from these SMF event records arestored in the SMF80_OPERATION records that are created by the SMF real-time dataprovider.

Table 56, Table 57 on page 212, Table 58 on page 212, Table 59 on page 213, Table 60on page 213, and Table 61 on page 213 describe the event code qualifiers for events2 - 7, which provide more information about the operation results.

Table 56. SMF80_OPERATION record type: event code qualifiers for event 2


0 Successful access


2 Profile not found—RACFIND specified onmacro

3 Access permitted due to warning

4 Failed due to PROTECTALL SETROPTS

5 Warning issued due to PROTECTALLSETROPTS

6 Insufficient category/SECLEVEL


8 Security label missing from job, user, orprofile

Reference 211

Table 56. SMF80_OPERATION record type: event code qualifiers for event 2 (continued)



10 Warning—data set not cataloged

11 Data set not cataloged

12 Profile not found—required for authoritychecking

13 Warning—insufficient category/SECLEVEL

14 Warning—non-main execution environment

15 Conditional access allowed via basic modeprogram



0 Successful processing of new volume



3 Less specific profile exists with differentsecurity label



0 Successful rename

1 Invalid group

2 User not in group


4 Resource name already defined

5 User not defined to RACF

6 Resource not protected SETROPTS

7 Warning——resource not protectedSETROPTS

8 User in second qualifier is not RACF defined

9 Less specific profile exists with differentsecurity label


11 Resource not protected by security label

12 New name not protected by security label

13 New security label must dominate oldsecurity label


15 Warning—resource not protected by securitylabel

16 Warning—new name not protected bysecurity label


Table 58. SMF80_OPERATION record type: event code qualifiers for event 4 (continued)


17 Warning—new security label must dominateold security label



0 Successful scratch

1 Resource not found

2 Invalid volume



0 Successful deletion



0 Successful definition

1 Group undefined

2 User not in group


4 Resource name already defined

5 User not defined to RACF

6 Resource not protected

7 Warning—resource not protected

8 Warning—security label missing from job,user, or profile


10 User in second qualifier in not defined toRACF


12 Less specific profile exists with a differentsecurity label

SMF80_RESOURCE record typeSMF record type 80 records for event 2 are created when a z/OS resource that isprotected by RACF is updated, deleted, or accessed by a user. Relevant fields fromthese SMF event records are stored in the SMF80_RESOURCE records that are createdby the SMF real-time data provider.

Table 62 on page 214 describes the event code qualifiers for event 2, which providemore information about the operation results.

Reference 213

Table 62. SMF80_RESOURCE record type: event code qualifiers for event 2


0 Successful access


2 Profile not found—RACFIND specified onmacro

3 Access permitted due to warning

4 Failed due to PROTECTALL SETROPTS

5 Warning issued due to PROTECTALLSETROPTS

6 Insufficient category/SECLEVEL


8 Security label missing from job, user, orprofile


10 Warning—data set not cataloged

11 Data set not cataloged

12 Profile not found—required for authoritychecking

13 Warning—insufficient category/SECLEVEL

14 Warning—non-main execution environment

15 Conditional access allowed via basic modeprogram

Variable reference for the environment configuration fileThis reference lists the environment variables that you can update in the z/OS LogForwarder environment configuration file.

JRELIBThis variable specifies the fully qualified path to a set of native librariesthat are required by the Java Runtime Environment (31-bit).

Update this variable only in the rare situation where the files are not in thedefault location.

The default value is /usr/lib/java_runtime.

JRELIB64This variable specifies the fully qualified path to a set of native librariesthat are required by the Java Runtime Environment (64-bit).


The default value is /usr/lib/java_runtime64.

REGJAR

This variable specifies the fully qualified path to the ifaedjreg.jar file,which provides access to z/OS product registration services.



The default value is /usr/include/java_classes/ifaedjreg.jar.

RESOLVER_CONFIGThis variable is optional.

Include a value for this variable if you use the variable to provide thelocation of the resolver configuration file. For more information about theresolver configuration file, see “Requirements for running the z/OS LogForwarder” on page 23.

No default value is assigned for this variable.

TZ This variable is required.

The variable specifies the time zone for the z/OS Log Forwarder. For adescription of the TZ variable, see the information about the format of theTZ environment variable in the z/OS UNIX System Services CommandReference.

The default value is EST5EDT (Eastern Standard time with a correction forDaylight Saving Time).

Important: The z/OS Log Forwarder does not obtain the time zone settingfrom z/OS IPL parameters (for example, the CLOCK IPL member). If theTZ variable is not set, the time zone offset defaults to CoordinatedUniversal Time (UTC), which is equivalent to a time zone offset of zero.

ZLF_GATHERERThis variable is required only if you collect data by using a data gatherertype that is provided by a third party organization in cooperation withIBM. In that situation, the third party organization must provide the valuefor this variable.

ZLF_HOMEThis variable is required.

The variable specifies the home (or installation) directory for the z/OS LogForwarder.

The default value is /usr/lpp/IBM/zscala/V2R2. Change the value if adifferent installation directory is used for the z/OS Log Forwarder.

ZLF_JAVA_HOMEThis variable is required.

The variable specifies the Java home (or installation) directory.

The following value is the default value, which is for Java 1.6 64-bit:/usr/lpp/java/J6.0_64

You can change this value to refer to any installed version of Java that issupported by the z/OS Log Forwarder.

ZLF_LOGThis variable is required.

The variable specifies the directory that the z/OS Log Forwarder uses tofind the logging.properties file.

Important: Unless IBM Software Support requests that you change thelogging.properties file, you should leave this file in the default

Reference 215

installation directory. The default value is /usr/lpp/IBM/zscala/V2R2/samples. Change the value if a different directory is used for the file.

ZLF_WAS_PLUGINS_ROOTThis variable is required only if you collect log data from WebSphereApplication Server High Performance Extensible Logging (HPEL).

The variable specifies the WebSphere Application Server installation rootdirectory for Web Server Plug-ins. This directory contains thecom.ibm.hpel.logging.jar file that is used by the z/OS Log Forwarder toretrieve log data from HPEL.

The default value is /usr/IBM/WebSphere/Plugins. Change this value torefer to the installation root directory for Web Server Plug-ins.

ZLF_WORKThis variable is required.

The variable specifies the z/OS Log Forwarder working directory, whichcontains the files that are necessary for retaining the state of the z/OS LogForwarder.

The default value is /etc/IBM/zscala/V2R2.

Variable reference for the z/OS Log Forwarder start procedureThis reference lists the variables that you can update in the z/OS Log Forwarderstart procedure, which is based on the sample GLAPROC procedure.

ENVDIR procedure variableAlthough this variable is optional, the best practice is to specify thevariable.

The variable specifies the directory where the data configuration file andthe environment configuration file are located. To indicate the variable, theoption identifier -e precedes the directory specification, as shown in thefollowing example:’-e /etc/IBM/zscala/V2R2’

The following directory is the default directory that is used if the ENVDIRprocedure variable is not specified:/usr/lpp/IBM/zscala/V2R2/samples

GLABASE procedure variableThis variable is required.

The variable specifies the directory where the startup.sh script is located.

The following directory is the default installation directory for thestartup.sh script:/usr/lpp/IBM/zscala/V2R2/samples

Change the value if a different installation directory was used during theSMP/E installation.

Variable reference for the z/OS SMF real-time data provider startprocedure

This reference lists the variables that you can update in the z/OS SMF real-timedata provider start procedure, which is based on the sample GLASMF procedure.


GLAJCLThis variable specifies the installation data set of the sample code.

The default data set is ’ZSCALA.V2R2M0.SGLASAMP’. If the HLQ of thesample code is not the default location, update this value.

GLALMODThis variable specifies the installation data set of the load modules.

The default data set is ’ZSCALA.V2R2M0.SGLALINK’. If the HLQ of the loadmodule data set is not the default location, update this value.

Reference 217

Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document inother countries. Consult your local IBM representative for information on theproducts and services currently available in your area. Any reference to an IBMproduct, program, or service is not intended to state or imply that only that IBMproduct, program, or service may be used. Any functionally equivalent product,program, or service that does not infringe any IBM intellectual property right maybe used instead. However, it is the user's responsibility to evaluate and verify theoperation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matterdescribed in this document. The furnishing of this document does not grant youany license to these patents. You can send license inquiries, in writing, to:

IBM Director of LicensingIBM CorporationNorth Castle DriveArmonk, NY 10504-1785U.S.A.

For license inquiries regarding double-byte character set (DBCS) information,contact the IBM Intellectual Property Department in your country or sendinquiries, in writing, to:

Intellectual Property LicensingLegal and Intellectual Property LawIBM Japan, Ltd.19-21, Nihonbashi-Hakozakicho, Chuo-kuTokyo 103-8510, Japan

The following paragraph does not apply to the United Kingdom or any othercountry where such provisions are inconsistent with local law:INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THISPUBLICATION “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHEREXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIEDWARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESSFOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express orimplied warranties in certain transactions, therefore, this statement may not applyto you.

This information could include technical inaccuracies or typographical errors.Changes are periodically made to the information herein; these changes will beincorporated in new editions of the publication. IBM may make improvementsand/or changes in the product(s) and/or the program(s) described in thispublication at any time without notice.

Any references in this information to non-IBM Web sites are provided forconvenience only and do not in any manner serve as an endorsement of those Websites. The materials at those Web sites are not part of the materials for this IBMproduct and use of those Web sites is at your own risk.


IBM may use or distribute any of the information you supply in any way itbelieves appropriate without incurring any obligation to you.

Licensees of this program who wish to have information about it for the purposeof enabling: (i) the exchange of information between independently createdprograms and other programs (including this one) and (ii) the mutual use of theinformation which has been exchanged, should contact:

IBM Corporation2Z4A/10111400 Burnet RoadAustin, TX 78758U.S.A.

Such information may be available, subject to appropriate terms and conditions,including in some cases, payment of a fee.

The licensed program described in this document and all licensed materialavailable for it are provided by IBM under terms of the IBM Customer Agreement,IBM International Program License Agreement or any equivalent agreementbetween us.

Any performance data contained herein was determined in a controlledenvironment. Therefore, the results obtained in other operating environments mayvary significantly. Some measurements may have been made on development-levelsystems and there is no guarantee that these measurements will be the same ongenerally available systems. Furthermore, some measurements may have beenestimated through extrapolation. Actual results may vary. Users of this documentshould verify the applicable data for their specific environment.

Information concerning non-IBM products was obtained from the suppliers ofthose products, their published announcements or other publicly available sources.IBM has not tested those products and cannot confirm the accuracy ofperformance, compatibility or any other claims related to non-IBM products.Questions on the capabilities of non-IBM products should be addressed to thesuppliers of those products.

All statements regarding IBM's future direction or intent are subject to change orwithdrawal without notice, and represent goals and objectives only.

This information contains examples of data and reports used in daily businessoperations. To illustrate them as completely as possible, the examples include thenames of individuals, companies, brands, and products. All of these names arefictitious and any similarity to the names and addresses used by an actual businessenterprise is entirely coincidental.

Terms and conditions for product documentationPermissions for the use of these publications are granted subject to the followingterms and conditions.

Applicability

These terms and conditions are in addition to any terms of use for the IBMwebsite.


Personal use

You may reproduce these publications for your personal, noncommercial useprovided that all proprietary notices are preserved. You may not distribute, displayor make derivative work of these publications, or any portion thereof, without theexpress consent of IBM.

Commercial use

You may reproduce, distribute and display these publications solely within yourenterprise provided that all proprietary notices are preserved. You may not makederivative works of these publications, or reproduce, distribute or display thesepublications or any portion thereof outside your enterprise, without the expressconsent of IBM.

Rights

Except as expressly granted in this permission, no other permissions, licenses orrights are granted, either express or implied, to the publications or anyinformation, data, software or other intellectual property contained therein.

IBM reserves the right to withdraw the permissions granted herein whenever, in itsdiscretion, the use of the publications is detrimental to its interest or, asdetermined by IBM, the above instructions are not being properly followed.

You may not download, export or re-export this information except in fullcompliance with all applicable laws and regulations, including all United Statesexport laws and regulations.

IBM MAKES NO GUARANTEE ABOUT THE CONTENT OF THESEPUBLICATIONS. THE PUBLICATIONS ARE PROVIDED "AS-IS" AND WITHOUTWARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDINGBUT NOT LIMITED TO IMPLIED WARRANTIES OF MERCHANTABILITY,NON-INFRINGEMENT, AND FITNESS FOR A PARTICULAR PURPOSE.

TrademarksIBM, the IBM logo, and ibm.com are trademarks or registered trademarks ofInternational Business Machines Corp., registered in many jurisdictions worldwide.Other product and service names might be trademarks of IBM or other companies.A current list of IBM trademarks is available on the Web at "Copyright andtrademark information" at http://www.ibm.com/legal/copytrade.shtml.

Java and all Java-based trademarks and logos are trademarks or registeredtrademarks of Oracle and/or its affiliates.

Linux is a registered trademark of Linus Torvalds in the United States, othercountries, or both.

UNIX is a registered trademark of The Open Group in the United States and othercountries.

Notices 221

http://www.ibm.com/legal/copytrade.shtml

IBM®

Printed in USA

I B M Op e r a ti o n s An a l yti cs f o r z S y s t e m s IBM able 1. New features in IBM...

Documents

Transcript of I B M Op e r a ti o n s An a l yti cs f o r z S y s t e m s IBM able 1. New features in IBM...