8/20/2019 8_Introscope Configuration and Administration Guide
1/304
Date: 10-2009
Version 8.2
CA Wily Introscope®
Configuration andAdministration Guide
8/20/2019 8_Introscope Configuration and Administration Guide
2/304
6000 Shoreline Court, Suite 300
South San Francisco, CA 94080
Copyright © 2009, CA. All rights reserved.
Wily Technology, the Wily Technology Logo, Introscope, and All Systems Green are registered
trademarks of CA.
Blame, Blame Game, ChangeDetector, Get Wily, Introscope BRT Adapter, Introscope
ChangeDetector, Introscope Environment Performance Agent, Introscope ErrorDetector, Introscope
LeakHunter, Introscope PowerPack, Introscope SNMP Adapter, Introscope SQL Agent, Introscope
Transaction Tracer, SmartStor, Web Services Manager, Whole Application, Wily Customer Experience
Manager, Wily Manager for CA SiteMinder, and Wily Portal Manager are trademarks of CA. Java is a
trademark of Sun Microsystems in the U.S. and other countries. All other names are the property of
their respective holders.
For help with Introscope or any other product from CA Wily Technology, contact Wily Technical
Support at 1-888-GET-WILY ext. 1 or [email protected].
If you are the registered support contact for your company, you can access the support Web site
directly at www.ca.com/wily/support.
We value your feedback
Please take this short online survey to help us improve the information we provide you. Link to the
survey at: http://tinyurl.com/6j6ugb
If you have other comments or suggestions about Wily documentation, please send us an e-mail at
US Toll Free 888 GET WILY ext. 1
US +1 630 505 6966
Fax +1 650 534 9340
Europe +44 (0)870 351 6752
Asia-Pacific +81 3 6868 2300
Japan Toll Free 0120 974 580
Latin America +55 11 5503 6167
www.ca.com/apm
mailto:[email protected]://www.ca.com/wily/supporthttp://tinyurl.com/6j6ugbmailto:[email protected]://www.ca.com/apmhttp://www.ca.com/apmmailto:[email protected]://tinyurl.com/6j6ugbhttp://www.ca.com/wily/supportmailto:[email protected]
8/20/2019 8_Introscope Configuration and Administration Guide
3/304
Contents iii
CONTENTS
Table of Contents
Chapter 1 Starting and Stopping the Enterprise Manager . . . . . . . 7
Starting the Enterprise Manager on Windows . . . . . . . . . . 8
Starting the Enterprise Manager on UNIX. . . . . . . . . . . 10
Starting the Enterprise Manager on OS/400 . . . . . . . . . . 11
Starting and configuring the Enterprise Manager on z/OS . . . . . 12
Configuring the Enterprise Manager to use a different JVM . . . . 20
Configuring the Enterprise Manager config directory . . . . . . . 20
Stopping the Enterprise Manager . . . . . . . . . . . . . 21
Enterprise Manager shell and terminal window commands. . . . . 22
Chapter 2 Administering the Enterprise Manager . . . . . . . . . . 23
Monitoring Enterprise Manager health and load . . . . . . . . 24
Managing Enterprise Manager capacity . . . . . . . . . . . 31
Shutting down the Enterprise Manager from the Workstation . . . . 32
Mounting and unmounting agents . . . . . . . . . . . . . 32
Shutting off agents . . . . . . . . . . . . . . . . . . 34
Shared state in multiple Workstations . . . . . . . . . . . . 39
Chapter 3 Configuring Enterprise Manager Communications . . . . . . 41
The default Enterprise Manager communications channel . . . . . 42
Connecting to the Enterprise Manager across a firewall . . . . . . 45
Enabling SSL for Enterprise Managers on OS/400 . . . . . . . . 45
Configuring the Enterprise Manager Web Server for HTTPS . . . . 46
Chapter 4 Configuring Enterprise Manager Data Storage . . . . . . . 51
Configuring SmartStor data storage . . . . . . . . . . . . 52
Configuring Transaction Event database location and aging . . . . 54
Configuring location of baseline storage . . . . . . . . . . . 54
8/20/2019 8_Introscope Configuration and Administration Guide
4/304
iv Contents
CA Wily Introscope
Chapter 5 Working with Enterprise Manager Clustering . . . . . . . . 55
Understanding Enterprise Manager clustering . . . . . . . . . 56
Configuring a cluster of Enterprise Managers . . . . . . . . . 63
Chapter 6 Creating and Editing Introscope Dashboards . . . . . . . . 67
Introscope Dashboard Editor overview. . . . . . . . . . . . 68
Creating Dashboards . . . . . . . . . . . . . . . . . . 71
Creating Data Viewers in a Dashboard. . . . . . . . . . . . 76
Setting data-viewing properties of a Data Viewer . . . . . . . . 82
Embedding an Investigator Tab View in a Dashboard . . . . . . 91
Adding shapes and lines to Dashboard. . . . . . . . . . . . 92
Coloring shapes, lines and connectors . . . . . . . . . . . . 95
Creating and editing text . . . . . . . . . . . . . . . . 97
Inserting an image on a Dashboard. . . . . . . . . . . . . 100
Manipulating Dashboard objects . . . . . . . . . . . . . . 101
Creating and managing custom hyperlinks . . . . . . . . . . 106
Chapter 7 Working with Management Modules . . . . . . . . . . 111
About Management Modules . . . . . . . . . . . . . . . 112
Creating and working with Management Modules . . . . . . . . 113
Configuring metric groupings . . . . . . . . . . . . . . . 119
Monitoring performance with Alerts. . . . . . . . . . . . . 125
Working with Alert Downtime Schedules . . . . . . . . . . . 145
Using Calculators . . . . . . . . . . . . . . . . . . . 154
Using JavaScript calculators . . . . . . . . . . . . . . . 155
Deploying Management Modules . . . . . . . . . . . . . . 160
Chapter 8 Using the Command-Line Workstation . . . . . . . . . 163
About the Command Line Workstation (CLW) . . . . . . . . . 164
Running the CLW . . . . . . . . . . . . . . . . . . . 164
CLW command reference . . . . . . . . . . . . . . . . 167
Sample scripts . . . . . . . . . . . . . . . . . . . . 202
Chapter 9 Configuring Introscope Security and Permissions . . . . . 207
About Introscope security and permissions . . . . . . . . . . 208
Configuring and managing realms . . . . . . . . . . . . . 208
Configuring and managing users and groups . . . . . . . . . 219
Defining User and Group Permissions . . . . . . . . . . . . 228
8/20/2019 8_Introscope Configuration and Administration Guide
5/304
Contents v
Configuration and Administration Guide
Chapter 10 Configuring WebView Options . . . . . . . . . . . . 235
Configuring WebView for Non-ISO-8859-1 characters . . . . . . 236
Configuring the WebView application context path . . . . . . . 236
Running WebView on z/OS. . . . . . . . . . . . . . . . 237
Chapter 11 Tailoring your Workstation Configuration . . . . . . . . 245
Running Workstation in verbose mode. . . . . . . . . . . . 246
Redirecting Workstation output to a file . . . . . . . . . . . 246
Configuring Workstation to supply login values. . . . . . . . . 246
Configuring the Workstation for Asian-language reports . . . . . 247
Chapter 12 System Configurations . . . . . . . . . . . . . . . 249
Configuring the Clock Skew Tolerance . . . . . . . . . . . . 250
Setting the Maximum Data Point Retrieval Property . . . . . . . 251
Increasing the Number of Query Results Returned . . . . . . . 251
Appendix A Introscope Properties Files . . . . . . . . . . . . . . 253
IntroscopeEnterpriseManager.properties . . . . . . . . . . . 254
Introscope Enterprise Manager.lax . . . . . . . . . . . . . 276
EMService.conf File . . . . . . . . . . . . . . . . . . 279
IntroscopeWorkstation.properties . . . . . . . . . . . . . 281
Introscope Workstation.lax. . . . . . . . . . . . . . . . 283
IntroscopeWebView.properties . . . . . . . . . . . . . . 286
Introscope WebView.lax. . . . . . . . . . . . . . . . . 290
Appendix B Using the JDBC API . . . . . . . . . . . . . . . . . 293
Configuring the JDBC API . . . . . . . . . . . . . . . . 294
Using the JDBC API . . . . . . . . . . . . . . . . . . 294
Index . . . . . . . . . . . . . . . . . . . . . . . . . . 297
8/20/2019 8_Introscope Configuration and Administration Guide
6/304
vi Contents
CA Wily Introscope
8/20/2019 8_Introscope Configuration and Administration Guide
7/304
Starting and Stopping the Enterprise Manager 7
CHAPTER 1
Starting and Stopping theEnterprise Manager
This chapter provides instructions for starting and stopping the Enterprise
Manager (EM), and for automating startup in supported environments. It includes
these topics:
Starting the Enterprise Manager on Windows . . . . . . . . . . . 8Starting the Enterprise Manager on UNIX . . . . . . . . . . . 10
Starting the Enterprise Manager on OS/400 . . . . . . . . . . 11
Starting and configuring the Enterprise Manager on z/OS . . . . . . 12
Configuring the Enterprise Manager to use a different JVM . . . . . . 20
Stopping the Enterprise Manager . . . . . . . . . . . . . . 21
Enterprise Manager shell and terminal window commands . . . . . . 22
» Note See Introscope Properties Files on page 253 for details of properties
discussed in this guide.
8/20/2019 8_Introscope Configuration and Administration Guide
8/304
8 Starting and Stopping the Enterprise Manager
CA Wily Introscope
Starting the Enterprise Manager on Windows
When running the Enterprise Manager in a Windows environment, you can either
start the Enterprise Manager manually, or run it as a Windows Service.
Starting the Enterprise Manager Manually on Windows
You can start the Enterprise Manager manually in one of these ways:
Navigate to Start > Programs > Introscope > Administration > Introscope
Enterprise Manager.
Run the IntroscopeEnterpriseManager.exe located in the
directory.
The Enterprise Manager starts in accordance with the property settings in
/Introscope Enterprise Manager.lax, described in
the Introscope Properties Files on page 253.
Running the Enterprise Manager as a Windows Service
Installing the Enterprise Manager as a Windows Service increases its availability.
If you configure the Enterprise Manager as a Windows Service, when the machine
on which it runs starts, the Enterprise Manager starts automatically. It shuts
down when the machine is shut down.
» Note Introscope uses a public domain Java Service Wrapper from Tanuki
Software (http://wrapper.tanukisoftware.org ) to enable the
Enterprise Manager to run as a Windows Service. See the Configuration
Property Overview (http://wrapper.tanukisoftware.org/doc/english/
properties.html ) for information about the Tanuki wrapper.
These instructions assume you are familiar with configuring applications as
Windows Services, and are familiar with using the Windows Services console.
Configuring the Enterprise Manager as a Windows Service
This section provides instructions on how to register a single Enterprise Manager
on a Windows machine that is running with Introscope-default JVM system
parameters.
1 Stop the Enterprise Manager—see Stopping the Enterprise Manager on page 21.
2 In the IntroscopeEnterpriseManager.lax file, in your
directory, set the lax.stdin.redirect property to blank, and save the file.
8/20/2019 8_Introscope Configuration and Administration Guide
9/304
Starting the Enterprise Manager on Windows 9
Configuration and Administration Guide
3 In the IntroscopeEnterpriseManager.properties file, in your
/config directory, set the
introscope.enterprisemanager.disableInteractiveMode property to true and
save the file.
4 Assign a unique name to the Enterprise Manager service. By default, a service
name is assigned in the EMService.conf, in your
directory. If no other services are configured on the machine, you can use the
default service name. Otherwise, modify EMService.conf to specify a unique
name and a display name for the service using these properties:
wrapper.ntservice.name=
wrapper.ntservice.displayname=
Save changes to the file.
5 Run the /RegisterEMService.bat script to register the
Enterprise Manager instance as a service.
The Windows Services console shows that the Enterprise Manager is configured
as a Windows Service.
Repeat these steps for each Enterprise Manager you want to run as a service.
Changing Windows Service properties
If you need to change configuration properties in the EMService.conf file, first
unregister the service, as described in the following section. Make the changes,
then re-register the service for the changes to take effect.
Unregistering an Enterprise Manager Windows Service
Unregister the Enterprise Manager Service:
before you make changes to the EMService.conf properties file.
to return the Enterprise Manager instance to Console control.
before you uninstall the Enterprise Manager instance.
To unregister an Enterprise Manager Service:
1 Stop the Enterprise Manager Service and shut down the Windows Services GUI.
2 Run /DeregisterEMService.bat.
» Note If either the Enterprise Manager Service or the Windows Services GUI isactive, the service is not deleted, but is marked for deletion. This is
standard Windows Service behavior, and occurs because the Windows
Registry lock prevents the service from being removed. When you restart
the machine, the service is deleted.
8/20/2019 8_Introscope Configuration and Administration Guide
10/304
10 Starting and Stopping the Enterprise Manager
CA Wily Introscope
Stopping and Restarting the Enterprise Manager Windows
Service
Use the standard controls in the Windows Services GUI to start, stop, and restart
the Enterprise Manager running as a Windows Service.
Starting the Enterprise Manager on UNIX
When running the Enterprise Manager in a UNIX environment, you can either
start the Enterprise Manager manually, or run it in nohup mode.
Starting the Enterprise Manager Manually on UNIX
To start the Enterprise Manager on UNIX, run Introscope_Enterprise_Manager by
typing this at the command prompt:
./Introscope_Enterprise_Manager
The Enterprise Manager starts in accordance with the property settings in
/Introscope Enterprise Manager.lax, described in
Introscope Properties Files on page 253.
Enterprise Manager output appears in the UNIX shell. The shell does not accept
user input, unless you start up the Enterprise Manager in nohup mode, as
described in the following section.
Running the Enterprise Manager on Unix
You can manage the Enterprise Manager by running control scripts that allow youto start, stop, and check the server application status. You can use the EMCtrl.sh
startup script to start and stop the Enterprise Manager instead of directly using
the ./Introscope_Enterprise_Manager command.
To run the Introscope Enterprise Manager on Unix:
Use the following commands at the bin folder:
./EMCtrl.sh start—This starts the Enterprise Manager.
./EMCtrl.sh stop—This stops the Enterprise Manager.
./EMCtrl.sh status—This gives the status of the Enterprise Manager.
./EMCtrl.sh help—This shows the supported options for the script.
8/20/2019 8_Introscope Configuration and Administration Guide
11/304
Starting the Enterprise Manager on OS/400 11
Configuration and Administration Guide
Running Enterprise Manager in nohup Mode on UNIX
The UNIX nohup command runs a command, but suppresses the action of the HUP
(hangup) signal, enabling the command to continue running after the user whoissued it logs out. In this case, output that normally goes to the terminal goes to
a file named nohup.out.
To run the Introscope Enterprise Manager in nohup mode on UNIX:
1 In the IntroscopeEnterpriseManager.lax file, in your
directory, set the lax.stdin.redirect property to blank, and save the file.
2 In the IntroscopeEnterpriseManager.properties file, in the
/config directory, set the property
introscope.enterprisemanager.disableInteractiveMode to true and save the file.
3 Start the Enterprise Manager with this command:
nohup Introscope_Enterprise_Manager&
» Note Do not run the Enterprise Manager in nohup mode without performing
the configuration described above. Otherwise, the Enterprise Manager
might not start, or might start and consume excessive system resources.
Starting the Enterprise Manager on OS/400
This section provides instructions for starting the Enterprise Manager on OS/400
using the OS/400 start script, and invoking from the command line.
Running the OS/400 start scriptWhen the Enterprise Manager is installed on OS/400, the installer automatically
generates two start scripts. One for Enterprise Manager and one for WebView. To
invoke the start script from a QShell command prompt from the
directory, use one of the following commands:
Enterprise Manager start script:
nohup ./runem.sh &
WebView start script:
nohup ./runwv.sh &
If you run the script from a directory other than , identify
the home directory as shown below:
nohup ./runem.sh / &
8/20/2019 8_Introscope Configuration and Administration Guide
12/304
12 Starting and Stopping the Enterprise Manager
CA Wily Introscope
where is the root of your Enterprise Manager installation
directory.
The output will default to the nohup.out in the directory from which the script is
executed.
» Note Use the runem.sh script to start the Enterprise Manager on Red Hat
Linux 5.
Using control scripts for OS/400 and Z/OS
You can use control scripts in OS/400 and Z/OS to start the Enterprise Manager.
Use these control scripts for OS/400 and Z/OS:
1 Go to \bin directory.
2 Use these commands:
./EMCtrl.sh start—Starts the Enterprise Manager.
./EMCtrl.sh status—Verifies whether the Enterprise Manager is running or
not.
./EMCtrl.sh —Stops the Enterprise Manager.
Running the Enterprise Manager as a Java program
To start Enterprise Manager from the command line, enter this command from
the directory:
lib/EnterpriseManager.jar:lib/WebServices.jar:lib/IntroscopeClient.jar:lib/
SNMPAdapter.jar:lib/Workstation.jar:lib/IntroscopeServices.jar
Start command syntax can vary, depending on your version of Java and other
environmental factors.
» Note If you are running the Enterprise Manager as a Java Program on SAP, use
this command:
lib/EnterpriseManager.jar:lib/WebServices.jar:lib/
IntroscopeClient.jar:lib/SNMPAdapter.jar:lib/
Workstation.jar:lib/IntroscopeServices.jar:lib/
WebViewAPI.jar:lib/WebViewServer.jar
Starting and configuring the Enterprise Manager onz/OS
To start the Enterprise Manager on z/OS, use the runem.sh file.
8/20/2019 8_Introscope Configuration and Administration Guide
13/304
Starting and configuring the Enterprise Manager on z/OS 13
Configuration and Administration Guide
For instructions on running the EM as a Java program, please see Running the
Enterprise Manager as a Java program on page 12.
To facilitate use of Introscope in the z/OS environment, run the product in a
manner familiar to z/OS operations and development staff—as a z/OS batch job,
or as a started task. You can also run Introscope Enterprise Manager as a shell
process. This section describes running the Introscope Enterprise Manager as a
batch job or started task:
Prerequisites for running Enterprise Manager as a z/OS batch job on page 13
briefly describes the z/OS environment and the BPXBATCH utility. This section
describes software prerequisites, security, authorization and address space
size requirements.
Configuring Enterprise Manager to run as a z/OS batch job on page 16
describes the tasks necessary to customize the sample JCL and other files in
preparation for running the Introscope Enterprise Manager as a batch job on
the z/OS platform.
Running the Enterprise Manager on z/OS on page 18 describes the activities
required to run the Introscope Enterprise Manager as a batch job or started
task on the z/OS platform.
Prerequisites for running Enterprise Manager as a z/OS batch job
z/OS base infrastructure for Java
The MVS utility program, BPXBATCH, enables users to invoke UNIX System
Services to run shell commands, shell scripts and executable files in MVS batch.
You can therefore use the BPXBATCH utility to start the IntroscopeEnterprise Manager.
Software prerequisites
The specific software prerequisites to run Java (and therefore the Introscope
Enterprise Manager) on z/OS can be found in the Java for z/OS Program Directory
GI10-0614. The minimum software requirements are:
The Introscope Enterprise Manager requires JVM 1.5 (either 32 or 64 bits).
» Note WebView needs JVM 1.5 to run.
UNIX System Services must be enabled
Language Environment 1.5 or higher
Java for z/OS
8/20/2019 8_Introscope Configuration and Administration Guide
14/304
14 Starting and Stopping the Enterprise Manager
CA Wily Introscope
Security and authorization
Authorization is determined by the RACF profile associated with a user’s USERID.
To successfully start the Introscope Enterprise Manager, a user must have
authority to access the following resources:
UNIX System Services as specified by the OMVS parameter in the RACF profile
EXEC PGM= BPXBATCH
EXEC WILYPROC
It is important to understand how the user is identified in these contexts:
Batch—When a user submits a batch job through the TSO/E or ISPF interface,
the user’s TSO UserID and password are propagated to the batch job. Provided
the submitted Batch JOB CARD does not explicitly contain a User and
Password, the batch job acquires the same access authority as the TSO user.
If User and Password are coded on the Batch JOB CARD, they override the
UserID and Password of the TSO/E user.
Started Task—If the Introscope Enterprise Manager is initiated as a started
task, it runs under the authorization of the STC user ID. A Site-specific
algorithm might be used to relate the name of the started PROC to a particular
UserID. You must make sure that the algorithm produces a UserID with the
correct authorization to run the Introscope Enterprise Manager.
Virtual memory
If the OMVS address space is not large enough, an out-of-memory condition
occurs when an attempt is made to run the Introscope Enterprise Manager. The
OMVS ASSIZEMAX(address-space-size) parameter in the RACF profile for theowner of the Introscope Enterprise Manager job or started task must be large
enough to execute the Introscope Enterprise Manager. The maximum value for
this parameter is 2G.
In addition, in order to run the Enterprise Manager with Java 1.5 and avoid
encountering Java 1.5 GC Heap Initialization errors, you may need to raise the
MEMLIMIT in your SMFPRMxy PARMLIB member. Update your SMFPRMxy to add
the following line if it does not already exist:
MEMLIMIT(256M) /* SET MINIMUM MEMLIMIT FOR JAVA 1.5 */
» Note The 256M value is minimum requirement. Recommended value can be
1024M or more.
DB2
1 To access JDBC and SQLJ from UNIX Systems Services, you must set the
following environment variables:
8/20/2019 8_Introscope Configuration and Administration Guide
15/304
Starting and configuring the Enterprise Manager on z/OS 15
Configuration and Administration Guide
STEPLIB must include the SDSNEXIT, SDSNLOAD and SDSNLOD2 data sets
LIBPATH and LD_LIBRARY_PATH must include the DLL libraries for the JDBC
and SQLJ drivers
2 In order to run the Introscope Enterprise Manager as a z/OS job, the runem.sh
shell script must contain the following statements where DSNXXX denotes the
High Level Qualifier of your DB2 libraries:
export STEPLIB=DSNXXX.SDSNEXIT:DSNXXX.SDSNLOAD:DSNXXX.SDSNLOD2
export LIBPATH=/usr/lpp/db2/db2710/lib:$LIBPATH
export LD_LIBRARY_PATH=/usr/lpp/db2/db2710/
lib:$LD_LIBRARY_PATH
» Note While DB2SQLJPROPERTIES—the directory containing the
db2sqljjdbc.properties file—is used to specify the name of a SQLJ
properties file, it is not always necessary to set this environment
variable.
3 The CLASSPATH must include the JDBC and SQLJ driver classes. Include this file
in the classpath in the runem.sh shell script:
/usr/lpp/dsn710/db2/db2710/classes/db2sqljclasses.zip
» Note The character string “710” in dsn710 and db2710 refers to the version
and release of DB2 being used. If another version of DB2 is being used,
these directories must be changed.
4 Specify the DB2 subsystem identifier:
SYS_DB2_SUB_SYSTEM_NAME=xxx , where xxx is the subsystem identifier.
JDBC
JDBC for z/OS is based on the DB2 Call Level Interface (CLI). You must ensure
that the DB2 CLI has been set up as follows:
1 The DB2 CLI application plan (the default name is DSNACLI) must be bound
before you can use the DB2 CLI. A sample bind job can be found in
DSNXXX.SDSNSAMP(DSNTIJCL) where DSNXXX denotes the high level qualifier
of your DB2 installation.
2 The DB2 CLI application plan must be public. Execute the following command
from SPUFI or from a batch job:
GRANT EXECUTE ON PLAN DSNACLI TO PUBLIC
3 Set DSNAOINI to point to the CLI initialization file. The CLI initialization file
provides information about DB2 subsystem names and additional configuration
parameters. You can place the file either in an HFS file or in an MVS data set using
one of the following statements:
8/20/2019 8_Introscope Configuration and Administration Guide
16/304
16 Starting and Stopping the Enterprise Manager
CA Wily Introscope
export DSNAOINI=/usr/lpp/db2/db2510/dsnaoini
export DSNAOINI=DSNxxx.CLIINI
export DSNAOINI=DSNxxx.CLIINI(CONF1)
4 Make sure that the DB2 subsystem referenced in the CLI initialization file
references the DB2 subsystem that you want to connect to. DB2 subsystem
names usually take the form DB2n where n is a unique qualifier.
Configuring Enterprise Manager to run as a z/OS batch job
Before the Introscope Enterprise Manager can run as a z/OS batch job, the
following files (extracted from the z/OS installer file) must be copied into the
z/OS environment and customized to ensure compatibility with site-specific
requirements.
IntroscopeEnterpriseManager.properties – the Introscope Enterprise Manager
properties file, which specifies information pertinent to the operation of the
Introscope Enterprise Manager
runem.sh – the shell script used to launch the Introscope Enterprise Manager
process
WILYPROC – the z/OS PROC used to execute z/OS BPXBATCH program that
launches the shell script
readme.txt - a description of the runem.sh and WILYPROC files and how to use
and customize them
Copying and customizing Introscope Enterprise Manager files
IntroscopeEnterpriseManager.properties and runem.sh are UNIX files that must
be copied into the Hierarchical File System (HFS) on the z/OS platform. These
files are used by UNIX System Services and the Introscope Enterprise Manager
respectively. Use a UNIX-based editor such as vi to make required changes to
these files. Do not use the ISPF editor because it does not properly handle
insertion of characters that increase line length.
WILYPROC is a z/OS file that is used by the JES2 component of z/OS and by DB2
for z/OS. Make changes to this file using the ISPF editor.
To copy the Enterprise Manager files:
Copy the IntroscopeEnterpriseManager.properties and runem.sh files into the
Introscope installation directory, as shown in the following examples, where
is the UID assigned to Wily Technology:
/u/ /introscope7.2/runem.sh
/u/ /introscope7.2/config/IntroscopeEnterpriseManager.properties
8/20/2019 8_Introscope Configuration and Administration Guide
17/304
Starting and configuring the Enterprise Manager on z/OS 17
Configuration and Administration Guide
To customize Enterprise Manager properties file:
1 Open the u/xxxx/introscope7.2/config/
IntroscopeEnterpriseManager.properties file.
2 Find the Interactive Mode heading and make sure the value for the following
property is set to true:
introscope.enterprisemanager.disableInteractiveMode=true
To customize the UNIX shell script runem.sh:
1 Verify the version and release of DB2. The DB2 files in the runem.sh shell script
reference DB2 V7.1.
» Note If you are connecting to a DB2 subsystem other than version 7.1, you
must change all instances of 710 to the appropriate character string for
the version being used.
2 Make the following changes to the runem.sh script:
In the CLASSPATH, /usr/lpp/dsn710/db2/db2710/classes/
db2sqljclasses.zip\
export STEPLIB=DSN710.SDSNEXIT:DSN710.SDSNLOAD,DSNXXX.SDSNLOD2
#export LIBPATH=/usr/lpp/dsn710/db2/db2710/lib:$LIBPATH
#export LD_LIBRARY_PATH=/usr/lpp/dsn710/db2/db2710/
lib:$LD_LIBRARY_PATH
» Note If LIBPATH and LD_LIBRARY_PATH have been commented out, remove
the #.
To copying Enterprise Manager files to a z/OS partitioned dataset (PDS)
1 Using ISPF, create a partition dataset named userid.JCL.CNTL, (where userid
is the userid assigned to WILY) with fixed-block (FBA) of 80 byte records.
2 Copy WILYPROC into the PDS userid.JCL.CNTL.
3 Do not try to copy the shell script runem.sh, or the properties file
IntroscopeEnterpriseManager.properties , into this PDS. These files have
lines that are longer than 80 bytes. An attempt to copy them either fails, or lines
are truncated.
To customize the z/OS PROC that invokes BPXBATCH
1 Edit WILYPROC using the ISPF editor. Make the following changes: On the ISPF command line, type CAPS OFF and press Enter.
On the ISPF command line, type NUM OFF and press Enter.
Set the IHOME parameter equal to the fully qualified path to the directory that
contains runem.sh.
8/20/2019 8_Introscope Configuration and Administration Guide
18/304
18 Starting and Stopping the Enterprise Manager
CA Wily Introscope
2 This statement is coded to end the batch job and the UNIX shell script within 20
seconds:
// PARM='SH nohup &IHOME/runem.sh &IHOME & sleep 20'
If z/OS is too busy to initiate the Introscope Enterprise Manager within 20
seconds, the Enterprise Manager does not start. If you encounter this problem,
try increasing the sleep time.
3 Modify the STDERR and STDOUT file names contained in the following statements
to satisfy installation naming conventions:
//STDOUT DD PATH='//stdout',
//STDERR DD PATH='//stderr',
To copy WILYPROC into an active PROCLIB
Using ISPF option 3, copy the PROC to xxxx .PROCLIB where xxxx represents an
site-dependent high-level qualifier. If WILYPROC is not an acceptable name inyour installation, rename the PROC accordingly.
Running the Enterprise Manager on z/OS
Executing the z/OS PROC
Running as a started task from the z/OS console or SDSF
You can run the Enterprise Manager as an MVS started task (STC). A started task
is a familiar, operator-friendly environment that allows the Enterprise Manager to
be started, monitored, and cancelled from the MVS console. In this case, the
Enterprise Manager runs under the authorization of the STC user ID, allowing
assignment of specific authorities to the started task.
To run the Introscope Enterprise Manager as a started task, type this command
on the MVS console or on the command line of SDSF:
/start WILYPROC
You must type the member name of the PROC stored in an active PROCLIB. If you
were required to change the name of WILYPROC, you must use the new name in
the /start command.
Running as a batch job
You can run the Enterprise Manager as a batch job, using JCL (job-control
language) to describe the job's requirements.
To run the Enterprise Manager as a batch job, submit the following JCL:
//jobname JOB installation jobcard parameters,NOTIFY=&SYSUID
ISCOPE EXEC WILYPROC
/*
8/20/2019 8_Introscope Configuration and Administration Guide
19/304
Starting and configuring the Enterprise Manager on z/OS 19
Configuration and Administration Guide
You must execute the member name of the PROC stored in an active PROCLIB. If
you were required to change the name WILYPROC, you must use the new name
in the EXEC statement.
The z/OS batch job and the OMVS shell end shortly after the Enterprise Managerprocesses start. If you submitted the batch JCL from your ISPF session and your
JOB card contains NOTIFY=&SYSUID, your ISPF session receives notification
when your batch job ends.
Verifying the Enterprise Manager is running
The most accurate way to determine that the Enterprise Manager has started
correctly is to view the STDOUT file. To view STDOUT, telnet into z/OS and view
the STDOUT file defined in the WILYPROC. The last line of the file indicates that
the Enterprise Manager has started.
You can also determine that the Enterprise Manager has started from SDSF using
one of the following procedures:
If you started the Enterprise Manager as a started task, on the DA screen you
should see at least two active started tasks whose owner is the owner
associated with the Enterprise Manager PROC.
If you started the Enterprise Manager by submitting a batch job from your TSO
session, look for two active tasks with your TSO USERID as the owner.
You might see the job that invoked the WILYPROC, as well as the OMVS shell
script. These two entries disappear from the list after the Enterprise Manager
starts.
The owner of your TSO session is also your TSO/E USERID. Do not mistake yourTSO/E session for one of the Enterprise Manager tasks.
Starting the Enterprise Manager automatically
To start the Enterprise Manager automatically when JES2 starts:
1 Place the following statement in JES2PARM:
$VS,'start WILYPROC
2 Type the member name of the PROC stored in an active PROCLIB. If you were
required to change the name of WILYPROC, use the new name in the /start
command.
You can also run the Enterprise Manager as a batch job using the job scheduling
package you use to manage your batch workload.
8/20/2019 8_Introscope Configuration and Administration Guide
20/304
20 Starting and Stopping the Enterprise Manager
CA Wily Introscope
Configuring the Enterprise Manager to use adifferent JVM
By default, an Introscope component uses the JVM it is bundled with. This section
has instructions on how to configure the Enterprise Manager to use a different
JVM.
If you start the Enterprise Manager with Introscope Enterprise
Manager.exe, edit the lax.nl.current.vm property in Introscope
Enterprise Manager.lax to point to the desired JVM executable.
You can use an absolute or relative path to the executable.
On Windows, escape backslashes, and include the JVM executable file
extension: java.exe instead of java.
If you run the Enterprise Manager as a Windows service, edit the
wrapper.java.command property in EMService.conf to point to the desired
JVM executable.
You can use absolute paths; backslashes must be escaped.
You can also use relative paths; Wily Technology Division recommends that
you use UNIX forward slashes and start with dot-slash (./).
The executable is java not java.exe.
If you start the Enterprise Manager with the runem.sh script on z/OS or AS/
400, edit the JAVA_HOME variable in runem.sh to point to the desired JVM root
directory.
Use an absolute to the executable, not a relative path.
Configuring the Enterprise Manager config directory
If you have multiple Enterprise Managers and want to run them using a single
instance of Introscope, you can create a separate configuration directory for each
Enterprise Manager. For each Enterprise Manager, make a copy of the config,
logs, data, and traces directories. In each copy of the config directory, set the
appropriate properties in the IntroscopeEnterpriseManager.properties file
to point to the copies of your config, logs, data, and traces directories.
By creating separate configuration directories, you can keep all the directories
within one instance of Introscope while maintaining separate config, logs, data,
and traces directories for each Enterprise Manager.
To specify a different configuration directory for your Enterprise Manager:
1 Open the IntroscopeEnterpriseManager.properties file and modify the
following property:
8/20/2019 8_Introscope Configuration and Administration Guide
21/304
Stopping the Enterprise Manager 21
Configuration and Administration Guide
introscope.enterprisemanager.directory.config=config
The default directory is config, which is the configuration directory created when
Introscope was first installed. Change the location of this directory using either
an absolute path (e.g. C:\\Introscope\em1\\config) or a path relative to theIntroscope installation directory (e.g. config2). For example:
introscope.enterprisemanager.directory.config=
C:\\Introscope\\em1\\config
» Note In a Windows environment, backslashes must be escaped (doubled) for
the path to function.
2 Save the changes made to the IntroscopeEnterpriseManager.properties
file.
3 Open the Introscope Enterprise Manager.lax file and modify the following
property:
lax.nl.java.option.additional
This property modifies the Java command-line and can contain multiple
properties. Use the location of the config file you selected in step one. For
example:
lax.nl.java.option.additional=-Xms512m -Xmx512m -Djava.awt.headless=false -
Dcom.wily.introscope.em.properties=C:\Introscope\em1\config
» Note This property can point either to a directory containing a
Enterprise Manager properties file, or to an
IntroscopeEnterpriseManager.properties file directly. For
example:
-Dcom.wily.introscope.em.properties=C:\Introscope\em1\configor
-Dcom.wily.introscope.em.properties=
C:\Introscope\em1\config\IntroscopeEnterpriseManager.properties
4 Save the changes to the Introscope Enterprise Manager.lax file.
5 Restart the Enterprise Manager.
Stopping the Enterprise Manager
When you stop the Enterprise Manager:
Agents stop reporting data.
SmartStor does not save data.
you are logged out of the Workstation.
all users connected to the Enterprise Manager are disconnected.
8/20/2019 8_Introscope Configuration and Administration Guide
22/304
22 Starting and Stopping the Enterprise Manager
CA Wily Introscope
Stopping the Enterprise Manager from the terminal window
From the Enterprise Manager terminal window, enter:
iscopeshutdown
» Note Interactive mode must be enabled to use this command. Set
introscope.enterprisemanager.disableInteractiveMode to
false in the Enterprise Manager properties file. For more information,
see Introscope Properties Files on page 253.
Stopping the Enterprise Manager from the Workstation
If you have shutdown privileges for the Enterprise Manager, you can stop the
Enterprise Manager from the Workstation in a Management Module Editor
window, by choosing Manager > Shut Down Enterprise Manager.
Automatic shutdown of Enterprise Manager
If the Enterprise Manager detects errors writing to disk, indicating a shortage of
disk space, it automatically shuts down. If failover is configured, agents can
failover to another Enterprise Manager. For more information, see the Java Agent
Guide or .NET Agent Guide.
Enterprise Manager shell and terminal windowcommands
When the Enterprise Manager is running, if lax.stdin.redirect (defined inIntroscope Properties Files on page 253) is set to console, you can enter the
following commands in the Enterprise Manager terminal window or shell:
» Note Interactive mode must be enabled to use these commands. Set
introscope.enterprisemanager.disableInteractiveMode to
false in the Enterprise Manager properties file. For more information,
see Introscope Properties Files on page 253.
Command Result
iscopedebugpolling [true | false]
[delay in seconds]
Periodically reports debugging information to the
log
iscopehelp Display help screen
iscopeshutdown Shuts down the Enterprise Manager
8/20/2019 8_Introscope Configuration and Administration Guide
23/304
Administering the Enterprise Manager 23
CHAPTER 2
Administering the Enterprise Manager
This chapter describes how to use the Introscope Workstation to manage the
information collected and reported by the Introscope Enterprise Manager. It
includes these topics:
Monitoring Enterprise Manager health and load . . . . . . . . . 24Shutting down the Enterprise Manager from the Workstation . . . . . 32
Mounting and unmounting agents. . . . . . . . . . . . . . 32
Shutting off agents . . . . . . . . . . . . . . . . . . 34
Shared state in multiple Workstations . . . . . . . . . . . . 39
8/20/2019 8_Introscope Configuration and Administration Guide
24/304
24 Administering the Enterprise Manager
CA Wily Introscope
Monitoring Enterprise Manager health and load
Understanding Enterprise Manager metrics
The Enterprise Manager generates metrics that are useful in managing and
diagnosing itself, assessing its health, and determining how well it is performing
under its workload. Metrics are reported every 15 seconds.
Enterprise Manager metrics appear in the Investigator tree under:
Custom Metric Host (Virtual)
Custom Metric Process (Virtual)
Custom Metric Agent (Virtual)(SuperDomain)
Enterprise Manager
In a clustered environment, the metrics appear under the Collector cluster node.
You can also create Dashboards that present custom views of Enterprise Manager
metrics, and set up alerts based on the values of Enterprise Manager metrics.
» Note For all installations, you must apply the appropriate Daylight SavingsTime patch to your operating system to ensure the correct operation of
your Enterprise Manager. For more information, see the Introscope
Installation and Upgrade Guide.
8/20/2019 8_Introscope Configuration and Administration Guide
25/304
Monitoring Enterprise Manager health and load 25
Configuration and Administration Guide
Enterprise Manager
Host—The name of the machine on which the Enterprise Manager is running.
Name—The name of the Enterprise Manager as defined in the propertyintroscope.enterprisemanager.name .
Overall Capacity (%)—Estimate of the percentage of the Enterprise Manager’s
capacity that is consumed. The value can range from 0% to over 100%. The value
can exceed 100% because the metric calculation assumes an upper bound of
desirable capacity usage that is less than 100%—the premise being that most
customers prefer to run an Enterprise Manager at 50-75% capacity during normal
times and allow headroom for handling increased load during peak periods.
Capacity is determined in part by the values of the Metrics described in Enterprise
Manager | Health on page 29.
Port—The port number as defined in the propertyintroscope.enterprisemanager.port.channel1
Enterprise Manager | CPU
EM CPU Used (%)—Shows what percent of the total available CPU was used by
running Enterprise Managers during the time period specified.
» Note This number does not reflect other processes running on the server or
overall server CPU in use, but rather how much CPU the particular
Enterprise Manager used.
This metric is acquired from the JVM using an API introduced in the JDK 1.5.
Therefore, it is supported only on some platforms.
Enterprise Manager | Configuration
Agent Clusters Metric Load—Number of Metrics reported by all Virtual Agents.
Number of Agent Clusters—Number of Virtual Agents.
Number of Metric Groupings—Number of Metric Groupings for the Enterprise
Manager, including user-defined and temporary system Metric Groupings. This
value is an indicator of how much business logic is running on the Enterprise
Manager—a significant factor in the volume of Metrics it can support.
Enterprise Manager | Configuration | Agent Clusters |VirtualAgentName
Metric Load—Number of Metrics reported by a Virtual Agent.
8/20/2019 8_Introscope Configuration and Administration Guide
26/304
26 Administering the Enterprise Manager
CA Wily Introscope
Enterprise Manager | Connections
Metrics Queued (%)—Percentage of incoming Metrics (for the current time
slice) waiting to be processed.
Number of Agents—Number of Agents currently connected and reporting live
data.
Number of Applications—The total number of applications reported by all the
connected Agents.
Number of Historical Metrics—All the metrics that are present in the SmartStor
database.
Number of Metrics—Number of Metrics the Enterprise Manager is aware of.
When agents disconnect, this number drops.
Number of Metrics Handled—Number of incoming Metrics processed per 15second interval. This number varies, but should be around the same value as
Number of Metrics. A value less than Number of Metrics indicates the Enterprise
Manager is overloaded and is not processing incoming data fast enough.
Number of Workstations—Number of connected Workstations.
Enterprise Manager | Data Store | SmartStor
Metrics Appended To Query Per Interval—Tracks the SmartStor data
appending task that runs every day, and shows how many Metrics are appended
per interval.
Metrics Converted From Spool to Query Per Interval—Tracks the SmartStorspool to query conversion task that runs every hour.
SmartStor Disk Usage (mb)—Megabytes of disk used by SmartStor.
Enterprise Manager | Data Store | SmartStor | MetaData
Agents with Data—The number of agents tracked by the Enterprise Manager for
which there is metric data.
Agents without Data—The number of agents tracked by the Enterprise Manager
for which there is no metric data.
» Note This is an approximate value.
Metrics with Data—The number of fully qualified metrics (agent name + partial
metric name) tracked by the Enterprise Manager for which there is metric data.
8/20/2019 8_Introscope Configuration and Administration Guide
27/304
Monitoring Enterprise Manager health and load 27
Configuration and Administration Guide
Partial Metrics with Data—The number of partial metrics (that is, metrics
without the agent name qualifier) tracked by the Enterprise Manager for which
there is metric data.
Partial Metrics without Data—The number of partial metrics tracked by the
Enterprise Manager for which there is no metric data. If this value is greater than
0, the Enterprise Manager is tracking partial metrics unnecessarily.
Write Duration (ms)—The time in milliseconds spent writing to the SmartStor
metadata file.
Enterprise Manager | Data Store | SmartStor |Tasks
Converting Spool To Data—Tracks whether the spool to query conversion task is
running. When this task is running, this Metric has the value 1; when not running,
the value is 0.
Data Append—Tracks whether the data appending task is running. When this
task is running, this Metric has the value 1; when not running, the value is 0.
Reperiodizing—Tracks whether the reperiodization task is running. When this
task is running, this Metric has the value 1; when not running, the value is 0.
Enterprise Manager | Data Store | Transactions
Number of Dropped Per Interval—This is the number of transaction traces
dropped per 15 second interval because transaction traces could not be
processed fast enough. If transaction traces are received faster than they can be
processed and inserted into the data store, then the excess are dropped.
Number of Inserts Per Interval—Number of transaction trace inserts per 15
second interval.
Number of Queries Per Interval—The number of transaction trace queries per
time slice that are included in the “greater than last 20 minute” category.
Number of Traces in Database—Number of transaction traces in the transaction
trace database.
Number of Traces in Insert Queue—The Transaction Trace storage system
includes an insert queue that is 1000 elements in size. If the Enterprise Manager
becomes overloaded with incoming transaction traces, they are put into this
queue. The Enterprise Manager might become overloaded if, for example, toomany metrics are reporting, or perhaps if an aggressive Transaction Trace
session was started and the rate of traces is too high. When the insert queue
reaches 2000, the transaction traces are dropped to prevent the Enterprise
Manager from running out of memory.
8/20/2019 8_Introscope Configuration and Administration Guide
28/304
28 Administering the Enterprise Manager
CA Wily Introscope
TT Database Disk Usage (mb)—Number of megabytes of disk storage the
transaction trace database is using.
Total Data Insertion Duration Per Interval (ms)—How much time spent per
time slice inserting the transaction trace data into internal data store.
Total Index Insertion Duration Per Interval (ms)—How much time spent per
time slice indexing the transaction trace data.
Total Query Duration Per Interval (ms)—How much time spent per time slice
fetching data from internal data store.
Enterprise Manager | Data Store | Volume Space Free
» Note The Volume Free Space metrics are not available if you are running the
Enterprise Manager on the z/OS platform.
Baseline Volume Free (mb)—How much baseline volume space is free inmegabytes.
Log Volume Free (mb)—How much log volume space is free in megabytes.
SmartStor Archive Volume Free (mb)—How much SmartStor Archive volume
space is free in megabytes.
SmartStor Data Volume Free (mb)—How much SmartStor data volume space
is free in megabytes.
Traces Volume Free (mb)—How much Traces volume space is free in
megabytes.
Volume Free (mb)—The number of megabytes of disk space available on thedisk partition holding the baselines file. (This is defined by the property
introscope.enterprisemanager.dbfile , and defaults to the local drive if
none is found.) It does this by running a OS specific file system utility: dir /-c
on Windows, df –kP on AIX/HP-UX, and df –k on other Linux and other Unix
implementations. It will not work if these utilities are either not available, or in
the case of df, do not support GNU format when passed –k (or –kP).
Enterprise Manager | Database
Metric Data Points Sent per Interval—A metric data point is one timeslice of
data for one metric. This metric shows the number of metric data points written
to the relational database per interval.
Queued Metric Data Points—Number of metric data points not yet written to the
relational database.
8/20/2019 8_Introscope Configuration and Administration Guide
29/304
Monitoring Enterprise Manager health and load 29
Configuration and Administration Guide
Enterprise Manager | GC Heap
GC Duration (ms)—The time in milliseconds (during a time slice) spent handling
high-priority system tasks—most likely garbage collection—that prevented
threads from running. As this number grows, the overall capacity of the system
is diminished. When memory runs low and the JVM is spending a lot of time
garbage collecting, this number will be high—perhaps 5000 to 10000 ms.
In Use (mb)—Megabytes of heap in use.
In Use Post GC (mb)—Tracks how much heap is used after a major garbage
collection.
Total (mb)—Size of the heap in megabytes.
Enterprise Manager | Health
CPU Capacity (%)—Shows what percent of the total available CPU was used by
running Enterprise Managers during the time period specified.
» Note This number does not reflect other processes running on the server or
overall server CPU in use, but rather how much CPU the particular
Enterprise Manager used.
This metric is acquired from the JVM using an API introduced in the JDK 1.5.
Therefore, it is supported only on some platforms.
GC Capacity (%)—An average of the last 8 values. This is used to normalize
abnormal numbers.
Harvest Capacity (%)—The current capacity of the Enterprise Manager toharvest data every 15 seconds. This value computes 100% in use based on how
long harvest takes vs. 15000ms. For example, if harvest takes 15000ms, then
this metric will be 100.
Heap Capacity (%)—The maximum possible total heap vs. current in use
(tracked by GC Heap: In Use Post GC (mb). For example, if the JVM is launched
with -Xmx1024m, then the total possible heap is 1024m. A 25% buffer remains
when this metric is at 100% and when the actual heap would be at 100%. For
example, if the total heap is 1000MB and the current heap usage is 750MB, then
this metric will be 100%. This is because Java needs heap space for "normal
operation".
8/20/2019 8_Introscope Configuration and Administration Guide
30/304
30 Administering the Enterprise Manager
CA Wily Introscope
Incoming Data Capacity (%)—The capacity of the Enterprise Manager to handle
incoming data, based on an internal Metric that indicates the number of incoming
Metrics yet to be processed. This internal Metric is divided by twice the total
number of Metrics. For example, if 150,000 Metrics are in the to-be-processedqueue and the Enterprise Manager has a total of 300,000 Metrics, the incoming
data capacity will be 25%.
SmartStor Capacity (%)—The capacity of the SmartStor subsystem. This value
computes 100% in use based on how long smartstor writes take vs 15000ms. For
example, if smartstor duration is 15000ms, then this metric will be 100.
Enterprise Manager | Internal
The Internal node contains metrics that relate to Enterprise Manager internals.
These metrics are for use by Wily Technology Division personnel only, and are
subject to change without notice.
Enterprise Manager | Problems| Management Modules
Warning Count—The total number of missing constructs in Management
Modules. For example, if a dashboard named, Dashboard XYZ , is missing and is
referenced by 11 other dashboards, it is considered one warning (not 11).
Enterprise Manager | MOM
Number of Collector Metrics—The number of metrics received from the
Collectors per interval.
Collector Metrics Received Per Interval—The number of metric data points
the MOM is receiving every (15 second) interval period. It is essentially the total
amount of traffic from the Collectors to the MOM, and shows the overall load that
queries are putting on the entire cluster.
Enterprise Manager | MOM | Collectors | CollectorHost@Port
Connected—The status of a Collector Enterprise Manager in an Enterprise
Manager cluster.
Values are based on Introscope alert values, and can be:
1 - connected
2 - connected to a slow Collector. A Collector is considered slow if the Ping Timeis greater than ten seconds.
3 - not connected
8/20/2019 8_Introscope Configuration and Administration Guide
31/304
Managing Enterprise Manager capacity 31
Configuration and Administration Guide
Ping Time (ms)—Measures the time in milliseconds to send a no-op message to
the Collector and return. It's a measure of the responsiveness of the MOM-
Collector connection taking into account the network round-trip time and the time
to process the message in the MOM and Collector.
Enterprise Manager | Tasks
Harvest Duration (ms)—The time in milliseconds (during a time slice) spent
harvesting data.
SmartStor Duration (ms)—The time in milliseconds (during a time slice) spent
writing Metric data to disk.
Managing Enterprise Manager capacity
An inappropriately configured agent can create thousands of metrics in quick
succession and overload the Enterprise Manager. To prevent this, the Enterprise
Manager uses a metric throttle that stops an agent when its metric output
becomes excessive.
Configuring the Enterprise Manager metric throttle
The default Metric throttle is 50,000 Metrics. To change the Metric threshold, you
add a property to the IntroscopeEnterpriseManager.properties file. For
more information, see Enterprise Manager Metric Clamping on page 264.
To configure the Enterprise Manager Metric throttle:
1 Open the IntroscopeEnterpriseManager.properties file.2 Add the property introscope.enterprisemanager.agent.metrics.limit to
the file, and enter a value for the Metric throttle limit.
3 Save your changes.
The new value is applied when the Enterprise Manager checks the property file
during a hot deploy. A hot deploy occurs about once every 60 seconds.
If an agent generates more than the specified number of Metrics, the Enterprise
Manager will drop Metrics.
If this occurs, modify the agent’s configuration to reduce the volume of Metrics
reported. Metric explosion typically is a result of inadequately filtered JMX data
or SQL Metrics. For more information, see the Java Agent Guide or .NET Agent
Guide, as appropriate to your environment.
8/20/2019 8_Introscope Configuration and Administration Guide
32/304
32 Administering the Enterprise Manager
CA Wily Introscope
Automatic shutdown of the Enterprise Manager
If the Enterprise Manager detects errors writing to disk, indicating a shortage of
disk space, it automatically shuts down so that agents can failover to another
Enterprise Manager. To enable failover, see the Java Agent Guide or the .NET
Agent Guide, as appropriate to your environment.
Shutting down the Enterprise Manager from theWorkstation
You can shut down the Enterprise Manager from the Workstation. Shutting down
the Enterprise Manager automatically logs you out of the Workstation.
To shut down the Enterprise Manager:
1 Within a Management Module Editor window, select Manager > Shut Down
Enterprise Manager.
The Shut Down Confirmation message appears.
» Note If the Shut Down Enterprise Manager command is disabled, you do
not have shutdown or full permission. Contact your administrator.
2 Click Yes to confirm Enterprise Manager shutdown.
The Enterprise Manager shuts down, logs you out of the Workstation, and opens
the Workstation login window.
Mounting and unmounting agents
Agents that disconnect and then reconnect to an Enterprise Manager are
recognized as the same agent, and simply become active again in the
Investigator tree. You might, however, encounter situations where an agent
disconnects and does not reconnect, such as agents used in QA environments.For example, you might have a load test running with an agent, but the agent
disconnects after tweaking the managed application for performance.
» Important To shut down the Enterprise Manager, you must have shutdown
or full permission for the Enterprise Manager.
8/20/2019 8_Introscope Configuration and Administration Guide
33/304
Mounting and unmounting agents 33
Configuration and Administration Guide
An agent that does not reconnect to and Enterprise Manager after a user-defined
time limit is automatically unmounted and removed from the Investigator tree
(see Auto-unmounting of disconnected agents on page 33); or it can be manually
unmounted at any time after disconnects.
To mount and unmount agents, you must have either historical_agent_control
or full permission for the Domain.
Unmounting disconnected agents
When an agent is disconnected, its icon in the Investigator tree is disabled.
Disconnected agents are automatically unmounted from the Workstation after a
user-defined time. Disconnected agents can also be unmounted manually.
» Note You can only unmount disconnected agents.
Disconnected and unmounted agents that have recorded at least one data point
can be remounted.
Auto-unmounting of disconnected agents
Disconnected agents are automatically unmounted from the Workstation after 60
minutes. This is the default setting, but this value can be changed in the
autoUnmountDelayInMinutes property in the
IntroscopeEnterpriseManager.properties file. See Introscope Properties
Files on page 253.
Unmounting an agent manually
To unmount a disconnected agent manually:
In the Investigator tree, right-click the agent and choose Unmount from the menu.
» Note If the Unmount command is disabled, you do not have
historical_agent_control or full permission. Contact your
administrator.
Mounting agents
You might want to remount a disconnected agent to view previously collected
Metrics from a database. Disconnected and unmounted agents that connectedwhile database recording was enabled can be remounted.
8/20/2019 8_Introscope Configuration and Administration Guide
34/304
34 Administering the Enterprise Manager
CA Wily Introscope
When an application process is disconnected from Introscope and is not
recognized as the same returning agent, (due to a change in host location, etc.),
and has been unmounted, you can remount and view it in the tree. Metrics it was
collecting appear in the tree.
If a user remounts an agent, any Metrics that agent ever reported to the database
appear as Resources in the Investigator tree.
To mount an agent:
1 Choose Manager > Mount Agent.
The agent Chooser opens.
» Note If the Mount Agent command is disabled, you do not have
historical_agent_control or full permission. Contact your
administrator.
2 Click the agent to mount, and click Choose.
» Note If you mount an agent, you must unmount it manually. It does not auto-
unmount.
Shutting off agents
Use the Agent Shutoff feature to shut off reporting at the agent, Resource, or
Metric level.
You can individually turn on or shut off each agent, Resource, or Metric. You can
also turn on or shut off all components under an agent all at once.
It is important to remember that when you shut off a component, you are only
shutting off that component’s reporting—the component is still present.
You can turn on or off a component’s reporting in two ways:
turn on or off the component itself
turn on or off a component that is upstream (one that either directly contains
that component, or is several levels above)
8/20/2019 8_Introscope Configuration and Administration Guide
35/304
Shutting off agents 35
Configuration and Administration Guide
To change the shutoff state of agents, Resources, or Metrics, you must have
either read and live_agent_control permissions, or full permission for the
Domain.
Shutting off and turning on components
You can shut off and turn on agent, Resource, and Metric components. To shut
off or turn on components, right-click the component and choose the shutoff
command:
You can shut off and turn on agents, resources, and metrics:
To Do this
Shut off metric
reporting
By default, individual metric reporting is on (Shutoff is not
checked).
To shut off an individual metric’s reporting, right-click the metric
and choose Shutoff from the menu.
When you shut off an individual Metric:
the metric’s toggle is shut off
the metric’s icon in the Investigator tree turns gray
the metric stops reporting (shows as a change in the Reporting
Status field in the settings)
Turn on metric
reporting
To turn on a metric’s reporting, right-click the Metric and choose
Shutoff from the menu (this unchecks the Shutoff
option).
If data is available and metrics are not shut off because of an
upstream toggle, the metric begins reporting, and the metric’s
icon regains color. The Reporting Status field shows that the
metric is currently reporting.
8/20/2019 8_Introscope Configuration and Administration Guide
36/304
36 Administering the Enterprise Manager
CA Wily Introscope
Shut off resource
reporting
By default, individual resource reporting is on.
To shut off resource reporting, right-click the resource and choose
Shutoff from the menu. This checks the Shutoffoption.
When you shut off a Resource’s reporting:
the resource’s toggle is shut off
the resource’s icon turns gray
the resource stops reporting
any resources below (downstream) of the Resource turn gray
and stop reporting
any metrics below (downstream) of the Resource turn gray and
stop reporting
Turn on resource
reporting
To turn on an individual Resource’s reporting, right-click the
Resource and select Shutoff from the menu. This
unchecks the Shutoff option.
If data is available and Metrics are not shut off because of an
upstream toggle, the Metrics under the Resource begin reporting,
and their icons regain color. The Reporting Status field shows that
the Resource contains Metrics that are currently reporting.
Shut off agent
reporting
By default, individual agent reporting is on (Shutoff option is not
selected).
To shut off agent reporting, right-click the agent and select Shutoff
from the menu. This checks the Shutoff option.
When you shut off an agent’s reporting:
the agent’s toggle is shut off
the agent’s icon turns gray
the agent stops reporting
any Resources below (downstream) of the agent turn gray and
stop reporting
any Metrics below (downstream) of the agent turn gray and stop
reporting
Turn on agent
reporting
To turn on an individual agent’s reporting, right-click the agent
and select Shutoff from the right-click menu. This
unchecks the Shutoff option.
If data is available, the Metrics under the agent begin reporting,
and their icons regain color. The Reporting Status field shows that
the agent contains Metrics that are currently reporting.
To Do this
8/20/2019 8_Introscope Configuration and Administration Guide
37/304
Shutting off agents 37
Configuration and Administration Guide
Discovering why a component is not reporting data
Icons for Metrics and Resources in the Investigator tree are grayed out when they
have no data available to report. With the Agent Shutoff feature, there might also
be no data available either because the item’s toggle is off, or an upstream toggle
is off.
Shut off all agent
components
You can quickly turn on or shut off all components under an agent
that is reporting.
To shut off reporting for everything under a specific agent:
1 Open an Investigator window.2 Expand the Investigator tree to show the agent whose settings
you want to change.
3 Right-click on the agent and select Shut Off All AgentComponents. A warning dialog notifies that you cannot undo this
action.
4 Click Continue to shut off agent components.
The Agent and all components contained by it turn gray and stop
reporting.
Note: It is possible to have Agent reporting shut off and
individual Metrics underneath turned on. However,
upstream toggles override downstream toggles.
When you select Shut Off All Agent Components:
the Agent’s toggle is shut off
the Agent’s icon turns gray
the Agent stops reporting
all components under the Agent have their toggles shut off
the icons of all components under the Agent turn gray
all components under the Agent stop reporting
Note: This function is for shutting off reporting for an individual
Agent and all components under that Agent. It is not for
shutting off reporting for all connected Agents. Agent
reporting must be specifically turned on or shut off for
each Agent.
Turn on all Agentcomponents
To turn on reporting for All Agent Components, right-click theagent and select Turn On All Agent Components.
After a short delay (usually the time slice during which new data
is being gathered), the Agent and all components contained by it
regain color and begin reporting if there is data available.
To Do this
8/20/2019 8_Introscope Configuration and Administration Guide
38/304
38 Administering the Enterprise Manager
CA Wily Introscope
Icon appearance
This table details reasons why a component might not be reporting data:
Reporting status messages
You can investigate why data is not available to a component by looking at the
component’s Shutoff status and its Reporting Status field.
If a component is currently reporting, one of these messages appears:
is currently reporting
contains Metrics that are currently reporting
contains Metrics that are currently reporting
If a component is not reporting, one of these messages appears:
is shut off due to its own toggle
is shut off due to an upstream toggle
is not known to the current Agent
has no data available
contains no Metrics that have data available
contains no Metrics known to the current agent
contains no Metrics known to the current agent
» Note The above messages might occur in combination.
Metric Resource Agent
its toggle is shut off
an upstream Resource
or Agent toggle is shut
off
there is no data
available
while an Agent was
disconnected, its
application code
changed, and the
Metric is no longer
recognized by the
Agent
its toggle is shut off
an upstream Resource
or Agent toggle is shut
off
there is no data
available because all
Resources and Metrics it
contains might be shut
off
there is no data
available for Metrics
underneath
its toggle is shut off
Agent has been
disconnected (but has not
yet been unmounted)
there is no data available
because all Resources and
Metrics it contains might be
shut off
there is no data available
for Metrics underneath
8/20/2019 8_Introscope Configuration and Administration Guide
39/304
Shared state in multiple Workstations 39
Configuration and Administration Guide
Shared state in multiple Workstations
Several Workstations can communicate with one Enterprise Manager, and when
changes are made on one Workstation, they are immediately reflected on otherWorkstations.
Because of the Workstation’s shared state, several users could be editing the
same piece of information in the Management Module Editor or in the Console at
the same time.
You’ll encounter competing changes in the Management Module Editor if, for
example, you and another user are editing the same alert in the Management
Module Editor, and the other user clicks the Apply button first to save changes.
As soon as that user’s changes are applied, you see this message in your
Workstation:
You’ll encounter competing changes in the Console if, for example, you and
another user are editing the same Dashboard, and the other user selects File
> Save first. As soon as that user’s changes are applied, you see this message
in your Workstation:
Click Continue to close this dialog
box and return to your own edits.
If you click Apply, your changes
override those made by the other
user.
Click Update to close this dialog
box and update your settings with
those made by the other user.
Click Save to override changes
made by the other user.
Click Save As to save the current
Dashboard as a new Dashboard.
8/20/2019 8_Introscope Configuration and Administration Guide
40/304
40 Administering the Enterprise Manager
CA Wily Introscope
8/20/2019 8_Introscope Configuration and Administration Guide
41/304
Configuring Enterprise Manager Communications 41
CHAPTER 3
Configuring Enterprise ManagerCommunications
This section includes information about how the Enterprise Manager
communicates with other components in an Introscope environment, default
communications behaviors, and options for configuring Enterprise Manager
communications.
The default Enterprise Manager communications channel . . . . . . 42
Connecting to the Enterprise Manager across a firewall . . . . . . . 45
Enabling SSL for Enterprise Managers on OS/400. . . . . . . . . 45
Configuring the Enterprise Manager Web Server for HTTPS. . . . . . 46
8/20/2019 8_Introscope Configuration and Administration Guide
42/304
42 Configuring Enterprise Manager Communications
CA Wily Introscope
The default Enterprise Manager communicationschannel
By default the Enterprise Manager communicates with Agents, Workstations,
WebView, and in a cluster of Enterprise Managers on port 5001. See , for more
information.
All socket communication to the EM is obfuscated, and passwords are encrypted
The properties in the Enterprise Manager properties file that define
communication channels and their use are:
introscope.enterprisemanager.serversocketfactory.
introscope.enterprisemanager.port.
introscope.enterprisemanager.serversockets.reuseaddr
introscope.enterprisemanager.enabled.channels
Each Agent that communicates with the Enterprise Manager has analogous
properties that must specify matching values, as described in Defining Agent
Connection and Name Properties in the Java Agent Guide or the .NET Agent
Guide, as appropriate for your environment.
Similarly, the WebView properties file specifies the Enterprise Manager port in the
introscope.webview.enterprisemanager.tcp.port property.
Rebinding a communication channel to a port in use
There are socket states where nothing is actually using the socket, but memoryis still in use, thus rendering the socket unusable. Examples of these states are
FIN_WAIT and TIME_WAIT.
The property introscope.enterprisemanager.serversockets.reuseaddr on
page 257 allows an Enterprise Manager communication channel to rebind to a
local port that is in use. Use this feature if an Enterprise Manager socket is stuck
in an unusable state, but you still want to start the Enterprise Manager.
» Note Do not enable this property by default and use it only when a socket is
in an unusable state. You should disable it after the Enterprise Manager
starts, as there may be a valid reason that the port is in use (for
example, a collector on the same machine is already bound to the port).
8/20/2019 8_Introscope Configuration and Administration Guide
43/304
The default Enterprise Manager communications channel 43
Configuration and Administration Guide
Configuring SSL
Configuring the Enterprise Manager for SSL is done in
IntroscopeEnterpriseManager.properties by enabling a securecommunications channel. This can be in addition to the default non-secure
channel.
The following is an example of enabling both a secure and non-secure
communications channel. It sets up the default non-secure listener on port 5001,
and a secure listener on port 5443:
# The enabled communication channels.
introscope.enterprisemanager.enabled.channels=channel1, channel2
# The default communication channel.
introscope.enterprisemanager.serversocketfactory.channel1=com.wily.isen
gard.postofficehub.link.net.server.DefaultServerSocketFactory
introscope.enterprisemanager.port.channel1=5001
# The secure communication channel.
introscope.enterprisemanager.serversocketfactory.channel2=com.wily.isen
gard.postofficehub.link.net.server.SSLServerSocketFactory
introscope.enterprisemanager.port.channel2=5443
# Location of a keystore containing certificates for authenticating the
EM to clients.
# Either an absolute path or a path relative to the config directory.
# On Windows, backslashes must be escaped. For example:
#
introscope.enterprisemanager.keystore.channel1=C:\\Introscope\\config
\\internal\\server\\keystore
introscope.enterprisemanager.keystore.channel2=internal/server/keystore
# The password for the keystore.
introscope.enterprisemanager.keypassword.channel2=password
# Location of a truststore containing trusted client certificates.
# Either an absolute path or a path relative to the config directory.
# On Windows, backslashes must be escaped. For example:
#
introscope.enterprisemanager.truststore.channel1=C:\\Introscope\\conf
ig\\internal\\server\\keystore
# A truststore is only needed if client authentication is required.
# If no truststore is specified, the EM trusts all client certificates.
8/20/2019 8_Introscope Configuration and Administration Guide
44/304
44 Configuring Enterprise Manager Communications
CA Wily Introscope
SSL property rules
The Enterprise Manager must be configured with an SSL server socket in
IntroscopeEnterpriseManager.properties by enabling a secure
communications channel in addition to the default channel.
When configuring SSL properties, the following rules:
The keystore property is the location of a keystore. It should contain a
certificate used by the EM to authenticate itself to the client. The EM's default
keystore contains a self-signed certificate that is inherently untrustworthy.
This doesn't matter because by default the Workstation and Agent don't
validate server certificates.
The keystore property is either an absolute path or a path relative to the
config directory.
If the needclientauth property is set to true, the EM requires the client toauthenticate.
The EM must be configured with a trustore.
Agents and Workstations must be configured with a keystore containing a
certificate trusted by the EM.
If needclientauth = true, and a truststore is not configured, the EM does
not start.
This requires setting the trustore property.
The truststore property is the location of a truststore. It should contain the
certificates of trusted clients. It is only needed if clients are required to
authenticate.
If no truststore is specified, all client certificates are trusted.
The ciphersuites property is used to set the enabled cipher suites.
This can be used to enable anonymous authentication or avoid data
encryption. For example, for anonymous authentication, you might set the
cipher suites to SSL_DH_anon_WITH_RC4_128_MD5 . For example:
introscope.enterprisemanager.ciphersuites.channel2=SSL_DH_anon_WITH_R
C4_128_MD5
A particular JSSE implementation may have several cipher suites available
that permit anonymous authentication (they all contain "anon"). However,
the cipher suite will be common to both client and server, and enabled on
both. You might also specify a cipher suite that does not encrypt data. For
example:
introscope.enterprisemanager.ciphersuites.channel2=SSL_RSA_WITH_NULL_
SHA
8/20/2019 8_Introscope Configuration and Administration Guide
45/304
Connecting to the Enterprise Manager across a firewall 45
Configuration and Administration Guide
Connecting to the Enterprise Manager across afirewall
HTTP tunneling is enabled by default. Tunneling technology enables one network
to send its data via another network’s connections.
To use the tunneling transport on the Workstation, connect to the port for the
Enterprise Manager’s embedded web server. By default the port is 8081. You can
configure it with the introscope.enterprisemanager.webserver.port
property in the Enterprise Manager properties file. For more information, see
introscope.enterprisemanager.webserver.port .
In addition to using tunneling for the Enterprise Manager, you can configure
Agents to send information using tunneling technology. This feature enables
Agents to connect to an Enterprise Manager remotely. See the Java Agent Guide
for more information about Agent HTTP tunneling.
Enabling SSL for Enterprise Managers on OS/400
By default the AS/400 JVM uses a native JSSE provider. You must configure use
of the pure Java JSSE provider to allow an Enterprise Manager hosted on an AS/
400 to communicate with other Introscope components such as the Workstation,
or in a cluster, other Enterprise Managers.
To enable SSL for Enterprise Managers on OS/400:
1 Open your Java security file, usually in your JVM's lib/security directory.
2 In the list of security providers, look for com.ibm.jsse.IBMJSSEProvider . If itexists, make sure it is higher up the list and thus takes priority over other SSL
providers, including com.ibm.as400.ibmonly.net.ssl.Provider . If it not
present, add it to the list.
3 Set the following SSL properties, replacing any previous definitions:
ssl.KeyManagerFactory.algorithm=IbmX509
ssl.TrustManagerFactory.algorithm=IbmX509
ssl.SocketFactory.provider=com.ibm.jsse.JSSESocketFactory
ssl.ServerSocketFactory.provider=com.ibm.jsse.
JSSEServerSocketFactory
8/20/2019 8_Introscope Configuration and Administration Guide
46/304
46 Configuring Enterprise Manager Communications
CA Wily Introscope
Configuring the Enterprise Manager Web Server forHTTPS
You can configure the Enterprise Manger web server for HTTPS by using the Jetty
configuration file. The Jetty file is provided for advanced configuration and
can be found in the config directory of the
installation.
This file is activated by uncommenting the
introscope.enterprisemanager.webserver.jetty.configurationFile
property in IntroscopeEnterpriseManager.properties .
For additional information about configuration, refer to the following:
Default Jetty configuration file on page 47 and Modifying the Jetty-config.xml
file on page 48 http://docs.codehaus.org/display/JETTY/How+to+configure+SSL to access the
Jetty website for addtional information.
Understanding how HTTPS works
The Enterprise Manager web server supports HTTP tunneling over SSL.
This is done by introducing an optional Jetty 6.0.2 XML configuration file for the
embedded Jetty web server in the EM. A default configuration file is provided that
sets up a secure listener on port 8444, which allows the WorkStation and Agent
to access the EM using HTTPS tunneling through the EM web Server.
The Jetty configuration file is specified by the
introscope.enterprisemanager.webserver.jetty.configurationFile
property in IntroscopeEnterpriseManager.properties . The property
specifies a path to a Jetty 6.0.2 XML configuration file. It can be absolute or a
path relative to the config directory. The configuration file is loaded at startup
and does not support hot configuration. The default value is em-jetty-
config.xml. For example, it would appear as follows:
introscope.enterprisemanager.webserver.jetty.configurationFile
=em-jetty-config.xml
When you configure the Jetty configuration properties in the EM Web Server
Settings section of IntroscopeEnterpriseManager.properties thefollowing rules apply:
If the Jetty config file is successfully loaded, the default non-secure listener
specified by introscope.enterprisemanager.webserver.port is not
created.
8/20/2019 8_Introscope Configuration and Administration Guide
47/304
Configuring the Enterprise Manager Web Server for HTTPS 47
Configuration and Administration Guide
If a non-secure HTTP listener is desired in addition to a secure HTTPS listener,
the default config file ha
Top Related