Auto_trouble
-
Upload
sathish-kandasamy -
Category
Documents
-
view
93 -
download
10
description
Transcript of Auto_trouble
CA AutoSys Workload Automation r11 Automation r11 TroubleshootingPresenters: Dan Shannon Elizabeth DexterPresenters: Dan Shannon, Elizabeth Dexter
Terms of This Presentation
This presentation was based on current information and resource allocations as of October 2009 and is subject to change or withdrawal by CA at any time without notice. Notwithstanding anything in this presentation to the contrary, this presentation shall not serve g y g p y, pto (i) affect the rights and/or obligations of CA or its licensees under any existing or future written license agreement or services agreement relating to any CA software product; or (ii) amend any product documentation or specifications for any CA software product. The development release and timing of any features or functionality described in this presentation development, release and timing of any features or functionality described in this presentation remain at CA’s sole discretion. Notwithstanding anything in this presentation to the contrary, upon the general availability of any future CA product release referenced in this presentation, CA will make such release available (i) for sale to new licensees of such product; and (ii) to existing licensees of such product on a when and if-available basis as part of CA maintenance and support, and in the form of a regularly scheduled major product release. Such releases may be made available to current licensees of such product who are current subscribers to CA maintenance and support on a when and if-available basis. In the event of a conflict between ppthe terms of this paragraph and any other information contained in this presentation, the terms of this paragraph shall govern.
2 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
For Informational Purposes Only
Certain information in this presentation may outline CA’s general product direction. All information in this presentation is for your informational purposes only and may not be incorporated into any contract. CA assumes no responsibility for the accuracy or completeness p y p y y pof the information. To the extent permitted by applicable law, CA provides this document “as is” without warranty of any kind, including without limitation, any implied warranties or merchantability, fitness for a particular purpose, or non-infringement. In no event will CA be liable for any loss or damage direct or indirect from the use of this document including liable for any loss or damage, direct or indirect, from the use of this document, including, without limitation, lost profits, lost investment, business interruption, goodwill, or lost data, even if CA is expressly advised of the possibility of such damages.
3 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
Agenda:
>Preparing for the Installation
>Installation and Initial Configurationg
>Customisation and Tuning
>Common Issues
4 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
Preparing for Installation
> ArchitectureWhat are you going to install, and where?
What are you planning to support?
> SystemM hi i iMachine sizing
> OS requirementsPatches, librariesPatches, libraries
> NetworkingCan the machines communicate?
5 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
Installation
> Verify that the common component installs were successful
> Verify that you can communicate between machines
6 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
Architecture
7 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
WCC PortsServices Related Applications Start Stop SSL
Launcher Server: WCC MainWCC Main LoginWeb ServicesWeb Services Samples ApplicationCredential Application
8080 8005 8443
C fi ti S C fi ti 10132 10133 10134Configuration Server: Configuration 10132 10133 10134
Event Server: Event ConsoleQuick Edit
10138 10139 10140
Job Status Console Servant: 10143 10144 10145
Job Status Console Server: Job Status Console 10122Job Status Views
Monitoring Server: Job Flow DesignJob Flow MonitoringCritical Path Monitoring
10135 10136 10137
h d lScheduling Server: Job EditorJob Editor Plus (BLOB editor)Application Editor
10129 10130 10131
UI Framework Server: Quick ViewQuick StartEnterprise Command Line
10146 10147 10148
8 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
HA Server: High AvailabilityReporting Collectors
10149 10150 10151
Installation and Initial Configuration
> Installing the Common ComponentsSSA
EEM
CCS/Event Management
> C ti it> ConnectivityEEM
AutoSys/WCC/SSAy / /
> Multicast (WCC)
9 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
Common Issues
> Collecting Information for SupportRelease, maintenance level, logs, system information
> SSA Issues
> EEM Issues
> AutoSys Issues
> WCC Issues
> Multiple NIC cards > Multiple NIC cards
> Command Sponsor is not installed
10 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
Logging and Debugging
> CA Workload Automation AutoSys Logging
> EEM Logginggg g
> WCC Logging
> SSA Logging
11 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
CA Workload Automation AutoSys Logging
> ISDBGACTIV
> Log Filesgunivagent.out
as_server.INS
t d INSevent_demon.INS
auto_rem.joid.runnum
12 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
EEM Logging (SDK)
> New With SDK 8.4 SR01
> EIAMCONFIG Environment
> Configuration Fileseiam.config
logger.config
13 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
WCC Logging
> WCC Logging
> Location: <WCC Install Root> Logs folderUses log4jUses log4j
– Access (not currently used)
– ApplicationAdmin, config, event, haserver, jobcontrolservant, jobcontrolserver, launcher, monitoring, reporting, uiframework
Log4j propertiesLog4j.properties
– Install
– Services
T t L> Tomcat Logs<WCC Install Root>/xxx/logs
14 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
Determining the Version
> AutoSysautoflags
> WCCuejmver.bat (Windows), uejmver.sh (*NIX)
15 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
SSA Logging
> Environment VariablesCSAM_SSL_DEBUG
CSAM_DEBUG
CSAM_LOG
16 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
Directory Layout/ o p t / C A / U n i c e n t e r A u t o S y s J M
a r c h i v e b i n
$ A U T O S Y S
a g e n t
o u t$ A U T O U S E R
o u t l i b
d b o b j O R A
I N G
t x
r e q u e s t
j o b s t
s n m p
d a t ai n s t a l l
S Y B
d o c m a n 8
r e s p o n s e
m e s s a g e s
t e s t j i l
b i n
n i t e l y
C o n f i g u r a t i o n F i l e s :
* c o n f i g . $ A U T O S E R V* a u t o s y s . s h . h o s t* a u t o s y s c s h h o s t b i b
17 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
d e m oO u t T h e D o o r
a u t o s y s . c s h . h o s t* a u t o s y s . k s h . h o s t* a u t o s y s . b a s h . h o s te t c .
b i n _ ba k . . .
l i b _ ba k . .
SSA Related Files
18 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
SSA Configuration Global
19 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
SSA Configuration Port (Cont)
20 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
SSA Application Logic
21 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
SSA Configuration Port
22 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
AutoSys Communication Problems
> CAUAJM_E_10029 Communication attempt with the Unicenter AutoSys JM Application Server has failed! [host:port]. [host:port].
Can you ping the App Server host?
Is the App Server running?
Are AutoServer/AutoServerPort values correct?
Is port 7163 (SSA) blocked by firewall?
Are SSA port configurations the same on client & App Are SSA port configurations the same on client & App Server machines the same?
– EnablePmux
E bl SSL– EnableSSL
23 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
AutoSys Communication Problems
> CAUAJM_E_10527 Timed out waiting for response from the Unicenter AutoSys JM Application Server [host:port]
Is the App Server running?Is the App Server running?
Are there messages in the App Server log file?– ISDBGACTIV=LIGHT,LOG
Is the SSA PortRange for 49152-50176 configured correctly?
– EnablePmux
– EnableSSL
Do the App Server and RDBMS machines have adequate resources?resources?
Does increasing DB_CONNECTIONS reduce problem?
24 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
Troubleshooting the Scheduler
Job stuck in STARTING
> Did Scheduler complete the Agent handshake?> Did Scheduler complete the Agent handshake?CAUAJM_I_10082 [POLAL02-GX280 connected for job1 626.225.1]
> What does corresponding Job Agent log file show?CAUAJM_E_10527 Timed out waiting for response from the Unicenter AutoSys JM Application Serverthe Unicenter AutoSys JM Application Server.
CAUAJM_E_10221 Exhausted list of application servers. Failing request.
25 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
Troubleshooting the Agent
CAUAJM_E_50026 ERROR: AutoPing WAS NOT SUCCESSFUL!
> Does univagent log file show activity under ISDBGACTIV=LIGHT,JOB traces?
> Do OS network tools reach Agent machine? (ping, nslookup, tracert/traceroute)
> Is firewall blocking SSA physical port 7163?> Is firewall blocking SSA physical port 7163?> Do SSA settings for AutoServerPort match (autoping –D)? > Do SSA settings for PortRange=49152-50176 match?
EnablePmuxEnableSSL
26 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
EEM Troubleshooting
> Performance IssuesCheck logging levels
Active Directory connectivity
Linux kernel
Cache settingsCache settings
> Unexpected permissions errorsCheck for mixed case directory entries y
> Active Directory Group RetrievalFilters
27 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
EEM Troubleshooting
CAUAJM_E_10448 FATAL SECURITY ERROR: Tampering or corruption of the security-related database keys detected
> Did someone install a new instance using an MDB from a previous instance installation?previous instance installation?
> Did someone directly modify the ujo_alamode or ujo_keymaster keys?j _ y y
> What is the state of the database tables? Did some unknown corruption of keys take place?
28 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
EEM Troubleshooting
CAUAJM_E_10436 Security server unreachable or invalid authentication certificate file
> Is the EEM backend running?> Do OS network tools reach EEM backend machine? (ping > Do OS network tools reach EEM backend machine? (ping,
nslookup, tracert/traceroute)> Is firewall blocking EEM physical port 5250?> Was the proper EEM backend host name entered?> Are you able to connect to the EEM backend via the
as safetool?as_safetool?> Does the AUTOUSER folder contain a valid certificate file?29 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
EEM Troubleshooting
Policy problems
> Does the EEM web UI Permission Check work as expected?
> Does autosec test work as expected?> Does autosec_test work as expected?> Does as_safetool work?
30 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
WCC - Other
> Job Flow MonitoringMonitor ID
Effective limit on the number of jobs in a flow
> Job Status ConsoleGl b l S i /Gl b l IDGlobal Session/Global ID
Prior to WCC r11.1 SP1, check alarms/alerts filters
> Unexpected login prompts> Unexpected login promptsIAM Security setting in server definition
31 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
Command Sponsor Troubleshooting
> Required for Quick View and ECLI in WCC
> Requires a credential user that is authorised on the OS to qexecute the command
> Restricted to the commands in the AutoSysCommandISponsorFilters txt fileAutoSysCommandISponsorFilters.txt file
> Uses iGateway
32 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA
Questions?
33 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright © 2009 CA