© copyright 2014 BMC Software, Inc.
10/2015
Control-M…As an Application
Management Platform
Vedran Vesel, ImavesControl-M stručnjak
© copyright 2014 BMC Software, Inc.
Control-M 9
Unmatched application workflow automation
Lowering TCO
Increasing application deployment speed
Gartner Magic Quadrant for Workload Automation
© copyright 2014 BMC Software, Inc.
BMC Control-M Workload AutomationRecent Releases
Big Data / Hadoop
Databases (Hana, Teradata, more…)
Oracle Retail
SAP
IBM Cognos
Backup
Java, Messaging & Web Services
Application Integrator
Application Hub
Archiving
Workload Change Manager
JCL Verify
Mobile App
Workload Conversion & Discovery
Single Point of Control Service Level Management Self Service & Mobility Audit & Compliance
BMC Confidential – Subject to Change
© copyright 2014 BMC Software, Inc.
The Only Constant isChange
© copyright 2014 BMC Software, Inc.
“Legacy” Application Lifecycle
5
RUN
DEPLOY
DESIGN
DEVELOP
TEST
Zero Value Value
© copyright 2014 BMC Software, Inc.
The Workload Change Request Lifecycle…
#&?!!!
#&?!!!
#&?!!!
#&?!!!
BMC Control-MWorkload Change Manager
#&?!!!
© copyright 2014 BMC Software, Inc.
Workload Change Manager : Functionality
• Web interface, easy and intuitive, for requesting or modifying batch processes
• Available items and features (job types, actions, etc) can be configured depending on the end user
• Site standards & naming conventions enforcement
• Follow-up for each request status, with optional email notification
• Tracks and audits all changes
• Integration with external Workflow and Change Management systems
© copyright 2014 BMC Software, Inc.
Workload Change Manager : Benefits
Standardize the workload change request process
– Avoid mistakes that can lead to potential problems on business services– Enforce the predefined standards and naming conventions– Totally audited process
Accelerate the go-live of new business processes
– Avoid a manual, slow and error prone process– Enable an easy development of batch workflows for developers and other users– Immediate notification of the requests statuses
Save time for both the users requesting the change and the administrators
– Thanks to a intuitive interface for users and a total integration in the administrator console– With an automatic update in the change management system, avoiding additional time needed
to update it manually
© copyright 2014 BMC Software, Inc.
Customer case
• 400K daily jobs; over a million defined in total
• About 60,000 change requests monthly
• 15 schedulers dedicated to making changes
• Current process is Help desk ticket, email, phone
• Expect 80% reduction in tickets and time
• Addresses compliance, audit and security gaps
80 % Reduction
Itau Unibanco accelerates change process by 80%
© copyright 2014 BMC Software, Inc.
BIG DATA
© copyright 2014 BMC Software, Inc.
Customer Loyalty• 360 degree view of the customer• 150+ variables about each customer• Enhance customer value• Cultivate high-value customers
Boeing 787 generates an average of 500GB of system data a flight
Airbus A380 is fitted with as many as 25,000 sensors capturing 8,000+ data points per second
Airlines are looking to use emerging technologies, such as Hadoop and sophisticated data mining algorithms, to capture unstructured data
Operations• Bag tracking data• Ticketing data• Scheduling data
Capturing sensor data• Optimize maintenance • Minimize downtime• Forecasting the weather to optimize fuel loads• Optimization of flight path / Trajectory corrections• Flaps – Data• Wingtips – Improved gas mileage• System Data
Engine data – One cross-country flight• GE jet engines collect information at 5,000 data points per second
20 TB x 2 x 6 x 28,537 x 365Info per engine
per hourEngines hours Commercial flights per
day in the U.S.Days per year
= 2,499,841,200 TB
© copyright 2014 BMC Software, Inc.
Traditional Hadoop
© copyright 2014 BMC Software, Inc.
Nearly 100% of processing in Hadoop is batch
Scheduling tools available for Hadoop:
– Open source tools like Oozie, Spring, Azkaban
– Batch scheduling tools are “version 1” at best
– Batch jobs are hand-written scripts
– Tools are used for scheduling jobs – not creating and managing workflows
Big Data –
The Batch Challenge
© copyright 2014 BMC Software, Inc.
SQL Query
#!/usr/bin/sh# Sample pmcmd scriptset pagesize 0 linesize 80 feedback off
SELECT 'The database ' || instance_name ||' has been running since ' || to_char(startup_time, 'HH24:MI MM/DD/YYYY')FROM v$instance;
SELECT 'There are ' || count(status) ||' data files with a status of ' || statusFROM dba_data_filesGROUP BY statusORDER BY status;
SELECT 'The total storage used by the data files is ' ||sum(bytes)/1024/1024 || ' MB'FROM dba_data_files;
#!/usr/bin/env bashbin=`dirname "$0"`bin=`cd "$bin"; pwd`. "$bin"/../libexec/hadoop-config.sh#set the hadoop command and the path to the hadoop jarHADOOP_CMD="${HADOOP_PREFIX}/bin/hadoop --config $HADOOP_CONF_DIR“#find the hadoop jarHADOOP_JAR='‘#find under HADOOP_PREFIX (tar ball install)HADOOP_JAR=`find ${HADOOP_PREFIX} -name 'hadoop--*.jar' | head -n1`#if its not found look under /usr/share/hadoop (rpm/deb installs)if [ "$HADOOP_JAR" == '' ]then
HADOOP_JAR=`find /usr/share/hadoop -name 'hadoop--*.jar' | head -n1`fi#if it is still empty then dont run the testsif [ "$HADOOP_JAR" == '' ]then
echo "Did not find hadoop--*.jar under '${HADOOP_PREFIX} or '/usr/share/hadoop'"
exit 1fi#dir where to store the data on hdfs. The data is relative of the users home dir on hdfs.PARENT_DIR="validate_deploy_`date+%s`“TERA_GEN_OUTPUT_DIR="${PARENT_DIR}/tera_gen_data“TERA_SORT_OUTPUT_DIR="${PARENT_DIR}/tera_sort_data“
Hadoop
#!/bin/ksh
cd /home/bmcU1ser/ftp_race_source
sftp -b /dev/stdin -o Cipher=blowfish -o Compression=yes -o BatchMode=yes -o IdentityFile=/export/home/user/.ssh/id_rsa -o Port=22 bmcUs1ser@hou-hadoop-mstr 1>sftp.log 2>&1 <<ENDSFTP
if [ -f /home/bmcU1ser/ftp_race_target/daily_shipment_log ]; thenexit 1
elseput daily_shipment_log /home/bmcU1ser/ftp_race_target
fiquitENDSFTPrc=$?if [[ $rc != 0 ]]; then
print "***Error occurred...$rc" `date "+%Y-%m-%d-%H.%M.%S"`if [[ -f /home/bmcU1ser/ftp_race_target/daily_shipment_log ]]; then
rm /home/bmcU1ser/ftp_race_target/daily_shipment_logfi
elsemv /home/bmcU1ser/ftp_race_source/daily_shipment_log
/home/bmcU1ser/ftp_race_source/old/daily_shipment_logprint "***Successful transfer...$rc" ̀ date "+%Y-%m-%d-%H.%M.%S"`
fi
File TransferInformatica
#!/usr/bin/bash# Sample pmcmd script# Check if the service is alivepmcmd pingservice -sv testService -d testDomainif [ "$?" != 0 ]; then# handle errorecho "Could not ping service"exitfi# Get service propertiespmcmd getserviceproperties -sv testService -d testDomainif [ "$?" != 0 ]; then# handle errorecho "Could not get service properties"exitfi# Get task details for session task "s_testSessionTask" of workflow# "wf_test_workflow" in folder "testFolder"pmcmd gettaskdetails -sv testService -d testDomain -u Administrator -p adminPass-folder testFolder -workflow wf_test_workflow s_testSessionTaskif [ "$?" != 0 ]; then# handle errorecho "Could not get details for task s_testSessionTask"exitfi
Programmers program
© copyright 2014 BMC Software, Inc.
What happens when this runs?
• What is related to what?
• Are we on time or late?
• What if something fails?– Which program was running?
– Where is the output?
– How do I fix it?
– Can I just rerun it? If so,from the beginning?
– Does any cleanup have to be done?
– How do I track this problem and the stepstaken to resolve the problem?
#!/usr/bin/env bashbin=`dirname "$0"`bin=`cd "$bin"; pwd`. "$bin"/../libexec/hadoop-config.sh#set the hadoop command and the path to the hadoop jarHADOOP_CMD="${HADOOP_PREFIX}/bin/hadoop --config $HADOOP_CONF_DIR“#find the hadoop jarHADOOP_JAR='‘#find under HADOOP_PREFIX (tar ball install)HADOOP_JAR=`find ${HADOOP_PREFIX} -name 'hadoop--*.jar' | head -n1`#if its not found look under /usr/share/hadoop (rpm/deb installs)if [ "$HADOOP_JAR" == '' ]then
HADOOP_JAR=`find /usr/share/hadoop -name 'hadoop--*.jar' | head -n1`fi#if it is still empty then dont run the testsif [ "$HADOOP_JAR" == '' ]then
echo "Did not find hadoop--*.jar under '${HADOOP_PREFIX} or '/usr/share/hadoop'"
exit 1fi#dir where to store the data on hdfs. The data is relative of the users home dir on hdfs.PARENT_DIR="validate_deploy_`date+%s`“TERA_GEN_OUTPUT_DIR="${PARENT_DIR}/tera_gen_data“TERA_SORT_OUTPUT_DIR="${PARENT_DIR}/tera_sort_data“
Hadoop
© copyright 2014 BMC Software, Inc.
SQL Query HadoopFile TransferInformatica
A better way
BMC Control-M
© copyright 2014 BMC Software, Inc.
Defining Control-M for Hadoop jobs
Hadoop Program parameters
HDFS commands
© copyright 2014 BMC Software, Inc.
A better way
© copyright 2014 BMC Software, Inc.
And the fun is just beginning…
© copyright 2014 BMC Software, Inc.
Integration for All Applications
© copyright 2014 BMC Software, Inc.
Applications.
And More
Applications.
Teradata
Splunk
Microsoft Access
Informix
SQLite
HP Vertica
Cassandra
MariaDB
Neo4j
Redis
Microsoft SQL Server Reporting Services (SSRS)
Microsoft SQL Server Analysis Services (SSAS)
SAS
Microstrategy
Tibco Analytics
Splunk Enterprise
Qlikview
Other
Tableau
Pentaho
SPSS
Salesforce.com
Oracle JD Edwards EnterpriseOne
Workdays
Other
Concur
Infor Lawson
Oracle Flexcube (iFlex)
Accenture Alnova
TCS BaNCS (Tata Consulting services) core banking solution
Infosys Finance
2015 Workload Automation Survey
© copyright 2014 BMC Software, Inc.
Control-M Application Integrator
• Build applications faster
• Reduce scripting effort
• Rapid deployment of new applications or application changes
• Minimize risk and downtime in case ofjob failures
© copyright 2014 BMC Software, Inc.
Applications & Platforms
© copyright 2014 BMC Software, Inc.
Native application plug-ins
Taking advantage of all CONTROL-M workload automation offerings
Predefined connection profiles holding secured credentials
Eliminate copy/paste or manually typing of job attributes
Automatic load of input parameters, allow use of variables inputs
Control level of details to include in job output for auto recovery
Advanced controls including restart from point of failure
© copyright 2014 BMC Software, Inc.
Automation steps in a job
Start the Job in the application
Monitor job progress and determine completion
Terminate the application job interactively
Retrieve the job output from the application
Execute
Progress
Abort
Output
© copyright 2014 BMC Software, Inc.
New Type of Job in Control-M
Job available in the Templates Manager
© copyright 2014 BMC Software, Inc.
Innovation Made Easy Increase Value The Power of Many
Application Integrator
A workload automation design tool that connects applications and process so business services are quickly and reliably delivered to customers
© copyright 2014 BMC Software, Inc.
AdvancedFile Transfer
© copyright 2014 BMC Software, Inc.
Control-M Control Module for Advanced File Transfer
• Integrate file transfers into automated workflows:
– Manage FTP transfers securely
– Eliminate lag time between file transfers and subsequent processing steps
– Gain instant visibility into the status of your transfers.
– Use automated recovery capabilities to improve reliability.
© copyright 2014 BMC Software, Inc.
Control-M for SAP 8
• Optimize your SAP environment
– XBP 3.0 certification
– Conversion
• Coming next
– SAP Process Integration (PI)
– Solution Manager
– Financial Closing Cockpit (FCC)
BMC Confidential – Subject to Change
© copyright 2014 BMC Software, Inc.
Oracle Retail
• Quickly integrate Oracle Retail
jobs into your workload
automation environment
– Oracle Retail Merchandising
System (RMS)
– Oracle Retail Price Management
(RPM)
– Support release 13.x and higher
© copyright 2014 BMC Software, Inc.
Leading Workload Automation Solution in the market
• History of innovationand investment
• On-going customers engagement
• Record breaking CXI
Why Control-M 9?
• Unmatched applicationworkflow automation
• Lowering TCO
• Increasing applicationdeployment speed
Key Elements
• Automated agent and client deployment
• Runtime analytics
• High Availability
• Automated application workflow
promotion between environments
Release Highlights
Control-M 9
Executive Summary
© copyright 2014 BMC Software, Inc.
Our commitment…
remaining focused on customers.
© copyright 2014 BMC Software, Inc.
Bring IT to Life.
Hvala na pažnji!
TM
Top Related