Control-M…As an Application Management Platform

© copyright 2014 BMC Software, Inc.

10/2015

Control-M…As an Application

Management Platform

Vedran Vesel, ImavesControl-M stručnjak


Control-M 9

Unmatched application workflow automation

Lowering TCO

Increasing application deployment speed

Gartner Magic Quadrant for Workload Automation


BMC Control-M Workload AutomationRecent Releases

Big Data / Hadoop

Databases (Hana, Teradata, more…)

Oracle Retail

SAP

IBM Cognos

Backup

Java, Messaging & Web Services

Application Integrator

Application Hub

Archiving

Workload Change Manager

JCL Verify

Mobile App

Workload Conversion & Discovery

Single Point of Control Service Level Management Self Service & Mobility Audit & Compliance

BMC Confidential – Subject to Change


The Only Constant isChange


“Legacy” Application Lifecycle

5

RUN

DEPLOY

DESIGN

DEVELOP

TEST

Zero Value Value


The Workload Change Request Lifecycle…

#&?!!!

#&?!!!

#&?!!!

#&?!!!

BMC Control-MWorkload Change Manager

#&?!!!


Workload Change Manager : Functionality

• Web interface, easy and intuitive, for requesting or modifying batch processes

• Available items and features (job types, actions, etc) can be configured depending on the end user

• Site standards & naming conventions enforcement

• Follow-up for each request status, with optional email notification

• Tracks and audits all changes

• Integration with external Workflow and Change Management systems


Workload Change Manager : Benefits

Standardize the workload change request process

– Avoid mistakes that can lead to potential problems on business services– Enforce the predefined standards and naming conventions– Totally audited process

Accelerate the go-live of new business processes

– Avoid a manual, slow and error prone process– Enable an easy development of batch workflows for developers and other users– Immediate notification of the requests statuses

Save time for both the users requesting the change and the administrators

– Thanks to a intuitive interface for users and a total integration in the administrator console– With an automatic update in the change management system, avoiding additional time needed

to update it manually


Customer case

• 400K daily jobs; over a million defined in total

• About 60,000 change requests monthly

• 15 schedulers dedicated to making changes

• Current process is Help desk ticket, email, phone

• Expect 80% reduction in tickets and time

• Addresses compliance, audit and security gaps

80 % Reduction

Itau Unibanco accelerates change process by 80%


BIG DATA


Customer Loyalty• 360 degree view of the customer• 150+ variables about each customer• Enhance customer value• Cultivate high-value customers

Boeing 787 generates an average of 500GB of system data a flight

Airbus A380 is fitted with as many as 25,000 sensors capturing 8,000+ data points per second

Airlines are looking to use emerging technologies, such as Hadoop and sophisticated data mining algorithms, to capture unstructured data

Operations• Bag tracking data• Ticketing data• Scheduling data

Capturing sensor data• Optimize maintenance • Minimize downtime• Forecasting the weather to optimize fuel loads• Optimization of flight path / Trajectory corrections• Flaps – Data• Wingtips – Improved gas mileage• System Data

Engine data – One cross-country flight• GE jet engines collect information at 5,000 data points per second

20 TB x 2 x 6 x 28,537 x 365Info per engine

per hourEngines hours Commercial flights per

day in the U.S.Days per year

= 2,499,841,200 TB


Traditional Hadoop


Nearly 100% of processing in Hadoop is batch

Scheduling tools available for Hadoop:

– Open source tools like Oozie, Spring, Azkaban

– Batch scheduling tools are “version 1” at best

– Batch jobs are hand-written scripts

– Tools are used for scheduling jobs – not creating and managing workflows

Big Data –

The Batch Challenge


SQL Query

#!/usr/bin/sh# Sample pmcmd scriptset pagesize 0 linesize 80 feedback off

SELECT 'The database ' || instance_name ||' has been running since ' || to_char(startup_time, 'HH24:MI MM/DD/YYYY')FROM v$instance;

SELECT 'There are ' || count(status) ||' data files with a status of ' || statusFROM dba_data_filesGROUP BY statusORDER BY status;

SELECT 'The total storage used by the data files is ' ||sum(bytes)/1024/1024 || ' MB'FROM dba_data_files;

#!/usr/bin/env bashbin=`dirname "$0"`bin=`cd "$bin"; pwd`. "$bin"/../libexec/hadoop-config.sh#set the hadoop command and the path to the hadoop jarHADOOP_CMD="${HADOOP_PREFIX}/bin/hadoop --config $HADOOP_CONF_DIR“#find the hadoop jarHADOOP_JAR='‘#find under HADOOP_PREFIX (tar ball install)HADOOP_JAR=`find ${HADOOP_PREFIX} -name 'hadoop--*.jar' | head -n1`#if its not found look under /usr/share/hadoop (rpm/deb installs)if [ "$HADOOP_JAR" == '' ]then

HADOOP_JAR=`find /usr/share/hadoop -name 'hadoop--*.jar' | head -n1`fi#if it is still empty then dont run the testsif [ "$HADOOP_JAR" == '' ]then

echo "Did not find hadoop--*.jar under '${HADOOP_PREFIX} or '/usr/share/hadoop'"

exit 1fi#dir where to store the data on hdfs. The data is relative of the users home dir on hdfs.PARENT_DIR="validate_deploy_`date+%s`“TERA_GEN_OUTPUT_DIR="${PARENT_DIR}/tera_gen_data“TERA_SORT_OUTPUT_DIR="${PARENT_DIR}/tera_sort_data“

Hadoop

#!/bin/ksh

cd /home/bmcU1ser/ftp_race_source

sftp -b /dev/stdin -o Cipher=blowfish -o Compression=yes -o BatchMode=yes -o IdentityFile=/export/home/user/.ssh/id_rsa -o Port=22 bmcUs1ser@hou-hadoop-mstr 1>sftp.log 2>&1 <<ENDSFTP

if [ -f /home/bmcU1ser/ftp_race_target/daily_shipment_log ]; thenexit 1

elseput daily_shipment_log /home/bmcU1ser/ftp_race_target

fiquitENDSFTPrc=$?if [[ $rc != 0 ]]; then

print "***Error occurred...$rc" `date "+%Y-%m-%d-%H.%M.%S"`if [[ -f /home/bmcU1ser/ftp_race_target/daily_shipment_log ]]; then

rm /home/bmcU1ser/ftp_race_target/daily_shipment_logfi

elsemv /home/bmcU1ser/ftp_race_source/daily_shipment_log

/home/bmcU1ser/ftp_race_source/old/daily_shipment_logprint "***Successful transfer...$rc" ̀ date "+%Y-%m-%d-%H.%M.%S"`

fi

File TransferInformatica

#!/usr/bin/bash# Sample pmcmd script# Check if the service is alivepmcmd pingservice -sv testService -d testDomainif [ "$?" != 0 ]; then# handle errorecho "Could not ping service"exitfi# Get service propertiespmcmd getserviceproperties -sv testService -d testDomainif [ "$?" != 0 ]; then# handle errorecho "Could not get service properties"exitfi# Get task details for session task "s_testSessionTask" of workflow# "wf_test_workflow" in folder "testFolder"pmcmd gettaskdetails -sv testService -d testDomain -u Administrator -p adminPass-folder testFolder -workflow wf_test_workflow s_testSessionTaskif [ "$?" != 0 ]; then# handle errorecho "Could not get details for task s_testSessionTask"exitfi

Programmers program


What happens when this runs?

• What is related to what?

• Are we on time or late?

• What if something fails?– Which program was running?

– Where is the output?

– How do I fix it?

– Can I just rerun it? If so,from the beginning?

– Does any cleanup have to be done?

– How do I track this problem and the stepstaken to resolve the problem?

#!/usr/bin/env bashbin=`dirname "$0"`bin=`cd "$bin"; pwd`. "$bin"/../libexec/hadoop-config.sh#set the hadoop command and the path to the hadoop jarHADOOP_CMD="${HADOOP_PREFIX}/bin/hadoop --config $HADOOP_CONF_DIR“#find the hadoop jarHADOOP_JAR='‘#find under HADOOP_PREFIX (tar ball install)HADOOP_JAR=`find ${HADOOP_PREFIX} -name 'hadoop--*.jar' | head -n1`#if its not found look under /usr/share/hadoop (rpm/deb installs)if [ "$HADOOP_JAR" == '' ]then

HADOOP_JAR=`find /usr/share/hadoop -name 'hadoop--*.jar' | head -n1`fi#if it is still empty then dont run the testsif [ "$HADOOP_JAR" == '' ]then

echo "Did not find hadoop--*.jar under '${HADOOP_PREFIX} or '/usr/share/hadoop'"

exit 1fi#dir where to store the data on hdfs. The data is relative of the users home dir on hdfs.PARENT_DIR="validate_deploy_`date+%s`“TERA_GEN_OUTPUT_DIR="${PARENT_DIR}/tera_gen_data“TERA_SORT_OUTPUT_DIR="${PARENT_DIR}/tera_sort_data“

Hadoop


SQL Query HadoopFile TransferInformatica

A better way

BMC Control-M


Defining Control-M for Hadoop jobs

Hadoop Program parameters

HDFS commands


A better way


And the fun is just beginning…


Integration for All Applications


Applications.

And More

Applications.

Teradata

Splunk

Microsoft Access

Informix

SQLite

HP Vertica

Cassandra

MariaDB

Neo4j

Redis

Microsoft SQL Server Reporting Services (SSRS)

Microsoft SQL Server Analysis Services (SSAS)

SAS

Microstrategy

Tibco Analytics

Splunk Enterprise

Qlikview

Other

Tableau

Pentaho

SPSS

Salesforce.com

Oracle JD Edwards EnterpriseOne

Workdays

Other

Concur

Infor Lawson

Oracle Flexcube (iFlex)

Accenture Alnova

TCS BaNCS (Tata Consulting services) core banking solution

Infosys Finance

2015 Workload Automation Survey


Control-M Application Integrator

• Build applications faster

• Reduce scripting effort

• Rapid deployment of new applications or application changes

• Minimize risk and downtime in case ofjob failures


Applications & Platforms


Native application plug-ins

Taking advantage of all CONTROL-M workload automation offerings

Predefined connection profiles holding secured credentials

Eliminate copy/paste or manually typing of job attributes

Automatic load of input parameters, allow use of variables inputs

Control level of details to include in job output for auto recovery

Advanced controls including restart from point of failure


Automation steps in a job

Start the Job in the application

Monitor job progress and determine completion

Terminate the application job interactively

Retrieve the job output from the application

Execute

Progress

Abort

Output


New Type of Job in Control-M

Job available in the Templates Manager


Innovation Made Easy Increase Value The Power of Many

Application Integrator

A workload automation design tool that connects applications and process so business services are quickly and reliably delivered to customers


AdvancedFile Transfer


Control-M Control Module for Advanced File Transfer

• Integrate file transfers into automated workflows:

– Manage FTP transfers securely

– Eliminate lag time between file transfers and subsequent processing steps

– Gain instant visibility into the status of your transfers.

– Use automated recovery capabilities to improve reliability.


Control-M for SAP 8

• Optimize your SAP environment

– XBP 3.0 certification

– Conversion

• Coming next

– SAP Process Integration (PI)

– Solution Manager

– Financial Closing Cockpit (FCC)

BMC Confidential – Subject to Change


Oracle Retail

• Quickly integrate Oracle Retail

jobs into your workload

automation environment

– Oracle Retail Merchandising

System (RMS)

– Oracle Retail Price Management

(RPM)

– Support release 13.x and higher


Leading Workload Automation Solution in the market

• History of innovationand investment

• On-going customers engagement

• Record breaking CXI

Why Control-M 9?

• Unmatched applicationworkflow automation

• Lowering TCO

• Increasing applicationdeployment speed

Key Elements

• Automated agent and client deployment

• Runtime analytics

• High Availability

• Automated application workflow

promotion between environments

Release Highlights

Control-M 9

Executive Summary


Our commitment…

remaining focused on customers.


Bring IT to Life.

Hvala na pažnji!

TM

Control-M…As an Application Management Platform

Documents

Transcript of Control-M…As an Application Management Platform