Post on 09-May-2015
description
NGTC Conference (c)
Service Virtualiza.on: Reality is Overrated.
a CA LISA Service Virtualiza.on Best Prac.ce: Transform Performance and Scale
Min Fang min.fang@ca.com+65 9636 8356
Agenda
• Constraints: The Enemy of Agility • What is Service Virtualiza.on • Capabili.es of Service Virtualiza.on • Best Prac.ce: Transform Performance & Scale
Next Genera.on Tes.ng Conference (c)
Accelera.ng Applica.on Time-‐to-‐Market HOW DO WE INCREASE “TOUCH TIME” AND REDUCE “IDLE TIME”?
DEVELOPMENT REGRESSION QA INTEGRATION
PERFORMANCE UAT PRODUCTION
§ Constraints of dependent systems needed to move the SDLC forward
§ Increasing Complexity of today’s heterogeneous enterprise IT architectures
§ Lack of CollaboraFon creates scheduling conflicts and buffers from development through opera.ons
§ Need Complete visibility into what happens in produc.on
CHALLENGES
Bridging the Gaps in DevOps
RUN BUILD PLAN
DevOps brings the applica.on porbolio forward to meet market challenges with collabora.on and efficiency from design to deployment
ENVIRONMENT MANAGEMENT
CONTINUOUS FEEDBACK
AGILE DEVELOP & TEST
SERVICE VIRTUALIZATION
RELEASE AUTOMATION COLLABORATIVE
PORTFOLIO PLANNING OPERATIONS MONITORING
DATA MINING
NGTC Conference (c)
Constraints: The Enemy of Agility
1. Insufficient environment/capacity for tes.ng and development
Copyright © 2010 CA. All rights reserved.
SOAP
AppServer
UAT Tester
100% u.liza.on
MainFrame Performance Tester
Developer
2. Uncontrollable environment growth
Copyright © 2010 CA. All rights reserved.
SOAP
UAT Tester AppServer1
MainFrame AppServer2
AppServer3
Dev Team1
SOAP
SOAP
Training
Keep adding MIPS to keep pace with new requirements
SYSTEM UNDER TEST
3. System Dependencies = Development & tes.ng constraint
MainFrame Frontend Tester/ Developer
1. Upgrading MQ version 2. Timesharing
Mainframe
ESB 3. ESB Team will only complete development in 2
weeks .me
NGTC Conference (c)
What is Service Virtualiza.on
Service Virtualiza.on: How does it work?
R Structured Conversations R Observe R Understand R Protocol-Level
§ Recorded traffic
§ Design specs
§ Sample RR pairs
§ Transaction logs
§ Manual Creation
§ Byte-Code
R Heuristics R Analytics R Algorithms R State
R Sophisticated Behavior R Dynamic Properties (Dates,
values, etc.) R Scenario Support R Test Data Sanitation R Compiled Model vs. Stubs R Automatic Healing
CAPTURE PROCESS MODEL
JDBC
MQ
Source ApplicaFon
ERP
Database
Mainframe
Legacy
SOAP
§ Automa.cally capture realis.c behavior § Op.mize models as desired
Capture
Create Virtual Services
Captures and Models Realis.c Virtual Services
LISA Virtualizes en.re test environments to eliminate costly constraints
W W W . I T K O . C O M
Virtual Test Environments
JDBC MQ
VS1 DB1
VS2’
IT resources now available on your terms,
accelera.ng .me-‐to-‐market
and team produc.vity, allowing 24/7
tes.ng
SOAP/HTTP Source ApplicaFon
ERP
Database
Mainframe
Legacy
§ Mainframes & Mainframe Components
§ Risk Solu.on Providers
§ Credit Bureau
§ SMS Gateway Providers
§ Teleconference Service Provider
§ Card Processing Services
§ Bill Payment Services
§ Travel System (GDS)
CICS-‐CICS
§ Plaborms
§ SOA Protocols
MQ DB2
SOAP JMS JDBC HTTP JAVA MQ
UDDI XML
CORBA REST .NET
CA LISA Virtual Service Environments Customers can virtualize almost any IT asset
NGTC Conference (c)
How Can Service Virtualiza.on Help?
Virtual Service
Model
Design arFfacts, transacFon logs,
WSDL files, flat files, and others
Source App
1. Model Applica.on Under Development
Virtual service can be modeled from design ar.facts. Allowing development and integra.on test to start earlier
2. Using SV for training environment (Use Client without Backend)
AppServer MainFrame
AppServer
Virtualize
Virtual Service Environment
3. Virtual Data Management -‐ For Development & Tes.ng
Copyright © 2010 CA. All rights reserved.
Applica.on Server
Abu Bakar IC: 1234 Acc: M123
Alvin Koh IC: 4567 Acc: 456
Arun Kailasam IC: 54354 Acc:D4345
You have full control over data in virtual service, you can add your data directly into the virtual service for the purpose of: a) Scenario-‐based tes.ng b) Nega.ve tes.ng
3. Virtual Data Management -‐ Create or Modify virtual data directly
1. If AccountID= =1313094687566198 2. Return user
details of JACKY LEE (exis.ng customer)
3. Nega.ve Tes.ng: How do my applica.on handle -‐10k as account
balance
100 tps (Virtual
Mainframe)
4. Using SV for Performance Engineering
SOAP
AppServer
1. Interface load tes.ng 2. Load tes.ng
without u.lizing mainframe MIPS
Test
Test
Test
Virtualize
3. At the same .me, understand the txn/sec requirement and to
size your mainframe more accurately
1000 VU (Interface)
Aligning to CIO Goals Do More with Less
0
0.5
1
1.5
2
2.5
3
3.5
4
Year 0 Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7
Normal
Number of Test Environments Number of Training Environments Complexity in the environments
Halt growth by using LISA
3Y TCO: Reduce overall future footprint by 50% or more
5Y TCO: Reduce overall future footprint by 70% or more
CIO Goal accomplished! CONTROL costs
#1 consumer retail bank of America
eliminated $100M in hardware
requirements when using LISA over 5
years. Over 10x ROI, 6x in year 1
NGTC Conference (c)
Best Prac.ce: Transform Performance & Scale
In the .me it takes to blink an eye—400 milliseconds to be exact, online AND mobile users will have begun to abandon your applica.on
When customers “struggle” with an app…
You Your compeFtor
Abandon the transac.on, try later on computer
43%
Become more likely to prefer a compe.tor
Log a complaint with customer service
Go directly to a compe.tor
16% 14% 12%
(Harris Interac.ve Mobile Transac.ons Survey for TeaLeaf, March 2011)
Performance will make or break an applica.on!.
When problems occur:
• Revenue is lost.
• Produc.vity decreases.
• Customers leave.
• The longer it takes to find the defect, the more it Costs to fix.
Development Phase Cost of Fixing RaFo
Requirements 1
Design 3 -‐ 6x
Coding 10x
Development Tes.ng 15 -‐ 40x
Acceptance Tes.ng 30 -‐ 70x
Opera.on/Produc.on 40 -‐ 1000
80% of the TCO of an applica.on is directly related to finding and fixing problems. (Source: “The Economic Impacts of Inadequate Infrastructure for So<ware Tes>ng, NIST)
The result: on average, a performance issue costs an organiza.on at least $45,000/hour. (Source: Mul>ple industry analysts have published es>mates ranging from EMA’s $45,000/hr cost to Yankee Group’s and Meta’s es>mate of over $90,000/hr or more depending on the industry and nature of the failing applica>on.)
Service Virtualiza.on: Where would I use this?
Service Virtualiza.on
Eliminate Capacity Constraints Major North American Bank Performance & Test CoE
Business Problem
• 13 performance, dev, test teams forced to .meshare on mainframe-‐based services
• Cost of addi.onal environments for L&P lab and the movement to Agile development/tes.ng
• Difficult to perform load tes.ng due to system dependencies
• Only 2 of 5 major releases were performance tested • Offshore teams unable to complete off hours
performance tes.ng due to availability issues
Business Impact
• Virtualize in 8 days what took 2 years of custom-‐coded stub development to create.
• Virtualize new systems in 1 hour with business analysts versus weeks of custom coding
• Virtualized environments enabled all 5 major releases plus 5 minor releases to include L&P tes.ng
• Over $30 million in hardware cost avoidance • Offshore teams performing 300% more L&P tes.ng due to
system availability
Or similar heavy-‐weight implementa.on environment
Shared Mainframe Service
VSE
VSE
VSE
Before A{er
Avoid costly transacFon fees when tesFng third-‐party services Leading Global Airline
Challenge § Test and valida.on of reserva.on booking system was incurring $1 to 2
million in monthly fees to access live third-‐party services
SoluFon using LISA Virtualize § LISA simulated the behavior of third-‐party reserva.on systems as virtual
services that were available 24/7 without usage fees § No longer needed to coordinate test schedules or mock transac.on data
sets with third-‐party service providers § Provided freedom for development and QA teams to test earlier in the
so{ware lifecycle
Client Results § Reduced service usage fees to save over $12M annually § Lower tes?ng costs by using stable, virtualized data sets § Higher quality by encouraging earlier and more frequent tes.ng
“We rely heavily on LISA to automate the tes?ng, configura?on and virtualiza?on aspects of our test and development environment.” -‐ Lead Architect
Customer Success Story
Example: Performance Tes.ng Middle-‐Tiers Using LISA’s Virtual Service Environments (VSEs)
Frequent Flyer System
Booking business service
PNR Service
Ticke.ng Applica.on
Integra.on Bus (ESB)
SOAP/HTTP(S SOAP/JMS
XML/MQ
XML/MQ
LISA Test
LISA Test
UI Load Test Tools
USERS: Agents / Adjusters
Reserva.on Service
GDS ROI LISA
Test
Isolate, test and tune components by virtualizing dependent services
19.0s Conven.onal load tes.ng can
only expose overall performance .me
Slow responding transac.on or component
pinpointed with LISA
LISA VSE
LISA VSE
LISA VSE
3.3s
2.5s
2.5s
10.7s
LISA manages Test data
Enabling High Performance with Constrained Resources & Time Leading P&C Insurance Fund
Challenge § Enterprise wide “go-‐live” of cri.cal quo.ng system across 50 states § Performance Tes.ng lags could not be pinpointed with UI tes.ng § Middle-‐.er and 3rd party credit systems too cri.cal/costly to test.
SoluFon using LISA Virtualize & Test § Simulated Web, IBM MQ/ESB, Cloud/3rd party Credit service, and Oracle
apps with Virtual Services for 24/7 inexpensive tes.ng § Drive 50K user+ load tests directly against isolated components § “Prove It” quick ROI project with iTKO engineers and SI partner
Client Results § Went from test overruns to unlimited Virtualized performance test access,
and star.ng next phase 4 months early § Delivered ROI within one month with skilled resources § Improved test bed availability by 64% and increased number of tests run by 10x
“When you can do isola?on at a component level against the performance budget, you can iden?fy the weak link in the chain and address it..” -‐ Sr. Director, IT Architecture & Performance
Customer Success Story
Best Prac.ce: Decompose Service Levels Increasing efficiency by building up to meet a “Performance Budget”
Order"
Lookup"
Price"Solution"
Test: "4.0"
Seconds"
Without component-‐level performance tesFng:
Isolated component-‐level performance tesFng miFgates risk:
Timeline 8 mo. 9 mo. PROJECT DUE!
Try adding more test Hardware!!"
Test"3.2
Seconds"
"SLA
2.1 seconds"
Order"
Lookup"
Price"
Solution"
0.5s" 0.4s"
0.5s"
1.2s"
"Test:"
2.1 Seconds"
0.5s"
1.2s"1.9s"
0.4s"
Use VS to isolate & tune components separately from dependencies!
Still not within SLA!!$!$!
Aggregate solution tuned for performance budgets!
Produc.on vs. Test: in produc.on
online banking shared infrastructure
datacenter – Kuala Lumpur
online banking shared infrastructure datacenter – Singapore
mobile banking shared infrastructure datacenter – Singapore
4G SingTel Singapore
3G Celcom Kaula Lumpur
2.5G Telkomsel Jakarta
Mobile TransacFons Singapore
Kaula Lumpur
Jakarta
Internet TransacFons
web service
web service transac.ons
online banking shared infrastructure datacenter – Singapore Simulated TransacFons
Simulated web services
online banking shared infrastructure datacenter – Singapore
mobile banking shared infrastructure datacenter – Singapore
4G SingTel Singapore
3G Celcom Kaula Lumpur
2.5G Telkomsel Jakarta
TransacFons with real world network condiFons
Singapore
Kaula Lumpur
Jakarta
virtual locaFons
Load Generator
Load Generator
Load Generator
without network virtualizaFon
with network virtualizaFon
ProducFon vs. Test: in test
RealisFc Mobile/Web/Internet Network
CondiFons
Xyz Bank iPad Edi.on Mobile App Summary Analysis
Time KBs Data
0
10
20
30
40
50
60
70
80
7.1
20.7 17.2 12.3 11.7 8.2
14.6
46
32.8
20.5 22.4 19.1 21.3
72.2
44.6
27 33.2 31.9
WiFi
3G
3G Busy
Performance Op.miza.on -‐ once bo}lenecks iden.fied than solu.on is to op.mize the applica.on.
References
Source : CA Press Release | ServiceVirtualiza.on.com
§ Service Virtualiza.on Site Home Page :
h}p://servicevirtualiza.on.com/profiles/blog/list?tag=best+prac.ces § The Book -‐ Service Virtualiza.on: Reality is Overrated.
h}p://www.amazon.com/Service-‐Virtualiza.on-‐Reality-‐Is-‐Overrated-‐ebook/dp/B009HAKER8/ref=dp_kinw_strp_1
§ Ge�ng Started with Service Virtualiza.on h}p://servicevirtualiza.on.com/video/ge�ng-‐started-‐with-‐service-‐virtualiza.on-‐sv-‐minute-‐2012
§ LISA Introduc.on – What Does LISA Do ? h}p://www.youtube.com/watch?v=2Nl69c3MLkQ
MIN FANG MIN.FANG@CA.COM
+65 9636 8356
NGTC Conference (c)
Thank You!