Stress Testing for the Digital Economy: Are YOU Ready to Deliver High Performance APIs, Streaming,...

IT & DATA MANAGEMENT RESEARCH,

INDUSTRY ANALYSIS & CONSULTING

Sven HammarCEO

Apica

Twitter: @svenhammar

Stress Testing for the

Digital Economy

Julie Craig

Research Director

Enterprise Management Associates

Twitter: @julie_craig

Are YOU Ready to Deliver High Performance APIs, Streaming, and Mobile Applications



Today’s Speakers

2 © 2016, Enterprise Management Associates

Julie Craig – Research Director Julie has over 20 years of experience in software engineering, IT

infrastructure engineering, and enterprise management. At EMA, Julie’s

focus areas include DevOps, application performance and availability

management, Application Discovery and Dependency Mapping, software

development, and software as a service.

Sven Hammar – CEOSven founded Apica in 2005. As CEO, he has led the company from concept

through development to becoming a premier provider of performance testing

and monitoring. Prior to founding Apica, he was CEO & co-founder of Celo

Communications. He also served as VP of Marketing at Gemplus (GEMP),

leading the market launch of Gemplus security solutions/products for banking

and PC-OEM with major players like RSA, VISA, MasterCard, ACI,

Panasonic, and Target.



Logistics for Today’s Webinar


An archived version of the event recording will be

available at www.enterprisemanagement.com

• Log questions in the Q&A panel located on the lower

right corner of your screen

• Questions will be addressed during the Q&A session

of the event

> QUESTIONS

> EVENT RECORDING

http://www.enterprisemanagement.com/



Stress Testing for the

Digital Economy

Julie Craig

Research Director

Enterprise Management Associates

Twitter: @julie_craig

Are YOU Ready to Deliver High Performance APIs, Streaming, and Mobile Applications



Overview

• A new technology landscape

• New technologies and use cases

• Continuous Delivery Supporting Digital Transformation

• Automating for Digital Transformation

• Wrap-up




A New Technology Landscape



“Incorporating new technologies” IT’s #1 challenge

7 © 2016 Enterprise Management Associates, Inc.

40%

36%

36%

34%

26%

24%

23%

21%

21%

18%

18%

Incorporating new technologies such as cloud and mobile into the technology…

Acquiring personnel with the right skills

High cost of administration ties up too much budget

Business alignment

Lack of application-focused versus silo-focused monitoring/mgmt tools

Internet of Things (IoT) considerations such as sensor networks, etc.

Difficulty meeting demands of the business in a timely manner

Poor application performance and/or availability

Lack of organizational cloud strategy

"Shadow IT"

Lack of executive vision or leadership

What are the top three (3) overall concerns/challenges affecting your IT department?



We are in the midst of an explosion of innovation….




Yet IT teams are still struggling with the “same old, same

old” in terms of performance & availability mgt.


“Supporting these applications requires skills which we don’t have.”

“We are unable to pinpoint sources and causes of problems.”

“Our applications are unstable.”

“We can’t see transactions or data flow between tiers”

“We have too many applications running on different devices and no way to monitor them all.”

“We want to become more proactive in detecting and fixing problems – but we can’t.”

“We are seeing performance issues with our web applications – such as slow loading times.”

“We had ten people on a 32 hour conference call.”

“We can’t get less than 30 second response time.”



Technology innovation is driving growth of a new breed

of massively interconnected, complex applications


Virtualization

IoT SaaS

PaaS

IaaS

APIs/Int

Mobile

Big Data Streaming

Agile

Docker

Continuous

Delivery



New Technologies and Use Cases



Mobile applications: high value, high risk

• Volume of mobile Business to Consumer (B2C)

interactions increases each year

% of online shoppers using mobile devices

continues to rise

• High risk of poor performance

User experience varies depending on device type,

network connection type/speed, & performance of

back-end systems

• Users perceive poor performance as the fault of

the app provider


#3 Technology-

related initiative

supporting

“Digital

Transformation”



APIs: Critical to doing business in an interconnected

marketplace


• APIs are the “de facto standard” for integrating

within & across organizational boundaries

• Essential to delivery of container-based

microservices

• Both consumer and provider APIs essential to

revenue growth

• Each integration point creates a performance

risk

• API consumption means being dependent on

third-party services for performance/availability

90% say providing

APIs TO partners, etc.

“critical” or “important” to

revenue generation

98% say consuming

APIs FROM partners, etc.

“critical” or “important” to

revenue generation



Streaming: A whole different animal

• Endpoint devices (i.e. set-top boxes, game

stations, etc.) each have specific Operating

System and hardware implementations

• “Players” for streamed content are different

for an iPhone, a PlayStation (PS4™), or a set-

top box

• Hi bandwidth, latency sensitive

• Network dependent

• Connection speed varies depending on

device, location, and service provider


50% report

“significant

amounts” of

streaming traffic on

their organization's

network in the

course of an

average day



20%

21%

21%

24%

25%

27%

29%

32%

32%

35%

38%

45%

51%

Omnichannel (mobile, mobile web, web) for internal non-IT stakeholders

Omnichannel (mobile, mobile web, web) for external business consumers

Omnichannel (mobile, mobile web, web) for IT stakeholders

Social IT in support of IT stakeholders

Social IT in support of IT to IT consumer dialog

New or enhanced CMDB/CMS/application dependency mapping

Social IT in support of business/company consumers

Cross-domain IT asset management and optimization

Improved third-party service oversight/management

Enhanced automation for self-service

New or enhanced project management capabilities

Improved application performance management

Improved user/customer experience management

Which four of the following functional capabilities are (will be) most important for your organization’s digital/IT transformation initiative in the coming 12 months?

Improved user/customer experience, APM are top

“functional” priorities for Digital Transformation



Continuous Delivery Supporting

Digital Transformation



Top 4 drivers for Continuous Delivery relate to business-

and customer- focused requirements

© 2016 Enterprise Management Associates, Inc.

42%

40%

39%

34%

33%

31%

30%

29%

22%

Business demands for new products or services

Customer demands for better application quality

Customer demands for new products or services

Business demands to better compete against competitors

Agile development practices

CIO-driven requirements for increased productivity

Executive-driven desire to rapidly implement new ideas

Regulatory or compliance requirements/changes

Industry vertical specific changes

Which of the following are the top drivers for the Continuous Delivery initiative at your company?



Accelerating delivery of IT services:

Good for business, Hard on IT…


52%

48%

45%

36%

31%

Operations is spending more time onproduction troubleshooting

Development is drawn into productiontroubleshooting more often

Service levels have degraded as ratesof change increased

We are experiencing a higher numberof performance and/or availability

problems

Minimal production impact

How has this increase in frequency of code delivery impacted production environments?

49%

44%

41%

35%

33%

Higher levels of customer satisfaction

Faster revenue growth

Minimal business impact

We are more agile in competing withinour industry

Less negative fallout on social media

How has this increase in frequency of code delivery impacted the business as a whole?



“Manual troubleshooting” of production environments top

bottleneck slowing Continuous Delivery


44%

37%

37%

37%

36%

34%

27%

24%

23%

Manual troubleshooting processes dealing with problems arising from…

Time spent testing code

Manual processes supporting package builds and production deployment

Fear/risk related to adverse production impacts of new code

Time spent creating code

Manual processes supporting code movement through lifecycle stages

Lack of testing/QA personnel

Lack of testing facilities, developers and/or QA must "wait their turn" to test

Lack of workflow/automation tools to drive the Continuous Delivery pipeline

Which of the following factors are the biggest "bottlenecks" slowing down your Continuous Delivery pipeline?



Automating for

Digital Transformation



24%

25%

27%

29%

32%

32%

35%

38%

45%

51%

Social IT in support of IT stakeholders

Social IT in support of IT to IT consumer dialog

New or enhanced CMDB/CMS/application dependency mapping

Social IT in support of business/company consumers

Cross-domain IT asset management and optimization

Improved third-party service oversight/management

Enhanced automation for self-service

New or enhanced project management capabilities

Improved application performance management

Improved user/customer experience management

Which four of the following functional capabilities are (will be) most important for your organization’s digital/IT transformation initiative in the coming 12 months?

Top priorities for functional Digital Transformation:

Improved user experience, app performance




Top priorities for organizational Digital Transformation:

Delivering new services faster and at higher quality


42%

42%

39%

38%

36%

36%

35%

33%

Compressed delivery time frames for IT services

Improvements in quality of the services delivered to IT and business

Improvements in IT-business alignment

Organizational shifts in IT focus from silo and technology-based to customer-and business-aligned

Cost reduction associated with delivery of IT services

Agility in terms of "time to innovation", i.e. time required to transform ideas intofull-blown organizational processes

Customer satisfaction (external customers)

Customer satisfaction (internal customers)

From the organizational perspective, which of the following are the top focus areas for your Digital Transformation initiatives?



Wrap-up



Wrap up

• Vast majority of companies report they are actively engaged in

Digital Transformation.

• Influx of new technologies is #1 challenge IT organizations are

facing

• New development practices– Agile, Continuous Delivery– are

impacting IT and business with high rates of change

• Automation focused on performance/availability testing across the

lifecycle is the answer


ApicaA Performance-as-a-Service CompanyStress Testing for the Digital Economy

2016

2626 www.apicasystems.com

Agenda• Apica Overview

• Apica LoadTest Suite

• Stress & Loadtesting 1-1

• Customer Cases

• Q&A


Apica Overview• Founded in 2005

• Sweden, US and the UK

• Leading global platform, network & tools for performance testing and monitoring

• 550+ global customers

• We test and monitor web and mobile applications


Apica’s Product Offering

Max

Synthetic Monitoring LoadTesting Test Tool & Experts 1 2 3

OK

Unified Performance Monitoring & Testing Platform

One GUI - One Platform


200+ Monitoring

Locations

50+ LoadTest

Clusters


Elite - social platform

Capacity planing

Stress Test

Test Automation

Device– iPhone

– Android App

– Web


Objectives We Help Address

• Site launch

• Stress test / Max Load

• Monitoring scripts

• Baseline test

• Test Automation

Test TypesTechnology

covered

• Mobile

• Streaming

• Web applications

• API / Authentication

• DDoS / SDoS

Capacity Planing

• Cloud, on-premise, hybrid

• Infrastructure plan test

• Mega Test

• DDoS Test

• Fail over and back up test


Apica LoadTest Suite

• Apica Zebratester Tool

• Apica SaaS Portal

• Cloud Capacity

• Team

• Test Automation

• Software

• SaaS & On Premise


How It Works

Record Usage

ScenariosScript Functions Generate Load APM & Client Side Analyze Report

Fast, via GUI,

Any Device

Dynamic Data

& Device

Scale in Apica

Portal & Network

Compile and

compare result files Status &

Re-runs


Project result

Load - Normal and Max

– How many users ,

can we handle?

Behavior in ”Danger Zone”

– become unstable

above maximum load?

Problem Analysis ?

– Where are the bottlenecks?

– Fix them via A/B test?


Mega Test Ready


Plan a head

Know your numbers !

• Traffic

• Behavior

• Devices

Integrations

Test plan

• Baseline (90%)

• Mega (10%)

High Volume Testing


Record real user scenarios– User interaction

– API endpoint vs App Usage

Any device Iphone, Android, Apple TV

Match Speed & Location

Combine with Apica Monitoring

Testing mobile applications


Test Automation

POST Action

Execute Apica Loadtest

• Continuous Delivery service.

• Customize release process model with API

• Apica’ test platform is full API driven and integrate direct with AWS

CodePipeline, Jenkins and Team City


Continuous Deployment


DDOS – Validation Test

• High Traffic…

• Network & Layer 7 Attacs – Simple URL

– Full Scripts

– Slowloris ...

– Network only modules

• Risk & Function & Protection


Launch with Confidence

Set Your Targets-> Use Metrics

Validate Baseline -> Design

Max Test -> Go / NoGo

Set a Process ->

Test Automation

1

2

3

4


Thank you - Questions?

[email protected]

Peter.Tollgå[email protected]

mailto:[email protected]

mailto:Peter.Tollgå[email protected]

Stress Testing for the Digital Economy: Are YOU Ready to Deliver High Performance APIs, Streaming,...

Technology

Transcript of Stress Testing for the Digital Economy: Are YOU Ready to Deliver High Performance APIs, Streaming,...