EMA - Measuring the User Experience in the Cloud

Measuring User Experience in the Cloud: Which Tactics are Worth the Time?

Julie Craig

Research Director, ApplicationsEnterprise Management Associates

5777 Central Avenue, Suite 105

Boulder, CO 80301

www.enterprisemanagement.com

Agenda

• Cloud Snapshot• Cloud Characteristics Create New Management Challenges• End User Experience Monitoring (EUE) and the Cloud• Summary and Closing

Slide 2

Which needs are effectively being addressed by cloud computing solutions for your organization?

Expand revenue channels by reselling Cloud-based services

Business model enhancements

Free up resources for strategic projects instead of routine op-erations/firefighting

Improve service resilience/ more consistent service per-formance

Enable disaster recovery/business continuity

Reduce complexity of IT management

Faster time to create new services

Increase infrastructure flexibility and agility (data center consol-idation, etc.)

Reduce the operational costs of IT management (e.g. staff, power, rent, maintenance, etc.)

Faster time to deploy existing services

Reduce the capital costs of IT management (e.g. hardware, facil-ities, licenses, etc.)

0% 10% 20% 30% 40% 50% 60%

25%

30%

39%

43%

45%

47%

48%

51%

53%

54%

55%

CapEx Reductions, Fast Deployment, OpEx Reductions are Top Three Reasons for Cloud Adoption

3

What cloud computing service model(s) has your organization deployed, or is planning to deploy?

Platform as a Service (PaaS) – an environment for application development and deployment is delivered by the (internal/ex-

ternal) cloud provider

Infrastructure as a Service (IaaS) – only basic system or machine-level resources are delivered by the (internal/ex-

ternal) cloud provider

Software as a Service (SaaS) – a fully-functional pre-built ap-plication environment is delivered by the (internal/external)

cloud provider

0% 20% 40% 60% 80% 100%

39%

39%

77%

SaaS Leads Public Cloud Deployments

4

31%

35%

75%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Public cloud – openlyavailable to any

organization or thegeneral public

Community cloud –shared by a few

selectedorganizations withcommon interests

Private cloud –operated solely for

your ownorganization

What cloud computing deployment model(s) has your organization deployed, or is planning to deploy?

5

Private Cloud Leads Public Cloud

Agenda


Slide 6

Which does your organization measure in order to quantify the success of cloud-related initiatives?

We don’t quantify the success of cloud-related services

Quantity of new application services

User productivity-specific metrics

Revenue-generation through cloud enabled services

Responsiveness of new application services to user requests

Business-productivity metrics (other than revenue)

Cost savings opex

Cost savings capex

Improved service quality (performance, availability)

0% 20% 40% 60% 80%

10%

29%

31%

33%

36%

39%

45%

50%

57%

Measuring Success

7

Rank Type of Tool

1 Infrastructure performance & availability monitoring

2 End-to-end application availability

3 End-to-end application response time

4 Service Desk/Help Desk Incident/problem management

5 Business impact management

6 Service Level Agreement (SLA) Reporting

7 Transaction aware User Experience Management/ Monitoring

8

Ranked Importance of Current Monitoring/ Management Tools

Please rate the importance of the following management technologies.

Which technologies does your org currently have deployed/full commitment to deploy in your environment?

BSM or service management portal

Application dependency mapping

Usage-based accounting and/or chargeback

End User Experience (EUE)

Service catalog

Run-book or IT Process Automation

CMDB/CMS

Integrated service dashboard

Business Intelligence or BAM

Unified Service Desk

Data warehouse IT analytics

Service level management

0% 10% 20% 30% 40% 50% 60%

23%

23%

25%

28%

29%

30%

33%

35%

41%

45%

46%

52%

End User Experience Management Under-represented

9

Agenda


Slide 10

“End user Experience”:Catch All Term for Multiple Types of Products

• Network tracing or capture• Deep visibility into network flows and packets

• Active or synthetic transaction monitoring• Script-based tests that run against an application at regular intervals.

• Passive or observed transaction monitoring• “Hard” or “soft” appliances scan protocol-based traffic such as web interactions

• End to end transaction reconstruction• Consolidate metrics from across the data center to deliver a reconciled end to

end view of transaction execution

• Client-side EUE• Monitor performance and availability from the perspective of the user workstation

Slide 11

Agentless, Client-Side EUE

• Deep endpoint monitoring for both “internal” and “external” users • Instrumentation at the endpoint via the browser

• “Watch” activity entering, exiting, and on the workstation or device• Diagnose application-related issues, identify delay sources

• Instrumentation for the Cloud• Monitoring can be embedded in VMs• “Cloud Edition” can be installed in Amazon EC2 instances• Central repository and collector monitors EC2 environments

• Instrumentation for Mobile

Slide 12

Correlsense CEO, Oren Elias

Slide 13 End-User-Monitoring.com

The Big Question: How do we Manage Today’s Ever Changing Environments?

Slide 14

Public

Private SOA

Virtualization

Agile

Lots of Moving Parts

End-User-Monitoring.com

With all of the Complexity and Moving Parts…

Slide 15

What Does Not Change?

Public

PrivateVirtualization SOAAgile


The Two Axioms

Slide 16

1. Transactions Drive the Business

2. User’s Expectation of Performance


Public, Private, Hybrid Datacenter

WEB

MQ/ESB

DCOM

CORBA

SERVER

APP SVR

ElasticLB

LDAP

WebServices

‘Under The Hood’


Cloud ≠ ‘Cure All’


Four New Challenges in the Cloud

“Old world” monitoring paradigms no longer fit


Challenge #1 – Dynamic Environments

Highly dynamic environment• Out: Auto-discovery• In: Auto-detection


Challenge #2 - Highly Virtualized Infrastructure

Infrastructure is highly virtualized• Out: Focus solely on Infrastructure and

resource monitoring• In: Focus on end-user and applications


More Demanding Applications

Customer facing applications are more demanding• Out: Focus solely on Availability• In: Focus on performance and service levels


Multi-Tenant Environment

Shared environment• Out: Hardware appliances• In: Software-based solutions


Top 3 Tips For Cloud Application Management

You Can’t Truly Understand Your Web Performance Without Monitoring the Experience of your Real Users

Here’s What You Need to Do:

1) Be Paranoid

• Watch Everything

2) Be Smart

• Watch Before and After

3) Be Lazy

• Use Automation to Make your Life Easier

Datacenter

WEB

MQ/ESB

DCOM

CORBA

SERVER

APP SVR

PROXY

LDAP

WebServices

1) Be ParanoidWatch Every Transaction from Every User


Total End-User Response TimeBrowser Rendering Time

Get the Key User-Experience Metrics

Ren

deri

ng

Real User Monitoring

Datacenter TimeNetwork Latency

Look Across Space

Houston, We Have A Problem

2) Be Smart.From Transactions to Performance Models

End-User Response

Time Breakdown

Transactions Modeled with Transaction

Profile

Compare Between Application Models

Slide 30

Before vs. After

Within SLA vs. Exceeded SLA


Insurance in Your Changing WorldAgile Roll-Outs, Virtual Migrations, Private Cloud Migrations

• Compare “new” and “old” production behavior• Prove response times are better or improved• Realize causes of degradations to avoid rollback


3) AutomateEnforce SLAs on End-User Experience

Application End-User SLAS

Monitor Service Levels

Detect Degradation

Automated Action

Auto-Scale Out

Measure Impact

Alert and Automate


Get Started with SharePath RUM Express for Free

http://www.real-user-monitoring.com


Agenda


Slide 36

Summary

• Cloud computing is a “hot topic” with a variety of potential benefits

• Cloud also brings its own set of challenges • How do you measure success of Cloud adoption?• How do you monitor performance and availability?• How do you get visibility to End User performance?• Which types of EUE tools are most useful for Cloud monitoring?

• Each type of product has its strengths

• Stretch your tools budget by making sure to match tools acquisitions with desired monitoring metrics

Slide 37

For more information on Enterprise Management Associates, please go to www.enterprisemanagement.com, or call +1 303-543-9500

Thank you for attendingtoday’s presentation

http://www.enterprisemanagement.com/

EMA - Measuring the User Experience in the Cloud

Technology

Transcript of EMA - Measuring the User Experience in the Cloud