HPE | Network Virtualization | POC

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

HP Network Virtualization (NV) Enablement – How to POCMatthew Brady – EMEA NV Pre-Sales15th May 2014

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.2

POC OverviewScoping (Discovery)ObjectivesPre-RequisitesHow to Select Network ScenariosWhat to CoverHow to TestHow to ReportFollow Up

Agenda

Content Intended as Aide Memoire/Checklist- Won’t Cover All Points- Questions/Feedback Throughout!


POC Overview


Proof of Concept

“A proof of concept (POC) … is a realization of a certain method or idea to demonstrate its feasibility, or a demonstration in principle, whose purpose is to verify that some concept or theory has the potential of being used. A proof of concept is usually small and may or may not be complete”

- Wikipedia

What is a POC?• Not a Trial (Self-Guided)• Not a Pilot (Actual Project Deliverables)• Limited in Scope and Length

• Low number of vusers • 2-3 days total time• Single Application• Limited Journeys

• Usually AssistedGive-Get?• Sponsor Required• Conditional Commitment/Access to Power• Final Results Presented to Power


POC Scoping


Overall• What Is The Name And Description Of The Application To Be Covered By The

POC?• Who Will Be Involved?• Where Will It Be Run, Using What Resources?

Architecture/Technology• What Is The Outline Architecture Of The Application (Overview Only Is

Required)?• Which Client Platforms Will Be Used (Windows/Android/Ios)? Which Client

Types Should Be Included In The POC Test Activities?• How Will It Be Delivered To Users (Web Browser, Client Application, Mobile

App Etc.) And Using Which Technologies (HTML5/JS/.Net/Java Etc.)?

Scoping Questions


Geography/Connectivity• Where Will It Be Hosted When In Production?• Who Will Be The End-users? Where Will They Be Located? How Will They Be

Connected?• Is The Client Communication Encrypted (Https)?• If Mobile Users, Which Operators And Connection Types (2.5g, 3g, 3.5g, 4g)?• If Web Users, Which Connection Types (Dialup, Dsl, Cable, Satellite, Other)?

Types Of Testing/Tools• Single-user Functional And/Or Multi-user Non-functional• Manual, UFT, LR/PC, Sprinter And/Or SV• Desktop, Web And/Or On-device

Scoping Questions


Scripting/Automation• Do LR Scripts Exist For The Application Already? If So, How Many Are There?

What Protocols Do They Use? Which Ones Will Be Selected For The POC?• If Lr Scripts Do Not Exist, When Will They Be Available? Which Protocols Will

They Use?• Which Lr Version Is Currently Installed? Will The POC Use This Environment

Or Will A New Environment Be Setup For The POC? What OS Is Used For LR Controllers/Load Generators?

Objectives• What Is The Test Objective?

• 5000 Concurrent Users, 100k Transaction Per Hour, Functional Tests?• How Will Test Success Be Determined (E.g. Response Times, Server CPU

Etc.)?• What Slas Or Other Requirements Exist?

Scoping Questions


POC Objectives


NV With LR/PC• Record/Sample Remote Network Conditions• Configure And Apply Network Conditions In Tests, Including Additional

Reporting Capabilities• Analyse The Impact Of Remote Users On System Load• Analyse And Optimize The Application Flows For Performance

Success Criteria

Client Objectives – Multi User Testing

Criterion Status after POC

Install, license and configure HP NV software components CompleteDemonstrate HP Network Catcher to profile/sample mobile network conditions

Complete

Execute automated tests using HP NV for HP to apply network conditions

Complete

Demonstrate additional HP NV reporting and metrics CompleteDemonstrate using HP NV Analytics to optimize and improve performance of application flows

Complete

Provide comparison report and analyse comparative throughput of network virtualization in tests

Complete


NV For Desktop/Mobile• Install, License And Deploy Of HP NV For Mobile• Record/Sample Mobile Network Conditions For UK Consumers• Apply Network Conditions, Time Transactions And Capture Traffic Flowing From

On-device Tests• Analyse And Optimize The Application Flows For Performance Including HTTPS

Decoding• Demonstrate Using API/Command Line To Control Network Conditions For

AutomationSuccess Criteria

Client Objectives – Single User Testing

Criterion Status after POC

Install, license and configure HP NV software components CompleteDemonstrate HP Network Catcher to profile/sample mobile network conditions

Complete

Execute manual and automated on-device tests using HP NV for Mobile to apply network conditions

Complete

Demonstrate additional HP NV reporting and metrics CompleteDemonstrate using HP NV Analytics to optimize and improve performance of application flows

Complete

Provide comparison report, and analyse comparative throughput of network virtualization in tests

Complete


1. Ensure Successful POC• Keep it Simple, Well-defined Scope• Ensure S/W Pre-installed• Check & Verify Objectives, Prerequisites• Limit Time/Effort but Provide Maximum Results2. Demonstrate Maximum Value/Impact• Work Towards the Final Report – Not Client-Defined Schedule• Carefully Select Network Test Scenarios – Include Worst Cases• Opt For Poorer Network Conditions3. Progress Sales Process• Agree Success Criteria Before POC• Demonstrate Compliance to Success Criteria• Provide Detailed Report With Executive Summary• Present Results to Executive Team

HP Objectives?


POC Prerequisites


Environment• Identify Testing Servers

• Controller(s), Load-Gens• Scripting Workstation• Testing Workstation

• Check Admin Rights• Check Server Specification/OS Version etc.

Resources• When can the Application be tested? Will it remain unchanged through the

POC?• Required Additional Software (Java, Silverlight)• Internet Connectivity (for NC Global Library etc.)• Software Download(s)• Demo Licenses & Extensions

Prerequisites


LoadGen

All Protocols Except CitrixNV for LR/PC

Network NV-M

LoadGenNV-M

LoadGenNV-M

NV-M(Controller Compone

nt)

NV-M(PC Server Compone

nt)

* 1 day minimum checkout period

LM??

License

NV Analytics

NV Predictor

(+MS Office)

Controller PC Server

Internet

OR

OR

OR

EITHERNC Global DB


Desktop & Web Apps, Manual or Automated Testing (UFT, Sprinter etc.)

NV for Mobile (Desktop Mode)

Network

App Under TestNV-M

Licenses

LM?

NV Analytics

EITHER

OR

NV TM


On-Device Native, Hybrid & Web AppsNV for Mobile (Mobile Mode)

Network

Proxy

New WifiHotspot

On-Device WebInterface

Desktop WebConsole

NV-M

* 1 day minimum checkout period

NV Analytics

SSL Key


How to Select NetworkTest Scenarios


Let’s Begin with a Bit of Theory

http://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=J6LVx4NwwnSyYM&tbnid=ig6L31ZL5Z6zBM:&ved=0CAUQjRw&url=http://joan-druett.blogspot.com/2010/12/kissing-and-albert-einstein.html&ei=rsCtUYzKDYKI9gSBsoDYBA&bvm=bv.47244034,d.eWU&psig=AFQjCNEAbE3fBx-AGmmqDx72gk4jPJrzpQ&ust=1370427909092776


The Network Impact

20

TokyoLondonNew York

New YorkData Center

WAN

Consider 3 Users - 1 in NY, 1 in London And 1 in Tokyo• All Users Access the Same Application in the NY Data Center• Will They All Experience the Application in the Same Way?• If Not, Why Not?


Network Impairments

21

Conditions which inhibit the flow of data across a network• Key Impairments are:

• Network Delay (Latency)• Jitter (Delay variation, inter-packet gap variation)• Packet Loss (inverse of delivery rate)• Bandwidth Constraints• Bandwidth Utilization


Mobile Network Impairments

22

Mobile Networks Experience Extreme Conditions of the Same Net Impairments• More Network Delay (Last Mile + Backhaul, RRC latency)• More Jitter (Delay variation, inter-packet gap variation)• More Packet Loss (inverse of delivery rate)• More Bandwidth Constraints• More Bandwidth Utilization


Why is There Delay on a Terrestrial Network?

23

A Bt (msec) = d / (2/3)c

Signald

Packet PacketPacket Packet PacketBUSY

Lost Packet

Network Delay• Propagation Delay• Processing / Insertion / Serialization Delay• Queueing Delay


New York

LondonFrankfurt

Tokyo

Sydney

Signal Speed - Geography

24• Speed of Light = 299,792.458 km/s

New York London Frankfu

rt Tokyo Sydney

New York 5,585km 6,219km 10,871k

m15,998k

m

London 639km 9,585km 16,991km

Frankfurt 9,357km 16,480k

mTokyo 7,793kmNew

YorkLondon Frankfu

rtTokyo Sydney

New York 28ms 31ms 54ms 80ms

London 3ms 48ms 85msFrankfurt 47ms 82ms

Tokyo 39msT = D / (2/3)c

Distance

TheoreticalMinimumSignal Time(One Way)


25

Combination of:• Server Connect Latency (First Mile)• Back-Haul/Propagation Latency• Client Connect Latency (Last Mile)

Total Latency

First Mile

Last MilePropagation Latency


Internet Consumer (Last Mile) Latency

26

ConnectionType RTT

Fiber 17msCable 28msDSL 44ms

http://www.fcc.gov/measuring-broadband-america

Broadband Latency (FCC)• “Fiber-to-the-home services provided 17 ms round-trip latency on average,

while cable-based services averaged 28 ms, and DSL-based services averaged 44 ms.”


Mobile Latency

MobileCell Tower

Mobile Last Mile

50ms to 800msUp to 50% Packet LossBW 56Kbps to 20Mbps

OriginServer

MobileDevice


Mobile Latency – 1st Data point

28

http://stakeholders.ofcom.org.uk/binaries/research/telecoms-research/bbspeeds2010/Mobile_BB_performance.pdf

Ofcom study, UK Mobile Networks, May 2012Static Tests:• “Latency… was an average of 117 milliseconds. Nearly two thirds of tests took more than

100 milliseconds”User Panel:• “Average latency was 192 milliseconds and around a third of tests delivered latency of

higher than 200 milliseconds”


Mobile Latency – 2nd Data point

29

Cisco Cloud Index, 2010-2015“Global average fixed latency is 125ms and average mobile latency is 290ms… Global mobile consumer average latency is 307.3ms”


Latency – Summary of Averages

30


Packet Loss

31

Random Early DetectionAverage queue monitored size & packets dropped based on statistical probabilities. If buffer is almost empty, all incoming packets are accepted. As queue grows, probability of dropping incoming packet grows. When buffer is full, probability reaches 1 and all incoming packets are dropped.

Packets get dropped on an end to end link due to the following:• Network congestion causes queues to saturate• Network congestion triggers RED• Interference on wireless networks• Hardware errors• Firewall Policies

Packet loss is the inverse of delivery rate100% - packet loss rate = delivery rate


Packet Loss

32

http://pam2012.ftw.at/papers/PAM2012paper6.pdf

Packet Loss, Akamai/MIT Research• Tested 149 Million connections in 51 countries

• Compared PL for mobile networks from non-mobile networks• Most connections have no packet loss (Median PL is 0%).

• 75th percentile usually zero too• Average connection packet loss: 2.5%

• Fixed network average is 2.2%• Implies connections are usually fine, but sometimes terrible

• The average PL for the non-zero half is 5%• Suggests “small” connections have higher Packet Loss

• HTTP Connections are often small

http://pam2012.ftw.at/papers/PAM2012paper6.pdf


Jitter

33

Jitter

No Jitter

Jitter-Sensitive Applications:• Video• VoIP

Jitter = Variability over time of latency across a network• PDV – Packet Delay VariationConstant Latency = Zero Jitter


Mobile Bandwidth in Theory

34

Protocol Max Downlink Max UplinkGPRS 56-115 KbpsCDMA200 (1X) 153.6 KbpsEdge 236.8 Kbps3G Bar 2 Mbps stationary

client384 Kbps moving client

EV-DO 2.5-4.9 Mbps 0.15-1.8 MbpsEDGE Evolution 1.6 Mbps 0.5 MbpsHSDPA 2 Mbps 200 KbpsHSPA+ 14.4 Mbps4G Bar 1 Gbps stationary

client100 Mbps moving client

HSPA+ Release 11 21-672 Mbps 5.8-168 MbpsLTE 100-300 Mbps 50-75 MbpsWiMax 37-365 Mbps 17-376 Mbps


Mobile Bandwidth

35

You only get those speeds it if you’re this guy…


Mobile Bandwidth

36

Source (PCWorld): http://bit.ly/JX7MQw

PCWorld/Novarum3G• Download: 0.59 – 3.84 Mbps• Upload: 0.56 – 1.44 Mbps

4G• Download: 2.81 – 9.12 Mbps• Upload: 0.97 – 5.86 Mbps

Date: Apr, 2013

http://bit.ly/JX7MQw


Download BW Impact on Load Time

DSL

Cable FIBER

Poor Cell Conn

Avg Cell Conn

Load Time (ms)

Download BW (Kbps)


Upload BW Impact on Load Time

DSL

Cable FIBER

Poor Cell Conn Avg Cell Conn

Load Time (ms)

Upload BW (Kbps)


Latency Impact

DSL

CableFIBER

Poor Cell Conn

Avg Cell Conn

Load Time (ms)

Latency (ms)


Network Profiles – Conclusions for Performance

Upload BW doesn’t matter much until <200 kbpsDownload BW Matters at Low Values <1Mbps

• Unless Transfer Volume (Data) is LargeLatency matters the mostPacket Loss is Important but causes variation in results

• Only use in Worst Network Conditions


How to Select Network Test Scenarios


1. Determine the Full Range of Client Connection Types, for example:

• Remote Office Tokyo, London• Consumer Web UK, Germany• Consumer Mobile 2.5G India, UK, Germany• Consumer Mobile 3G UK, Germany, Japan• Consumer Mobile 4G UK, Japan

2. Select Test Range of Network Conditions (Not All):• Remote Office London• Remote Office Tokyo• Consumer Web Europe Typical • Consumer Mobile 4G UK Best • Consumer Mobile 3G Europe Typical• Consumer Mobile 2.5G India Worst

Selecting the Network Values for a POC


1. Work Out One-way Propagation Delay Server To Clients• Calculate KM Distance/200,000

2. Attempt To Validate Using Ping Tests• Or Use Other Tools – Try To Exclude The Last Mile

3. Demonstrate Mechanisms To Capture Network Conditions …• NC Server/Agents• NC App• NC Global Library• Other Tools/DataBut Do Not Use The Results In The POC

Selecting the Network Values for a POC


5. Select Test Range of Network Conditions (Not All):• Remote Office London• Remote Office Tokyo• Consumer Web Europe Typical • Consumer Mobile 4G UK Best • Consumer Mobile 3G Europe Typical• Consumer Mobile 2.5G India Worst

6. … Which Are:• Representative • Predictable• In Sequence

Selecting the Network Values for a POC (cont.)

Use:• Manual, Static Definitions• Stepped Latency & Bandwidth (Equally Spread?)• Static (Zero?) Packet Loss (Except Final Case)


Example Network Test Profiles• Baseline 0ms 100Mbps/100Mbps 0% PL• Remote Office London 20ms 10Mbps/10Mbps 0% PL• Consumer Web Europe Typical 50ms 5Mbps/2Mbps 0% PL

• Consumer Mobile 4G UK Best 80ms 10Mbps/2Mbps 0% PL• Remote Office Tokyo 100ms 10Mbps/10Mbps 0% PL• Consumer Mobile 3G Europe Typical 130ms 2Mbps/512kbps 0% PL• Consumer Mobile 2.5G India Worst 250ms 512kbps/128kbps 1% PL

Notes:• Always One Way Latency in NV• Baseline Should Have BW Defined to Ensure Stability Between Tests• Network Profiles Should be Common Between Test Types (Functional/Non-

functional)

Selecting the Network Values for a POC (cont.)

(Too Many?)


What to Cover


CapabilitiesNetwork Discovery

• (NC Server/Agents – Avoid)?• NC Global DB• NC Mobile App• Other Tools

Single User Test - Desktop• Define Network Conditions• Proxy (SSL/HTTPS)?• Transaction Manager• (Integration with UFT Etc. – Avoid?)

Single User Test - Mobile• Define Network Conditions• Proxy (SSL/HTTPS)• Web Console/Mobile Web

Multi User Test• Define Network Conditions• Runtime Metrics• LR Analysis• NV Predictor• (Packet Capture – NV Analytics –

Avoid?)• (API Transaction Marking – Avoid?)


How to Test


Check SSL/HTTPS• Use Proxy (Always Needed for Mobile)• Verify Decryption Works (Doesn’t Always)Assess Test Steps• Analyse User Journeys using NV Transaction Manager• Use NV Analytics to Focus on Network-Impacted Steps• Create Modified (Focussed) TM Steps for Network TestingExecute Tests• Baseline• Network Models

Test Process – Single User


Remember• Save .Shunra files AND TM (Timing) Exports

• Unless using Mobile Console• Good Naming Conventions!Gotchas …• Check the Adapter• Connect TM!• TM Marking (Before 1st/After Last Packet)• TM Times vs NV Analytics Times

Test Process – Single User (cont.)


Check SSL/HTTPS• If Yes – Avoid Packet Capture During Test• Use Manual Test?Check Script Timings• Script Think Time• Paced vs Real-World?• RampDesign Test Scenario• Add/Define Groups to Align with NV Virtual LocationsExecution Sequence:1. Full Test w/o NV2. Low-Volume Test with NV3. Full Test with NV

Test Process – Multi User (LR/PC)

Will Network Make Any Difference to Capacity?• 0.1s Baseline Execution Time – 25s Think Time• 0.2s Baseline Execution Time – 300s Pacing


Remember• Save Scenarios• Save Results & Analysis Files• Document Network Profiles (Export?)Gotchas• “Local User” – Required For LR/PC Analysis• Blank Virtual Location – For NV Predictor• BW per Virtual Location vs Per Vuser• Packet Capture• Transaction Marking

Test Process – Multi User (LR/PC – cont.)

HPE | Network Virtualization | POC

Software

Transcript of HPE | Network Virtualization | POC