Identify and Fix Performance Problemswith Visual Studio Ultimate

Benjamin DayBenjamin Day Consulting, Inc.www.benday.com | blog.benday.com

DEV302

Benjamin Day

Consultant, Coach, TrainerProfessional Scrum Development Trainer

http://scrum.org Unit testing enthusiastMicrosoft MVP for Visual Studio ALMSilverlight, Windows Azure, C#,Team Foundation Serverhttp://blog.benday.combenday@benday.com

Thanks.

To my content reviewersJerri Chiu, MicrosoftDennis Bass, Microsoft

The Agenda

Basics of Web Performance TestsLoad TestsLoad Test RigsPerformance Explorer

Existing application Identify performance problemsGo from bad performance to goodASP.NET & WCF

Disclaimer: I’m going to use the term “load testing” imprecisely…a lot.

Here we go.

The Purpose of Load Testing

Identify the capabilities of your applicationPossible goals

ExploreVerify Find the limitsCrush

The Obligatory “Eat Your Vegetables”

Load test throughout the development cycle.Catch problems earlyEstablish performance baselineWatch for performance trendsCheaper to fix earlyPlan for performance

Best Practice

Cheaper To Fix Early

Early Less Early Nearly Done Production0

10

20

30

40

50

60

Time

Cos

t to

fix

Dose Of Reality

You know you should work that way.You probably don’t.This talk is for you.You have problems.How do you figure out what’s going wrong?

Inspiration for this talk

Real customer engagementSpent years developing a web applicationReleased it Seriously grumpy customersHuge hardware~20 simultaneous usersFixed it

4000+ simultaneous usersReduced hardware

The Tools

Visual Studio

Ultimate

• Web Performance Tests• Load Tests• Load Test Rigs

Visual Studio

Premium

• Profiling Tools• Performance Wizard

demo

Tour Of The Application

Disaster Strikes.

The application is deployed to production.Users are losing their minds.

Intermittent errors on the site.Site is slow.

Your boss is seriously angry.“Fix it!”

The Problem.

“No Repro”“Works on my box.”You don’t know what the errors are.The site seems to work fine.

Wild guess: it dies under load.You need to create some load.

The Plan.

Script user activity with Web Performance TestsSimulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefullyFind code problems with the Performance WizardFix the code

Web Performance Tests.

The Plan.

Script user activity with Web Performance TestsSimulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefullyFind code problems with the Performance WizardFix the code

What is a Web Performance Test?

Test type in Visual Studio 2010 UltimateSimulate a user using an ASP.NET application

HTTP trafficBuilding block of Load Tests

Tip: Web Tests & User Stories

Web Tests should simulate a User StoryExample:

As an administrator, I need to search for a person by username so that I can deactivate that person’s account.As a customer, I want to order a pizza and pay with a credit card.

Helps to organize your suite

demo

Create a basic Web Test

Data-driven Web Tests

Run web test once per row in data sourceSimulate multiple users doing *similar* actionsData source types

OLE DB, CSV, XMLParameterizationBetter tests, better coverage

demo

Web Tests in the Underperforming Application

Load Tests.

The Plan.

Script user activity with Web Performance TestsSimulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefullyFind code problems with the Performance WizardFix the code

What are Load Tests?

Mix of Web Performance and Unit TestsSimulates a mix of users doing a mix of thingsSimulate

Network speedsDifferent browsersVarying user activity loads

Why would you put a Unit Test in a Load Test?

Helpful for testing WCFCumbersome to simulate WCF via Web Test

(You’d have to re-invent the wheel.)Unit tests can re-use the WCF libraries

(Done and done.)Important for Silverlight and SOAhttp://wcfloadtest.codeplex.com/http://sqlloadtest.codeplex.com/

demo

Hey Kids, Let’s create some Load Tests!(Hooray!)

Running Load Tests.

The Plan.

Script user activity with Web Performance TestsSimulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefullyFind code problems with the Performance WizardFix the code

Two ways to run load tests.

Generate load from Visual Studio 2010Uses only 1 core/processorLimited to 250 simulated users

Generate load from a Test RigFar better testSimulate lots more users

Load Test Agent Licensing

Microsoft Visual Studio Team System 2010 Test Load Virtual User PackSimulate up to 1000 virtual usersUsed to cost $$$

Visual Studio 2010 Load Test Feature Pack

Used to cost ~$3500.00 per 1000 simulated usersFree starting March 8, 2011Visual Studio 2010 Ultimate with MSDNSimulate unlimited amount of load

(Limited only by your hardware.)http://www.microsoft.com/visualstudio/en-us/products/2010-editions/load-test-virtual-user-pack

Load Test Rig & The Application

Application Under Test

Generates Load

Coordinates &

Gathers Information

Controller

Agent 1 Web Server

DatabaseAgent 2

Agent n

ControllerOrchestratesLicensingPerfMon Counters

Agent (1..n)Simulates users

The Plan.

Script user activity with Web Performance TestsSimulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefullyFind code problems with the Performance WizardFix the code

demo

Run Load Tests on the Rig

Performance Profiling

The Plan.

Script user activity with Web Performance TestsSimulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefullyFind code problems with the Performance WizardFix the code

Profiling Tools

Load Tests help you find issues in a system“Surface” issues

Profiling helps you find issues in the codeUnit Tests, Web Tests, Load Tests

RepeatableGood for targeted tuning

System under “natural” loadNot repeatableLooking for rough patterns

Profiling Methods• Gathers information at intervals (clock cycles)• No code modifications• Use this for first pass explorations

Sampling

• Modifies your code• Gathers detailed timing and count informationInstrumentation

• Multi-threaded code profilingConcurrency

• Object creation• Garbage collection.NET Memory

• Interaction between your application and SQL Server via ADO.NETTier Interaction

Load Test Performance Sessions Via Wizard

Must be only Web Performance TestsCan’t profile via Load Test RigASP.NET must be running in x86 mode

demo

Let’s go profile some stuff.

The Plan.

Script user activity with Web Performance TestsSimulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefullyFind code problems with the Performance WizardFix the code

Any last questions?

For More Information: Articles

http://tinyurl.com/3xjqgvz

http://tinyurl.com/2ulvvvr

For More Information: Microsoft Links

Visual Studio Performance Testing Quick Reference Guidehttp://vsptqrg.codeplex.com/ Content Index for Web Tests & Load Testshttp://tinyurl.com/ye97e52

For More Information: Microsoft Bloggers

VS Team Test Blog Site http://blogs.msdn.com/b/vstsqualitytools/Ed Glas - http://blogs.msdn.com/edglas/Bill Barnett - http://blogs.msdn.com/billbar/Sean Lumley - http://blogs.msdn.com/slumley/Dennis Stone - http://blogs.msdn.com/densto/Mike Taute - http://blogs.msdn.com/mtaute/Neelesh Kamkolkar - http://blogs.msdn.com/b/nkamkolkar/

Thank you.

http://blog.benday.com | http://www.benday.com | benday@benday.com

DEV Track Resources

http://www.microsoft.com/visualstudio http://www.microsoft.com/visualstudio/en-us/lightswitch http://www.microsoft.com/expression/http://blogs.msdn.com/b/somasegar/http://blogs.msdn.com/b/bharry/http://www.microsoft.com/sqlserver/en/us/default.aspxhttp://www.facebook.com/visualstudio

Resources

www.microsoft.com/teched

Sessions On-Demand & Community Microsoft Certification & Training Resources

Resources for IT Professionals Resources for Developers

www.microsoft.com/learning

http://microsoft.com/technet http://microsoft.com/msdn

Learning

http://northamerica.msteched.com

Connect. Share. Discuss.

Complete an evaluation on CommNet and enter to win!

Scan the Tag to evaluate this session now on myTech•Ed Mobile

Miscellaneous Tips.

Include Load Tests In Your Nightly Build

Create a Test List (*.vsmdi)Include the Test List in the buildCaptures trend data at 24 hours intervalsCompare the load test data with Excel

Static Code Analysis / FxCop

Help you to find issues in your codeShow you IDispose errorsRules are configurableUse your judgment about which are relevant for your team

Avoid Static Methods & CA1822

Static Code Analysis Rule CA1822 is evil.

Sacrifices maintainability for perfomance.Blog post: “Static Methods Are A Code Smell”http://bit.ly/cnEsuV

Only optimize known performance problems.

You’d be surprised by what *ISN’T* a performance problemDon’t spend a ton of time coding fancy solutions to performance problems you *THINK* you might haveFind and fix *REAL* problems

Tip: WCF Message Size

Data structure formatting can make a huge differenceExample: Banks and the States they operate inOption 1: Dictionary<string, string>

Key = Bank IdValue = State Abbreviation

Option 2: Dictionary<string, string[]>Key = State AbbreviationValue = Array of Bank Id’s

Option 2 was approximately 90% smaller

Tip: Add Performance Counters

Create Performance Counters (perfmon) in your appDo this early in the development cycleMore directed profiling of your app during Load TestsGood for managing your app when it’s in production

(Operations people love this.)

Using Performance Sessions on x64

You can’t profile x64 IIS processes from the IDE

Option #1: Profile from the command lineStart ASP.NET profiling using VSPerfAspNetCmd.exe Run the web test

Option #2: Run the ASP.NET process in x86 mode

How To Run ASP.NET in x86 Mode

Set “Enable 32-bit Applications” to true on your application’s AppPool