itcampro @ itcamp12 # Premium conference on Microsoft technologies

Serving 10 million

requests per day

How to solve the architecture challenges

of a high traffic ASP.NET website

Level 400

Dan Fizesan, Lead Developer,

NetMatch – Travel Technology Solutions

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud ITCamp 2012 sponsors

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

• Dan Fizesan, 37 years old.

• 15 year experience with Progress / C# / ASP.NET / T-SQL.

• Technical Architect / Lead developer in NetMatch / Zoover

project.

• NetMatch international company e-business provider for

tourism industry

• Zoover is a website with reviews about trips on average

500.000 unique visitors per day

• Contact: d.fizesan@netmatch.ro

Who am I

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

• Architecture of a web application

• Deep dive into the macro architecture

• Serving 10 million requests a day - final view

• Demo

• Q & A

Agenda

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

• More and more users are using the online

websites

• Applications on mobile devices are using

data from the web data services

• The FOCUS in the presentation are on-

premise applications that are also subject to

above trends

Context

itcampro @ itcamp12 # Premium conference on Microsoft technologies

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

• Be online as much time as possible

• Serve as many users as possible

• Low cost of ownership

• Good performance

Architectural Goals

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

• Be online as much time as possible

• Serve as many users as possible

• Low cost of ownership

• Good performance

Architectural Goals

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

• We reach high availability when more than 90% of

the time the website is working correctly

• 99% availability = the website is down 87,6 hours in

a year (or 15 minutes every day)

• 99.99% availability = the system is down less then 1

hour per year

• Our systems go for 99.9% availability

Availability

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

How can we reach 99.9% availability?

• Through passive redundancy – failover

• Through active redundancy

• Through data sources replication

Challenge

itcampro @ itcamp12 # Premium conference on Microsoft technologies

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

• Hardware load balancers

• Software load balancers: Network Load

Balancer, Ha Proxy, etc.

• Algorithms to spread the load

• Implementing failover is crucial

• Alternative: Producer-consumer pattern –

used in IIS

Load Balancers + Failover

itcampro @ itcamp12 # Premium conference on Microsoft technologies

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

Web is stateless, what happens with the state when using a web farm?

Challenge

No sessions stickiness: • SQL (session) state

server,

• distributed cache

session state,

• use cookies

With session stickiness • performs worse

• can lead to

overloaded servers

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

• Be online as much time as possible

• How can we serve as many users as possible? Partly through using load balancers

Partly through scalability of the website

• Low cost of ownership

• Good performance

Architectural Goals

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

• The ability of the website to grow the user

base by adding more hardware

• The more linear the relation is, the more

scalable the website is

• Vertical scalability

• Horizontal scalability, lower costs using

commodity hardware

Scalability

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

• Be online as much time as possible

• How can we serve as many users as possible?

Partly through using load balancers

Partly through scalability of the website

• Low cost of ownership

• Good performance

Architectural Goals

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

• Helps the backend (database/services)

• Advantage: brings consistency in an webfarm

• Disadvantages: latency

• Some of the free options:

Windows Server AppFabric Caching, memcached

• Some of the commercial options:

NCache, Azure AppFabric caching

Distributed Caching

itcampro @ itcamp12 # Premium conference on Microsoft technologies

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

How can we protect the website in the case the distributed cache goes down or is not reachable?

• Through implementing failover caching

mechanisms in the website

Challenge

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

• Be online as much time as possible

Through high availability

• Serve as many users as possible

Partly through using load balancers

Partly through scalability of the website

• Low cost of ownership

• Good performance

Architectural Goals

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

• Web code/ business layer performance

• Database queries performance

Flat databases

Replicated databases

• Distributed architecture

splitting into sub-applications

• Parallelization of the code (demo)

Perceived good performance

itcampro @ itcamp12 # Premium conference on Microsoft technologies

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

How can we use distributed architecture and have a good performance?

• We should externalize the resource-intensive

parts of the application into sub-applications

• Choosing wrong parts will only add (network)

latency

Challenge

itcampro @ itcamp12 # Premium conference on Microsoft technologies

itcampro @ itcamp12 # Premium conference on Microsoft technologies

OUR ARCHITECTURE TO SERVE 10 MILLION REQUESTS PER DAY

itcampro @ itcamp12 # Premium conference on Microsoft technologies

itcampro @ itcamp12 # Premium conference on Microsoft technologies

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

300 requests

Demo Results

Approach % s

ThreadPool parallelism 1200% 21,6 s

Task Parallel Library parallelism 4200% 76 s

Simple Threads parallelism 155% 2,8 s

Custom ThreadPool parallelism 100% 1,8 s

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Q & A

Contact: d.fizesan@netmatch.ro

www.netmatch.ro/itcamp2012/challenges.pptx

www.netmatch.ro/itcamp2012/demo.zip

itcampro @ itcamp12 # Premium conference on Microsoft technologies

Private &

Public Cloud

• Availability

• Redundancy

• Load balancer

• Scalability

• Distributed caching

• Performance

To Remember