Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development
-
Upload
michael-figuiere -
Category
Technology
-
view
3.368 -
download
0
Transcript of Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development
![Page 1: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/1.jpg)
Large Scale Web Development
Theory and practice with Java
2/3/2011 Michaël Figuière
![Page 2: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/2.jpg)
Scalability best practices
![Page 3: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/3.jpg)
Typical Web Architecture
Backend A
Load BalancerApplication
Instance
ApplicationInstance
Backends may be slow, fast, highly available or not
Backend B
Backend C
Backend D
Backend E
ApplicationInstance
![Page 4: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/4.jpg)
Facing the network’s reality
• Some requests will be slow
• Some requests won’t answer
• Some requests will just fail
Server / proxy overloaded, network traffic, ...
Server failure, network failure, OS and JVM pressure
Server application’s bugs, GC, connection rejected...
![Page 5: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/5.jpg)
Handling the network’s reality
• Timeout must be set and handled for every remote request
• Use Circuit Breaker pattern
• Setting a deadline for your answer may be helpful
If API doesn’t offer it, ExecutorService and Future can help
Whatever happen, answer will be sent as is within N sec. If mandatory goals aren’t achieved, return error.
Avoid requesting an already overloaded service
![Page 6: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/6.jpg)
Requests make the load
• Two requests instead of one double the load of the backend
• Cache must be sized with care
Here, counting requests isn’t about optimization, it’s critical
Cache Misses increase load on backends
![Page 7: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/7.jpg)
Make requests in parallel
• Parallel requests reduce overall duration
• Thread pools make it possible easily
• Thread pools also act as a throttle to shield a backend
Mandatory when backends are slow
No more than N concurrent requests
ExecutorService and Future do the job
![Page 8: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/8.jpg)
Make requests in parallel
D = Sum of requests durations
D = Max of requests durations
![Page 9: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/9.jpg)
From separated thread pools to semaphores
• When you have a thousand threads, merging thread pools can help
• A semaphore can then do the throttling job
• Semaphore can also be tuned for live throttle tuning !
Mutualize resources
Allowing to slow down a requests stream to a dying backend
Limit the concurrent users of a resource
![Page 10: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/10.jpg)
Serialized caches in Java heap
• Garbage Collector tuning can be time consuming
• Serializing data structures in Java heap caches reduces pressure on GC
• Don’t use Java Standard Serialization, use Avro, Kryo, or ProtocolBuffer
Especially when production environment is hard to simulate
Very low CPU overhead and a so compact format
GC time complexity partly depends on amount of references
![Page 11: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/11.jpg)
Memcached instead of Java heap cache
• Memcached is a simple and efficient Unix daemon
• Several Java clients available
Only two parameters to set : memory size and listening port
All based on NIO !
![Page 12: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/12.jpg)
Partitioned Memcached
Application Memcached
Memcached
Memcached
Memcached
MemcachedClient
Application
MemcachedClient
Application
MemcachedClient
R/W requests between the application and one of the memcached instances (depending on hashing)
![Page 13: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/13.jpg)
Monitor everything
• A JMX attribute only costs an AtomicInteger and is priceless
• Spring JMX offers efficient annotations
• Hyperic can do the aggregating job
AtomicInteger doesn’t cost synchronization
But so awful to configure and use. SpringSource promises to makes it better !
@ManagedResource, @ManagedAttribute
![Page 14: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/14.jpg)
Logging with care
• With high traffic, strange things happen
• These strange things may be hard to reproduce in development environment
• Logs are the only way to track them
Synchronization issues, connection losses, weird requests, ...
You’ll have a lot of logs to store, but it’s ok
Production environment’s behavior can’t be fully simulated
![Page 15: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/15.jpg)
Concurrency Playground
![Page 16: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/16.jpg)
What can be done with java.util.concurrent ?
• Parallel invocations, with or without dependencies between requests
• Making synchronous and asynchronous code collaboration possible
• Blocking IO code in pooled threads mixed with NIO code
Wrapping Future, CountDownLatch, NIO callbacks, ...
CountDownLatch, custom Future implementation, ...
ExecutorService with Future will do the job
![Page 17: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/17.jpg)
Basic Parallel Requests
ServletThread
executorService.submit()
Thread A(from Pool)
Thread B(from Pool)
future.get()
future.get()
![Page 18: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/18.jpg)
Basic Parallel Requests
ServletThread
executorService.submit()
Thread A(from Pool)
Thread B(from Pool)
Callable.call() Callable.call()
future.get()
future.get()
![Page 19: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/19.jpg)
Basic Parallel Requests
ServletThread
executorService.submit()
Thread A(from Pool)
Thread B(from Pool)
Callable.call() Callable.call()
future.get()
future.get()
![Page 20: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/20.jpg)
Thread Pooled Requests + Memcached NIO Client
ServletThread
Thread A(from Pool)
Thread B(from Pool)
get()
NIO Thread(Memcached)
invoke()(custom
ExecutorService)
future.get()
future.get()
![Page 21: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/21.jpg)
Thread Pooled Requests + Memcached NIO Client
ServletThread
submit()(in read
callback)
Thread A(from Pool)
Thread B(from Pool)
get()
NIO Thread(Memcached)
invoke()(custom
ExecutorService)
future.get()
future.get()
![Page 22: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/22.jpg)
Thread Pooled Requests + Memcached NIO Client
ServletThread
submit()(in read
callback)
Thread A(from Pool)
Thread B(from Pool)
call()
get()
NIO Thread(Memcached)
call()
invoke()(custom
ExecutorService)
future.get()
future.get()
![Page 23: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/23.jpg)
Thread Pooled Requests + Memcached NIO Client
ServletThread
submit()(in read
callback)
Thread A(from Pool)
Thread B(from Pool)
call()
get()
NIO Thread(Memcached)
call()
invoke()(custom
ExecutorService)
future.get()
future.get()
![Page 24: Xebia Knowledge Exchange (feb 2011) - Large Scale Web Development](https://reader035.fdocuments.net/reader035/viewer/2022062513/55635fdfd8b42a5c598b4c0d/html5/thumbnails/24.jpg)
Questions / Answers
?@mfiguiere
blog.xebia.fr