Zero-Copy Event-Driven Servers with Netty
-
Upload
daniel-bimschas -
Category
Technology
-
view
13.582 -
download
0
description
Transcript of Zero-Copy Event-Driven Servers with Netty
![Page 1: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/1.jpg)
Zero-Copy Event-Driven Servers with Netty
Daniel Bimschas Institute of Telematics, University of Lübeck
http://www.itm.uni-luebeck.de
1
![Page 2: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/2.jpg)
Content • Basics
– Zero-Copy Techniques – Event-Driven Architectures
• Introduction to Netty – Buffers – Codecs – Pipelines & Handlers
• Netty Examples – Discard – HTTP Server
2
![Page 3: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/3.jpg)
Zero-Copy Techniques
3
![Page 4: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/4.jpg)
Traditional Approach • copying and context switches between kernel
and user space à poor performance!
4
App
Kernel Context
Application Context
Application Buffer
Read Buffer Socket Buffer NIC Buffer Fast
Slow
Slide Source: Distributed Systems Course 2011/2012 by Dennis Pfisterer, Institute of Telematics, University of Lübeck, Germany
![Page 5: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/5.jpg)
Zero-Copy Approach • Kernel handles the copy process via Direct Memory Access (DMA)
– No CPU load – Lower load on bus system – No copying between kernelspace and userspace
5
App
Kernel Context
Application Context
Application Buffer
Read Buffer Socket Buffer NIC Buffer Fast
Perfect! Task
Slide Source: Distributed Systems Course 2011/2012 by Dennis Pfisterer, Institute of Telematics, University of Lübeck, Germany
![Page 6: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/6.jpg)
Simple Benchmark: Copy vs. Zero-Copy D
urat
ion
[ms]
Data [Mbyte] 6 Slide Source: Distributed Systems Course 2011/2012 by Dennis Pfisterer, Institute of Telematics, University of Lübeck, Germany
![Page 7: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/7.jpg)
Zero-Copy Between Communication Layers • Often copying is not necessary
– If data is not modi"ed a slice can be passed forward without copying to a different buffer
7
Link Layer
Internet
Transport
Application
Ethernet IP TCP HTTP XML
Ethernet IP TCP HTTP XML
Ethernet IP TCP HTTP XML
Ethernet IP TCP HTTP XML
Ethernet IP TCP HTTP XML
Slide Source: Distributed Systems Course 2011/2012 by Dennis Pfisterer, Institute of Telematics, University of Lübeck, Germany
![Page 8: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/8.jpg)
Zero-Copy Between Communication Layers • Sometimes slices of multiple packages can be
combined to extract e.g., a payload that is split over multiple packages
• Newly “created” buffer points to original buffers à No copying necessary
8
TCP HTTP (Part 1) TCP HTTP (Part 2)
HTTP (Part 1) HTTP (Part 2)
Received Buffers
Virtual Buffer
![Page 9: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/9.jpg)
Event-Driven Architecture in Networking
9
![Page 10: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/10.jpg)
Request Processing in Multi-Thread Servers t1: Thread S1: ServerSocket
d1: Decoder
s1: Servlet
db1: DataBase
t2: Thread
waitForData()
bytes = read()
<<create>>(socket)
run()
<<create>>
decode(bytes)
<<create>>
processRequest(req) query(...)
results response write(response)
socket = accept()
Waits most of time without doing
actual work! socket = accept()
waitForData()
bytes = read() req = decode(bytes)
= thread idle
s2: Socket
10
![Page 11: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/11.jpg)
Request Processing in Multi-Thread Servers • Usually one thread per request
– Thread idle most of the time (e.g. waiting for I/O) – Thread even more idle when network slow – Number of simultaneous clients mostly limited by
maximum number of threads
• Thread construction is expensive – Higher latency when constructing on request – Can be improved using pools of Threads
(see Java‘s ExecutorService & Executors classes)
11
![Page 12: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/12.jpg)
Request Processing in Event-Driven Servers io1: NioWorker s1: Socket s2: Socket
dataAvailable()
bytes = read()
e1: ExecutorThread
handleEvent(s1, bytes)
Disclaimer: this slide may contain errors and is far away from real implementation code but should do good for illustrative purposes
d1: Decoder <<create>>
bytes = read() handleEvent(s2, bytes)
dataAvailable()
d2: Decoder <<create>>
decode(bytes)
decode(bytes) bytes = read() handleEvent(s1, bytes)
dataAvailable()
req = decode(bytes)
resp = processRequest(bytes)
write(resp)
bytes = read() handleEvent(s2, bytes)
dataAvailable()
resp = processRequest(bytes)
req = decode(bytes)
write(resp)
= request 1
= request 2
12
![Page 13: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/13.jpg)
Request Processing in Event-Driven Servers • Calls to I/O functions of OS are non-blocking • Heavy usage of zero-copy strategies • Everything is an event
– Data available for reading – Writing data – Connection established / shut down
• Different requests share threads • Work is split into small tasks
– Tasks are solved by worker threads from a pool – Thread number and control decoupled from individual
connections / simultaneous requests • Number of simultaneous clients can be very high
– Netty: 50.000 on commodity hardware!
13
![Page 14: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/14.jpg)
Introduction to Netty
14
![Page 15: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/15.jpg)
Introduction to Netty
• „The Netty project is an effort to provide an asynchronous event-driven network application framework for rapid development of maintainable high-performance protocol servers & clients.“
• Good reasons to use Netty: • Simpli"es development • Amazing performance • Amazing documentation (Tutorials, JavaDocs), clean concepts • Lots of documenting examples • Unit testability for protocols • Heavily used at e.g., twitter for performance critical applications
Source: http://netty.io
15
![Page 16: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/16.jpg)
Netty – Feature Overview
16
![Page 17: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/17.jpg)
Introduction to Netty - Buffers • Netty uses a zero-copy strategy for efficiency • Primitive byte[] are wrapped in a ChannelBuffer • Simple read/write operations, e.g.:
– writeByte() – writeLong() – readByte() – readLong() – …
• Hides complexities such as byte order • Uses low overhead index pointers for realization:
17
![Page 18: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/18.jpg)
Introduction to Netty - Buffers • Combine & slice ChannelBuffers without copying
any payload data by e.g., – ChannelBuffer.slice(int index, int length) – ChannelBuffers.wrappedBuffer(ChannelBuffer... Buffers)
• Pseudo-Code Example:
TCP HTTP (Part 1) TCP HTTP (Part 2)
HTTP (Part 1) HTTP (Part 2)
Received Buffers
Virtual Buffer
requestPart1 = buffer1.slice(OFFSET_PAYLOAD, buffer1.readableBytes() – OFFSET_PAYLOAD);
requestPart2 = buffer2.slice(OFFSET_PAYLOAD, buffer2.readableBytes() – OFFSET_PAYLOAD);
request = ChannelBuffers.wrappedBuffer(requestPart1, requestPart2);
18
![Page 19: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/19.jpg)
Netty – Feature Overview
19
![Page 20: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/20.jpg)
Introduction to Netty - Codes • Many protocol encoders/decoders included
– Base64 – Zlib – Framing/Deframing – HTTP – WebSockets – Google Protocol Buffers – Real-Time Streaming Protocol (RTSP) – Java Object Serialization – String – (SSL/TLS)
20
![Page 21: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/21.jpg)
Introduction to Netty - Codecs • Abstract base classes for easy implementation
– OneToOneEncoder – OneToOneDecoder
• Converts one Object (e.g. a ChannelBuffer) into another (e.g. a HttpServletRequest)
– ReplayingDecoder • The bytes necessary to decode an Object (e.g. a
HttpServletRequest) may be split over multiple data events • Manual implementation forces to check and accumulate data
all the time • ReplayingDecoder allows you to implement decoding
methods just like all required bytes were already received
21
![Page 22: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/22.jpg)
Netty – Putting it all together
22
![Page 23: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/23.jpg)
Introduction to Netty – Pipelines & Handlers
• Every socket is attached to a ChannelPipeline
• It contains a stack of handlers – Protocol
Encoders / Decoders – Security Layers
(SSL/TLS, Authentication) – Application Logic – ...
23
![Page 24: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/24.jpg)
Introduction to Netty – Pipelines & Handlers • One ChannelPipeline per
Connection • Handlers can handle
– Downstream events – Upstream events – Or both
• Everything is an event – Data available for reading – Writing data – Connection established /
shut down – …
24
![Page 25: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/25.jpg)
Netty – ChannelPipeline Example: HTTP(S) Client
Client Application
Channel
write(httpRequest) read(httpResponse)
HttpRequestDecoder HttpRequestEncoder
httpRequest httpResponse
ChannelP
ipeline StringEncoder StringDecoder
SSLDecoder SSLEncoder
String
ChannelBuffer
String
ChannelBuffer
OS Socket object
ChannelBuffer ChannelBuffer
Disclaimer: this slide is imprecise, may contain errors and there’s no one-to-one implementation. It shows a logic conceptual view of the Netty pipeline.
• Applications based on Netty are built as a stack
• Application Logic sites on top of the channel
• Everything else (decoding, securing, ...) is done inside the pipeline
25
![Page 26: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/26.jpg)
Netty Examples
DISCARD Server
26
![Page 27: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/27.jpg)
Netty Examples – DISCARD Server http://tools.ietf.org/html/rfc863
Source: Netty project source code @ http://github.com/netty/netty 27
![Page 28: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/28.jpg)
Netty Examples – DISCARD Server Bootstrap
Source: Netty project source code @ http://github.com/netty/netty 28
![Page 29: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/29.jpg)
Netty Examples – DISCARD Server
Source: Netty project source code @ http://github.com/netty/netty
29
![Page 30: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/30.jpg)
Netty Examples – Echo Server
Source: Netty project source code @ http://github.com/netty/netty 30
![Page 31: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/31.jpg)
Netty Examples – Static HTTP File Server
Source: Netty project source code @ http://github.com/netty/netty 31
![Page 32: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/32.jpg)
Netty Examples – Static HTTP File Server
Source: Netty project source code @ http://github.com/netty/netty 32
![Page 33: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/33.jpg)
Netty Examples – Static HTTP File Server
Source: Netty project source code @ http://github.com/netty/netty ...
33
![Page 34: Zero-Copy Event-Driven Servers with Netty](https://reader033.fdocuments.net/reader033/viewer/2022050804/546c732faf795985298b4fed/html5/thumbnails/34.jpg)
References • Netty
– Project: http://netty.io – Tutorial: http://netty.io/docs/ – JavaDoc: http://netty.io/docs/3.2.6.Final/api/ – Sources: http://netty.io/docs/3.2.6.Final/xref/ – Development: https://github.com/netty/netty
34