NOTES ON NETTY PART 3

RICK HIGHTOWER’S

CHANNEL PIPELINES, AND EVENT LOOPS

About Rick hightowerAbout Rick

• Implemented Microservices, Vert.x/Netty at massive scale

• Author of QBit, microservices lib and Boon, Json parser and utility lib

• Founder of Mammatus Technology

• Rick’s Twitter, Rick’s LinkedIn, Rick’s Blog, Rick’s Slideshare

Great book on Netty!

https://www.manning.com/books/netty-in-action

Great talk about Netty Best Practices given at Facebook,then Twitter University

https://goo.gl/LbXheq

http://www.infoq.com/presentations/apple-netty

Great talk about Netty at scale

Previous slide deckPrevious SLIDE DECK

• Notes on Netty Basics Slideshare

• Part 1: http://www.slideshare.net/richardhightower/notes-on-netty-baics

• Part 2: http://www.slideshare.net/richardhightower/netty-notes-part-2-transports-and-buffers

• Notes on Netty Basics Google slides

• Part 1: https://goo.gl/aUGm2N

• Part 2: https://goo.gl/xZZhVs

CHANNEL PIPELINES

Chaining channel handlersChannelPipeline

• Channel - Socket

• ByteBuf - container for bytes of message

• ChannelHandler process / transform messages

• ChannelPipeline - forms chain of ChannelHandlers

Lifecycle of a channel Channel Lifecycle states

• Active

• connected to remote peer

• Inactive

• disconnected from remote peer

• Unregistered

• created

• Registered

• channel associated with event loop

methods on ChannelHandler for lifecycle eventChannelHandler Lifecycle Events

• handlerAdded()

• added to ChannelPipeline

• handlerRemoved()

• removed from ChannelPipeline

• exceptionCaught()

• error during ChannelPipeline processing

event notification event methods for ChannelInboundHanderChannelInboundHandler lifecycle methods for

Channel

• channelRegistered() - Channel married to an event loop

• channelUnregistered() - Channel divorced from event loop

• channelActive() - connected

• channelInactive() - disconnected

• channelReadComplete() - read operation completed

• channelRead() - data is read on channel

• channelWritabilityChanged() - outgoing IO buffer/limit is met or not

• userEventTriggered() - someone passed a POJO to the event loop

Let it go. Let it go. Can't hold it back anymoreReleasing messages

• Not a good idea to override read unless you know what you are doing

• you must free up pooled resources calling ReferenceCountUtil.release(message)

• Use SimpleChannelInboundHandler and override channelRead0() instead, netty will free up message resource for you

java -Dio.netty.leakDetectionLevel=ADVANCED

You can leak on write or read

Handler for outbound ChannelOutboundHandler

• Outbound operations

• methods invoked by

• Channel, ChannelPipeline, and ChannelHandlerContext.

• Can defer operation or event

• Powerful level of control

• Many Methods take a ChannelPromise which extends ChannelFuture and provides ability to mark as succeeded or failed (setSuccess(), setFailure())

• You should mark promise failed or succeeded and also release resources

l ifecycle event methodsChannelOutboundHandler

• bind(channelHandlerContext, localSocketAddress, channelPromise)

• listen to address for connections request event

• connect(channelHandlerContext, socketAddress, channelPromise)

• connect to remote peer request event

• disconnect(channelHandlerContext, channelPromise)

• disconnect from remote peer request event

• close(channelHandlerContext, channelPromise)

• close channel request event

l ifecycle event methodsChannelOutboundHandler

• deregister(channelHandlerContext, channelPromise)

• removed from event loop request event

• read(channelHandlerContext)

• read more data from channel request event

• flush(channelHandlerContext)

• flush data to remote peer request event

• write(channelHandlerContext, message:Object, channelPromise)

• write data to channel request event

chain of channel handlersChannelPipeline

• ChannelPipeline is chain of ChannelHandlers

• handlers intercept inbound and outbound events through Channel

• ChannelHandlers process application IO data

• Channel processes IO events

• New Channel assigned to new ChannelPipeline

• relationship is permanent

• no cheating! Channel can’t attach to another ChannelPipeline

• Channel can’t leave ChannelPipeline

• Direction determines if event handled by ChannelInboundHandler or a ChannelOutboundHandler

• Unhanded events go to next Channel in chain

ChannelHandlers can modify ChannelPipelineChannelPipeline can be edited

• ChannelHandler methods to change ChannelPipeline

• addFirst()

• addBefore()

• addAfter()

• addLast()

• remove()

• replace()

Used by Netty to fire events inboundChannelPipeline methods

• fireChannelRegistered

• fireChannelUnregistered

• fireChannelActive

• fireChannelInactive

• fireExceptionCaught

• fireUserEventTriggered - user defined event so you can pass a POJO to the channel

• fireChannelRead

• fireChannelReadComplete

Outbound methodsChannelPipeline methods

• bind - listen to a port, binds channel to port/address

• connect - connect to a remote address

• disconnect - disconnect from a remote address

• close - close the channel after next channel handler is called

• deregister

• flush

• write

• writeAndFlush

• read

Managing passing the events to next handler in chainChannelHandlerContext

• Associates ChannelHandler and ChannelPipeline

• Created when ChannelHandler added to a ChannelPipeline

• Manages interaction of its ChannelHandler to others in ChannelPipeline

• Use context instead of pipeline (most often) as it involves a shorter event flow

• otherwise must go through whole chain from start

MethodsChannelHandlerContext

• channel - returns the Channel

• bind, close, connect, disconnect

• read, write

• deregister

• executor - returns EventExecutor (has thread pool scheduling interface)

• fireChannelActive, fireChannelInactive, fireChannelRead, fireChannelReadComplete

• handler - returns corresponding handler

• isRemoved - has the handler been removed?

• name - name of the instance

• pipeline - returns the associated pipeline

In Bound Exception handling

public class MyInboundExceptionHandler

extends ChannelInboundHandlerAdapter {

…

@Override

public void exceptionCaught(ChannelHandlerContext

channelHandlerContext,

Throwable cause) {

logger.error(cause);

channelHandlerContext.close();

}

}

Outbound Exception handling

public class MyHandler extends ChannelOutboundHandlerAdapter {

@Override

public void write( final ChannelHandlerContext channelHandlerContext,

final Object message,

final ChannelPromise channelPromise) {

channelPromise.addListener(new ChannelFutureListener() {

@Override

public void operationComplete(final ChannelFuture channelFuture) {

if (!channelFuture.isSuccess()) {

logger.error(channelFuture.cause());

channelFuture.channel().close();

}

}

});

}

}

EVENT LOOPS

Threading model specificsEvent Loop And Threading model

• Threading model specifies is key to understanding Netty

• When threads are spawned is very important to your application code

• You have to understand trade-offs and specifics of Netty

• Multi cores are common occurrence

• Netty uses Executor API interfaces to schedule EventLoop tasks

• Netty tries to reduce cost of thread hand off and CPU cache line moving about by limiting the #of threads to handle IO

• Less is more by reducing “context switching"

• Increases CPU Register, L1, L2, cache hits by keeping things in same thread

• Reduce wake up cost of threads and synchronization of shared variables

Tasks can be submitted to EventLoopEventLoop tasks

• EventLoop indirectly extends Java’s ScheduledExecutorService

• Events and Tasks are executed in order received

• You can use ScheduledExecutorService methods to schedule a task for later execution

• Tasks will run in same thread as IO so no sync needed

Use event loop to schedule tasksEventLoop task schedule

ScheduledFuture<?> future =

channel.eventLoop().scheduleAtFixedRate(...);

Netty does task management to make sure only one thread can handle IO

Ensure Channel is handled by ONE Thread

• Any Thread can call methods on a Channel

• No synchronization in is needed

• How?

• If you call a method on a Channel and its not from the IO Thread (EventLoop thread)

• Netty will schedule a task for that method to be run and the task will run in EventLoop thread

• “Netty’s threading model hinges on determining the identity of the currently executing Thread; that is, whether or not it is the one assigned to the current Channel and its EventLoop.” (Netty in Action)

• If method call is from EventLoop Thread, then method is executed right away

• Each EventLoop has a task queue

• task queue is not shared

• Do not put long running tasks in the task queue of an event loop

• If long running use another thread pool (not Netty) to execute and then call channel when done

Relationship for NIOFor NIO

• An EventLoop manages many Channels

• Channels can be sockets/connections/clients

• If you block in one, you block all clients

• Do not block

THANKS AND BUY NORMAN’S BOOK

Netty in ActionIdeas for slides

• Many ideas for slides are directly derived from Netty in Action book by Norman et al and/or his talks on Netty

• BUY THE BOOK Netty in Action!

• The slides are a study aid for myself so I can better learn Netty

• I’ve worked with ByteBuffer, NIO, Vert.x, and have often wanted to use parts of Netty on projects but lacked the knowledge of Netty internals so used NIO or ByteBuffer or OIO when I really wanted to use Netty instead

Previous slide deckPrevious SLIDE DECK

• Notes on Netty Basics Slideshare

• Part 1: http://www.slideshare.net/richardhightower/notes-on-netty-baics

• Part 2: http://www.slideshare.net/richardhightower/netty-notes-part-2-transports-and-buffers

• Notes on Netty Basics Google slides

• Part 1: https://goo.gl/aUGm2N

• Part 2: https://goo.gl/xZZhVs

About Rick hightowerAbout Rick

• Implemented Microservices, Vert.x/Netty at massive scale

• Author of QBit, microservices lib and Boon, Json parser and utility lib

• Founder of Mammatus Technology

• Rick’s Twitter, Rick’s LinkedIn, Rick’s Blog, Rick’s Slideshare

The end… sleepless dev.