HTTP2 and gRPC
62
gRPC Jee-Arr-Pee-See
Transcript of HTTP2 and gRPC
gRPCJee-Arr-Pee-See
gRPC introduction➤ what is gRPC➤ why we us gRPC➤ HTTP 1.1/ HTTP2➤ TCP Lazy Start / Heartbeat➤ Protobuf➤ how gRPC works➤ the good part➤ the bad part➤ what we do next
What is gRPC➤ gRPC is a modern, open source remote procedure call (RPC) framework
that can run anywhere. It enables client and server applications to communicate transparently, and makes it easier to build connected systems.
Python Client
gRPC Server
Java Client
Proto Request
Proto Request
gRPC with GO!
why we us gRPC - HTTP➤ We try to use Protobuf and HTTP➤ HTTP is easy for everyone!➤ We need to do a lot to make HTTP better performace➤ We need to think about time, package size and safety➤ Transport need to trunked
Web Proxy Jedi
Account
NOSQL
DB
200ms
200ms
200ms
100ms
10ms
why we us gRPC - How about thrift?➤ Thrift is based on raw socket➤ This is super fast!➤ But hard for develop➤ We need to look into the connection➤ Not very good for mobile
App Server
sorry? we lose connection?
OK! We create a new one! I response! But no FIN!
I have to WAIT!
why we us gRPC - gRPC➤ IT IS HTTP but version 2➤ Better performance than HTTP1➤ It using Protobuf which is binary➤ HTTP2 Transport layer using binary➤ Persistent Connection
What's problem with HTTP1➤ textual➤ ordered and blocking➤ large size of HEADER - most are cookies➤ Persistent Connection may not work for every system➤ multi connection - TCP Slow Start➤ can't push from server - web socket is not for everywhere
HTTP1 - textual
https://www.w3.org/Protocols/HTTP/1.1/rfc2616bis/draft-lafon-rfc2616bis-latest.html#message.types
HEADER
BODY
CRLF
HTTP1 - textual
Cookie: A=1HEADER-1: HELLO\n\rThis is the body of http request or response.What da luo li ji?
HTTP1 - ordered and blocking
Browser
or
APP
Server
give me html
give me css
give me js
That why we put js at the end of the content, it will blockand why we make icon together and file small...
HTTP1 - ordered and blocking
Browser
or
APP
Server
give me more ...
Limit of TCP connectionmay block heartbeat
Slow Start of TCP
HTTP1 - Large size of HEADERCookie: AWSELB: AD910F851AAED57C481223E777E0EB4FC465E931AB9639541C1EC1B1D5723D2290FDEAD3216B4F9B93C1B30CD47F245B24175467919E3B654E480345440B279CA95626A3C8; Hm_lpvt_5e1b9d2c20eff9c9278a43f0870f4593: 1458451373; Hm_lvt_5e1b9d2c20eff9c9278a43f0870f4593: 1458451373; __ads_session: k34wk5FrsQjw5w0ACgA
Response: 200 OK
Access-Control-Allow-Credentials:true
Access-Control-Allow-Headers:X-Requested-With, content-type, accept, origin, authorization, x-csrftoken
Access-Control-Allow-Methods:GET, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Origin:*
Connection:keep-alive
Content-Encoding:gzip
Content-Language:en-US
Content-Length:6833 Lots of content make things bad
HTTP1 - Persistent Connection➤ Persistent Connection need heartbeat but...➤ large size content may block heartbeat
BODY❤
This is a connection!
➤ So we need to make body small➤ Make them fail fast
BODY❤ BODY❤
Read more...
HTTP1 - TCP Slow Start➤ Yes! But we could use multi connection to solve this problem, but ...➤ TCP Slow Start...
End Point End Point
let me send you 1
It's ok send me 2
let me send you 2
It's ok send me 4
HTTP1 - TCP Slow Start➤ If there is a problem it will make less data➤ So multi connection with large data is not so good hmmm...
End Point End Point
let me send you 4
It's ok send me 8
let me send you 8
no! my buffer is full, send me 4!
HTTP1 - Can't push from server➤ Web socket is good➤ Different browser has different support➤ What if the network is so bad?➤ What about the APP?
HTTP2➤ binary➤ multiplexed➤ one connection for parallelism➤ make data smaller➤ uses header compression to reduce overhead➤ server push
HTTP2 - binary, so this THE frame
https://http2.github.io/http2-spec/
LENGTH (24)
Stream Identifier
Type (8) Flags (8)
Frame payload
R
HTTP2 - DATA FARME
PAD LENGTH (8)
Stream Identifier
Frame payload
➤ DATA frames (type=0x0) convey arbitrary, variable-length sequences of octets associated with a stream. One or more DATA frames are used, for instance, to carry HTTP request or response payloads.
HTTP2 - HEADER FARME
PAD LENGTH (8)
Stream Dependency ? (31)
Weight (8)
➤ The HEADERS frame (type=0x1) is used to open a stream (Section 5.1), and additionally carries a header block fragment.
E
Header Block Fragment (*)
Padding (*)
HTTP2 - CONTINUATION FARME
Header Block Fragment (*)
➤ The CONTINUATION frame (type=0x9) is used to continue a sequence of header block fragments .
HTTP2 - Stream➤ A "stream" is an independent, bidirectional sequence of frames exchanged between
the client and server within an HTTP/2 connection➤ A single HTTP/2 connection can contain multiple concurrently open streams, with
either endpoint interleaving frames from multiple streams.➤ Streams can be established and used unilaterally or shared by either the client or
server.➤ Streams can be closed by either endpoint.➤ The order in which frames are sent on a stream is significant.➤ Streams are identified by an integer. Stream identifiers are assigned to streams by
the endpoint initiating the stream.
HTTP2 - Stream Prioritization➤ Once an HTTP message can be split into many individual frames, and we allow for frames from multiple
streams to be multiplexed.➤ Each stream may be assigned an integer weight between 1 and 256➤ Each stream may be given an explicit dependency on another stream
Stream 1
Stream 2
Stream 3
Stream 4
F
F
F
F
HTTP2 - Stream Prioritization➤ So...
*
A 12 B 4
*
D 1
C 8
*
E 8 F 8
D 4
B 4 A 12
root stream root stream root stream
CONTINUATIONCONTINUATIONCONTINUATION
END_HEADERS
HTTP2 - Header Compression and Decompression➤ The CONTINUATION frame (type=0x9) is used to continue a sequence of header block
fragments .➤ a single HEADERS or PUSH_PROMISE frame, with the END_HEADERS flag set➤ a HEADERS or PUSH_PROMISE frame with the END_HEADERS flag cleared and one or more
CONTINUATION frames, where the last CONTINUATION frame has the END_HEADERS flag set.
HEADERS
END_HEADERS
PUSH_PROMISE
END_HEADERS
HEADERS
HTTP2 - Server Push➤ of course➤ but we did not do that :)
“so we choose Jee-Arr-Pee-See
-DaGuGuJi
gRPC➤ based on HTTP2➤ based on Protobuf
Protobuf 3➤ Protobuf is binary➤ Support format to JSON, XML and Thrift➤ Support multi Programming Language
C++
C#
Java
GOPython
Protobuf 3➤ Simple and Easy
message Person { required string name = 1; required int32 id = 2; optional string email = 3;}
Person john = Person.newBuilder()
.setId(1234) .setName("Guojing") .setEmail("[email protected]
") .build();output = new
FileOutputStream(args[0]);john.writeTo(output);
“Show me the gRPC!
-Big Family
gRPC➤ gRPC support multi Programming language too➤ Python, PHP, Ruby ... are based on gRPC C Core➤ gRPC-Java➤ gRPC-GO➤ simple RPC and Stream RPC!
gRPC➤ scripting language is easy for develop➤ but lower performance than C/C++/Java/GO➤ but language performace is not what we consider these days➤ best choice is GO➤ run gRPC-Java on JVM - not a web project
gRPC➤ run as a service, not part of web project
gRPC server gRPC server gRPC server
Proxy like tomcat or flask
FE
gRPC➤ run as a service, if part of web project, we may go die
FE
tomcat or flask
gRPC Server
tomcat or flask
gRPC Server
tomcat or flask
gRPC Server
I don't know which one is dead with what protocol
gRPC➤ deploy will be easier if we ...
gRPC server
Proxy like tomcat or flask
FE
Proxy like tomcat or flask
nginx
gRPC server gRPC server gRPC server gRPC server
One gRPC Server is downweb is still working!
How gRPC works➤ Server➤ Stub➤ Channel➤ CompletionQueue
How gRPC works - C Core➤ Server
service Greeter { rpc SayHello (HelloRequest) returns (HelloReply) {}}
message HelloRequest { string name = 1;}
message HelloReply { string message = 1;}
def serve(): server = route_guide_pb2.beta_create_RouteGuide_server(RouteGuideServicer()) server.add_insecure_port('[::]:50051') server.start()
How gRPC works - C Core - Server➤ Server is just Start multi TCP Server Listener➤ Server Start ---> for each tcp server start➤ Tcp Server waiting for stream and frame➤ Check frame type then call the Frame parser➤ Run callback
How gRPC works - C Core - Channel➤ Channel is connection maintainer➤ Channel init status is IDLE➤ If a client call, Channel status will be READY➤ We could subscribe a channel with callback➤ Also unsubscribe channel➤ Channel could reconnect automatically
Channel
ConnectionConnection
How gRPC works - C Core - Stub➤ Stub is just a function wrapper➤ It create a Call with a Ticket
Channel
Call With Ticket
How gRPC works - C Core - Channel➤ Channel is complicate➤ Channel has it stack➤ Channel has filter➤ Every Call to Channel need to copy Channel filter to Call element filter
How gRPC works - C Core - CompletionQueue➤ When server receive stream➤ After parse to object and then call the callback function➤ It may enqueue➤ The Programming language layer (Python) will create a thread to get the
element from the queue with registered callback
How gRPC works - C Core - CompletionQueue
Completion Queue
Main Thread
Sub Thread
EventEventEvent
gRPC - The good part➤ Based on HTTP2➤ Fast➤ We could use Thread Pool➤ From Google and someone is working with gRPC
this is important because we don't want a frame work no on maintain
gRPC - The bad part➤ We need to know everything is not good enough➤ So still bug
gRPC - The bad part➤ Stream stack overflow then abort() ...
➤ Because a channel has max request number➤ Server may not create such threads > 1000
➤ Install monit to restart when it down➤ We have haproxy to make thread and request number in control
gRPC - The bad part➤ Persistent Connection will (maybe) not load balanced
➤ Because IT IS Persistent Connection
gRPC - The bad part
Proxy
gRPC gRPC
at least one machine is IDLE
gRPC - The bad part
Proxy
gRPC gRPC
gRPC - The bad part
Proxy
gRPC gRPC
we need zookeeper or consul
gRPC - The bad part
Proxy
gRPC gRPC
Channel subscribe callback
gRPC - The bad part➤ We change service, then we need to rebuild in both client➤ We have lots of end points, how to maintain?
➤ That it is, we must be smart.
gRPC - What problem we have now
DEVELOPwith
UNIT TESTCI
➤ We will have a lot service➤ We need to have UNIT TEST and CI➤ We need to make everyone start a project easy
docker
vagrant
Mac OS
Linux
gRPC - What problem we have now
ServiceTest
+
SmokeTest
+
EndTest
➤ QA team to make things faster➤ Service deploy need to be fast➤ We can't test every part of the service➤ This need to be auto
gRPC - What problem we have now
Deploy onTHE
VERSIONthat
passedCI
andTest
➤ SA Team need to➤ Deploy fast➤ Rollback fast➤ Pre-Release➤ A/B Test
gRPC - What problem we have now
AfterRelease
➤ Log !➤ Performance➤ PM need to check the result➤ Exception Collection➤ Hot fix! (add smoke test)
gRPC - We need to have a public develop team to
VIPKID
Toolkit
➤ We need to have a public develop team to:
➤ SDK➤ Toolkit➤ Automatically system for whole thing like - (test
request in gitlab CI)
We are a team➤ We need to be faster➤ We need to think more and different➤ We don't blame A person➤ We make things better
May the force be with you
FAQ
Thanks
