Fscons scalable appplication transfers

Scalable application layer transfers

Daniel Stenberg

Free Software

Network hacker, code and IETF

Embedded developer

Consultant

Email: [email protected]: @bagderWeb: daniel.haxx.seBlog: daniel.haxx.se/blog

Agenda

Traditional network client

applied to libcurl

C10K

applied to libcurl

Protocol stack

ContentsApplicationTransportInternetLink

libcurl

curl.haxx.se

Very portable C library

Application protocol transfers: DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP

Roots from 1997

MIT licensed

Daniel is lead developer

A typical client

Create socket

Connect socket

Select() for data to read or write

Read or write data

Loop

pseudo C code

WARNINGthe following source code examples are not complete and they resemble C code

int main(int argc, char *argv[]){ int sock; /* Socket descriptor */

if ((sock = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0) DieWithError("socket() failed");

/* it would resolve the host name here */

if (connect(sock to server)) DieWithError("connect() failed");

do {

select(maxfd+1, sock, ); /* wait for socket to become readable or writable */

if(check sock if writable) send(sock, string, string_len, 0);

if(check sock if readable) recv(sock, string, string_len, 0);

} while(everything is fine);

close(sock); exit(0);}

School book example

1

2

1

3

4

5

Simple case works fine

Easy to read

Fast to write

Easy to learn and teach

int main(int argc, char *argv[]){ int sock[NUM_CONNS]; /* Socket descriptors */

for(i=0; i< NUM_CONNS; i++) { if ((sock[i] = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0) DieWithError("socket() failed");

if (connect(sock[i] to server[i])) DieWithError("connect() failed");

do { /* set up the bitmasks for select() */

select(maxfd+1, sock, ); /* wait for any socket to become readable or writable */

for(i=0; i< NUM_CONNS; i++) { if(check sock[i] if writable) send(sock, string, string_len, 0);

if(check sock[i] if readable) recv(sock, string, string_len); }



More connections

1

2

1

3

4

5

int main(int argc, char *argv[]){ int sock[NUM_CONNS]; /* Socket descriptors */

for(i=0; i< NUM_CONNS; i++) { if ((sock[i] = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0) DieWithError("socket() failed");

if (connect(sock[i] to server[i])) DieWithError("connect() failed");

do { /* set up the bitmasks for select() */

select(maxfd+1, sock, ); /* wait for any socket to become readable or writable */

for(i=0; i< NUM_CONNS; i++) { if(check sock[i] if writable) send(sock, string, string_len, 0);

if(check sock[i] if readable) recv(sock, string, string_len); }



More connections

Many connections

Quickly degrades

select() and poll() suck

select() slightly more due to FD_SETSIZE

Threads help to some degree, but...

what about libcurl

examples in C

has bindings for 40 languages

libcurl simple app

int main(void){ CURL *curl; CURLcode res;

curl = curl_easy_init(); if(curl) { curl_easy_setopt(curl, CURLOPT_URL, "http://example.com"); res = curl_easy_perform(curl);

/* always cleanup */ curl_easy_cleanup(curl); } return 0;}

easy interface

easy to write

synchronous

supports many protocols in one go

made for single transfers

libcurl multi app

int main(int argc, char **argv){ handle1 = curl_easy_init(); handle2 = curl_easy_init();

curl_easy_setopt(handle1, CURLOPT_URL, "http://www.example.com/"); curl_easy_setopt(handle2, CURLOPT_URL, "http://fscons.org/");

multi_handle = curl_multi_init();

curl_multi_add_handle(multi_handle, handle1); curl_multi_add_handle(multi_handle, handle2);

curl_multi_perform(multi_handle, &still_running);

while(still_running) { curl_multi_timeout(multi_handle, &curl_timeo); curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd);

rc = select(maxfd+1, &fdread, &fdwrite, &fdexcep, &timeout);

if(rc says timeout or readable/writable sockets) curl_multi_perform(multi_handle, &still_running); }

curl_multi_cleanup(multi_handle);

curl_easy_cleanup(http_handle); curl_easy_cleanup(http_handle2);

return 0;}

1

2

5

4

6

3

7

8

libcurl multi app

int main(int argc, char **argv){ handle1 = curl_easy_init(); handle2 = curl_easy_init();

curl_easy_setopt(handle1, CURLOPT_URL, "http://www.example.com/"); curl_easy_setopt(handle2, CURLOPT_URL, "http://fscons.org/");

multi_handle = curl_multi_init();

curl_multi_add_handle(multi_handle, handle1); curl_multi_add_handle(multi_handle, handle2);

curl_multi_perform(multi_handle, &still_running);

while(still_running) { curl_multi_timeout(multi_handle, &curl_timeo); curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd);

rc = select(maxfd+1, &fdread, &fdwrite, &fdexcep, &timeout);

if(rc says timeout or readable/writable sockets) curl_multi_perform(multi_handle, &still_running); }

curl_multi_cleanup(multi_handle);

curl_easy_cleanup(http_handle); curl_easy_cleanup(http_handle2);

return 0;}

multi interface

several easy handles dealt with combined

made for simultaneous transfers

still easy to write and read

but: relies on select()

select with many

Scale to c10K

10000 simultaneous connections

not as easy as it may sound

A better way

event-based

react only on the sockets that have changed actions

no check for action

no loops

really scalable

not that portable

There's no POSIX for event-based

The portable libs don't support all platforms

epoll (Linux), kqueue (FreeBSD, NetBSD, OpenBSD, Darwin), /dev/poll (Solaris, HPUX), pollset (AIX), Event Completion (Solaris 10), I/O Completion Ports (Microsoft Windows)

libev and libevent

potent event libraries we base the examples on in this presentation

they support a subset of platforms

the examples are not complete now either

event-based basics

detail what sockets we work with and what do we wait for on the sockets

timeouts

wait until something happens

simple event client

int function(int socketfd){ev = ev_default_loop();

ev_timer_init(..., timer_callback, );

ev_io_init(ev, event_callback, socketfd, WHAT);

ev_loop(ev); /* sit here until done */}

int timer_callback(...){/* timer expired! */}

int event_callback(..., int socketfd, ...){/* something happened on my socket, now we recv or send etc */

}

event based compared

different code-flow

might require a different state machine for the protocol

again: not widely portable

Other ways to avoid wrongs

one thread per connection with blocking sockets

threads with non-blocking i/o to handle N connections each

using libcurl scalable

multi_socket API added 2006

tells the app what sockets to wait for what on

app then speaks with event library

avoids select()

event system agnostic

libcurl multi_socket bindings

not always supported

a matter of writing the glue code correct you can help!

multi_socket example

int main(){CURLM *multi = curl_multi_init();CURL *easy;

curl_multi_setopt(..., CURLMOPT_SOCKETFUNCTION, socket_cb);curl_multi_setopt(..., CURLMOPT_TIMERFUNCTION, timer_cb);

easy = curl_easy_init();curl_easy_setopt(..., CURLOPT_URL, );

curl_multi_add_handle(..., easy);

event_dispatch(); /* start the libevent dispatch loop */}

int event_cb(...){/* event library found event on libcurl's socket, act */curl_multi_socket_action(..., action, ...);}

int socket_cb(...){/* called when libcurl adds, removes or changes a socket that we must deal with */

event_set(..., event_cb); /* tell event lib about setup */}

Scales?

done 70,000+ connections

port numbers are 16 bit

callbacks take time thus limit throughput

Which protocols?

(almost) all network based ones libcurl support! (DICT, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS and TFTP)

Caveats?

yeah sure, but...

c-ares for name resolving

not everything is non-blocking

More factors

Network stack inefficiency

Stack size?

Memory / mallocs

Other code may slow down things

Copy copy copy copy ...

For you who fell asleep

old style socket applications don't scale

use event based concepts

libcurl can too

libcurl is...

entirely open source

in need of your help!

at http://curl.haxx.se/libcurl/

Fscons scalable appplication transfers

Technology

Transcript of Fscons scalable appplication transfers