MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation...
Transcript of MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation...
![Page 1: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/1.jpg)
MultiPath TCP : Linux Kernelimplementation
Presenter: Christoph PaaschIP Networking Lab
Universite catholique de Louvain
August 22, 2012
http://mptcp.info.ucl.ac.be
IP Networking Lab http://mptcp.info.ucl.ac.be 1 / 32
![Page 2: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/2.jpg)
Networks are becoming Multipath
Mobile devices can connect to the Internet via differentinterfaces
3G
WiFi
Internet
Data-centers have a large redundant infrastructure
IP Networking Lab http://mptcp.info.ucl.ac.be 2 / 32
![Page 3: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/3.jpg)
Networks are becoming Multipath
Mobile devices can connect to the Internet via differentinterfaces
3G
WiFi
Internet
Data-centers have a large redundant infrastructure
IP Networking Lab http://mptcp.info.ucl.ac.be 2 / 32
![Page 4: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/4.jpg)
However, the protocols are single-path
TCP is used for 95% of the Internet communications
TCP identifies connections by the 5-tuple
IP: 192.168.1.1 IP: 4.4.4.4
regular TCP
Port: 42424 Port: 80
A single TCP connection cannot be used across differentinterfaces.
IP Networking Lab http://mptcp.info.ucl.ac.be 3 / 32
![Page 5: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/5.jpg)
However, the protocols are single-path
TCP is used for 95% of the Internet communications
TCP identifies connections by the 5-tuple
IP: 192.168.1.1 IP: 4.4.4.4
regular TCP
Port: 42424 Port: 80
A single TCP connection cannot be used across differentinterfaces.
IP Networking Lab http://mptcp.info.ucl.ac.be 3 / 32
![Page 6: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/6.jpg)
However, the protocols are single-path
TCP is used for 95% of the Internet communications
TCP identifies connections by the 5-tuple
IP: 192.168.1.1 IP: 4.4.4.4
regular TCP
Port: 42424 Port: 80
A single TCP connection cannot be used across differentinterfaces.
IP Networking Lab http://mptcp.info.ucl.ac.be 3 / 32
![Page 7: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/7.jpg)
End-hosts don’t use the multipath network
Smartphones have to restart their data-transfer when movingaway from the WiFi access-point.
IP Networking Lab http://mptcp.info.ucl.ac.be 4 / 32
3G
WiFi
Internet
![Page 8: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/8.jpg)
End-hosts don’t use the multipath network
Smartphones have to restart their data-transfer when movingaway from the WiFi access-point.
IP Networking Lab http://mptcp.info.ucl.ac.be 4 / 32
3G
WiFi
Internet
![Page 9: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/9.jpg)
End-hosts don’t use the multipath network
Smartphones have to restart their data-transfer when movingaway from the WiFi access-point.
IP Networking Lab http://mptcp.info.ucl.ac.be 4 / 32
3G
WiFi
Internet
![Page 10: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/10.jpg)
End-hosts don’t use the multipath network
Collisions in data-center reduce the bandwidth and result insuboptimal load-balancing
IP Networking Lab http://mptcp.info.ucl.ac.be 5 / 32
![Page 11: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/11.jpg)
End-hosts don’t use the multipath network
Collisions in data-center reduce the bandwidth and result insuboptimal load-balancing
A B
IP Networking Lab http://mptcp.info.ucl.ac.be 5 / 32
![Page 12: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/12.jpg)
End-hosts don’t use the multipath network
Collisions in data-center reduce the bandwidth and result insuboptimal load-balancing
A B
IP Networking Lab http://mptcp.info.ucl.ac.be 5 / 32
![Page 13: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/13.jpg)
Mismatch between the multipath networkand the single-path transport protocol.
IP Networking Lab http://mptcp.info.ucl.ac.be 6 / 32
![Page 14: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/14.jpg)
Previous work
Transport layer
SCTP
Needs modifications in the applications
Does not pass by most middleboxes/firewalls
Network layer
Mobile IP(v6), shim6, HIP,. . .
Some are only designed for IPv6
Do not pass by middleboxes/firewalls
Are hard to deploy
IP Networking Lab http://mptcp.info.ucl.ac.be 7 / 32
![Page 15: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/15.jpg)
MultiPath TCP
Runs with unmodified applications
Works over today’s Internet
IPv4/IPv6 are both supported (even simultaneously)
IP Networking Lab http://mptcp.info.ucl.ac.be 8 / 32
![Page 16: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/16.jpg)
MultiPath TCP
Standard Stream Socket API
standard Socket API
Transport Layer
MultiPath TCP
Application Layer
IP Networking Lab http://mptcp.info.ucl.ac.be 9 / 32
![Page 17: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/17.jpg)
MultiPath TCP
Multiple TCP subflows to pass middleboxes
standard Socket API
Transport Layer
MultiPath TCP
TCPsubflow
TCPsubflow
TCPsubflow
Application Layer
IP Networking Lab http://mptcp.info.ucl.ac.be 9 / 32
![Page 18: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/18.jpg)
MultiPath TCP
IPv4/IPv6 capable
standard Socket API
Transport Layer
MultiPath TCP
TCPsubflow
TCPsubflow
TCPsubflow
Application Layer
Network Layer
WiFi Wired3G
IPv4 IPv6IPv4
IP Networking Lab http://mptcp.info.ucl.ac.be 9 / 32
![Page 19: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/19.jpg)
MultiPath TCP
Is the server MPTCP-capable?
IP Networking Lab http://mptcp.info.ucl.ac.be 10 / 32
SYNMP_CAPABLE
![Page 20: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/20.jpg)
MultiPath TCP
Is the server MPTCP-capable?
IP Networking Lab http://mptcp.info.ucl.ac.be 10 / 32
SYN + ACKMP_CAPABLE
![Page 21: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/21.jpg)
MultiPath TCP
Create new subflows
IP Networking Lab http://mptcp.info.ucl.ac.be 10 / 32
SYNMP_JOIN
![Page 22: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/22.jpg)
MultiPath TCP
Create new subflows
IP Networking Lab http://mptcp.info.ucl.ac.be 10 / 32
SYN+ACKMP_JOIN
![Page 23: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/23.jpg)
MultiPath TCP
Create new subflows
IP Networking Lab http://mptcp.info.ucl.ac.be 10 / 32
![Page 24: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/24.jpg)
MultiPath TCP
Separate sequence-number space
IP Networking Lab http://mptcp.info.ucl.ac.be 10 / 32
write_seqsnd_cwndrcv_nxt
Subflow 1
write_seqsnd_cwndrcv_nxt
Subflow 2
write_seqsnd_cwndrcv_nxt
Data-level
![Page 25: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/25.jpg)
MultiPath TCP
Separate sequence-number space
IP Networking Lab http://mptcp.info.ucl.ac.be 10 / 32
SubSeq: 1DataSeq: 1000
SubSeq: 200DataSeq: 1001
SubSeq: 2DataSeq: 1002
write_seqsnd_cwndrcv_nxt
Subflow 1
write_seqsnd_cwndrcv_nxt
Subflow 2
write_seqsnd_cwndrcv_nxt
Data-level
![Page 26: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/26.jpg)
MultiPath TCP
Cross-subflow reinjection
IP Networking Lab http://mptcp.info.ucl.ac.be 10 / 32
SubSeq: 200DataSeq: 1001
SubSeq: 3DataSeq: 1001
SubSeq: 1DataSeq: 1000
SubSeq: 2DataSeq: 1002
![Page 27: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/27.jpg)
MultiPath TCP
standard Socket API
Transport Layer
MultiPath TCP
TCPsubflow
TCPsubflow
TCPsubflow
Application Layer
Network Layer
WiFi Wired3G
IP Networking Lab http://mptcp.info.ucl.ac.be 11 / 32
![Page 28: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/28.jpg)
MultiPath TCP
Linux Kernel Implementation
Available at http://mptcp.info.ucl.ac.be
IP Networking Lab http://mptcp.info.ucl.ac.be 12 / 32
![Page 29: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/29.jpg)
Establishing first subflow
Exchanged Messages
SYNMP_CAPABLE
IP Networking Lab http://mptcp.info.ucl.ac.be 13 / 32
![Page 30: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/30.jpg)
Establishing first subflow
High-Level Kernel design
standard Socket API
Transport Layer
Application Layer
Network Layer
Connection EstablishmentIs the Peer MPTCP-Capable?
TCPsubflow
structtcp_sock
socket(AF_INET, SOCK_STREAM, 0);connect(...);
IP Networking Lab http://mptcp.info.ucl.ac.be 13 / 32
![Page 31: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/31.jpg)
Establishing first subflow
In-depth call-stacksocket(AF_INET, SOCK_STREAM, 0);connect(...);
tcp_v4_connect(...);
tcp_connect(...);
tcp_connect_init(...);
Generate a new keyfor this connection
create SYN
tcp_transmit_skb(...);
tcp_syn_options(...);
mptcp_syn_options(...);
tcp_options_write(...);
mptcp_options_write(...);
pass packet to IP-stack
IP Networking Lab http://mptcp.info.ucl.ac.be 13 / 32
![Page 32: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/32.jpg)
Establishing first subflow
Exchanged Messages
SYN + ACKMP_CAPABLE
IP Networking Lab http://mptcp.info.ucl.ac.be 14 / 32
![Page 33: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/33.jpg)
Establishing first subflow
In-depth call-stack
tcp_v4_rcv(...);
tcp_v4_conn_request(...);
tcp_openreq_init(...);
* Store remote-key in mptcp_request_sock* Generate unique local key
SYN
create the request-socket
tcp_v4_send_synack(...);
mptcp_synack_options(...);
SYN/ACK
IP Networking Lab http://mptcp.info.ucl.ac.be 14 / 32
![Page 34: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/34.jpg)
Establishing first subflow
High-Level Kernel design
standard Socket API
Transport Layer
Application Layer
Network Layer
MultiPath TCPMeta-socket
Creating the Meta-socketand linking the application to this socket.
TCPsubflow
structtcp_sock
IP Networking Lab http://mptcp.info.ucl.ac.be 14 / 32
![Page 35: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/35.jpg)
Establishing first subflow
In-depth call-stack
tcp_v4_rcv(...);
tcp_v4_do_rcv(...);
tcp_rcv_state_process(...);
tcp_rcv_synsent_state_process(...);
mptcp_alloc_mpcb(...);
create the meta-socket
mptcp_add_sock(...);
attach master to meta
SYN/ACK
IP Networking Lab http://mptcp.info.ucl.ac.be 14 / 32
![Page 36: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/36.jpg)
Establishing first subflow
Exchanged Messages
ACKMP_CAPABLE
IP Networking Lab http://mptcp.info.ucl.ac.be 14 / 32
![Page 37: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/37.jpg)
Establishing first subflow
In-depth call-stack
tcp_v4_hnd_req(...);tcp_v4_do_rcv(...);tcp_v4_rcv(...);
tcp_check_req(...);
ACK
tcp_v4_syn_recv_sock(...);
Creates the master-socket
mptcp_check_req_master(...);
Creates the meta-socketInitializes everything for MPTCP
Handles the third ackof the 3-way Handshake
IP Networking Lab http://mptcp.info.ucl.ac.be 14 / 32
![Page 38: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/38.jpg)
Establishing first subflow
High-Level Kernel design
standard Socket API
Transport Layer
Application Layer
Network Layer
MultiPath TCPMeta-socket
TCPsubflow
structtcp_sock
IP Networking Lab http://mptcp.info.ucl.ac.be 14 / 32
![Page 39: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/39.jpg)
Establishing additional subflows
Exchanged Messages
SYNMP_JOIN
IP Networking Lab http://mptcp.info.ucl.ac.be 15 / 32
![Page 40: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/40.jpg)
Establishing additional subflows
High-Level Kernel design
standard Socket API
Transport Layer
Application Layer
Network Layer
MultiPath TCPMeta-socket
An additional subflowis being createdTCP
subflow
structtcp_sock
TCPsubflow
structtcp_sock
IP Networking Lab http://mptcp.info.ucl.ac.be 15 / 32
![Page 41: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/41.jpg)
Establishing additional subflows
In-depth call-stackmptcp_init4_subsockets(...);
tp->mptcp->slave_sk = 1;
connect(...);
mptcp_add_sock(...);
Creates a new IPv4 subflow
Thanks to slave_sk headds an MP_JOIN
tcp_v4_connect(...);
tcp_connect(...);
SYN
Attach subflow to meta-sk
IP Networking Lab http://mptcp.info.ucl.ac.be 15 / 32
![Page 42: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/42.jpg)
Establishing additional subflows
Exchanged Messages
SYN+ACKMP_JOIN
IP Networking Lab http://mptcp.info.ucl.ac.be 16 / 32
![Page 43: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/43.jpg)
Establishing additional subflows
In-depth call-stack
tcp_v4_rcv(...);
SYN
tcp_v4_send_synack(...);
mptcp_lookup_join(...);
SYN/ACK
Look for the MP_JOIN in a SYNAnd see if we know this token
mptcp_v4_do_rcv(...);
mptcp_v4_join_request(...);
Similar to tcp_v4_conn_request()Generate the truncated MAC
IP Networking Lab http://mptcp.info.ucl.ac.be 16 / 32
![Page 44: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/44.jpg)
Establishing additional subflows
Exchanged Messages
ACKMP_JOIN
IP Networking Lab http://mptcp.info.ucl.ac.be 16 / 32
![Page 45: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/45.jpg)
Establishing additional subflows
In-depth call-stack
tcp_v4_rcv(...);
ACK
mptcp_check_req_child(...);
Verify the hmac of the ACK
mptcp_syn_recv_sock(...);
Look for a request-sock
mptcp_v4_do_rcv(...);
tcp_check_req(...);
mptcp_add_sock(...);
Attach subflow to meta-sk
IP Networking Lab http://mptcp.info.ucl.ac.be 16 / 32
![Page 46: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/46.jpg)
Sending Data
Exchanged Messages
SubSeq: 1DataSeq: 1000
SubSeq: 200DataSeq: 1001
SubSeq: 2DataSeq: 1002
write_seqsnd_cwndrcv_nxt
Subflow 1
write_seqsnd_cwndrcv_nxt
Subflow 2
write_seqsnd_cwndrcv_nxt
Data-level
IP Networking Lab http://mptcp.info.ucl.ac.be 17 / 32
![Page 47: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/47.jpg)
Sending Data
High-Level Kernel design
standard Socket API
Transport Layer
Application Layer
Network Layer
MultiPath TCPMeta-socket
TCPsubflow
Mastersubsocket
TCPsubflow
Slavesubsocket
send-queue
write(sock, &buf, size);
21 3 4 5
IP Networking Lab http://mptcp.info.ucl.ac.be 17 / 32
![Page 48: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/48.jpg)
Sending Data
High-Level Kernel design
standard Socket API
Transport Layer
Application Layer
Network Layer
MultiPath TCPMeta-socket
TCPsubflow
Mastersubsocket
TCPsubflow
Slavesubsocket
MultiPath TCP Scheduler
send-queue
write(sock, &buf, size);
21 3 4 5
IP Networking Lab http://mptcp.info.ucl.ac.be 17 / 32
![Page 49: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/49.jpg)
Sending Data
High-Level Kernel design
standard Socket API
Transport Layer
Application Layer
Network Layer
MultiPath TCPMeta-socket
TCPsubflow
Mastersubsocket
TCPsubflow
Slavesubsocket
MultiPath TCP Scheduler
send-queue
send-queue send-queue
21 3 4 5
21 3 4 5
write(sock, &buf, size);
IP Networking Lab http://mptcp.info.ucl.ac.be 17 / 32
![Page 50: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/50.jpg)
Receiving Data
Packets can be reordered at the data-level due to delay-differences.
IP Networking Lab http://mptcp.info.ucl.ac.be 18 / 32
SubSeq: 1DataSeq: 1000
SubSeq: 200DataSeq: 1001
SubSeq: 2DataSeq: 1002
![Page 51: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/51.jpg)
Receiving Data
Packets can be reordered at the data-level due to delay-differences.
IP Networking Lab http://mptcp.info.ucl.ac.be 18 / 32
1000
SubSeq: 200DataSeq: 1001
receive-queue
MPTCP-Level
SubSeq: 2DataSeq: 1002
![Page 52: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/52.jpg)
Receiving Data
Packets can be reordered at the data-level due to delay-differences.
IP Networking Lab http://mptcp.info.ucl.ac.be 18 / 32
1000
SubSeq: 200DataSeq: 1001
out-of-order queue
1002
receive-queue
MPTCP-Level
![Page 53: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/53.jpg)
Receiving Data
A loss at the subflow-level (or network-reordering) can also causereordering at the subflow-level
IP Networking Lab http://mptcp.info.ucl.ac.be 19 / 32
SubSeq: 3DataSeq: 1003
SubSeq: 4DataSeq: 1004
![Page 54: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/54.jpg)
Receiving Data
A loss at the subflow-level (or network-reordering) can also causereordering at the subflow-level
IP Networking Lab http://mptcp.info.ucl.ac.be 19 / 32
SubSeq: 4DataSeq: 1004 Packet Dropped
![Page 55: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/55.jpg)
Receiving Data
Subflow-level out-of-order queues are necessary to handle theretransmission at the subflow-level
IP Networking Lab http://mptcp.info.ucl.ac.be 20 / 32
SubSeq: 4DataSeq: 1004
out-of-order queue
Subflow-Level
![Page 56: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/56.jpg)
Receiving Data
High-Level Kernel design
standard Socket API
Transport Layer
Application Layer
Network Layer
MultiPath TCPMeta-socket
TCPsubflow
Mastersubsocket
TCPsubflow
Slavesubsocket
ofo-queuereceive-queue
ofo-queue ofo-queue
read(sock, &buf, size);
IP Networking Lab http://mptcp.info.ucl.ac.be 21 / 32
![Page 57: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/57.jpg)
MultiPath TCP
Performance Evaluation
IP Networking Lab http://mptcp.info.ucl.ac.be 22 / 32
![Page 58: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/58.jpg)
MPTCP on Amazon EC2
2-8 paths available between hosts not on the samemachine/rack
ECMP load-balancing
40 medium CPU instances running MPTCP
During 12 hours, all-to-all iperf with:
TCPMPTCP (2 subflows)MPTCP (4 subflows)
IP Networking Lab http://mptcp.info.ucl.ac.be 23 / 32
![Page 59: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/59.jpg)
MPTCP on Amazon EC2
0
100
200
300
400
500
600
700
800
900
1000
0 500 1000 1500 2000 2500 3000
Thro
ughput (M
b/s
)
Flow Rank
TCPMPTCP, 4 subflowsMPTCP, 2 subflows
IP Networking Lab http://mptcp.info.ucl.ac.be 24 / 32
![Page 60: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/60.jpg)
MultiPath TCP on a Smartphone/Mobile Node
Smartphones have multiple interfaces (WiFi/3G)
MultiPath TCP can be a benefit for these devices
Increased BandwidthIncreased Resilience
IP Networking Lab http://mptcp.info.ucl.ac.be 25 / 32
![Page 61: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/61.jpg)
MultiPath TCP handover from WiFi to 3G
DSEQ: 1100
DSEQ: 1000
IP Networking Lab http://mptcp.info.ucl.ac.be 26 / 32
![Page 62: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/62.jpg)
MultiPath TCP handover from WiFi to 3G
DSEQ: 1100
DSEQ: 1000
IP Networking Lab http://mptcp.info.ucl.ac.be 26 / 32
![Page 63: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/63.jpg)
MultiPath TCP handover from WiFi to 3G
DSEQ: 1100
DSEQ: 1000
REMOVE-ADDRESS
IP Networking Lab http://mptcp.info.ucl.ac.be 26 / 32
![Page 64: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/64.jpg)
MultiPath TCP handover from WiFi to 3G
DSEQ: 1100
Teardown
Reinject 1100
IP Networking Lab http://mptcp.info.ucl.ac.be 26 / 32
![Page 65: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/65.jpg)
MultiPath TCP handover from WiFi to 3G
Regular TCP would break!
Some applications support recovering from a broken TCP(HTTP-Header Range)
Thanks to the REMOVE ADDR-option, MPTCP is able tohandle this without the need for application support.
IP Networking Lab http://mptcp.info.ucl.ac.be 27 / 32
![Page 66: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/66.jpg)
MultiPath TCP handover from WiFi to 3G
1.0 0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0Relative Time w.r.t the Wi-Fi loss [seconds]
0
1
2
3
4
5
6
7
8
9Goodput[Mbps]
Wi-Fi loss
Application Handover
IP Networking Lab http://mptcp.info.ucl.ac.be 28 / 32
![Page 67: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/67.jpg)
MultiPath TCP handover from WiFi to 3G
1.0 0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0Relative Time w.r.t the Wi-Fi loss [seconds]
0
1
2
3
4
5
6
7
8
9Goodput[Mbps]
Wi-Fi lossFull MPTCPApplication Handover
IP Networking Lab http://mptcp.info.ucl.ac.be 28 / 32
![Page 68: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/68.jpg)
Flow-to-core affinity
Flow-to-core affinity
Individual TCP-flows are steered to the same CPU-core toavoid reordering inside the receive-code.
MPTCP has lots of L1/L2 cache-misses because theindividual subflows are steered on different CPU-cores.
MPTCP-aware Receive-Flow-Steering sends all subflows onthe same CPU-core.
CPU 0Core 0 Core 1
CPU 1Core 2 Core 3
L2 cache L2 cache
DRAM
L1 cache L1 cache L1 cache L1 cache
IP Networking Lab http://mptcp.info.ucl.ac.be 29 / 32
![Page 69: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/69.jpg)
Flow-to-core affinity - 10 Gbps interfaces
0 10 20 30 40 50 60 70
Number of iperf-sessions0
2
4
6
8
10
12
14
Ave
rage
Goo
dput
inG
bps
MPTCP
IP Networking Lab http://mptcp.info.ucl.ac.be 30 / 32
![Page 70: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/70.jpg)
Flow-to-core affinity - 10 Gbps interfaces
0 10 20 30 40 50 60 70
Number of iperf-sessions0
2
4
6
8
10
12
14
Ave
rage
Goo
dput
inG
bps
MPTCPMPTCP-aware RFS
IP Networking Lab http://mptcp.info.ucl.ac.be 30 / 32
![Page 71: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/71.jpg)
Conclusion
MultiPath TCP increases the bandwidth andallows seamless mobile handover.
MultiPath TCP can be used with unmodifiedapplications, over today’s Internet.
IP Networking Lab http://mptcp.info.ucl.ac.be 31 / 32
![Page 72: MultiPath TCP : Linux Kernel implementation · MultiPath TCP : Linux Kernel implementation Presenter: Christoph Paasch IP Networking Lab Universit e catholique de Louvain August 22,](https://reader033.fdocuments.net/reader033/viewer/2022042600/5ace376e7f8b9a27628e916d/html5/thumbnails/72.jpg)
Conclusion
Freely available at http://mptcp.info.ucl.ac.beDownload it, try it out, contribute!
UCLouvain MPTCP-Team:Christoph Paasch
Gregory DetalFabien Duchene
Prof. Olivier Bonaventure
Thanks to our previous and present partners/contributors:
IP Networking Lab http://mptcp.info.ucl.ac.be 32 / 32