TheTransportLayerisDead–LongLivetheTransportLayer!

MichaelWelzlUniversityofOslo/UniversityofRomeTorVergata

15.01.20181

MichaelWelzlUniversityofOslo/UniversityofRomeTorVergata

15.01.20182

Wakeupcall• Thetransportlayerisinterestingagain!

– Somehow,(mostlyUS?)industryisaware,butacademia+(European?)fundingbodiesdon'trecognize;onlyexception:MPTCP

• Longlistofnew,interestingstuff!– HTTP/2+QUIC;LEDBAT;WebRTC(withRMCAT congestioncontrols)– Focusonlowlatency:Bufferbloatcommunity=>AQMalgorithms:

(FQ-)CoDel,PIE andnowalsosomeWiFiwork;ECN:ABE,L4S

• Whyhastransport"gonecold"?– Noproblemtosolve,TCPworks? nottrue! TCP=latency,complexity,bad

fairness,badinteractionswithdelay-sensitiveornon-greedytraffic,...– ChangingTCPisfrustratinglyhard,elsenochancetochangeanything?

Notsotrueanymore!=>storyofthistalk! 3

Outline

1. The"transportlayerossification"problem

2. Thesolution1. IETFTransportServices(TAPS)WG2. NEATECproject(andsoftware!)

4

Terminology

5

Ossification Petrification

TheTransportLayerisDead....

(theproblem)

6

Thebeauty;-)ofOSI...

• AN-layerprovidesaservicetotheN+1-layer– what'sbelowtheserviceinterfaceishidden

•

• Protocolsinsidelayersareinterchangeable

• Transportlayer(andabove)shouldbeespecially easytochange!

node A node B

protocol Layer N+1

Layer N: services N1, N2

service interface

entitiy vertical comm.

horizontal comm.

N1

N2

N1

N2

7

Internettransportisossified

è Applicationsexpectprecisely thebehaviorofTCPandUDP

node A nodeB

e.g.HTTP Application Layer

TransportLayer

entityUDP

TCP

è Routersexpecttoseeprecisely TCPorUDP(sometimeswithevenmorelimitations)

8

Transportlayerdevelopment• Computer,30yearsago

• AprogramthatsendsdataovertheInternet,30yearsago

(usingBerkeleysockets)

9

Transportlayerdevelopment• Computer,today

• AprogramthatsendsdataovertheInternet,today

(usingBerkeleysockets)

10

Morethanjustsockets...

• Previousexample:Berkeleysockets– Today,applicationsusemanydifferentAPIsfromvariouslibraries/middlewares

• Buttheselibraries/middlewaresuse socketsè limitedbytheservicesofTCPandUDP1. Areliablebytestream2. Unreliablepacket(“datagram”)transmission

It’sawayofthinking aboutcommunication!11

...andit’sfromthe80s!

1) Reliablebytestream,TCP

2) Datagram,UDP

12

Whatabout...

• Priorities?– Amongyourownflows,andbetweenyoursandothers?

• Careful“donotdisturbanybody”background(“scavenger”)communication?

• Intelligentuseofmultiplenetworkinterfaces?

• Maybeusingdataevenwhenachecksumfails?– Codecscanhandlethat,butUDPwillthrowawayallyourdata

• Usingprotocolfeaturesthatyieldlowerlatency?– Maybetradelatencyagainstbandwidth?– Controlthesendbuffer?

13

Example:unordered message delivery

• Some applications can accept data chunks outof order,even when requiring reliability– TCPcauses Head-Of-Line(HOL)blocking delay

Chunk 2 Chunk 3 Chunk 4 Chunk 1TCP receiver buffer

App waits in vain!

• Some protocols can deliver data outof order (e.g.SCTP)• If suchaprotocol is notavailable:fine to use TCP!

• in-orderdelivery is correct!Justslower

...BUT:unless anapplication asks for it,we can NEVERgiveitdatainthe wrong order.è Thisservice needs to be inALLAPIs!(notjustsocketlevel)

14

...LongLiveTheTransportLayer!

(thesolution)

15

Solutionpart1:AStandard.

IETFTransportServices(TAPS)

16

TAPSBackground

Application

Protocol X

ISP A (supports

X only)

Application

Protocol X

ISP A (supports

X only)

TAPS system

Application

Protocol Y

ISP B (supports X and Y)

TAPS system

Application

Protocol X

ISP B (supports X and Y)

without TAPS with TAPS

• mid-2013,thetimehadcome:LEDBAT,RTMFP,MPTCP,SPDY,Minion• Canwegetsomeorderintothischaos?

• 1yearofcommunity-convincing;1barBoF;2BoFs=>TAPSchartered24.9.2014• Historyavailableat:https://sites.google.com/site/transportprotocolservices/17

HowTAPSwantstosolvetheproblem

• Unorderedreliablemessagedeliveryisonlyoneexample

• Whatistheminimumsetoftransportservices...– fromtheservicesthatIETFtransportprotocolsoffer...thatwe“must”makeavailableeverywhereforthemtobecomeusable?

• Onceweknowthat,wecanspecifyhowtoimplementaTAPSsystem

18

TAPScharter:plannedoutcomes

1. Listofservices providedbytoday’stransports

2. List:subsetofservices thatsystemssupportingTAPSwillprovide+guidanceonchoosingamongavailablemechanismsandprotocols

3. Experimentalspec:mechanismstoprovidetheservicesidentifiedinitem2(“Thisdocumentwillexplainhowtoselectandengageanappropriateprotocolandhowtodiscoverwhichprotocolsareavailablefortheselectedservicebetweenagivenpairofendpoints.Further,itwillprovideabasisforincrementaldeployment.”)

19

Bottom-upmethodtoderiveaminimumsetoftransportservices

• IETFTransportServices(TAPS)WorkingGroup1. Survey ofservicesprovidedbytransportprotocols– RFC

8095(54pages)2. In-depthanalysisof30+RFCs:TCP,MPTCP,UDP,UDP-

Lite,SCTP,LEDBAT (RFCs-to-be8303&8304~79pages)3. Derivationofaminimumset:from2),keeponlyservices

thatrequireapplication-specificknowledgeanddonotprohibitfallingbacktoTCP/UDP;resolvesomeresultingpeculiarities(1Internet-draft,53pages)

20

Fall-backsenableone-sideddeployment

SeveralProtocols

NewAPI

NewApp NewProtocolX

NewAPI

NewApp

TCP

NewAPI

NewApp

UDP

NewAPI

NewApp

TCP

Sockets

CurrentApp

UDP

Sockets

CurrentApp

21

Result(latestversion):high-levelview

1. Flowcreationandflowgrouping– Mostconfigurationfeaturesonlyconfigureagroup(e.g.,can’tconfiguretimeoutforanSCTPstreamonly)

– Shoulduseconfigurationearly,ideallybeforeconnecting

2. Send-before-connect– Couldalsobeimplementedasaparameterto“connect”– Specify“idempotentdata”forextra-fasttransmission(TFO)

3. Limitedconnect/listen/close/abortsemantics(supportUDP,ortransparentlyusestreams)– Connectmaynotinvokeaccept!– Nohalf-closedconnections 22

Note:assumeTCPfall-backonly(assumingUDPfall-backlimitsthislist)

Result(latestversion):high-levelview/2

• Beinformedaboutsenderbufferrunningdry– Latedecisionaboutchoiceofdatatosend

• Configure:per-flowpriorities,network-widepriorities(DSCP),timeout,authentication,checksums

• Somequeries,e.g.relatedtopacketsizes• Sendmessages,receivebytestream

– Messagesstayintact,butmaybereorderedordropped(configuredbysender)

– Receivermustbeaware(detectboundaries)23

Movingtoasystemspecification...

• TAPScharteritem3:currentlyvariousdocuments/proposals– post-sockets:TLSfeatures,connectionmigration,variousotherextras,relatedtoApple'ssoon-to-be-shippedcode

– guidelinesforTAPSsystems,securitysurvey– NEATprojectoutputs(happyeyeballs,API,..)– socketintents(mostlyfocusedoninterfacechoice)

24

Solutionpart2:Code.

Theproject(andsoftware)

25

WhatdoesNEAToffer?• Auser-spacelibrary forefficiente2enetworkInternetcomm.,designedtobeeasytouse– Thinkofitas"TAPSminimumset++"

• Easy,platform-independentaccesstothefeaturesofTCP,MPTCP,SCTP,UDP,UDP-LITEandmoreresearchythings viaacallback-basedAPI– Policysystemtocontroleverythingviajsonfiles

• One-sideddeploymentpossible,candofall-backstoTCPorUDP,cantalktonativepeers(e.g.TCP,SCTP,evenWebRTCbrowser!)

26

Internals:sequenceofeventsinNEAT

27

Happy

Eye-balls

28

Moredeploymentconsiderations• “Application”couldbealibraryormiddleware

– e.g.,pub-subdoesn’tneed100%reliability=>interestingresearchpossibilities("networkingmeetsdistributedsystems")

– wouldonlyexploitasubsetofTAPScapabilities,butre-compilinganappwiththenewmiddlewareversionwouldmaketheappuseTAPS

• RelatedNEATdevelopment:NEATsocketAPI– RunlegacyapplicationsoverNEATusing"withneat commandname params"– Benefitfrom:policysettings;transparentSCTP-stream-mapping

Non TAPS-enabled application

M's API

Legacy transport(TCP, UDP)

M's internals

Non TAPS-enabled application

M's API

Available transports (TCP, UDP, Minion,

SCTP...)

M + TAPS internalsMiddleware M Middleware M

29

Conclusion

• Wehave:– IETFTAPS– Apple– TheNEATproject,includingMozillaandthemainSCTPdevelopers

• ....allworkingtowardsareal transportlayer– Newflexibility,newresearchpossibilities

• PleasejointheNEATeffort– download,test,extendandplay!

30

node A node B

protocol Layer N+1

Layer N: services N1, N2

service interface

entitiy vertical comm.

horizontal comm.

N1

N2

N1

N2

Stuff!

• TAPS:– https://datatracker.ietf.org/wg/taps/

• NEAT:– http://www.neat-project.org– https://github.com/NEAT-project/neat– "NEAT:APlatform- andProtocol-IndependentInternetTransportAPI",

IEEECommunicationsMagazine55(6),2017.– "De-ossifyingtheInternetTransportLayer:ASurveyandFuture

Perspectives",IEEECommunicationsSurveys&Tutorials19(1),Firstquarter2017.

31

Questions,comments?

32

Backupslides

33

Sendingmessages,receivingabytestream

• Canwemakethiscombinationwork?– BecompatibletoTCPbutstillbenefitfrommessages?

• Alternativenotveryattractive:alwaystellinganapplication“sorry,youonlygetastreamhere”isnotmuchdifferentthansaying“sorry,useTCPinstead”– Let’sminimize#hoopsanappdeveloperhastojumpthrough

• Message-orientedTCPappsalreadyframetheirdata– Unnecessarytorepeatthisintransportlayer– Requirementtotellreceiverapp“hereisyourcompletemessage”createsa

majorlimitationandisoftenunnecessary

34

Application-Framed(AFra-)Bytestream

• NormalTCP-likebytestream API– Optional:someadditionalinformationprovidedbysenderapp

• Senderapp: handsoverastreamofbytes,informstransportaboutframeboundariesandrequirements(order,reliability,..)– Delimitedframesstayintact,inorder– Morerelaxedrulespossiblebetween frames– Delimitersassumedtobeknownbyapplication

• Receiverapp:receivesstreamofbytes– App-leveldelimitersturnitintomessages

• TCP=specialcase:nodelimitersused– Cantalkto“normal”TCPapplicationsonbothsides

35

Unorderedmessagedelivery:SCTP

36

API API

Msg 3

Msg 2

Msg 1

SenderappMsg 1

Msg 3

Msg 2

Receiverapp

• Informwhereframebegins• Configure:“unordered”

Block1Block3 Block2

App-definedheader.Couldalsobee.g.implicitknowledgeaboutsize

Justabytestream!

Appknowshowtoidentifymessages

Block1Block2 Block3

• Informwhereframeends

Unorderedmessagedelivery:TCP

37

API API

Msg 3

Msg 2

Msg 1

SenderappMsg 1

Msg 2

Msg 3

Receiverapp

• Informwhereframebegins• Configure:“unordered”

… TCPjustignoresthis!

Block1Block3 Block2

Justabytestream!

Appknowshowtoidentifymessages

Block1Block2 Block3

• Informwhereframeends

Unreliableunorderedmsg delivery:SCTP

38

API API

Msg 3

Msg 2

Msg 1

SenderappMsg 3

Msg 2

Receiverapp

• Informwhereframebegins• Configure:“unreliable,unordered”

Block1Block3 Block2

Justabytestream!

Appknowshowtoidentifymessages

Block2 Block3

• Informwhereframeends

Unreliableunorderedmsg delivery:TCP

39

API API

Msg 3

Msg 2

Msg 1

SenderappMsg 1

Msg 2

Receiverapp

• Informwhereframebegins• Configure:“unreliable,unordered”

… TCPjustignoresthis!

Block1Block3 Block2

Justabytestream!

Appknowshowtoidentifymessages

Block2 Block3

Block1

Msg 3

• Informwhereframeends

Unreliablemessagedelivery:SCTP,largemessages

40

API API

Msg 3

Msg 2

Msg 1

Senderapp Receiverapp

• Informwhereblockbegins• Configure:“Unreliable”

Block1Block3 Block2

Packets

• Informwhereframeends

Unreliablemessagedelivery:SCTP,largemessages

41

API API

SenderappMsg 2

Receiverapp

Justabytestream!

Appknowshowtoidentifymessages

Block1Block3 Block2

Packets

SCTP

Init:

exampledecisiontree

-Willyouneedsomeformofreliability?Yes: SCTPorTCP canbeused.- Isanyofthefollowingusefultotheapplication?*Choosingascheduler,choosingpriorities*Configurablemessagereliability*Unorderedmessagedelivery*RequestnottodelaymessageSACKsYes:SCTPispreferred.No:- Isanyofthefollowingusefultotheapplication?*Handoveramessagetoreliablytransfer(possiblymultipletimes)beforeconnectionestablishment*Suggesttimeouttothepeer*NotificationofExcessiveRetransmissions(earlywarningbelowabortionthreshold)*NotificationofICMPerrormessagearrivalYes:TCPispreferred.No:SCTPandTCPareequallypreferable.

No: allprotocolscanbeused.- Isanyofthefollowingusefultotheapplication?*Specifychecksumcoverageusedbythesender*Specifymin.checksumcoveragerequiredbyreceiverYes:UDP-Liteispreferred;No:UDPispreferred. 42

Avoids,e.g.:1)chooseUDP2)appasksforreliability

TCP UDP SCTP

APP Class 0 APP Class 1 APP Class 2 APP Class 3

TCP Minion Experimental Mechanisms

Traditional Socket NEAT Socket

Middleware

NEAT Framework

NEAT User API

NEAT APP Support API

NEAT Policy

ManagerUSER

KERNEL

Policy Information

Base

Characteristic Information

Base

Policy Interface

SCTP/UDP

APP Class 4

PCAP RAW IP Experimental Mechanisms

KPI

Selection Components

H and SComponents

NEAT APP Support Module

IP

DIAG & STATS

NEAT Kernel Module

Policy Interface

Transport Components

SCTP/UDP

SPUD/UDP…

Userspace TransportExp

Mech

NEATarchitecture

43