in the real world of real time: Data Channels€¦ · Sharefest Lessons Learned 1. Unreliable is...
33
Data Channels http://slid.es/whadar/data-channels in the real world of real time: Hadar Weiss CTO and Co-Founder, Peer5 Hacker @Sharefest // // twt:whadar [email protected]
Transcript of in the real world of real time: Data Channels€¦ · Sharefest Lessons Learned 1. Unreliable is...
DataChannels
http://slid.es/whadar/data-channels
intherealworldofrealtime:
HadarWeissCTOandCo-Founder,Peer5Hacker@Sharefest
// //twt:whadar [email protected]
Again,whyisthisso
huge
ThewebmeanttobeDecentralized
Inrealitywearequitecentralizedduetothefact:
webbrowserscanonlycommunicatewithservers
(HTTP,WebSockets)
WebRTCshouldmakethewebfullydistributed
cool,doyouhaztehcodez?(actuallyit'snotthatsimple)
Tasks:SecuritySignallingNATtraversalDecideonreliablevsunreliableP2PencapsulationandserializationBrowsersupportandinteroperabilityWebRTChacksP2PcoordinationHTTPaugmentation
ReliableorUnreliable(orpartiallyreliable)
SCTP!(StreamControlTransmissionProtocol)
TCPvsUDPvsSCTP
HighPerformanceBrowserNetworking-byIlyaGrigorik
SCTPSamplevarchannel_confs={TCPLIKE_CHANNEL:{},//reliable,orderedUNORDERED_RELIABLE:{ordered:false},ORDERED_PARTIAL_3ATTEMPS:{ordered:true,maxRetransmits:3},UNORDERED_PARTIAL_3ATTEMPS:{ordered:false,maxRetransmits:3},ORDERED_PARTIAL_1S:{ordered:true,maxRetransmitTime:1000},UNORDERED_PARTIAL_1S:{ordered:false,maxRetransmitTime:1000},UNORDERED_UNRELIABLE:{ordered:false,maxRetransmits:0}}
varpc=newRTCPeerConnection(servers)//don'tdo{RtpDataChannels:true}vardc=pc.createDataChannel("dc_label",channel_confs.UNORDERED_PARTIAL_3ATTEMPS);
UnderlyingProtocols
HighPerformanceBrowserNetworking-byIlyaGrigorik
P2PMulticasting
Why?
FileSharing
VideoStreamingRadio
VOD
live
gaming
Whynotjustservers
How
GiganticMesh!
MatchingAlgorithm1. AvailableChunks2. Location3. ISP4. LocalNetwork5. AvailableBandwidth6. AvailableCPU7. #ofconnectedpeers8. Expectedpeerlifetime9. WebRTCspecific
1. Browservendor2. Browserversion
10. Manyotherfactorsyoucanconsidersuchasnetworkcongestion,mobiledataplan,shareratio,etc.
MoreP2PlogicWhatpeersarethereintheswarm?
WhatpeerscanIconnectto?
Whatparts(chunks)oftheresourceeachpeerhas?
Whichpeersareavailabletosendmedata?
DoIreallygetthechunksthatIaskedfor?
AmIgettingchunksinthebestspeed?
Coordinationisneeded#1metadataencapsulation
/****@paramswarmId*@paramseedertrueifseeder*@paramcompletetrueifavailabilityMap,falseifupdate*@paramblockIdsinlistofchunks,oravailabilityMap*@constructor*/functionHave(swarmId,seeder,availabilityMap,blockIds){this.tag=exports.P2P_HAVE;this.swarmId=swarmId;this.seeder=seeder;this.blockIds=[];if(!seeder){if(availabilityMap){this.complete=truethis.availabilityMap=availabilityMap;}else{this.complete=falsethis.blockIds=blockIds;}}}//swarmId(4bytes)//chunkIds-optional-arrayofidseach4bytes//Exampleof2chunksencoding:0x010203040A0B0C0D.0x01-0x04arethefirstencodedchunk,0x0A-0x0Darethesecondencodedchunk)functionRequest(swarmId,chunkIds){this.tag=exports.P2P_REQUESTthis.swarmId=swarmId;if(!chunkIds)chunkIds=[];this.chunkIds=chunkIds;}
//attributes(ordered)
Coordinationisneeded#2(serialization)
varp2p_have_encode=function(message){varbuffers=[];varswarmId=ascii2ab(BinaryProtocol.packSwarmId(message.swarmId));buffers.push(swarmId);varseeder=BoolToUInt8Array(message.seeder);buffers.push(seeder);varcomplete=BoolToUInt8Array(message.complete);buffers.push(complete);if(message.complete){buffers.push(message.availabilityMap);}else{varblockIdsEncoded=UInt32ToUInt8Array(message.blockIds);buffers.push(blockIdsEncoded);}
returnBinaryProtocol.concat(buffers);};varp2p_have_decode=function(buffer,index,length){varblockIds;varavailabilityMap;varswarmId=ab2ascii(buffer.slice(index,index+BinaryProtocol.transferedSwarmIdSize));index+=BinaryProtocol.transferedSwarmIdSize;length-=BinaryProtocol.transferedSwarmIdSize;
varseeder=UInt8ArrayToBool(buffer,index);index++;length--;
if(!seeder){varcomplete=UInt8ArrayToBool(buffer,index);index++;length--;if(complete){
WebRTCMulticastingisevenharder
Allinjavascript(forgoodandbad)
HighchurnUsersenterandleaverapidlyNode.jshelps
Browsercompatibility,interopissuesSomeusershaveunsupportedbrowsers-IE,Safari,OperaSomebrowsersdon'tlikeotherbrowsers-Chrome,Firefox
Sharefest
EasiestfilesharingOpensource(jointhegrowingcommunity )Anyonecancreatea"torrent"UsesmanyHTML5apis
1. FileAPI2. FilesystemAPI3. IndexDB4. WebSockets5. And...WebRTCDataChannels
ongithub
SharefestArchitecture
SharefestArchitecture(meshnetwork)
SharefestLessonsLearned1. Unreliableisusable2. Chromememorymanagementisproblematic3. Latencyofhandshakeislong!4. WebRTChasquirks1. SDPhacks2. Overcapacityconnectionsthrowerrors3. Manyotherswereresolved
5. 256connectionsperuseristhehardlimit-butyourchromewillprobablycrashsooner
6. Filesizeisalimit,butFSAPIcanhelpifthere'ssomefreespaceattheendpoint
7. Binaryserializationofchunksandmetadataoverbase64isheavy
Sharefestnumbers1. 91%oftheconnectionsareSTUN2. Medianp2pconnectionyield90KB/s3. Localnetworksusuallyyield2MB/s4. Units
1. ChunkSize=800bytes2. BlockSize=1MB
isusefulchrome://webrtc-internals/
LastConclusionfromSharefest:
Node.jsisyourfriend
Peer-assisted(orHTTP-assistedifyouarerealp2pfan)
HybridApproaches1. Aggressive--UseP2PandHTTPallthetime.Goodforfiledownloading
2. Server-friendly--UseP2Pasmuchaspossible.Goodforvideostreaming.Requiresclientsidelogic
3. P2Pfriendly--Useserversasmuchaspossiblewhenthere'sacapacityproblem,useP2P.GoodforwebsitesthatwanttouseP2Pasan"insurance"
Peer5XHR-likeAPIAnAPIcompatiblewithW3CXMLHttpRequest(AJAX):
Withthepowerofthepeerstoassistseamlessly(seeonpeer5.com)
varxhr=newXMLHttpRequest();//(takenfromhtml5rocks.com)xhr.open('GET','/path/to/image.png',true);xhr.responseType='blob';
xhr.onload=function(e){if(this.status==200){//Note:.responseinsteadof.responseTextvarblob=newBlob([this.response],{type:'image/png'});...}};xhr.send();
//ctortakesoptionalparameterstocontrolhybriddownloadingvarrequest=newpeer5.Request({downloadMode:'aggressive'});request.open('GET','http://path/to/a/big/file.json');request.onload=function(e){console.log(this.response);...};
request.send();
[email protected]//sharefest.me
