Exploring the Possibilities of

Sencha & WebRTC Alexandru Lazar - Engineer, ModusCreate Rich Waters - Chief Software Architect, Extnd LLC Tobias Uhlig - Solutions Engineer, Sencha

@alexlazar86@rwaters@tobiasuhlig

Monday, September 23, 13

Monday, September 23, 13

Ext JS DataView

http://docs.sencha.com/extjs/4.2.1/extjs-build/examples/view/data-view.html

Monday, September 23, 13

https://code.google.com/p/css-vfx

Charles Ying (2009)

Snow Stack

Monday, September 23, 13

Best of both worlds

Demo Time

Monday, September 23, 13

Ext JS and HTML5

Monday, September 23, 13

Circles Component

Monday, September 23, 13

And Sencha Touch?

Monday, September 23, 13

Custom Drag & Drop

Monday, September 23, 13

Coding Session

Monday, September 23, 13

Coding Session

Monday, September 23, 13

Animations for Ext JS

Monday, September 23, 13

Animations for Touch

Monday, September 23, 13

What is WebRTC ?

A new communication

Platform

Open Source

Javascript API that enables Real Time

Communication(RTC)

Monday, September 23, 13

The need for WebRTC

•Reduce infrastructure requirements

•Need for speed & quality

•Live streaming & data transfers

•Easy Video/Audio chat sessions

Monday, September 23, 13

Business Perspective

ServerCloud

Client Client

ServerCloud

Client ClientWebRTC

HttpRequest/WS HttpRequest/WS

Monday, September 23, 13

Advantages

• Higher performance

• No server latency

• Less server-side infrastructure

• Easy to implement

• Privacy & Security

• No browser plugins needed

Monday, September 23, 13

Disadvantages• Not fully implemented yet

• Specification still in Draft

• Cross-browser support very limited & requires polyfill

• Broadcast activity implementation becomes much more complex & quirky

• Data persistency

Monday, September 23, 13

Browser Compatibility

• PC

• Google Chrome 23

• Mozilla Firefox 22

• Mobile - only Android for the moment

• Google Chrome 28 (Enabled by default since 29)

• Mozilla Firefox 24

• Bowser (Ericsson Labs)

Monday, September 23, 13

Browser Compatibility

Monday, September 23, 13

Key Features

• Media Streams

• Peer Connection

• Data Channels

Monday, September 23, 13

Media Streamnavigator.getUserMedia ( constraints, successCallback, errorCallback );

Monday, September 23, 13

Peer Connection

• API for establishing audio/video call sessions

• The Built-in concepts: p2p, codec control, encryption and bandwidth management

• 2 Main Protocols:

• ROAP - RTCWeb Offer/Answer Protocol

• JSEP - JavaScript Session Establishment Protocol

Monday, September 23, 13

Peer Connection

• To start a session the client needs:

• Local Session Description

• Remote Session Description

• Remote Transport Candidates

• Caller - Offer

• Callee - Answer

Monday, September 23, 13

Signaling Flow

Browser

Caller

Application

Server/Cloud

Browser

Callee

Application

App Level

WebRTC

Signaling

Session DescriptionSession Description

Media Flow

Signaling

Monday, September 23, 13

Signaling Flow #1

Browser

Caller

Application

Server/Cloud

Browser

Callee

Application

App Level

WebRTC

Session Description

Session Description

Caller sends the Offer

Monday, September 23, 13

Signaling Flow #2

Browser

Caller

Application

Server/Cloud

Browser

Callee

Application

App Level

WebRTC

Session Description

Session Description

Callee receives the Offer

Monday, September 23, 13

Signaling Flow #3

Browser

Caller

Application

Server/Cloud

Browser

Callee

Application

App Level

WebRTC

Session Description

Session Description

Callee Sends the Answer

Monday, September 23, 13

Signaling Flow #4

Browser

Caller

Application

Server/Cloud

Browser

Callee

Application

App Level

WebRTC

Session Description

Session Description

Caller receives the Answer

Monday, September 23, 13

Signaling Flow #5

Browser

Caller

Application

Server/Cloud

Browser

Callee

Application

App Level

WebRTC

Connection Established - Media Flows

Media Flow

Monday, September 23, 13

Peer Connection

//Config peer connection & add streamPeerConnection(config, iceCallback);addStream(stream);

//Create the local session description& apply itcreateOffer(args);setLocalDescription(type, desc);startIce();

.

.

. --- wait for the Callee to respond ---. ..

//Got the Remote descriptionsetRemoteDescription(type, desc);

-- Receive the call from caller --

//Create Peer connection & set descriptionPeerConnection(config, iceCallback);setRemoteDescription(type, desc);

//Create local session desc & apply itcreateAnswer(args);setLocalDescription(type, desc);startIce();

Caller CalleeTimeline

Monday, September 23, 13

How does communication work?

• Connections to remote peers are established using NAT-traversal technologies such as ICE, STUN, and TURN.

• NAT - network address translation

• ICE - Interactive Connectivity Establishment

• STUN - Session Traversal Utilities for NAT

• TURN - Traversal Using Relays around NAT

• Sending the locally-produced streams to remote peers and receiving streams from remote peers.

• Sending arbitrary data directly to remote peers.

Monday, September 23, 13

NAT (Network Address Translation)

• The process of modifying IP address information into IPv4 Headers while in transit across a traffic routing device (address:port)

Monday, September 23, 13

ICE (Interactive Connectivity Establishment)

• The protocol used by NAT Traversal to create Offer/Answer Protocols.

• Uses STUN or TURN protocols

• Usage:

• Internet applications of Voice over Internet Protocol (VoIP)

• p2p communications

• video, instant messaging(chats) and other interactive media

• NAT transversal is an important component to facilitate communications involving hosts on private networks, often located behind firewalls.

Monday, September 23, 13

STUN (Session Traversal Utilities for NAT)

• It is a client-server protocol

• Returns the public IP to a client + information from which the client can infer the type of NAT it sits behind.

• Used to permit NAT transversal for applications.

• Intended to be a tool to be used by other protocols such as ICE (previous slide)

Monday, September 23, 13

TURN (Traversal Using Relays around NAT )

• Places a third party server to relay messages between two clients where peer to peer media traffic is not allowed by a firewall.

• Turns the tables so that the client on the inside can be on the receiving end, rather than the sending end, of a connection that is requested by the client.

• Provides the same security functions provided by symmetric NATs and firewalls

Monday, September 23, 13

Create the RTCPeerConnection

Monday, September 23, 13

Create the Offer

Monday, September 23, 13

Create the Answer

Monday, September 23, 13

Create the ICE Candidate

Monday, September 23, 13

Modus Talk Demo

Monday, September 23, 13

Multiple Peers

• Mesh

• Chrome has hard limit of 256 peer connections

• Maximum bandwidth used by each video RTP port (media-track) is about 1MB by default though we can force lower quality

• sdp = sdp.replace( /a=mid:video\r\n/g , 'a=mid:video\r\nb=AS:256\r\n'); // Cap outgoing bandwidth at 256Kb/s

• Repeater

Monday, September 23, 13

Sample Multi Peer Flow

callUser(userId)

onAcceptCallinitiateVideoCallcreateRoom-- Create PeerConnection, query local media, attach stream & become marked as broadcaster --

-- Invite 2nd Peer ---- Repeat above flow minus createRoom --

-- Once 2+ peer video begin streaming --announceNewParticipant(room)

---- Peer #1 ----onIncomingCallacceptCall

onInitiateVideoCall-- Create PeerConnection, query local media & attach stream --

---- Peer #2 ------ Repeat above flow --

---- Peer #1 ----onNewParticipant-- Create another PeerConnection instance, establish separate signaling channel & begin exchanging SDP, attach local stream to new Peer --

Caller CalleeTimeline

Monday, September 23, 13

Live Streaming (1-n)

Monday, September 23, 13

S-Circles Demo #2

Monday, September 23, 13

Sources

• http://www.webrtc.org/

• http://www.w3.org/TR/2013/WD-webrtc-20130910/

• http://en.wikipedia.org/wiki/Interactive_Connectivity_Establishment

• http://en.wikipedia.org/wiki/STUN

• http://en.wikipedia.org/wiki/Network_address_translation

• https://dvcs.w3.org/hg/FXTF/raw-file/tip/filters/index.html

Monday, September 23, 13

Code Resources• http://peerjs.com/

• Simplified communication & provided signaling server (no streaming support)

• https://www.webrtc-experiment.com/

• Examples of video chat, screen sharing & DataChannels

• https://code.google.com/p/webrtc/source/browse/trunk/samples/js/base/adapter.js

• Chrome/Firefox polyfill

Monday, September 23, 13

Questions?

Monday, September 23, 13