Recording and media manipulation of WebRTC streams
-
Upload
luis-lopez -
Category
Internet
-
view
906 -
download
3
description
Transcript of Recording and media manipulation of WebRTC streams
Developing WebRTC applications with advanced media processing
Luis Lopez
www.kurento.org
WebRTC infrastructures: at the heart of the long tail
Peer-to-Peer WebRTC Application (without media infrastructure)
WebRTC video stream
WebRTC Application based on media infrastructure
media infrastructure
WebRTC media servers: the simplest WebRTC infrastructure
• Media server– Monolithic software
– Based on shared memory
– Scales well vertically
– Can’t scale horizontally
• Media server vendors– Software vendors
– Hardware vendors
WebRTC media server vendors
Common functions of WebRTC media serversTranscoding media server
VP8 H.264
MCU/SFU media server
Recording media server
Evolution of media servers: functionalityWhat common WebRTC Media Servers do:• Transcoding• MCU• Recording
What future MediaServers will do:• Flexible processing• Augmented reality• Blending• Mixing• Analyzing• Etc.
Media ishere
Media goesthere
TranscodingMCU/SFURecording
Current media servers
Media ishere
Rich Mediagoes there
Future media servers
Media Events
Transcoding, MCU/SFU,Recording,
Enrich,Augment, Analyze,
Combine,Transform,Adapt, …
ContextContent
Commands
Why is this important?
Implicit augmentationwith external knowledge
Explicit augmentationwith target information
Why do you communicate? To see other’s face?
WebRTC convergence with TelcoWhy don’t converge with Content?
Kurento Media Server: an open source software WebRTC media server
SendReceiveAnalyze
AugmentEnrich
TransformTranscode
RecordProcess
Replicate
MediaSource
MediaSink
KMS
JavaAPI
JavaScriptAPI
Otherlanguages
Applications define the processingof streams getting through KMS
What does Kurento mean?
Kurento History
2012
•Kurento goes open source LGPL v2.1. Professional services provided by Naevatec(startup)
June’13
•1M€ of investment for providing multimedia capabilities to FI-WARE
Dec’13
•Kurento Featured at WebRTC Expo (Paris)
Feb’14
•3.3M€ of investment for making Kurento an elastic cloud platform (NUBOMEDIA)
April’14
•Kurento featured at IMS World Conference (Barcelone)
July’14
•Kurento featured at DevCon5 (New York)
Sep’14
•500K€ of investment for maintenance of Kurento FI-WARE APIs
Kurento Client API
Sink
SRC Sink
SRC
SRC
Sink
Sink
Media Element
• Provides a specific media
functionality
› Send/receive media
› Process media
› Transform media
• Exchange media through
› Sources
› Sinks
Media pipeline
• Chain of media elements
implementing the desired media
logic.
• The Media API provides the
capability of creating media
pipelines by joining media
elements of the toolbox
Media Element
Sin
k
SRC
Creating applications: the Lego game
Protocols&Codecs
ComputerVision
AugmentedReality
Mul sensoryMul media
MediaRepository
IPTVIntegra on
GroupCommunica ons
Sink%
H pEndpoint
Sink%
SRC%
RtpEndpoint
SRC$
DataChannelEndpoint
Sink%
SRC%
WebRtcEndpoint
Sink%
SRC%
BarCodeReader
Sink%
SRC%
PointerTracker
Sink%
SRC%
FaceDetector
Sink%
SRC%
FaceOverlay
Sink%
SRC%
ChromaFilter
Sink%
SRC%
HeartRateBlender
Sink%
RecorderEndpoint
SRC$
PlayerEndpoint
Sink%
CdnUpload
Sink%
IPTVConnector Mixer
Sink
SRC Sink
SRC
Sink
SRC
Sink
Sink
Application 2
Sink
SRC
Sink
SRC
SinkSink
SRC
Sink
Application 3
Sink
SRC
SRC
Sink
SRC Sink
SinkSRC
Application 1
Toolbox of media elements
Architecting a Kurento application
Ready?
Installing Kurento
• The doc– http://www.kurento.org/docs/current/installation_guide.html
• The video– https://www.youtube.com/watch?v=RGdDpyZAPus
• In shortsudo add-apt-repository ppa:kurento/kurento
wget -O - http://ubuntu.kurento.org/kurento.gpg.key | sudo apt-key add -
sudo apt-get update
sudo apt-get install kurento-media-server
Get a Kurento cloud instance for free
• FI-WARE (http://www.fi-ware.org)– Funded through European Commission research programmes
– PaaS cloud infrastructure offering a bunch of capabilities and enablers for the “Future Internet”
– You can get money developing WebRTC applications with Kurento and FI-WARE• http://www.fi-ware.org/accelerators/
• How to get a Kurento instance in FIWARE in 5 minutes– https://www.youtube.com/watch?v=lFM12pXeWu8
Kurento “Hello World”
Source & demo
https://github.com/Kurento/kurento-tutorial-js/tree/master/kurento-hello-world
Recording and playing
Media Pipeline
WebRTCStreaming
Sin
kSR
C
Sin
k
Mediato
file or URI
Media Pipeline
WebRTCStreaming
Sin
kSR
C
Mediafrom
file or URI
SRC
WebRtcEndpoint
WebRtcEndpoint
RecorderEndpoint
PlayerEndpoint
Source codekurentoClient.create("MediaPipeline", function(error, p) {
if(error) return onError(error);
pipeline = p;
pipeline.create("WebRtcEndpoint", function(error, webRtc){
if(error) return onError(error);
pipeline.create(”RecorderEndpoint”, function(error, recorder){
if(error) return onError(error);
webRtc.connect(recorder, function(error){
if(error) return onError(error);
webRtc.connect(webRtc, function(error){
if(error) return onError(error);
webRtc.processOffer(sdpOffer, …
if(error) return onError(error);
webRtcPeer.processSdpAnswer(sdpAnswer);
});
});
});
});
});
I don’t like callbacks
From that code structureto this code structure
Second try … generators
• https://github.com/Kurento/kurento-tutorial-js/tree/master/kurento-hello-world-recorder-generator
• Transactions very soon …
– By end Q4’14
Advanced media processing
Source & demo
• https://github.com/Kurento/kurento-tutorial-js/tree/master/kurento-magic-mirror
Create your own filter: working with Kurento external modules
WebRTCStreaming
Sin
kSR
C
WebRtcEndpoint
Sin
k
SRC
ArMarkerDetector
Source & demo
• https://github.com/Kurento/kurento-tutorial-js/tree/master/kurento-alvar
Putting it altogether: calls with recording and filtering
Source & demo: let’s switch to Java for fun
• https://github.com/Kurento/kurento-tutorial-java/tree/master/kurento-one2one-call-advanced
Bonus: going beyond the media server vision
WebRTC Application based on media infrastructure
WebRTC Cloud
WebRTC cloud infrastructures
PaaSIaaS SaaSProvider- Computing resourcesDeveloper- Installation- Administration- Security- Application logic
Provider- Development API
Developer- Application logic
Provider- Service
Developer- Nothing to do
Multimedia PaaS: scalability types
Number of concurrent sessions
Nu
mb
er o
f u
sers
per
ses
sio
n
WebRTC PaaS Phone system
TV Broadcasting
CDNs
Media Serverbased Services
“RTC calls”
“Media serverRTC”
“Broadcasting Non-RTC”
“Beyond RTC calls”
The scalability problem in call clouds
CallCallCall Call CallCall
CallCallCall Call CallCall
CallCallCall Call CallCall
The scalability problem beyond call clouds
Media stream
On
e to
MA
NY
WebRTC services beyond the call model
Users in call Monitoring users
Users can dynamically change their role
Nubomedia: Kurento as an elastic media server
Load%Balancer%Func. on%
IaaS%Cloud%Manager%
Applica. on%Server%Func. on%
Broker%Func. on%
Media%Server%Func. on%
Cloud%Orchestrator%