Introduce native html5 streaming player
-
Upload
jesse-chien-chen-chen -
Category
Engineering
-
view
1.008 -
download
1
Transcript of Introduce native html5 streaming player
HOW DID I CREATE A HTML5 VIDEO STREAMING PLAYER
and what I’ve learned
KKBOXVideo陳建辰 jessechen
AGENDA• Some beforehand knowhow
• Why developing this player
• HTML5 extensions
• Introduce our player
• Future of this player
SOME BEFOREHAND KNOWHOW
analog-to-digital adapter
digital raw data
PROCESS OF PLAYING VIDEO1. Make it digital
analog signal
H.264 / H.265Encode
PROCESS OF PLAYING VIDEO2. Encode video file
Raw data .mp4 / .ts file
Easier for storage and transference
PROCESS OF PLAYING VIDEO3. Play it !
.mp4 / .ts filea/v sync
codec decode and
display
extract contentfrom container
WAIT, WHERE IS PLAYER?
WHAT PLAYER DOES
.mp4 / .ts filea/v sync
codec decode and
display
HERE
extract contentfrom container
THEY ARE ALREADY HANDLED
.mp4 / .ts file
HTML5
a/v sync
codec decode and
display
extract contentfrom container
SO WHY DEVELOPING THIS PLAYER?
OUR PLAYER FOCUS ON (1)
extract content and a/v info
from container
.mp4 / .ts file
get filesin smart way
a/v sync
codec decode and
display
HOW PLAYER GET FILE ?
server holds .mp4
content
VINTAGE WAY
Player get it and play
single file
slow and inflexible
server holds .mp4
content
EFFICIENT WAY
Player get part of video and
start play
progressive download
Keep requesting rest part of video during
playback
still inflexible
prepare different qualities
of contenton server
MODERN WAYadaptive streaming
Player playsbest quality depends on
network status
Keep requesting rest part of video during
playback
HOW ABOUT WE SPLIT FILE• split file into segments, described by manifest
• manifest tells
- available bitrates
- how file is fragmented
- other information, e.g encryption
+
+
knows how toget fragments
ADAPTIVE STREAMING IN SHORT
fragments
manifestfile
ADAPTIVE STREAMING TYPES
SSSmooth Streaming
HLSHTTP Live Streaming
HDS
INTRODUCE DASHDynamic Adaptive Streaming through HTTP
PROS OF DASH
• open source, with industry standard, which means more universal usage
• support 2 types of containers, the MPEG-4 file format or the MPEG-2 Transport Stream
GET FILES IN SMART WAY
• manifest parser - able to read different type of manifest
• adaptive bitrate - able to decide which quality to load
OUR PLAYER FOCUS ON (2)
.mp4 / .ts file
get filesin smart way
protectionlogic
extract contentfrom container
a/v sync
codec decode and
display
DRM ?(DIGITAL RIGHTS MANAGEMENT)
Content provider / DRM provider likes it
For end user
CLIENT SIDE PROTECTION• Client get a license instead of a key from DRM
service
• A blackbox or sandbox get the key by processing license
• That blackbox / sandbox decrypt content and directly output it to display
H.264 / H.265Encode
HOW TO PROTECT CONTENT
Raw data .mp4 / .ts file
DRM implemented
here
PROTECTION LOGIC
• give what DRM server needs and retrieve license
• negotiate with browser in order to implement protection on client side
• deal with different browsers
knows how to
NOW, LET’S START TALK ABOUT BROWSER
<video>, MSE and EME
HTML5 VIDEO ELEMENT• <video></video> support progressive download
• For adaptive streaming, MSE is required
extract content from container
WHAT ROLE MSE PLAYS
.mp4 / .ts file
MSE handles
a/v sync
codec decode and
display
MSE handles
INTRODUCE MSE
• Given different bitrate segments and it helpshtml5 video element to play smoothly
• SourceBuffer provide video and audio buffer condition
Media Source Extension
EME IS ABOUT PROTECTION
HOW BROWSER ADOPT DRM
• For browser, that blackbox called CDM (Content Decrypt Module)
• Each browser support different DRM
context - “a blackbox or sandbox get the key by processing license”
DRM ON BROWSER
Widevine FairplayPlayready Primetime
INTRODUCE EME
• Even though browser support its own DRM, W3C defines a EME spec, in order to expose same api for client
• prefixed api was implemented on earlier version of chrome(smart tv)
Encrypted Media Extension
EMECDM
provide context from encrypted content
get ‘challenge’
DRM license server
request with challenge
get license
provide license for CDM to decrypt content
player
EMECDM
PROTECTION LOGIC FLOW
WHAT ROLE EME PLAYS
extract contentfrom container
.mp4 / .ts file
a/v synccodec decode
anddisplay
decrypt content in blackbox
INTRODUCE OUR PLAYER
OUR GOAL
• play not single file but sequence of segments, with different bitrate, a.k.a adaptive streaming
• play protected stuffs, for content providers’ goods
a player able to
影片ya pi
yapi.js
DEVELOP PROCESS
make it work stable refactor
MAKE IT WORK FIRST
only 2 files in the very beginning
BE STABLE
• well structured
• modularized
• dependency management (dijon.js)
• consistent code style
MODULARIZED
api
adaptive
extension
fragment
manifest
protection
stats
stream
utils
UTILS• Capabilities
• Debug
• ErrorHandler
• EventBus
• IntervalBus
• UrlModel
api
adaptive
extension
fragment
manifest
protection
stats
stream
utils
EXTENSION
• MediaSourceExtension
• SourceBufferExtension
• VideoModel
api
adaptive
extension
fragment
manifest
protection
stats
stream
utils
MANIFEST
• ManifestLoader
• ManifestExtension
• ManifestModel
api
adaptive
extension
fragment
manifest
protection
stats
stream
utils
STREAM
• StreamCtrl
• BufferCtrlapi
adaptive
extension
fragment
manifest
protection
stats
stream
utils
FRAGMENT
• FragmentCtrl
• FragmentLoader
• SegmentTemplate
api
adaptive
extension
fragment
manifest
protection
stats
stream
utils
ADAPTIVE
• AbrCtrl
• BandwidthRecorderapi
adaptive
extension
fragment
manifest
protection
stats
stream
utils
PROTECTION
• ProtectionCtrl
• ProtectionModel
• ProtectionRequest
• Playready / Widevine
api
adaptive
extension
fragment
manifest
protection
stats
stream
utils
STATS
• MetricsCtrl
• Metrics
• StatsCtrl
• BitrateHistory
api
adaptive
extension
fragment
manifest
protection
stats
stream
utils
REFACTOR
• review flow
• redefine api
APPLICATION FLOW
REDEFINE API
• use jsdoc to generate spec document
• define api, event and vo(value object)
• spec
• result
DEMOSample player
HOW TO MANAGE THOSE MODULES
DEPENDENCY INJECTION
• not necessary in the beginning, but became very important after it went much more complex
• use dijon.js as di framework
INTRODUCE DIJON.JS
• a system holds all dependencies, after mapping modules to it
• an object get dependencies after injection
DEFINE DEPENDENCIES
// A depfunction A() {}
// B requires Afunction B() { this.a = undefined; // how dijon knows where to inject}
// instantiate dijonvar system = new dijon.System();
3 KINDS OF MAPPING
var a = new A();system.mapValue(‘aValue’, a);// system.getObject(‘a’) would return an ‘a’ instance
system.mapSingleton(aSingleton, A);// system.getObject(‘aSingleton’) would return a singleton a
system.mapClass(bClass, B);// system.getObject(‘bClass’) would return a new inited B
OUTLET MAPPING
// map outlet to make dependency work// @param sourceKey {String} - the key mapped to system would be injected// @param targetKey {String|'global'} - the key outlet is assigned to// @param outletName {String|sourceKey} - the name of property used as an outletsystem.mapOutlet('aSingleton', 'bClass', ‘a');
var b = system.getObject(‘bClass’);// b.a is aSingleton
DIRECTLY INJECTION
var b = new B();system.injectInto(b);
// b WOULDN’T have dependency singleton A// b/c we only mapOutlet to key bClass of system
// map outlet to globalsystem.mapOutlet(‘aSingleton’, ‘global’, ‘a’); system.injectInto(b); // this would work
AUTO MAP OUTLET
function B(){this.aSingleton = undefined;this.c = undefined;
}function C(){}
// auto mapped outlet would assign to ‘global’// and outlet name is the same as mapped key (before mapping)system.autoMapOutlets = true;
system.mapClass(‘bClass’, B); // map again b/c B is changedsystem.mapClass(‘c’, C);// system.getObject(‘b’) would have c dep
COMMUNICATE IN BETWEEN
Now B has dependencies of A and C
How would you do in this scenario:
A is doing something when it’s done, invoke a method of B
–this is a very common situation in player
MAP HANDLER AND NOTIFYfunction A() {
this.say = function() {system.notify(‘aDoneSaying’);
};}
function B() {this.a = undefined;this.afterAsays = function(){
// do something};
}
system.mapSingleton(‘a’, A); system.mapSingleton(‘b’, B);
system.mapHandler(‘aDoneSaying’, ’b’, ‘afterAsays’);// system.getObject(‘b’).a.say() would invoke b.afterAsays
NOTIFYING CLASSsystem.mapSingleton(‘a’, A); system.mapClass(‘b’, B); // map class here
var b1 = system.getObject(‘b’);// b1.say() would invoke a newly instantiated b.afterAsays// instead of b1.afterAsays
system.mapValue(‘b1’, b1);system.unmapHandler(‘aDoneSaying’, ‘b’, ‘afterAsays’) system.mapHandler(‘aDoneSaying’, ‘b1’, ‘afterAsays’)// b1.say() would invoke b1.afterAsays
function E() { this.a = undefined; this.setup = function () {};}
var e = new E();
system.injectInto(e);// e.setup invoked
CONVENTIONsetup method of module would be invoked
after getObject or injectInto
INJECT SYSTEM ITSELF
system.mapValue('system', system);
function D() { this.system = undefined;}
system.mapSingleton('d', D);
// system.getObject(‘d’) has system as dependency
function A() {}; function B() {}
function Dep() { return { system: undefined, setup: function () { this.system.autoMapOutlets = true; // map dep here this.system.mapSingleton('a', A); this.system.mapClass('b', B); } };}
function App() { var b; var system = new dijon.System(); system.mapValue('system', system); system.mapOutlet('system'); var dep = new Dep(); system.injectInto(dep); // inject system to dep and invoke setup function, which map all deps
return { system: undefined, a: undefined, setup: function () { b = system.getObject('b'); },
init: function () { system.injectInto(this); // after init, app.a exists } };}// after new App().init(), app get all dep setup in Dep
SUMMARY OF DIJONJS• 3kb
• very little convention
• focus on di and notify / handler
BUILD PROJECT
• list all source files in index.html of testing page
• grunt as task manager
• concat and minify scripts with grunt-usemin
• export to dist/player.min.js
ONE LAST NOTICE
• ui should be totally separated
• ui <-> player -> videoElement
• ui order player by api, respond to player behavior by event
playervideoElement
ui
FUTURE OF YAPI.JS
• firefox / safari drm support
• live
• ads
• smart tv / chromecast / nexus player
THANK YOUquestions ?