Introduce native html5 streaming player

Click here to load reader

  • date post

    20-Jan-2017
  • Category

    Engineering

  • view

    1.004
  • download

    1

Embed Size (px)

Transcript of Introduce native html5 streaming player

  • HOW DID I CREATE A HTML5 VIDEO STREAMING PLAYER

    and what Ive learned

    KKBOXVideo jessechen

  • AGENDA Some beforehand knowhow

    Why developing this player

    HTML5 extensions

    Introduce our player

    Future of this player

  • SOME BEFOREHAND KNOWHOW

  • analog-to-digital adapter

    digital raw data

    PROCESS OF PLAYING VIDEO1. Make it digital

    analog signal

  • H.264 / H.265Encode

    PROCESS OF PLAYING VIDEO2. Encode video file

    Raw data .mp4 / .ts file

    Easier for storage and transference

  • PROCESS OF PLAYING VIDEO3. Play it !

    .mp4 / .ts filea/v sync

    codec decode and

    display

    extract contentfrom container

  • WAIT, WHERE IS PLAYER?

  • WHAT PLAYER DOES

    .mp4 / .ts filea/v sync

    codec decode and

    display

    HERE

    extract contentfrom container

  • THEY ARE ALREADY HANDLED

    .mp4 / .ts file

    HTML5

    a/v sync

    codec decode and

    display

    extract contentfrom container

  • SO WHY DEVELOPING THIS PLAYER?

  • OUR PLAYER FOCUS ON (1)

    extract content and a/v info

    from container

    .mp4 / .ts file

    get filesin smart way

    a/v sync

    codec decode and

    display

  • HOW PLAYER GET FILE ?

  • server holds .mp4

    content

    VINTAGE WAY

    Player get it and play

    single file

    slow and inflexible

  • server holds .mp4

    content

    EFFICIENT WAY

    Player get part of video and

    start play

    progressive download

    Keep requesting rest part of video during

    playback

    still inflexible

  • prepare different qualities

    of contenton server

    MODERN WAYadaptive streaming

    Player playsbest quality depends on

    network status

    Keep requesting rest part of video during

    playback

  • HOW ABOUT WE SPLIT FILE split file into segments, described by manifest

    manifest tells

    - available bitrates

    - how file is fragmented

    - other information, e.g encryption

    +

    +

  • knows how toget fragments

    ADAPTIVE STREAMING IN SHORT

    fragments

    manifestfile

  • ADAPTIVE STREAMING TYPES

    SSSmooth Streaming

    HLSHTTP Live Streaming

    HDS

  • INTRODUCE DASHDynamic Adaptive Streaming through HTTP

  • PROS OF DASH

    open source, with industry standard, which means more universal usage

    support 2 types of containers, the MPEG-4 file format or the MPEG-2 Transport Stream

  • GET FILES IN SMART WAY

    manifest parser - able to read different type of manifest

    adaptive bitrate - able to decide which quality to load

  • OUR PLAYER FOCUS ON (2)

    .mp4 / .ts file

    get filesin smart way

    protectionlogic

    extract contentfrom container

    a/v sync

    codec decode and

    display

  • DRM ?(DIGITAL RIGHTS MANAGEMENT)

    Content provider / DRM provider likes it

    For end user

  • CLIENT SIDE PROTECTION Client get a license instead of a key from DRM

    service

    A blackbox or sandbox get the key by processing license

    That blackbox / sandbox decrypt content and directly output it to display

  • H.264 / H.265Encode

    HOW TO PROTECT CONTENT

    Raw data .mp4 / .ts file

    DRM implemented

    here

  • PROTECTION LOGIC

    give what DRM server needs and retrieve license

    negotiate with browser in order to implement protection on client side

    deal with different browsers

    knows how to

  • NOW, LETS START TALK ABOUT BROWSER

    , MSE and EME

  • HTML5 VIDEO ELEMENT support progressive download

    For adaptive streaming, MSE is required

  • extract content from container

    WHAT ROLE MSE PLAYS

    .mp4 / .ts file

    MSE handles

    a/v sync

    codec decode and

    display

    MSE handles

  • INTRODUCE MSE

    Given different bitrate segments and it helpshtml5 video element to play smoothly

    SourceBuffer provide video and audio buffer condition

    Media Source Extension

  • EME IS ABOUT PROTECTION

  • HOW BROWSER ADOPT DRM

    For browser, that blackbox called CDM (Content Decrypt Module)

    Each browser support different DRM

    context - a blackbox or sandbox get the key by processing license

  • DRM ON BROWSER

    Widevine FairplayPlayready Primetime

  • INTRODUCE EME

    Even though browser support its own DRM, W3C defines a EME spec, in order to expose same api for client

    prefixed api was implemented on earlier version of chrome(smart tv)

    Encrypted Media Extension

  • EMECDM

    provide context from encrypted content

    get challenge

    DRM license server

    request with challenge

    get license

    provide license for CDM to decrypt content

    player

    EMECDM

    PROTECTION LOGIC FLOW

  • WHAT ROLE EME PLAYS

    extract contentfrom container

    .mp4 / .ts file

    a/v synccodec decode

    anddisplay

    decrypt content in blackbox

  • INTRODUCE OUR PLAYER

  • OUR GOAL

    play not single file but sequence of segments, with different bitrate, a.k.a adaptive streaming

    play protected stuffs, for content providers goods

    a player able to

  • ya pi

    yapi.js

  • DEVELOP PROCESS

    make it work stable refactor

  • MAKE IT WORK FIRST

    only 2 files in the very beginning

  • BE STABLE

    well structured

    modularized

    dependency management (dijon.js)

    consistent code style

  • MODULARIZED

    api

    adaptive

    extension

    fragment

    manifest

    protection

    stats

    stream

    utils

  • UTILS Capabilities

    Debug

    ErrorHandler

    EventBus

    IntervalBus

    UrlModel

    api

    adaptive

    extension

    fragment

    manifest

    protection

    stats

    stream

    utils

  • EXTENSION

    MediaSourceExtension

    SourceBufferExtension

    VideoModel

    api

    adaptive

    extension

    fragment

    manifest

    protection

    stats

    stream

    utils

  • MANIFEST

    ManifestLoader

    ManifestExtension

    ManifestModel

    api

    adaptive

    extension

    fragment

    manifest

    protection

    stats

    stream

    utils

  • STREAM

    StreamCtrl

    BufferCtrlapi

    adaptive

    extension

    fragment

    manifest

    protection

    stats

    stream

    utils

  • FRAGMENT

    FragmentCtrl

    FragmentLoader

    SegmentTemplate

    api

    adaptive

    extension

    fragment

    manifest

    protection

    stats

    stream

    utils

  • ADAPTIVE

    AbrCtrl

    BandwidthRecorderapi

    adaptive

    extension

    fragment

    manifest

    protection

    stats

    stream

    utils

  • PROTECTION

    ProtectionCtrl

    ProtectionModel

    ProtectionRequest

    Playready / Widevine

    api

    adaptive

    extension

    fragment

    manifest

    protection

    stats

    stream

    utils

  • STATS

    MetricsCtrl

    Metrics

    StatsCtrl

    BitrateHistory

    api

    adaptive

    extension

    fragment

    manifest

    protection

    stats

    stream

    utils

  • REFACTOR

    review flow

    redefine api

  • APPLICATION FLOW

  • REDEFINE API

    use jsdoc to generate spec document

    define api, event and vo(value object)

    spec

    result

    http://usejsdoc.org/http://localhost:9900/doc/spec/KKPlayer.MediaPlayer.htmlhttp://localhost:9900/doc/dist/

  • DEMOSample player

    http://localhost:9900

  • HOW TO MANAGE THOSE MODULES

  • DEPENDENCY INJECTION

    not necessary in the beginning, but became very important after it went much more complex

    use dijon.js as di framework

  • INTRODUCE DIJON.JS

    a system holds all dependencies, after mapping modules to it

    an object get dependencies after injection

  • DEFINE DEPENDENCIES

    // A depfunction A() {}

    // B requires Afunction B() { this.a = undefined; // how dijon knows where to inject}

    // instantiate dijonvar system = new dijon.System();

  • 3 KINDS OF MAPPING

    var a = new A();system.mapValue(aValue, a);// system.getObject(a) would return an a instance

    system.mapSingleton(aSingleton, A);// system.getObject(aSingleton) would return a singleton a

    system.mapClass(bClass, B);// system.getObject(bClass) would return a new inited B

  • OUTLET MAPPING

    // map outlet to make dependency work// @param sourceKey {String} - the key mapped to system would be injected// @param targetKey {String|'global'} - the key outlet is assigned to// @param outletName {String|sourceKey} - the name of property used as an outletsystem.mapOutlet('aSingleton', 'bClass', a');

    var b = system.getObject(bClass);// b.a is aSingleton

  • DIRECTLY INJECTION

    var b = new B();system.injectInto(b);

    // b WOULDNT have dependency singleton A// b/c we only mapOutlet to key bClass of system

    // map outlet to globalsystem.mapOutlet(aSingleton, global, a); system.injectInto(b); // this would work

  • AUTO MAP OUTLET

    function B(){this.aSingleton = undefined;this.c = undefined;

    }function C(){}

    // auto mapped outlet would assign to global// and outlet name is the same as m