2016 Streaming Media West: Transitioning from Flash to HTML5

Choosing an HTML5 player An overview of how media engines work & benchmark of open-source frameworks

Streaming Media West – Track DTuesday, May 10, 2016 1:45 to 2:30 pm

● Pioneers in hybrid video delivery systems to accompany exponential growth in OTT traffic

● Experts in HTML5 and consultants in the transition from Flash

Infinite scale, limitless delivery.

Streamroot: Who are we?

And who are you?


I. HTML5 basicsA. Why make the transitionB. Formats, APIs, standardization

II. Delving in to the migrationA. Formats & EncodingB. PlayersC. DRMs, ads and business featuresD. HTTPs

III. Choosing a playerA. Modern player architectureB. Considerations & what to look forC. Benchmark of open-source and proprietary options

What we’ll be talking about today.


IV. Examples: hls.js & dash.jsDebugging, subtitles, encryption, ads, analytics, Streamroot, ABR

V. Going from POC to productionA. What could go wrongB. What happens after I do a POC? Workflow, fallbacks, etc.C. Smoothly transitioning into production: QoS metrics, AB testingD. Useful tools & links

What we’ll be talking about today.



I. It’s time to switch to HTML5.


I. HTML5 BasicsAPIs

1. Media Source Extensions

• Makes HTTP adaptive streaming possible in HMTL5

• Use JavaScript to build streams and inject data into the video tag’s buffer

Source: W3C specs


I. HTML5 BasicsAPIs

2. Encrypted Media Extensions

• Allows for DRMs in HMTL5

• Provides a way to interact with content protection systems

• Plugin-free

• Common Encryption (CENC): standardized key and encryption methods - multiple DRMs for same file

Source: W3C specs


I. HTML5 BasicsAPIs

Source: Bitmovin


I. HTML5 BasicsFormats available

HDS


I. HTML5 BasicsFormats available

Source: Encoding.com


I. HTML5 BasicsStandardization

CMAF…!


II. Delving into the migration

What are you using today?


II. Delving into the migrationFormats & Encoding

Significant changes to the encoding side that can take some time

Encode to MP4 and then repackage as necessary for target platforms

Lots of packaging solutions out there: Wowza, Unified Streaming Platform, etc.


II. Delving into the migrationPlayers

More flexible architecture, with separation between UX and media engine logic

When choosing: look at use case and feature requirements

Section III.


II. Delving into the migrationBusiness features

DRMs: switch from token authorization to HMTL5 DRM

Ads: rewrite the ad logic in HTML5

Business logic: make sure 3rd party plugins are also compatible with HTML5


II. Delving into the migrationHTTPS

Beware!

Playing HTTP streams even in an HTTPS environment is not an option in HTML5.

Changing to HTTPs can be expensive and long, depending on your CDN.


It’s like...

III. Player architectureModern player architecture

… your favorite burger.Infinite scale, limitless delivery.

It’s like...

III. Player architectureModern player architecture


UI

III. Player architecture


Skin

UI



Skin

AdsAuthentication

UI



Skin

Ads

PlaylistsAuthentication

UI

Social sharing



Skin

Ads


UI

Media Engine(s)

Social sharing



Skin

Ads


Media Engine(s)

UI

Media Engine(s)

Social sharing



Skin

Ads


Social sharing

Media Engine(s)

UI

Media Engine(s)

Playback& DRM



Skin

Ads


Content decryption

module

Media Engine(s)

DRM Manager

UI

Media Engine(s)

Playback& DRM

Social sharing



Skin

Ads

Playlists

DRM ManagerDecoder / Renderer

Authentication

Content decryption

module

Media Engine(s)

UI

Media Engine(s)

Playback& DRM

Social sharing



1. Skin - the graphic design of your player

III. Player architectureUser Interface


1. Skin - the graphic design of your player

2. UI logic - features defining all interaction with the user on top of video playback

Lots of plugins available, video.js for example


https://github.com/videojs/video.js/wiki/Plugins#community-plugins


3. Business logic

Authentication

Payment

Ads

*Configuration / device detection logic!



Sample UI workflow with authentication, channels and pre-roll advertisement



Easily customizable, with many plugins available for use or inspiration.

Add features as plugins/modules to the core UI base.

Create a unified user experience across browsers, even if the media engine behind it may vary from device to device.

Check out tools such as React native, Haxe



III. Player architectureMedia Engine


Extremely important yet too often neglected.

Most often necessary to have several media engines to reach your audience.

III. Player architectureMedia Engine


III. Player architectureDecoder & DRM manager

1. General criteria - device, format and codec dependencies


III. What to look for

2. Custom features- DRMs - DVR subtitles- QoS, etc.

3. Perfs & Quality - startup time - ABR- error strategies

4. Street cred, robustness, ease of use

Light-touch dev: - Simplicity and stablity of the media engine- Assess extensibility & ease of incorporating special features- Media engine customization- Events exposed- Debugging


Digging deeper: - Robust yet flexible core design- Tests and testing coverage- community & support

III. What to look for


III. What’s out there?

Disclaimer!

- What follows are all GOOD options - probably the best out there.

- We’ve tried to be solely objective based on our research and experience.

- Features, support and upkeep are always changing.


PROs

Supported by DASH-IFPushed & maintained by Akamai + tier-1s (BBC)Highest visibility, big communityLots of features and use-cases handledWidely used in production Huge test suite + online test page

CONs

A little bit of technical debt and complexityNon-trivial API and customization configH264/AAC support only


http://dashif.org/reference/players/javascript/index.html

http://dashif.org/reference/players/javascript/index.html


PROs

Built by an entire at Google: solid & smartPerformances and robustness, quick to improveSimple to get started, good tutorialsOnly one supporting WebM, VP8, VP9, open audio codecs Good support on github and Google groups

CONs

Google-centricToday lacks some features for large broadcastersStricter PR & features policyNo ES6 supportFewer OVP and open-source all-in-one integrations



PROs

Lead by Dailymotion, built from scratch by author of Flashls Enormous traction and visibilityClear architecture design, easily extendableGood robustness and debug demo, responsive supportWidely used in prod by all-in-one players + tier-1s

CONsNot all HLS features supported yetSome restrictions from the transmuxing & HLS: no DRMs today,

only AES128


http://dailymotion.github.io/hls.js/demo/



PROs

Seamless Flash fallback with a MediaSource polyfillDe facto solution for HLS with VideoJSLarge community of users (Brightcove + Videojs)In production with Brightcove with a wide range of customers

CONsVideojs plugin: not usable without videojsLack of public debug tools or pagesLearning curve on providers & tech behind it


But there are also a lot of off-the-shelf options.



All available online on github:

https://github.com/streamroot/benchmarking-player

Contains:- Media engines code- Examples


IV. Examples

https://github.com/streamroot/benchmarking-player

MSE example

https://github.com/streamroot/benchmarking-player/blob/master/samples/mse.html


IV. Examples



https://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs.htmlhttps://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-AES.htmlhttps://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-subtitles.htmlhttps://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-ads.htmlhttps://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-streamroot.html

Online debug demo: debug http://dailymotion.github.io/hls.js/demo/ Videojs Ads plugin: https://github.com/videojs/videojs-contrib-adsAds state diagram: https://raw.githubusercontent.com/videojs/videojs-contrib-ads/master/ad-states.png Hls.js Events: https://github.com/dailymotion/hls.js/blob/master/API.md#runtime-events Video Events simulator: http://docs.youbora.com/video-event-simulator/

Streamroot Clappr integration: https://github.com/streamroot/clappr-p2phls-plugin Strearmoot Videojs integration: https://github.com/streamroot/videojs5-hlsjs-p2p-source-handler


IV. Examples

https://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs.html

https://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-AES.html


https://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-subtitles.html


https://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-ads.html


https://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-streamroot.html

https://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-streamroot.html



https://github.com/videojs/videojs-contrib-ads

https://raw.githubusercontent.com/videojs/videojs-contrib-ads/master/ad-states.png

https://github.com/dailymotion/hls.js/blob/master/API.md#runtime-events

http://docs.youbora.com/video-event-simulator/

https://github.com/streamroot/clappr-p2phls-plugin

https://github.com/streamroot/videojs5-hlsjs-p2p-source-handler

https://github.com/dailymotion/hls.js/blob/master/src/controller/cap-level-controller.js#L68

Checks the max CapLevel corresponding to current player size

Frequency: every 1000 ms

IV. Examples: hls.js CapLevel


https://github.com/dailymotion/hls.js/blob/master/src/controller/cap-level-controller.js#L68

https://github.com/dailymotion/hls.js/blob/master/src/controller/fps-controller.js#L33

Calculates the dropped frames per second ratio.

If > 0.2, bans the level forever goes into restricted capping levels

fpsDroppedMonitoringThresholdfpsDroppedMonitoringPeriod

IV. Examples: hls.js dropped frames


https://github.com/dailymotion/hls.js/blob/master/src/controller/fps-controller.js#L33

https://github.com/dailymotion/hls.js/blob/master/src/controller/stream-controller.js#L131

First segment is loaded from the first level in the playlist, then continues with normal ABR rule.

IV. Examples: hls.js startup strat


https://github.com/dailymotion/hls.js/blob/master/src/controller/stream-controller.js#L131

https://github.com/dailymotion/hls.js/blob/master/src/controller/abr-controller.js

Before: Simple algorithm,inspired by Android’s AVController’s ABR algo

Now: EWMA smoothing


IV. Examples: hls.js bandwidth est.

https://github.com/dailymotion/hls.js/blob/master/src/controller/abr-controller.js

https://github.com/dailymotion/hls.js/blob/master/src/controller/abr-controller.js#L51


IV. Examples: hls.js abort rule

https://github.com/dailymotion/hls.js/blob/master/src/controller/abr-controller.js#L51

STRONG POINTS COULD BE IMPROVED

Very simple and understandable Startup time constraint could be improved to get the lowest level first

Handles CPU & player size constraints

Conservative BW adjustment to avoid oscillation, with EWMA smoothing

Sound emergency abort mechanism


IV. Examples: hls.js sumup

1. Tweak the parametershttps://github.com/dailymotion/hls.js/blob/master/API.md#fine-tuning

Dropped FPS:

capLevelOnFPSDrop: false,fpsDroppedMonitoringPeriod: 5000,fpsDroppedMonitoringThreshold: 0.2

PlayerSize:

capLevelToPlayerSize: false,

2. Write your own rules!AbrController: AbrControllercapLevelController: CapLevelController,fpsController: fpsController

Bandwidth estimation:

abrEwmaFastLive: 5.0,abrEwmaSlowLive: 9.0,abrEwmaFastVoD: 4.0,abrEwmaSlowVoD: 15.0,abrEwmaDefaultEstimate: 500000,abrBandWidthFactor: 0.8,abrBandWidthUpFactor: 0.7,


IV. Examples: hls.js how to improve

https://github.com/dailymotion/hls.js/blob/master/API.md#fine-tuning

https://github.com/dailymotion/hls.js/blob/master/API.md#fine-tuning

https://github.com/streamroot/benchmarking-player/blob/master/samples/dash.js/dashjs.htmlhttps://github.com/streamroot/benchmarking-player/blob/master/samples/dash.js/dashjs-widewine.htmlhttps://github.com/streamroot/benchmarking-player/blob/master/samples/dash.js/dashjs-subtitles.htmlhttps://github.com/streamroot/benchmarking-player/blob/master/samples/dash.js/dashjs-ads.htmlhttps://github.com/streamroot/benchmarking-player/blob/master/samples/dash.js/dashjs-streamroot.html

Online debug page: http://dashif.org/reference/players/javascript/v2.3.0/samples/dash-if-reference-player/index.html Dash.js metrics & events: http://cdn.dashjs.org/latest/jsdoc/module-DashMetrics.htmlDash.js ABR logic: https://github.com/Dash-Industry-Forum/dash.js/wiki/ABR-Logic

Streamroot Videojs integration: https://github.com/streamroot/videojs5-dashjs-p2p-source-handler Infinite scale, limitless delivery.

IV. Examples: Dash.js

https://github.com/streamroot/benchmarking-player/blob/master/samples/dash.js/dashjs.html



https://github.com/streamroot/benchmarking-player/blob/master/samples/dash.js/hlsjs.html



















http://dashif.org/reference/players/javascript/v2.3.0/samples/dash-if-reference-player/index.html

http://cdn.dashjs.org/latest/jsdoc/module-DashMetrics.html

https://github.com/Dash-Industry-Forum/dash.js/wiki/ABR-Logic

https://github.com/streamroot/videojs5-dashjs-p2p-source-handler

Source: DASH-IF, Maxdome


IV. Examples: Dash.js

STRONG POINTS COULD BE IMPROVED

Smoothes bandwidth No quantization of bitrates

Segment abort mechanism to avoid buffering during network drops Doesn’t handle CPU & Player size constraints

Rich buffer threshold to avoid BW oscillations


IV. Examples: Dash.js sumup

1. Tweak the ParametersThroughputRule:

AVERAGE_THROUGHPUT_SAMPLE_AMOUNT_LIVE = 2;AVERAGE_THROUGHPUT_SAMPLE_AMOUNT_VOD = 3;

AbandonRequestRule:

GRACE_TIME_THRESHOLD = 500;ABANDON_MULTIPLIER = 1.5;

2. Write your own ruleshttps://github.com/Dash-Industry-Forum/dash.js/wiki/Migration-2.0#extending-dashjshttps://github.com/Dash-Industry-Forum/dash.js/blob/development/src/streaming/rules/abr/ABRRulesCollection.js

BufferOccupancyRule:

RICH_BUFFER_THRESHOLD = 20


IV. Examples: Dash.js how to improve

https://github.com/Dash-Industry-Forum/dash.js/wiki/Migration-2.0#extending-dashjs

https://github.com/Dash-Industry-Forum/dash.js/blob/development/src/streaming/rules/abr/ABRRulesCollection.js

https://github.com/Dash-Industry-Forum/dash.js/blob/development/src/streaming/rules/abr/ABRRulesCollection.js

Other media engines

https://github.com/streamroot/benchmarking-player/blob/master/samples/shaka.html https://github.com/streamroot/benchmarking-player/blob/master/samples/videojs.hls.html


IV. Examples

https://github.com/streamroot/benchmarking-player/blob/master/samples/shaka.html

https://github.com/streamroot/benchmarking-player/blob/master/samples/videojs.hls.html

https://github.com/streamroot/benchmarking-player/blob/master/samples/videojs.hls.html

What could go wrong?

- - Encoding- - DRMs- - HTTPS


V. The Path to Production!

I did a POC. Now what?

- - Do you need a Flash Fallback?- - Moving the UX to HTML5- - Build the media engine extendable architecture (providers)- - Fallback alternatives

https://github.com/streamroot/videojs5-hlsjs-source-handler/blob/master/lib/videojs5-hlsjs-source-handler.jshttps://github.com/videojs/video.js/blob/66922a818e588b8954989c439e1be790777966ae/docs/guides/tech.md

https://github.com/faisalman/ua-parser-js



https://github.com/videojs/video.js/blob/66922a818e588b8954989c439e1be790777966ae/docs/guides/tech.md

https://github.com/faisalman/ua-parser-js

Making a smooth transition.

- - HTML5 player rollout- - Playback & QoS Metrics- - AB Testing!




Streamroot workflow

V. The Path to Production!AB testing pipeline: General workflow

Axel Delmas

You can also put a map/reduce framework between the message broker and the analytics database.(or any kind of data processing consumer. Could be a hyperloglog consumer, or whatever)

Many options based on your requirements: open-source, proprietary, Cloud...


V. The Path to Production!AB testing pipeline: What to chose from

Message broker / stream processor

Storage + possible query/visualization

BI/viz tools Advanced analytics

Kafka (Kinesis) Druid Caravel R

RabbitMQ InfluxDB Grafana Matlab

ActiveMQ Cassandra (DynamoDB) Tableau

PostGresSQL QlikView

ElasticSearch Kibana


V. The Path to Production!AB testing pipeline: General setup

Client AB testing methods:

1) Dynamic Config Injection2) Different builds with a reverse proxy


V. The Path to Production!AB testing pipeline: General setup


V. The Path to Production!AB testing pipeline: Config injection


V. The Path to Production!AB testing pipeline: Dynamic config injection

PROs

Simple & easy splitsSeparate - fast to deploy & rollbackScalable

CONs

Limited scope Complications in codeNeed to handle scaling & high availability

WARNING

Async implementation!


V. The Path to Production!AB testing pipeline: Different builds with reverse proxy


PROs

Serve very different (major) versions of playerPerfect for rolling out new features smoothly (10% then 30% then...)

CONs

Good CI/CD necessary to roll out new versions quicklyHeavy stack to maintain and scale

WARNING

Make sure the file request is not slower!Single point of failure!

V. The Path to Production!AB testing pipeline: Different builds with reverse proxy

How much time does it take?

The build vs. buy tradeoff.

- Do I have the internal resources to develop, integrate, customize and maintain?

- Are the features I need already available in off-the-shelf and/or open-source solutions?

Consider time to market.



Questions?


Nikolay Rodionov, Co-Founder and CPO, [email protected]

Erica Beavers, Head of Partnerships, [email protected]


Get in touch!

mailto:[email protected]



We’d like to send out a huge thanks to Ludovic Bostral and Benjamin Pott from Afrostream for letting us borrow their burger analogy. Without them

this talk would not have been half as juicy.

(Also their platform rocks!)


Credits

2016 Streaming Media West: Transitioning from Flash to HTML5

Engineering

Transcript of 2016 Streaming Media West: Transitioning from Flash to HTML5