SIGCOMM 2011. Outline Introduction Datasets and Metrics Analysis Techniques Engagement View...
-
Upload
ursula-lambert -
Category
Documents
-
view
220 -
download
0
Transcript of SIGCOMM 2011. Outline Introduction Datasets and Metrics Analysis Techniques Engagement View...
UNDERSTANDING THE IMPACT OF VIDEO QUALITY ON USER ENGAGEMENT
SIGCOMM 2011
Outline
Introduction Datasets and Metrics Analysis Techniques Engagement
View Level Viewer Level
Lessons Conclusion
Introduction
Internet video has become more and more popular
What impacts engagement?! Not well understood yet
Introduction
Given the same video, does Quality impact Engagement?! What are the most critical metrics? Do these critical metrics differ across
genres? How much does optimizing a metric
help?
Datasets and Metrics
Data Collection A week of data from multiple premium
video sites & full census measurement from video player
Video Genres Live LVoD SVoD
Datasets and Metrics
Quality Metrics Buffering Ratio Rate of Buffering Join time Rendering Quality Average Bit Rate
Datasets and Metrics
Two Engagement Granularities View
Play time of a video session Viewer
Total play time by a viewer in a period of time
Total number of views by a viewer in a period of time
Analysis Techniques
Which metrics matter most
Are metrics independent?
How do we quantify the impact?
Analysis Techniques
Qualitative Correlation Coefficient Information Gain
Linear Regression Quantitative
Analysis Techniques
An simple example
View Level Engagement
Long VoD Content - Correlation
View Level Engagement
Long VoD Content - Correlation Most important metric
Buffering ratio
Less important metrics Rendering quality, Join time
View Level Engagement
Long VoD Content – Information Gain
View Level Engagement
Long VoD Content – Information Gain Bit rate becomes the most important
metric Why??????
View Level Engagement
Live Content
View Level Engagement
Live Content Buffering Ration remains the most
significant Bitrate and Rate of Buffering matter
much more
View Level Engagement
Live Content Rendering Quality negatively
correlated?!
User behavior matters
View Level Engagement
Short VoD Content
View Level Engagement
Short VoD Content Similar to long VoD content Buffering ration remains the strongest Rendering Quality is less important
View Level Engagement
Quantitative Impact Not apply regression to all the data Only apply regression to the segment
that looks like linear 0-10% range of Buffering ratio
View Level Engagement
Summary BufRatio is the most important quality metric. For live content, AvgBitrate in addition to
BufRatio is a key quality metric. A 1% increase in BufRatio can decrease 1 to 3
minutes ofviewing time. JoinTime has significantly lower impact on
view-level engagement than the other metrics
RendQual in live video highlights the need of considering context of actual user and system behavior
Viewer Level Engagement
Buffering ratio vs. play time
Viewer Level Engagement
Buffering ratio vs. # of views
Viewer Level Engagement
Summary Both the # of views and the total play
time are impacted by the quality metrics Correlation between the engagement
metrics and the quality metrics becomes visually and quantitatively more striking at the viewer level
The join time, which seemed less relevant at the view level, has non-trivial impact at the viewer level
Lesson Learned
The need for complementary analysis All of you are right. The reason every one of
you is telling it differently is because each one of you touched a different part of the elephant. So, actually the elephant has all the features you mentioned.
Combination of Correlation and Information gain
Lesson Learned
The importance of context Lies, damned lies, and statistics
Together with the context of the human and operating factors
Lesson Learned
Toward video quality index Provide objective index for service
providers and researchersex: MOS
More dimensions More play type More Content type Etc…
Conclusion