Mm presentation bkk

download Mm presentation bkk

of 34

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Mm presentation bkk

Modeling the Qoe of Rate Changes in Skype/Silk VoIP calls

Modeling the Qoe of Rate Changes in Skype/Silk VoIP callsChien-nan Chencing-yu chusu-ling yehhao-hua chupolly huangUniversity of Illinois, Urbana-ChampaignNational Taiwan university1Hello everyone, my name is Cing-Yu Chu. Today, on behalf of our research group, I am going to introduce our paper "Modeling the QoE of Rate Changes in SKYPE/SILK VoIP Calls". It is a joint work with Chien-Nan Chen who is now in UIUC, and Su-Ling Yeh, Hao-Hua Chu and Polly Huang from National Taiwan University.1OutlineMotivationPreliminary ExperimentProposed ModelLarge-Scale ExperimentEvaluationConclusion2Here is the outline of my presentation today. I will start with our motivation and then introduce the preliminary experiment we used to proposed the models for quality prediction. We then tried to find out the coefficients of these models using a large-scale experiment, and evaluate the derived models with both training data and other dataset that are independent to the training data. 2Voice over IP

InternetBandwidth FluctuationPacket LossDelayJitterMotivationPre. Exp.Proposed ModelLarge Exp.EvaluationConclusion3So let's begin with what is Voice over IP, or VoIP, application. As shown in this slide, VoIP applications allow users to send their voice data or voice packet into the Internet. The Internet will then help to forward these data or packets to the receivers, or another end user. In this process, we can easily see that the quality of VoIP applications would be influenced by different network conditions such as delay, jitter and packet loss. And all of these can actually be attributed to the bandwidth fluctuation.3Rate AdaptationAvailable bandwidth Ramping up the sending rate

Available bandwidth Tuning down the sending rate

Is the quality improved proportionally?Rate change Disturbing users?MotivationPre. Exp.Proposed ModelLarge Exp.EvaluationConclusion4Therefore, in order to deal with the bandwidth fluctuation, the rate adaptation has long been a classical topic in VoIP application. Traditionally, to fully utilize the network resource, we ramp up the sending rate when there is extra bandwidth available. We believe this can improve the service quality. However, our concern here is that if the improved quality is proportional to the increased sending rate. On the other hand, when the available bandwidth is insufficient, we have to tune down the sending rate to avoid congestion or packet loss. Combining ramping up and tuning down the sending rate, we actually introduce the quality fluctuation, or we call it rate change event. So our second concern is we wonder if such rate change would disturb users and make them unhappy.4GoalInvestigating the relationship ofSending rate vs. Perceived qualityTo explore the influence ofRate change magnitude/frequency

MethodologySynthesized VoIP callsUser study experimentsMotivationPre. Exp.Proposed ModelLarge Exp.EvaluationConclusion5So, in this work, we tried to understand the relationship between the sending bitrate and the perceived quality. Also, we are interested in how the rate change, both magnitude and frequency, would influence the perceived quality. We answer these questions by synthesizing VoIP calls with different sending bitrate and rate change magnitude and frequency. And then we conducted a series of user study experiment to see how real human perceive these VoIP calls.5ContributionSending bitrate vs. user perception Logarithmic RelationshipFrequency of rate change Logarithmic RelationshipMagnitude of rate changeComplicated, but InterestingClosed-form models to predict user perception under bandwidth fluctuationMotivationPre. Exp.Proposed ModelLarge Exp.EvaluationConclusion6In the end of this work, we were able to find that the relationship between the sending bitrate and perceived quality is logarithmic. And such logarithmic relationship can also be observed in the influence of rate change frequency. As for the rate change magnitude, we found it to be kind of complicated but it's interesting, and we will talk about the detail later. After we had the above findings, we were able to derive closed-form models to predict the perceived quality.6Preliminary experimentTo confirm the influence ofsending bitraterate change magnituderate change frequency

5-level MOS (Mean Opinion Score)14 participants

MotivationPre. Exp.Proposed ModelLarge Exp.EvaluationConclusion7All of our observations are based on a preliminary experiment. The purpose of this preliminary experiment is to confirm if sending bitrate, rate change magnitude/frequency would really influence the perceived quality. If yes, then how do they influence the perceived quality. In our work, we adopted the 5-level Mean Opinion Score (MOS) which is recommended by ITU, with 5 is the best quality while 1 is the worst. We use MOS to represent the perceived quality throughout our work. In this experiment, totally 14 particpiants were recruited.7

Audio Track ProductionSkype/SILK audio codec30s audio tracksentences without contextual connectionFixed-rate tracks

Variable-rate tracksBitrate (kbps)5.69.513.317. Exp.Proposed ModelLarge Exp.EvaluationConclusion8Here we have to explain how did we produce the synthesized VoIP calls. Since Skype might be the most popular VoIP application, we chose it as our research target, and SILK is the audio codec adopted by Skype in its latest version. The most desirable property of SILK is that it allows arbitrary coding bitrate from 5 kbps to 40 kbps, which is very suitable for investigating the influence of sending rate and rate change. We used SILK to endcode and decode an original audio track. The length of this audio track is 30 seconds, and it is composed of several simple and meaningful sentences. After being processed by SILK, we could get the degraded audio tracks. These audio tracks could be classified into 2 categories. The first on is called fixed-rate tracks. We evenly chose 10 different rates between the maximum and minimum of SILK's coding bitrate, and we used these bitrate to encode and decode the audio files to form the fixed-rate tracks. The second on is called variable-rate tracks. A variable-rate track is defined by three parameters: high rate, low rate and delta T. We switched the coding bitrate between high rate and low rate every delta T period to introduce the quality fluctuation or rte change.8ResultFixed-rateMOS vs. sending bitrate

User VariationLogarithmic TrendMotivationPre. Exp.Proposed ModelLarge Exp.EvaluationConclusion9Here is the result of the fixed-rate case and the figure is the MOS-bitrate plot where the x-axis is the bitrate and the y-axis is the MOS. From this figure, we can see that there exists user variation which is indicated by the error bar of one standard deviation. However, we can still observe a trend if we look at the average user score which is indicated by the red points. The trend we found here is a logarithmic relationsip between the bitrate and the perceived quality.9ResultVariable-RateMOS - T plot

Rate change matters!MotivationPre. Exp.Proposed ModelLarge Exp.EvaluationConclusion10As for the variable-rate case, the MOS-delta T plot here can tell us how the rate change influence the perceived quality. Again, the red points here are the average user scores, and they are the result of variable-rate tracks whose high rate is 40.6 and low rat is 17.2 kbps. We can clearly observe that the perceived quality changes when we vary the delta T, and the perceived quality becomes better when the delta T is bigger. Furthermore, the three horizontal lines in this figure are the quality of fixed-rate case with bitrate equals the high rate, low rate and the average of high rate and low rate. So it tells us that the quality of a variable-rate track is different from the quality of average bitrate. And also, even though the bitrate of this variable-rate track is never below the low rate, its quality could be worse than the low rate when the rate change is rapid. So, we can conclude that, the rate change plays an important role!10Effect of Rate Change FrequencyWhen T varies

Logarithmic TrendMotivationPre. Exp.Proposed ModelLarge Exp.EvaluationConclusion11Then, let's look at the influence of frequency and magnitude separately. Here is the result of a few variable-rate tracks. Because the logarithmic regression on all these curves reveal good fitting result. We simply conclude that the influence of rate change frequency is logarithmic.11Effect of Rate Change Magnitude When sharing the same average bitrate

Magnitude MOS MotivationPre. Exp.Proposed ModelLarge Exp.EvaluationConclusion12As for the rate change magnitude, here is the result of variable-rate tracks that shared the same average bitrate. But the upper curve has smaller rate change magnitude and its quality is always better than the lower one. This suggests that when the rate change magnitude is bigger, the quality is worse. However, does it mean the difference between high rate and low rate is the only factor that influence that quality? The answer is NO.12Effect of Rate Change Magnitude However, with the same magnitude

Higher (hr + lr)Lower (hr + lr)MotivationPre. Exp.Proposed ModelLarge Exp.EvaluationConclusion13Here are the cases where delta T is 3 seconds. This time we plot x-axis as the difference between high rate and low rate while the y-axis is still the MOS. The red line is the result of tracks that have higher high rate and the blue line is the result of tracks that have lower high rate. So, from this figure, we can see that even though the rate difference is the same, tracks with both high rate and low rate are high have better quality. So we can conclude the quality is determined by only the rate difference but also the level of both high rate and low rate.13Short SummaryFixed-rateMOS bitrate log