A Novel Video Transmission Optimization Mechanism Based on ...

Research ArticleA Novel Video Transmission Optimization Mechanism Based onReinforcement Learning and Edge Computing

Nan Hu 1 Xuming Cen 1 Fangjun Luan1 Liangliang Sun1 and Chengdong Wu 2

1School of Information and Control Engineering Shenyang Jianzhu University Shenyang 110168 China2Robot Science amp Engineering Faculty Northeastern University Shenyang 110819 China

Correspondence should be addressed to Nan Hu hunansjzueducn

Received 18 September 2021 Accepted 11 October 2021 Published 31 October 2021

Academic Editor Xingsi Xue

Copyright copy 2021NanHu et al(is is an open access article distributed under the Creative Commons Attribution License whichpermits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

As we know the video transmission traffic already constitutes 60 of Internet downlink traffic (e optimization of videotransmission efficiency has become an important challenge in the network (is paper designs a video transmission optimizationstrategy that takes reinforcement learning and edge computing (TORE) to improve the video transmission efficiency and qualityof experience Specifically first we design the popularity prediction model for video requests based on the RL (reinforcementlearning) and introduce the adaptive video encoding method for optimizing the efficiency of computing resource distributionSecond we design a video caching strategy which adopts EC (edge computing) to reduce the redundant video transmission Lastsimulations are conducted and the experimental results fully demonstrate the improvement of video quality and response time

1 Introduction

In 2019 video transmission traffic made up 606 of overallInternet downlink traffic [1] In the future with the rapiddevelopment of 4K8K ARVR holographic communica-tion smart city intelligent transportation and other tech-nologies network video transmission demand and trafficwill be further inspired In addition the number of videousers on the Internet has maintained a rapid growth ten-dency not only due to the rapid improvement of traditionalnetwork bandwidth but also because the quick expansion ofmobile Internet has further stimulated the potential of thevideo transmission market

(e industry of video transmission service has beenshown in a trend of explosive growth where fierce com-petition has existed among video transmission serviceproviders First the traditional film-television industry hasgradually begun to promote online video media library(including commercial television providers and publicservice broadcasters) Second video portals (YouTubeYouku Tencent Video etc) have developed rapidly Inaddition the service scale of professional video streamingmedia providers (such as Netflix and Maxdome) also shows

an explosive growth trend (erefore the competitionamong Internet video transmission services has becomeincreasingly fierce As of Q3 in 2018 Netflixrsquos globalmembership reached 137 million far ahead of more than 30streaming media companies in the United States It wasreported by Sandvine providing intelligent bandwidthmanagement services in October that Netflix has accountedfor 15 of global network traffic (excluding China andIndia) Although Netflix has basically won the first place itscompetitive pressure is increasing Domestic competition isalso fierce video portals such as Youku iQIYI and TencentVideo are making efforts one after another Although thesethree video portals still occupy the first echelon of domesticvideo transmission business Bilibili Mango TV and othervideo platforms also develop rapidly In addition domesticP2P live video services short video and other services arealso developing rapidly

Both domestic and foreign video transmission serviceproviders are committed to optimizing the user experienceso as to provide users with higher-quality video transmissionservices (erefore whether from the perspective of users orvideo service providers both investigations on the videotransmission mechanisms and algorithms to obtain the best

HindawiMobile Information SystemsVolume 2021 Article ID 6258200 10 pageshttpsdoiorg10115520216258200

user experience have significant industrial value and aca-demic value In short the object of the video transmissionoptimization is to provide videos for users with maximumvideo resolution and minimum video stream stuck In ad-dition there are other factors to be considered such as theswitching frequency of video rate (ese factors are finallysummarized into the quality of experience (QoE) scorewhich is used to evaluate the quality of video transmission

At present academia mainly studies the optimizationmechanism of video transmission according to the trans-mission architecture scheduling algorithm and evaluationmechanism In terms of architecture there are inevitableconflicts between the best-efforts transmission mode oftraditional Internet based on TCPIP and the deterministicquality assurance of video transmission (erefore someresearchers put forward the novel network transmissionarchitectures such as information-centric networking(ICN) to improve the network transmission efficiency so asto optimize video transmission efficiency some researchersproposed content-distributed network (CDN) in the ap-plication layer based on the current Internet architecturenetwork to optimize the efficiency of video transmission inorder to make up for the low efficiency of Internet trans-mission some researchers proposed the intelligent videotransmission strategy using edge computing aiming tobreak through the limitations of end-to-end transmissionand achieve the joint intelligent video transmission mech-anism of ldquoend-edge-cloudrdquo In terms of scheduling algo-rithms researchers have explored the adaptive rateadjustment from the end side intelligent video compressionmechanism based on server side intelligent video cachingand scheduling based on edge computing etc In addition inthe aspect of evaluation mechanism researchers haveconducted a lot of research on both subjective evaluationmechanism and objective evaluation mechanism in order toaccurately evaluate the video quality so as to design a moreaccurate QoE model and finally achieve the accuratematching between algorithm objectives and user QoE

In this paper we propose a video transmission opti-mization mechanism based on RL and EC and the maincontributions are summarized as follows (i) we design thepopularity prediction model for video requests based on theRL and introduce the adaptive video encoding method foroptimizing the efficiency of computing resource distribu-tion and (ii) we design a video caching strategy whichadopts EC to reduce the redundant video transmission

(e rest of the paper is structured as follows Section 2reviews the related work In Section 3 a novel videotransmission optimization mechanism based on RL and ECis proposed(e experimental results are reported in Section4 and finally Section 5 concludes this paper

2 Related Work

21 Optimization of Transmission Architecture In order toimprove the network transmission efficiency researchershave proposed many transmission architecture optimizationalgorithms (e objective of the algorithm is to analyze thekey problems from the traditional best-effort packet

forwarding mode of the Internet and improve the networktransmission efficiency in terms of eliminating transmissionredundancy cooperation between end side and networkside etc so as to reach an all-purpose video transmissionoptimization scheme

211 Content-Centric Network Content-centric network(CCN) [2] is a novel Internet architecture proposed by Lv in2009 which is content-centric and is a typical representativeof ICN [3 4] ICN achieves the interconnection of infor-mation in the network based on the information-centricnetwork communicationmode ICN abandons the traditionalInternet IP protocol and the corresponding packet switchingmechanism and reaches an efficient and secure contentdistribution mechanism by redesigning the network archi-tecture and introducing identification and network cachingICN extremely reduces the network redundancy by contentlabeling and network caching so as to improve the trans-mission efficiency of network videos Based on the namingmechanism routing mechanism distribution mechanismand caching mechanism ICN has evolved in many formsamong which the most representative scheme is CCN [2]

One of the key technologies of CCN is name-basedcontent routing(e implementation of name-based contentrouting includes twomain modules forwarding informationbase (FIB) and pending interest table (PIT) which are shownin Figure 1 FIB is able to forward interest messages to nodesthat may cache corresponding data Another key technologyof CCN is network caching(e router in CCN can achieve acontent storage module (as shown in Figure 1) which issimilar to the buffer space in the IP network but it can havedifferent content replacement strategies

212 Content Distribution Network Related schemes ofICN can improve the efficiency of network video trans-mission but it is difficult to implement such schemes in thecurrent Internet in the short term (erefore the contentdistribution scheme in the application layer has emerged inthe current Internet namely content distribution networkMany content service providers have been developed athome and abroad including Akamai in the United StatesNetresidence Technology and Blue Flood in China etc Inaddition Alibaba Cloud and Tencent can all provide contentacceleration services in CDN

As a platform of content accelerated distribution serviceproviders of CDN can provide content accelerated servicesfor content providers (CPs) CDN service providers deploymultiple CDN cloud platforms and CDN points of presence(PoPs) in the country or even around the world to promotethe content to be distributed (such as volume video) to thePoP in advance (e content requested by users is obtainedby redirecting to the nearest CDN PoP through DNS ca-nonical name As shown in Figure 2 from the perspective ofcontent transmission CDN can greatly decrease thebandwidth requirements of the backbone network andeliminate huge redundant transmission between CDN cloudplatforms and CDN PoPs so as to improve the networktransmission efficiency

2 Mobile Information Systems

In the current Internet content transmission CDN hasbeen widely deployed and applied and achieved obviousperformance improvement However CDN also faces someproblems and challenges First of all the cost of node de-ployment is high in CDN which is difficult to be widelydeployed as a heavy asset platform So the number of CDNPoPs is often limited and it is difficult to achieve extensivecoverage for massive users (erefore the transmissionefficiency between CDN PoPs is still low Second the processof content request in CDN is complex which results inadditional content request delay so as to affect the userexperience especially the time latency In addition CDN isclosed and independent based on the application layerwhere content service providers and network service

providers cannot participate in the optimization of contentdistribution so the available communication and jointoptimization mode between the network side and the endside are impossible to form

22 Video Transmission Optimization Algorithm

221 Adaptive Bitrate Adjustment Some researchers haveproposed adaptive bitrate (ABR) algorithms [5ndash7] Aimingat dynamic network available bandwidth the object of ABRis to achieve end-to-side adaptive bitrate adjustment andavoid lags to improve the user QoE of watching videos

(e detailed comparison and analysis of ABR algorithmhave been provided in literature [8 9] Specifically the

request content1

request content2

content1 hellip

content2 hellip

content1 port2 3

content2 port2 3

content1 port0 1

content storage

FIB

PIT

port0

port1

port2

port3

Figure 1 Illustration of CCN routing mechanism

CDN cloud platform

CDN PoP

Figure 2 Illustration of CDN architecture

Mobile Information Systems 3

comparison of paper [9] found that the configuration ofvarious parameters has a significant impact on ABR per-formance (erefore in practical application dynamicallyadjustment for ABR according to network state character-istics user system characteristics and other factors that issetting ABR parameters was a huge challenge To solve thisproblem MITrsquos research team proposed a reinforcementlearning-based intelligent dynamic bitrate adjustmentscheme called pensive [10] As shown in Figure 3 thescheme achieves intelligent and dynamic end-to-end videobitrate adjustment by reinforcement learning which effec-tively addressed challenges of the complex parameter con-figuration in ABR and showed an efficient videotransmission application effect

However the end-based ABR strategy still has limita-tions In this kind of scheme each video client dynamicallyadjusts the request policy according to its own network statewhich is based on the local optimal decision and is difficult toensure the global optimization of network bandwidth re-source utilization

222 Intelligent Video Transmission Based on EdgeComputing (e scheme to optimize the video user QoEshould have the following characteristics (1) it can optimizethe video transmission globally for users sharing bottleneckbandwidth rather than only making decisions locally (2) itcan reduce the redundancy of video transmission and ensurethe efficient utilization of network bandwidth and (3) it canobtain the network state in real time and it resists thedynamic network jitter by designing corresponding mech-anisms to ensure the smooth watching experience for videousers

Recently edge computing has emerged as a noveltechnology which can satisfy the demand for video trans-mission scheduling First the edge computing platform isclose to the terminal users and can provide the ability tooptimize video transmission for all users globally Secondthe edge computing platform has a strong ability of sensorystorage and computing which can effectively address theinsufficiency of network transmission ability (ereforevideo transmission based on edge computing can improvethe utilization of network bandwidth and transmission ef-ficiency of massive videos and it plays an important role inrealizing the joint optimization of user QoE

In literature [10 11] the authors proposed joint bitrateoptimization mechanisms based on edge computing (eseschemes make intelligent joint bitrate decisions throughdeep learning Compared with the traditional end-basedQoE optimization mechanism the optimization schemebased on edge computing has prominent advantages interms of total QoE

3 Video Transmission Optimization Based onRL and EC

31 Cloud-Based Intelligent Video Coding Mechanism(e video is increasingly popular as a core experience ofpeoplersquos online activities Only on Facebook more than 8

billion videos are viewed every day [12] (e client down-loads videos from the cloud server of the video provider byABR to watch videos [13 14] (e ABR algorithm candynamically select the highest bitrate that the networkbandwidth can support and avoid the jam phenomenonduring watching Higher bitrate can provide higher videoquality but it also results in more video transmissions so theend-to-end connection with the higher bandwidth is re-quired for clients

When the original videos are uploaded different basicbitrate versions of the videos are generated [15] whichconsumes huge computing resources In the network videotransmission there are more than 100 video resolutions andthe same resolution also contains multiple different videobitrates so the number of potential output types of videobitrates is large By default FFmpeg is used to encode thevideo uploaded to the server into a small number of standardversions More computation can improve the userrsquos videoviewing experience by improving the coding performance(decreasing the amount of transmitted data for the samevideo quality) or increasing the coding selection (providingmore fine-grained bitrate selection to adapt to the dynamicnetwork bandwidth) However the computing power ofvideo coding in the cloud is limited and it is impossible togenerate enough coding versions for all videos (ereforedynamically allocating appropriate coding power to thecloud among different videos to achieve the optimal globaluser experience is one of the problems to be solved in thenetwork video transmission

In this paper we propose a cloud-based intelligentvideo coding mechanism with popularity considerationassigning computing power and encoding bitrate versionsof videos according to the popularity However the pop-ularity of videos in the real situation is extremely imbal-anced where less than 1 of the videos contribute morethan 80 of the time spent in viewing so the imbalance isvery obvious (is feature is of great value to computingpower allocation for cloud dynamic coding In the cloudthe highest quality coding or more customized bitrateversions are produced on demand for a small number of themost popular videos so that the overall video viewingquality can be significantly improved with only a smallamount of computing power

311 Prediction of Video Popularity Based on ReinforcementLearning Analysis and prediction of video popularity arerequired for targeting cloud coding based on the feature ofhigh concentration of video watching In our scheme therequest processing logging mode is in charge of logging thesequence of video user requests including video ID requestbitrate request time terminal parameters (such as resolu-tion) etc (e popularity prediction should have followingcharacteristics first the prediction should be quick so that itcan decrease the number of missing video requests secondthe prediction should be accurate which can ensure that thecomputing is consumed on the most valuable videos andthird the prediction should be scalable to analyze andpredict massive request records


(e popularity prediction methods proposed in papers[16 17] mainly aimed at the analysis and prediction ofpopularity at the day level (ese methods need great pre-diction delay and the goal of this paper is to quickly predictthe popularity at the minute level so it is very important todesign a fast-incremental popularity prediction algorithmTo be able to further maintain stability and adaptability tonetwork dynamics we use reinforcement learning to predictthe popularity of videos

Video requests that occurred in the past time t will have animpact on the popularity of future moment T which is rep-resented by f (T-t) f is a function of probability distributiondefined on the space [0 +infin] which is generallymonotonicallydecreasing(erefore in principle themore recent the visit thegreater the effect on the popularity and the effect of a particularvisit on the popularity gradually converges to zero over timeFor a video ti represents the time of the visit i and the totalnumber of times to watch the video in the future time Tcan becalculated by the following formula

F(T) 1113944tileT

1113946+infin

tf t minus ti( 1113857dt (1)

(e key problem is to set the core probability densityfunction f to make incremental update possible so as toaccelerate the process of video popularity prediction Pre-vious works [18ndash20] used power law distribution as theprobability density function to predict the popularityHowever a complete calculation is required to solve thepopularity every time in this method which greatly de-creases the prediction speed and affects the timeliness of thepopularity feedback In this paper we use exponentialdistribution as the probability density function which canlargely reduce the computations needed for the popularityprediction and is expressed as follows

f(t) 1w

1113874 1113875eminus (tw)

(2)

where w indicates the range of the time window for futureimpact and it mainly serves to remove visits made long ago

which have minimal effect on the accuracy of popularityprediction and can be ignored For a video we suppose T2 isthe present request time of the video to trigger the presentpopularity upgrade and T1 is the last request time of therequest Aiming at current time T2 the future popularity ofthe video can be calculated by the following formula

F T2( 1113857 1113944tileT2

1113946+infin

t

1w

eminus tminus ti( )w( )dt

1w

+ 1113944tileT1

eminus T2minus ti( )w( )

1w

+ eminus T2minus T1( )w( ) 1113944

tileT1


1w

+ eminus T2minus T1( )w( )F T1( 1113857

(3)

Reinforcement learning is a field of machine learningwhich selects the action based on the environment tomaximize the expected benefits In reinforcement learningthe agent chooses an action to be acted in the environment[21ndash23] After the environment receives the action the statechanges and generates a reward according to the quality ofaction and the reward is forwarded to the agent (e agentselects the next action according to the reward and thecurrent state of the environment which form a positivefeedback mechanism and increase the probability to choosethe optimal action for each state [24 25] In this paper weapply reinforcement learning to popularity prediction anddesign a popularity prediction strategy based on rein-forcement learning which is able to further support thedynamic network and improve the accuracy of prediction

(e process of popularity prediction based on rein-forcement learning is shown in Figure 4 (e video requestsin the past time period t are regarded as the state thepredicted popularity on time T is regarded as the action andthe network performance of video transmission is consid-ered the reward (e agent chooses action as predicted

State

240 p

480 p

720 p

1080 p

QoE

ABR Neural network Bitrate

Network bandwidth

Bitrate

Player buffer

Reward

End-side network and player state awareness

Figure 3 Reinforcement learning-based intelligent pensive-ABR mechanism


popularity according to the reward (e proposed methodcan choose the popularity with the highest video trans-mission performance as the prediction popularity whichensures the accuracy of popularity prediction adapted to thedynamic network [26ndash28]

Specifically we adopt the Q-learning method to predictthe request popularity of videos We consider the videopopularity of the past time t as the state expressed as s andthe video request popularity at the moment T as the actionexpressed as a (en the Q value expressed as Q (s a) iscalculated as follows

Q(s a) Q(s a) + α r + cQ sprime aprime( 1113857 minus Q(s a)( 1113857 (4)

where α represents learning step c represents discountfactor for rewards and Q(sprime aprime) is the maximum Q of thestate sprime and action aprime at the next moment FurthermoreQ (sa) is obtained by the performance of the video transmissioncorresponding to that state and action In this paper theperformance expressed as P is set to be related to the requestdelay which is calculated as follows

P k times delay (5)

where k is the coefficient of impact of time delay onperformance

(en the action corresponding to the maximumQ valueis selected as the predicted video request popularity atmoment T which is expressed as

F(T) argmaxa

Q(s a) (6)

312 Adaptive Computing Power Allocation for VideoCoding (e computation distributionmanagementmode isresponsible for accepting both raw video regular encodingrequests and popularity-sensitive on-demand customencoding requests which also dynamically allocates andbalances CPU computational resources at the core level ofgranularity according to the different workloads of the tworequest types

Based on the above popularity prediction a set ofpopularity-sensitive customized coding task is obtainedPopularity prediction of the video is triggered and tasks inthe on-demand coding set are generated with differentpriorities because the bitrate requested by the user doesnot exist At the same time in our mechanism we

consider that even videos with the same popularity shouldhave different priorities because overall improvement ofthe user QoE may be different under the same computingpower For example the requested bitrate of video A is720p while there are only 180p 480p and 1080p in theactual video caching module Due to the bandwidthlimitation of the userrsquos requested bitrate the closest videoversion is 480p (1080p may cause huge lags due to in-sufficient bandwidth) If the requested bitrate of video B is720p and there are only 180p and 1080p in the actualvideo caching module the actual bitrate should be 180pIn the above case although the popularity prediction of Aand B is the same B should be given priority to conducton-demand coding to maximize the effect of QoE(erefore we introduce the QoE increment factorexpressed as

θ(x) x

radic (7)

where x indicates the multiplication coefficient between therequest bitrate and the response bitrate (at is when therequest and response bitrate are 720p and 480p respectivelythen x takes the value 15

(e computing power distribution managementmode receives the conventional original video codingrequest such as regular encoding requests encode the rawvideo in both 480p and 1080p by default In fact theamount of conventional coding can be increased or de-creased according to the computing power of the cloudvideo encoding platform (e remaining potentialencoding options including 180p 360p 720p in ac-cordance with the popularity of user video requeststrigger on-demand specialized encoding services thusproviding intelligent and specific encoding services Inthe actual video cloud platform the coding types involvedare far more than those mentioned in this paper (ecloud transcoding platform can dynamically allocate thecomputing power according to the actual computingpower and the conventional transcoding requirements oforiginal videos

32 Popularity-Based Intelligent Edge Caching MechanismMuch transmission redundancy is generated in theprocess of video transmission which has a strong localityin time and space that is a small number of videos arerequested by users in the same area many times in a shorttime (erefore as shown in Figure 5 the mechanismintroduces an edge computing platform which breaks thelimitation of traditional end-to-end video transmissionand achieves an intelligent video transmission mecha-nism of end-edge cooperation by edge caching And theprocess of the mechanism is specifically described asfollows

Step 1 (e edge computing platform receives videorequests from all users within its coverage areaStep 2 Search the local cache space in the edge com-puting platform

Transmission performanceT

Predicted popularity of T

Optional popularity

All popularity

Figure 4 Reinforcement learning-based popularity prediction


(1) if there is a corresponding video and the bitratematches completely the video cached in the edgeplatform is directly used to respond to the userrequest

(2) if the corresponding video is available but thebitrate does not exactly match and no superiorchoice is found in the cloud respond to the userrequest directly with the cached video and at thesame time inform the cloud of the request in orderto count and predict the video popularity

(3) if the corresponding video is available but thebitrate does not exactly match but a better optioncan be found in the cloud forward the video re-quest to the cloud for that user

(4) if no corresponding video is available the requestwill be directly forwarded to the cloud

Step 3 For videos responded by the cloud the edgecomputing platform caches these videos according tothe predicted popularity within the edge coverage andthe videos with lower popularity will be replacedpreferentially

4 Experimental Simulation and Result Analysis

41 Setups To prove the effectiveness and efficiency of themechanism simulations are conducted based on four partsvideo data mechanism settings video requests and com-parison simulations

411 Video Data We use 1000 videos for simulations and25 new videos will incrementally be uploaded every 1 sduring the experiments For each video 10 blocks arecontained and the playing time of each block is 2 s

412 Mechanism Settings

(1) Computing power model setting CPU computingpower is set to 400 cores With a single-core CPUcomputing power the video encoding time for eachbitrate is uniformly set to 5 s which means that thecloud computing power could handle 80 videoencoding missions at a second

(2) Regular transcoding power distribution setting (eencoding range of the video is considered 180p240p 360p 480p 540p 720p 960p 1080p All ofthese except for the regular coding are used as

potential on-demand custom coding requirementstriggered by the popularity of user requests Bydefault the original videos are encoded as 360p and720p bitrates Since the beginning of the experimentthe regular encoding of all new videos is requiredand this approach allocates 12 of the computingpower to regular encoding

(3) Edge computing platform (e cache capacity isconfigured depending on the storage capacity of 400videos with the bitrate of 180p If 360p is targeted theplatform is able to cache 200 videos and so on (etransmission latency between the edge and the user is5ms and the transmission latency from the cloud tothe edge platform is 200ms

(4) Network bandwidth setting Assuming that nobandwidth bottleneck exists between the edge andthe user and the downlink traffic between the cloudvideo platform and the edge platform can transmit400 video blocks (each video has 10 video blocks)with the bitrate of 480p per second For 960p only200 video blocks can be completed per second andso on

413 Video Requests Video requests distribution setting(e user chooses a video according to the Zipf (parameter107) probability distribution to request and randomlychooses a bitrate from 180p 240p 360p 480p 540p 720p960p 1080p

414 Comparison Simulations Comparison simulations areconducted among our mechanism joint coding-transmis-sion optimization (TOSO) [29] and joint rate control andbuffer management (JRCBM) [30] under different numbersof requests and the results are analyzed in terms of videorelative quality and video lag degree

42 Results Analysis

421 Video Relative Quality (e comparison simulationson video relative quality under different numbers of requestsare shown in Figure 6 Our algorithm is always the bestunder different numbers of requests because when the basicbitrates do not match the user request the coding task can becustomized to ensure the relative quality of the video

Edge computing platform Cloud video platform

Figure 5 Intelligent edge caching with popularity consideration


422 Video Lag Degree To analyze and compare the videolag degree in different algorithms the relative smoothnessindex of video viewing is considered the metric to evaluatethe video lag degree which is calculated as

r twch

twch + twit (8)

where twch indicates the duration of video viewing and twitindicates the duration of waiting during video viewingwhich includes the time of buffering process during startup

Comparison simulations on the video lag degree underdifferent request numbers are shown in Figure 7 and theproposed TORE is always the best under different numbersof requests We can explain the advantages of the proposedapproach in two aspects On one hand the EC-based in-telligent caching strategy adaptively allocates arithmeticpower and tasks to edge-side nodes which will decrease thetransmission latency of the requests On the other hand thepopularity-based edge intelligent caching reduces the re-dundant transmission of the network As a result the pathwill not be jammed to ensure the stability of the hugenetwork video transmission

423 Video Response Time As can be seen from Figure 8the proposed TORE has a good performance in responsetime (e intelligent caching method is implementedaccording to the regional popularity characteristics in the ECplatform which is combined with video forwarding tominimize the network transmission redundancy and max-imize the video transmission efficiency (e proposedscheme is of significant value for optimizing the video re-sponse time which can improve the network transmissionefficiency and user QoE

5 Conclusions

In this paper we propose a dynamic computing power al-location mechanism based on intelligent popularity pre-diction for video user distribution(e proposedmechanismcan take into account both the conventional encoding de-mand and the dynamic on-demand customized encodingdemand of users and can fully and reasonably utilize thelimited computing power in the cloud to adaptively allocatethe computing power to each server to reduce the responselatency of requests and thus improve QoE At the same time

200 400 600 800 1000e number of requests

0

1

Vide

o re

lativ

e qua

lity

TORETOSOJRCBM

01

02

03

04

05

06

07

08

09

Figure 6 Comparison on video relative quality under differentnumbers of requests

200 400 600 800 1000The number of requests

0

1

Vide

o la

g de

gree

01

02

03

04

05

06

07

08

09

TORETOSOJRCBM

Figure 7 Comparison on video lag degree under differentnumbers of requests


0

20

40

60

80

100

120

Resp

onse

tim

e (s)

TORETOSOJRCBM

Figure 8 Comparison on video response time under differentnumbers of requests


this scheme introduces the edge computing architecture andreinforcement learning method to achieve video popularityprediction which further realizes intelligent caching basedon video popularity We experimentally demonstrate thatthe proposed method can optimize the efficiency of videotransmission and reduce network latency

(e key research of the proposed optimization mecha-nism is to improve the video quality and response time ofusers in watching videos However compression anddecoding in video transmission optimization are not ana-lyzed In the future we can try to optimize the video contentby using different bitrates to encode the video streaming thatusers are interested in and uninterested in so as to directlyreduce the redundant traffic in video transmission

In particular to make the readers more easily follow thispaper the commonly used abbreviations are listed below

Abbreviation

ABR Adaptive bitrateAR Augmented realityCCN Content-centric networkCDN Content-distributed networkCP Content providerEC Edge computingFIB Forwarding information baseICN Information-centric networkJRCBM Joint rate control and buffer managementPIT Pending interest tablePoP Point of presenceQoE Quality of experienceRL Reinforcement learningTORE Transmission optimization with reinforcement

learning and edge computingTOSO Joint coding-transmission optimizationVR Virtual reality

Data Availability

All the data used to support the findings of the study areincluded within the article

Conflicts of Interest

(e authors declare that they have no conflicts of interest

Acknowledgments

(is paper was supported by the Youth Program ResearchProjects of Liaoning Higher Education Institutions (Grantno lnqn202014) the National Natural Science Foundationof China 61873174 and the Liaoning Provincial NaturalScience Foundation of China 2020-KF-11-07

References

[1] A Ghotbou and M Khansari ldquoVE-CoAP a constrainedapplication layer protocol for IoT video transmissionrdquoJournal of Network and Computer Applications vol 173pp 1ndash14 2021

[2] J Lv X Wang K Ren M Huang and K Li ldquoACO-inspiredinformation-centric networking routing mechanismrdquo Com-puter Networks vol 126 pp 200ndash217 2017

[3] B Ahlgren C Dannewitz C Imbrenda D Kutscher andB Ohlman ldquoA survey of information-centric networkingrdquoIEEE Communications Magazine vol 50 no 7 pp 26ndash362011

[4] D Trossen M Sarela and K Sollins ldquoArguments for aninformation-centric internetworking architecturerdquo ACMComputer Communications Review vol 40 no 4 pp 26ndash332010

[5] Z Akhtar Y S Nam R Govindan et al ldquoOboe auto-tuningvideo abr algorithms to network conditionsrdquo in Proceedings ofthe 2018 Conference of the ACM Special Interest Group onData Communication pp 44ndash58 ACM Budapest HungaryAugust 2018

[6] HMao R Netravali andM Alizadeh ldquoNeural adaptive videostreaming with pensieverdquo in Proceedings of the Conference ofthe ACM Special Interest Group on Data Communicationpp 197ndash210 ACM Los Angeles CA USA August 2017

[7] Y Qin R Jin S Hao et al ldquoA control theoretic approach toabr video streaming a fresh look at pid-based rate adapta-tionrdquo in Proceedings of the INFOCOM 2017-IEEE Conferenceon Computer Communications pp 1ndash9 IEEE Atlanta GAUSA May 2017

[8] F Y Yan H Ayers C Zhu et al ldquoLearning in situ a ran-domized experiment in video streamingrdquo 2019

[9] D Stohr A Frommgen A Rizk M Zink R Steinmetz andW Effelsberg ldquoWhere are the sweet spots a systematicapproach to reproducible dash player comparisonsrdquo inProceedings of the 25th ACM international conference onMultimedia ACM Mountain View CA USA October 2017

[10] HMao R Netravali andM Alizadeh ldquoNeural adaptive videostreaming with pensieverdquo in Proceedings of the 2016 ACMSIGCOMM Conference pp 197ndash210 ACM FlorianopolisBrazil August 2016

[11] X Ma Q Li J Chai X Xiao S Xia and Y Jiang ldquoStewardsmart edge based joint QoE optimization for adaptive videostreamingrdquo in Proceedings of the 29th ACM Workshop onNetwork and Operating Systems Support for Digital Audio andVideo pp 31ndash36 ACM Amherst MA USA June 2019

[12] L Zhang A Sun S Ryan J Liu and M Zhang ldquoRenderingmulti-party mobile augmented reality from edgerdquo in Pro-ceedings of the 29th ACM Workshop on Network and Oper-ating Systems Support for Digital Audio and Video pp 31ndash36ACM Amherst MA USA June 2019

[13] Facebook Facebook Community Update Facebook Cam-bridge MA USA 2021 httpswwwfacebookcomphotophpfbid=10102457977071041

[14] I Sodagar ldquo(e MPEG-DASH standard for multimediastreaming over the internetrdquo IEEE MultiMedia vol 18 no 42011

[15] T C(ang Q-D Ho J W Kang and A T Pham ldquoAdaptivestreaming of audiovisual content using MPEG DASHrdquo IEEETransactions on Consumer Electronics vol 58 no 1 2012

[16] Facebook Facebookrsquos Streaming Video Engine Scale TalkFacebook Cambridge MA USA 2021 httpswwwfacebookcomatscaleeventsvideos174171200496102047

[17] G Gursun M Crovella and I Matta ldquoDescribing andforecasting video access patternsrdquo in Proceedings of the 2011IEEE INFOCOM pp 16ndash20 IEEE Shanghai China April2011

[18] G Szabo and B A Huberman ldquoPredicting the popularity ofonline contentrdquo Communications of the ACM vol 53 2010


[19] R Crane and D Sornette ldquoRobust dynamic classes revealedby measuring the response function of a social systemrdquoProceedings of the National Academy of Sciences of the UnitedStates of America vol 105 no 41 pp 15649ndash15653 2008

[20] Q Zhao M A Erdogdu H Y He A Rajaraman andJ Leskovec ldquoSEISMIC a self-exciting point process model forpredicting tweet popularityrdquo 2015 httparxivorgabs150602594v1

[21] G Vecchio S Palazzo D Giordano F Rundo andC Spampinato ldquoMASK-RL multiagent video object seg-mentation framework through reinforcement learningrdquo IEEETransactions on Neural Networks and Learning Systemsvol 31 no 12 pp 1ndash13 2020

[22] L Ma S Cheng and Y Shi ldquoEnhancing learning efficiency ofbrain storm optimization via orthogonal learning designrdquoIEEE Transactions on Systems Man and Cybernetics Systemsvol 51 no 11 2020

[23] M Xu Y Song J Wang M Qiao L Huo and Z WangldquoPredicting head movement in panoramic video a deep re-inforcement learning approachrdquo IEEE Transactions on Pat-tern Analysis and Machine Intelligence vol 41 no 11pp 2693ndash2708 2018

[24] J Luo F R Yu Q Chen and L Tang ldquoAdaptive videostreaming with edge caching and video transcoding oversoftware-defined mobile networks a deep reinforcementlearning approachrdquo IEEE Transactions on Wireless Commu-nications vol 19 no 3 pp 1577ndash1592 2020

[25] K Arulkumaran M P Deisenroth M Brundage andA A Bharath ldquoDeep reinforcement learning a brief surveyrdquoIEEE Signal Processing Magazine vol 34 no 6 pp 26ndash382017

[26] E Skordilisa and R Moghaddass ldquoA deep reinforcementlearning approach for real-time sensor-driven decisionmaking and predictive analyticsrdquo Computers amp IndustrialEngineering vol 147 2020

[27] P-Y Yin and C-H Chao ldquoAutomatic selection of fittestenergy demand predictors based on cyber swarm optimiza-tion and reinforcement learningrdquo Applied Soft Computingvol 71 2018

[28] E Chalmers E B Contreras B Robertson A Luczak andA Gruber ldquoLearning to predict consequences as a method ofknowledge transfer in reinforcement learningrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 29no 6 pp 2259ndash2270 2018

[29] J-Y Wu K Wu and M Wang ldquoPower-constrained qualityoptimization for mobile video chatting with coding-trans-mission adaptationrdquo IEEE Transactions onMobile Computingvol 20 no 9 pp 2862ndash2876 2021

[30] C Liu and Y Dong ldquoQoE-aware video transmission opti-mization method for joint rate control and buffer manage-ment in LTE networksrdquo Journal of Nanjing University of Postsand Telecommunications vol 36 no 3 pp 59ndash67 2016


user experience have significant industrial value and aca-demic value In short the object of the video transmissionoptimization is to provide videos for users with maximumvideo resolution and minimum video stream stuck In ad-dition there are other factors to be considered such as theswitching frequency of video rate (ese factors are finallysummarized into the quality of experience (QoE) scorewhich is used to evaluate the quality of video transmission

At present academia mainly studies the optimizationmechanism of video transmission according to the trans-mission architecture scheduling algorithm and evaluationmechanism In terms of architecture there are inevitableconflicts between the best-efforts transmission mode oftraditional Internet based on TCPIP and the deterministicquality assurance of video transmission (erefore someresearchers put forward the novel network transmissionarchitectures such as information-centric networking(ICN) to improve the network transmission efficiency so asto optimize video transmission efficiency some researchersproposed content-distributed network (CDN) in the ap-plication layer based on the current Internet architecturenetwork to optimize the efficiency of video transmission inorder to make up for the low efficiency of Internet trans-mission some researchers proposed the intelligent videotransmission strategy using edge computing aiming tobreak through the limitations of end-to-end transmissionand achieve the joint intelligent video transmission mech-anism of ldquoend-edge-cloudrdquo In terms of scheduling algo-rithms researchers have explored the adaptive rateadjustment from the end side intelligent video compressionmechanism based on server side intelligent video cachingand scheduling based on edge computing etc In addition inthe aspect of evaluation mechanism researchers haveconducted a lot of research on both subjective evaluationmechanism and objective evaluation mechanism in order toaccurately evaluate the video quality so as to design a moreaccurate QoE model and finally achieve the accuratematching between algorithm objectives and user QoE

In this paper we propose a video transmission opti-mization mechanism based on RL and EC and the maincontributions are summarized as follows (i) we design thepopularity prediction model for video requests based on theRL and introduce the adaptive video encoding method foroptimizing the efficiency of computing resource distribu-tion and (ii) we design a video caching strategy whichadopts EC to reduce the redundant video transmission

(e rest of the paper is structured as follows Section 2reviews the related work In Section 3 a novel videotransmission optimization mechanism based on RL and ECis proposed(e experimental results are reported in Section4 and finally Section 5 concludes this paper

2 Related Work

21 Optimization of Transmission Architecture In order toimprove the network transmission efficiency researchershave proposed many transmission architecture optimizationalgorithms (e objective of the algorithm is to analyze thekey problems from the traditional best-effort packet

forwarding mode of the Internet and improve the networktransmission efficiency in terms of eliminating transmissionredundancy cooperation between end side and networkside etc so as to reach an all-purpose video transmissionoptimization scheme

211 Content-Centric Network Content-centric network(CCN) [2] is a novel Internet architecture proposed by Lv in2009 which is content-centric and is a typical representativeof ICN [3 4] ICN achieves the interconnection of infor-mation in the network based on the information-centricnetwork communicationmode ICN abandons the traditionalInternet IP protocol and the corresponding packet switchingmechanism and reaches an efficient and secure contentdistribution mechanism by redesigning the network archi-tecture and introducing identification and network cachingICN extremely reduces the network redundancy by contentlabeling and network caching so as to improve the trans-mission efficiency of network videos Based on the namingmechanism routing mechanism distribution mechanismand caching mechanism ICN has evolved in many formsamong which the most representative scheme is CCN [2]

One of the key technologies of CCN is name-basedcontent routing(e implementation of name-based contentrouting includes twomain modules forwarding informationbase (FIB) and pending interest table (PIT) which are shownin Figure 1 FIB is able to forward interest messages to nodesthat may cache corresponding data Another key technologyof CCN is network caching(e router in CCN can achieve acontent storage module (as shown in Figure 1) which issimilar to the buffer space in the IP network but it can havedifferent content replacement strategies

212 Content Distribution Network Related schemes ofICN can improve the efficiency of network video trans-mission but it is difficult to implement such schemes in thecurrent Internet in the short term (erefore the contentdistribution scheme in the application layer has emerged inthe current Internet namely content distribution networkMany content service providers have been developed athome and abroad including Akamai in the United StatesNetresidence Technology and Blue Flood in China etc Inaddition Alibaba Cloud and Tencent can all provide contentacceleration services in CDN

As a platform of content accelerated distribution serviceproviders of CDN can provide content accelerated servicesfor content providers (CPs) CDN service providers deploymultiple CDN cloud platforms and CDN points of presence(PoPs) in the country or even around the world to promotethe content to be distributed (such as volume video) to thePoP in advance (e content requested by users is obtainedby redirecting to the nearest CDN PoP through DNS ca-nonical name As shown in Figure 2 from the perspective ofcontent transmission CDN can greatly decrease thebandwidth requirements of the backbone network andeliminate huge redundant transmission between CDN cloudplatforms and CDN PoPs so as to improve the networktransmission efficiency







request content1

request content2

content1 hellip

content2 hellip

content1 port2 3

content2 port2 3

content1 port0 1

content storage

FIB

PIT

port0

port1

port2

port3


CDN cloud platform

CDN PoP

















F(T) 1113944tileT

1113946+infin



f(t) 1w

1113874 1113875eminus (tw)

(2)



F T2( 1113857 1113944tileT2

1113946+infin

t

1w


1w

+ 1113944tileT1


1w


tileT1


1w


(3)



State

240 p

480 p

720 p

1080 p

QoE


Network bandwidth

Bitrate

Player buffer

Reward








P k times delay (5)



F(T) argmaxa

Q(s a) (6)




θ(x) x

radic (7)







Optional popularity

All popularity



















42 Results Analysis






r twch

twch + twit (8)




5 Conclusions



0

1

Vide

o re

lativ

e qua

lity

TORETOSOJRCBM

01

02

03

04

05

06

07

08

09



0

1

Vide

o la

g de

gree

01

02

03

04

05

06

07

08

09

TORETOSOJRCBM



0

20

40

60

80

100

120

Resp

onse

tim

e (s)

TORETOSOJRCBM






Abbreviation



Data Availability




Acknowledgments


References






































request content1

request content2

content1 hellip

content2 hellip

content1 port2 3

content2 port2 3

content1 port0 1

content storage

FIB

PIT

port0

port1

port2

port3


CDN cloud platform

CDN PoP

















F(T) 1113944tileT

1113946+infin



f(t) 1w

1113874 1113875eminus (tw)

(2)



F T2( 1113857 1113944tileT2

1113946+infin

t

1w


1w

+ 1113944tileT1


1w


tileT1


1w


(3)



State

240 p

480 p

720 p

1080 p

QoE


Network bandwidth

Bitrate

Player buffer

Reward








P k times delay (5)



F(T) argmaxa

Q(s a) (6)




θ(x) x

radic (7)







Optional popularity

All popularity



















42 Results Analysis






r twch

twch + twit (8)




5 Conclusions



0

1

Vide

o re

lativ

e qua

lity

TORETOSOJRCBM

01

02

03

04

05

06

07

08

09



0

1

Vide

o la

g de

gree

01

02

03

04

05

06

07

08

09

TORETOSOJRCBM



0

20

40

60

80

100

120

Resp

onse

tim

e (s)

TORETOSOJRCBM






Abbreviation



Data Availability




Acknowledgments


References















































F(T) 1113944tileT

1113946+infin



f(t) 1w

1113874 1113875eminus (tw)

(2)



F T2( 1113857 1113944tileT2

1113946+infin

t

1w


1w

+ 1113944tileT1


1w


tileT1


1w


(3)



State

240 p

480 p

720 p

1080 p

QoE


Network bandwidth

Bitrate

Player buffer

Reward








P k times delay (5)



F(T) argmaxa

Q(s a) (6)




θ(x) x

radic (7)







Optional popularity

All popularity



















42 Results Analysis






r twch

twch + twit (8)




5 Conclusions



0

1

Vide

o re

lativ

e qua

lity

TORETOSOJRCBM

01

02

03

04

05

06

07

08

09



0

1

Vide

o la

g de

gree

01

02

03

04

05

06

07

08

09

TORETOSOJRCBM



0

20

40

60

80

100

120

Resp

onse

tim

e (s)

TORETOSOJRCBM






Abbreviation



Data Availability




Acknowledgments


References





































P k times delay (5)



F(T) argmaxa

Q(s a) (6)




θ(x) x

radic (7)







Optional popularity

All popularity



















42 Results Analysis






r twch

twch + twit (8)




5 Conclusions



0

1

Vide

o re

lativ

e qua

lity

TORETOSOJRCBM

01

02

03

04

05

06

07

08

09



0

1

Vide

o la

g de

gree

01

02

03

04

05

06

07

08

09

TORETOSOJRCBM



0

20

40

60

80

100

120

Resp

onse

tim

e (s)

TORETOSOJRCBM






Abbreviation



Data Availability




Acknowledgments


References

















































42 Results Analysis






r twch

twch + twit (8)




5 Conclusions



0

1

Vide

o re

lativ

e qua

lity

TORETOSOJRCBM

01

02

03

04

05

06

07

08

09



0

1

Vide

o la

g de

gree

01

02

03

04

05

06

07

08

09

TORETOSOJRCBM



0

20

40

60

80

100

120

Resp

onse

tim

e (s)

TORETOSOJRCBM






Abbreviation



Data Availability




Acknowledgments


References


































r twch

twch + twit (8)




5 Conclusions



0

1

Vide

o re

lativ

e qua

lity

TORETOSOJRCBM

01

02

03

04

05

06

07

08

09



0

1

Vide

o la

g de

gree

01

02

03

04

05

06

07

08

09

TORETOSOJRCBM



0

20

40

60

80

100

120

Resp

onse

tim

e (s)

TORETOSOJRCBM






Abbreviation



Data Availability




Acknowledgments


References




































Abbreviation



Data Availability




Acknowledgments


References

































A Novel Video Transmission Optimization Mechanism Based on ...

Documents

Transcript of A Novel Video Transmission Optimization Mechanism Based on ...