Video Codec Comparison Report
Transcript of Video Codec Comparison Report
-
8/8/2019 Video Codec Comparison Report
1/17
Video compression: Performance of
available codec software
1 Introduction1.1 DigitalVideoA digital video is a collection of images presented sequentially to produce the effect of continuous
motion. Ittakesadvantageofthespatiotemporalpropertiesofthehumaneyetosimulatecontinuityin
motion. Thepersistenceofthehumaneyeissuchthatnanosecondsofexposuretoanimageresultsin
millisecondsofimageontheretina.Hence, imagesplayedataspeedgreaterthanamillisecondwould
appeartobecontinuous.Ingeneral,theeyecannotdifferentiatebetweenindividualimageswhenthey
areplayedatarateof25persecondorhigher. Severalstandardsfortelevisionexist,whichdefinethe
framerateofthevideobeingdisplayed.SomeofthemaretheNTSC,PALetc.Theframeratevariesfrom
25fpsto60fpsdependingonthestandard.Thevideofileconsistsoftheindividualimages(alsoknownasframes)andthesequencinginformation.
1.2 TheSizeBarrierConsideravideothatisbeingplayedoutattherateof30 imagespersecond.Fora640x480grayscale
video represented in the raw lossless format, thatwouldbe640x480x30bytesper second. Fora30
minutevideo,thiswouldbeapproximately16GB.Foracolourvideo,usingthreebytesperpixel,that
wouldbe48GBnotevenincludingtheaudioandthesequencinginformation. Thisisalmostthesizeof2
blueraydiscsforasmallsizedSDvideo.ForsomemodernHDtransmissions,theframesizesare ashigh
as1920x1080whichwouldworkouttovideosizesbegreaterthan300GBwhenuncompressed.
1.3 VideoCompressionItisimpossibletoevenimaginetransmittingvideosofsuchhugesizes.Toreducethesizeofthevideo
tomanageableproportions,thevideosareusuallyneverstoredortransmittedintherawformat.Even
in situations where compression is not required, the video is still compressed. This is because, the
human eye is insensitive tohigher frequenciesandminute variations in colour and transmitting this
informationwouldbeawasteofresources.Everyvideoissubjectedtosomekindofcompression.The
compressionmethodisbasedupontheapplicationandbandwidthconstraintswherethevideoisused.
Compressionmaybeclassified
I. Baseduponthereproducibilityintoa. Losslesscompression
-
8/8/2019 Video Codec Comparison Report
2/17
As the name indicates, videos compressed using thismethod can be reproduced to the
original content without any change in data. Some methods which perform lossless
compression are Huffman coding, Run Length Coding etc. The amount of compression
achievedusingthesemethodsisverylesscomparedtothoseachievedusinglossmethods(
whicharediscussednext).Further, theamountof compression isalsogreatlydependent
uponthecontentofthevideo.
b. LossycompressionThiscompression isperformedbydropping informationwhichdoesnotsignificantlyaffect
thevisualizationofthevideo.Forexample,thehumaneyeisinsensitivetohighfrequencies
and also does not recognizeminor variations in colours. Hence this information can be
droppedwhileencodingthevideo.MethodssuchasJPEGperformlossycompression.
II. Baseduponwherethecompressionisperformedintoa. Intraframecompression
Thismethod
takes
advantage
of
the
spatial
redundancy
present
in
each
frame
of
the
video and compresses each frame based upon one of the compressionmethods. In
general, indoorvideoshaveauniform,nonchangingbackgroundandhaveaveryhigh
spatialredundancywhichcanbegreatlycompressed.
b. InterframecompressionThis method identifies the temporal redundancies between consecutive frames in a
videoandattemptstoremovethem.Usually,videosdonothavemuchscenechanges
andhencewillhavealotoftemporalredundancy.
UsuallygoodvideoformatsimplementbothInterandintraframecompressiontechniques.
1.4 VideoencodingformatsA video encoding format is a representation for compressed video. Such a format specifies the
representation of each frame, the sequencing information between frames and compression and
decompressionmethodsforinterandintraframeredundancy.
Althoughmaximumcompressionistargeted,usually,allformatshaveacertainamountofredundancy
inthem.Thisistomaintainperformanceinenvironmentswherethereisframedroppinganddataloss.
Errorpropagation resistancemechanismsarepartof thespecificationsofallvideoencoding formats.Thesealsoassistinseekingofdata.Withoutthese,everytimewewatchamove,wewouldhavetostart
fromthebeginningwithoutbeingabletocueforward.
-
8/8/2019 Video Codec Comparison Report
3/17
Somepopularformatsforvideoencodingare:
wmv Mpeg1 Mpeg4 Asf
1.5 ContainerformatsContainerformatsaredifferentfromencodingformats.Theyholdcombinationsofthevideoandaudio
encoded formats. They specify the bitrates of the audio and video and help maintain the
synchronizationbetween the audio and video. Some containers aredesigned to holdonly a specific
combinationofaudioandvideowhilesomearecapableofholdingseveralcombinations(butonlyone
combinationata time).Twopopular container formatsare .aviand .wmv. .avi canbeused tohold
severalvideoformatsincludingmpeg4andmpeg1
Somevideoencodingformatsarecontainersinthemselvesandarecapableofholdingbothaudioand
video.Forexample,MPEG1,MPEG4
1.6 CodecAcodec isanacronym forCoderDecoder. It iscapableofencodinga setof images intoavideoand
decodingavideo intoasetof images.Each imageusuallyconstitutesa frame in thevideo.However,
severaladditionalframesareaddedforthereasonsdiscussedearlier.
Eachcodeciscapableofworkingwithonlyaspecificvideoformat.However,severalcodecscanexistfor
asingleformat.Usually,eachmultimediacompanyformathasitsowncodecforitsplayerforaformat.
Forexample, theora,movareallcodecs for thempeg4 format.Codecs canbe ineither softwareor
hardware.Thesoftwarecodecsareslowerandinexpensiveascomparedtothehardwarecodecswhich
aremuchfaster.
The specifications for a format are not rigid and provide for some variations. Although codecs
implement a specified format, theymay vary in theirmethodof operation resulting in variations in
qualityandperformance.
2 CodecEvaluationWith the ever increasing need for bandwidth, codec designers tend to be over greedy and design
algorithmswhichmightbadlyaffecttheaesthecityofthevideocontent.Hence,evaluationcriteriafor
codecperformancesarerequiredtoverifythequalityofthecompressedvideos.
2.1 CriteriaforComparisonThecodecsarecomparedbasedonthefollowingcriteria
-
8/8/2019 Video Codec Comparison Report
4/17
1. QualityofVideo2. Performanceofthecodec
2.2 QualityofvideoQualityofvideocorrespondstothelookandfeelofthevideo,theresolution,theartifacts,theblurring
andothervisualaestheticcomponents.Thequalityofvideodependsonboththe formatofthevideoand the codec used to encode to that format. Usually, several codecs implement a single format.
However,eachonediffers fromtheother.Qualityalsodependsontheamountof informationonthe
video being encoded. Also, the performancewill not be constant throughout the video. Clipswith
higherinformationhavemoreartifactsthansceneswithlittlemovementandscenechanges.Qualitycan
bemeasuredasobjectiveorsubjective.
2.2.1 ObjectiveQualityObjectivequalityistomeasurethequalityinmathematicaltermswhichmakesitveryeasytocompare
andevaluate.Someofthemetricsavailabletomeasureobjectivequalityare:
a. MeanSquareError(MSE):Itisthesecondmomentofthedifferenceanddescribesthevariancebetweentheoriginalframeandtheencodedframe.
b. PeakSignaltoNoiseRatio(PSNR):Theratiobetweenthemaximumsignallevelandthenoise.Mathematically,itisgivenby:
c. ColourDifference:Thisistheabsolutedifferenceoftheindividualcolourcomponentsbetweentheinputframeandtheoutputframe.Itiscalculatedby
d. StructuralSimilarity(SSIM)[2]Thisisusedtomeasurethesimilaritybetweentwoimages.Itisanumberbetween0and1. Itisafunctionofluminance,contrastandstructuralsimilarity.Itis
independentofthecolourcomponents.
2.2.2 SubjectiveQualitySubjectivequality ismeasuredbyvisually inspectingtheencodedvideo forartifacts,blurring,blocking
andoverallquality.
-
8/8/2019 Video Codec Comparison Report
5/17
2.3 PerformanceoftheCodecTheperformanceofthecodecismeasuredasafunctionofthreequantities
1. Compressionratioofthecodec(Filesize)2. Speedofencoding(compression)3. Speedofdecoding(decompression)
2.3.1 CompressionratioofthecodecThecompressionratioofthecodecismeasuredbyencodingarepetitivesetofframesusingthecodecs
toyieldvideosofdifferentformats.Thefilesizeoftheencodedvideototheuncompressedvideowill
actasameasureofthecapacityforcompression.Byselectingappropriateframestocompress,wecan
measureboththebestandworstcasescenarios.
2.3.2 EncodingandDecodingspeedTheencodinganddecoding speedvary fromcodec tocodecandwithin the samecodec fordifferentframes.Higher the redundancy, slower the encoding and smaller is the size of the file. By selecting
appropriateframestocompress,wecanmeasureboththebestandworstcasescenarios.
2.4 BitRatesBitRateismeasuredinKiloBitspersecondandrepresentstheamountofdataflowperunittime.Itis
animportantfactorthatdecidesthequalityofthevideo.Forexample,consideravideowhichhasabit
rateof1000KbPS.Fora standarddefinitionvideo, thiswouldmean that therewouldbeabout29.8
framesinthe1000KiloBitsi.eabout33KiloBitsperframeor4Kilobytesperframe.Thisrestrictsthe
amountofdatathatcanbeusedtorepresentaframe.Lowerbitratesmeanhighercompressionand
lowerqualityofvideo,morenoise,blocking,discolourationetc.
Application BitRatesa Videostreaming 100500KbPS
b SDvideo 5002000KbPS
c HDvideo >2000KbPS
Bymeasuringeachofthequantitiesdiscussedin2.3and2.4,wewillbeabletoidentifytheappropriate
codecforaspecificapplication.
3 Implementation3.1 CodecsThefollowingcodecsarebeingevaluatedinthisstudy
-
8/8/2019 Video Codec Comparison Report
6/17
Sl.No Codec Designer/Developer Format Container1 WMV2 Microsoft wmv wmv2 Theora Xiph.org MPEG4 avi3 Asf Microsoft asf asf4 MPEG4 MPEG MPEG4 mp45 Quicktime Apple MPEG4 mov6 MPEG1 MPEG MPEG1 mpeg
Allcodecsarepartoftheffmpeglibrary.
3.2 DatasetThefollowingvideosareusedforevaluationofthecodecs.Thereasonforselectionofthevideoisalso
described.Allvideosareof352x288pixeldimension,butmayappearstretchedinthisdocument.
3.2.1 QualityMeasurement3.2.1.1Akiyo
Figure1 AframefromtheAkiyovideosequenceThis isa300framevideointheuncompressedYUVformat.Thisvideoshowsanewsreader.It hasno
backgroundchangesandalmostnegligibleforegroundchanges.
-
8/8/2019 Video Codec Comparison Report
7/17
3.2.1.2Foreman
Figure2AframefromtheForemanvideosequenceThisisa300framevideointheuncompressedYUVformat.Thisvideohasasuddenscenechangeatthe
end.Other
than
that,
there
is
no
background
change.
Only
the
face
shows
rich
emotions
which
can
be
hardtocompress
3.2.1.3Football
Figure3AframefromtheFootballvideosequenceThisisa125framevideointheuncompressedYUVformat.Thisvideohasaconstantbackgroundanda
veryrapidandlargechangeintheforegroundasplayerkeepcominginandgoingoutoftheframes.
-
8/8/2019 Video Codec Comparison Report
8/17
3.2.1.4Stephan
Figure4 AframefromtheStephanvideosequence
ThisisarecordingofStephanEdbergstennismatch.Thisis300framesinlengthandisalsointhe
uncompressedYUV
format.
This
video
has
a
fast
foreground
change
as
the
player
runs
about,
and
a
backgroundchangeasthecamerafollowshim.Thiswouldbethehardestkindofnaturalvideoto
encode.
Video ForegroundChange BackgroundChangeAkiyo
Foreman Football Stephan
3.2.2 PerformanceMeasurementIn order tomeasure the performance in terms of compression ratio and speed of encoding, I have
proposedasetofframesasshownbelow.Theseframeswilltogetherallowustomeasurethebestand
worstcasescenarios.
-
8/8/2019 Video Codec Comparison Report
9/17
Alternateframes SpatialRedundancy TemporalRedundancy3.2.2.1
100% 100%
3.2.2.2100% 0%
3.2.2.3=0% 100%
3.2.2.4=0% =0%
Thesepairsofalternatingframesincorporatethebestandworstcasescenariosforcompression.
3.2.3 BitRatesInordertocovertheentirerangeofapplications,thevideoswillbeencodedtothefollowingBitrates:
a. 600KbPSThisistherangeatwhichyoutubeplaysitsvideos.
b. 1,000KbPSThisisthebitratesgenerallyusedinvideoconferencing
c. 3,000KbPSThesebitratesaregenerallyusedinopticaldiscplaybacks.
4 ResultsandDiscussionAspartoftheexercise,Iwasabletomesuremostoftheevaluationparameters.However,duetoissues
withtheffmpeglibrary,Ididnotgetanaccuratemeasureofthecodinganddecodingtimes.
4.1 Akiyo
-
8/8/2019 Video Codec Comparison Report
10/17
Akio MSE
1.5
2
2.5
3000
wmv2
theora
asf
mpeg41
qt
0.5 mpeg1
0
600 1000
Figure5MeanSquaredErrorforAkiyo
Akio PSNR
48
wmv247
theora
43
44
45
46
1000 3000
asfmpeg4
qt
mpeg1
600
Figure6PSNRforAkiyo
Akio Absolute Color Distance
0
0.2
0.4
0.6
0.8
1
2
3000
1.
wmv2
theora
asf
mpeg4
qt
mpeg1
600 1000
Figure7AbsoluteColourdistanceforAkiyo
Akio SSIM
0.9965
0.997
0.9975
0.998
0.9985
0.999
0.9995
1
3000
wmv2
theora
asf
mpeg4
qt
mpeg1
0.996
600 1000
Figure8StructuralSimilarityforAkiyo
-
8/8/2019 Video Codec Comparison Report
11/17
F ootballMS E
2
3
4
5
6
7
8
9
10
3000
wmv2
Theora
as f
mpeg4
Qt
mpeg1
1
0
600 1000
Figure9MeanSquaredErrorforFootball
F ootballPSNR
36
600
37
38
39
40
41
42
43
1000 3000
wmv
Theora
as f
mpeg4
Qt
mpeg1
F ootballAbs oluteColorDistance
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
600 1000 3000
wmv2
theora
as f
mpeg4
Qt
mpeg1
F ootballSS IM
0.88
600 1000
0.9
0.92
0.94
0.96
0.98
1
3000
wmv2
Theora
as f
mpeg4
Qt
mpeg1
Figure10PSNRforFootball
Figure11AbsoluteColourDistanceforFootball
Figure12StructuralSimilarityforFootball
-
8/8/2019 Video Codec Comparison Report
12/17
ForemanMSE
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
600 1000 3000
wmv2
Theora
as f
mpeg4
Qt
mpeg1
Figure13MeanSquaredErrorforforemanForemanPS NR
39
40
41
42
43
44
45
46
47
600 1000 3000
wmv2
Theora
as f
mpeg4
Qt
mpeg1
Figure14PSNRforforemanForemanAbsoluteColorDistance
0
0.5
1
1.5
2
2.5
3
600 1000 3000
wmv2
theora
as f
mpeg4
Qt
mpeg1
Figure15AbsoluteColourDistanceforforeman
F oremanSS IM
0.975
0.98
0.985
0.99
0.995
1
1.005
600 1000 3000
wmv2
Theora
as f
mpeg4
Qt
mpeg1
Figure16StructuralSimilarityforforeman
-
8/8/2019 Video Codec Comparison Report
13/17
Stephan MSE
0
2
4
6
8
10
600 1000 3000
wmv2
theora
asf
mpeg4
qt
mpeg1
Figure17MeanSquaredErrorforStephan
Stephan PSNR
36
37
38
39
40
41
42
43
44
45
600 1000 3000
wmv2
theora
asf
mpeg4
qt
mpeg1
Figure18PSNRforStephan
Stephan Absolute Color Distance
0
1
2
3
4
5
600 1000 3000
wmv2theora
asf
mpeg4
qt
mpeg1
Figure19AbsoluteColourDistanceforStephan
Stephan SSIM
0.88
0.9
0.92
0.94
0.96
0.98
11.02
600 1000 3000
wmv2
theora
asf
mpeg4
qt
mpeg1
Figure20StructuralSimilarityforStephan
-
8/8/2019 Video Codec Comparison Report
14/17
0 200 400 600 800 1000 1200 1400 1600
Size in KB
1
B-W File Sizes
0 100 200 300 400
Size in KB
1
W-W File Sizes
mpeg-1
Qt
mpeg-4
asf
theora
w mv2
Figure21 Filesizeswithhighspatialandlowtemporalredunda
Figure22Filesizeswithhighspatialandtemporalredundancy
Figure24 FilesizeswithlowspatialandlowtemporalredundaFigure23Filesizeswithlowspatialandhightemporalredundancy
0 500 1000 1500 2000 2500 3000
Size in KB
1
C-C File Sizes
mpeg-1
Qt
mpeg-4
asf
theora
w mv2
0 5000 10000 15000 20000
Size in KB
1
C-N File Sizes
-
8/8/2019 Video Codec Comparison Report
15/17
Followingaresomesampleframesfromtheencodedvideos
Figure25 CounterClockwisefromthetopaframefromtheStephanvideooriginalframe,wmvencodedat600kbpsandwmvencodedat3000kbps
Figure26 CounterClockwisefromthetopaframefromtheAkiyovideooriginalframe,wmvencodedat600kbpsandwmvencodedat1000kbps
-
8/8/2019 Video Codec Comparison Report
16/17
InFigure25,thedistortionisclearlyvisiblewhenencodedat600kbps,butat3000kbps,itisalmost
negligible. However,inFigure26,thereisnovisibledistortionevenat600kbps.Thisimpliesthatthe
encodingprocessissensitivetothecontentofthevideoalso.
5 ConclusionSelection of a format for encoding or representation depends upon the applicationwhich uses the
video.Thevariouscriteriatobeconsideredbeforeselectingaformatare:
Applicationo Transmission
Videos used for transmission and viewing over the internet require a high
compression ratio.They can compromiseon thequalityas suchvideosare rarely
usedforimportantapplications.
o VideoConferencingVideoconferencingapplicationshavespecificcriteriawhenitcomestoquality.They
need thevideostobeclear,butthe frameratecanbecompromised.Surveillance
videos also fall into this category. The encoding and decoding speed are of
significancehere.
o ArchivingVideos used for this purpose do not have significant demands on encoding or
decodingspeed.Theyrequirehigherresolutionandqualitywithlowerfilesizes.
PerformanceRequirementso RealtimevideoprocessingforUAVsetc.
Therequirementhereisforfasterencodingspeedandverylittleblurring
o VideoViewingVideo viewing, in general, does not have much processing requirements. This is
because of the availability of sufficient processing capability and non real time
natureoftheapplication.
Qualityrequirementso Entertainmento Conferencingo Surgicalprocedures
6 FutureWorkPossiblefutureworkincludes
-
8/8/2019 Video Codec Comparison Report
17/17
a. Measuringblurringeffectsofthecodecsb. Measuringblockingeffectsandimpactonedgedetectionalgorithmsc. Evaluatingcodinganddecodingtimes.d. Identifyingimpactofframesizeoncodingspeedandcompressionratio.
7 References[1] Madhuri Khambete, and Madhuri Joshi, Blur and Ringing Artifact Measurement Image Compression
using Wavelet Transform, PROCEEDINGS OF WORLD ACADEMY OF SCIENCE, ENGINEERING AND
TECHNOLOGY VOLUME 20 APRIL 2007 ISSN 1307-6884
[2] Zhou Wang, Alan Conrad Bovik, Hamid Rahim Sheikh, and Eero P. Simoncelli, Image Quality
Assessment: From Error Visibility to Structural Similarity , IEEE TRANSACTIONS ON IMAGEPROCESSING, VOL. 13, NO. 4, APRIL 2004