Learning Image Processing with OpenCV - DropPDFTools to develop new projects Creating an OpenCV C++...

LearningImageProcessingwithOpenCV

TableofContents

Credits

AbouttheAuthors

AbouttheReviewers

www.PacktPub.com

Supportfiles,eBooks,discountoffers,andmore

Whysubscribe?

FreeaccessforPacktaccountholders

Preface

Whatthisbookcovers

Whatyouneedforthisbook

Whothisbookisfor

Conventions

Readerfeedback

Customersupport

Downloadingtheexamplecode

Downloadingthecolorimagesofthisbook

Errata

Piracy

Questions

1.HandlingImageandVideoFiles

AnintroductiontoOpenCV

DownloadingandinstallingOpenCV

GettingacompilerandsettingCMake

ConfiguringOpenCVwithCMake

Compilingandinstallingthelibrary

ThestructureofOpenCV

CreatinguserprojectswithOpenCV

Generalusageofthelibrary

Toolstodevelopnewprojects

CreatinganOpenCVC++programwithQtCreator

Readingandwritingimagefiles

ThebasicAPIconcepts

Imagefile-supportedformats

Theexamplecode

Readingimagefiles

Eventhandlingintotheintrinsicloop

Writingimagefiles

Readingandwritingvideofiles

Theexamplecode

User-interactionstools

Trackbars

Mouseinteraction

Buttons

Drawinganddisplayingtext

Summary

2.EstablishingImageProcessingTools

Basicdatatypes

Pixel-levelaccess

Measuringthetime

Commonoperationswithimages

Arithmeticoperations

Datapersistence

Histograms

Theexamplecode

Summary

3.CorrectingandEnhancingImages

Imagefiltering

Smoothing

Theexamplecode

Sharpening

Theexamplecode

Workingwithimagepyramids

Gaussianpyramids

Laplacianpyramids

Theexamplecode

Morphologicaloperations

Theexamplecode

Geometricaltransformations

Affinetransformation

Scaling

Theexamplecode

Translation

Theexamplecode

Imagerotation

Theexamplecode

Skewing

Theexamplecode

Reflection

Theexamplecode

Perspectivetransformation

Theexamplecode

Inpainting

Theexamplecode

Denoising

Theexamplecode

Summary

4.ProcessingColor

Colorspaces

Conversionbetweencolorspaces(cvtColor)

Theexamplecode

Grayscale

Examplecode

CIEXYZ

Theexamplecode

CIEL*a*b*

Theexamplecode

CIEL*u*v*

Theexamplecode

Color-space-basedsegmentation

HSVsegmentation

YCrCbsegmentation

Colortransfer

Theexamplecode

Summary

5.ImageProcessingforVideo

Videostabilization

Superresolution

Stitching

Summary

6.ComputationalPhotography

High-dynamic-rangeimages

CreatingHDRimages

Example

Tonemapping

Alignment

Exposurefusion

Seamlesscloning

Decolorization

Non-photorealisticrendering

Summary

7.AcceleratingImageProcessing

OpenCVwiththeOpenCLinstallation

AquickrecipetoinstallOpenCVwithOpenCL

ChecktheGPUusage

Acceleratingyourownfunctions

CheckingyourOpenCL

Thecodeexplanation

YourfirstGPU-basedprogram

Thecodeexplanation

Goingrealtime

Thecodeexplanation

Theperformance

Summary

Allrightsreserved.Nopartofthisbookmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformorbyanymeans,withoutthepriorwrittenpermissionofthepublisher,exceptinthecaseofbriefquotationsembeddedincriticalarticlesorreviews.

Everyefforthasbeenmadeinthepreparationofthisbooktoensuretheaccuracyoftheinformationpresented.However,theinformationcontainedinthisbookissoldwithoutwarranty,eitherexpressorimplied.Neithertheauthors,norPacktPublishing,anditsdealersanddistributorswillbeheldliableforanydamagescausedorallegedtobecauseddirectlyorindirectlybythisbook.

PacktPublishinghasendeavoredtoprovidetrademarkinformationaboutallofthecompaniesandproductsmentionedinthisbookbytheappropriateuseofcapitals.However,PacktPublishingcannotguaranteetheaccuracyofthisinformation.

Firstpublished:March2015

Productionreference:1230315

PublishedbyPacktPublishingLtd.

LiveryPlace

35LiveryStreet

BirminghamB32PB,UK.

ISBN978-1-78328-765-9

www.packtpub.com

CreditsAuthors

GloriaBuenoGarcía

OscarDenizSuarez

JoséLuisEspinosaAranda

JesusSalidoTercero

IsmaelSerranoGracia

NoeliaVállezEnano

Reviewers

WalterLucetti

AndrédeSouzaMoreira

MarvinSmith

CommissioningEditor

JulianUrsell

AcquisitionEditor

SamWood

ContentDevelopmentEditor

KirtiPatil

TechnicalEditor

FaisalSiddiqui

CopyEditor

StutiSrivastava

ProjectCoordinator

NidhiJoshi

Proofreaders

MartinDiver

MariaGould

SamanthaLyon

Indexer

TejalSoni

Graphics

AbhinashSahu

ProductionCoordinator

ConidonMiranda

CoverWork

ConidonMiranda

AbouttheAuthorsGloriaBuenoGarcíaholdsaPhDinmachinevisionfromCoventryUniversity,UK.Shehasexperienceworkingastheprincipalresearcherinseveralresearchcenters,suchasUMR7005researchunitCNRS/LouisPasteurUniv.Strasbourg(France),GilbertGilkes&GordonTechnology(UK),andCEITSanSebastian(Spain).Sheistheauthoroftwopatents,oneregisteredtypeofsoftware,andmorethan100refereedpapers.Herinterestsarein2D/3Dmultimodalityimageprocessingandartificialintelligence.SheleadstheVISILABresearchgroupattheUniversityofCastilla-LaMancha.ShehascoauthoredabookonOpenCVprogrammingformobiledevices:OpenCVessentials,PacktPublishing.

Thisisdedicatedtooursonsforthetimewehavenotbeenabletoplaywiththemandourparentsfortheirunconditionalsupportduringourlifetime.ThanksfromGloriaandOscar.

OscarDenizSuarez‘sresearchinterestsaremainlyfocusedoncomputervisionandpatternrecognition.Heistheauthorofmorethan50refereedpapersinjournalsandconferences.Hereceivedtherunner-upawardforthebestPhDworkoncomputervisionandpatternrecognitionbyAERFAIandtheImageFileandReformattingSoftwareChallengeAwardbyInnocentiveInc.Hehasbeenanationalfinalistforthe2009CorBaayenaward.Hisworkisusedbycutting-edgecompanies,suchasExistor,Gliif,Tapmedia,E-Twenty,andothers,andhasalsobeenaddedtoOpenCV.Currently,heworksasanassociateprofessorattheUniversityofCastilla-LaManchaandcontributestoVISILAB.HeisaseniormemberofIEEEandisaffiliatedwithAAAI,SIANI,CEA-IFAC,AEPIA,andAERFAI-IAPR.HeservesasanacademiceditorofthePLoSONEjournal.HehasbeenavisitingresearcheratCarnegieMellonUniversity,ImperialCollegeLondon,andLeicaBiosystems.HehascoauthoredtwobooksonOpenCVpreviously.

JoséLuisEspinosaArandaholdsaPhDincomputersciencefromtheUniversityofCastilla-LaMancha.HehasbeenafinalistforCertamenUniversitarioArquímedesdeIntroducciónalaInvestigacióncientíficain2009forhisfinaldegreeprojectinSpain.Hisresearchinterestsinvolvecomputervision,heuristicalgorithms,andoperationalresearch.HeiscurrentlyworkingattheVISILABgroupasanassistantresearcheranddeveloperincomputervisiontopics.

Thisisdedicatedtomyparentsandmybrothers.

JesusSalidoTercerogainedhiselectricalengineeringdegreeandPhD(1996)fromUniversidadPolitécnicadeMadrid(Spain).Hethenspent2years(1997and1998)asavisitingscholarattheRoboticsInstitute(CarnegieMellonUniversity,Pittsburgh,USA),workingoncooperativemultirobotsystems.SincehisreturntotheSpanishUniversityofCastilla-LaMancha,hespendshistimeteachingcoursesonroboticsandindustrialinformatics,alongwithresearchonvisionandintelligentsystems.Overthelast3years,hiseffortshavebeendirectedtodevelopvisionapplicationsonmobiledevices.HehascoauthoredabookonOpenCVprogrammingformobiledevices.

ThisisdedicatedtothosetowhomIoweallIam:myparents,SagrarioandMaria.

IsmaelSerranoGraciareceivedhisdegreeincomputersciencein2012fromthe

UniversityofCastilla-LaMancha.Hegotthehighestmarksforhisfinaldegreeprojectonpersondetection.ThisapplicationusesdepthcameraswithOpenCVlibraries.Currently,heisaPhDcandidateatthesameuniversity,holdingaresearchgrantfromtheSpanishMinistryofScienceandResearch.HeisalsoworkingattheVISILABgroupasanassistantresearcheranddeveloperondifferentcomputervisiontopics.

Thisisdedicatedtomyparents,whohavegivenmetheopportunityofeducationandhavesupportedmethroughoutmylife.Itisalsodedicatedtomysupervisor,Dr.OscarDeniz,whohasbeenafriend,guide,andhelper.Finally,itisdedicatedtomyfriendsandmygirlfriend,whohavealwayshelpedmeandbelievedthatIcoulddothis.

NoeliaVállezEnanohaslikedcomputerssinceherchildhood,thoughshedidn’thaveonebeforehermid-teens.In2009,shefinishedherstudiesincomputerscienceattheUniversityofCastilla-LaMancha,whereshegraduatedwithtophonors.ShestartedworkingattheVISILABgroupthroughaprojectonmammographyCADsystemsandelectronichealthrecords.Sincethen,shehasobtainedamaster’sdegreeinphysicsandmathematicsandhasenrolledforaPhDdegree.Herworkinvolvesusingimageprocessingandpatternrecognitionmethods.Shealsolikesteachingandworkinginotherareasofartificialintelligence.

AbouttheReviewersWalterLucetti,knownontheInternetasMyzhar,isanItaliancomputerengineerwithaspecializationinroboticsandroboticsperception.Hereceivedthelaureadegreein2005whilestudyingatResearchCenter“E.Piaggio”inPisa(Italy),wherehewroteathesisabout3Dmappingoftherealworldusinga2Dlasertiltedwithaservomotor,fusing3DwithRGBdata.Whilewritingthethesis,heencounteredOpenCVforthefirsttime—itwasearly2004andOpenCVwasatitslarvalstage.

Afterthelaurea,hestartedworkingassoftwaredeveloperforlow-levelembeddedsystemsandhigh-leveldesktopsystems.HegreatlyimprovedhisknowledgeofcomputervisionandmachinelearningasaresearcheratGustavoStefaniniAdvancedRoboticsCenterinLaSpezia(Italy),aspinoffofPERCROLaboratoryofScuolaSuperioreSant’AnnaofPisa(Italy).

Currently,heisworkinginthesoftwareindustry,writingfirmwareforembeddedARMsystems,softwarefordesktopsystemsbasedontheQtframework,andintelligentalgorithmsforvideosurveillancesystemsbasedonOpenCVandCUDA.

Heisalsoworkingonapersonalroboticproject:MyzharBot.MyzharBotisatrackedgroundmobilerobotthatusescomputervisiontodetectobstaclesandtoanalyzeandexploretheenvironment.TherobotisguidedbyalgorithmsbasedonROS,CUDA,andOpenCV.Youcanfollowtheprojectonthiswebsite:http://myzharbot.robot-home.it.

AndrédeSouzaMoreirahasamaster’sdegreeincomputerscience,withemphasisoncomputergraphicsfromthePontificalCatholicUniversityofRiodeJaneiro(Brazil).

HegraduatedwithabachelorofcomputersciencedegreefromUniversidadeFederaldoMaranhão(UFMA)inBrazil.Duringhisundergraduatedegree,hewasamemberofLabmint’sresearchteamandworkedwithmedicalimaging,specifically,breastcancerdetectionanddiagnosisusingimageprocessing.

Currently,heworksasaresearcherandsystemanalystatInstitutoTecgraf,oneofthemajorresearchanddevelopmentlabsincomputergraphicsinBrazil.HehasbeenworkingextensivelywithPHP,HTML,andCSSsince2007,andnowadays,hedevelopsprojectsinC++11/C++14,alongwithSQLite,Qt,Boost,andOpenGL.Moreinformationabouthimcanbeacquiredonhispersonalwebsiteatwww.andredsm.com.

MarvinSmithiscurrentlyasoftwareengineerinthedefenseindustry,specializinginphotogrammetryandremotesensing.HereceivedhisBSdegreeincomputersciencefromtheUniversityofNevadaReno.Histechnicalinterestsincludehighperformancecomputing,distributedimageprocessing,andmultispectralimageryexploitation.Priortoworkingindefense,MarvinheldinternshipswiththeIntelligentRoboticsGroupattheNASAAmesResearchCenterandtheNevadaAutomotiveTestCenter.

www.PacktPub.com

Supportfiles,eBooks,discountoffers,andmoreForsupportfilesanddownloadsrelatedtoyourbook,pleasevisitwww.PacktPub.com.

DidyouknowthatPacktofferseBookversionsofeverybookpublished,withPDFandePubfilesavailable?YoucanupgradetotheeBookversionatwww.PacktPub.comandasaprintbookcustomer,youareentitledtoadiscountontheeBookcopy.Getintouchwithusat<service@packtpub.com>formoredetails.

Atwww.PacktPub.com,youcanalsoreadacollectionoffreetechnicalarticles,signupforarangeoffreenewslettersandreceiveexclusivediscountsandoffersonPacktbooksandeBooks.

https://www2.packtpub.com/books/subscription/packtlib

DoyouneedinstantsolutionstoyourITquestions?PacktLibisPackt’sonlinedigitalbooklibrary.Here,youcansearch,access,andreadPackt’sentirelibraryofbooks.

Whysubscribe?FullysearchableacrosseverybookpublishedbyPacktCopyandpaste,print,andbookmarkcontentOndemandandaccessibleviaawebbrowser

FreeaccessforPacktaccountholdersIfyouhaveanaccountwithPacktatwww.PacktPub.com,youcanusethistoaccessPacktLibtodayandview9entirelyfreebooks.Simplyuseyourlogincredentialsforimmediateaccess.

PrefaceOpenCV,arguablythemostwidelyusedcomputervisionlibrary,includeshundredsofready-to-useimagingandvisionfunctionsandisusedinbothacademiaandindustry.Ascamerasgetcheaperandimagingfeaturesgrowindemand,therangeofapplicationsusingOpenCVincreasessignificantly,bothfordesktopandmobileplatforms.

Thisbookprovidesanexample-basedtourofOpenCV’smainimageprocessingalgorithms.WhileotherOpenCVbookstrytoexplaintheunderlyingtheoryorprovidelargeexamplesofnearlycompleteapplications,Thisbookisaimedatpeoplewhowanttohaveaneasy-to-understandworkingexampleassoonaspossible,andpossiblydevelopadditionalfeaturesontopofthat.

Thebookstartswithanintroductorychapterinwhichthelibraryinstallationisexplained,thestructureofthelibraryisdescribed,andbasicimageandvideoreadingandwritingexamplesaregiven.Fromthis,thefollowingfunctionalitiesarecovered:handlingofimagesandvideos,basicimageprocessingtools,correctingandenhancingimages,color,videoprocessing,andcomputationalphotography.Lastbutnotleast,advancedfeaturessuchasGPU-basedaccelerationsarealsoconsideredinthefinalchapter.Newfunctionsandtechniquesinthelatestmajorrelease,OpenCV3,areexplainedthroughout.

WhatthisbookcoversChapter1,HandlingImageandVideoFiles,showsyouhowtoreadimageandvideofiles.Italsoshowsbasicuser-interactiontools,whichareveryusefulinimageprocessingtochangeaparametervalue,selectregionsofinterest,andsoon.

Chapter2,EstablishingImageProcessingTools,describesthemaindatastructuresandbasicproceduresneededinsubsequentchapters.

Chapter3,CorrectingandEnhancingImages,dealswithtransformationstypicallyusedtocorrectimagedefects.Thischaptercoversfiltering,pointtransformationsusingLookUpTables,geometricaltransformations,andalgorithmsforinpaintinganddenoisingimages.

Chapter4,ProcessingColor,dealswithcolortopicsinimageprocessing.Thischapterexplainshowtousedifferentcolorspacesandperformcolortransfersbetweentwoimages.

Chapter5,ImageProcessingforVideo,coverstechniquesthatuseavideoorasequenceofimages.Thischapterisfocusedonalgorithms’implementationforvideostabilization,superresolution,andstitching.

Chapter6,ComputationalPhotography,explainshowtoreadHDRimagesandperformtonemappingonthem.

Chapter7,AcceleratingImageProcessing,coversanimportanttopicinimageprocessing:speed.ModernGPUsarethebestavailabletechnologytoacceleratetime-consumingimageprocessingtasks.

WhatyouneedforthisbookThepurposeofthisbookistoteachyouOpenCVimageprocessingbytakingyouthroughanumberofpracticalimageprocessingprojects.Thelatestversion,Version3.0ofOpenCV,willbeused.

Eachchapterprovidesseveralready-to-useexamplestoillustratetheconceptscoveredinit.Thebookis,therefore,focusedonprovidingyouwithaworkingexampleassoonaspossiblesothattheycandevelopadditionalfeaturesontopofthat.

Tousethisbook,onlyfreesoftwareisneeded.AlltheexampleshavebeendevelopedandtestedwiththefreelyavailableQtCreatorIDEandGNU/GCCcompiler.TheCMaketoolisalsousedtoconfigurethebuildprocessoftheOpenCVlibraryonthetargetplatform.Moreover,thefreelyavailableOpenCLSDKisrequiredfortheGPUaccelerationexamplesshowninChapter7,AcceleratingImageProcessing.

WhothisbookisforThisbookisintendedforreaderswhoalreadyknowC++programmingandwanttolearnhowtodoimageprocessingusingOpenCV.Youareexpectedtohaveaminimalbackgroundinthetheoryofimageprocessing.Thebookdoesnotcovertopicsthataremorerelatedtocomputervision,suchasfeatureandobjectdetection,tracking,ormachinelearning.

ConventionsInthisbook,youwillfindanumberofstylesoftextthatdistinguishbetweendifferentkindsofinformation.Herearesomeexamplesofthesestyles,andanexplanationoftheirmeaning.

Codewordsintext,foldernames,filenames,fileextensions,pathnames,systemvariables,URLs,anduserinputareshownasfollows:“Eachmodulehasanassociatedheaderfile(forexamplecore.hpp).”

Ablockofcodeissetasfollows:

#include<opencv2/core/core.hpp>

#include<opencv2/highgui/highgui.hpp>

usingnamespacestd;

usingnamespacecv;

intmain(intargc,char*argv[])

Matframe;//Containerforeachframe

Whenwewishtodrawyourattentiontoaparticularpartofacodeblock,therelevantlinesoritemsaresetinbold:

#include<opencv2/highgui/highgui.hpp>

#include<iostream>

usingnamespacestd;

usingnamespacecv;

Anycommand-lineinputoroutputiswrittenasfollows:

C:\opencv-buildQt\install

Newtermsandimportantwordsareshowninbold.Wordsthatyouseeonthescreen,forexample,inmenusordialogboxes,appearinthetextlikethis:“ClickingtheNextbuttonmovesyoutothenextscreen.”

NoteWarningsorimportantnotesappearinaboxlikethis.

TipTipsandtricksappearlikethis.

ReaderfeedbackFeedbackfromourreadersisalwayswelcome.Letusknowwhatyouthinkaboutthisbook—whatyoulikedordisliked.Readerfeedbackisimportantforusasithelpsusdeveloptitlesthatyouwillreallygetthemostoutof.

Tosendusgeneralfeedback,simplye-mail<feedback@packtpub.com>,andmentionthebook’stitleinthesubjectofyourmessage.

Ifthereisatopicthatyouhaveexpertiseinandyouareinterestedineitherwritingorcontributingtoabook,seeourauthorguideatwww.packtpub.com/authors.

CustomersupportNowthatyouaretheproudownerofaPacktbook,wehaveanumberofthingstohelpyoutogetthemostfromyourpurchase.

DownloadingtheexamplecodeYoucandownloadtheexamplecodefilesfromyouraccountathttp://www.packtpub.comforallthePacktPublishingbooksyouhavepurchased.Ifyoupurchasedthisbookelsewhere,youcanvisithttp://www.packtpub.com/supportandregistertohavethefilese-maileddirectlytoyou.

DownloadingthecolorimagesofthisbookWealsoprovideyouwithaPDFfilethathascolorimagesofthescreenshots/diagramsusedinthisbook.Thecolorimageswillhelpyoubetterunderstandthechangesintheoutput.Youcandownloadthisfilefrom:https://www.packtpub.com/sites/default/files/downloads/ImageProcessingwithOpenCV_Graphics.pdf

ErrataAlthoughwehavetakeneverycaretoensuretheaccuracyofourcontent,mistakesdohappen.Ifyoufindamistakeinoneofourbooks—maybeamistakeinthetextorthecode—wewouldbegratefulifyoucouldreportthistous.Bydoingso,youcansaveotherreadersfromfrustrationandhelpusimprovesubsequentversionsofthisbook.Ifyoufindanyerrata,pleasereportthembyvisitinghttp://www.packtpub.com/submit-errata,selectingyourbook,clickingontheErrataSubmissionFormlink,andenteringthedetailsofyourerrata.Onceyourerrataareverified,yoursubmissionwillbeacceptedandtheerratawillbeuploadedtoourwebsiteoraddedtoanylistofexistingerrataundertheErratasectionofthattitle.

Toviewthepreviouslysubmittederrata,gotohttps://www.packtpub.com/books/content/supportandenterthenameofthebookinthesearchfield.TherequiredinformationwillappearundertheErratasection.

PiracyPiracyofcopyrightedmaterialontheInternetisanongoingproblemacrossallmedia.AtPackt,wetaketheprotectionofourcopyrightandlicensesveryseriously.IfyoucomeacrossanyillegalcopiesofourworksinanyformontheInternet,pleaseprovideuswiththelocationaddressorwebsitenameimmediatelysothatwecanpursuearemedy.

Pleasecontactusat<copyright@packtpub.com>withalinktothesuspectedpiratedmaterial.

Weappreciateyourhelpinprotectingourauthorsandourabilitytobringyouvaluablecontent.

QuestionsIfyouhaveaproblemwithanyaspectofthisbook,youcancontactusat<questions@packtpub.com>,andwewilldoourbesttoaddresstheproblem.

Chapter1.HandlingImageandVideoFilesThischapterisintendedasafirstcontactwithOpenCV,itsinstallation,andfirstbasicprograms.Wewillcoverthefollowingtopics:

AbriefintroductiontoOpenCVforthenovice,followedbyaneasystep-by-stepguidetotheinstallationofthelibraryAquicktourofOpenCV’sstructureaftertheinstallationintheuser’slocaldiskQuickrecipestocreateprojectsusingthelibrarywithsomecommonprogrammingframeworksHowtousethefunctionstoreadandwriteimagesandvideosFinally,wedescribethelibraryfunctionstoaddrichuserinterfacestothesoftwareprojects,includingmouseinteraction,drawingprimitives,andQtsupport

AnintroductiontoOpenCVInitiallydevelopedbyIntel,OpenCV(OpenSourceComputerVision)isafreecross-platformlibraryforreal-timeimageprocessingthathasbecomeadefactostandardtoolforallthingsrelatedtoComputerVision.Thefirstversionwasreleasedin2000underBSDlicenseandsincethen,itsfunctionalityhasbeenverymuchenrichedbythescientificcommunity.In2012,thenonprofitfoundationOpenCV.orgtookonthetaskofmaintainingasupportsitefordevelopersandusers.

NoteAtthetimeofwritingthisbook,anewmajorversionofOpenCV(Version3.0)isavailable,stillonbetastatus.Throughoutthebook,wewillpresentthemostrelevantchangesbroughtwiththisnewversion.

OpenCVisavailableforthemostpopularoperatingsystems,suchasGNU/Linux,OSX,Windows,Android,iOS,andsomemore.ThefirstimplementationwasintheCprogramminglanguage;however,itspopularitygrewwithitsC++implementationasofVersion2.0.NewfunctionsareprogrammedwithC++.However,nowadays,thelibraryhasafullinterfaceforotherprogramminglanguages,suchasJava,Python,andMATLAB/Octave.Also,wrappersforotherlanguages(suchasC#,Ruby,andPerl)havebeendevelopedtoencourageadoptionbyprogrammers.

Inanattempttomaximizetheperformanceofcomputingintensivevisiontasks,OpenCVincludessupportforthefollowing:

MultithreadingonmulticorecomputersusingThreadingBuildingBlocks(TBB)—atemplatelibrarydevelopedbyIntel.AsubsetofIntegratedPerformancePrimitives(IPP)onIntelprocessorstoboostperformance.ThankstoIntel,theseprimitivesarefreelyavailableasofVersion3.0beta.InterfacesforprocessingonGraphicProcessingUnit(GPU)usingComputeUnifiedDeviceArchitecture(CUDA)andOpenComputingLanguage(OpenCL).

TheapplicationsforOpenCVcoverareassuchassegmentationandrecognition,2Dand3Dfeaturetoolkits,objectidentification,facialrecognition,motiontracking,gesturerecognition,imagestitching,highdynamicrange(HDR)imaging,augmentedreality,andsoon.Moreover,tosupportsomeofthepreviousapplicationareas,amodulewithstatisticalmachinelearningfunctionsisincluded.

DownloadingandinstallingOpenCVOpenCVisfreelyavailablefordownloadathttp://opencv.org.Thissiteprovidesthelastversionfordistribution(currently,3.0beta)andolderversions.

NoteSpecialcareshouldbetakenwithpossibleerrorswhenthedownloadedversionisanonstablerelease,forexample,thecurrent3.0betaversion.

Onhttp://opencv.org/downloads.html,suitableversionsofOpenCVforeachplatformcanbefound.Thecodeandinformationofthelibrarycanbeobtainedfromdifferentrepositoriesdependingonthefinalpurpose:

Themainrepository(athttp://sourceforge.net/projects/opencvlibrary),devotedtofinalusers.Itcontainsbinaryversionsofthelibraryandready-to‑compilesourcesforthetargetplatform.Thetestdatarepository(athttps://github.com/itseez/opencv_extra)withsetsofdatatotestpurposesofsomelibrarymodules.Thecontributionsrepository(athttp://github.com/itseez/opencv_contrib)withthesourcecodecorrespondingtoextraandcutting-edgefeaturessuppliedbycontributors.Thiscodeismoreerror-proneandlesstestedthanthemaintrunk.

TipWiththelastversion,OpenCV3.0beta,theextracontributedmodulesarenotincludedinthemainpackage.Theyshouldbedownloadedseparatelyandexplicitlyincludedinthecompilationprocessthroughtheproperoptions.Becautiousifyouincludesomeofthosecontributedmodules,becausesomeofthemhavedependenciesonthird‑partysoftwarenotincludedwithOpenCV.Thedocumentationsite(athttp://docs.opencv.org/master/)foreachofthemodules,includingthecontributedones.Thedevelopmentrepository(athttps://github.com/Itseez/opencv)withthecurrentdevelopmentversionofthelibrary.Itisintendedfordevelopersofthemainfeaturesofthelibraryandthe“impatient”userwhowishestousethelastupdateevenbeforeitisreleased.

RatherthanGNU/LinuxandOSX,whereOpenCVisdistributedassourcecodeonly,intheWindowsdistribution,onecanfindprecompiled(withMicrosoftVisualC++v10,v11,andv12)versionsofthelibrary.EachprecompiledversionisreadytobeusedwithMicrosoftcompilers.However,iftheprimaryintentionistodevelopprojectswithadifferentcompilerframework,weneedtocompilethelibraryforthatspecificcompiler(forexample,GNUGCC).

TipThefastestroutetoworkingwithOpenCVistouseoneoftheprecompiledversionsincludedwiththedistribution.Then,abetterchoiceistobuildafine-tunedversionofthe

librarywiththebestsettingsforthelocalplatformusedforsoftwaredevelopment.ThischapterprovidestheinformationtobuildandinstallOpenCVonWindows.FurtherinformationtosetthelibraryonLinuxcanbefoundathttp://docs.opencv.org/doc/tutorials/introduction/linux_installandhttps://help.ubuntu.com/community/OpenCV.

GettingacompilerandsettingCMakeAgoodchoiceforcross‑platformdevelopmentwithOpenCVistousetheGNUtoolkit(includinggmake,g++,andgdb).TheGNUtoolkitcanbeeasilyobtainedforthemostpopularoperatingsystems.OurpreferredchoiceforadevelopmentenvironmentconsistsoftheGNUtoolkitandthecross‑platformQtframework,whichincludestheQtlibraryandtheQtCreatorIntegratedDevelopmentEnvironment(IDE).TheQtframeworkisfreelyavailableathttp://qt-project.org/.

NoteAfterinstallingthecompileronWindows,remembertoproperlysetthePathenvironmentvariable,addingthepathforthecompiler’sexecutable,forexample,C:\Qt\Qt5.2.1\5.2.1\mingw48_32\binfortheGNU/compilersincludedwiththeQtframework.OnWindows,thefreeRapidEnvironmentEditortool(availableathttp://www.rapidee.com)providesaconvenientwaytochangePathandotherenvironmentvariables.

TomanagethebuildprocessfortheOpenCVlibraryinacompiler-independentway,CMakeistherecommendedtool.CMakeisafreeandopensourcecross‑platformtoolavailableathttp://www.cmake.org/.

ConfiguringOpenCVwithCMakeOncethesourcesofthelibraryhavebeendownloadedintothelocaldisk,itisrequiredthatyouconfigurethemakefilesforthecompilationprocessofthelibrary.CMakeisthekeytoolforaneasyconfigurationofOpenCV’sinstallationprocess.Itcanbeusedfromthecommandlineorinamoreuser‑friendlywaywithitsGraphicalUserInterface(GUI)version.

ThestepstoconfigureOpenCVwithCMakecanbesummarizedasfollows:

1. Choosethesource(let’scallitOPENCV_SRCinwhatfollows)andtarget(OPENCV_BUILD)directories.Thetargetdirectoryiswherethecompiledbinarieswillbelocated.

2. MarktheGroupedandAdvancedcheckboxesandclickontheConfigurebutton.3. Choosethedesiredcompiler(forexample,GNUdefaultcompilers,MSVC,andso

on).4. Setthepreferredoptionsandunsetthosenotdesired.5. ClickontheConfigurebuttonandrepeatsteps4and5untilnoerrorsareobtained.6. ClickontheGeneratebuttonandcloseCMake.

ThefollowingscreenshotshowsyouthemainwindowofCMakewiththesourceandtargetdirectoriesandthecheckboxestogroupalltheavailableoptions:

ThemainwindowofCMakeafterthepreconfigurationstep

NoteForbrevity,weuseOPENCV_BUILDandOPENCV_SRCinthistexttodenotethetargetandsourcedirectoriesoftheOpenCVlocalsetup,respectively.Keepinmindthatalldirectoriesshouldmatchyourcurrentlocalconfiguration.

Duringthepreconfigurationprocess,CMakedetectsthecompilerspresentandmanyotherlocalpropertiestosetthebuildprocessofOpenCV.ThepreviousscreenshotdisplaysthemainCMakewindowafterthepreconfigurationprocess,showingthegroupedoptionsinred.

Itispossibletoleavethedefaultoptionsunchangedandcontinuetheconfigurationprocess.However,someconvenientoptionscanbeset:

BUILD_EXAMPLES:ThisissettobuildsomeexamplesusingOpenCV.BUILD_opencv_<module_name>:Thisissettoincludethemodule(module_name)in

thebuildprocess.OPENCV_EXTRA_MODULES_PATH:Thisisusedwhenyouneedsomeextracontributedmodule;setthepathforthesourcecodeoftheextramoduleshere(forexample,C:/opencv_contrib-master/modules).WITH_QT:ThisisturnedontoincludetheQtfunctionalityintothelibrary.WITH_IPP:Thisoptionisturnedonbydefault.ThecurrentOpenCV3.0versionincludesasubsetoftheIntelIntegratedPerformancePrimitives(IPP)thatspeeduptheexecutiontimeofthelibrary.

TipIfyoucompilethenewOpenCV3.0(beta),becautiousbecausesomeunexpectederrorshavebeenreportedrelatedtotheIPPinclusion(thatis,withthedefaultvalueofthisoption).WerecommendthatyouunsettheWITH_IPPoption.

IftheconfigurationstepswithCMake(loopthroughsteps4and5)don’tproduceanyfurthererrors,itispossibletogeneratethefinalmakefilesforthebuildprocess.ThefollowingscreenshotshowsyouthemainwindowofCMakeafteragenerationstepwithouterrors:

CompilingandinstallingthelibraryThenextstepafterthegenerationprocessofmakefileswithCMakeisthecompilationwiththepropermaketool.Thistoolisusuallyexecutedonthecommandline(theconsole)fromthetargetdirectory(theonesetattheCMakeconfigurationstep).Forexample,inWindows,thecompilationshouldbelaunchedfromthecommandlineasfollows:

OPENCV_BUILD>mingw32-make

ThiscommandlaunchesabuildprocessusingthemakefilesgeneratedbyCMake.Thewholecompilationtypicallytakesseveralminutes.Ifthecompilationendswithouterrors,theinstallationcontinueswiththeexecutionofthefollowingcommand:

OPENCV_BUILD>mingw32-makeinstall

ThiscommandcopiestheOpenCVbinariestotheOPENCV_BUILD\installdirectory.

Ifsomethingwentwrongduringthecompilation,weshouldrunCMakeagaintochangetheoptionsselectedduringtheconfiguration.Then,weshouldregeneratethemakefiles.

Theinstallationendsbyaddingthelocationofthelibrarybinaries(forexample,inWindows,theresultingDLLfilesarelocatedatOPENCV_BUILD\install\x64\mingw\bin)tothePathenvironmentvariable.WithoutthisdirectoryinthePathfield,theexecutionofeveryOpenCVexecutablewillgiveanerrorasthelibrarybinarieswon’tbefound.

Tocheckthesuccessoftheinstallationprocess,itispossibletorunsomeoftheexamplescompiledalongwiththelibrary(iftheBUILD_EXAMPLESoptionwassetusingCMake).Thecodesamples(writteninC++)canbefoundatOPENCV_BUILD\install\x64\mingw\samples\cpp.

NoteTheshortinstructionsgiventoinstallOpenCVapplytoWindows.AdetaileddescriptionwiththeprerequisitesforLinuxcanbereadathttp://docs.opencv.org/doc/tutorials/introduction/linux_install/linux_install.html.AlthoughthetutorialappliestoOpenCV2.0,almostalltheinformationisstillvalidforVersion3.0.

ThestructureofOpenCVOnceOpenCVisinstalled,theOPENCV_BUILD\installdirectorywillbepopulatedwiththreetypesoffiles:

Headerfiles:ThesearelocatedintheOPENCV_BUILD\install\includesubdirectoryandareusedtodevelopnewprojectswithOpenCV.Librarybinaries:Thesearestaticordynamiclibraries(dependingontheoptionselectedwithCMake)withthefunctionalityofeachoftheOpenCVmodules.Theyarelocatedinthebinsubdirectory(forexample,x64\mingw\binwhentheGNUcompilerisused).Samplebinaries:Theseareexecutableswithexamplesthatusethelibraries.Thesourcesforthesesamplescanbefoundinthesourcepackage(forexample,OPENCV_SRC\sources\samples).

OpenCVhasamodularstructure,whichmeansthatthepackageincludesastaticordynamic(DLL)libraryforeachmodule.Theofficialdocumentationforeachmodulecanbefoundathttp://docs.opencv.org/master/.Themainmodulesincludedinthepackageare:

core:ThisdefinesthebasicfunctionsusedbyalltheothermodulesandthefundamentaldatastructuresincludingtheimportantmultidimensionalarrayMat.highgui:Thisprovidessimpleuserinterface(UI)capabilities.BuildingthelibrarywithQtsupport(theWITH_QTCMakeoption)allowsUIcompatibilitywithsuchaframework.imgproc:Theseareimageprocessingfunctionsthatincludefiltering(linearandnonlinear),geometrictransformations,colorspaceconversion,histograms,andsoon.imgcodecs:Thisisaneasy-to-useinterfacetoreadandwriteimages.

NotePayattentiontothechangesinmodulessinceOpenCV3.0assomefunctionalityhasbeenmovedtoanewmodule(forexample,readingandwritingimagesfunctionsweremovedfromhighguitoimgcodecs).

photo:ThisincludesComputationalPhotographyincludinginpainting,denoising,HighDynamicRange(HDR)imaging,andsomeothers.stitching:Thisisusedforimagestitching.videoio:Thisisaneasy-to-useinterfaceforvideocaptureandvideocodecs.video:Thissuppliesthefunctionalityofvideoanalysis(motionestimation,backgroundextraction,andobjecttracking).features2d:Thesearefunctionsforfeaturedetection(cornersandplanarobjects),featuredescription,featurematching,andsoon.objdetect:Thesearefunctionsforobjectdetectionandinstancesofpredefineddetectors(suchasfaces,eyes,smile,people,cars,andsoon).

Someothermodulesarecalib3d(cameracalibration),flann(clusteringandsearch),ml(machinelearning),shape(shapedistanceandmatching),superres(superresolution),video(videoanalysis),andvideostab(videostabilization).

NoteAsofVersion3.0beta,thenewcontributedmodulesaredistributedinaseparatepackage(opencv_contrib-master.zip)thatcanbedownloadedfromhttps://github.com/itseez/opencv_contrib.Thesemodulesprovideextrafeaturesthatshouldbefullyunderstoodbeforeusingthem.ForaquickoverviewofthenewfunctionalityinthenewreleaseofOpenCV(Version3.0),refertothedocumentathttp://opencv.org/opencv-3-0-beta.html.

CreatinguserprojectswithOpenCVInthisbook,weassumethatC++isthemainlanguageforprogrammingimageprocessingapplications,althoughinterfacesandwrappersforotherprogramminglanguagesareactuallyprovided(forinstance,Python,Java,MATLAB/Octave,andsomemore).

Inthissection,weexplainhowtodevelopapplicationswithOpenCV’sC++APIusinganeasy-to-usecross-platformframework.

GeneralusageofthelibraryTodevelopanOpenCVapplicationwithC++,werequireourcodeto:

IncludetheOpenCVheaderfileswithdefinitionsLinktheOpenCVlibraries(binaries)togetthefinalexecutable

TheOpenCVheaderfilesarelocatedintheOPENCV_BUILD\install\include\opencv2directorywherethereisafile(*.hpp)foreachofthemodules.Theinclusionoftheheaderfileisdonewiththe#includedirective,asshownhere:

#include<opencv2/<module_name>/<module_name>.hpp>

//Includingtheheaderfileforeachmoduleusedinthecode

Withthisdirective,itispossibletoincludeeveryheaderfileneededbytheuserprogram.Ontheotherhand,iftheopencv.hppheaderfileisincluded,alltheheaderfileswillbeautomaticallyincludedasfollows:

#include<opencv2/opencv.hpp>

//IncludingalltheOpenCV'sheaderfilesinthecode

NoteRememberthatallthemodulesinstalledlocallyaredefinedintheOPENCV_BUILD\install\include\opencv2\opencv_modules.hppheaderfile,whichisgeneratedautomaticallyduringthebuildingprocessofOpenCV.

Theuseofthe#includedirectiveisnotalwaysaguaranteeforthecorrectinclusionoftheheaderfiles,becauseitisnecessarytotellthecompilerwheretofindtheincludefiles.Thisisachievedbypassingaspecialargumentwiththelocationofthefiles(suchasI\<location>forGNUcompilers).

Thelinkingprocessrequiresyoutoprovidethelinkerwiththelibraries(dynamicorstatic)wheretherequiredOpenCVfunctionalitycanbefound.Thisisusuallydonewithtwotypesofargumentsforthelinker:thelocationofthelibrary(suchas‑L<location>forGNUcompilers)andthenameofthelibrary(suchas-l<module_name>).

NoteYoucanfindacompletelistofavailableonlinedocumentationforGNUGCCandMakeathttps://gcc.gnu.org/onlinedocs/andhttps://www.gnu.org/software/make/manual/.

ToolstodevelopnewprojectsThemainprerequisitestodevelopourownOpenCVC++applicationsare:

OpenCVheaderfilesandlibrarybinaries:OfcourseweneedtocompileOpenCV,andtheauxiliarylibrariesareprerequisitesforsuchacompilation.Thepackageshouldbecompiledwiththesamecompilerusedtogeneratetheuserapplication.AC++compiler:Someassociatetoolsareconvenientasthecodeeditor,debugger,projectmanager,buildprocessmanager(forinstanceCMake),revisioncontrolsystem(suchasGit,Mercurial,SVN,andsoon),andclassinspector,amongothers.Usually,thesetoolsaredeployedtogetherinaso-calledIntegratedDevelopmentEnvironment(IDE).Anyotherauxiliarylibraries:Optionally,anyotherauxiliarylibrariesneededtoprogramthefinalapplication,suchasgraphical,statistical,andsoonwillberequired.

ThemostpopularavailablecompilerkitstoprogramOpenCVC++applicationsare:

MicrosoftVisualC(MSVC):ThisisonlysupportedonWindowsanditisverywellintegratedwiththeIDEVisualStudio,althoughitcanbealsointegratedwithothercross-platformIDEs,suchasQtCreatororEclipse.VersionsofMSVCthatcurrentlycompatiblewiththelatestOpenCVreleaseareVC10,VC11,andVC12(VisualStudio2010,2012,and2013).GNUCompilerCollectionGNUGCC:Thisisacross‑platformcompilersystemdevelopedbytheGNUproject.ForWindows,thiskitisknownasMinGW(MinimalGNUGCC).TheversioncompatiblewiththecurrentOpenCVreleaseisGNUGCC4.8.ThiskitmaybeusedwithseveralIDEs,suchasQtCreator,Code::Blocks,Eclipse,amongothers.

Fortheexamplespresentedinthisbook,weusedtheMinGW4.8compilerkitforWindowsplustheQt5.2.1libraryandtheQtCreatorIDE(3.0.1).Thecross-platformQtlibraryisrequiredtocompileOpenCVwiththenewUIcapabilitiesprovidedbysuchalibrary.

NoteForWindows,itispossibletodownloadaQtbundle(includingQtlibrary,QtCreator,andtheMinGWkit)fromhttp://qt-project.org/.Thebundleisapproximately700MB.

QtCreatorisacross-platformIDEforC++thatintegratesthetoolsweneedtocodeapplications.InWindows,itmaybeusedwithMinGWorMSVC.ThefollowingscreenshotshowsyoutheQtCreatormainwindowwiththedifferentpanelsandviewsforanOpenCVC++project:

ThemainwindowofQtCreatorwithsomeviewsfromanOpenCVC++project

CreatinganOpenCVC++programwithQtCreatorNext,weexplainhowtocreateacodeprojectwiththeQtCreatorIDE.Inparticular,weapplythisdescriptiontoanOpenCVexample.

WecancreateaprojectforanyOpenCVapplicationusingQtCreatorbynavigatingtoFile|NewFileorFile|Project…andthennavigatingtoNon-QtProject|PlainC++Project.Then,wehavetochooseaprojectnameandthelocationatwhichitwillbestored.Thenextstepistopickakit(thatis,thecompiler)fortheproject(inourcase,DesktopQt5.2.1MinGW32bit)andthelocationforthebinariesgenerated.Usually,twopossiblebuildconfigurations(profiles)areused:debugandrelease.Theseprofilessettheappropriateflagstobuildandrunthebinaries.

WhenaprojectiscreatedusingQtCreator,twospecialfiles(with.proand.pro.userextensions)aregeneratedtoconfigurethebuildandrunprocesses.Thebuildprocessisdeterminedbythekitchosenduringthecreationoftheproject.WiththeDesktopQt5.2.1MinGW32bitkit,thisprocessreliesontheqmakeandmingw32‑maketools.Usingthe*.profileastheinput,qmakegeneratesthemakefilethatdrivesthebuildprocessforeachprofile(thatis,releaseanddebug).TheqmaketoolisusedfromtheQtCreatorIDEasanalternativetoCMaketosimplifythebuildprocessofsoftwareprojects.Itautomatesthegenerationofmakefilesfromafewlinesofinformation.

Thefollowinglinesrepresentanexampleofa*.profile(forexample,showImage.pro):

TARGET:showImage

TEMPLATE=app

CONFIG+=console

CONFIG-=app_bundle

CONFIG-=qt

SOURCES+=\

showImage.cpp

INCLUDEPATH+=C:/opencv300-buildQt/install/include

LIBS+=-LC:/opencv300-buildQt/install/x64/mingw/lib\

-lopencv_core300.dll\

-lopencv_imgcodecs300.dll\

-lopencv_highgui300.dll\

-lopencv_imgproc300.dll

Theprecedingfileillustratestheoptionsthatqmakeneedstogeneratetheappropriatemakefilestobuildthebinariesforourproject.Eachlinestartswithatagindicatinganoption(TARGET,CONFIG,SOURCES,INCLUDEPATH,andLIBS)followedwithamarktoadd(+=)orremove(-=)thevalueoftheoption.Inthissampleproject,weusethenon-Qtconsoleapplication.TheexecutablefileisshowImage.exe(TARGET)andthesourcefileisshowImage.cpp(SOURCES).AsthisprojectisanOpenCV-basedapplication,thetwolasttagsindicatethelocationoftheheaderfiles(INCLUDEPATH)andtheOpenCVlibraries(LIBS)usedbythisparticularproject(core,imgcodecs,highgui,andimgproc).Notethatabackslashattheendofthelinedenotescontinuationinthenextline.

NoteForadetaileddescriptionofthetools(includingQtCreatorandqmake)developedwithintheQtproject,visithttp://doc.qt.io/.

ReadingandwritingimagefilesImageprocessingreliesongettinganimage(forinstance,aphotographoravideofame)and“playing”withitbyapplyingsignalprocessingtechniquesonittogetthedesiredresults.Inthissection,weshowyouhowtoreadimagesfromfilesusingthefunctionssuppliedbyOpenCV.

ThebasicAPIconceptsTheMatclassisthemaindatastructurethatstoresandmanipulatesimagesinOpenCV.Thisclassisdefinedinthecoremodule.OpenCVhasimplementedmechanismstoallocateandreleasememoryautomaticallyforthesedatastructures.However,theprogrammershouldstilltakespecialcarewhendatastructuressharethesamebuffermemory.Forinstance,theassignmentoperatordoesnotcopythememorycontentfromanobject(MatA)toanother(MatB);itonlycopiesthereference(thememoryaddressofthecontent).Then,achangeinoneobject(AorB)affectsbothobjects.ToduplicatethememorycontentofaMatobject,theMat::clone()memberfunctionshouldbeused.

NoteManyfunctionsinOpenCVprocessdensesingleormultichannelarrays,usuallyusingtheMatclass.However,insomecases,adifferentdatatypemaybeconvenient,suchasstd::vector<>,Matx<>,Vec<>,orScalar.Forthispurpose,OpenCVprovidestheproxyclassesInputArrayandOutputArray,whichallowanyoftheprevioustypestobeusedasparametersforfunctions.

TheMatclassisusedfordensen-dimensionalsingleormultichannelarrays.Itcanactuallystorerealorcomplex-valuedvectorsandmatrices,coloredorgrayscaleimages,histograms,pointclouds,andsoon.

TherearemanydifferentwaystocreateaMatobject,themostpopularbeingtheconstructorwherethesizeandtypeofthearrayarespecifiedasfollows:

Mat(nrows,ncols,type,fillValue)

TheinitialvalueforthearrayelementsmightbesetbytheScalarclassasatypicalfour-elementvector(foreachRGBandtransparencycomponentoftheimagestoredinthearray).Next,weshowyouausageexampleofMatasfollows:

Matimg_A(4,4,CV_8U,Scalar(255));

//Whiteimage:

//4x4single-channelarraywith8bitsofunsignedintegers

//(upto255values,validforagrayscaleimage,forexample,

//255=white)

TheDataTypeclassdefinestheprimitivedatatypesforOpenCV.Theprimitivedatatypescanbebool,unsignedchar,signedchar,unsignedshort,signedshort,int,float,double,oratupleofvaluesofoneoftheseprimitivetypes.Anyprimitivetypecanbedefinedbyanidentifierinthefollowingform:

CV_<bitdepth>{U|S|F}C(<numberofchannels>)

IntheprecedingcodeU,S,andFstandforunsigned,signed,andfloat,respectively.Forthesinglechannelarrays,thefollowingenumerationisapplied,describingthedatatypes:

enum{CV_8U=0,CV_8S=1,CV_16U=2,CV_16S=3,CV_32S=4,CV_32F=5,CV_64F=6};

Here,itshouldbenotedthatthesethreedeclarationsareequivalent:CV_8U,CV_8UC1,andCV_8UC(1).Thesingle-channeldeclarationfitswellforintegerarraysdevotedtograyscaleimages,whereasthethreechanneldeclarationofanarrayismoreappropriateforimageswiththreecomponents(forexample,RGB,BRG,HSV,andsoon).Forlinearalgebraoperations,thearraysoftypefloat(F)mightbeused.

Wecandefinealloftheprecedingdatatypesformultichannelarrays(upto512channels).Thefollowingscreenshotsillustrateanimage’sinternalrepresentationwithonesinglechannel(CV_8U,grayscale)andthesameimagerepresentedwiththreechannels(CV_8UC3,RGB).ThesescreenshotsaretakenbyzoominginonanimagedisplayedinthewindowofanOpenCVexecutable(theshowImageexample):

An8-bitrepresentationofanimageinRGBcolorandgrayscale

NoteItisimportanttonoticethattoproperlysaveaRGBimagewithOpenCVfunctions,theimagemustbestoredinmemorywithitschannelsorderedasBGR.Inthesameway,whenanRGBimageisreadfromafile,itisstoredinmemorywithitschannelsinaBGRorder.Moreover,itneedsasupplementaryfourthchannel(alpha)tomanipulateimageswiththreechannels,RGB,plusatransparency.ForRGBimages,alargerintegervaluemeansabrighterpixelormoretransparencyforthealphachannel.

AllOpenCVclassesandfunctionsareinthecvnamespace,andconsequently,wewillhavethefollowingtwooptionsinoursourcecode:

Addtheusingnamespacecvdeclarationafterincludingtheheaderfiles(thisistheoptionusedinallthecodeexamplesinthisbook).Appendthecv::prefixtoalltheOpenCVclasses,functions,anddatastructuresthat

weuse.ThisoptionisrecommendediftheexternalnamesprovidedbyOpenCVconflictwiththeoften-usedstandardtemplatelibrary(STL)orotherlibraries.

Imagefile-supportedformatsOpenCVsupportsthemostcommonimageformats.However,someofthemneed(freelyavailable)third-partylibraries.ThemainformatssupportedbyOpenCVare:

Windowsbitmaps(*.bmp,*dib)Portableimageformats(*.pbm,*.pgm,*.ppm)Sunrasters(*.sr,*.ras)

Theformatsthatneedauxiliarylibrariesare:

JPEG(*.jpeg,*.jpg,*.jpe)JPEG2000(*.jp2)PortableNetworkGraphics(*.png)TIFF(*.tiff,*.tif)WebP(*.webp).

Inadditiontotheprecedinglistedformats,withtheOpenCV3.0version,itincludesadriverfortheformats(NITF,DTED,SRTM,andothers)supportedbytheGeographicDataAbstractionLibrary(GDAL)setwiththeCMakeoption,WITH_GDAL.NoticethattheGDALsupporthasnotbeenextensivelytestedonWindowsOSesyet.InWindowsandOSX,codecsshippedwithOpenCVareusedbydefault(libjpeg,libjasper,libpng,andlibtiff).Then,intheseOSes,itispossibletoreadtheJPEG,PNG,andTIFFformats.Linux(andotherUnix-likeopensourceOSes)looksforcodecsinstalledinthesystem.ThecodecscanbeinstalledbeforeOpenCVorelsethelibrariescanbebuiltfromtheOpenCVpackagebysettingtheproperoptionsinCMake(forexample,BUILD_JASPER,BUILD_JPEG,BUILD_PNG,andBUILD_TIFF).

TheexamplecodeToillustratehowtoreadandwriteimagefileswithOpenCV,wewillnowdescribetheshowImageexample.Theexampleisexecutedfromthecommandlinewiththecorrespondingoutputwindowsasfollows:

<bin_dir>\showImage.exefruits.jpgfruits_bw.jpg

TheoutputwindowfortheshowImageexample

Inthisexample,twofilenamesaregivenasarguments.Thefirstoneistheinputimagefiletoberead.Thesecondoneistheimagefiletobewrittenwithagrayscalecopyoftheinputimage.Next,weshowyouthesourcecodeanditsexplanation:

#include<iostream>

usingnamespacestd;

usingnamespacecv;

intmain(int,char*argv[])

Matin_image,out_image;

//Usage:<cmd><file_in><file_out>

//Readoriginalimage

in_image=imread(argv[1],IMREAD_UNCHANGED);

if(in_image.empty()){

//Checkwhethertheimageisreadornot

cout<<"Error!Inputimagecannotberead…\n";

return-1;

//Createstwowindowswiththenamesoftheimages

namedWindow(argv[1],WINDOW_AUTOSIZE);

namedWindow(argv[2],WINDOW_AUTOSIZE);

//Showstheimageintothepreviouslycreatedwindow

imshow(argv[1],in_image);

cvtColor(in_image,out_image,COLOR_BGR2GRAY);

imshow(argv[2],in_image);

cout<<"Pressanykeytoexit…\n";

waitKey();//Waitforkeypress

//Writingimage

imwrite(argv[2],in_image);

return0;

Here,weusethe#includedirectivewiththeopencv.hppheaderfilethat,infact,includesalltheOpenCVheaderfiles.Byincludingthissinglefile,nomorefilesneedtobeincluded.Afterdeclaringtheuseofcvnamespace,allthevariablesandfunctionsinsidethisnamespacedon’tneedthecv::prefix.Thefirstthingtodointhemainfunctionistocheckthenumberofargumentspassedinthecommandline.Then,ahelpmessageisdisplayedifanerroroccurs.

ReadingimagefilesIfthenumberofargumentsiscorrect,theimagefileisreadintotheMatin_imageobjectwiththeimread(argv[1],IMREAD_UNCHANGED)function,wherethefirstparameteristhefirstargument(argv[1])passedinthecommandlineandthesecondparameterisaflag(IMREAD_UNCHANGED),whichmeansthattheimagestoredintothememoryobjectshouldbeunchanged.Theimreadfunctiondeterminesthetypeofimage(codec)fromthefilecontentratherthanfromthefileextension.

Theprototypefortheimreadfunctionisasfollows:

Matimread(constString&filename,

intflags=IMREAD_COLOR)

Theflagspecifiesthecoloroftheimagereadandtheyaredefinedandexplainedbythefollowingenumerationintheimgcodecs.hppheaderfile:

enum{IMREAD_UNCHANGED=-1,//8bit,colorornot

IMREAD_GRAYSCALE=0,//8bit,gray

IMREAD_COLOR=1,//unchangeddepth,color

IMREAD_ANYDEPTH=2,//anydepth,unchangedcolor

IMREAD_ANYCOLOR=4,//unchangeddepth,anycolor

IMREAD_LOAD_GDAL=8//Usegdaldriver

NoteAsofVersion3.0ofOpenCV,theimreadfunctionisintheimgcodecsmoduleandnotinhighguilikeinOpenCV2.x.

AsseveralfunctionsanddeclarationsaremovedintoOpenCV3.0,itispossibletogetsomecompilationerrorsasoneormoredeclarations(symbolsand/orfunctions)arenotfoundbythelinker.Tofigureoutwhere(*.hpp)asymbolisdefinedandwhichlibrarytolink,werecommendthefollowingtrickusingtheQtCreatorIDE:

Addthe#include<opencv2/opencv.hpp>declarationtothecode.PresstheF2functionkeywiththemousecursoroverthesymbolorfunction;thisopensthe*.hppfilewherethesymbolorfunctionisdeclared.

Aftertheinputimagefileisread,checktoseewhethertheoperationsucceeded.Thischeckisachievedwiththein_image.empty()memberfunction.Iftheimagefileisreadwithouterrors,twowindowsarecreatedtodisplaytheinputandoutputimages,respectively.Thecreationofwindowsiscarriedoutwiththefollowingfunction:

voidnamedWindow(constString&winname,intflags=WINDOW_AUTOSIZE)

OpenCVwindowsareidentifiedbyaunivocalnameintheprogram.Theflags’definitionandtheirexplanationaregivenbythefollowingenumerationinthehighgui.hppheaderfile:

enum{WINDOW_NORMAL=0x00000000,

//theusercanresizethewindow(noconstraint)

//alsousetoswitchafullscreenwindowtoanormalsize

WINDOW_AUTOSIZE=0x00000001,

//theusercannotresizethewindow,

//thesizeisconstrainedbytheimagedisplayed

WINDOW_OPENGL=0x00001000,//windowwithopenglsupport

WINDOW_FULLSCREEN=1,

WINDOW_FREERATIO=0x00000100,

//theimageexpendsasmuchasitcan(noratioconstraint)

WINDOW_KEEPRATIO=0x00000000

//theratiooftheimageisrespected

Thecreationofawindowdoesnotshowanythingonscreen.Thefunction(belongingtothehighguimodule)todisplayanimageinawindowis:

voidimshow(constString&winname,InputArraymat)

Theimage(mat)isshownwithitsoriginalsizeifthewindow(winname)wascreatedwiththeWINDOW_AUTOSIZEflag.

IntheshowImageexample,thesecondwindowshowsagrayscalecopyoftheinputimage.Toconvertacolorimagetograyscale,thecvtColorfunctionfromtheimgprocmoduleisused.Thisfunctioncanactuallybeusedtochangetheimagecolorspace.

Anywindowcreatedinaprogramcanberesizedandmovedfromitsdefaultsettings.Whenanywindowisnolongerrequired,itshouldbedestroyedinordertoreleaseitsresources.Thisresourceliberationisdoneimplicitlyattheendofaprogram,likeintheexample.

Eventhandlingintotheintrinsicloop

Ifwedonothingmoreaftershowinganimageonawindow,surprisingly,theimagewillnotbeshownatall.Aftershowinganimageonawindow,weshouldstartalooptofetchandhandleeventsrelatedtouserinteractionwiththewindow.Suchataskiscarriedoutbythefollowingfunction(fromthehighguimodule):

intwaitKey(intdelay=0)

Thisfunctionwaitsforakeypressedduringanumberofmilliseconds(delay>0)returningthecodeofthekeyor-1ifthedelayendswithoutakeypressed.Ifdelayis0ornegative,thefunctionwaitsforeveruntilakeyispressed.

NoteRememberthatthewaitKeyfunctiononlyworksifthereisacreatedandactivewindowatleast.

WritingimagefilesAnotherimportantfunctionintheimgcodecsmoduleis:

boolimwrite(constString&filename,InputArrayimg,constvector<int>&

params=vector<int>())

Thisfunctionsavestheimage(img)intoafile(filename),beingthethirdoptionalargumentavectorofproperty-valuepairsspecifyingtheparametersofthecodec(leaveitemptytousethedefaultvalues).Thecodecisdeterminedbytheextensionofthefile.

NoteForadetailedlistofcodecproperties,takealookattheimgcodecs.hppheaderfileandtheOpenCVAPIreferenceathttp://docs.opencv.org/master/modules/refman.html.

ReadingandwritingvideofilesRatherthanstillimages,avideodealswithmovingimages.Thesourcesofvideocanbeadedicatedcamera,awebcam,avideofile,orasequenceofimagefiles.InOpenCV,theVideoCaptureandVideoWriterclassesprovideaneasy-to-useC++APIforthetaskofcapturingandrecordinginvolvedinvideoprocessing.

TheexamplecodeTherecVideoexampleisashortsnippetofcodewhereyoucanseehowtouseadefaultcameraasacapturedevicetograbframes,processthemforedgedetection,andsavethisnewconvertedframetoafile.Also,twowindowsarecreatedtosimultaneouslyshowyoutheoriginalframeandtheprocessedone.Theexamplecodeis:

#include<iostream>

usingnamespacestd;

usingnamespacecv;

intmain(int,char**)

Matin_frame,out_frame;

constcharwin1[]="Grabbing…",win2[]="Recording…";

doublefps=30;//Framespersecond

charfile_out[]="recorded.avi";

VideoCaptureinVid(0);//Opendefaultcamera

if(!inVid.isOpened()){//Checkerror

cout<<"Error!Cameranotready…\n";

return-1;

//Getsthewidthandheightoftheinputvideo

intwidth=(int)inVid.get(CAP_PROP_FRAME_WIDTH);

intheight=(int)inVid.get(CAP_PROP_FRAME_HEIGHT);

VideoWriterrecVid(file_out,

VideoWriter::fourcc('M','S','V','C'),

fps,Size(width,height));

if(!recVid.isOpened()){

cout<<"Error!Videofilenotopened…\n";

return-1;

//Createtwowindowsfororig.andfinalvideo

namedWindow(win1);

namedWindow(win2);

while(true){

//Readframefromcamera(grabbinganddecoding)

inVid>>in_frame;

//Converttheframetograyscale

cvtColor(in_frame,out_frame,COLOR_BGR2GRAY);

//Writeframetovideofile(encodingandsaving)

recVid<<out_frame;

imshow(win1,in_frame);//Showframeinwindow

imshow(win2,out_frame);//Showframeinwindow

if(waitKey(1000/fps)>=0)

break;

inVid.release();//Closecamera

return0;

Inthisexample,thefollowingfunctionsdeserveaquickreview:

doubleVideoCapture::get(intpropId):ThisreturnsthevalueofthespecifiedpropertyforaVideoCaptureobject.AcompletelistofpropertiesbasedonDC1394(IEEE1394DigitalCameraSpecifications)isincludedwiththevideoio.hppheaderfile.staticintVideoWriter::fourcc(charc1,charc2,charc3,charc4):Thisconcatenatesfourcharacterstoafourcccode.Intheexample,MSVCstandsforMicrosoftVideo(onlyavailableforWindows).Thelistofvalidfourcccodesispublishedathttp://www.fourcc.org/codecs.php.boolVideoWriter::isOpened():Thisreturnstrueiftheobjectforwritingthevideowassuccessfullyinitialized.Forinstance,usinganimpropercodecproducesanerror.

TipBecautious;thevalidfourcccodesinasystemdependonthelocallyinstalledcodecs.Toknowtheinstalledfourcccodecsavailableinthelocalsystem,werecommendtheopensourcetoolMediaInfo,availableformanyplatformsathttp://mediaarea.net/en/MediaInfo.

VideoCapture&VideoCapture::operator>>(Mat&image):Thisgrabs,decodes,andreturnsthenextframe.ThismethodhastheequivalentboolVideoCapture::read(OutputArrayimage)function.ItcanbeusedratherthanusingtheVideoCapture::grab()function,followedbyVideoCapture::retrieve().VideoWriter&VideoWriter::operator<<(constMat&image):Thiswritesthenextframe.ThismethodhastheequivalentvoidVideoWriter::write(constMat&image)function.

Inthisexample,thereisareading/writingloopwherethewindoweventsarefetchedandhandledaswell.ThewaitKey(1000/fps)functioncallisinchargeofthat;however,inthiscase,1000/fpsindicatesthenumberofmillisecondstowaitbeforereturningtotheexternalloop.Althoughnotexact,anapproximatemeasureofframespersecondisobtainedfortherecordedvideo.

voidVideoCapture::release():Thisreleasesthevideofileorcapturingdevice.Althoughnotexplicitlynecessaryinthisexample,weincludeittoillustrateitsuse.

User-interactionstoolsIntheprevioussections,weexplainedhowtocreate(namedWindow)awindowtodisplay(imshow)animageandfetch/handleevents(waitKey).TheexamplesweprovideshowyouaveryeasymethodforuserinteractionwithOpenCVapplicationsthroughthekeyboard.ThewaitKeyfunctionreturnsthecodeofakeypressedbeforeatimeoutexpires.

Fortunately,OpenCVprovidesmoreflexiblewaysforuserinteraction,suchastrackbarsandmouseinteraction,whichcanbecombinedwithsomedrawingfunctionstoprovidearicheruserexperience.Moreover,ifOpenCVislocallycompiledwithQtsupport(theWITH_QToptionofCMake),asetofnewfunctionsareavailabletoprogramanevenbetterUI.

Inthissection,weprovideaquickreviewoftheavailablefunctionalitytoprogramuserinterfacesinanOpenCVprojectwithQtsupport.WeillustratethisreviewonOpenCVUIsupportwiththenextexamplenamedshowUI.

Theexampleshowsyouacolorimageinawindow,illustratinghowtousesomebasicelementstoenrichtheuserinteraction.ThefollowingscreenshotdisplaystheUIelementscreatedintheexample:

TheoutputwindowfortheshowUIexample

ThesourcecodeoftheshowUIexample(withoutthecallbackfunctions)isasfollows:

#include<iostream>

usingnamespacestd;

usingnamespacecv;

//Callbackfunctionsdeclarations

voidcbMouse(intevent,intx,inty,intflags,void*);

voidtb1_Callback(intvalue,void*);

voidtb2_Callback(intvalue,void*);

voidcheckboxCallBack(intstate,void*);

voidradioboxCallBack(intstate,void*id);

voidpushbuttonCallBack(int,void*font);

//Globaldefinitionsandvariables

Matorig_img,tmp_img;

constcharmain_win[]="main_win";

charmsg[50];

intmain(int,char*argv[]){

constchartrack1[]="TrackBar1";

constchartrack2[]="TrackBar2";

constcharcheckbox[]="CheckBox";

constcharradiobox1[]="RadioBox1";

constcharradiobox2[]="RadioBox2";

constcharpushbutton[]="PushButton";

inttb1_value=50;//Initialvalueoftrackbar1

inttb2_value=25;//Initialvalueoftrackbar1

orig_img=imread(argv[1]);//Openandreadtheimage

if(orig_img.empty()){

cout<<"Error!!!Imagecannotbeloaded…"<<endl;

return-1;

namedWindow(main_win);//Createsmainwindow

//Createsafontforaddingtexttotheimage

QtFontfont=fontQt("Arial",20,Scalar(255,0,0,0),

QT_FONT_BLACK,QT_STYLE_NORMAL);

//CreationofCallBackfunctions

setMouseCallback(main_win,cbMouse,NULL);

createTrackbar(track1,main_win,&tb1_value,

100,tb1_Callback);

createButton(checkbox,checkboxCallBack,0,

QT_CHECKBOX);

//Passingvalues(font)totheCallBack

createButton(pushbutton,pushbuttonCallBack,

(void*)&font,QT_PUSH_BUTTON);

createTrackbar(track2,NULL,&tb2_value,

50,tb2_Callback);

//PassingvaluestotheCallBack

createButton(radiobox1,radioboxCallBack,

(void*)radiobox1,QT_RADIOBOX);

createButton(radiobox2,radioboxCallBack,

(void*)radiobox2,QT_RADIOBOX);

imshow(main_win,orig_img);//Showsoriginalimage

cout<<"Pressanykeytoexit…"<<endl;

waitKey();//Infiniteloopwithhandleforevents

return0;

WhenOpenCVisbuiltwithQtsupport,everycreatedwindow—throughthehighguimodule—showsadefaulttoolbar(seetheprecedingfigure)withoptions(fromlefttoright)forpanning,zooming,saving,andopeningthepropertieswindow.

Additionaltothementionedtoolbar(onlyavailablewithQt),inthenextsubsections,wecommentthedifferentUIelementscreatedintheexampleandthecodetoimplementthem.

TrackbarsTrackbarsarecreatedwiththecreateTrackbar(constString&trackbarname,constString&winname,int*value,intcount,TrackbarCallbackonChange=0,void*

userdata=0)functioninthespecifiedwindow(winname),withalinkedintegervalue(value),amaximumvalue(count),anoptionalcallbackfunction(onChange)tobecalledonchangesoftheslider,andanargument(userdata)tothecallbackfunction.Thecallbackfunctionitselfgetstwoarguments:value(selectedbytheslider)andapointertouserdata(optional).WithQtsupport,ifnowindowisspecified,thetrackbariscreatedinthepropertieswindow.IntheshowUIexample,wecreatetwotrackbars:thefirstinthemainwindowandthesecondoneinthepropertieswindow.Thecodeforthetrackbarcallbacksis:

voidtb1_Callback(intvalue,void*){

sprintf(msg,"Trackbar1changed.Newvalue=%d",value);

displayOverlay(main_win,msg);

return;

voidtb2_Callback(intvalue,void*){

sprintf(msg,"Trackbar2changed.Newvalue=%d",value);

displayStatusBar(main_win,msg,1000);

return;

MouseinteractionMouseeventsarealwaysgeneratedsothattheuserinteractswiththemouse(movingandclicking).Bysettingtheproperhandlerorcallbackfunctions,itispossibletoimplementactionssuchasselect,draganddrop,andsoon.Thecallbackfunction(onMouse)isenabledwiththesetMouseCallback(constString&winname,MouseCallbackonMouse,void*userdata=0)functioninthespecifiedwindow(winname)andoptionalargument(userdata).

Thesourcecodeforthecallbackfunctionthathandlesthemouseeventis:

voidcbMouse(intevent,intx,inty,intflags,void*){

//Staticvarsholdvaluesbetweencalls

staticPointp1,p2;

staticboolp2set=false;

//Leftmousebuttonpressed

if(event==EVENT_LBUTTONDOWN){

p1=Point(x,y);//Setorig.point

p2set=false;

}elseif(event==EVENT_MOUSEMOVE&&

flags==EVENT_FLAG_LBUTTON){

//Checkmovingmouseandleftbuttondown

//Checkoutbounds

if(x>orig_img.size().width)

x=orig_img.size().width;

elseif(x<0)

//Checkoutbounds

if(y>orig_img.size().height)

y=orig_img.size().height;

elseif(y<0)

p2=Point(x,y);//Setfinalpoint

p2set=true;

//Copyorig.totemp.image

orig_img.copyTo(tmp_img);

//Drawsrectangle

rectangle(tmp_img,p1,p2,Scalar(0,0,255));

//Drawtemporalimagewithrect.

imshow(main_win,tmp_img);

}elseif(event==EVENT_LBUTTONUP

&&p2set){

//Checkifleftbuttonisreleased

//andselectedanarea

//Setsubarrayonorig.image

//withselectedrectangle

Matsubmat=orig_img(Rect(p1,p2));

//Heresomeprocessingforthesubmatrix

//Marktheboundariesofselectedrectangle

rectangle(orig_img,p1,p2,Scalar(0,0,255),2);

imshow("main_win",orig_img);

return;

IntheshowUIexample,themouseeventsareusedtocontrolthroughacallbackfunction(cbMouse),theselectionofarectangularregionbydrawingarectanglearoundit.Intheexample,thisfunctionisdeclaredasvoidcbMouse(intevent,intx,inty,intflags,void*),theargumentsbeingthepositionofthepointer(x,y)wheretheeventoccurs,theconditionwhentheeventoccurs(flags),andoptionally,userdata.

NoteTheavailableevents,flags,andtheircorrespondingdefinitionsymbolscanbefoundinthehighgui.hppheaderfile.

ButtonsOpenCV(onlywithQtsupport)allowsyoutocreatethreetypesofbuttons:checkbox(QT_CHECKBOX),radiobox(QT_RADIOBOX),andpushbutton(QT_PUSH_BUTTON).Thesetypesofbuttoncanbeusedrespectivelytosetoptions,setexclusiveoptions,andtakeactionsonpush.ThethreearecreatedwiththecreateButton(constString&button_name,ButtonCallbackon_change,void*userdata=0,int

type=QT_PUSH_BUTTON,boolinit_state=false)functioninthepropertieswindowarrangedinabuttonbarafterthelasttrackbarcreatedinthiswindow.Theargumentsforthebuttonareitsname(button_name),thecallbackfunctioncalledonthestatuschange(on_change),andoptionally,anargument(userdate)tothecallback,thetypeofbutton(type),andtheinitialstateofthebutton(init_state).

Next,weshowyouthesourcecodeforthecallbackfunctionscorrespondingtobuttonsintheexample:

voidcheckboxCallBack(intstate,void*){

sprintf(msg,"Checkboxchanged.Newstate=%d",state);

displayStatusBar(main_win,msg);

return;

voidradioboxCallBack(intstate,void*id){

//IdoftheradioboxpassedtothecallBack

sprintf(msg,"%schanged.Newstate=%d",

(char*)id,state);

displayStatusBar(main_win,msg);

return;

voidpushbuttonCallBack(int,void*font){

//Addtexttotheimage

addText(orig_img,"Pushbuttonclicked",

Point(50,50),*((QtFont*)font));

imshow(main_win,orig_img);//Showsoriginalimage

return;

Thecallbackfunctionforabuttongetstwoarguments:itsstatusand,optionally,apointertouserdata.IntheshowUIexample,weshowyouhowtopassaninteger(radioboxCallBack(intstate,void*id))toidentifythebuttonandamorecomplexobject(pushbuttonCallBack(int,void*font)).

DrawinganddisplayingtextAveryefficientwaytocommunicatetheresultsofsomeimageprocessingtotheuserisbydrawingshapesor/anddisplayingtextoverthefigurebeingprocessed.Throughtheimgprocmodule,OpenCVprovidessomeconvenientfunctionstoachievesuchtasksasputtingtext,drawinglines,circles,ellipses,rectangles,polygons,andsoon.TheshowUIexampleillustrateshowtoselectarectangularregionoveranimageanddrawarectangletomarktheselectedarea.Thefollowingfunctiondraws(img)arectangledefinedbytwopoints(p1,p2)overanimagewiththespecifiedcolorandotheroptionalparametersasthickness(negativeforafillshape)andthetypeoflines:

voidrectangle(InputOutputArrayimg,Pointpt1,Pointpt2,constScalar&

color,intthickness=1,intlineType=LINE_8,intshift=0)

Additionaltoshapes’drawingsupport,theimgprocmoduleprovidesafunctiontoputtextoveranimagewiththefunction:

voidputText(InputOutputArrayimg,constString&text,Pointorg,int

fontFace,doublefontScale,Scalarcolor,intthickness=1,int

lineType=LINE_8,boolbottomLeftOrigin=false)

NoteTheavailablefontfacesforthetextcanbeinspectedinthecore.hppheaderfile.

Qtsupport,inthehighguimodule,addssomeadditionalwaystoshowtextonthemainwindowofanOpenCVapplication:

Textovertheimage:WegetthisresultusingtheaddText(constMat&img,constString&text,Pointorg,constQtFont&font)function.ThisfunctionallowsyoutoselecttheoriginpointforthedisplayedtextwithafontpreviouslycreatedwiththefontQt(constString&nameFont,intpointSize=-1,Scalarcolor=Scalar::all(0),intweight=QT_FONT_NORMAL,int

style=QT_STYLE_NORMAL,intspacing=0)function.IntheshowUIexample,thisfunctionisusedtoputtextovertheimagewhenthepushbuttonisclickedon,callingtheaddTextfunctioninsidethecallbackfunction.Textonthestatusbar:UsingthedisplayStatusBar(constString&winname,constString&text,intdelayms=0)function,wedisplaytextinthestatusbarforanumberofmillisecondsgivenbythelastargument(delayms).IntheshowUIexample,thisfunctionisused(inthecallbackfunctions)todisplayaninformativetextwhenthebuttonsandtrackbarofthepropertieswindowchangetheirstate.Textoverlaidontheimage:UsingthedisplayOverlay(constString&winname,constString&text,intdelayms=0)function,wedisplaytextoverlaidontheimageforanumberofmillisecondsgivenbythelastargument.IntheshowUIexample,thisfunctionisused(inthecallbackfunction)todisplayinformativetextwhenthemainwindowtrackbarchangesitsvalue.

SummaryInthischapter,yougotaquickreviewofthemainpurposeoftheOpenCVlibraryanditsmodules.Youlearnedthefoundationsofhowtocompile,install,andusethelibraryinyourlocalsystemtodevelopC++OpenCVapplicationswithQtsupport.Todevelopyourownsoftware,weexplainedhowtostartwiththefreeQtCreatorIDEandtheGNUcompilerkit.

Tostartwith,fullcodeexampleswereprovidedinthechapter.Theseexamplesshowedyouhowtoreadandwriteimagesandvideo.Finally,thechaptergaveyouanexampleofdisplayingsomeeasy-to-implementuserinterfacecapabilitiesinOpenCVprograms,suchastrackbars,buttons,puttingtextonimages,drawingshapes,andsoon.

Thenextchapterwillbedevotedtoestablishingthemainimageprocessingtoolsandtasksthatwillsetthebasisfortheremainingchapters.

Chapter2.EstablishingImageProcessingToolsThischapterdescribesthemaindatastructuresandbasicproceduresthatwillbeusedinsubsequentchapters:

ImagetypesPixelaccessBasicoperationswithimagesHistograms

Thesearesomeofthemostfrequentoperationsthatwewillhavetoperformonimages.Mostofthefunctionalitycoveredhereisinthecoremoduleofthelibrary.

BasicdatatypesThefundamentaldatatypeinOpenCVisMat,asitisusedtostoreimages.Basically,animageisstoredasaheaderplusamemoryzonecontainingthepixeldata.Imageshaveanumberofchannels.Grayscaleimageshaveasinglechannel,whilecolorimagestypicallyhavethreeforthered,green,andbluecomponents(althoughOpenCVstorestheminareverseorder,thatisblue,green,andred).Afourthtransparency(alpha)channelcanalsobeused.Thenumberofchannelsforanimgimagecanberetrievedwithimg.channels().

Eachpixelinanimageisstoredusinganumberofbits.Thisiscalledtheimagedepth.Forgrayscaleimages,pixelsarecommonlystoredin8bits,thusallowing256graylevels(integervalues0to255).Forcolorimages,eachpixelisstoredinthreebytes,onepercolorchannel.Insomeoperations,itwillbenecessarytostorepixelsinafloating-pointformat.Theimagedepthcanbeobtainedwithimg.depth(),andthevaluesreturnedare:

CV_8U,8-bitunsignedintegers(0..255)CV_8S,8-bitsignedintegers(-128..127)CV_16U,16-bitunsignedintegers(0..65,535)CV_16S,16-bitsignedintegers(-32,768..32,767)CV_32S,32-bitsignedintegers(-2,147,483,648..2,147,483,647)CV_32F,32-bitfloating-pointnumbersCV_64F,64-bitfloating-pointnumbers

NotethatthemostcommonimagedepthwillbeCV_8Uforbothgrayscaleandcolorimages.ItispossibletoconvertfromonedepthtoanotherusingtheconvertTomethod:

Matimg=imread("lena.png",IMREAD_GRAYSCALE);

Matfp;

img.convertTo(fp,CV_32F);

Itiscommontoperformanoperationonfloating-pointimages(thatis,pixelvaluesaretheresultofamathematicaloperation).Ifweuseimshow()todisplaythisimage,wewillnotseeanythingmeaningful.Inthiscase,wehavetoconvertpixelstotheintegerrange0..255.TheconvertTofunctionimplementsalineartransformationandhastwoadditionalparameters,alphaandbeta,whichrepresentascalefactorandadeltavaluetoadd,respectively.Thismeansthateachpixelpisconvertedwith:

newp=alpha*p+beta

Thiscanbeusedtodisplayfloatingpointimagesproperly.AssumingthattheimgimagehasmandMminimumandmaximumvalues(refertothefollowingcodetoseehowtoobtainthesevalues),wewouldusethis:

Matm1=Mat(100,100,CV_32FC1);

randu(m1,0,1e6);//randomvaluesbetween0and1e6

imshow("original",m1);

doubleminRange,MaxRange;

PointmLoc,MLoc;

minMaxLoc(m1,&minRange,&MaxRange,&mLoc,&MLoc);

Matimg1;

m1.convertTo(img1,CV_8U,255.0/(MaxRange-minRange),-255.0/minRange);

imshow("result",img1);

Thiscodemapstherangeoftheresultimagevaluestotherange0-255.Thefollowingimageshowsyoutheresultofrunningthecode:

TheresultofconvertTo(notethattheimageontheleft-handsideisdisplayedaswhite)

Theimagesizecanbeobtainedwiththerowsandcolsattributes.Thereisalsoasizeattributethatretrievesboth:

MatSizes=img.size;

intr=l[0];

intc=l[1];

Apartfromtheimageitself,otherdatatypesareverycommon;refertothefollowingtable:

Type Typekeyword Example

(Small)vector VecAB,whereAcanbe2,3,4,5or6,Bcanbeb,s,i,f,ord Vec3brgb;

rgb[0]=255;

(Upto4)scalars Scalar

Scalara;

a[0]=0;

a[1]=0;

Point PointAB,whereAcanbe2or3andBcanbei,f,ordPoint3dp;

p.x=0;

p.y=0;

p.z=0;

Size Size

Sizes;

s.width=30;

s.height=40;

Rectangle Rect

Rectr;

r.x=r.y=0;

r.width=r.height=100;

Someofthesetypeshaveadditionaloperations.Forexample,wecancheckwhetherapointliesinsidearectangle:

p.inside(r)

Thepandrargumentsare(two-dimensional)pointandrectangle,respectively.Notethatinanycase,theprecedingtableisnotexhaustive;OpenCVprovidesmanymoresupport

structureswithassociatedmethods.

Pixel-levelaccessToprocessimages,wehavetoknowhowtoaccesseachpixelindependently.OpenCVprovidesanumberofwaystodothis.Inthissection,wecovertwomethods;thefirstoneiseasyfortheprogrammer,whilethesecondoneismoreefficient.

Thefirstmethodusestheat<>templatefunction.Inordertouseit,wehavetospecifythetypeofmatrixcells,suchasinthisshortexample:

Matsrc1=imread("lena.jpg",IMREAD_GRAYSCALE);

ucharpixel1=src1.at<uchar>(0,0);

cout<<"Valueofpixel(0,0):"<<(unsignedint)pixel1<<endl;

Matsrc2=imread("lena.jpg",IMREAD_COLOR);

Vec3bpixel2=src2.at<Vec3b>(0,0);

cout<<"Bcomponentofpixel(0,0):"<<(unsignedint)pixel2[0]<<endl;

Theexamplereadsanimageinbothgrayscaleandcolorandaccessesthefirstpixelat(0,0).Inthefirstcase,thepixeltypeisunsignedchar(thatis,uchar).Inthesecondcase,whentheimageisreadinfullcolor,wehavetousetheVec3btype,whichreferstoatripletofunsignedchars.Ofcourse,theat<>functioncanalsoappearontheleft-handsideofanassignment,thatis,tochangethevalueofapixel.

Thefollowingisanothershortexampleinwhichafloating-pointmatrixisinitializedtothePivalueusingthismethod:

MatM(200,200,CV_64F);

for(inti=0;i<M.rows;i++)

for(intj=0;j<M.cols;j++)

M.at<double>(i,j)=CV_PI;

Notethattheat<>methodisnotveryefficientasithastocalculatetheexactmemorypositionfromthepixelrowandcolumn.Thiscanbeverytimeconsumingwhenweprocessthewholeimagepixelbypixel.Thesecondmethodusestheptrfunction,whichreturnsapointertoaspecificimagerow.Thefollowingsnippetobtainsthepixelvalueofeachpixelinacolorimage:

ucharR,G,B;

for(inti=0;i<src2.rows;i++)

Vec3b*pixrow=src2.ptr<Vec3b>(i);

for(intj=0;j<src2.cols;j++)

B=pixrow[j][0];

G=pixrow[j][1];

R=pixrow[j][2];

Intheexampleabove,ptrisusedtogetapointertothefirstpixelineachrow.Usingthispointer,wecannowaccesseachcolumnintheinnermostloop.

MeasuringthetimeProcessingimagestakestime(comparablymuchmorethanthetimeittakestoprocess1Ddata).Often,processingtimeisthecrucialfactorthatdecideswhetherasolutionispracticalornot.OpenCVprovidestwofunctionstomeasuretheelapsedtime:getTickCount()andgetTickFrequency().You’llusethemlikethis:

doublet0=(double)getTickCount();

//yourstuffhere…

elapsed=((double)getTickCount()–t0)/getTickFrequency();

Here,elapsedisinseconds.

CommonoperationswithimagesThefollowingtablesummarizesthemosttypicaloperationswithimages:

Operation Codeexamples

Setmatrixvalues img.setTo(0);//for1-channelimg

img.setTo(Scalar(B,G,R);//3-channelimg

MATLAB-stylematrixinitializationMatm1=Mat::eye(100,100,CV_64F);

Matm3=Mat::zeros(100,100,CV_8UC1);

Matm2=Mat::ones(100,100,CV_8UC1)*255;

Randominitialization Matm1=Mat(100,100,CV_8UC1);

randu(m1,0,255);

Createacopyofthematrix Matimg1=img.clone();

Createacopyofthematrix(withthemask)

img.copy(img1,mask);

Referenceasubmatrix(thedataisnotcopied)

Matimg1=img(Range(r1,r2),Range(c1,c2));

Imagecrop Rectroi(r1,c2,width,height);

Matimg1=img(roi).clone();//datacopied

Resizeimage resize(img,imag1,Size(),0.5,0.5);//decimatebyafactor

Flipimageflip(imgsrc,imgdst,code);

//code=0=>verticalflipping

//code>0=>horizontalflipping

//code<0=>vertical&horizontalflipping

SplitchannelsMatchannel[3];

split(img,channel);

imshow("B",channel[0]);//showblue

Mergechannels merge(channel,img);

Countnonzeropixels intnz=countNonZero(img);

Minimumandmaximum doublem,M;

PointmLoc,MLoc;minMaxLoc(img,&m,&M,&mLoc,&MLoc);

ThemeanpixelvalueScalarm,stdd;

meanStdDev(img,m,stdd);

uintmean_pxl=mean.val[0];

Checkwhethertheimagedataisnull If(img.empty())

cout<<"couldn'tloadimage";

ArithmeticoperationsArithmeticoperatorsareoverloaded.ThismeansthatwecanoperateonMatimageslikewecaninthisexample:

imgblend=0.2*img1+0.8*img2;

InOpenCV,theresultvalueofanoperationissubjecttotheso-calledsaturationarithmetic.Thismeansthatthefinalvalueisactuallythenearestintegerinthe0..255range.

Bitwiseoperationsbitwise_and(),bitwise_or(),bitwise_xor(),andbitwise_not()canbeveryusefulwhenworkingwithmasks.Masksarebinaryimagesthatindicatethepixelsinwhichanoperationistobeperformed(insteadofthewholeimage).Thefollowingbitwise_andexampleshowsyouhowtousetheANDoperationtocroppartofanimage:

usingnamespacecv;

usingnamespacestd;

intmain()

Matimg1=imread("lena.png",IMREAD_GRAYSCALE);

if(img1.empty())

cout<<"Cannotloadimage!"<<endl;

return-1;

imshow("Original",img1);//Original

//Createmaskimage

Matmask(img1.rows,img1.cols,CV_8UC1,Scalar(0,0,0));

circle(mask,Point(img1.rows/2,img1.cols/2),150,255,-1);

imshow("Mask",mask);

//performAND

bitwise_and(img1,mask,r);

//filloutsidewithwhite

constucharwhite=255;

for(inti=0;i<r.rows;i++)

for(intj=0;j<r.cols;j++)

if(!mask.at<uchar>(i,j))

r.at<uchar>(i,j)=white;

imshow("Result",r);

waitKey(0);

return0;

Afterreadinganddisplayingtheinputimage,wecreateamaskbydrawingafilledwhitecircle.IntheANDoperation,thismaskisused.Thelogicaloperationisonlyappliedinthosepixelsinwhichthemaskvalueisnotzero;otherpixelsarenotaffected.Finally,inthisexample,wefilltheouterpartoftheresultimage(thatis,outsidethecircle)withwhite.Thisisdoneusingoneofthepixelaccessmethodsexplainedpreviously.Seetheresultingimagesinthefollowingscreenshot:

Theresultofthebitwise_andexample

Next,anothercoolexampleisshowninwhichweestimatethevalueofPi.Let’sconsiderasquareanditsenclosedcircle:

Theirareasaregivenby:

Fromthis,wehave:

Let’sassumethatwehaveasquareimageofunknownsidelengthandanenclosedcircle.Wecanestimatetheareaoftheenclosedcirclebypaintingmanypixelsinrandompositionswithintheimageandcountingthosethatfallinsidetheenclosedcircle.Ontheotherhand,theareaofthesquareisestimatedasthetotalnumberofpixelspainted.ThiswouldallowyoutoestimatethevalueofPiusingthepreviousequation.

Thefollowingalgorithmsimulatesthis:

1. Onablacksquareimage,paintasolidwhiteenclosedcircle.2. Onanotherblacksquareimage(samedimensions),paintalargenumberofpixelsat

randompositions.3. PerformanANDoperationbetweenthetwoimagesandcountnonzeropixelsinthe

resultingimage.4. EstimatePiusingtheequation.

ThefollowingisthecodefortheestimatePiexample:

usingnamespacecv;

usingnamespacestd;

intmain()

constintside=100;

constintnpixels=8000;

inti,j;

Mats1=Mat::zeros(side,side,CV_8UC1);

Mats2=s1.clone();

circle(s1,Point(side/2,side/2),side/2,255,-1);

imshow("s1",s1);

for(intk=0;k<npixels;k++)

i=rand()%side;

j=rand()%side;

s2.at<uchar>(i,j)=255;

bitwise_and(s1,s2,r);

imshow("s2",s2);

imshow("r",r);

intAcircle=countNonZero(r);

intAsquare=countNonZero(s2);

floatPi=4*(float)Acircle/Asquare;

cout<<"EstimatedvalueofPi:"<<Pi<<endl;

waitKey();

return0;

Theprogramfollowstheprecedingalgorithmexactly.NotethatweusethecountNonZerofunctiontocountnonzero(white,inthiscase)pixels.Fornpixels=8000,theestimatewas3.125.Thelargerthevalueofnpixels,thebettertheestimation.

TheoutputoftheestimatePiexample

DatapersistenceApartfromthespecificfunctionstoreadandwriteimagesandvideo,inOpenCV,thereisamoregenericwaytosave/loadthedata.Thisisreferredtoasdatapersistence:thevalueofobjectsandvariablesintheprogramcanberecorded(serialized)onthedisk.Thiscanbeveryusefultosaveresultsandloadtheconfigurationdata.ThemainclassistheaptlynamedFileStorage,whichrepresentsafileonadisk.DataisactuallystoredinXMLorYAMLformats.

Thesearethestepsinvolvedwhenwritingdata:

1. CalltheFileStorageconstructor,passingafilenameandaflagwiththeFileStorage::WRITEvalue.Thedataformatisdefinedbythefileextension(thatis,.xml,.yml,or.yaml).

2. Usethe<<operatortowritedatatothefile.Dataistypicallywrittenasstring-valuepairs.

3. Closethefileusingthereleasemethod.

Readingdatarequiresthatyoufollowthesesteps:

1. CalltheFileStorageconstructor,passingafilenameandaflagwiththeFileStorage::READvalue.

2. Usethe[]or>>operatortoreaddatafromthefile.3. Closethefileusingthereleasemethod.

Thefollowingexampleusesdatapersistencetosaveandloadtrackbarvalues.

usingnamespacecv;

usingnamespacestd;

Matimg1;

voidtb1_Callback(intvalue,void*)

Mattemp=img1+value;

imshow("main_win",temp);

intmain()

img1=imread("lena.png",IMREAD_GRAYSCALE);

if(img1.empty())

cout<<"Cannotloadimage!"<<endl;

return-1;

inttb1_value=0;

//loadtrackbarvalue

FileStoragefs1("config.xml",FileStorage::READ);

tb1_value=fs1["tb1_value"];//method1

fs1["tb1_value"]>>tb1_value;//method2

fs1.release();

//createtrackbar

namedWindow("main_win");

createTrackbar("brightness","main_win",&tb1_value,

255,tb1_Callback);

tb1_Callback(tb1_value,NULL);

waitKey();

//savetrackbarvalueuponexiting

FileStoragefs2("config.xml",FileStorage::WRITE);

fs2<<"tb1_value"<<tb1_value;

fs2.release();

return0;

TipWhenOpenCVhasbeencompiledwithQtsupport,windowpropertiescanbesaved,includingtrackbarvalues,withthesaveWindowParameters()function.

Oncethetrackbarisusedtocontrolanintegervalue,itissimplyaddedtotheoriginalimage,makingitbrighter.Thisvalueisreadwhentheprogramstarts(thevaluewillbe0thefirsttime)andsavedwhentheprogramexitsnormally.Notethattwoequivalentmethodsareshowntoreadthevalueofthetb1_valuevariable.Thecontentsoftheconfig.xmlfileare:

<?xmlversion="1.0"?>

<opencv_storage>

<tb1_value>112</tb1_value>

</opencv_storage>

HistogramsOncetheimagehasbeendefinedwithadatatypeandweareabletoaccessitsgraylevelvalues,thatis,thepixels,wemaywanttoobtainaprobabilitydensityfunctionofthedifferentgraylevels,whichiscalledthehistogram.Theimagehistogramrepresentsthefrequencyofoccurrenceofthevariousgraylevelsintheimage.Thehistogramcanbemodeledsothattheimagemaychangeitscontrastlevels.Thisisknownashistogramequalization.Histogrammodelingisapowerfultechniqueforimageenhancementbymeansofcontrastvariation.Theequalizationallowsforimageareasoflowercontrasttogainahighercontrast.Thefollowingimageshowsyouanexampleofanequalizedimageanditshistogram:

Anexampleofanequalizedimagehistogram

InOpenCV,theimagehistogramcanbecalculatedwiththevoidcalcHistfunctionandhistogramequalizationisperformedwiththevoidequalizeHistfunction.

Theimagehistogramcalculationisdefinedwithtenparameters:voidcalcHist(constMat*images,intnimages,constint*channels,InputArraymask,OutputArrayhist,intdims,constint*histSize,constfloat**ranges,booluniform=true,andboolaccumulate=false).

constMat*images:Thefirstparameteristheaddressofthefirstimagefromacollection.Thiscanbeusedtoprocessabatchofimages.intnimages:Thesecondparameteristhenumberofsourceimages.constint*channels:Thethirdinputparameteristhelistofthechannelsusedtocomputethehistogram.Thenumberofchannelsgoesfrom0to2.InputArraymask:Thisisanoptionalmasktoindicatetheimagepixelscountedinthehistogram.OutputArrayhist:Thefifthparameteristheoutputhistogram.intdims:Thisparameterallowsyoutoindicatethedimensionofthehistogram.constint*histSize:Thisparameteristhearrayofhistogramsizesineachdimension.constfloat**ranges:Thisparameteristhearrayofthedimsarraysofthehistogrambinboundariesineachdimension.booluniform=true:Bydefault,theBooleanvalueistrue.Itindicatesthatthehistogramisuniform.boolaccumulate=false:Bydefault,theBooleanvalueisfalse.Itindicatesthatthe

histogramisnonaccumulative.

Thehistogramequalizationrequiresonlytwoparameters,voidequalizeHist(InputArraysrc,OutputArraydst).Thefirstparameteristheinputimageandthesecondoneistheoutputimagewiththehistogramequalized.

Itispossibletocalculatethehistogramofmorethanoneinputimage.Thisallowsyoutocompareimagehistogramsandcalculatethejointhistogramofseveralimages.Thecomparisonoftwoimagehistograms,histImage1andhistImage2,canbeperformedwiththevoidcompareHist(InputArrayhistImage1,InputArrayhistImage2,method)function.TheMethodmetricistheusedtocomputethematchingbetweenbothhistograms.TherearefourmetricsimplementedinOpenCV,thatis,correlation(CV_COMP_CORREL),chi-square(CV_COMP_CHISQR),intersectionorminimumdistance(CV_COMP_INTERSECT),andBhattacharyyadistance(CV_COMP_BHATTACHARYYA).

Itispossibletocalculatethehistogramofmorethanonechannelofthesamecolorimage.Thisispossiblethankstothethirdparameter.

Thefollowingsectionsshowyoutwoexamplecodesforcolorhistogramcalculation(ColourImageEqualizeHist)andcomparisonColourImageComparison.InColourImageEqualizeHist,itisalsoshownhowtocalculatethehistogramequalizationaswellasthe2Dhistogramfortwochannels,thatis,hue(H)andsaturation(S),intheColourImageComparisonexample.

TheexamplecodeThefollowingColourImageEqualizeHistexampleshowsyouhowtoequalizeacolorimageanddisplaythehistogramofeachchannelatthesametime.ThehistogramcalculationofeachcolorchannelintheRGBimageisdonewiththehistogramcalculation(InputArrayImagesrc,OutputArrayhistoImage)function.Tothisend,thecolorimageissplitintothechannels:R,G,andB.Thehistogramequalizationisalsoappliedtoeachchannelthatisthenmergedtoformtheequalizedcolorimage:

#include"opencv2/highgui/highgui.hpp"

#include"opencv2/imgproc/imgproc.hpp"

#include<iostream>

#include<stdio.h>

usingnamespacecv;

usingnamespacestd;

voidhistogramcalculation(constMat&Image,Mat&histoImage)

inthistSize=255;

//Settheranges(forB,G,R))

floatrange[]={0,256};

constfloat*histRange={range};

booluniform=true;boolaccumulate=false;

Matb_hist,g_hist,r_hist;

vector<Mat>bgr_planes;

split(Image,bgr_planes);

//Computethehistograms:

calcHist(&bgr_planes[0],1,0,Mat(),b_hist,1,&histSize,&histRange,

uniform,accumulate);

calcHist(&bgr_planes[1],1,0,Mat(),g_hist,1,&histSize,&histRange,

calcHist(&bgr_planes[2],1,0,Mat(),r_hist,1,&histSize,&histRange,

//DrawthehistogramsforB,GandR

inthist_w=512;inthist_h=400;

intbin_w=cvRound((double)hist_w/histSize);

MathistImage(hist_h,hist_w,CV_8UC3,Scalar(0,0,0));

//Normalizetheresultto[0,histImage.rows]

normalize(b_hist,b_hist,0,histImage.rows,NORM_MINMAX,-1,Mat());

normalize(g_hist,g_hist,0,histImage.rows,NORM_MINMAX,-1,Mat());

normalize(r_hist,r_hist,0,histImage.rows,NORM_MINMAX,-1,Mat());

//Drawforeachchannel

for(inti=1;i<histSize;i++){

line(histImage,Point(bin_w*(i-1),hist_h-cvRound(b_hist.at<float>(i-

1))),Point(bin_w*(i),hist_h-cvRound(b_hist.at<float>(i))),Scalar(

255,0,0),2,8,0);

line(histImage,Point(bin_w*(i-1),hist_h-cvRound(g_hist.at<float>(i-

1))),Point(bin_w*(i),hist_h-cvRound(g_hist.at<float>(i))),Scalar(

0,255,0),2,8,0);

line(histImage,Point(bin_w*(i-1),hist_h-cvRound(r_hist.at<float>(i-

1))),Point(bin_w*(i),hist_h-cvRound(r_hist.at<float>(i))),Scalar(

0,0,255),2,8,0);

histoImage=histImage;

intmain(int,char*argv[])

Matsrc,imageq;

MathistImage;

//Readoriginalimage

src=imread("fruits.jpg");

if(!src.data)

{printf("Errorimagen\n");exit(1);}

//Separatetheimagein3places(B,GandR)

split(src,bgr_planes);

//Displayresults

imshow("Sourceimage",src);

//Calculatethehistogramtoeachchannelofthesourceimage

histogramcalculation(src,histImage);

//Displaythehistogramforeachcolourchannel

imshow("ColourImageHistogram",histImage);

//EqualizedImage

//ApplyHistogramEqualizationtoeachchannel

equalizeHist(bgr_planes[0],bgr_planes[0]);

//Mergetheequalizedimagechannelsintotheequalizedimage

merge(bgr_planes,imageq);

//DisplayEqualizedImage

imshow("EqualizedImage",imageq);

//Calculatethehistogramtoeachchanneloftheequalizedimage

histogramcalculation(imageq,histImage);

//DisplaytheHistogramoftheEqualizedImage

imshow("EqualizedColourImageHistogram",histImage);

//Waituntiluserexitstheprogram

waitKey();

return0;

Theexamplecreatesfourwindowswith:

Thesourceimage:Thisisshowninthefollowingfigureintheupper-leftcorner.Theequalizedcolorimage:Thisisshowninthefollowingfigureintheupper-rightcorner.Thehistogramofthreechannels:Here,R=Read,G=GreenandB=Blue,forthesourceimage.Thisisshowninthefollowingfigureinthelower-leftcorner.ThehistogramofRGBchannelfortheequalizedimage:Thisisshowninnextfigureinthelower-rightcorner.ThefigureshowsyouhowthemostfrequentintensityvaluesforR,G,andBhavebeenstretchedoutduetotheequalizationprocess.

Thefollowingfigureshowsyoutheresultsofthealgorithm:

TheexamplecodeThefollowingColourImageComparisonexampleshowsyouhowtocalculatea2Dhistogramcomposedoftwochannelsfromthesamecolorimage.Theexamplecodealsoperformsacomparisonbetweentheoriginalimageandtheequalizedimagebymeansofhistogrammatching.Themetricsusedforthematchingarethefourmetricsthathavebeenmentionedpreviously,thatis,Correlation,Chi-Square,Minimumdistance,andBhattacharyyadistance.The2DhistogramcalculationoftheHandScolorchannelisdonewiththehistogram2Dcalculation(InputArrayImagesrc,OutputArrayhisto2D)function.Toperformthehistogramcomparison,thenormalized1DhistogramhasbeencalculatedfortheRGBimage.Inordertocomparethehistogram,theyhavebeennormalized.ThisisdoneinhistogramRGcalculation(InputArrayImagesrc,OutputArrayhisto):

voidhistogram2Dcalculation(constMat&src,Mat&histo2D)

Mathsv;

cvtColor(src,hsv,CV_BGR2HSV);

//Quantizethehueto30-255levels

//andthesaturationto32-255levels

inthbins=255,sbins=255;

inthistSize[]={hbins,sbins};

//huevariesfrom0to179,seecvtColor

floathranges[]={0,180};

//saturationvariesfrom0(black-gray-white)to

//255(purespectrumcolor)

floatsranges[]={0,256};

constfloat*ranges[]={hranges,sranges};

MatNDhist,hist2;

//wecomputethehistogramfromthe0-thand1-stchannels

intchannels[]={0,1};

calcHist(&hsv,1,channels,Mat(),hist,1,histSize,ranges,true,

false);

doublemaxVal=0;

minMaxLoc(hist,0,&maxVal,0,0);

intscale=1;

MathistImg=Mat::zeros(sbins*scale,hbins*scale,CV_8UC3);

for(inth=0;h<hbins;h++)

for(ints=0;s<sbins;s++)

floatbinVal=hist.at<float>(h,s);

intintensity=cvRound(binVal*255/maxVal);

rectangle(histImg,Point(h*scale,s*scale),

Point((h+1)*scale-1,(s+1)*scale-1),

Scalar::all(intensity),

CV_FILLED);

histo2D=histImg;

voidhistogramRGcalculation(constMat&src,Mat&histoRG)

//Using50binsforredand60forgreen

intr_bins=50;intg_bins=60;

inthistSize[]={r_bins,g_bins};

//redvariesfrom0to255,greenfrom0to255

floatr_ranges[]={0,255};

floatg_ranges[]={0,255};

constfloat*ranges[]={r_ranges,g_ranges};

//Usetheo-thand1-stchannels

intchannels[]={0,1};

//Histograms

MatNDhist_base;

//CalculatethehistogramsfortheHSVimages

calcHist(&src,1,channels,Mat(),hist_base,2,histSize,ranges,

true,false);

normalize(hist_base,hist_base,0,1,NORM_MINMAX,-1,Mat());

histoRG=hist_base;

Matsrc,imageq;

MathistImg,histImgeq;

MathistHSorg,histHSeq;

//Readoriginalimage

src=imread("fruits.jpg");

if(!src.data)

{printf("Errorimagen\n");exit(1);}

//Separatetheimagein3places(B,GandR)

split(src,bgr_planes);

//Displayresults

namedWindow("Sourceimage",0);

//Calculatethehistogramofthesourceimage

histogram2Dcalculation(src,histImg);

//Displaythehistogramforeachcolourchannel

imshow("H-SHistogram",histImg);

//EqualizedImage

//ApplyHistogramEqualizationtoeachchannel

//Mergetheequalizedimagechannelsintotheequalizedimage

merge(bgr_planes,imageq);

//DisplayEqualizedImage

namedWindow("EqualizedImage",0);

imshow("EqualizedImage",imageq);

//Calculatethe2DhistogramforHandSchannels

histogram2Dcalculation(imageq,histImgeq);

//Displaythe2DHistogram

imshow("H-SHistogramEqualized",histImgeq);

histogramRGcalculation(src,histHSorg);

histogramRGcalculation(imageq,histHSeq);

///Applythehistogramcomparisonmethods

for(inti=0;i<4;i++)

intcompare_method=i;

doubleorig_orig=compareHist(histHSorg,histHSorg,compare_method

doubleorig_equ=compareHist(histHSorg,histHSeq,compare_method);

printf("Method[%d]Original-Original,Original-Equalized:%f,%f

\n",i,orig_orig,orig_equ);

printf("Done\n");

waitKey();

Theexamplecreatesfourwindowswiththesourceimage,theequalizedcolorimageandthe2DhistogramforHandSchannelsforbothimagestheoriginal,andtheequalizedimage.ThealgorithmalsodisplaysthefournumericalmatchingparametersobtainedfromthecomparisonoftheoriginalRGBimagehistogramwithitselfandwiththeequalizedRGBimage.Forthecorrelationandintersectionmethods,thehigherthemetric,themoreaccuratethematch.Forthechi-squareandBhattacharyyadistance,thelesstheresult,thebetterthematch.ThefollowingfigureshowsyoutheoutputoftheColourImageComparisonalgorithm:

Finally,youcanrefertoChapter3,CorrectingandEnhancingImages,aswellastheexampleswithintocoveressentialaspectsofthisbroadtopic,suchasimageenhancementbymeansofhistogrammodeling.

NoteFormoreinformation,refertoOpenCVEssentials,DenizO.,FernándezM.M.,VállezN.,BuenoG.,SerranoI.,Patón.A.,SalidoJ.byPacktPublishing,https://www.packtpub.com/application-development/opencv-essentials.

SummaryThischaptercoveredandestablishedthebasisofapplyingimageprocessingmethodsusedincomputervision.Imageprocessingisoftenthefirststeptofurthercomputervisionapplications,andtherefore,ithasbeencoveredhere:basicdatatypes,pixellevelaccess,commonoperationswithimages,arithmeticoperations,datapersistence,andhistograms.

YoucanalsorefertoChapter3,CorrectingandEnhancingImages,ofOpenCVEssentialsbyPacktPublishingtocoverfurtheressentialaspectsofthisbroadtopic,suchasimageenhancement,imagerestorationbymeansoffiltering,andgeometricalcorrection.

Thenextchapterwillcoverfurtheraspectsofimageprocessingtocorrectandenhanceimagesbymeansofsmoothing,sharpening,imageresolutionanalysis,morphologicalandgeometricaltransforms,inpainting,anddenoising.

Chapter3.CorrectingandEnhancingImagesThischapterpresentsmethodsforimageenhancementandcorrection.Sometimes,itisnecessarytoreducethenoiseinanimageoremphasizeorsuppresscertaindetailsinit.Theseproceduresareusuallycarriedoutbymodifyingpixelvalues,performingsomeoperationsonthem,orontheirlocalneighborhoodaswell.Bydefinition,image-enhancementoperationsareusedtoimproveimportantimagedetails.Enhancementoperationsincludenoisereduction,smoothing,andedgeenhancement.Ontheotherhand,imagecorrectionattemptstorestoreadamagedimage.InOpenCV,theimgprocmodulecontainsfunctionsforimageprocessing.

Inthischapter,wewillcover:

Imagefiltering.Thisincludesimagesmoothing,imagesharpening,andworkingwithimagepyramids.Applyingmorphologicaloperations,suchasdilation,erosion,opening,orclosing.Geometricaltransformations(affineandperspectivetransformations).Inpainting,whichisusedtoreconstructdamagedpartsofimages.Denoising,whichisnecessarytoreducetheimagenoiseproducedbytheimage-capturedevice.

ImagefilteringImagefilteringisaprocesstomodifyorenhanceimages.Emphasizingcertainfeaturesorremovingothersinanimageareexamplesofimagefiltering.Filteringisaneighborhoodoperation.Theneighborhoodisasetofpixelsaroundaselectedone.Imagefilteringdeterminestheoutputvalueofacertainpixellocatedataposition(x,y)byperformingsomeoperationswiththevaluesofthepixelsinitsneighborhood.

OpenCVprovidesseveralfilteringfunctionsforcommonimage-processingoperations,suchassmoothingorsharpening.

SmoothingSmoothing,alsocalledblurring,isanimage-processingoperationthatisfrequentlyusedtoreducenoise,amongotherpurposes.Asmoothingoperationisperformedbyapplyinglinearfilterstotheimage.Then,thepixelvaluesoftheoutputatpositions(xi,yj)arecomputedasaweightedsumoftheinputpixelvaluesatpositions(xi,yj)andtheirneighborhoods.Theweightsforthepixelsinthelinearoperationareusuallystoredinamatrixcalledkernel.Therefore,afiltercouldberepresentedasaslidingwindowofcoefficients.

Therepresentationofthepixelneighborhood

LetKbethekernelandIandOtheinputandoutputimages,respectively.Then,eachoutputpixelvalueat(i,j)iscalculatedasfollows:

Median,Gaussian,andbilateralarethemostusedOpenCVsmoothingfilters.Medianfilteringisverygoodtogetridofsalt-and-pepperorspecklenoise,whileGaussianisamuchbetterpreprocessingstepforedgedetection.Ontheotherhand,bilateralfilteringisagoodtechniquetosmoothanimagewhilerespectingstrongedges.

ThefunctionsincludedinOpenCVforthispurposeare:

voidboxFilter(InputArraysrc,OutputArraydst,intddepth,Sizeksize,

Pointanchor=Point(-1,-1),boolnormalize=true,intborderType=

BORDER_DEFAULT):Thisisaboxfilterwhosekernelcoefficientsareequal.Withnormalize=true,eachoutputpixelvalueisthemeanofitskernelneighborswithallcoefficientsequalto1/n,wheren=thenumberofelements.Withnormalize=false,allcoefficientsareequalto1.Thesrcargumentistheinputimage,whilethefilteredimageisstoredindst.Theddepthparameterindicatestheoutputimagedepththatis-1tousethesamedepthastheinputimage.Thekernelsizeisindicatedinksize.Theanchorpointindicatesthepositionoftheso-calledanchorpixel.The(-1,-1)defaultvaluemeansthattheanchorisatthecenterofthekernel.Finally,theborder-typetreatmentisindicatedintheborderTypeparameter.voidGaussianBlur(InputArraysrc,OutputArraydst,Sizeksize,double

sigmaX,doublesigmaY=0,intborderType=BORDER_DEFAULT):ThisisdonebyconvolvingeachpointinthesrcinputarraywithaGaussiankerneltoproducethe

dstoutput.ThesigmaXandsigmaYparametersindicatetheGaussiankernelstandarddeviationinXandYdirections.IfsigmaYiszero,itissettobeequaltosigmaX,andifbothareequaltozero,theyarecomputedusingthewidthandheightgiveninksize.

NoteConvolutionisdefinedastheintegraloftheproductoftwofunctionsinwhichoneofthemispreviouslyreversedandshifted.

voidmedianBlur(InputArraysrc,OutputArraydst,intksize):Thisrunsthrougheachelementoftheimageandreplaceseachpixelwiththemedianofitsneighboringpixels.voidbilateralFilter(InputArraysrc,OutputArraydst,intd,double

sigmaColor,doublesigmaSpace,intborderType=BORDER_DEFAULT):ThisisanalogoustotheGaussianfilterconsideringtheneighboringpixelswithweightsassignedtoeachofthembuthavingtwocomponentsoneachweight,whichisthesameusedbytheGaussianfilter,andanotheronethattakesintoaccountthedifferenceinintensitybetweentheneighboringandevaluatedpixels.Thisfunctionneedsthediameterofthepixelneighborhoodasparameterd,andsigmaColorsigmaSpacevalues.AlargervalueofthesigmaColorparametermeansthatfarthercolorswithinthepixelneighborhoodwillbemixedtogether,generatinglargerareasofsemi-equalcolors,whereasalargervalueofthesigmaSpaceparametermeansthatfartherpixelswillinfluenceeachotheraslongastheircolorsarecloseenough.voidblur(InputArraysrc,OutputArraydst,Sizeksize,Point

anchor=Point(-1,-1),intborderType=BORDER_DEFAULT):Thisblursanimageusingthenormalizedboxfilter.ItisequivalenttousingboxFilterwithnormalize=true.Thekernelusedinthisfunctionis:

NoteThegetGaussianKernelandgetGaborKernelfunctionscanbeusedinOpenCVtogeneratecustomkernels,whichcanthenbepassedontofilter2D.

Inallcases,itisnecessarytoextrapolatethevaluesofthenon-existentpixelsoutsidetheimageboundary.OpenCVenablesthespecificationoftheextrapolationmethodinmostofthefilterfunctions.Thesemethodsare:

BORDER_REPLICATE:Thisrepeatsthelastknownpixelvalue:aaaaaa|abcdefgh|hhhhhhhBORDER_REFLECT:Thisreflectstheimageborder:fedcba|abcdefgh|hgfedcb

TheexamplecodeThefollowingSmoothexampleshowsyouhowtoloadanimageandapplyGaussianandmedianblurringtoitthroughGaussianBlurandmedianBlurfunctions:

#include"opencv2/opencv.hpp"

usingnamespacecv;

intmain(intargc,char**argv)

//Readthesourcefile

Matsrc;

src=imread(argv[1]);

//Applythefilters

Matdst,dst2;

GaussianBlur(src,dst,Size(9,9),0,0);

medianBlur(src,dst2,9);

//Showtheresults

namedWindow("ORIGINAL",WINDOW_AUTOSIZE);

imshow("ORIGINAL",src);

namedWindow("GAUSSIANBLUR",WINDOW_AUTOSIZE);

imshow("GAUSSIANBLUR",dst);

namedWindow("MEDIANBLUR",WINDOW_AUTOSIZE);

imshow("MEDIANBLUR",dst2);

waitKey();

return0;

Thefollowingfigureshowsyoutheoutputofthecode:

OriginalandblurredimagesfromGaussianandMedianblurringtransformations

SharpeningSharpeningfiltersareusedtohighlightbordersandotherfinedetailswithinimages.Theyarebasedonfirst-andsecond-orderderivatives.Thefirstderivativeofanimagecomputesanapproximationoftheimageintensitygradient,whereasthesecondderivativeisdefinedasthedivergenceofthisgradient.Sincedigitalimageprocessingdealswithdiscretequantities(pixelvalues),thediscreteversionsofthefirstandsecondderivativesareusedforsharpening.

First-orderderivativesproducethickerimageedgesandarewidelyusedforedge-extractionpurposes.However,second-orderderivativesareusedforimageenhancementduetotheirbetterresponsetofinedetails.TwopopularoperatorsusedtoobtainderivativesaretheSobelandtheLaplacian.

TheSobeloperatorcomputesthefirstimagederivativeofanimage,I,through:

TheSobelgradientmagnitudecanbeobtainedbycombiningthegradientapproximationsinthetwodirections,asfollows:

Ontheotherhand,thediscreteLaplacianofanimagecanbegivenasaconvolutionwiththefollowingkernel:

ThefunctionsincludedinOpenCVforthispurposeare:

voidSobel(InputArraysrc,OutputArraydst,intddepth,intdx,intdy,

intksize=3,doublescale=1,doubledelta=0,intborderType=

BORDER_DEFAULT):Thiscalculatesthefirst,second,third,ormixed-imagederivativeswiththeSobeloperatorfromanimageinsrc.Theddepthparameterindicatestheoutputimagedepth,thatis,-1tousethesamedepthastheinputimage.Thekernelsizeisindicatedinksizeandthedesiredderivativeordersindxanddy.Ascalefactorforthecomputedderivativevalescanbeestablishedwithscale.Finally,theborder-typetreatmentisindicatedintheborderTypeparameterandadeltavaluecanbeaddedtotheresultsbeforestoringthemindst.voidScharr(InputArraysrc,OutputArraydst,intddepth,intdx,int

dy,doublescale=1,doubledelta=0,intborderType=BORDER_DEFAULT

):Thiscalculatesamoreaccuratederivativeforakernelofsize3x3.Scharr(src,dst,ddepth,dx,dy,scale,delta,borderType)isequivalenttoSobel(src,dst,ddepth,dx,dy,CV_SCHARR,scale,delta,borderType).voidLaplacian(InputArraysrc,OutputArraydst,intddepth,intksize=

1,doublescale=1,doubledelta=0,intborderType=BORDER_DEFAULT):ThiscalculatestheLaplacianofanimage.AlltheparametersareequivalenttotheonesfromtheSobelandScharrfunctionsexceptforksize.Whenksize>1,itcalculatestheLaplacianoftheimageinsrcbyaddingupthesecondxandyderivativescalculatedusingSobel.Whenksize=1,theLaplacianiscalculatedbyfilteringtheimagewitha3x3kernelthatcontains-4forthecenter,0forthecorners,and1fortherestofthecoefficients.

NotegetDerivKernelscanbeusedinOpenCVtogeneratecustomderivativekernels,whichcanthenbepassedontosepFilter2D.

TheexamplecodeThefollowingSharpenexampleshowsyouhowtocomputeSobelandLaplacianderivativesfromanimagethroughSobelandLaplacianfunctions.Theexamplecodeis:

usingnamespacecv;

//Readthesourcefile

Matsrc;

//ApplySobelandLaplacian

Matdst,dst2;

Sobel(src,dst,-1,1,1);

Laplacian(src,dst2,-1);

//Showtheresults

namedWindow("SOBEL",WINDOW_AUTOSIZE);

imshow("SOBEL",dst);

namedWindow("LAPLACIAN",WINDOW_AUTOSIZE);

imshow("LAPLACIAN",dst2);

waitKey();

return0;

ContoursobtainedbySobelandLaplacianderivatives

WorkingwithimagepyramidsOnsomeoccasions,workingwithafixedimagesizeisnotpossible,andwewillneedtheoriginalimageatdifferentresolutions.Forexample,inobject-detectionproblems,examiningthewholeimagetryingtofindtheobjecttakestoomuchtime.Inthiscase,searchingforobjectsbystartingatsmallerresolutionsismoreefficient.Thistypeofimagesetiscalledpyramidormipmapduetothesimilaritywiththepyramidstructuretypeifimagesareorganizedfromthelargesttothesmallestfromthebottomtothetop.

TheGaussianpyramid

Therearetwokindsofimagepyramids:GaussianpyramidsandLaplacianpyramids.

GaussianpyramidsGaussianpyramidsarecreatedbyalternatelyremovingrowsandcolumnsinthelowerlevelandthenobtainingthevalueofthehigher-levelpixelbyapplyingaGaussianfilterusingtheneighborhoodfromtheunderlyinglevel.Aftereachpyramidstep,theimagereducesitswidthandheightbyhalfanditsareaisaquarterofthepreviouslevelimagearea.InOpenCV,GaussianpyramidscanbecomputedusingthepyrDown,pyrUp,andbuildPyramidfunctions:

voidpyrDown(InputArraysrc,OutputArraydst,constSize&dstsize=

Size(),intborderType=BORDER_DEFAULT):Thissubsamplesandblursansrcimage,savingtheresultindst.ThesizeoftheoutputimageiscomputedasSize((src.cols+1)/2,(src.rows+1)/2)whenitisnotsetwiththedstsizeparameter.voidpyrUp(InputArraysrc,OutputArraydst,constSize&dstsize=

Size(),intborderType=BORDER_DEFAULT):ThiscomputestheoppositeprocessofpyrDown.voidbuildPyramid(InputArraysrc,OutputArrayOfArraysdst,int

maxlevel,intborderType=BORDER_DEFAULT):ThisbuildsaGaussianpyramidforanimagestoredinsrc,obtainingmaxlevelnewimagesandstoringtheminthedstarrayfollowingtheoriginalimagethatisstoredindst[0].Thus,dststoresmaxlevel+1imagesasaresult.

Pyramidsarealsousedforsegmentation.OpenCVprovidesafunctiontocomputemean-shiftpyramidsbasedonthefirststepofthemean-shiftsegmentationalgorithm:

voidpyrMeanShiftFiltering(InputArraysrc,OutputArraydst,doublesp,

doublesr,intmaxLevel=1,TermCriteriatermcrit=TermCriteria

(TermCriteria::MAX_ITER+TermCriteria::EPS,5,1)):Thisimplementsthefilteringstageofthemean-shiftsegmentation,obtaininganimage,dst,withcolorgradientsandfine-graintextureflattened.Thespandsrparametersindicatethespatialwindowandthecolorwindowradii.

NoteMoreinformationaboutthemean-shiftsegmentationcanbefoundathttp://docs.opencv.org/trunk/doc/py_tutorials/py_video/py_meanshift/py_meanshift.html?highlight=meanshift.

LaplacianpyramidsLaplacianpyramidsdonothaveaspecificfunctionimplementationinOpenCV,buttheyareformedfromtheGaussianpyramids.Laplacianpyramidscanbeseenasborderimageswheremostofitselementsarezeros.TheithlevelintheLaplacianpyramidisthedifferencebetweentheithlevelintheGaussianpyramidandtheexpandedversionoftheith+1levelintheGaussianpyramid.

TheexamplecodeThefollowingPyramidsexampleshowsyouhowtoobtaintwolevelsfromaGaussianpyramidthroughthepyrDownfunctionandtheoppositeoperationthroughpyrUp.NoticethattheoriginalimagecannotbeobtainedafterusingpyrUp:

usingnamespacecv;

//Readthesourcefile

Matsrc;

//ApplytwotimespyrDown

Matdst,dst2;

pyrDown(src,dst);

pyrDown(dst,dst2);

//Showtheresults

namedWindow("1stPYRDOWN",WINDOW_AUTOSIZE);

imshow("1stPYRDOWN",dst);

namedWindow("2stPYRDOWN",WINDOW_AUTOSIZE);

imshow("2stPYRDOWN",dst2);

//ApplytwotimespyrUp

pyrUp(dst2,dst);

pyrUp(dst,src);

//Showtheresults

namedWindow("NEWORIGINAL",WINDOW_AUTOSIZE);

imshow("NEWORIGINAL",dst2);

namedWindow("1stPYRUP",WINDOW_AUTOSIZE);

imshow("1stPYRUP",dst);

namedWindow("2stPYRUP",WINDOW_AUTOSIZE);

imshow("2stPYRUP",src);

waitKey();

return0;

TheoriginalandtwolevelsoftheGaussianpyramid

MorphologicaloperationsMorphologicaloperationsprocessimagesaccordingtoshapes.Theyapplyadefined“structuringelement”toanimage,obtaininganewimagewherethepixelsatpositions(xi,yj)arecomputedbycomparingtheinputpixelvaluesatpositions(xi,yj)andtheirneighborhoods.Dependingonthestructuringelementselected,amorphologicaloperationismoresensitivetoonespecificshapeortheother.

Thetwobasicmorphologicaloperationsaredilationanderosion.Dilationaddspixelsfromthebackgroundtotheboundariesoftheobjectsinanimage,whileerosionremovespixels.Hereiswherethestructuringelementistakenintoaccounttoselectthepixelsthataretobeaddedordeleted.Indilation,thevalueoftheoutputpixelisthemaximumofallthepixelsintheneighborhood.Usingerosion,thevalueoftheoutputpixelistheminimumvalueofallthepixelsintheneighborhood.

Anexampleofdilationanderosion

Otherimage-processingoperationscanbedefinedbycombiningdilationanderosion,suchastheopeningandclosingoperations,andthemorphologicalgradient.Theopeningoperationisdefinedaserosion,followedbydilation,whileclosingisitsreverseoperation—dilationfollowedbyerosion.Therefore,openingremovessmallobjectsfromanimagewhilepreservingthelargeronesandclosingisusedtoremovesmallholeswhilepreservingthelargeronesinamannersimilartoopening.Themorphologicalgradientisdefinedasthedifferencebetweenthedilationandtheerosionofanimage.Furthermore,twomoreoperationsaredefinedusingopeningandclosing:top-hatandblack-hatoperations.Theyaredefinedasthedifferencebetweenthesourceimageanditsopeninginthecaseoftophatandthedifferencebetweentheclosingofanimageandthesourceimageinthecaseofblackhat.Alltheoperationsareappliedwiththesamestructuringelement.

InOpenCV,itispossibletoapplydilation,erosion,opening,andclosingthroughthefollowingfunctions:

voiddilate(InputArraysrc,OutputArraydst,InputArraykernel,Point

anchor=Point(-1,-1),intiterations=1,intborderType=

BORDER_CONSTANT,constScalar&borderValue=

morphologyDefaultBorderValue()):Thisdilatesanimagestoredinsrcusinga

specificstructuringelement,savingtheresultindst.Thekernelparameteristhestructuringelementused.Theanchorpointindicatesthepositionofanchorpixel.The(-1,-1)valuemeansthattheanchorisatthecenter.Theoperationcanbeappliedseveraltimesusingiterations.Theborder-typetreatmentisindicatedintheborderTypeparameterandisthesameasinotherfiltersfromprevioussections.Finally,aconstantisindicatedinborderValueiftheBORDER_CONSTANTbordertypeisused.voiderode(InputArraysrc,OutputArraydst,InputArraykernel,Point

anchor=Point(-1,-1),intiterations=1,intborderType=

BORDER_CONSTANT,constScalar&borderValue=

morphologyDefaultBorderValue()):Thiserodesanimageusingaspecificstructuringelement.Itsparametersarethesameasthatindilate.voidmorphologyEx(InputArraysrc,OutputArraydst,intop,InputArray

kernel,Pointanchor=Point(-1,-1),intiterations=1,intborderType

=BORDER_CONSTANT,constScalar&borderValue=

morphologyDefaultBorderValue()):Thisperformsadvancedmorphologicaloperationsdefinedusingtheopparameter.PossibleopvaluesareMORPH_OPEN,MORPH_CLOSE,MORPH_GRADIENT,MORPH_TOPHAT,andMORPH_BLACKHAT.MatgetStructuringElement(intshape,Sizeksize,Pointanchor=

Point(-1,-1)):Thisreturnsastructuringelementofthespecifiedsizeandshapeformorphologicaloperations.SupportedtypesareMORPH_RECT,MORPH_ELLIPSE,andMORPH_CROSS.

TheexamplecodeThefollowingMorphologicalexampleshowsyouhowtosegmentredcheckersinacheckerboard,applyingabinarythreshold(theinRangefunction)andthenrefiningtheresultswithdilationanderosionoperations(throughdilateanderodefunctions).Thestructureusedisacircleof15x15pixels.Theexamplecodeis:

usingnamespacecv;

usingnamespacestd;

//Readthesourcefile

Matsrc;

//Applythefilters

Matdst,dst2,dst3;

inRange(src,Scalar(0,0,100),Scalar(40,30,255),dst);

Matelement=getStructuringElement(MORPH_ELLIPSE,Size(15,15));

dilate(dst,dst2,element);

erode(dst2,dst3,element);

//Showtheresults

namedWindow("SEGMENTED",WINDOW_AUTOSIZE);

imshow("SEGMENTED",dst);

namedWindow("DILATION",WINDOW_AUTOSIZE);

imshow("DILATION",dst2);

namedWindow("EROSION",WINDOW_AUTOSIZE);

imshow("EROSION",dst3);

waitKey();

return0;

Original,redcolorsegmentation,dilation,anderosion

LUTsLook-uptables(LUTs)areverycommonincustomfiltersinwhichtwopixelswiththesamevalueintheinputinvolvesthesamevalueintheoutputtoo.AnLUTtransformationassignsanewpixelvaluetoeachpixelintheinputimageaccordingtothevaluesgivenbyatable.Inthistable,theindexrepresentstheinputintensityvalueandthecontentofthecellgivenbytheindexrepresentsthecorrespondingoutputvalue.Asthetransformationisactuallycomputedforeachpossibleintensityvalue,thisresultsinareductioninthetimeneededtoapplythetransformationoveranimage(imagestypicallyhavemorepixelsthanthenumberofintensityvalues).

TheLUT(InputArraysrc,InputArraylut,OutputArraydst,intinterpolation=0)OpenCVfunctionappliesalook-uptabletransformationoveran8-bitsignedoransrcunsignedimage.Thus,thetablegiveninthelutparametercontains256elements.Thenumberofchannelsinlutiseither1orsrc.channels.Ifsrchasmorethanonechannelbutluthasasingleone,thesamelutchannelisappliedtoalltheimagechannels.

TheexamplecodeThefollowingLUTexampleshowsyouhowtodivide(bytwo)theintensityofthepixelsfromanimageusingalook-uptable.TheLUTneedstobeinitializedbeforeusingitwiththiscode:

uchar*M=(uchar*)malloc(256*sizeof(uchar));

for(inti=0;i<256;i++){

M[i]=i*0.5;//Theresultisroundedtoanintegervalue

Matlut(1,256,CV_8UC1,M);

AMatobjectiscreatedwhereeachcellcontainsthenewvalue.Theexamplecodeis:

usingnamespacecv;

//Readthesourcefile

Matsrc;

//CreatetheLUT

uchar*M=(uchar*)malloc(256*sizeof(uchar));

M[i]=i*0.5;

Matlut(1,256,CV_8UC1,M);

//ApplytheLUT

Matdst;

LUT(src,lut,dst);

//Showtheresults

namedWindow("LUT",WINDOW_AUTOSIZE);

imshow("LUT",dst);

waitKey();

return0;

TheoriginalandLUT-transformedimages

GeometricaltransformationsGeometricaltransformationsdonotchangetheimagecontentbutinsteaddeformtheimagebydeformingtheirgrid.Inthiscase,outputimagepixelvaluesarecomputedbyfirstobtainingthecoordinatesoftheappropriateinputpixelsbyapplyingthecorrespondingmappingfunctionsandcopyingtheoriginalpixelvaluesfromtheobtainedpositionstothenewones:

Thistypeofoperationhastwoproblems:

Extrapolation:fx(x,y)andfy(x,y)couldobtainvaluesthatindicateapixeloutsidetheimageboundary.TheextrapolationmethodsusedingeometricaltransformationsarethesameastheonesusedinimagefilteringplusanotheronecalledBORDER_TRANSPARENT.Interpolation:fx(x,y)andfy(x,y)areusuallyfloating-pointnumbers.InOpenCV,itispossibletoselectbetweennearest-neighborandpolynomialinterpolationmethods.Nearest-neighborinterpolationconsistsofroundingthefloating-pointcoordinatetothenearestinteger.Thesupportedinterpolationmethodsare:

INTER_NEAREST:Thisisthenearest-neighborinterpolationexplainedpreviously.INTER_LINEAR:Thisisabilinearinterpolationmethod.Itisusedbydefault.INTER_AREA:Thisresamplesusingpixelarearelation.INTER_CUBIC:Thisisbicubicinterpolationmethodovera4x4pixelneighborhood.INTER_LANCZOS4:ThisistheLanczosinterpolationmethodoveran8x8pixelneighborhood.

ThegeometricaltransformationssupportedinOpenCVincludeaffine(scaling,translation,rotation,andsoon)andperspectivetransformations.

AffinetransformationAnaffinetransformationisageometrictransformationthatpreservesallpointsfromaninitiallineonalineafterapplyingit.Furthermore,distanceratiosfromeachofthesepointstotheendsofthelinesarealsopreserved.Ontheotherhand,affinetransformationsdon’tnecessarilypreserveanglesandlengths.

Geometrictransformationssuchasscaling,translation,rotation,skewing,andreflectionareallaffinetransformations.

ScalingScalinganimageisresizingitbyshrinkingorzooming.ThefunctioninOpenCVforthispurposeisvoidresize(InputArraysrc,OutputArraydst,Sizedsize,doublefx=0,doublefy=0,intinterpolation=INTER_LINEAR).Apartfromsrcanddst,theinputandoutputimages,ithassomeparameterstospecifythesizetowhichtheimageistoberescaled.Ifthenewimagesizeisspecifiedbysettingdsizetoavaluedifferentfrom0,thescaledfactorparameters,fxandfy,areboth0andfxandfyarecalculatedfromdsizeandtheoriginalsizeoftheinputimage.Iffxandfyaredifferentfrom0anddsizeequals0,dsizeiscalculatedfromtheotherparameters.Ascaleoperationcouldberepresentedbyitstransformationmatrix:

Here,sxandsyarethescalefactorsinthexandyaxis.

Theexamplecode

ThefollowingScaleexampleshowsyouhowtoscaleanimagethroughtheresizefunction.Theexamplecodeis:

usingnamespacecv;

//Readthesourcefile

Matsrc;

//Applythescale

Matdst;

resize(src,dst,Size(0,0),0.5,0.5);

//Showtheresults

namedWindow("SCALED",WINDOW_AUTOSIZE);

imshow("SCALED",dst);

waitKey();

return0;

Originalandscaledimages;fxandfyareboth0.5

TranslationTranslationissimplymovingtheimagealongaspecificdirectionanddistance.Thus,atranslationcouldberepresentedbymeansofavector,(tx,ty),oritstransformationmatrix:

InOpenCV,itispossibletoapplytranslationsusingthevoidwarpAffine(InputArraysrc,OutputArraydst,InputArrayM,Sizedsize,intflags=INTER_LINEAR,

intborderMode=BORDER_CONSTANT,constScalar&borderValue=Scalar())

function.TheMparameteristhetransformationmatrixthatconvertssrcintodst.Theinterpolationmethodisspecifiedusingtheflagsparameter,whichalsosupportstheWARP_INVERSE_MAPvalue,whichmeansthatMistheinversetransformation.TheborderModeparameteristheextrapolationmethod,andborderValueisusedwhenborderModeisBORDER_CONSTANT.

Theexamplecode

TheTranslationexampleshowsyouhowtousethewarpAffinefunctiontotranslateanimage.Theexamplecodeis:

usingnamespacecv;

//Readthesourcefile

Matsrc;

//Applytranslation

Matdst;

MatM=(Mat_<double>(2,3)<<1,0,200,0,1,150);

warpAffine(src,dst,M,src.size());

//Showtheresults

namedWindow("TRANSLATED",WINDOW_AUTOSIZE);

imshow("TRANSLATED",dst);

waitKey();

return0;

Originalanddisplacedimages.Thehorizontaldisplacementis200andverticaldisplacementis150.

ImagerotationImagerotationinvolvesaspecificangle,θ.OpenCVsupportsscaledrotationsinaspecificlocationusingatransformationmatrixdefinedasfollows:

Here,xandyarethecoordinatesoftherotationpointandsfisthescalefactor.

RotationsareappliedliketranslationsbymeansofthewarpAffinefunctionbutusingtheMatgetRotationMatrix2D(Point2fcenter,doubleangle,doublescale)functiontocreatetherotationtransformationmatrix.TheMparameteristhetransformationmatrixthatconvertssrcintodst.Asthenamesoftheparametersindicate,centeristhecenterpointoftherotation,angleistherotationangle(inacounter-clockwisedirection),andscaleisthescalefactor.

Theexamplecode

ThefollowingRotateexampleshowsyouhowtousethewarpAffinefunctiontorotateanimage.A45-degreecenteredrotationmatrixisfirstlyobtainedbygetRotationMatrix2D(Point2f(src.cols/2,src.rows/2),45,1).Theexamplecodeis:

usingnamespacecv;

//Readthesourcefile

Matsrc;

//Applytherotation

Matdst;

MatM=getRotationMatrix2D(Point2f(src.cols/2,src.rows/2),45,1);

//Showtheresults

namedWindow("ROTATED",WINDOW_AUTOSIZE);

imshow("ROTATED",dst);

waitKey();

return0;

Theoriginalimageandtheimageaftera45-degreecenteredrotationisapplied

SkewingAskewingtransformationdisplaceseachpointinafixeddirectionbyanamountproportionaltoitssigneddistancefromalinethatisparalleltothatdirection.Therefore,itwillusuallydistorttheshapeofageometricfigure,forexample,turningsquaresintonon-squareparallelogramsandcirclesintoellipses.However,askewingpreservestheareaofgeometricfigures,thealignment,andrelativedistancesofcollinearpoints.Askewingmappingisthemaindifferencebetweenuprightandslanted(oritalic)stylesofletters.

Skewingcanalsobedefinedbyitsangle,θ.

Theoriginalanditsrotated45degreesfromcenterimage

Usingtheskewingangle,thetransformationmatricesforhorizontalandverticalskewing

Duetothesimilaritieswithprevioustransformations,thefunctionusedtoapplyskewingiswarpAffine.

TipOnmostoccasions,itwillbenecessarytoaddsomesizetotheoutputimageand/orapplytranslation(changingthelastcolumnonthesheartransformationmatrix)inordertodisplaytheoutputimagecompletelyandinacenteredmanner.

Theexamplecode

ThefollowingSkewexampleshowsyouhowtousethewarpAffinefunctiontoskewθ=π/3horizontallyinanimage.Theexamplecodeis:

#include<math.h>

usingnamespacecv;

//Readthesourcefile

Matsrc;

//Applyskew

Matdst;

doublem=1/tan(M_PI/3);

MatM=(Mat_<double>(2,3)<<1,m,0,0,1,0);

warpAffine(src,dst,M,Size(src.cols+0.5*src.cols,src.rows));

//Showtheresults

namedWindow("SKEWED",WINDOW_AUTOSIZE);

imshow("SKEWED",dst);

waitKey();

return0;

Theoriginalimageandtheimagewhenskewedhorizontally

ReflectionAsreflectionisdoneoverthexandyaxesbydefault,itisnecessarytoapplytranslation(thelastcolumnofthetransformationmatrix).Then,thereflectionmatrixis:

Here,txisthenumberofimagecolumnsandtyisthenumberofimagerows.

Aswithprevioustransformations,thefunctionusedtoapplyreflectioniswarpAffine.

NoteOtheraffinetransformationscanbeappliedusingthewarpAffinefunctionwiththeircorrespondingtransformationmatrices.

Theexamplecode

ThefollowingReflectexampleshowsyouanexampleofhorizontal,vertical,andcombinedreflectionofanimageusingthewarpAffinefunction.Theexamplecodeis:

usingnamespacecv;

//Readthesourcefile

Matsrc;

//Applythereflections

Matdsth,dstv,dst;

MatMh=(Mat_<double>(2,3)<<-1,0,src.cols,0,1,0

MatMv=(Mat_<double>(2,3)<<1,0,0,0,-1,src.rows);

MatM=(Mat_<double>(2,3)<<-1,0,src.cols,0,-1,src.rows);

warpAffine(src,dsth,Mh,src.size());

warpAffine(src,dstv,Mv,src.size());

//Showtheresults

namedWindow("H-REFLECTION",WINDOW_AUTOSIZE);

imshow("H-REFLECTION",dsth);

namedWindow("V-REFLECTION",WINDOW_AUTOSIZE);

imshow("V-REFLECTION",dstv);

namedWindow("REFLECTION",WINDOW_AUTOSIZE);

imshow("REFLECTION",dst);

waitKey();

return0;

Thefollowingfigureshowsyoutheoutputforthecode:

TheoriginalandrotatedimagesinX,Y,andbothaxes

PerspectivetransformationForperspectivetransformation,a3x3transformationmatrixisneeded,althoughtheworkisperformedovertwo-dimensionalimages.Straightlinesremainstraightintheoutputimage,butinthiscase,proportionschange.Findingthetransformationmatrixismorecomplexthanwithaffinetransformations.Whenworkingwithperspective,thecoordinatesoffourpointsoftheinputimagematrixandtheircorrespondingcoordinatesontheoutputimagematrixareusedtoperformthisoperation.

WiththesepointsandthegetPerspectiveTransformOpenCVfunction,itispossibletofindtheperspectivetransformationmatrix.Afterobtainingthematrix,warpPerspectiveisappliedtoobtaintheoutputoftheperspectivetransformation.Thetwofunctionsareexplainedindetailhere:

MatgetPerspectiveTransform(InputArraysrc,InputArraydst)andMatgetPerspectiveTransform(constPoint2fsrc[],constPoint2fdst[]):Thisreturnstheperspectivetransformationmatrixcalculatedfromsrcanddst.voidwarpPerspective(InputArraysrc,OutputArraydst,InputArrayM,

Sizedsize,intflags=INTER_LINEAR,intborderMode=BORDER_CONSTANT,

constScalar&borderValue=Scalar()):ThisappliesanMaffinetransformationtoansrcimage,obtainingthenewdstimage.Therestoftheparametersarethesameasinothergeometricaltransformationsdiscussed.

TheexamplecodeThefollowingPerspectiveexampleshowsyouanexampleofhowtochangetheperspectiveofanimageusingthewarpPerspectivefunction.Inthiscase,itisnecessarytoindicatethecoordinatesoffourpointsfromthefirstimageandanotherfourfromtheoutputtocalculatetheperspectivetransformationmatrixthroughgetPerspectiveTransform.Theselectedpointsare:

Point2fsrc_verts[4];

src_verts[2]=Point(195,140);

Point2fdst_verts[4];

dst_verts[2]=Point(160,100);

Theexamplecodeis:

usingnamespacecv;

//Readthesourcefile

Matsrc;

Matdst;

Point2fsrc_verts[4];

Point2fdst_verts[4];

//ObtainandApplytheperspectivetransformation

MatM=getPerspectiveTransform(src_verts,dst_verts);

warpPerspective(src,dst,M,src.size());

//Showtheresults

namedWindow("PERSPECTIVE",WINDOW_AUTOSIZE);

imshow("PERSPECTIVE",dst);

waitKey();

return0;

Perspectiveresultswiththepointsmarkedintheoriginalimage

InpaintingInpaintingistheprocessofreconstructingdamagedpartsofimagesandvideos.Thisprocessisalsoknownasimageorvideointerpolation.Thebasicideaistosimulatetheprocessdonebyrestorerswithantiques.Nowadays,withthewideuseofdigitalcameras,inpaintinghasbecomeanautomaticprocessthatisusednotonlyforimagerestorationbydeletingscratches,butalsoforothertasks,suchasobjectortextremoval.

OpenCVsupportsaninpaintingalgorithmasofVersion2.4.Thefunctionforthispurposeis:

voidinpaint(InputArraysrc,InputArrayinpaintMask,OutputArraydst,

doubleinpaintRadius,intflags):Thisrestorestheareasindicatedwithnon-zerovaluesbytheinpaintMaskparameterinthesource(src)image.TheinpaintRadiusparameterindicatestheneighborhoodtobeusedbythealgorithmspecifiedbyflags.TwomethodscouldbeusedinOpenCV:

INPAINT_NS:ThisistheNavier-Stokes-basedmethodINPAINT_TELEA:ThisisthemethodproposedbyAlexandruTelea

Finally,therestoredimageisstoredindst.

NoteMoredetailsabouttheinpaintingalgorithmsusedinOpenCVcanbefoundathttp://www.ifp.illinois.edu/~yuhuang/inpainting.html

TipForvideoinpainting,considerthevideoasasequenceofimagesandapplythealgorithmoverallofthem.

TheexamplecodeThefollowinginpaintingexampleshowsyouhowtousetheinpaintfunctiontoinpainttheareasofanimagespecifiedinanimagemask.

Theexamplecodeis:

usingnamespacecv;

//Readthesourcefile

Matsrc;

//Readthemaskfile

Matmask;

mask=imread(argv[2]);

cvtColor(mask,mask,COLOR_RGB2GRAY);

//Applytheinpaintingalgorithms

Matdst,dst2;

inpaint(src,mask,dst,10,INPAINT_TELEA);

inpaint(src,mask,dst2,10,INPAINT_NS);

//Showtheresults

namedWindow("MASK",WINDOW_AUTOSIZE);

imshow("MASK",mask);

namedWindow("INPAINT_TELEA",WINDOW_AUTOSIZE);

imshow("INPAINT_TELEA",dst);

namedWindow("INPAINT_NS",WINDOW_AUTOSIZE);

imshow("INPAINT_NS",dst2);

waitKey();

return0;

Resultsofapplyinginpainting

NoteThefirstrowcontainstheoriginalimageandthemaskused.ThesecondrowcontainstheresultsfromtheinpaintingproposedbyTeleaontheleft-handsideandtheNavier-Stokes-basedmethodontheright-handside.

Gettingtheinpaintingmaskisnotaneasytask.Theinpainting2examplecodeshowsyouanexampleofhowwecanobtainthemaskfromthesourceimageusingbinarythresholdingthroughthreshold(mask,mask,235,255,THRESH_BINARY):

usingnamespacecv;

//Readthesourcefile

Matsrc;

//Createthemask

Matmask;

cvtColor(src,mask,COLOR_RGB2GRAY);

threshold(mask,mask,235,255,THRESH_BINARY);

//Applytheinpaintingalgorithms

Matdst,dst2;

inpaint(src,mask,dst,10,INPAINT_TELEA);

inpaint(src,mask,dst2,10,INPAINT_NS);

//Showtheresults

namedWindow("MASK",WINDOW_AUTOSIZE);

imshow("MASK",mask);

namedWindow("INPAINT_TELEA",WINDOW_AUTOSIZE);

imshow("INPAINT_TELEA",dst);

namedWindow("INPAINT_NS",WINDOW_AUTOSIZE);

imshow("INPAINT_NS",dst2);

waitKey();

return0;

Resultsofapplyinginpaintingalgorithmswithoutknowingthemask

NoteThefirstrowcontainstheoriginalimageandtheextractedmask.ThesecondrowcontainstheresultsfromtheinpaintingproposedbyTeleaontheleft-handsideandtheNavier-Stokes-basedmethodontheright-handside.

Theresultsfromthisexampleshowyouthatobtainingaperfectmaskisnotalwayspossible.Someotherpartsoftheimage,suchasthebackgroundornoise,areincludedsometimes.However,theinpaintingresultsremainacceptableastheresultingimagesareclosetotheonesobtainedintheothercase.

DenoisingDenoisingornoisereductionistheprocessofremovingnoisefromsignalsobtainedfromanalogordigitaldevices.Thissectionfocusesitsattentiononreducingnoisefromdigitalimagesandvideos.

Althoughsmoothingandmedianfilteringaregoodoptionstodenoiseanimage,OpenCVprovidesotheralgorithmstoperformthistask.ThesearethenonlocalmeansandtheTVL1(TotalVariationL1)algorithms.Thebasicideaofthenonlocalmeansalgorithmistoreplacethecolorofapixelwithanaverageofthecolorsfromseveralimagesub-windowsthataresimilartotheonethatcomprisesthepixelneighborhood.Ontheotherhand,theTVL1variationaldenoisingmodel,whichisimplementedwiththeprimal-dualoptimizationalgorithm,considerstheimage-denoisingprocessavariationalproblem.

NoteMoreinformationaboutthenonlocalmeansandtheTVL1denoisingalgorithmscanbefoundathttp://www.ipol.im/pub/art/2011/bcm_nlmandhttp://znah.net/rof-and-tv-l1-denoising-with-primal-dual-algorithm.html,respectively.

OpenCVprovidesfourfunctionstodenoisecolorandgrayscaleimagesfollowingthenonlocalmeansapproach.FortheTVL1model,onefunctionisprovided.Thesefunctionsare:

voidfastNlMeansDenoising(InputArraysrc,OutputArraydst,floath=3,

inttemplateWindowSize=7,intsearchWindowSize=21):Thisdenoisesasinglegrayscaleimageloadedinsrc.ThetemplateWindowSizeandsearchWindowSizeparametersarethesizesinpixelsofthetemplatepatchthatisusedtocomputeweightsandthewindowthatisusedtocomputetheweightedaverageforthegivenpixel.Theseshouldbeoddandtheirrecommendedvaluesare7and21pixels,respectively.Thehparameterregulatestheeffectofthealgorithm.Largerhvaluesremovemorenoisedefectsbutwiththedrawbackofremovingmoreimagedetails.Theoutputisstoredindst.voidfastNlMeansDenoisingColored(InputArraysrc,OutputArraydst,float

h=3,floathForColorComponents=3,inttemplateWindowSize=7,int

searchWindowSize=21):Thisisamodificationofthepreviousfunctionforcoloredimages.ItconvertsthesrcimagetotheCIELABcolorspaceandthenseparatelydenoisestheLandABcomponentswiththefastNlMeansDenoisingfunction.voidfastNlMeansDenoisingMulti(InputArrayOfArrayssrcImgs,OutputArray

dst,intimgToDenoiseIndex,inttemporalWindowSize,floath=3,int

templateWindowSize=7,intsearchWindowSize=21):Thisusesanimagesequencetoobtainadenoisedimage.Twomoreparametersareneededinthiscase:imgToDenoiseIndexandtemporalWindowSize.ThevalueofimgToDenoiseIndexisthetargetimageindexinsrcImgstobedenoised.Finally,temporalWindowSizeisusedtoestablishthenumberofsurroundingimagestobeusedfordenoising.Thisshouldbeodd.

voidfastNlMeansDenoisingColoredMulti(InputArrayOfArrayssrcImgs,

OutputArraydst,intimgToDenoiseIndex,inttemporalWindowSize,floath

=3,floathForColorComponents=3,inttemplateWindowSize=7,int

searchWindowSize=21):ThisisbasedonthefastNlMeansDenoisingColoredandfastNlMeansDenoisingMultifunctions.Theparametersareexplainedintherestofthefunctions.voiddenoise_TVL1(conststd::vector<Mat>&observations,Mat&result,

doublelambda,intniters):Thisobtainsadenoisedimageinresultfromoneormorenoisyimagesstoredinobservations.Thelambdaandnitersparameterscontrolthestrengthandthenumberofiterationsofthealgorithm.

TheexamplecodeThefollowingdenoisingexampleshowsyouhowtouseoneofthedenoisingfunctionsfornoisereductionoveracoloredimage(fastNlMeansDenoisingColored).Astheexampleusesanimagewithoutnoise,somethingneedstobeadded.Forthispurpose,thefollowinglinesofcodeareused:

Matnoisy=src.clone();

Matnoise(src.size(),src.type());

randn(noise,0,50);

noisy+=noise;

AMatelementiscreatedwiththesamesizeandtypeoftheoriginalimagetostorenoisegeneratedbytherandnfunctiononit.Finally,thenoiseisaddedtotheclonedimagetoobtainthenoisyimage.

Theexamplecodeis:

usingnamespacecv;

//Readthesourcefile

Matsrc;

//Addsomenoise

Matnoisy=src.clone();

Matnoise(src.size(),src.type());

randn(noise,0,50);

noisy+=noise;

//Applythedenoisingalgorithm

Matdst;

fastNlMeansDenoisingColored(noisy,dst,30,30,7,21);

//Showtheresults

namedWindow("ORIGINALWITHNOISE",WINDOW_AUTOSIZE);

imshow("ORIGINALWITHNOISE",noisy);

namedWindow("DENOISED",WINDOW_AUTOSIZE);

imshow("DENOISED",dst);

waitKey();

return0;

Thefollowingfigureshowsyounoisyanddenoisedimagesfromexecutingthepreviouscode:

Resultsfromapplyingdenoising

SummaryInthischapter,weexplainedmethodsforimageenhancementandcorrection,includingnoisereduction,edgeenhancement,morphologicaloperations,geometricaltransformations,andtherestorationofdamagedimages.DifferentoptionshavebeenpresentedineachcasetoprovidethereaderwithalltheoptionsthatcanbeusedinOpenCV.

Thenextchapterwillcovercolorspacesandhowtoconvertthem.Inaddition,color-space-basedsegmentationandcolor-transfermethodswillbeexplained.

Chapter4.ProcessingColorColorisaperceptualresultcreatedinresponsetotheexcitationofourvisualsystembylightincidentupontheretinainthevisibleregionofthespectrum.Thecolorofanimagemaycontainagreatdealofinformation,whichcanbeusedforsimplifyingimageanalysis,objectidentification,andextractionbasedoncolor.Theseproceduresareusuallycarriedoutconsideringthepixelvaluesinthecolorspaceinwhichitisdefined.Inthischapter,thefollowingtopicswillbecovered:

ThecolorspacesusedinOpenCVandhowtoconvertanimagefromonecolormodeltoanotherAnexampleofhowtosegmentapictureconsideringthecolorspaceinwhichitisdefinedHowtotransfertheappearanceofanimagetoanotherusingthecolortransfermethod

ColorspacesThehumanvisualsystemisabletodistinguishhundredsofthousandsofcolors.Toobtainthisinformation,thehumanretinahasthreetypesofcolorphotoreceptorconecells,whichrespondtoincidentradiation.Becauseofthis,mosthumancolorperceptionscanbegeneratedwiththreenumericalcomponentscalledprimaries.

Tospecifyacolorintermsofthreeormoreparticularcharacteristics,thereareanumberofmethodscalledcolorspacesorcolormodels.Selectingbetweenthemtorepresentanimagedependsontheoperationstobeperformed,becausesomearemoreappropriateaccordingtotherequiredapplication.Forexample,insomecolorspacessuchasRGB,thebrightnessaffectsthethreechannels,afactthatcouldbeunfavorableforsomeimage-processingoperations.ThenextsectionexplainscolorspacesusedinOpenCVandhowtoconvertapicturefromonecolormodeltoanother.

Conversionbetweencolorspaces(cvtColor)Therearemorethan150color-spaceconversionmethodsavailableinOpenCV.ThefunctionprovidedbyOpenCVintheimgprocmoduleisvoidcvtColor(InputArraysrc,OutputArraydst,intcode,intdstCn=0).Theargumentsofthisfunctionare:

src:Thisisaninputimage8-bitunsigned,16-bitunsigned(CV_16UC),orsingle-precisionfloating-point.dst:Thisistheoutputimageofthesamesizeanddepthassrc.code:Thisisthecolorspaceconversioncode.ThestructureofthisparameterisCOLOR_SPACEsrc2SPACEdst.SomeexamplevaluesareCOLOR_BGR2GRAYandCOLOR_YCrCb2BGR.dstCn:Thisisthenumberofchannelsinthedestinationimage.Ifthisparameteris0oromitted,thenumberofthechannelsisderivedautomaticallyfromsrcandcode.

Examplesofthisfunctionwillbedescribedintheupcomingsections.

TipThecvtColorfunctioncanonlyconvertfromRGBtoanothercolorspaceorfromanothercolorspacetoRGB,soifthereaderwantstoconvertbetweentwocolorspacesotherthanRGB,afirstconversiontoRGBmustbedone.

VariouscolorspacesinOpenCVarediscussedintheupcomingsections.

RGBRGBisanadditivemodelinwhichanimageconsistsofthreeindependentimageplanesorchannels:red,green,andblue(andoptionally,afourthchannelforthetransparency,sometimescalledalphachannel).Tospecifyaparticularcolor,eachvalueindicatestheamountofeachofthecomponentspresentoneachpixel,withhighervaluescorrespondingtobrighterpixels.Thiscolorspaceiswidelyusedbecauseitcorrespondstothethreephotoreceptorsofthehumaneye.

NoteThedefaultcolorformatinOpenCVisoftenreferredtoasRGBbutitisactuallystoredasBGR(thechannelsarereversed).

Theexamplecode

ThefollowingBGRsplitexampleshowsyouhowtoloadanRGBimage,splittingandshowingeachparticularchannelingrayandinacolor.Thefirstpartofthecodeisusedtoloadandshowthepicture:

#include<opencv2/imgproc.hpp>

usingnamespacestd;

usingnamespacecv;

vector<Mat>showSeparatedChannels(vector<Mat>channels);

intmain(intargc,constchar**argv)

//Loadtheimage

Matimage=imread("BGR.png");

imshow("Picture",image);

Thenextpartofthecodesplitsthepictureintoeachchannelandshowsit:

vector<Mat>channels;

split(image,channels);

//showchannelsingrayscale

namedWindow("Bluechannel(gray)",WINDOW_AUTOSIZE);

imshow("Bluechannel(gray)",channels[0]);

namedWindow("Greenchannel(gray)",WINDOW_AUTOSIZE);

imshow("Greenchannel(gray)",channels[1]);

namedWindow("Redchannel(gray)",WINDOW_AUTOSIZE);

imshow("Redchannel(gray)",channels[2]);

//showchannelsinBGR

vector<Mat>separatedChannels=showSeparatedChannels(channels);

namedWindow("Bluechannel",WINDOW_AUTOSIZE);

imshow("Bluechannel",separatedChannels[0]);

namedWindow("Greenchannel",WINDOW_AUTOSIZE);

imshow("Greenchannel",separatedChannels[1]);

namedWindow("Redchannel",WINDOW_AUTOSIZE);

imshow("Redchannel",separatedChannels[2]);

waitKey(0);

return0;

Itisworthnotingtheuseofthevoidsplit(InputArraym,OutputArrayOfArraysmv)OpenCVfunctiontosplittheimageminitsthreechannelsandsaveitinavectorofMatcalledmv.Onthecontrary,thevoidmerge(InputArrayOfArraysmv,OutputArraydst)functionisusedtomergeallthemvchannelsinonedstimage.Furthermore,afunctiondenominatedasshowSeparatedChannelsisusedtocreatethreecolorimagesrepresentingeachofthechannels.Foreachchannel,thefunctiongeneratesvector<Mat>auxcomposedbythechannelitselfandtwoauxiliarychannelsorderedwithalltheirvaluessetto0,whichrepresenttheothertwochannelsofthecolormodel.Finally,theauxpictureismerged,generatinganimagewithonlyonechannelfulfilled.Thisfunctioncode,whichwillalsobeusedinotherexamplesofthischapter,isasfollows:

vector<Mat>showSeparatedChannels(vector<Mat>channels){

vector<Mat>separatedChannels;

//createeachimageforeachchannel

Matzer=Mat::zeros(channels[0].rows,channels[0].cols,

channels[0].type());

vector<Mat>aux;

for(intj=0;j<3;j++){

if(j==i)

aux.push_back(channels[i]);

aux.push_back(zer);

Matchann;

merge(aux,chann);

separatedChannels.push_back(chann);

returnseparatedChannels;

Thefollowingfigureshowsyoutheoutputoftheexample:

TheoriginalRGBimageandchannelsplitting

GrayscaleIngrayscale,thevalueofeachpixelisrepresentedasasinglevaluecarryingonlytheintensityinformation,composinganimageexclusivelyformedfromdifferentshadesofgray.ThecolorspaceconversioncodetoconvertbetweenRGBandgrayscale(Y)inOpenCVusingcvtColorisCOLOR_BGR2GRAY,COLOR_RGB2GRAY,COLOR_GRAY2BGR,andCOLOR_GRAY2RGB.Thesetransformationsareinternallycomputedasfollows:

Notefromtheprecedingformulathatitisnotpossibletoretrievecolorsdirectlyfromagrayscaleimage.

Examplecode

ThefollowingGrayexampleshowsyouhowtoconvertanRGBimagetograyscale,showingthetwopictures.Theexamplecodeis:

usingnamespacecv;

//Loadtheimage

Matimage=imread("Lovebird.jpg");

namedWindow("Picture",WINDOW_AUTOSIZE);

MatimageGray;

cvtColor(image,imageGray,COLOR_BGR2GRAY);

namedWindow("Graypicture",WINDOW_AUTOSIZE);

imshow("Graypicture",imageGray);

waitKey(0);

return0;

TheoriginalRGBimageandthegrayscaleconversion

NoteThismethodtoconvertfromRGBtograyscalehasthedisadvantageoflosingthecontrastoftheoriginalimage.Chapter6,ComputationalPhotography,ofthisbookdescribesthedecolorizationprocess,whichmakesthissameconversionwhileovercomingthisissue.

CIEXYZTheCIEXYZsystemdescribescolorwithaluminancecomponentY,whichisrelatedtothebrightnesssensitivityofhumanvisionandtwoadditionalchannels,XandZ,standardizedbytheCommissionInternationaledeL’Éclairage(CIE)usingstatisticsfromexperimentswithseveralhumanobservers.Thiscolorspaceisusedtoreportcolorfrommeasuringinstruments,suchasacolorimeteroraspectrophotometer,anditisusefulwhenaconsistentcolorrepresentationacrossdifferentdevicesisneeded.Themainproblemofthiscolorspaceisthatthecolorsarescaledinanon-uniformmanner.ThisfactcausedtheCIEtoadopttheCIEL*a*b*andCIEL*u*v*colormodels.

ThecolorspaceconversioncodetoconvertbetweenRGBandCIEXYZinOpenCVusingcvtColorisCOLOR_BGR2XYZ,COLOR_RGB2XYZ,COLOR_XYZ2BGR,andCOLOR_XYZ2RGB.Thesetransformationsarecomputedasfollows:

Theexamplecode

ThefollowingCIExyzexampleshowsyouhowtoconvertanRGBimagetotheCIEXYZcolorspace,splittingandshowingeachparticularchannelingrayandinacolor.Thefirstpartofthecodeisusedtoloadandconvertthepicture:

usingnamespacestd;

usingnamespacecv;

//Loadtheimage

//transformtoCIEXYZ

cvtColor(image,image,COLOR_BGR2XYZ);

ThenextpartofthecodesplitsthepictureineachoftheCIEXYZchannelsandshowsthem:

namedWindow("Xchannel(gray)",WINDOW_AUTOSIZE);

imshow("Xchannel(gray)",channels[0]);

namedWindow("Ychannel(gray)",WINDOW_AUTOSIZE);

imshow("Ychannel(gray)",channels[1]);

namedWindow("Zchannel(gray)",WINDOW_AUTOSIZE);

imshow("Zchannel(gray)",channels[2]);

//showchannelsinBGR

cvtColor(separatedChannels[i],separatedChannels[i],COLOR_XYZ2BGR);

namedWindow("Xchannel",WINDOW_AUTOSIZE);

imshow("Xchannel",separatedChannels[0]);

namedWindow("Ychannel",WINDOW_AUTOSIZE);

imshow("Ychannel",separatedChannels[1]);

namedWindow("Zchannel",WINDOW_AUTOSIZE);

imshow("Zchannel",separatedChannels[2]);

waitKey(0);

return0;

TheoriginalRGBimageandtheCIEXYZchannelsplitting

YCrCbThiscolorspaceiswidelyusedinvideo-andimage-compressionschemes,anditisnotanabsolutecolorspacebecauseitisawaytoencodetheRGBcolorspace.TheYchannelrepresentsluminance,whileCrandCbrepresentred-difference(thedifferencebetweentheRchannelintheRGBcolorspaceandY)andblue-difference(thedifferencebetweentheBchannelintheRGBcolorspaceandY)chromacomponents,respectively.Itisusedwidelyinvideo-andimage-compressionschemes,suchasMPEGandJPEG.

ThecolorspaceconversioncodetoconvertbetweenRGBandYCrCbinOpenCVusingcvtColorisCOLOR_BGR2YCrCb,COLOR_RGB2YCrCb,COLOR_YCrCb2BGR,andCOLOR_YCrCb2RGB.Thesetransformationsarecomputedasfollows:

Then,takealookatthefollowing:

Theexamplecode

ThefollowingYCrCbcolorexampleshowsyouhowtoconvertanRGBimagetotheYCrCbcolorspace,splittingandshowingeachparticularchannelingrayandinacolor.Thefirstpartofthecodeisusedtoloadandconvertthepicture:

usingnamespacestd;

usingnamespacecv;

//Loadtheimage

//transformtoYCrCb

cvtColor(image,image,COLOR_BGR2YCrCb);

ThenextpartofthecodesplitsthepictureintoeachoftheYCrCbchannelsandshowsthem:

namedWindow("Ychannel(gray)",WINDOW_AUTOSIZE);

imshow("Ychannel(gray)",channels[0]);

namedWindow("Crchannel(gray)",WINDOW_AUTOSIZE);

imshow("Crchannel(gray)",channels[1]);

namedWindow("Cbchannel(gray)",WINDOW_AUTOSIZE);

imshow("Cbchannel(gray)",channels[2]);

//showchannelsinBGR

cvtColor(separatedChannels[i],separatedChannels[i],COLOR_YCrCb2BGR);

namedWindow("Ychannel",WINDOW_AUTOSIZE);

imshow("Ychannel",separatedChannels[0]);

namedWindow("Crchannel",WINDOW_AUTOSIZE);

imshow("Crchannel",separatedChannels[1]);

namedWindow("Cbchannel",WINDOW_AUTOSIZE);

imshow("Cbchannel",separatedChannels[2]);

waitKey(0);

return0;

TheoriginalRGBimageandtheYCrCbchannelsplitting

HSVTheHSVcolorspacebelongstothegroupoftheso-calledhue-orientedcolor-coordinatesystems.Thistypeofcolormodelcloselyemulatesmodelsofhumancolorperception.Whileinothercolormodels,suchasRGB,animageistreatedasanadditiveresultofthreebasecolors,thethreechannelsofHSVrepresenthue(Hgivesameasureofthespectralcompositionofacolor),saturation(Sgivestheproportionofpurelightofthedominantwavelength,whichindicateshowfaracolorisfromagrayofequalbrightness),andvalue(Vgivesthebrightnessrelativetothebrightnessofasimilarlyilluminatedwhitecolor)correspondingtotheintuitiveappealoftint,shade,andtone.HSViswidelyusedtomakeacomparisonofcolorsbecauseHisalmostindependentlightvariations.Thefollowingfigureshowsyouthiscolormodelrepresentingeachofthechannelsasapartofacylinder:

ThecolorspaceconversioncodetoconvertbetweenRGBandHSVinOpenCVusingcvtColorisCOLOR_BGR2HSV,COLOR_RGB2HSV,COLOR_HSV2BGR,andCOLOR_HSV2RGB.Inthiscase,itisworthnotingthatifthesrcimageformatis8-bitor16-bit,cvtColorfirstconvertsittoafloating-pointformat,scalingthevaluesbetween0and1.Afterthat,thetransformationsarecomputedasfollows:

IfH<0,thenH=H+360.Finally,thevaluesarereconvertedtothedestinationdatatype.

Theexamplecode

ThefollowingHSVcolorexampleshowsyouhowtoconvertanRGBimagetotheHSVcolorspace,splittingandshowingeachparticularchannelingrayscaleandtheHSVimage.Theexamplecodeis:

usingnamespacestd;

usingnamespacecv;

//Loadtheimage

//transformtoHSV

cvtColor(image,image,COLOR_BGR2HSV);

namedWindow("Hchannel(gray)",WINDOW_AUTOSIZE);

imshow("Hchannel(gray)",channels[0]);

namedWindow("Schannel(gray)",WINDOW_AUTOSIZE);

imshow("Schannel(gray)",channels[1]);

namedWindow("Vchannel(gray)",WINDOW_AUTOSIZE);

imshow("Vchannel(gray)",channels[2]);

namedWindow("HSVimage(allchannels)",WINDOW_AUTOSIZE);

imshow("HSVimage(allchannels)",image);

waitKey(0);

return0;

TheoriginalRGBimage,HSVconversion,andchannelsplitting

NoteTheimshowfunctionofOpenCVassumesthatthecoloroftheimagetobeshownisRGB,

soitdisplaysitincorrectly.Ifyouhaveanimageinanothercolorspaceandyouwanttodisplayitcorrectly,youfirsthavetoconvertitbacktoRGB.

HLSTheHLScolorspacebelongstothegroupofhue-orientedcolor-coordinatesystems,suchastheHSVcolormodelexplainedpreviously.Thismodelwasdevelopedtospecifythevaluesofhue,lightness,andsaturationofacolorineachchannel.ThedifferencewithrespecttotheHSVcolormodelisthatthelightnessofapurecolordefinedbyHLSisequaltothelightnessofamediumgray,whilethebrightnessofapurecolordefinedbyHSVisequaltothebrightnessofwhite.

ThecolorspaceconversioncodetoconvertbetweenRGBandHLSinOpenCVusingcvtColorisCOLOR_BGR2HLS,COLOR_RGB2HLS,COLOR_HLS2BGR,andCOLOR_HLS2RGB.Inthiscase,aswithHSV,ifthesrcimageformatis8-bitor16-bit,cvtColorfirstconvertsittoafloating-pointformat,scalingthevaluesbetween0and1.Afterthat,thetransformationsarecomputedasfollows:

IfH<0,thenH=H+360.Finally,thevaluesarereconvertedtothedestinationdatatype.

Theexamplecode

ThefollowingHLScolorexampleshowsyouhowtoconvertanRGBimagetoHLScolor

space,splittingandshowingeachparticularchannelingrayscaleandtheHLSimage.Theexamplecodeis:

usingnamespacestd;

usingnamespacecv;

//Loadtheimage

//transformtoHSV

cvtColor(image,image,COLOR_BGR2HLS);

namedWindow("Hchannel(gray)",WINDOW_AUTOSIZE);

imshow("Hchannel(gray)",channels[0]);

namedWindow("Lchannel(gray)",WINDOW_AUTOSIZE);

imshow("Lchannel(gray)",channels[1]);

namedWindow("Schannel(gray)",WINDOW_AUTOSIZE);

imshow("Schannel(gray)",channels[2]);

namedWindow("HLSimage(allchannels)",WINDOW_AUTOSIZE);

imshow("HLSimage(allchannels)",image);

waitKey(0);

return0;

TheoriginalRGBimage,HLSconversion,andchannelsplitting

CIEL*a*b*TheCIEL*a*b*colorspaceistheseconduniformcolorspacestandardizedbyCIEafterCIEL*u*v*,whichisderivedbasedontheCIEXYZspaceandwhitereferencepoint.Actually,itisthemostcompletecolorspacespecifiedbyCIEandwascreatedtobedevice-independent,liketheCYEXYZmodel,andtobeusedasareference.Itisabletodescribethecolorsvisibletothehumaneye.Thethreechannelsrepresentthelightnessofthecolor(L*),itspositionbetweenmagentaandgreen(a*),anditspositionbetweenyellowandblue(b*).

ThecolorspaceconversioncodetoconvertbetweenRGBandCIEL*a*b*inOpenCVusingcvtColorisCOLOR_BGR2Lab,COLOR_RGB2Lab,COLOR_Lab2BGR,andCOLOR_Lab2RGB.Theprocedureusedtocomputethesetransformationsisexplainedathttp://docs-hoffmann.de/cielab03022003.pdf.

Theexamplecode

ThefollowingCIElabexampleshowsyouhowtoconvertanRGBimagetotheCIEL*a*b*colorspace,splittingandshowingeachparticularchannelingrayscaleandtheCIEL*a*b*image.Theexamplecodeis:

usingnamespacestd;

usingnamespacecv;

//Loadtheimage

//transformtoCIELab

cvtColor(image,image,COLOR_BGR2Lab);

namedWindow("achannel(gray)",WINDOW_AUTOSIZE);

imshow("achannel(gray)",channels[1]);

namedWindow("bchannel(gray)",WINDOW_AUTOSIZE);

imshow("bchannel(gray)",channels[2]);

namedWindow("CIELabimage(allchannels)",WINDOW_AUTOSIZE);

imshow("CIELabimage(allchannels)",image);

waitKey(0);

return0;

TheoriginalRGBimage,CIEL*a*b*conversion,andchannelsplitting

CIEL*u*v*TheCIEL*u*v*colorspaceisthefirstuniformcolorspacestandardizedbyCIE.Itisasimple-to-computetransformationoftheCIEXYZspaceandwhitereferencepoint,whichattemptsperceptualuniformity.LiketheCIEL*a*b*colorspace,itwascreatedtobedevice-independent.Thethreechannelsrepresentthelightnessofthecolor(L*)anditspositionbetweengreenandred(u*),andthelastonerepresentsmostlyblueandpurpletypecolors(v*).Thiscolormodelisusefulforadditivemixturesoflightsduetoitslinearadditionproperties.

ThecolorspaceconversioncodetoconvertbetweenRGBandCIEL*u*v*inOpenCVusingcvtColorisCOLOR_BGR2Luv,COLOR_RGB2Luv,COLOR_Luv2BGR,andCOLOR_Luv2RGB.Theprocedureusedtocomputethesetransformationscanbeseenathttp://docs.opencv.org/trunk/modules/imgproc/doc/miscellaneous_transformations.html#cvtcolor

Theexamplecode

ThefollowingCIELuvcolorexampleshowsyouhowtoconvertanRGBimagetotheCIEL*u*v*colorspace,splittingandshowingeachparticularchannelingrayscaleandtheCIEL*u*v*image.Theexamplecodeis:

usingnamespacestd;

usingnamespacecv;

//Loadtheimage

//transformtoCIELuv

cvtColor(image,image,COLOR_BGR2Luv);

namedWindow("uchannel(gray)",WINDOW_AUTOSIZE);

imshow("uchannel(gray)",channels[1]);

namedWindow("vchannel(gray)",WINDOW_AUTOSIZE);

imshow("vchannel(gray)",channels[2]);

namedWindow("CIELuvimage(allchannels)",WINDOW_AUTOSIZE);

imshow("CIELuvimage(allchannels)",image);

waitKey(0);

return0;

OriginalRGBimage,CIEL*u*v*conversion,andchannelsplitting

BayerTheBayerpixel-spacecompositioniswidelyusedindigitalcameraswithonlyoneimagesensor.Unlikecameraswiththreesensors(oneperRGBchannel,whichisabletoobtainalltheinformationofaparticularcomponent),inonesensorcamera,everypixeliscoveredbyadifferentcolorfilter,soeachpixelisonlymeasuredinthiscolor.ThemissingcolorinformationisextrapolatedfromitsneighborsusingtheBayermethod.Itallowsyoutogetcompletecolorpicturesfromasingleplanewherethepixelsareinterleavedasfollows:

ABayerpatternexample

NoteNotethattheBayerpatternisrepresentedbymoreGpixelsthanRandBbecausethehumaneyeismoresensitivetogreenfrequencies.

Thereareseveralmodificationsoftheshownpatternobtainedbyshiftingthepatternbyonepixelinanydirection.ThecolorspaceconversioncodetoconvertfromBayertoRGBinOpenCVisdefinedconsideringthecomponentsofthesecondandthirdcolumnsofthesecondrow(XandY,respectively)asCOLOR_BayerXY2BGR.Forexample,thepatternofthepreviouspicturehasa“BG”type,soitsconversioncodeisCOLOR_BayerBG2BGR.

Theexamplecode

ThefollowingBayerexampleshowsyouhowtoconvertapicturedefinedbyanRGBayerpatternobtainedfromanimagesensortoanRGBimage.Theexamplecodeis:

usingnamespacecv;

//Showbayeredimageincolor

Matbayer_color=imread("Lovebird_bayer_color.jpg");

namedWindow("Bayerpictureincolor",WINDOW_AUTOSIZE);

imshow("Bayerpictureincolor",bayer_color);

//Loadbayeredimage

Matbayer=imread("Lovebird_bayer.jpg",CV_8UC3);

namedWindow("Bayerpicture",WINDOW_AUTOSIZE);

imshow("Bayerpicture",bayer);

MatimageColor;

cvtColor(bayer,imageColor,COLOR_BayerRG2BGR);

namedWindow("Colorpicture",WINDOW_AUTOSIZE);

imshow("Colorpicture",imageColor);

waitKey(0);

return0;

TheBayerpatternimageandRGBconversion

Color-space-basedsegmentationEachcolorspacerepresentsanimageindicatingthenumericvalueofthespecificcharacteristicmeasuredbyeachchanneloneachpixel.Consideringthesecharacteristics,itispossibletopartitionthecolorspaceusinglinearboundaries(forexample,planesinthree-dimensionalspacesandonespaceperchannel),allowingyoutoclassifyeachpixelaccordingtothepartitionitliesin,thereforeallowingyoutoselectasetofpixelswithpredefinedcharacteristics.Thisideacanbeusedtosegmentobjectsofanimageweareinterestedin.

OpenCVprovidesthevoidinRange(InputArraysrc,InputArraylowerb,InputArrayupperb,OutputArraydst)functiontocheckwhetheranarrayofelementsliebetweentheelementsoftwootherarrays.Withrespecttocolor-space-basedsegmentation,thisfunctionallowsyoutoobtainthesetofpixelsofansrcimage,thevaluesofwhosechannelsliebetweenthelowerblowerboundariesandupperbupperboundaries,obtainingthedstimage.

NoteThelowerbandupperbboundariesareusuallydefinedasScalar(x,y,z),wherex,y,andzarethenumericalvaluesofeachchanneldefinedaslowerorupperboundaries.

Thefollowingexamplesshowyouhowtodetectpixelsthatcanbeconsideredtobeskin.Ithasbeenobservedthatskincolordiffersmoreinintensitythanchrominance,sonormally,theluminancecomponentisnotconsideredforskindetection.ThisfactmakesitdifficulttodetectskininapicturerepresentedinRGBbecauseofthedependenceofthiscolorspaceonluminance,soHSVandYCrCbcolormodelsareused.Itisworthnotingthatforthistypeofsegmentation,itisnecessarytoknoworobtainthevaluesoftheboundariesperchannel.

HSVsegmentationAsstatedpreviously,HSViswidelyusedtomakeacomparisonofcolorsbecauseHisalmostindependentoflightvariations,soitisusefulinskindetection.Inthisexample,thelowerboundaries(0,10,60)andtheupperboundaries(20,150,255)areselectedtodetecttheskinineachpixel.Theexamplecodeis:

usingnamespacestd;

usingnamespacecv;

intmain()

//Loadtheimage

Matimage=imread("hand.jpg");

Mathsv;

cvtColor(image,hsv,COLOR_BGR2HSV);

//selectpixels

Matbw;

inRange(hsv,Scalar(0,10,60),Scalar(20,150,255),bw);

namedWindow("Selectedpixels",WINDOW_AUTOSIZE);

imshow("Selectedpixels",bw);

waitKey(0);

return0;

SkindetectionusingtheHSVcolorspace

YCrCbsegmentationTheYCrCbcolorspacereducestheredundancyofRGBcolorchannelsandrepresentsthecolorwithindependentcomponents.Consideringthattheluminanceandchrominancecomponentsareseparated,thisspaceisagoodchoiceforskindetection.

ThefollowingexampleusestheYCrCbcolorspaceforskindetectionusingthelowerboundaries(0,133,77)andtheupperboundaries(255,173,177)ineachpixel.Theexamplecodeis:

usingnamespacestd;

usingnamespacecv;

intmain()

//Loadtheimage

Matimage=imread("hand.jpg");

Matycrcb;

cvtColor(image,ycrcb,COLOR_BGR2HSV);

//selectpixels

Matbw;

inRange(ycrcb,Scalar(0,133,77),Scalar(255,173,177),bw);

namedWindow("Selectedpixels",WINDOW_AUTOSIZE);

imshow("Selectedpixels",bw);

waitKey(0);

return0;

SkindetectionusingtheYCrCbcolorspace

NoteFormoreimage-segmentationmethods,refertoChapter4ofOpenCVEssentialsbyPacktPublishing.

ColortransferAnothertaskcommonlycarriedoutinimageprocessingistomodifythecolorofanimage,specificallyincaseswhereitisnecessarytoremoveadominantorundesirablecolorcast.Oneofthesemethodsiscalledcolortransfer,whichcarriesoutasetofcolorcorrectionsthatborrowonesourceimage’scolorcharacteristics,andtransfertheappearanceofthesourceimagetothetargetimage.

TheexamplecodeThefollowingcolorTransferexampleshowsyouhowtotransferthecolorfromasourcetotargetimage.ThismethodfirstconvertstheimagecolorspacetoCIEL*a*b*.Next,itsplitsthechannelsforsourceandtargetimages.Afterthat,itfitsthechanneldistributionfromoneimagetoanotherusingthemeanandthestandarddeviation.Finally,thechannelsaremergedbacktogetherandconvertedtoRGB.

NoteForfulltheoreticaldetailsofthetransformationusedintheexample,refertoColorTransferbetweenImagesathttp://www.cs.tau.ac.il/~turkel/imagepapers/ColorTransfer.pdf.

ThefirstpartofthecodeconvertstheimagetotheCIEL*a*b*colorspace,whilealsochangingthetypeoftheimagetoCV_32FC1:

usingnamespacestd;

usingnamespacecv;

//Loadtheimages

Matsrc=imread("clock_tower.jpg");

Mattar=imread("big_ben.jpg");

//ConverttoLabspaceandCV_32F1

Matsrc_lab,tar_lab;

cvtColor(src,src_lab,COLOR_BGR2Lab);

cvtColor(tar,tar_lab,COLOR_BGR2Lab);

src_lab.convertTo(src_lab,CV_32FC1);

tar_lab.convertTo(tar_lab,CV_32FC1);

Thenextpartofthecodeperformsthecolortransferasstatedpreviously:

//Findmeanandstdofeachchannelforeachimage

Matmean_src,mean_tar,stdd_src,stdd_tar;

meanStdDev(src_lab,mean_src,stdd_src);

meanStdDev(tar_lab,mean_tar,stdd_src);

//Splitintoindividualchannels

vector<Mat>src_chan,tar_chan;

split(src_lab,src_chan);

split(tar_lab,tar_chan);

//Foreachchannelcalculatethecolordistribution

tar_chan[i]-=mean_tar.at<double>(i);

tar_chan[i]*=(stdd_src.at<double>(i)/stdd_src.at<double>(i));

tar_chan[i]+=mean_src.at<double>(i);

//Mergethechannels,converttoCV_8UC1eachchannelandconvertto

Matoutput;

merge(tar_chan,output);

output.convertTo(output,CV_8UC1);

cvtColor(output,output,COLOR_Lab2BGR);

//showpictures

namedWindow("Sourceimage",WINDOW_AUTOSIZE);

namedWindow("Targetimage",WINDOW_AUTOSIZE);

imshow("Targetimage",tar);

namedWindow("Resultimage",WINDOW_AUTOSIZE);

imshow("Resultimage",output);

waitKey(0);

return0;

Anightappearancecolor-transferexample

SummaryInthischapter,weprovidedadeeperviewofthecolorspacesusedinOpenCVandshowedyouhowtoconvertbetweenthemusingthecvtColorfunction.Furthermore,thepossibilitiesofimageprocessingusingdifferentcolormodelsandtheimportanceofselectingthecorrectcolorspaceconsideringtheoperationsweneedtomakewashighlighted.Tothisend,color-space-basedsegmentationandcolor-transfermethodswereimplemented.

Thenextchapterwillcoverimage-processingtechniquesusedforvideoorasequenceofimages.Wewillseehowtoimplementvideostabilization,superresolution,andstitchingalgorithmswithOpenCV.

Chapter5.ImageProcessingforVideoThischaptershowsyoudifferenttechniquesrelatedtoimageprocessingforvideo.Whilemostclassicalimageprocessingdealswithstaticimages,video-basedprocessingisbecomingpopularandaffordable.

Thischaptercoversthefollowingtopics:

VideostabilizationThevideosuperresolutionprocessImagestitching

Inthischapter,wewillworkwithavideosequenceoralivecameradirectly.Theoutputofimageprocessingmaybeeitherasetofmodifiedimagesorusefulhigh-levelinformation.Mostimage-processingtechniquesconsiderimagesasatwo-dimensionaldigitalsignalandapplydifferenttechniquestoit.Inthischapter,asequenceofimagesfromavideoorlivecamerawillbeusedtomakeorimproveanewenhancedsequencewithdifferenthigh-leveltechniques.Thus,moreusefulinformationisobtained,thatis,athirdtimedimensionisincorporated.

VideostabilizationVideostabilizationreferstoafamilyofmethodsusedtoreducetheblurringassociatedwiththemotionofthecamera.Inotherwords,itcompensatesforanyangularmovement,equivalenttoyaw,pitch,roll,andxandytranslationsofthecamera.Thefirstimagestabilizersappearedintheearly60s.Thesesystemswereabletoslightlycompensateforcamerashakesandinvoluntarymovements.Theywerecontrolledbygyroscopesandaccelerometersbasedonmechanismsthatcouldcancelorreduceunwantedmovementbychangingthepositionofalens.Currently,thesemethodsarewidelyusedinbinoculars,videocameras,andtelescopes.

Therearevariousmethodsforimageorvideostabilization,andthischapterfocusesonthemostextendedfamiliesofmethods:

Mechanicalstabilizationsystems:Thesesystemsuseamechanicalsystemonthecameralenssothatwhenthecameraismoved,motionisdetectedbyaccelerometersandgyroscopes,andthesystemgeneratesamovementonthelens.Thesesystemswillnotbeconsideredhere.Digitalstabilizationsystems:Thesearenormallyusedinvideoandtheyactdirectlyontheimageobtainedfromthecamera.Inthesesystems,thesurfaceofthestabilizedimageisslightlysmallerthanthesourceimage’ssurface.Whenthecameraismoved,thecapturedimageismovedtocompensatethismovement.Althoughthesetechniqueseffectivelyworktocancelmovementbyreducingtheusableareaofmovementsensor,resolutionandimageclarityaresacrificed.

Video-stabilizationalgorithmsusuallyencompassthefollowingsteps:

GeneralstepsofVideo-Stabilizationalgorithms

ThischapterfocusesonthevideostabmoduleinOpenCV3.0Alpha,whichcontainsasetoffunctionsandclassesthatcanbeusedtosolvethevideo-stabilizationproblem.

Let’sexplorethegeneralprocessinmoredetail.Videostabilizationisachievedbyafirstestimationoftheinter-framemotionbetweenconsecutiveframesusingtheRANSACmethod.Attheendofthisstep,anarrayof3x3matricesisobtained,andeachofthemdescribesthemotionofthetwopairsofconsecutiveframes.Globalmotionestimationisveryimportantforthisstepanditaffectstheaccuracyofthestabilizedfinalsequence.

YoucanfindmoredetailedinformationabouttheRANSACmethodathttp://en.wikipedia.org/wiki/RANSAC.

Thesecondstepgeneratesanewsequenceofframesbasedontheestimatedmotion.Additionalprocessingisperformed,suchassmoothing,deblurring,borderextrapolation,andsoon,toimprovethequalityofstabilization.

Thethirdstepremovestheannoyingirregularperturbations—refertothefollowingfigure.Thereareapproachesthatassumeacamera-motionmodel,whichworkwellwhensomeassumptionscanbemadeabouttheactualcameramotion.

Removingtheirregularperturbations

IntheOpenCVexamples([opencv_source_code]/samples/cpp/videostab.cpp),avideo-stabilizationprogramexamplecanbefound.ForthefollowingvideoStabilizerexample,thevideoStabilizer.proprojectneedstheselibraries:lopencv_core300,lopencv_highgui300,lopencv_features2d300,lopencv_videoio300,andlopencv_videostab300.

ThefollowingvideoStabilizerexamplehasbeencreatedusingthevideostabmoduleofOpenCV3.0Alpha:

#include<string>

#include<iostream>

#include<opencv2/videostab.hpp>

usingnamespacestd;

usingnamespacecv;

usingnamespacecv::videostab;

voidprocessing(Ptr<IFrameSource>stabilizedFrames,stringoutputPath);

Ptr<IFrameSource>stabilizedFrames;

//1-Preparetheinputvideoandcheckit

stringinputPath;

stringoutputPath;

if(argc>1)

inputPath=argv[1];

inputPath=".\\cube4.avi";

if(argc>2)

outputPath=argv[2];

outputPath=".\\cube4_stabilized.avi";

Ptr<VideoFileSource>source=makePtr<VideoFileSource>(inputPath);

cout<<"framecount(rough):"<<source->count()<<endl;

//2-Preparethemotionestimator

//first,preparethemotiontheestimationbuilder,RANSACL2

doublemin_inlier_ratio=0.1;

Ptr<MotionEstimatorRansacL2>est=makePtr<MotionEstimatorRansacL2>

(MM_AFFINE);

RansacParamsransac=est->ransacParams();

ransac.size=3;

ransac.thresh=5;

ransac.eps=0.5;

est->setRansacParams(ransac);

est->setMinInlierRatio(min_inlier_ratio);

//second,createafeaturedetector

intnkps=1000;

Ptr<GoodFeaturesToTrackDetector>feature_detector=

makePtr<GoodFeaturesToTrackDetector>(nkps);

//third,createthemotionestimator

Ptr<KeypointBasedMotionEstimator>motionEstBuilder=

makePtr<KeypointBasedMotionEstimator>(est);

motionEstBuilder->setDetector(feature_detector);

Ptr<IOutlierRejector>outlierRejector=makePtr<NullOutlierRejector>();

motionEstBuilder->setOutlierRejector(outlierRejector);

//3-Preparethestabilizer

StabilizerBase*stabilizer=0;

//first,preparetheoneortwopassstabilizer

boolisTwoPass=1;

intradius_pass=15;

if(isTwoPass)

//withatwopassstabilizer

boolest_trim=true;

TwoPassStabilizer*twoPassStabilizer=newTwoPassStabilizer();

twoPassStabilizer->setEstimateTrimRatio(est_trim);

twoPassStabilizer-

>setMotionStabilizer(makePtr<GaussianMotionFilter>(radius_pass));

stabilizer=twoPassStabilizer;

//withanonepassstabilizer

OnePassStabilizer*onePassStabilizer=newOnePassStabilizer();

onePassStabilizer-

>setMotionFilter(makePtr<GaussianMotionFilter>(radius_pass));

stabilizer=onePassStabilizer;

//second,setuptheparameters

intradius=15;

doubletrim_ratio=0.1;

boolincl_constr=false;

stabilizer->setFrameSource(source);

stabilizer->setMotionEstimator(motionEstBuilder);

stabilizer->setRadius(radius);

stabilizer->setTrimRatio(trim_ratio);

stabilizer->setCorrectionForInclusion(incl_constr);

stabilizer->setBorderMode(BORDER_REPLICATE);

//caststabilizertosimpleframesourceinterfacetoread

stabilizedframes

stabilizedFrames.reset(dynamic_cast<IFrameSource*>(stabilizer));

//4-Processingthestabilizedframes.Theresultsareshowedandsaved.

processing(stabilizedFrames,outputPath);

catch(constexception&e)

cout<<"error:"<<e.what()<<endl;

stabilizedFrames.release();

return-1;

stabilizedFrames.release();

return0;

voidprocessing(Ptr<IFrameSource>stabilizedFrames,stringoutputPath)

VideoWriterwriter;

MatstabilizedFrame;

intnframes=0;

doubleoutputFps=25;

//foreachstabilizedframe

while(!(stabilizedFrame=stabilizedFrames->nextFrame()).empty())

nframes++;

//initwriter(once)andsavestabilizedframe

if(!outputPath.empty())

if(!writer.isOpened())

writer.open(outputPath,VideoWriter::fourcc('X','V','I','D'),

outputFps,stabilizedFrame.size());

writer<<stabilizedFrame;

imshow("stabilizedFrame",stabilizedFrame);

charkey=static_cast<char>(waitKey(3));

if(key==27){cout<<endl;break;}

cout<<"processedframes:"<<nframes<<endl;

cout<<"finished"<<endl;

Thisexampleacceptsthenameofaninputvideofileasadefaultvideofilename(.\cube4.avi).Theresultingvideowillbedisplayedandthensavedas.\cube4_stabilized.avi.Notehowthevideostab.hppheaderisincludedandthecv::videostabnamespaceisused.Theexampletakesfourimportantsteps.Thefirststeppreparestheinputvideopath,andthisexampleusesthestandardcommand-lineinputarguments(inputPath=argv[1])toselectthevideofile.Ifitdoesnothaveaninputvideofile,thenitusesthedefaultvideofile(.\cube4.avi).

Thesecondstepbuildsamotionestimator.ArobustRANSAC-basedglobal2Dmethodiscreatedforthemotionestimatorusingasmartpointer(Ptr<object>)ofOpenCV(Ptr<MotionEstimatorRansacL2>est=makePtr<MotionEstimatorRansacL2>(MM_AFFINE)).Therearedifferentmotionmodelstostabilizethevideo:

MM_TRANSLATION=0MM_TRANSLATION_AND_SCALE=1MM_ROTATIO=2MM_RIGID=3MM_SIMILARITY=4MM_AFFINE=5MM_HOMOGRAPHY=6MM_UNKNOWN=7

Thereisatrade-offbetweenaccuracytostabilizethevideoandcomputationaltime.Themorebasicmotionmodelshaveworseaccuracyandbettercomputationaltime;however,themorecomplexmodelshavebetteraccuracyandworsecomputationaltime.

TheRANSACobjectisnowcreated(RansacParamsransac=est->ransacParams())andtheirparametersareset(ransac.size,ransac.threshandransac.eps).Afeaturedetectorisalsoneededtoestimatethemovementbetweeneachconsecutiveframethatwillbeusedbythestabilizer.ThisexampleusestheGoodFeaturesToTrackDetectormethodtodetect(nkps=1000)salientfeaturesineachframe.Then,itusestherobustRANSACandfeature-detectormethodstocreatethemotionestimatorusingthePtr<KeypointBasedMotionEstimator>motionEstBuilder=

makePtr<KeypointBasedMotionEstimator>(est)classandsettingthefeaturedetector

withmotionEstBuilder->setDetector(feature_detector).

RANSACparameters

Size Thesubsetsize

Thresh Themaximumerrortoclassifyasinliers

Eps Themaximumoutliersratio

Prob Theprobabilityofsuccess

Thethirdstepcreatesastabilizerthatneedsthepreviousmotionestimator.Youcanselect(isTwoPass=1)aone-ortwo-passstabilizer.Ifyouusethetwo-passstabilizer(TwoPassStabilizer*twoPassStabilizer=newTwoPassStabilizer()),theresultsareusuallybetter.However,inthisexample,thisiscomputationallyslower.Ifyouusetheotheroption,one-passstabilizer(OnePassStabilizer*onePassStabilizer=newOnePassStabilizer()),theresultsareworsebuttheresponseisfaster.Thestabilizerneedstosetotheroptionstoworkcorrectly,suchasthesourcevideofile(stabilizer->setFrameSource(source))andthemotionestimator(stabilizer->setMotionEstimator(motionEstBuilder)).Italsoneedstocastthestabilizertosimpleframesourcevideotoreadstabilizedframes(stabilizedFrames.reset(dynamic_cast<IFrameSource*>(stabilizer))).

Thelaststepstabilizesthevideousingthecreatedstabilizer.Theprocessing(Ptr<IFrameSource>stabilizedFrames)functioniscreatedtoprocessandstabilizeeachframe.Thisfunctionneedstointroduceapathtosavetheresultingvideo(stringoutputPath=".//stabilizedVideo.avi")andsettheplaybackspeed(doubleoutputFps=25).Afterwards,thisfunctioncalculateseachstabilizedframeuntiltherearenomoreframes(stabilizedFrame=stabilizedFrames->nextFrame().empty()).Internally,thestabilizerfirstestimatesthemotionofeveryframe.Thisfunctioncreatesavideowriter(writer.open(outputPath,VideoWriter::fourcc('X','V','I','D'),outputFps,stabilizedFrame.size()))tostoreeachframeintheXVIDformat.Finally,itsavesandshowseachstabilizedframeuntiltheuserpressestheEsckey.

TodemonstratehowtostabilizeavideowithOpenCV,thepreviousvideoStabilizerexampleisused.Theexampleisexecutedfromthecommandlineasfollows:

<bin_dir>\videoStabilizer.exe.\cube4.avi.\cube4_stabilized.avi

NoteThiscube4.avivideocanbefoundintheOpenCVsamplesfolder.Italsohasagreatdealofcameramovement,whichisperfectforthisexample.

Toshowthestabilizationresults,first,seefourframesofcube4.aviinthefollowingfigure.Thefigurefollowingtheseframesshowsthefirst10framesofcube4.aviandcube4_stabilized.avisuperimposedwithout(left-handsideofthefigure)andwith(right-handsideofthefigure)stabilization.

Thefourconsecutiveframesofcube4.avivideothatarecameramovements

10superimposedframesofcube4.aviandcube4_stabilizatedvideoswithoutandwithstabilization

Lookingattheprecedingfigure,ontheright-handside,youcanseethatthevibrationsproducedbythecameramovementhavebeenreducedduetothestabilization.

SuperresolutionSuperresolutionreferstothetechniquesoralgorithmsdesignedtoincreasethespatialresolutionofanimageorvideo,usuallyfromasequenceofimagesoflowerresolution.Itdiffersfromthetraditionaltechniquesofimagescaling,whichuseasingleimagetoincreasetheresolution,keepingthesharpedges.Incontrast,superresolutionmergesinformationfrommultipleimagestakenfromthesamescenetorepresentdetailsthatwerenotinitiallycapturedintheoriginalimages.

Theprocessofcapturinganimageorvideofromareal-lifescenerequiresthefollowingsteps:

Sampling:ThisisthetransformationofthecontinuoussystemfromthesceneofanidealdiscretesystemwithoutaliasingGeometrictransformation:Thisreferstoapplyingasetoftransformations,suchastranslationorrotation,duetothecamerapositionandlenssystemtoinfer,ideally,detailsofthescenethatarriveateachsensorBlur:ThishappensduetothelenssystemortheexistingmotioninthesceneduringtheintegrationtimeSubsampling:Withthis,thesensoronlyintegratesthenumberofpixelsatitsdisposal(photosites)

Youcanseethisprocessofimagecapturinginthefollowingfigure:

Theprocessofcapturinganimagefromarealscene

Duringthiscaptureprocess,thedetailsofthesceneareintegratedbydifferentsensorssothateachpixelineachcaptureincludesdifferentinformation.Therefore,superresolutionisbasedontryingtofindtherelationshipbetweendifferentcapturesthathaveobtaineddifferentdetailsofthesceneinordertocreateanewimagewithmoreinformation.Superresolutionis,therefore,usedtoregenerateadiscretizedscenewithahigherresolution.

Superresolutioncanbeobtainedbyvarioustechniques,rangingfromthemostintuitivein

thespatialdomaintotechniquesbasedonanalyzingthefrequencyspectrum.Techniquesarebasicallydividedintooptical(usinglenses,zoom,andsoon)orimage-processing-basedtechniques.Thischapterfocusesonimage-processing-basedsuperresolution.Thesemethodsuseotherpartsofthelower-resolutionimages,orotherunrelatedimages,toinferwhatthehigh-resolutionimageshouldlooklike.Thesealgorithmscanbealsodividedintothefrequencyorspatialdomain.Originally,superresolutionmethodsonlyworkedwellongrayscaleimages,butnewmethodshavebeendevelopedtoadaptthemtocolorimages.

Ingeneral,superresolutioniscomputationallydemanding,bothspatiallyandtemporally,becausethesizeoflow-resolutionandhigh-resolutionimagesishighandhundredsofsecondsmaybeneededtogenerateanimage.Totrytoreducethecomputationaltime,preconditionersareusedcurrentlyforoptimizersthatareresponsibleforminimizingthesefunctions.AnotheralternativeistouseGPUprocessingtoimprovethecomputationaltimebecausethesuperresolutionprocessisinherentlyparallelizable.

ThischapterfocusesonthesuperresmoduleinOpenCV3.0Alpha,whichcontainsasetoffunctionsandclassesthatcanbeusedtosolvetheproblemofresolutionenhancement.Themoduleimplementsanumberofmethodsbasedonimage-processingsuperresolution.ThischapterspecificallyfocusesontheBilateralTV-L1(BTVL1)superresolutionmethodimplemented.Amajordifficultyofthesuperresolutionprocessistoestimatethewarpingfunctiontobuildthesuperresolutionimage.TheBilateralTV-L1usesopticalflowtoestimatethewarpingfunction.

NoteYoucanfindmoredetailedinformationabouttheBilateralTV-Lmethodathttp://www.ipol.im/pub/art/2013/26/andopticalflowathttp://en.wikipedia.org/wiki/Optical_flow.

IntheOpenCVexamples([opencv_source_code]/samples/gpu/super_resolution.cpp),abasicexampleofsuperresolutioncanbefound.

NoteYoucanalsodownloadthisexamplefromtheOpenCVGitHubrepositoryathttps://github.com/Itseez/opencv/blob/master/samples/gpu/super_resolution.cpp.

Forthefollowingsuperresolutionexampleproject,thesuperresolution.proprojectfileshallincludetheselibraries:lopencv_core300,lopencv_imgproc300,lopencv_highgui300,lopencv_features2d300,lopencv_videoio300,andlopencv_superres300toworkcorrectly:

#include<iostream>

#include<iomanip>

#include<string>

#include<opencv2/core.hpp>

#include<opencv2/core/utility.hpp>

#include<opencv2/highgui.hpp>

#include<opencv2/superres.hpp>

#include<opencv2/superres/optical_flow.hpp>

#include<opencv2/opencv_modules.hpp>

usingnamespacestd;

usingnamespacecv;

usingnamespacecv::superres;

staticPtr<DenseOpticalFlowExt>createOptFlow(stringname);

//1-Initializetheinitialparameters

//Inputandoutputvideo

stringinputVideoName;

stringoutputVideoName;

if(argc>1)

inputVideoName=argv[1];

inputVideoName=".\\tree.avi";

if(argc>2)

outputVideoName=argv[2];

outputVideoName=".\\tree_superresolution.avi";

constintscale=4;//Scalefactor

constintiterations=180;//Iterationscount

constinttemporalAreaRadius=4;//Radiusofthetemporalsearcharea

stringoptFlow="farneback";//Opticalflowalgorithm

//optFlow="farneback";

//optFlow="tvl1";

//optFlow="brox";

//optFlow="pyrlk";

doubleoutputFps=25.0;//Playbackspeedoutput

//2-Createanopticalflowmethod

Ptr<DenseOpticalFlowExt>optical_flow=createOptFlow(optFlow);

if(optical_flow.empty())return-1;

//3-Createthesuperresolutionmethodandsetitsparameters

Ptr<SuperResolution>superRes;

superRes=createSuperResolution_BTVL1();

superRes->set("opticalFlow",optical_flow);

superRes->set("scale",scale);

superRes->set("iterations",iterations);

superRes->set("temporalAreaRadius",temporalAreaRadius);

Ptr<FrameSource>frameSource;

frameSource=createFrameSource_Video(inputVideoName);

superRes->setInput(frameSource);

//Notusethefirstframe

Matframe;

frameSource->nextFrame(frame);

//4-Processingtheinputvideowiththesuperresolution

//Showtheinitialoptions

cout<<"Input:"<<inputVideoName<<""<<

frame.size()<<endl;

cout<<"Output:"<<outputVideoName<<endl;

cout<<"Playbackspeedoutput:"<<outputFps<<endl;

cout<<"Scalefactor:"<<scale<<endl;

cout<<"Iterations:"<<iterations<<endl;

cout<<"Temporalradius:"<<temporalAreaRadius<<endl;

cout<<"OpticalFlow:"<<optFlow<<endl;

cout<<endl;

VideoWriterwriter;

doublestart_time,finish_time;

for(inti=0;;++i)

cout<<'['<<setw(3)<<i<<"]:";

Matresult;

//Calculatetheprocessingtime

start_time=getTickCount();

superRes->nextFrame(result);

finish_time=getTickCount();

cout<<(finish_time-start_time)/getTickFrequency()<<"secs,

Size:"<<result.size()<<endl;

if(result.empty())break;

//Showtheresult

imshow("SuperResolution",result);

if(waitKey(1000)>0)break;

//Savetheresultonoutputfile

if(!outputVideoName.empty())

if(!writer.isOpened())

writer.open(outputVideoName,VideoWriter::fourcc('X','V',

'I','D'),outputFps,result.size());

writer<<result;

writer.release();

return0;

staticPtr<DenseOpticalFlowExt>createOptFlow(stringname)

if(name=="farneback")

returncreateOptFlow_Farneback();

elseif(name=="tvl1")

returncreateOptFlow_DualTVL1();

elseif(name=="brox")

returncreateOptFlow_Brox_CUDA();

elseif(name=="pyrlk")

returncreateOptFlow_PyrLK_CUDA();

cerr<<"IncorrectOpticalFlowalgorithm-"<<name<<endl;

returnPtr<DenseOpticalFlowExt>();

Thisexamplecreatesaprogram(superresolution)toobtainvideoswithsuperresolution.Ittakesthepathofaninputvideoorusesadefaultvideopath(.\tree.avi).Theresultingvideoisdisplayedandsavedas.\tree_superresolution.avi.Inthefirstplace,thesuperres.hppandsuperres/optical_flow.hppheadersareincludedandthecv::superresnamespaceisused.Theexamplefollowsfourimportantsteps.

Thefirststepsetstheinitialparameters.Itistheinputvideopaththatusesthestandardinput(inputVideoName=argv[1])toselectthevideofile,andifitdoesnothaveaninputvideofile,thenitusesadefaultvideofile.Theoutputvideopathalsousestheinputstandard(outputVideoName=argv[2])toselecttheoutputvideofile,andifitdoesnothaveanoutputvideofile,thenitusesthedefaultoutputvideofile(.\tree_superresolution)andtheoutputplaybackspeedisalsoset(doubleoutputFps=25.0).Otherimportantparametersofthesuperresolutionmethodarethescalefactor(constintscale=4),theiterationcount(constintiterations=100),theradiusofthetemporalsearcharea(constinttemporalAreaRadius=4),andtheoptical-flowalgorithm(stringoptFlow="farneback").

Thesecondstepcreatesanoptical-flowmethodtodetectsalientfeaturesandtrackthemforeachvideoframe.Anewmethod(staticPtr<DenseOpticalFlowExt>createOptFlow(stringname))hasbeencreatedtoselectbetweenthedifferentoptical-flowmethods.Youcanselectbetweenfarneback,tvl1,brox,andpyrlkoptical-flowmethods.Anewmethod(staticPtr<DenseOpticalFlowExt>createOptFlow(stringname))iswrittentocreateanoptical;-flowmethodtotrackfeatures.ThetwomostimportantmethodsareFarneback(createOptFlow_Farneback())andTV-L1(createOptFlow_DualTVL1()).ThefirstmethodisbasedonGunnerFarneback’salgorithmthatcomputestheopticalflowforallpointsintheframe.ThesecondmethodcalculatestheopticalflowbetweentwoimageframesthatarebasedondualformulationontheTVenergyandemploysanefficientpoint-wisethresholdingstep.Thissecondmethodiscomputationallymoreefficient.

Comparisonbetweenthedifferentopticalflowmethods

Method Complexity Parallelizable

Farneback Quadratic No

TV-L1 Lineal Yes

Brox Lineal Yes

PyrLK Lineal No

NoteYoucanalsolearnmoreabouttheFarnebackoptical-flowmethodusedathttp://www.diva-portal.org/smash/get/diva2:273847/FULLTEXT01.pdf.

Thethirdstepcreatesandsetsthesuperresolutionmethod.Aninstanceofthismethodiscreated(Ptr<SuperResolution>superRes),whichusestheBilateralTV-L1algorithm(superRes=createSuperResolution_BTVL1()).Thismethodhasthefollowingparametersforthealgorithm:

scale:Thisisthescalefactoriterations:Thisistheiterationcounttau:Thisisanasymptoticvalueofthesteepestdescentmethodlambda:Thisistheweightparametertobalancethedatatermandsmoothnesstermalpha:ThisisaparameterofspatialdistributioninBilateral-TVbtvKernelSize:ThisisthekernelsizeoftheBilateral-TVfilterblurKernelSize:ThisistheGaussianblurkernelsizeblurSigma:ThisistheGaussianblursigmatemporalAreaRadius:ThisistheradiusofthetemporalsearchareaopticalFlow:Thisisadenseoptical-flowalgorithm

Theseparametersaresetasfollows:

superRes->set("parameter",value);

Onlythefollowingparametersareset;theotherparametersusetheirdefaultvalues:

superRes->set("opticalFlow",optical_flow);

superRes->set("scale",scale);

superRes->set("iterations",iterations);

superRes->set("temporalAreaRadius",temporalAreaRadius);

Afterwards,theinputvideoframeisselected(superRes->setInput(frameSource)).

Thelaststepprocessestheinputvideotocomputethesuperresolution.Foreachvideoframe,thesuperresolutioniscalculated(superRes->nextFrame(result));thiscalculationiscomputationallyveryslow,thustheprocessingtimeisestimatedtoshowprogress.Finally,eachresultframeisshown(imshow("SuperResolution",result))andsaved(writer<<result).

Toshowthesuperresolutionresults,asmallpartofthefirstframeofthetree.aviandtree_superresolution.avivideosarecomparedwithandwithoutsuperresolution:

Partofthefirstframeoftree.aviandtree_superresolution.avivideoswithoutandwiththesuperresolutionprocess

Intheright-hand-sidesectionoftheprecedingfigure,youcanobservemoredetailsintheleavesandbranchesofthetreeduetothesuperresolutionprocess.

StitchingImagestitching,orphotostitching,candiscoverthecorrespondencerelationshipbetweenimageswithsomedegreeofoverlap.Thisprocesscombinesasetofimageswithoverlappingfieldsofviewtoproduceapanoramaorhigher-resolutionimage.Mostofthetechniquesforimagestitchingneednearlyexactoverlapsbetweentheimagestoproduceseamlessresults.Somedigitalcamerascaninternallystitchasetofimagestobuildapanoramaimage.Anexampleisshowninthefollowingfigure:

Apanoramaimagecreatedwithstitching

NoteTheprecedingimageexampleandmoreinformationaboutimagestitchingcanbefoundathttp://en.wikipedia.org/wiki/Image_stitching.

Stitchingcannormallybedividedintothreeimportantsteps:

Registration(images)impliesmatchingfeaturesinasetofimagestosearchforadisplacementthatminimizesthesumofabsolutevaluesinthedifferencesbetweenoverlappingpixels.Direct-alignmentmethodscouldbeusedtogetbetterresults.Theusercouldalsoaddaroughmodelofthepanoramatohelpthefeaturematchingstage,inwhichcase,theresultsaretypicallymoreaccurateandcomputationallyfaster.Calibration(images)focusesonminimizingdifferencesbetweenanidealmodelandthecamera-lenssystem:differentcamerapositionsandopticaldefectssuchasdistortions,exposure,chromaticaberrations,andsoon.Compositing(images)usestheresultsofthepreviousstep,calibration,combinedwiththeremappingoftheimagestoanoutputprojection.Colorsarealsoadjustedbetweenimagestocompensateforexposuredifferences.Imagesareblendedtogetherandseam-lineadjustmentisdonetominimizethevisibilityofseamsbetweenimages.

Whenthereareimagesegmentsthathavebeentakenfromthesamepointinspace,stitchingcanbeperformedusingoneofvariousmapprojections.Themostimportantmapprojectionsareshownasfollows:

Rectilinearprojection:Here,thestitchingimageisviewedonatwo-dimensionalplaneintersectingthepanoramasphereinasinglepoint.Linesthatarestraightin

realityareshownsimilarregardlessoftheirdirectionontheimage.Whentherearewideviews(around120degrees),imagesaredistortedneartheborders.Cylindricalprojection:Here,thestitchingimageshowsa360-degreehorizontalfieldofviewandalimitedverticalfieldofview.Thisprojectionismeanttobeviewedasthoughtheimageiswrappedintoacylinderandviewedfromwithin.Whenviewedona2Dplane,horizontallinesappearcurved,whileverticallinesremainstraight.Sphericalprojection:Here,thestitchingimageshowsa360-degreehorizontalfieldofviewand180-degreeverticalfieldofview,thatis,thewholesphere.Panoramaimageswiththisprojectionaremeanttobeviewedasthoughtheimageiswrappedintoasphereandviewedfromwithin.Whenviewedona2Dplane,horizontallinesappearcurvedasinacylindricalprojection,whileverticallinesremainvertical.Stereographicprojectionorfisheyeprojection:Thiscanbeusedtoformalittleplanetpanoramabypointingthevirtualcamerastraightdownandsettingthefieldofviewlargeenoughtoshowthewholegroundandsomeoftheareasaboveit;pointingthevirtualcameraupwardscreatesatunneleffect.Paniniprojection:Thishasspecializedprojectionsthatmayhavemoreaestheticallypleasingadvantagesovernormalcartographyprojections.Thisprojectioncombinesdifferentprojectionsinthesameimagetofine-tunethefinallookoftheoutputpanoramaimage.

ThischapterfocusesonthestitchingmoduleandthedetailsubmoduleinOpenCV3.0Alpha,whichcontainsasetoffunctionsandclassesthatimplementastitcher.Usingthesemodules,itispossibletoconfigureorskipsomesteps.Theimplementedstitchingexamplehasthefollowinggeneraldiagram:

IntheOpenCVexamples,therearetwobasicexamplesofstitching,whichcanbefoundat([opencv_source_code]/samples/cpp/stitching.cpp])and([opencv_source_code]/samples/cpp/stitching_detailed.cpp]).

Forthefollowing,moreadvancedstitchingAdvancedexample,thestitchingAdvanced.proprojectfilemustincludethefollowinglibrariestoworkcorrectly:lopencv_core300,lopencv_imgproc300,lopencv_highgui300,lopencv_features2d300,lopencv_videoio300,lopencv_imgcodecs300,andlopencv_stitching300:

#include<iostream>

#include<string>

#include<opencv2/opencv_modules.hpp>

#include<opencv2/core/utility.hpp>

#include<opencv2/imgcodecs.hpp>

#include<opencv2/features2d.hpp>

#include<opencv2/stitching/detail/blenders.hpp>

#include<opencv2/stitching/detail/camera.hpp>

#include<opencv2/stitching/detail/exposure_compensate.hpp>

#include<opencv2/stitching/detail/matchers.hpp>

#include<opencv2/stitching/detail/motion_estimators.hpp>

#include<opencv2/stitching/detail/seam_finders.hpp>

#include<opencv2/stitching/detail/util.hpp>

#include<opencv2/stitching/detail/warpers.hpp>

#include<opencv2/stitching/warpers.hpp>

usingnamespacestd;

usingnamespacecv;

usingnamespacecv::detail;

//Defaultparameters

vector<String>img_names;

doublescale=1;

stringfeatures_type="orb";//"surf"or"orb"featurestype

floatmatch_conf=0.3f;

floatconf_thresh=1.f;

stringadjuster_method="ray";//"reproj"or"ray"adjustermethod

booldo_wave_correct=true;

WaveCorrectKindwave_correct_type=WAVE_CORRECT_HORIZ;

stringwarp_type="spherical";

intexpos_comp_type=ExposureCompensator::GAIN_BLOCKS;

stringseam_find_type="gc_color";

floatblend_strength=5;

intblend_type=Blender::MULTI_BAND;

stringresult_name="panorama_result.jpg";

doublestart_time=getTickCount();

//1-Inputimages

if(argc>1)

for(inti=1;i<argc;i++)

img_names.push_back(argv[i]);

img_names.push_back("./panorama_image1.jpg");

img_names.push_back("./panorama_image2.jpg");

//Checkifhaveenoughimages

intnum_images=static_cast<int>(img_names.size());

if(num_images<2){cout<<"Needmoreimages"<<endl;return-1;}

//2-Resizeimagesandfindfeaturessteps

cout<<"Findingfeatures…"<<endl;

doublet=getTickCount();

Ptr<FeaturesFinder>finder;

if(features_type=="surf")

finder=makePtr<SurfFeaturesFinder>();

elseif(features_type=="orb")

finder=makePtr<OrbFeaturesFinder>();

else{cout<<"Unknown2Dfeaturestype:'"<<features_type<<endl;

return-1;}

Matfull_img,img;

vector<ImageFeatures>features(num_images);

vector<Mat>images(num_images);

vector<Size>full_img_sizes(num_images);

for(inti=0;i<num_images;++i)

full_img=imread(img_names[i]);

full_img_sizes[i]=full_img.size();

if(full_img.empty()){cout<<"Can'topenimage"<<img_names[i]

<<endl;return-1;}

resize(full_img,img,Size(),scale,scale);

images[i]=img.clone();

(*finder)(img,features[i]);

features[i].img_idx=i;

cout<<"Featuresinimage#"<<i+1<<"are:"<<

features[i].keypoints.size()<<endl;

finder->collectGarbage();

full_img.release();

img.release();

cout<<"Findingfeatures,time:"<<((getTickCount()-t)/

getTickFrequency())<<"sec"<<endl;

//3-Matchfeatures

cout<<"Pairwisematching"<<endl;

t=getTickCount();

vector<MatchesInfo>pairwise_matches;

BestOf2NearestMatchermatcher(false,match_conf);

matcher(features,pairwise_matches);

matcher.collectGarbage();

cout<<"Pairwisematching,time:"<<((getTickCount()-t)/

//4-Selectimagesandmatchessubsettobuildpanorama

vector<int>indices=leaveBiggestComponent(features,pairwise_matches,

conf_thresh);

vector<Mat>img_subset;

vector<String>img_names_subset;

vector<Size>full_img_sizes_subset;

for(size_ti=0;i<indices.size();++i)

img_names_subset.push_back(img_names[indices[i]]);

img_subset.push_back(images[indices[i]]);

full_img_sizes_subset.push_back(full_img_sizes[indices[i]]);

images=img_subset;

img_names=img_names_subset;

full_img_sizes=full_img_sizes_subset;

//Estimatecameraparametersrough

HomographyBasedEstimatorestimator;

vector<CameraParams>cameras;

if(!estimator(features,pairwise_matches,cameras)){cout<<

"Homographyestimationfailed."<<endl;return-1;}

for(size_ti=0;i<cameras.size();++i)

cameras[i].R.convertTo(R,CV_32F);

cameras[i].R=R;

cout<<"Initialintrinsic#"<<indices[i]+1<<":\n"<<

cameras[i].K()<<endl;

//5-Refinecameraparametersglobally

Ptr<BundleAdjusterBase>adjuster;

if(adjuster_method=="reproj")

//"reproj"method

adjuster=makePtr<BundleAdjusterReproj>();

else//"ray"method

adjuster=makePtr<BundleAdjusterRay>();

adjuster->setConfThresh(conf_thresh);

if(!(*adjuster)(features,pairwise_matches,cameras)){cout<<"Camera

parametersadjustingfailed."<<endl;return-1;}

//Findmedianfocallength

vector<double>focals;

cout<<"Camera#"<<indices[i]+1<<":\n"<<cameras[i].K()<<

focals.push_back(cameras[i].focal);

sort(focals.begin(),focals.end());

floatwarped_image_scale;

if(focals.size()%2==1)

warped_image_scale=static_cast<float>(focals[focals.size()/2]);

warped_image_scale=static_cast<float>(focals[focals.size()/2-

1]+focals[focals.size()/2])*0.5f;

//6-Wavecorrelation(optional)

if(do_wave_correct)

vector<Mat>rmats;

rmats.push_back(cameras[i].R.clone());

waveCorrect(rmats,wave_correct_type);

cameras[i].R=rmats[i];

//7-Warpimages

cout<<"Warpingimages(auxiliary)..."<<endl;

t=getTickCount();

vector<Point>corners(num_images);

vector<UMat>masks_warped(num_images);

vector<UMat>images_warped(num_images);

vector<Size>sizes(num_images);

vector<UMat>masks(num_images);

//Prepareimagesmasks

masks[i].create(images[i].size(),CV_8U);

masks[i].setTo(Scalar::all(255));

//Mapprojections

Ptr<WarperCreator>warper_creator;

if(warp_type=="rectilinear")

warper_creator=makePtr<cv::CompressedRectilinearWarper>(2.0f,1.0f);

elseif(warp_type=="cylindrical")

warper_creator=makePtr<cv::CylindricalWarper>();

elseif(warp_type=="spherical")

warper_creator=makePtr<cv::SphericalWarper>();

elseif(warp_type=="stereographic")

warper_creator=makePtr<cv::StereographicWarper>();

elseif(warp_type=="panini")

warper_creator=makePtr<cv::PaniniWarper>(2.0f,1.0f);

if(!warper_creator){cout<<"Can'tcreatethefollowingwarper'"<<

warp_type<<endl;return1;}

Ptr<RotationWarper>warper=warper_creator->create(static_cast<float>

(warped_image_scale*scale));

Mat_<float>K;

cameras[i].K().convertTo(K,CV_32F);

floatswa=(float)scale;

K(0,0)*=swa;K(0,2)*=swa;

K(1,1)*=swa;K(1,2)*=swa;

corners[i]=warper->warp(images[i],K,cameras[i].R,INTER_LINEAR,

BORDER_REFLECT,images_warped[i]);

sizes[i]=images_warped[i].size();

warper->warp(masks[i],K,cameras[i].R,INTER_NEAREST,BORDER_CONSTANT,

masks_warped[i]);

vector<UMat>images_warped_f(num_images);

images_warped[i].convertTo(images_warped_f[i],CV_32F);

cout<<"Warpingimages,time:"<<((getTickCount()-t)/

//8-Compensateexposureerrors

Ptr<ExposureCompensator>compensator=

ExposureCompensator::createDefault(expos_comp_type);

compensator->feed(corners,images_warped,masks_warped);

//9-Findseammasks

Ptr<SeamFinder>seam_finder;

if(seam_find_type=="no")

seam_finder=makePtr<NoSeamFinder>();

elseif(seam_find_type=="voronoi")

seam_finder=makePtr<VoronoiSeamFinder>();

elseif(seam_find_type=="gc_color")

seam_finder=makePtr<GraphCutSeamFinder>

(GraphCutSeamFinderBase::COST_COLOR);

elseif(seam_find_type=="gc_colorgrad")

seam_finder=makePtr<GraphCutSeamFinder>

(GraphCutSeamFinderBase::COST_COLOR_GRAD);

elseif(seam_find_type=="dp_color")

seam_finder=makePtr<DpSeamFinder>(DpSeamFinder::COLOR);

elseif(seam_find_type=="dp_colorgrad")

seam_finder=makePtr<DpSeamFinder>(DpSeamFinder::COLOR_GRAD);

if(!seam_finder){cout<<"Can'tcreatethefollowingseamfinder'"<<

seam_find_type<<endl;return1;}

seam_finder->find(images_warped_f,corners,masks_warped);

//Releaseunusedmemory

images.clear();

images_warped.clear();

images_warped_f.clear();

masks.clear();

//10-Createablender

Ptr<Blender>blender=Blender::createDefault(blend_type,false);

Sizedst_sz=resultRoi(corners,sizes).size();

floatblend_width=sqrt(static_cast<float>(dst_sz.area()))*

blend_strength/100.f;

if(blend_width<1.f)

blender=Blender::createDefault(Blender::NO,false);

elseif(blend_type==Blender::MULTI_BAND)

MultiBandBlender*mb=dynamic_cast<MultiBandBlender*>(blender.get());

mb->setNumBands(static_cast<int>(ceil(log(blend_width)/log(2.))-

cout<<"Multi-bandblender,numberofbands:"<<mb->numBands()

<<endl;

elseif(blend_type==Blender::FEATHER)

FeatherBlender*fb=dynamic_cast<FeatherBlender*>(blender.get());

fb->setSharpness(1.f/blend_width);

cout<<"Featherblender,sharpness:"<<fb->sharpness()<<endl;

blender->prepare(corners,sizes);

//11-Compositingstep

cout<<"Compositing…"<<endl;

t=getTickCount();

Matimg_warped,img_warped_s;

Matdilated_mask,seam_mask,mask,mask_warped;

for(intimg_idx=0;img_idx<num_images;++img_idx)

cout<<"Compositingimage#"<<indices[img_idx]+1<<endl;

//11.1-Readimageandresizeitifnecessary

full_img=imread(img_names[img_idx]);

if(abs(scale-1)>1e-1)

resize(full_img,img,Size(),scale,scale);

img=full_img;

full_img.release();

Sizeimg_size=img.size();

cameras[img_idx].K().convertTo(K,CV_32F);

//11.2-Warpthecurrentimage

warper->warp(img,K,cameras[img_idx].R,INTER_LINEAR,BORDER_REFLECT,

img_warped);

//Warpthecurrentimagemask

mask.create(img_size,CV_8U);

mask.setTo(Scalar::all(255));

warper->warp(mask,K,cameras[img_idx].R,INTER_NEAREST,

BORDER_CONSTANT,mask_warped);

//11.3-Compensateexposureerrorstep

compensator->apply(img_idx,corners[img_idx],img_warped,mask_warped);

img_warped.convertTo(img_warped_s,CV_16S);

img_warped.release();

img.release();

mask.release();

dilate(masks_warped[img_idx],dilated_mask,Mat());

resize(dilated_mask,seam_mask,mask_warped.size());

mask_warped=seam_mask&mask_warped;

//11.4-Blendingimagesstep

blender->feed(img_warped_s,mask_warped,corners[img_idx]);

Matresult,result_mask;

blender->blend(result,result_mask);

cout<<"Compositing,time:"<<((getTickCount()-t)/

imwrite(result_name,result);

cout<<"Finished,totaltime:"<<((getTickCount()-start_time)/

return0;

ThisexamplecreatesaprogramtostitchimagesusingOpenCVsteps.Ittakesaninputpathtoselectthedifferentinputimagesorusesdefaultinputimages(.\panorama_image1.jpgandpanorama_image2.jpg),whichareshownlater.Finally,theresultingimageisshownandsavedas.\panorama_result.jpg.Inthefirstplace,thestitching.hppanddetailheadersareincludedandthecv::detailnamespaceisused.Themoreimportantparametersarealsoset,andyoucanconfigurethestitchingprocesswiththeseparameters.Ifyouneedtouseacustomconfiguration,itisveryusefultounderstandthegeneraldiagramofthestitchingprocess(thepreviousfigure).Thisadvancedexamplehas11importantsteps.Thefirststepreadsandcheckstheinputimages.Thisexampleneedstwoormoreimagestowork.

Thesecondstepresizestheinputimagesusingthedoublescale=1parameterandfindsthefeaturesoneachimage;youcanselectbetweentheSurf(finder=makePtr<SurfFeaturesFinder>())orOrb(finder=makePtr<OrbFeaturesFinder>())featurefindersusingthestringfeatures_type="orb"parameter.Afterwards,thisstepresizestheinputimages(resize(full_img,img,Size(),scale,scale))andfindsthefeatures((*finder)(img,features[i])).

NoteFormoreinformationaboutSURFandORBdescriptors,refertoChapter5ofOpenCVEssentialsbyPacktPublishing.

Thethirdstepmatchesthefeaturesthathavebeenfoundpreviously.Amatcheriscreated(BestOf2NearestMatchermatcher(false,match_conf))withthefloatmatch_conf=0.3fparameter.

Thefourthstepselectsimagesandmatchessubsetstobuildthepanorama.Then,thebestfeaturesareselectedandmatchedusingthevector<int>indices=leaveBiggestComponent(features,pairwise_matches,conf_thresh)function.Withthesefeatures,anewsubsetiscreatedtobeused.

Thefifthsteprefinesparametersgloballyusingbundleadjustmenttobuildanadjuster(Ptr<BundleAdjusterBase>adjuster).Givenasetofimagesdepictinganumberof2Dor3Dpointsfromdifferentviewpoints,bundleadjustmentcanbedefinedastheproblemofsimultaneouslyrefiningthe2Dor3Dcoordinates,describingthescenegeometryaswellastheparametersoftherelativemotionandopticalcharacteristicsofthecamerasemployedtoacquiretheimagesaccordingtoanoptimalitycriterioninvolvingthecorrespondingimageprojectionsofallpoints.Therearetwomethodstocalculatethisbundleadjustment,reproj(adjuster=makePtr<BundleAdjusterReproj>())orray(adjuster=makePtr<BundleAdjusterRay>()),whichareselectedwiththestringadjuster_method="ray"parameter.Finallythisbundleadjustmentisusedas(*adjuster)(features,pairwise_matches,cameras).

Thesixthstepisanoptionalstep(booldo_wave_correct=true)thatcalculatesthewavecorrelationtoimprovethecamerasetting.ThetypeofwavecorrelationisselectedwiththeWaveCorrectKindwave_correct_type=WAVE_CORRECT_HORIZparameterandcalculatedaswaveCorrect(rmats,wave_correct_type).

Theseventhstepcreatesawarperimagethatneedsamapprojection.Themapprojectionshavebeendescribedpreviously,andtheycanberectilinear,cylindrical,spherical,stereographic,orpanini.ThereareactuallymoremapprojectionsimplementedinOpenCV.Themapprojectionscanbeselectedwiththestringwarp_type="spherical"parameter.Afterwards,awarperiscreated(Ptr<RotationWarper>warper=warper_creator->create(static_cast<float>(warped_image_scale*scale)))andeachimageiswarped(warper->warp(masks[i],K,cameras[i].R,INTER_NEAREST,BORDER_CONSTANT,masks_warped[i])).

Theeighthstepcompensatesexposureerrorsbycreatingacompensator(Ptr<ExposureCompensator>compensator=

ExposureCompensator::createDefault(expos_comp_type))anditisappliedtoeachwarpedimage(compensator->feed(corners,images_warped,masks_warped)).

Theninthstepfindsseammasks.Thisprocesssearchesforthebestareasofattachmentforeachpanoramaimage.TherearesomemethodsimplementedinOpenCVtoperformthistask,andthisexampleusesthestringseam_find_type="gc_color"parametertoselectthem.ThesemethodsareNoSeamFinder(there’snouseofthismethod),VoronoiSeamFinder,GraphCutSeamFinderBase::COST_COLOR,GraphCutSeamFinderBase::COST_COLOR_GRAD,DpSeamFinder::COLOR,andDpSeamFinder::COLOR_GRAD.

Thetenthstepcreatesablendertocombineeachimagetobuildthepanorama.TherearetwotypesofblendersimplementedinOpenCV,MultiBandBlender*mb=dynamic_cast<MultiBandBlender*>(blender.get())andFeatherBlender*fb=dynamic_cast<FeatherBlender*>(blender.get()),whichcanbeselectedwiththeintblend_type=Blender::MULTI_BANDparameter.Finally,theblenderisprepared(blender->prepare(corners,sizes)).

Thelaststepcompositesthefinalpanorama.Thisstepneedswhatthepreviousstepshavedonetoconfigurethestitching.Foursub-stepsareperformedtocalculatethefinalpanorama.First,eachinputimageisread(full_img=imread(img_names[img_idx]))and,ifnecessary,resized(resize(full_img,img,Size(),scale,scale)).Second,theseimagesarewarpedwiththecreatedwarper(warper->warp(img,K,cameras[img_idx].R,INTER_LINEAR,BORDER_REFLECT,img_warped)).Third,theseimagesarecompensatedforexposureerrorswiththecreatedcompensator(compensator->apply(img_idx,corners[img_idx],img_warped,mask_warped)).Finally,theseimagesareblendedusingthecreatedblender.Thefinalresultpanoramaisnowsavedinthestringresult_name="panorama_result.jpg"file.

ToshowyouthestitchingAdvancedresults,twoinputimagesarestitchedandtheresultingpanoramaisshownasfollows:

SummaryInthischapter,youlearnedhowtousethreeimportantmodulesofOpenCVthathandleimageprocessinginvideo.Thesemodulesarevideostabilization,superresolution,andstitching.Someofthetheoreticalunderpinningshavealsobeenexplainedforeachmodule.

Ineachsectionofthischapter,acompleteexample,developedinC++,isexplained.Animageresultwasalsoshownforeachmodule,showingthemaineffect.

Thenextchapterintroduceshigh-dynamic-rangeimagesandshowsyouhowtohandlethemwithOpenCV.High-dynamic-rangeimagingistypicallyconsideredwithinwhatisnowcalledcomputationalphotography.Roughlyspeaking,computationalphotographyreferstotechniquesthatallowyoutoextendthetypicalcapabilitiesofdigitalphotography.Thismayincludehardwareadd-onsormodifications,butitmostlyreferstosoftware-basedtechniques.Thesetechniquesmayproduceoutputimagesthatcannotbeobtainedwitha“traditional”digitalcamera.

Chapter6.ComputationalPhotographyComputationalphotographyreferstotechniquesthatallowyoutoextendthetypicalcapabilitiesofdigitalphotography.Thismayincludehardwareadd-onsormodifications,butitmostlyreferstosoftware-basedtechniques.Thesetechniquesmayproduceoutputimagesthatcannotbeobtainedwitha“traditional”digitalcamera.Thischapterintroducessomeofthelesser-knowntechniquesavailableinOpenCVforcomputationalphotography:high-dynamic-rangeimaging,seamlesscloning,decolorization,andnon-photorealisticrendering.Thesethreeareinsidethephotomoduleofthelibrary.Notethatothertechniquesinsidethismodule(inpaintinganddenoising)havebeenalreadyconsideredinpreviouschapters.

High-dynamic-rangeimagesThetypicalimagesweprocesshave8bitsperpixel(bpp).Colorimagesalsouse8bitstorepresentthevalueofeachchannel,thatis,red,green,andblue.Thismeansthatonly256differentintensityvaluesareused.This8bpplimithasprevailedthroughoutthehistoryofdigitalimaging.However,itisobviousthatlightinnaturedoesnothaveonly256differentlevels.Weshould,therefore,considerwhetherthisdiscretizationisdesirableorevensufficient.Thehumaneye,forexample,isknowntocaptureamuchhigherdynamicrange(thenumberoflightlevelsbetweenthedimmestandbrightestlevels),estimatedatbetween1and100millionlightlevels.Withonly256lightlevels,therearecaseswherebrightlightsappearoverexposedorsaturated,whiledarkscenesaresimplycapturedasblack.

Therearecamerasthatcancapturemorethan8bpp.However,themostcommonwaytocreatehigh-dynamic-rangeimagesistousean8bppcameraandtakeimageswithdifferentexposurevalues.Whenwedothis,problemsofalimiteddynamicrangeareevident.Consider,forexample,thefollowingfigure:

Ascenecapturedwithsixdifferentexposurevalues

NoteThetop-leftimageismostlyblack,butwindowdetailsarevisible.Conversely,thebottom-rightimageshowsdetailsoftheroom,butthewindowdetailsarebarelyvisible.

Wecantakepictureswithdifferentexposurelevelsusingmodernsmartphonecameras.WithiPhoneandiPads,forexample,asofiOS8,itisveryeasytochangetheexposurewiththenativecameraapp.Bytouchingthescreen,ayellowboxappearswithasmallsunonitsside.Swipingupordowncanthenchangetheexposure(seethefollowingscreenshot).

Therangeofexposurelevelsisquitelarge,sowemayhavetorepeattheswipinggestureanumberoftimes.

IfyouusepreviousversionsofiOS,youcandownloadcameraappssuchasCamera+thatallowyoutofocusonaspecificpointandchangeexposure.

ForAndroid,tonsofcameraappsareavailableonGooglePlaythatcanadjusttheexposure.OneexampleisCameraFV-5,whichhasbothfreeandpaidversions.

TipIfyouuseahandhelddevicetocapturetheimages,makesurethedeviceisstatic.Infact,youmaywelluseatripod.Otherwise,imageswithdifferentexposureswillnotbealigned.Also,movingsubjectswillinevitablyproduceghostartifacts.Threeimagesaresufficientformostcases,withlow,medium,andhighexposurelevels.

TheexposurecontrolusingthenativecameraappinaniPhone5S

Smartphonesandtablesarehandytocaptureanumberofimageswithdifferentexposures.TocreateHDRimages,weneedtoknowtheexposure(orshutter)timeforeachcapturedimage(seethefollowingsectionforthereason).Notallappsallowyoutocontrol(orevensee)thismanually(theiOS8nativeappdoesn’t).Atthetimeofwritingthis,atleasttwofreeappsallowthisforiOS:ManuallyandManualShot!InAndroid,thefreeCameraFV-5allowsyoutocontrolandseeexposuretimes.NotethatF/StopandISOaretwootherparametersthatcontroltheexposure.

ImagesthatarecapturedcanbetransferredtothedevelopmentcomputerandusedtocreatetheHDRimage.

AsofiOS7,thenativecameraapphasanHDRmodethatautomaticallycapturesthreeimagesinarapidsequence,eachwithdifferentexposure.Theseimagesarealsoautomaticallycombinedintoasingle(sometimesbetter)image.

CreatingHDRimagesHowdowecombinemultiple(three,forexample)exposureimagesintoanHDRimage?Ifweconsideronlyoneofthechannelsandagivenpixel,thethreepixelvalues(oneforeachexposurelevel)mustbemappedtoasinglevalueinthelargeroutputrange(say,16bpp).Thismappingisnoteasy.Firstofall,wehavetoconsiderthatpixelintensitiesarea(rough)measureofsensorirradiance(theamountoflightincidentonthecamerasensor).Digitalcamerasmeasureirradiancebutinanonlinearway.Camerashaveanonlinearresponsefunctionthattranslatesirradiancetopixelintensityvaluesintherangeof0to255].Inordertomapthesevaluestoalargersetofdiscretevalues,wemustestimatethecameraresponsefunction(thatis,theresponsewithinthe0to255range).

Howdoweestimatethecameraresponsefunction?Wedothatfromthepixelsthemselves!TheresponsefunctionisanS-shapedcurveforeachcolorchannel,anditcanbeestimatedfromthepixels(withthreeexposuresofapixel,wehavethreepointsonthecurveforeachcolorchannel).Asthisisverytimeconsuming,usually,asetofrandompixelsischosen.

There’sonlyonethingleft.Wepreviouslytalkedaboutestimatingtherelationshipbetweenirradianceandpixelintensity.Howdoweknowirradiance?Sensorirradianceisdirectlyproportionaltotheexposuretime(orequivalently,theshutterspeed).Thisisthereasonwhyweneedexposuretime!

Finally,theHDRimageiscomputedasaweightedsumoftherecoveredirradiancevaluesfromthepixelsofeachexposure.Notethatthisimagecannotbedisplayedonconventionalscreens,whichalsohavealimitedrange.

NoteAgoodbookonhigh-dynamic-rangeimagingisHighDynamicRangeImaging:Acquisition,Display,andImage-BasedLightingbyReinhardetal,MorganKaufmannPub.ThebookisaccompaniedbyaDVDcontainingimagesindifferentHDRformats.

ExampleOpenCV(asof3.0only)providesfunctionstocreateHDRimagesfromasetofimagestakenwithdifferentexposures.There’sevenatutorialexamplecalledhdr_imaging,whichreadsalistofimagefilesandexposuretimes(fromatextfile)andcreatestheHDRimage.

NoteInordertorunthehdr_imagingtutorial,youwillneedtodownloadtherequiredimagefilesandtextfileswiththelist.Youcandownloadthemfromhttps://github.com/Itseez/opencv_extra/tree/master/testdata/cv/hdr.

TheCalibrateDebevecandMergeDebevecclassesimplementDebevec’smethodtoestimatethecameraresponsefunctionandmergetheexposuresintoanHDRimage,respectively.ThefollowingcreateHDRexampleshowsyouhowtousebothclasses:

#include<opencv2/photo.hpp>

#include<iostream>

usingnamespacecv;

usingnamespacestd;

intmain(int,char**argv)

vector<Mat>images;

vector<float>times;

//Loadimagesandexposures…

Matimg1=imread("1div66.jpg");

if(img1.empty())

cout<<"Error!Inputimagecannotberead…\n";

return-1;

images.push_back(img1);

times.push_back((float)1/66);

//Estimatecameraresponse…

Matresponse;

Ptr<CalibrateDebevec>calibrate=createCalibrateDebevec();

calibrate->process(images,response,times);

//Showtheestimatedcameraresponsefunction…

cout<<response;

//CreateandwritetheHDRimage…

Mathdr;

Ptr<MergeDebevec>merge_debevec=createMergeDebevec();

merge_debevec->process(images,hdr,times,response);

imwrite("hdr.hdr",hdr);

cout<<"\nDone.Pressanykeytoexit…\n";

return0;

Theexampleusesthreeimagesofacup(theimagesareavailablealongwiththecodeaccompanyingthisbook).TheimagesweretakenwiththeManualShot!appmentionedpreviously,usingexposuresof1/66,1/32,and1/12seconds;refertothefollowingfigure:

Thethreeimagesusedintheexampleasinputs

NotethatthecreateCalibrateDebevecmethodexpectstheimagesandexposuretimesinanSTLvector(STLisakindoflibraryofusefulcommonfunctionsanddatastructuresavailableinstandardC++).Thecameraresponsefunctionisgivenasa256real-valuedvector.Thisrepresentsthemappingbetweenthepixelvalueandirradiance.Actually,itisa256x3matrix(onecolumnpereachofthethreecolorchannels).Thefollowingfigureshowsyoutheresponsegivenbytheexample:

TheestimatedRGBcameraresponsefunctions

TipThecoutpartofcodedisplaysthematrixintheformatusedbyMATLABandOctave,twowidelyusedpackagesfornumericalcomputation.ItisstraightforwardtocopythematrixintheoutputandpasteitinMATLAB/Octaveinordertodisplayit.

TheresultingHDRimageisstoredinthelosslessRGBEformat.Thisimageformatusesonebytepercolorchannelplusonebyteasasharedexponent.Theformatusesthesameprincipleastheoneusedinthefloating-pointnumberrepresentation:thesharedexponentallowsyoutorepresentamuchwiderrangeofvalues.RGBEimagesusethe.hdrextension.Notethatasitisalosslessimageformat,.hdrfilesarerelativelylarge.Inthisexample,theRGBinputimagesare1224x1632each(100to200KBeach),whiletheoutput.hdrfileoccupies5.9MB.

TheexampleusesDebevecandMalik’smethod,butOpenCValsoprovidesanothercalibrationfunctionbasedonRobertson’smethod.Bothcalibrationandmergefunctionsareavailable,thatis,createCalibrateRobertsonandMergeRobertson.

NoteFormoreinformationontheotherfunctionsandthetheorybehindthem,refertohttp://docs.opencv.org/trunk/modules/photo/doc/hdr_imaging.html.

Finally,notethattheexampledoesnotdisplaytheresultingimage.TheHDRimagecannotbedisplayedinconventionalscreens,soweneedtoperformanotherstepcalledtonemapping.

TonemappingWhenhigh-dynamic-rangeimagesaretobedisplayed,informationcanbelost.Thisisduetothefactthatcomputerscreensalsohavealimitedcontrastratio,andprintedmaterialisalsotypicallylimitedto256tones.Whenwehaveahigh-dynamic-rangeimage,itisnecessarytomaptheintensitiestoalimitedsetofvalues.Thisiscalledtonemapping.

SimplyscalingtheHDRimagevaluestothereducedrangeofthedisplaydeviceisnotsufficientinordertoprovidearealisticoutput.Scalingtypicallyproducesimagesthatappearaslackingdetail(contrast),eliminatingtheoriginalscenecontent.Ultimately,tone-mappingalgorithmsaimatprovidingoutputsthatappearvisuallysimilartotheoriginalscene(thatis,similartowhatahumanwouldseewhenviewingthescene).Varioustone-mappingalgorithmshavebeenproposedanditisstillamatterofextensiveresearch.ThefollowinglinesofcodecanapplytonemappingtotheHDRimageobtainedinthepreviousexample:

Matldr;

Ptr<TonemapDurand>tonemap=createTonemapDurand(2.2f);

tonemap->process(hdr,ldr);//ldrisafloatingpointimagewith

ldr=ldr*255;//valuesininterval[0..1]

imshow("LDR",ldr);

ThemethodwasproposedbyDurandandDorseyin2002.Theconstructoractuallyacceptsanumberofparametersthataffecttheoutput.Thefollowingfigureshowsyoutheoutput.Notehowthisimageisnotnecessarilybetterthananyofthethreeoriginalimages:

Thetone-mappedoutput

Threeothertone-mappingalgorithmsareavailableinOpenCV:createTonemapDrago,

createTonemapReinhard,andcreateTonemapMantiuk.

AnHDRimage(theRGBEformat,thatis,fileswiththe.hdrextension)canbedisplayedusingMATLAB.Allittakesisthreelinesofcode:

hdr=hdrread('hdr.hdr');

rgb=tonemap(hdr);

imshow(rgb);

Notepfstoolsisanopensourcesuiteofcommand-linetoolstoread,write,andrenderHDRimages.Thesuite,whichcanread.hdrandotherformats,includesanumberofcameracalibrationandtone-mappingalgorithms.LuminanceHDRisfreeGUIsoftwarebasedonpfstools.

AlignmentThescenethatwillbecapturedwithmultipleexposureimagesmustbestatic.Thecameramustalsobestatic.Evenifthetwoconditionsmet,itisadvisabletoperformanalignmentprocedure.

OpenCVprovidesanalgorithmforimagealignmentproposedbyG.Wardin2003.Themainfunction,createAlignMTB,takesaninputparameterthatdefinesthemaximumshift(actually,alogarithmthebasetwoofthemaximumshiftineachdimension).Thefollowinglinesshouldbeinsertedrightbeforeestimatingthecameraresponsefunctioninthepreviousexample:

vector<Mat>images_(images);

Ptr<AlignMTB>align=createAlignMTB(4);//4=max16pixelshift

align->process(images_,images);

ExposurefusionWecanalsocombineimageswithmultipleexposureswithneithercameraresponsecalibration(thatis,exposuretimes)norintermediateHDRimage.Thisiscalledexposurefusion.ThemethodwasproposedbyMertensetalin2007.Thefollowinglinesperformexposurefusion(imagesistheSTLvectorofinputimages;refertothepreviousexample):

Matfusion;

Ptr<MergeMertens>merge_mertens=createMergeMertens();

merge_mertens->process(images,fusion);//fusionisa

fusion=fusion*255;//float.pointimagew.valuesin[0..1]

imwrite("fusion.png",fusion);

Thefollowingfigureshowsyoutheresult:

Exposurefusion

SeamlesscloningInphotomontages,wetypicallywanttocutanobject/personinasourceimageandinsertitintoatargetimage.Ofcourse,thiscanbedoneinastraightforwardwaybysimplypastingtheobject.However,thiswouldnotproducearealisticeffect.See,forexample,thefollowingfigure,inwhichwewantedtoinserttheboatinthetophalfoftheimageintotheseaatthebottomhalfoftheimage:

Cloning

AsofOpenCV3,thereareseamlesscloningfunctionsavailableinwhichtheresultismorerealistic.ThisfunctioniscalledseamlessCloneanditusesamethodproposedbyPerezandGangnetin2003.ThefollowingseamlessCloningexampleshowsyouhowitcanbeused:

#include<iostream>

usingnamespacecv;

usingnamespacestd;

//Loadandshowimages…

Matsource=imread("source1.png",IMREAD_COLOR);

Matdestination=imread("destination1.png",IMREAD_COLOR);

Matmask=imread("mask.png",IMREAD_COLOR);

imshow("source",source);

imshow("mask",mask);

imshow("destination",destination);

Matresult;

Pointp;//pwillbeneartoprightcorner

p.x=(float)2*destination.size().width/3;

p.y=(float)destination.size().height/4;

seamlessClone(source,destination,mask,p,result,NORMAL_CLONE);

imshow("result",result);

return0;

Theexampleisstraightforward.TheseamlessClonefunctiontakesthesource,destination,andmaskimagesandapointinthedestinationimageinwhichthecroppedobjectwillbeinserted(thesethreeimagescanbedownloadedfromhttps://github.com/Itseez/opencv_extra/tree/master/testdata/cv/cloning/Normal_Cloning).Seetheresultinthefollowingfigure:

Seamlesscloning

ThelastparameterofseamlessClonerepresentstheexactmethodtobeused(therearethreemethodsavailablethatproduceadifferentfinaleffect).Ontheotherhand,thelibraryprovidesthefollowingrelatedfunctions:

Function Effect

colorChangeMultiplieseachofthethreecolorchannelsofthesourceimagebyafactor,applyingthemultiplicationonlyintheregiongivenbythemask

illuminationChange Changesilluminationofthesourceimage,onlyintheregiongivenbythemask

textureFlattening Washesouttexturesinthesourceimage,onlyintheregiongivenbythemask

AsopposedtoseamlessClone,thesethreefunctionsonlyacceptsourceandmaskimages.

DecolorizationDecolorizationistheprocessofconvertingacolorimagetograyscale.Giventhisdefinition,thereadermaywellask,don’twealreadyhavegrayscaleconversion?Yes,grayscaleconversionisabasicroutineinOpenCVandanyimage-processinglibrary.ThestandardconversionisbasedonalinearcombinationoftheR,G,andBchannels.Theproblemisthatsuchaconversionmayproduceimagesinwhichcontrastintheoriginalimageislost.Thereasonisthattwodifferentcolors(whichareperceivedascontrastsintheoriginalimage)mayendupbeingmappedtothesamegrayscalevalue.Considertheconversionoftwocolors,AandB,tograyscale.Let’ssupposethatBisavariationofAintheRandGchannels:

A=(R,G,B)=>G=(R+G+B)/3

B=(R-x,G+x,B)=>G=(R-x+G+x+B)/3=(R+G+B)/3

Eventhoughtheyareperceivedasdistinct,thetwocolorsAandBaremappedtothesamegrayscalevalue!Theimagesfromthefollowingdecolorizationexampleshowthis:

#include<iostream>

usingnamespacecv;

usingnamespacestd;

//Loadandshowimages…

Matsource=imread("color_image_3.png",IMREAD_COLOR);

imshow("source",source);

//firstcomputeandshowstandardgrayscaleconversion…

Matgrayscale=Mat(source.size(),CV_8UC1);

cvtColor(source,grayscale,COLOR_BGR2GRAY);

imshow("grayscale",grayscale);

//nowcomputeandshowdecolorization…

Matdecolorized=Mat(source.size(),CV_8UC1);

Matdummy=Mat(source.size(),CV_8UC3);

decolor(source,decolorized,dummy);

imshow("decolorized",decolorized);

return0;

Decolorizationexampleoutput

Theexampleisstraightforward.Afterreadingtheimageandshowingtheresultofastandardgrayscaleconversion,itusesthedecolorfunctiontoperformthedecolorization.Theimageused(thecolor_image_3.pngfile)isincludedintheopencv_extrarepositoryathttps://github.com/Itseez/opencv_extra/tree/master/testdata/cv/decolor.

NoteTheimageusedintheexampleisactuallyanextremecase.Itscolorshavebeenchosensothatthestandardgrayscaleoutputisfairlyhomogeneous.

Non-photorealisticrenderingAspartofthephotomodule,fourfunctionsareavailablethattransformaninputimageinawaythatproducesanon-realisticbutstillartisticoutput.ThefunctionsareveryeasytouseandaniceexampleisincludedwithOpenCV(npr_demo).Forillustrativepurposes,hereweshowyouatablethatallowsyoutograsptheeffectofeachfunction.Takealookatthefollowingfruits.jpginputimage,includedwithOpenCV:

Theinputreferenceimage

Theeffectsare:

Function Effect

edgePreservingFilterSmoothingisahandyandfrequentlyusedfilter.Thisfunctionperformssmoothingwhilepreservingobjectedgedetails.

detailEnhance Enhancesdetailsintheimage

pencilSketch Apencil-likelinedrawingversionoftheinputimage

stylization Watercoloreffect

SummaryInthischapter,youlearnedwhatcomputationalphotographyisandtherelatedfunctionsavailableinOpenCV3.Weexplainedthemostimportantfunctionswithinthephotomodule,butnotethatotherfunctionsofthismodule(inpaintingandnoisereduction)werealsoconsideredinpreviouschapters.Computationalphotographyisarapidlyexpandingfield,withstrongtiestocomputergraphics.Therefore,thismoduleofOpenCVisexpectedtogrowinfutureversions.

Thenextchapterwillbedevotedtoanimportantaspectthatwehavenotyetconsidered:time.Manyofthefunctionsexplainedtakeasignificanttimetocomputetheresults.Thenextchapterwillshowyouhowtodealwiththatusingmodernhardware.

Chapter7.AcceleratingImageProcessingThischapterdealswiththeaccelerationofimageprocessingtasksusingGeneralPurposeGraphicsProcessingUnits(GPGPUs)or,inshort,GPUswithparallelprocessing.AGPUisessentiallyacoprocessordedicatedtographicsprocessingorfloatingpointoperations,aimedatimprovingperformanceonapplicationssuchasvideogamesandinteractive3Dgraphics.WhilethegraphicsprocessingisexecutedintheGPU,theCPUcanbededicatedtoothercalculations(suchastheartificialintelligencepartingames).EveryGPUisequippedwithhundredsofsimpleprocessingcoresthatallowmassiveparallelexecutiononhundredsof“simple”mathematicaloperationson(normally)floatingpointnumbers.

CPUsseemtohavereachedtheirspeedandthermalpowerlimits.BuildingacomputerwithseveralCPUshasbecomeacomplexproblem.ThisiswhereGPUscomeintoplay.GPUprocessingisanewcomputingparadigmthatusestheGPUtoimprovethecomputationalperformance.GPUsinitiallyimplementedcertainparalleloperationscalledgraphicsprimitivesthatwereoptimizedforgraphicsprocessing.Oneofthemostcommonprimitivesfor3Dgraphicsprocessingisantialiasing,whichmakestheedgesofthefigureshaveamorerealisticappearance.Otherprimitivesaredrawingsofrectangles,triangles,circles,andarcs.GPUscurrentlyincludehundredsofgeneral-purposeprocessingfunctionsthatcandomuchmorethanrenderinggraphics.Particularly,theyareveryvaluableintasksthatcanbeparallelized,whichisthecaseformanycomputervisionalgorithms.

OpenCVlibrariesincludesupportfortheOpenCLandCUDAGPUarchitectures.CUDAimplementsmanyalgorithms;however,itonlyworkswithNVIDIAgraphiccards.CUDAisaparallelcomputingplatformandprogrammingmodelcreatedbyNVIDIAandimplementedbytheGPUsthattheyproduce.ThischapterfocusesontheOpenCLarchitecture,asitissupportedbymoredevicesandisevenincludedinsomeNVIDIAgraphiccards.

TheOpenComputingLanguage(OpenCL)isaframeworkthatwritesprogramsthatcanbeexecutedonCPUsorGPUsattachedtoahostprocessor(aCPU).ItdefinesaC-likelanguagetowritefunctions,calledkernels,whichareexecutedonthecomputingdevices.UsingOpenCL,kernelscanberunonallormanyoftheindividualprocessingelements(PEs)inparalleltotheCPUsorGPUs.

Inaddition,OpenCLdefinesanApplicationProgrammingInterface(API)thatallowsprogramsrunningonthehost(theCPU)tolaunchkernelsonthecomputerdevicesandmanagetheirdevicememories,whichare(atleastconceptually)separatedfromthehostmemory.OpenCLprogramsareintendedtobecompiledatruntimesothatapplicationsthatuseOpenCLareportablebetweenimplementationsofvarioushostcomputerdevices.OpenCLisalsoanopenstandardmaintainedbythenonprofittechnologyconsortiumKhronosGroup(https://www.khronos.org/).

OpenCVcontainsasetofclassesandfunctionsthatimplementandacceleratetheOpenCVfunctionalityusingOpenCL.OpenCVcurrentlyprovidesatransparentAPIthatenablestheunificationofitsoriginalAPIwithOpenCL-acceleratedprogramming.

Therefore,youonlyneedtowriteyourcodeonce.Thereisanewunifieddatastructure(UMat)thathandlesdatatransferstotheGPUswhenitisneededandpossible.

SupportforOpenCLinOpenCVhasbeendesignedforeaseofuseanddoesnotrequireanyknowledgeofOpenCL.Ataminimumlevel,itcanbeviewedasasetofaccelerations,whichcantakeadvantageofthehighcomputingpowerwhenusingmodernCPUandGPUdevices.

TocorrectlyrunOpenCLprograms,theOpenCLruntimeshouldbeprovidedbythedevicevendor,typicallyintheformofadevicedriver.Also,touseOpenCVwithOpenCL,acompatibleSDKisneeded.Currently,therearefiveavailableOpenCLSDKs:

AMDAPPSDK:ThisSDKsupportsOpenCLonCPUsandGPUs,suchasX86+SSE2(orhigher)CPUsandAMDFusion,AMDRadeon,AMDMobility,andATIFireProGPUs.IntelSDK:ThisSDKsupportsOpenCLonIntelCoreprocessorsandIntelHDGPUs,suchasIntel+SSE4.1,SSE4.2orAVX,IntelCorei7,i5andi3(1st,2nd,and3rdGeneration),IntelHDGraphics,IntelCore2Solo(DuoQuadandExtreme),andIntelXeonCPUs.IBMOpenCLDevelopmentKit:ThisSDKsupportsOpenCLonAMDserverssuchasIBMPower,IBMPERCS,andIBMBladeCenter.IBMOpenCLCommonRuntime:ThisSDKsupportsOpenCVonCPUsandGPUs,suchasX86+SSE2(orhigher)CPUsandAMDFusionandRaedon,NVIDIAIon,NVIDIAGeForce,andNVIDIAQuadroGPUs.NvidiaOpenCLDriverandTools:ThisSDKsupportsOpenCLonsomeNvidiagraphicdevicessuchasNVIDIATesla,NVIDIAGeForce,NVIDIAIon,andNVIDIAQuadroGPUs.

OpenCVwiththeOpenCLinstallationTheinstallationstepsalreadypresentedinChapter1,HandlingImageandVideoFiles,needsomeadditionalstepstoincludeOpenCL.Thenewlyrequiredsoftwareisexplainedinthefollowingsection.

TherearenewrequirementstocompileandinstallOpenCVwithOpenCLonWindows:

OpenCL-capableGPUorCPU:Thisisthemostimportantrequirement.NotethatOpenCLsupportsmanycomputingdevicesbutnotall.YoucancheckwhetheryourgraphiccardsorprocessorsarecompatiblewithOpenCL.ThischapterusestheAMDAPPSDKforanAMDFireProW5000GPUtoexecutetheexamples.

NoteThereisalistwiththesupportedcomputerdevicesforthisSDKathttp://developer.amd.com/tools-and-sdks/opencl-zone/amd-accelerated-parallel-processing-app-sdk/system-requirements-driver-compatibility/.There,youcanalsoconsulttheminimumSDKversionthatyouneed.

Compilers:OpenCVwithOpenCLiscompatiblewithMicrosoftandMinGWcompilers.ItispossibletoinstallthefreeVisualStudioExpressedition.However,ifyouchooseMicrosofttocompileOpenCV,atleastVisualStudio2012isrecommended.However,theMinGWcompilerisusedinthischapter.AMDAPPSDK:ThisSDKisasetofadvancedsoftwaretechnologiesthatenableustousecompatiblecomputingdevicestoexecuteandacceleratemanyapplicationsbeyondjustgraphics.ThisSDKisavailableathttp://developer.amd.com/tools-and-sdks/opencl-zone/amd-accelerated-parallel-processing-app-sdk/.ThischapterusesVersion2.9oftheSDK(for64bitWindows);youcanseetheinstallationprogressinthefollowingscreenshot.

NoteIfthisstepfails,maybeyoumightneedtoupdatethecontrollerofyourgraphiccard.TheAMDcontrollersareavailableathttp://www.amd.com/en-us/innovations/software-technologies.

InstallingtheAMDAPPSDK

OpenCLBLAS:BasicLinearAlgebraSubroutines(BLAS)isasetofopensourcemathlibrariesforparallelprocessingonAMDdevices.Itcanbedownloadedfromhttp://developer.amd.com/tools-and-sdks/opencl-zone/amd-accelerated-parallel-processing-math-libraries/.Thischapterusesthe1.1BLASversionforWindows32/64bits,andyoucanseetheinstallationprogressinthefollowingscreenshot(theleft-handside).OpenCLFFT:FastFourierTransform(FFT)isaveryusefulfunctionthatmanyalgorithmsofimageprocessingneed.Therefore,thisfunctionisimplementedforparallelprocessingonAMDdevices.ItcanbedownloadedfromthesameURLasgivenpreviously.Thischapterusesthe1.1FFTversionforWindows32/64bits,andyoucanseetheinstallationprogressinthefollowingscreenshot(theright-handside):

InstallingBLASandFFTforOpenCL

QtlibrariesforC++compiler:Inthischapter,theMinGWbinariesofQtlibrariesareusedtocompileOpenCVwithOpenCL.TheotheralternativeistoinstallthelatestversionofQtandusetheVisualC++compiler.YoucanchoosetheQtversionandusedcompiler.Thepackagemanager,bymeansoftheMaintenanceTool.exeapplicationlocatedatC:\Qt\Qt5.3.1,canbeusedtodownloadotherQtversions.ThischapterusesQt(5.3.1)andMinGW(4.8.2)32bitstocompileOpenCVwithOpenCL.

Whenthepreviousrequirementsaremet,youcangenerateanewbuildconfigurationwithCMake.Thisprocessdiffersinsomepointsfromthetypicalinstallationthatwasexplainedinthefirstchapter.Thedifferencesareexplainedinthislist:

Whenselectingthegeneratorfortheproject,youcanchoosethecompilerversioncorrespondingwiththeinstalledenvironmentinyourmachine.ThischapterusesMinGWtocompileOpenCVwithOpenCL,andthentheMinGWMakefilesoptionisselected,specifyingthenativecompilers.Thefollowingscreenshotshowsthisselection:

CMakeselectingthegeneratorproject

TheoptionsshowninthefollowingscreenshotareneededtobuildtheOpenCVwithOpenCLproject.TheWITH_OPENCL,WITH_OPENCLAMDBLAS,andWITH_OPENCLAMDFFToptionsmustbeenabled.TheBLASandFFTpathsmustbeintroducedonCLAMDBLAS_INCLUDE_DIR,CLAMDBLAS_ROOT_DIR,CLAMDFFT_INCLUDE_DIR,andCLAMDFFT_ROOT_DIR.Inaddition,asshowninChapter1,HandlingImageandVideoFiles,youwillneedtoenableWITH_QTanddisabletheWITH_IPPoptionaswell.ItisalsoadvisabletoenableBUILD_EXAMPLES.Thefollowingscreenshotshowsyouthemainoptionsselectedinthebuildconfiguration:

CMakeselectingthemainoptions

Finally,tobuildtheOpenCVwithOpenCLproject,theCMakeprojectthatwaspreviouslygeneratedmustbecompiled.TheprojectwasgeneratedforMinGW,andtherefore,theMinGWcompilerisneededtobuildthisproject.Firstly,the[opencv_build]/folderisselectedwithWindowsConsole,andweexecutethis:

./mingw32-make.exe-j4install

The-j4parameteristhenumberofthecoreCPUsofthesystemthatwewanttousefor

theparallelizationofthecompilation.

NowtheOpenCVwithOpenCLprojectisreadytobeused.Thepathofthenewbinariesfilesmustbeaddedtothesystempath,inthiscase,[opencv_build]/install/x64/mingw/bin.

NoteDonotforgettoremovetheoldbinaryfilesofOpenCVfromthepathenvironmentvariable.

AquickrecipetoinstallOpenCVwithOpenCLTheinstallationprocesscanbesummarizedinthefollowingsteps:

1. DownloadandinstallAMDAPPSDK,whichisavailableathttp://developer.amd.com/tools-and-sdks/opencl-zone/amd-accelerated-parallel-processing-app-sdk.

2. DownloadandinstallBLASandFFTAMD,whichareavailableathttp://developer.amd.com/tools-and-sdks/opencl-zone/amd-accelerated-parallel-processing-math-libraries.

3. ConfiguretheOpenCVbuildwithCMake.EnabletheWITH_OPENCL,WITH_OPENCLAMDBLAS,WITH_QT,andBuild_EXAMPLESWITH_OPENCLAMDFFToptions.DisabletheWITH_IPPoption.Finally,introducetheBLASandFFTpathsonCLAMDBLAS_INCLUDE_DIR,CLAMDBLAS_ROOT_DIR,CLAMDFFT_INCLUDE_DIR,andCLAMDFFT_ROOT_DIR.

4. CompiletheOpenCVprojectwithmingw32-make.exe.5. Finally,modifythepathenvironmentvariabletoupdatetheOpenCVbindirectory

(forexample,[opencv_build]/install/x64/mingw/bin).

ChecktheGPUusageWhentheGPUisbeingusedonaWindowsplatform,thereisnoapplicationtomeasureitsusage.TheGPUusageisusefulfortworeasons:

ItispossibletoknowwhetheryouareusingtheGPUcorrectlyYoucanmonitortheGPUusagepercentage

Therearesomeapplicationsinthemarketforthispurpose.ThischapterusesAMDSystemMonitortochecktheGPUusage.ThisapplicationmonitorstheCPU,memoryRAM,andGPUusage.Refertothefollowingscreenshotforthis:

AMDSystemMonitormonitorstheCPU,GPU,andMemoryRAMusages

TheAMDSystemMonitorcanbedownloadedfromhttp://support.amd.com/es-xl/kb-articles/Pages/AMDSystemMonitor.aspxforMicrosoftWindows(32or64bits).

AcceleratingyourownfunctionsInthissection,therearethreeexamplesofusingOpenCVwithOpenCL.ThefirstexampleallowsyoutocheckwhethertheinstalledSDKisavailableandobtainusefulinformationaboutthecomputingdevicesthatsupportOpenCL.ThesecondexampleshowsyoutwoversionsofthesameprogramusingCPUandGPUprogramming,respectively.Thelastexampleisacompleteprogramtodetectandmarkfaces.Inaddition,acomputationalcomparativeisperformed.

CheckingyourOpenCLThefollowingisasimpleprogramthatisshowntocheckyourSDKandtheavailablecomputingdevices.ThisexampleiscalledcheckOpenCL.ItallowsyoutodisplaythecomputerdevicesusingtheOCLmoduleofOpenCV:

#include<opencv2/core/ocl.hpp>

usingnamespacestd;

usingnamespacecv;

usingnamespacecv::ocl;

intmain()

vector<ocl::PlatformInfo>info;

getPlatfomsInfo(info);

PlatformInfosdk=info.at(0);

if(sdk.deviceNumber()<1)

return-1;

cout<<"******SDK*******"<<endl;

cout<<"Name:"<<sdk.name()<<endl;

cout<<"Vendor:"<<sdk.vendor()<<endl;

cout<<"Version:"<<sdk.version()<<endl;

cout<<"Numberofdevices:"<<sdk.deviceNumber()<<endl;

for(inti=0;i<sdk.deviceNumber();i++){

Devicedevice;

sdk.getDevice(device,i);

cout<<"\n\n*********************\nDevice"<<i+1<<endl;

cout<<"VendorID:"<<device.vendorID()<<endl;

cout<<"Vendorname:"<<device.vendorName()<<endl;

cout<<"Name:"<<device.name()<<endl;

cout<<"Driverversion:"<<device.driverVersion()<<endl;

if(device.isAMD())cout<<"IsanAMDdevice"<<endl;

if(device.isIntel())cout<<"IsaInteldevice"<<endl;

cout<<"GlobalMemorysize:"<<device.globalMemSize()<<endl;

cout<<"Memorycachesize:"<<device.globalMemCacheSize()<<endl;

cout<<"Memorycachetype:"<<device.globalMemCacheType()<<endl;

cout<<"LocalMemorysize:"<<device.localMemSize()<<endl;

cout<<"LocalMemorytype:"<<device.localMemType()<<endl;

cout<<"MaxClockfrequency:"<<device.maxClockFrequency()<<

return0;

ThecodeexplanationThisexampledisplaystheinstalledSDKandtheavailablecomputingdevicesthatarecompatiblewithOpenCL.Firstly,thecore/ocl.hppheaderisincludedandthecv::ocl

namespaceisdeclared.

TheinformationabouttheavailableSDKinyourcomputerisobtainedusingthegetPlatfomsInfo(info)method.Thisinformationisstoredinthevector<ocl::PlatformInfo>infovectorandselectedwithPlatformInfosdk=info.at(0).Afterwards,themaininformationaboutyourSDKisshown,suchasthename,vendor,SDKversion,andthenumberofcomputingdevicescompatiblewithOpenCL.

Finally,foreachcompatibledevice,itsinformationisobtainedwiththesdk.getDevice(device,i)method.Nowdifferentinformationabouteachcomputingdevicecanbeshown,suchasthevendorID,vendorname,driverversion,globalmemorysize,memorycachesize,andsoon.

Thefollowingscreenshotshowsyoutheresultsofthisexampleforthecomputerused:

InformationabouttheSDKusedandcompatiblecomputingdevices

YourfirstGPU-basedprogramInthefollowingcode,twoversionsofthesameprogramareshown:oneonlyusestheCPU(native)toperformthecomputationsandtheotherusestheGPU(withOpenCL).ThesetwoexamplesarecalledcalculateEdgesCPUandcalculateEdgesGPU,respectively,andallowyoutoobservethedifferencesbetweenCPUandGPUversions.

ThecalculateEdgesCPUexampleisshowninthefirstplace:

usingnamespacestd;

usingnamespacecv;

if(argc<2)

cout<<"./calculateEdgesCPU<image>"<<endl;

return-1;

MatcpuFrame=imread(argv[1]);

MatcpuBW,cpuBlur,cpuEdges;

namedWindow("CannyEdgesCPU",1);

cvtColor(cpuFrame,cpuBW,COLOR_BGR2GRAY);

GaussianBlur(cpuBW,cpuBlur,Size(1,1),1.5,1.5);

Canny(cpuBlur,cpuEdges,50,100,3);

imshow("CannyEdgesCPU",cpuEdges);

waitKey();

return0;

Now,thecalculateEdgesGPUexampleisshown:

#include"opencv2/core/ocl.hpp"

usingnamespacestd;

usingnamespacecv;

if(argc<2)

cout<<"./calculateEdgesGPU<image>"<<endl;

return-1;

setUseOpenCL(true);

MatcpuFrame=imread(argv[1]);

UMatgpuFrame,gpuBW,gpuBlur,gpuEdges;

cpuFrame.copyTo(gpuFrame);

namedWindow("CannyEdgesGPU",1);

cvtColor(gpuFrame,gpuBW,COLOR_BGR2GRAY);

GaussianBlur(gpuBW,gpuBlur,Size(1,1),1.5,1.5);

Canny(gpuBlur,gpuEdges,50,100,3);

imshow("CannyEdgesGPU",gpuEdges);

waitKey();

return0;

ThecodeexplanationThesetwoexamplesobtainthesameresult,asshowninthefollowingscreenshot.Theyreadanimagefromthestandardcommand-lineinputarguments.Afterwards,theimageisconvertedtograyscaleandtheGaussianBlurandtheCannyfilterfunctionsareapplied.

Inthesecondexample,therearesomedifferencesthatarerequiredtoworkwiththeGPU.First,OpenCLmustbeactivatedwiththesetUseOpenCL(true)method.Second,UnifiedMats(UMat)areusedtoallocatememoryintheGPU(UMatgpuFrame,gpuBW,gpuBlur,gpuEdges).Third,theinputimageiscopiedfromtheRAMtoGPUmemorywiththecpuFrame.copyTo(gpuFrame)method.Now,whenthefunctionsareused,iftheyhaveanOpenCLimplementation,thenthesefunctionswillbeexecutedontheGPU.IfsomeofthesefunctionsdonothaveanOpenCLimplementation,thenormalfunctionwillbeexecutedontheCPU.Inthisexample,thetimeelapseusingtheGPUprogramming(secondexample)is10timesbetter:

Resultsoftheprecedingtwoexamples

GoingrealtimeOneofthemainadvantagesofGPUprocessingistoperformcomputationsinamuchfasterway.Thisincreaseinspeedallowsyoutoexecuteheavycomputationalalgorithmsinreal-timeapplications,suchasstereovision,pedestriandetection,opticalflow,orfacedetection.ThefollowingdetectFacesexampleshowsyouanapplicationtodetectfacesfromavideocamera.ThisexamplealsoallowsyoutoselectbetweentheCPUorGPUprocessinginordertocomparethecomputationaltime.

IntheOpenCVexamples([opencv_source_code]/samples/cpp/facedetect.cpp),arelatedfacedetectorexamplecanbefound.ForthefollowingdetectFacesexample,thedetectFace.proprojectneedstheselibraries:-lopencv_core300,-opencv_imgproc300,-lopencv_highgui300,-lopencv_videoio300,andlopencv_objdetct300.

ThedetectFacesexampleusestheoclmoduleofOpenCV:

#include<opencv2/core/ocl.hpp>

#include<opencv2/objdetect.hpp>

#include<opencv2/videoio.hpp>

#include<iostream>

#include<stdio.h>

usingnamespacestd;

usingnamespacecv;

//1-Settheinitialparameters

//Vectortostorethefaces

vector<Rect>faces;

CascadeClassifierface_cascade;

Stringface_cascade_name=argv[2];

intface_size=30;

doublescale_factor=1.1;

intmin_neighbours=2;

VideoCapturecap(0);

UMatframe,frameGray;

boolfinish=false;

//2-Loadthefilexmltousetheclassifier

if(!face_cascade.load(face_cascade_name))

cout<<"Cannotloadthefacexml!"<<endl;

return-1;

namedWindow("VideoCapture");

//3-SelectbetweentheCPUorGPUprocessing

if(argc<2)

cout<<"./detectFaces[CPU/GPU|C/G]"<<endl;

cout<<"TryingtouseGPU…"<<endl;

setUseOpenCL(true);

cout<<"./detectFacestryingtouse"<<argv[1]<<endl;

if(argv[1][0]=='C')

//TryingtousetheCPUprocessing

setUseOpenCL(false);

//TryingtousetheGPUprocessing

setUseOpenCL(true);

Rectr;

doublestart_time,finish_time,start_total_time,finish_total_time;

intcounter=0;

//4-Detectthefacesforeachimagecapture

start_total_time=getTickCount();

while(!finish)

start_time=getTickCount();

cap>>frame;

if(frame.empty())

cout<<"Nocaptureframe-->finish"<<endl;

break;

cvtColor(frame,frameGray,COLOR_BGR2GRAY);

equalizeHist(frameGray,frameGray);

//Detectthefaces

face_cascade.detectMultiScale(frameGray,faces,scale_factor,

min_neighbours,0|CASCADE_SCALE_IMAGE,Size(face_size,face_size));

//Foreachdetectedface

for(intf=0;f<faces.size();f++)

r=faces[f];

//Drawarectangleovertheface

rectangle(frame,Point(r.x,r.y),Point(r.x+r.width,r.y+r.height),

Scalar(0,255,0),3);

//Showtheresults

imshow("VideoCapture",frame);

//Calculatethetimeprocessing

finish_time=getTickCount();

cout<<"Timeperframe:"<<(finish_time-start_time)/getTickFrequency()

<<"secs"<<endl;

counter++;

//PressEsckeytofinish

if(waitKey(1)==27)finish=true;

finish_total_time=getTickCount();

cout<<"Averagetimeperframe:"<<((finish_total_time-

start_total_time)/getTickFrequency())/counter<<"secs"<<endl;

return0;

ThecodeexplanationThefirststepsetstheinitialparameters,suchasthexmlfile(Stringface_cascade_nameargv[2])thatusestheclassifiertodetectthefaces,theminimumsizeofeachdetectedface(face_size=30),thescalefactor(scale_factor=1.1),andtheminimumnumberofneighbors(min_neighbours=2)tofindatrade-offbetweentruepositiveandfalsepositivedetections.YoucanalsoseethemoreimportantdifferencesbetweentheCPUandGPUsourcecodes;youonlyneedtousetheUnifiedMat(UMatframe,frameGray).

NoteThereareotheravailablexmlfilesinthe[opencv_source_code]/data/haarcascades/foldertodetectdifferentbodypartssuchaseyes,lowerbodies,smiles,andsoon.

Thesecondstepcreatesadetectorusingtheprecedingxmlfiletodetectfaces.ThisdetectorisbasedonaHaarfeature-basedclassifierthatisaneffectiveobject-detectionmethodproposedbyPaulViolaandMichaelJones.Thisclassifierhasahighaccuracyofdetectingfaces.Thissteploadsthexmlfilewiththeface_cascade.load(face_cascade_name)method.

NoteYoucanfindmoredetailedinformationaboutPaulViolaandMichaelJonesmethodathttp://en.wikipedia.org/wiki/Viola%E2%80%93Jones_object_detection_framework.

ThethirdstepallowsyoutoselectbetweentheCPUorGPUprocessing(setUseOpenCL(false)orsetUseOpenCL(true),respectively).Thisexampleusesthestandardcommand-lineinputarguments(argv[1])toselectthis.TheusercanexecutethefollowingfromtheWindowsconsoletoselectbetweenCPUorGPUprocessing,respectively,andtheclassifierpath:

<bin_dir>/detectFacesCPUpathClassifier

<bin_dir>/detectFacesGPUpathClassifier

Iftheuserdoesnotintroduceaninputargument,thenGPUprocessingisused.

Thefourthstepdetectsfacesforeachimagecapturedfromthevideocamera.Beforethat,

eachcapturedimageisconvertedtograyscale(cvtColor(frame,frameGray,COLOR_BGR2GRAY))anditshistogramisequalized(equalizeHist(frameGray,frameGray)).Afterwards,usingthecreatedfacedetector,thedifferentfacesaresearchedoverthecurrentframeusingtheface_cascade.detectMultiScale(frameGray,faces,scale_factor,min_neighbours,0|CASCADE_SCALE_IMAGE,

Size(face_size,face_size))multiscaledetectionmethod.Finally,agreenrectangleisdrawnovereachdetectedface,andthenitisdisplayed.Ascreenshotofthisexamplerunningisshowninthefollowingscreenshot:

Theprecedingexampledetectingaface

TheperformanceIntheprecedingexample,thecomputationaltimeiscalculatedtocomparetheCPUandGPUprocessing.Theaverageprocessingtimeperframeisobtained.

AbigadvantageofchoosingGPUprogrammingisperformance.Therefore,thepreviousexamplecalculatesthetimemeasurementstocomparethespeedupsobtainedwithrespecttotheCPUversion.ThetimeisstoredatthebeginningoftheprogramusingthegetTickCount()method.Afterwards,attheendoftheprogram,thesamefunctiontoestimatethetimeisusedagain.Acounterisstoredtoknowthenumberofiterationsaswell.Finally,theaverageprocessingtimeperframeiscalculated.Theprecedingexamplehasanaverageprocessingtimeperframeof0.057secondsperframe(or17.5FPS)usingtheGPU,whereasthesameexampleusingtheCPUhasanaverageprocessingtimeperframeof0.335secondsperframe(or2.9FPS).Inconclusion,thereisaspeedincrementof6x.Thisincrementissignificant,especiallywhenyouonlyneedtochangeafewlines

ofthecode.However,itispossibletoachieveamuchhigherrateofspeedincrements,whichisrelatedtotheproblemandevenwithhowthekernelsaredesigned.

SummaryInthischapter,youlearnedhowtoinstallOpenCVwithOpenCLonyourcomputeranddevelopapplicationsusingyourcomputerdevices,whicharecompatiblewithOpenCL,withthelastversionofOpenCV.

ThefirstsectionexplainswhatOpenCLisandtheavailableSDKs.Rememberthatdependingyourcomputingdevices,youwillneedaspecificSDKtoworkcorrectlywithOpenCL.Inthesecondsection,theinstallationprocesstoinstallOpenCVwithOpenCLisexplained,andtheAMDAPPSDKhasbeenused.Inthelastsection,therearethreeexamplesusingGPUprogramming(thesecondexamplealsohasaCPUversioninordertocomparethem).Inaddition,inthelastsection,thereisacomputationalcomparativebetweenCPUandGPUprocessing,wheretheGPUisshowntobesixtimesfasterthantheCPUversion.

IndexA

affinetransformationabout/Affinetransformationscaling/Scalingtranslation/Translationimagerotation/Imagerotationskewing/Skewingreflection/Reflection

alignment,HDRimages/AlignmentAMDAPPSDK

URL/OpenCVwiththeOpenCLinstallation,AquickrecipetoinstallOpenCVwithOpenCL

AMDcontrollersURL/OpenCVwiththeOpenCLinstallation

AMDSystemMonitorURL,fordownloading/ChecktheGPUusage

arithmeticoperationsabout/Arithmeticoperations

BbasicAPIconcepts

about/ThebasicAPIconceptsbasicdatatypes

about/BasicdatatypesBayer

about/Bayerexamplecode/Theexamplecode

BilateralTV-L1algorithmabout/Superresolutionparameters/Superresolution

BilateralTV-Lmethodabout/SuperresolutionURL/Superresolution

bitsperpixel(bpp)/High-dynamic-rangeimagesBLAS

URL/OpenCVwiththeOpenCLinstallation,AquickrecipetoinstallOpenCVwithOpenCL

BSDlicenseabout/AnintroductiontoOpenCV

bundleadjustment/Stitchingbuttons

about/Buttons

OpenCVapplication,creatingwith/GeneralusageofthelibrarycalculateEdgesCPUexample

about/YourfirstGPU-basedprogramcode/Thecodeexplanation

calculateEdgesGPUexampleabout/YourfirstGPU-basedprogramcode/Thecodeexplanation

checkOpenCLexampleabout/CheckingyourOpenCLcode/Thecodeexplanation

CIEL*a*b*about/CIEL*a*b*examplecode/Theexamplecode

CIEL*u*v*about/CIEL*u*v*examplecode/Theexamplecode

CIEXYZabout/CIEXYZexamplecode/Theexamplecode

CMakesetting/GettingacompilerandsettingCMakeabout/GettingacompilerandsettingCMakeURL/GettingacompilerandsettingCMakeOpenCV,configuringwith/ConfiguringOpenCVwithCMakelibrary,installing/Compilingandinstallingthelibrarylibrary,compiling/Compilingandinstallingthelibrary

color-space-basedsegmentationabout/Color-space-basedsegmentationHSVsegmentation/HSVsegmentationYCrCbsegmentation/YCrCbsegmentation

colorspacesabout/Colorspacesconversion,withcvtColormethod/Conversionbetweencolorspaces(cvtColor)RGB/RGBgrayscale/GrayscaleCIEXYZ/CIEXYZYCrCb/YCrCbHSV/HSVHLS/HLSCIEL*a*b*/CIEL*a*b*CIEL*u*v*/CIEL*u*v*

Bayer/Bayercolortransfer

about/Colortransferexamplecode/Theexamplecode

ColourImageComparisonexample,histogramsabout/Theexamplecode

ColourImageEqualizeHistexample,histogramsabout/Theexamplecodesourceimagewindow/Theexamplecodeequalizedcolorimagewindow/Theexamplecodehistogramofthreechannelswindow/TheexamplecodehistogramofRGBchannelfortheequalizedimagewindow/Theexamplecode

CommissionInternationaledeL’Éclairage(CIE)about/CIEXYZ

compilerkits,OpenCVC++applicationsMicrosoftVisualC(MSVC)/ToolstodevelopnewprojectsGNUGCC(GNUCompilerCollection)/Toolstodevelopnewprojects

ComputeUnifiedDeviceArchitecture(CUDA)about/AnintroductiontoOpenCV

cvtColormethodused,forcolorspacesconversion/Conversionbetweencolorspaces(cvtColor)srcargument/Conversionbetweencolorspaces(cvtColor)dstargument/Conversionbetweencolorspaces(cvtColor)codeargument/Conversionbetweencolorspaces(cvtColor)dstCnargument/Conversionbetweencolorspaces(cvtColor)

cylindricalprojection/Stitching

Ddatapersistence

about/Datapersistencedecolorization

about/Decolorizationexample/Decolorization

denoisingabout/Denoisingreferencelink/Denoisingfunctions/Denoisingexamplecode/Theexamplecode

detectFacesexampleabout/Goingrealtimecode/Goingrealtime,Thecodeexplanation

digitalstabilizationsystemsabout/Videostabilization

EestimatePiexample

about/Arithmeticoperationsexposurefusion,HDRimages/Exposurefusionextrapolationmethods

about/Geometricaltransformations

FFarnebackopticalflowmethod

URL/SuperresolutionFFTAMD

URL/AquickrecipetoinstallOpenCVwithOpenCLfilestructure,OpenCV

headerfiles/ThestructureofOpenCVlibrarybinaries/ThestructureofOpenCVsamplebinaries/ThestructureofOpenCV

fourcccodeURL/Theexamplecode

functions,acceleratingabout/AcceleratingyourownfunctionscheckOpenCLexample/CheckingyourOpenCLcalculateEdgesCPUexample/YourfirstGPU-basedprogramcalculateEdgesGPUexample/YourfirstGPU-basedprogramdetectFacesexample/GoingrealtimeGPUprogramming,performance/Theperformance

functions,seamlessCloningexamplecolorChange/SeamlesscloningilluminationChange/SeamlesscloningtextureFlattening/Seamlesscloning

GGaussianpyramids

about/Gaussianpyramidsfunctions/Gaussianpyramids

GDAL(GeographicDataAbstractionLibrary)about/Imagefile-supportedformats

geometricaltransformationsabout/Geometricaltransformationsextrapolationmethods/Geometricaltransformationsinterpolationmethods/Geometricaltransformationsaffinetransformation/Affinetransformationperspectivetransformation/Perspectivetransformation

GNUGCConlinedocumentation,URL/Generalusageofthelibrary

GNUtoolkit/GettingacompilerandsettingCMakeGPUusage

checking/ChecktheGPUusageGraphicProcessingUnit(GPU)

about/AnintroductiontoOpenCVgrayscale

about/Grayscaleexamplecode/Examplecode

GUI(GraphicalUserInterface)/ConfiguringOpenCVwithCMake

HHaarfeature-basedclassifier/ThecodeexplanationHDRimages

about/High-dynamic-rangeimagescreating/CreatingHDRimagescreateHDRexample/Exampletonemapping/Tonemappingalignment/Alignmentexposurefusion/Exposurefusion

hdr_imagingtutorialURL,forfileprerequisites/Example

HighDynamicRange(HDR)/ThestructureofOpenCVhighdynamicrange(HDR)

about/AnintroductiontoOpenCVhistogramequalization

about/Histogramshistograms

about/HistogramsColourImageEqualizeHistexample/TheexamplecodeColourImageComparisonexample/Theexamplecode

HLSabout/HLSexamplecode/Theexamplecode

HSVabout/HSVhue/HSVsaturation/HSVvalue/HSVexamplecode/Theexamplecode

HSVsegmentationabout/HSVsegmentation

IIDE(IntegratedDevelopmentEnvironment)/GettingacompilerandsettingCMakeimagecapturingprocess

about/Superresolutionsampling/Superresolutiongeometrictransformation/Superresolutionblur/Superresolutionsubsampling/Superresolution

imagefile-supportedformatsabout/Imagefile-supportedformats

imagefilesreading/Readingandwritingimagefiles,Readingimagefileswriting/Readingandwritingimagefiles,Writingimagefilesexamplecode/Theexamplecodeeventhandling,intointrinsicloop/Eventhandlingintotheintrinsicloop

imagefilteringabout/Imagefilteringsmoothing/Smoothingsharpening/Sharpeningimagepyramids/Workingwithimagepyramids

imageprocessingtimemeasuring/Measuringthetime

imagepyramidsabout/WorkingwithimagepyramidsLaplacianpyramids/Workingwithimagepyramids,LaplacianpyramidsGaussianpyramids/Gaussianpyramidsexamplecode/Theexamplecode

imagerotationabout/Imagerotationexamplecode/Theexamplecode

imagestitchingabout/StitchingURL/Stitchingregistration/Stitchingcalibration/Stitchingcompositing/StitchingstitchingAdvancedexample/Stitching

imshowfunctionabout/Theexamplecode

inpaintingabout/Inpaintingfunctions/Inpaintingreferencelink/Inpainting

examplecode/TheexamplecodeIntegratedDevelopmentEnvironment(IDE/ToolstodevelopnewprojectsIntegratedPerformancePrimitives(IPP)

about/AnintroductiontoOpenCVIntelIntegratedPerformancePrimitives(IPP)/ConfiguringOpenCVwithCMakeinterpolationmethods

about/Geometricaltransformations

Kkernel

about/Smoothing

LLaplacianpyramids

about/Workingwithimagepyramids,LaplacianpyramidsLinux/CompilingandinstallingthelibraryLuminanceHDR/TonemappingLUTs

about/LUTsexamplecode/Theexamplecode

onlinedocumentation,URL/Generalusageofthelibrarymapprojections

about/Stitchingrectilinear/Stitchingcylindrical/Stitchingspherical/Stitchingstereographic/Stitchingpanini/Stitching

mean-siftsegmentationreferencelink/Gaussianpyramids

mechanicalstabilizationsystemsabout/Videostabilization

MinGW(MinimalGNUGCC)/Toolstodevelopnewprojectsmipmap

about/Workingwithimagepyramidsmodules,OpenCV

core/ThestructureofOpenCVhighgui/ThestructureofOpenCVimgproc/ThestructureofOpenCVimgcodecs/ThestructureofOpenCVphoto/ThestructureofOpenCVstitching/ThestructureofOpenCVvideoio/ThestructureofOpenCVvideo/ThestructureofOpenCVfeatures2d/ThestructureofOpenCVobjdetect/ThestructureofOpenCV

morphologicaloperationsabout/Morphologicaloperationsfunctions/Morphologicaloperationsexamplecode/Theexamplecode

mouseinteractionabout/Mouseinteraction

Nnon-photorealisticrendering

about/Non-photorealisticrenderingedgePreservingFiltereffect/Non-photorealisticrenderingdetailEnhanceeffect/Non-photorealisticrenderingstylizationeffect/Non-photorealisticrendering

OOpenComputingLanguage(OpenCL)

about/AnintroductiontoOpenCVOpenCV

about/AnintroductiontoOpenCVURL,fordownloading/DownloadingandinstallingOpenCVdownloading/DownloadingandinstallingOpenCVinstalling/DownloadingandinstallingOpenCVURL,formainrepository/DownloadingandinstallingOpenCVURL,fortestdatarepository/DownloadingandinstallingOpenCVURL,forcontributionsrepository/DownloadingandinstallingOpenCVURL,fordocumentationsite/DownloadingandinstallingOpenCVURL,fordevelopmentrepository/DownloadingandinstallingOpenCVURL,fortutorials/DownloadingandinstallingOpenCVcompiler,obtaining/GettingacompilerandsettingCMakeconfiguring,withCMake/ConfiguringOpenCVwithCMakestructure/ThestructureofOpenCVreferencelink,formodules/ThestructureofOpenCVuserprojects,creatingwith/CreatinguserprojectswithOpenCV

OpenCV,withOpenCLinstallationprocess/AquickrecipetoinstallOpenCVwithOpenCL

OpenCVAPIURL/Writingimagefiles

OpenCVapplicationdeveloping,withC++/Generalusageofthelibrary

OpenCVC++applicationsprerequisites/Toolstodevelopnewprojects

OpenCVC++programcreating,withQtCreator/CreatinganOpenCVC++programwithQtCreator

operations,withimagesabout/Commonoperationswithimages

opticalflowmethodsbrox/Superresolutionpyrlk/Superresolutionfarneback/Superresolutiontvl1/Superresolutioncomparing/Superresolution

Ppaniniprojection/Stitchingparameters,BilateralTV-L1algorithm

scale/Superresolutioniterations/Superresolutiontau/Superresolutionlamba/Superresolutionalpha/SuperresolutionbtvKernelSize/SuperresolutionblurKernelSize/SuperresolutionblurSigma/SuperresolutiontemporalAreaRadius/SuperresolutionopticalFlow/Superresolution

perspectivetransformationabout/Perspectivetransformationfunctions/Perspectivetransformationexamplecode/Theexamplecode

pfstools/Tonemappingpixel-levelaccess

about/Pixel-levelaccessprerequisites,OpenCVC++applications

OpenCVheaderfilesandlibrarybinaries/ToolstodevelopnewprojectsC++compiler/Toolstodevelopnewprojectsauxiliarylibraries/Toolstodevelopnewprojects

pyramidabout/Workingwithimagepyramids

QQtbundle

URL,fordownloading/ToolstodevelopnewprojectsQtCreator

about/ToolstodevelopnewprojectsOpenCVC++program,creatingwith/CreatinganOpenCVC++programwithQtCreator

Qtframeworkabout/GettingacompilerandsettingCMakeURL/GettingacompilerandsettingCMake

QtprojectURL/CreatinganOpenCVC++programwithQtCreator

RRANSACmethod

URL/VideostabilizationRapidEnvironmentEditortool

URL/GettingacompilerandsettingCMakerectilinearprojection/Stitchingreflection

about/Reflectionexamplecode/Theexamplecode

requisites,forinstallingOpenCVwithOpenCLabout/OpenCVwiththeOpenCLinstallationOpenCL-capableGPUorCPU/OpenCVwiththeOpenCLinstallationcompilers/OpenCVwiththeOpenCLinstallationAMDAPPSDK/OpenCVwiththeOpenCLinstallationOpenCLBLAS(BasicLinearAlgebraSubroutines)/OpenCVwiththeOpenCLinstallationOpenCLFFT(FastFourierTransform)/OpenCVwiththeOpenCLinstallationQtlibraries,forC++compiler/OpenCVwiththeOpenCLinstallation

RGBabout/RGBexamplecode/Theexamplecode

Sscaling

about/Scalingexamplecode/Theexamplecode

seamlesscloningabout/Seamlesscloning

seamlessCloningexampleabout/Seamlesscloningfunctions/Seamlesscloning

sharpeningabout/Sharpeningfunctions/Sharpeningexamplecode/Theexamplecode

skewingabout/Skewingexamplecode/Theexamplecode

smoothingabout/Smoothingfunctions/Smoothingexamplecode/Theexamplecode

sphericalprojection/Stitchingstandardtemplatelibrary(STL)

about/ThebasicAPIconceptsstereographicprojection/Stitchingsticher/Stitchingsuperresmodule/Superresolutionsuperresolution

about/SuperresolutionURL,forexample/Superresolutionexample/Superresolution

TThreadingBuildingBlocks(TBB)

about/AnintroductiontoOpenCVtonemapping,HDRimages/Tonemappingtrackbars

about/Trackbarstranslation

about/Translationexamplecode/Theexamplecode

TVL1(TotalVariationL1)about/Denoising

UUnifiedMats(UMat)/Thecodeexplanationuser-interactionstools

about/User-interactionstoolstrackbars/Trackbarsmouseinteraction/Mouseinteractionbuttons/Buttonstext,drawing/Drawinganddisplayingtexttext,displaying/Drawinganddisplayingtext

userinterface(UI)/ThestructureofOpenCVuserprojects

creating,withOpenCV/CreatinguserprojectswithOpenCV

Vvideofiles

writing/Readingandwritingvideofilesreading/ReadingandwritingvideofilesrecVideoexample/Theexamplecode

videostabilizationabout/Videostabilizationmechanicalstabilizationsystems/Videostabilizationdigitalstabilizationsystems/VideostabilizationvideoStabilizerexample/Videostabilization

videostabilizationalgorithmssteps/Videostabilization

WWindows/Compilingandinstallingthelibrary

YYCrCb

about/YCrCbexamplecode/Theexamplecode

YCrCbsegmentationabout/YCrCbsegmentation

Learning Image Processing with OpenCV - DropPDFTools to develop new projects Creating an OpenCV C++...

Documents

Transcript of Learning Image Processing with OpenCV - DropPDFTools to develop new projects Creating an OpenCV C++...

INDEX ∞ Image Processing ∞ OpenCV ∞ Download & Setup ∞ Make Project ∞ Show Result ∞ Q & A Setup OpenCV 2.4.2 & Tutorial.

OpenCV · 2018-04-18 · OpenCV 5 OpenCV is a cross-platform library using which we can develop real-time computer vision applications. It mainly focuses on image processing, video

· blueprints using TensorAow, OpenCV augmentation, and a a-box label tool THE NITOR APPROACH TECHNOLOGY TensorFlow OpenCV Object Recognition Created an image-detection algorithm

Arles Image Web Page Creator Help -

Introduction to Image Processing using OpenCV

Intro to the OpenCV Library€¦ · The OpenCV Library Mat - The Basic Image Container How to scan images, lookup tables and time measurement with OpenCV Mask operations on matrices

Introduction to OpenCV - RealTechSupport1-2. Include OpenCV headers first 7. IplImage is OpenCV image type 7, 19. Use cvLoadImage and cvSaveImage to load/save an image in multiple

Arles Image Web Page Creator Help

Exploring the Opencv Library for Image Processing in Long ...

OPENCV/CV BRIDGEchrome.ws.dei.polimi.it/images/4/47/Robotics_L9_2019_ex.pdf · Add cv_bridge and image transport Add OpenCV Include both opencv and catkin library Link both opencv

INDEX ∞ Image Processing ∞ OpenCV ∞ Download & Setup ∞ Make Project ∞ Show Result ∞ Q & A

OpenCV and Visual c++ Programming in image processing

Image Processing in OpenCV-Emgu

Introduction to OpenCV Marvin Smith. Introduction OpenCV is an Image Processing library created by Intel and maintained by Willow Garage. Available for.

Arles Image Web Page Creator Help - DigitalDutch.com Arles Image Web Page Creator Help Digital Dutch Table of Contents Part IAbout 1 Part IIGetting Started Creating a Image Web Page

Emgu CV - Robofest · What is Emgu CV? A cross platform .Net wrapper to the OpenCV image processing library. Allowing OpenCV functions to be called …

2D Image Processing Introduction to OpenCV

Learning Image Processing with OpenCV - Sample Chapter

Practical Python and OpenCV: An Introductory, Example ... · OpenCV: An Introductory, Example Driven Guide to Image Processing and Computer Vision 3rd Edition Dr. Adrian Rosebrock.

— Basic Concepts and Introduction of OpenCV —swk/lecture/ic2014/kagami_ic...Intelligent Control Systems Image Processing (1) — Basic Concepts and Introduction of OpenCV — Shingo