Siamese Neural Network based Gait Recognition for Human...
Transcript of Siamese Neural Network based Gait Recognition for Human...
![Page 1: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/1.jpg)
SiameseNeuralNetworkbasedGaitRecognitionforHumanIdentification
ChengZhang,WuLiu,Huadong Ma,Huiyuan FuBeijingUniversityofPostsandTelecommunications
ICASSP2016
![Page 2: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/2.jpg)
Outline
• Introduction• Proposedmethod– ConventionalCNNbasedGaitRecognition– SiameseNetworkbasedGaitRecognition
• Experiments• Conclusions
![Page 3: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/3.jpg)
Definition• Gait analysis is the systematic study of animal locomotion,
more specifically the study of human motion, using the eyeand the brain of observers, augmented by instrumentation formeasuring body movements, body mechanics, and theactivity of the muscles.
![Page 4: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/4.jpg)
Background
• Socialsecurity– Videobigdataandcameranetwork– Remotesurveillance– Identificationandattributeclassification
• Biometricauthenticationtechniques– Facialrecognition– Irisrecognition– Fingerprinttechnologies– Voiceverification– Handgeometry
![Page 5: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/5.jpg)
Characteristics
• Remoteaccessed– Itcanidentifysubjectsfromadistancewithoutinterruptingthesubject
• Robust– Eveninlowresolutionvideos,thegaitstillworkswell
• Security– Itisdifficulttoimitateorcamouflagehumangait
iris fingerprintface voice gait
![Page 6: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/6.jpg)
Whygaitworks?
• A plethora of technique and data continue to showthat a person’s walking is indeed unique
MurrayMP,DroughtAB,KoryRC.Walkingpatternsofnormalmen[J].TheJournalofBone&JointSurgery,1964,46(2):335-360.JohanssonG.Visualperceptionofbiologicalmotionandamodelforitsanalysis[J].Attention,Perception,&Psychophysics,1973,14(2):201-211.
![Page 7: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/7.jpg)
Challenges• Inconspicuousinter-classdifferencefromthedifferentpeople
• Thelargeintra-classvariationsfromthesameperson– Walkingspeeds– Viewpoints– Clothing– Belongings– Occasion normal clothes backpack
Gaitsilhouettesofdifferentsubject
![Page 8: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/8.jpg)
RecentEffortsandMajorDrawback• Model-basedmethods
– Extractinghumanbodystructurefromtheimages– Requiringahighresolutionaswellashighercomputationalcostand
arenotyetsuitableforoutdoorsurveillance
• Model-freemethods– Usingthewholemotionpattern/featuresofthehumanbody,and
performingrecognitionatlowerresolutions– Human-craftedgaitfeaturescanextremelyhardtobreakthrough
featurerepresentationbottleneckwhenfacingwiththegaitandappearancechanges
![Page 9: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/9.jpg)
GeneralStepsofOurSystem
![Page 10: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/10.jpg)
GaitEnergyImage
• Averagingofsilhouetteoveronegaitcycle– Representahumanmotionsequenceinasingleimagewhilepreservingtemporalinformation
– Robusttoincidentalsilhouetteerrorsinindividualimage
![Page 11: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/11.jpg)
ConventionalCNNbasedGR
• RetraintheCNNsonthegaitdataset– CNNsareabletolearndiscriminativefeatures– Fine-tuningfromapre-trainedmodel(e.g.,AlexNet)isagoodsolutiontosolvethedatalimitationproblemandspeeduptheconvergenceofnewmodel
– EmploytheAlexNet andonlychangethe1,000labeloutputtothenumberofsubjectsingaitdataset
![Page 12: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/12.jpg)
Problems
• Datalimitation– Tolearnsufficientfeatures,theCNNrequiresamassoftrainingdataforallcategories
– Forgaitrecognition,thenumberofsubjectscanbelarge,whilewithonlyafewexamplespersubjectinpublicdatabase
• Domaingap– Gaitrecognitionforhumanidentificationisessentiallyasearchproblembutnotclassification
![Page 13: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/13.jpg)
MetricLearning
= ≠
![Page 14: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/14.jpg)
ProposedFramework
• SiameseNeuralNetworkbasedgaitrecognition
![Page 15: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/15.jpg)
Sampling
• Trainingdataishighlyunbalanced– Usingasamplertogenerateequalnumberofpositiveandnegativeineachmini-batch,avoidoverlybiased towardstonegativedecisions
– Usingasamplertoenforcevarietytopreventoverfitting toalimitednegativeset
• Specially,thetrainingsetisselectedfromOULP-C1V1-A-Gallery dataset,with20,000similarGEIpairsandrandomlyselected20,000dissimilarpairs
![Page 16: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/16.jpg)
LossFunction
• ThedistancebetweenapairofGEIscanbemeasuredby:
• Wecandefinethecontrastivefunctionasfollows:
![Page 17: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/17.jpg)
TrainingandFeatureExtraction
• Supervisedsetting• MinimizedthecontrastivelossfunctionoveratrainingsetofNpatchpairsusingstochasticgradientdescent
• Experimentedwithdifferentparametersandgavethebestperformanceoffeaturerepresentation
![Page 18: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/18.jpg)
Experiments• Database:OU-ISIRLargePopulation
• Evaluation:Rank-1andRank-5identificationrates
• Baselines:STOAgaitrecognitionmethods,i.e.,GEI,FDF,HWLD,VTM,andRankSVM
• Pipeline:Backgroundsegmentation->Periodicidentification->GEIsgeneration->DNNtraining->DNNfeatureextraction->K-Nearest-Neighborsearching
![Page 19: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/19.jpg)
Database• OU-ISIRLargePopulationGaitDatabase
– Containstheworld’slargestnumberofsubjects– Recordstwosequencesforeachsubject:probe(query)andgallery
(source)sequence,offersfaircomparisontestbed
![Page 20: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/20.jpg)
Intra-viewrecognition
[9]H.Iwama,M.Okumura,Y.Makihara,andY.Yagi,“Theou-isirgaitdatabasecomprisingthelargepopulationdatasetandperformanceevaluationofgaitrecognition,”IEEETIFS.[7]Sivapalan,D.Chen,S.Denman,S.Sridharan,andC.Fookes,“Histogramofweightedlocaldirectionsforgaitrecognition,”inCVPRW,2013.
![Page 21: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/21.jpg)
SomeResults
![Page 22: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/22.jpg)
Inter-viewrecognition
D.Muramatsu,A.Shiraishi,Y.Makihara,M.Uddin,andY.Yagi,“Gait-basedpersonrecognitionusingarbitraryviewtransformationmodel,”IEEETIP,2015.R.Mart´ın-F´elez andT.Xiang,“Gaitrecognitionbyranking,”inECCV,2012.
![Page 23: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/23.jpg)
Conclusions• Wepresentoneofthefirstattemptstostudythedeepneural
networkbasedgaitrecognitionforhumanidentificationwithdistancemetriclearning
• Intheend-to-endframework,weleveragethecompetitiveGEIpresentationastheinputofnetworkwhileholisticallyexploittheSiameseneuralnetworktolearneffectivefeaturerepresentationsforhumanidentification
• Thecomprehensiveevaluationsshowthatweimpressivelyoutperformthestate-of-the-artsontheworld’slargestchallengegaitbenchmarkdataset
![Page 24: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/24.jpg)
FutureWorks
• 3-DimensionalSiameseneuralnetwork• Quasi-periodicorsub-framegaitrecognition• Unconstrainedenvironment,likeilluminationchanges,darkillumination,clutteredbackground,motionblur,andimagecompressionnoise
• …
“High’st Queenofstate,GreatJunocomes;Iknowherbyhergait”—— TheTempest[Act4Scene1],Shakespeare
![Page 25: Siamese Neural Network based Gait Recognition for Human ...web.cse.ohio-state.edu/~zhang.7804/icassp2016.pdf · network based gait recognition for human identification with distance](https://reader035.fdocuments.net/reader035/viewer/2022070810/5f08d5437e708231d423f110/html5/thumbnails/25.jpg)
Anyquestions?