Post on 19-Jun-2020
SiameseNeuralNetworkbasedGaitRecognitionforHumanIdentification
ChengZhang,WuLiu,Huadong Ma,Huiyuan FuBeijingUniversityofPostsandTelecommunications
ICASSP2016
Outline
• Introduction• Proposedmethod– ConventionalCNNbasedGaitRecognition– SiameseNetworkbasedGaitRecognition
• Experiments• Conclusions
Definition• Gait analysis is the systematic study of animal locomotion,
more specifically the study of human motion, using the eyeand the brain of observers, augmented by instrumentation formeasuring body movements, body mechanics, and theactivity of the muscles.
Background
• Socialsecurity– Videobigdataandcameranetwork– Remotesurveillance– Identificationandattributeclassification
• Biometricauthenticationtechniques– Facialrecognition– Irisrecognition– Fingerprinttechnologies– Voiceverification– Handgeometry
Characteristics
• Remoteaccessed– Itcanidentifysubjectsfromadistancewithoutinterruptingthesubject
• Robust– Eveninlowresolutionvideos,thegaitstillworkswell
• Security– Itisdifficulttoimitateorcamouflagehumangait
iris fingerprintface voice gait
Whygaitworks?
• A plethora of technique and data continue to showthat a person’s walking is indeed unique
MurrayMP,DroughtAB,KoryRC.Walkingpatternsofnormalmen[J].TheJournalofBone&JointSurgery,1964,46(2):335-360.JohanssonG.Visualperceptionofbiologicalmotionandamodelforitsanalysis[J].Attention,Perception,&Psychophysics,1973,14(2):201-211.
Challenges• Inconspicuousinter-classdifferencefromthedifferentpeople
• Thelargeintra-classvariationsfromthesameperson– Walkingspeeds– Viewpoints– Clothing– Belongings– Occasion normal clothes backpack
Gaitsilhouettesofdifferentsubject
RecentEffortsandMajorDrawback• Model-basedmethods
– Extractinghumanbodystructurefromtheimages– Requiringahighresolutionaswellashighercomputationalcostand
arenotyetsuitableforoutdoorsurveillance
• Model-freemethods– Usingthewholemotionpattern/featuresofthehumanbody,and
performingrecognitionatlowerresolutions– Human-craftedgaitfeaturescanextremelyhardtobreakthrough
featurerepresentationbottleneckwhenfacingwiththegaitandappearancechanges
GeneralStepsofOurSystem
GaitEnergyImage
• Averagingofsilhouetteoveronegaitcycle– Representahumanmotionsequenceinasingleimagewhilepreservingtemporalinformation
– Robusttoincidentalsilhouetteerrorsinindividualimage
ConventionalCNNbasedGR
• RetraintheCNNsonthegaitdataset– CNNsareabletolearndiscriminativefeatures– Fine-tuningfromapre-trainedmodel(e.g.,AlexNet)isagoodsolutiontosolvethedatalimitationproblemandspeeduptheconvergenceofnewmodel
– EmploytheAlexNet andonlychangethe1,000labeloutputtothenumberofsubjectsingaitdataset
Problems
• Datalimitation– Tolearnsufficientfeatures,theCNNrequiresamassoftrainingdataforallcategories
– Forgaitrecognition,thenumberofsubjectscanbelarge,whilewithonlyafewexamplespersubjectinpublicdatabase
• Domaingap– Gaitrecognitionforhumanidentificationisessentiallyasearchproblembutnotclassification
MetricLearning
= ≠
ProposedFramework
• SiameseNeuralNetworkbasedgaitrecognition
Sampling
• Trainingdataishighlyunbalanced– Usingasamplertogenerateequalnumberofpositiveandnegativeineachmini-batch,avoidoverlybiased towardstonegativedecisions
– Usingasamplertoenforcevarietytopreventoverfitting toalimitednegativeset
• Specially,thetrainingsetisselectedfromOULP-C1V1-A-Gallery dataset,with20,000similarGEIpairsandrandomlyselected20,000dissimilarpairs
LossFunction
• ThedistancebetweenapairofGEIscanbemeasuredby:
• Wecandefinethecontrastivefunctionasfollows:
TrainingandFeatureExtraction
• Supervisedsetting• MinimizedthecontrastivelossfunctionoveratrainingsetofNpatchpairsusingstochasticgradientdescent
• Experimentedwithdifferentparametersandgavethebestperformanceoffeaturerepresentation
Experiments• Database:OU-ISIRLargePopulation
• Evaluation:Rank-1andRank-5identificationrates
• Baselines:STOAgaitrecognitionmethods,i.e.,GEI,FDF,HWLD,VTM,andRankSVM
• Pipeline:Backgroundsegmentation->Periodicidentification->GEIsgeneration->DNNtraining->DNNfeatureextraction->K-Nearest-Neighborsearching
Database• OU-ISIRLargePopulationGaitDatabase
– Containstheworld’slargestnumberofsubjects– Recordstwosequencesforeachsubject:probe(query)andgallery
(source)sequence,offersfaircomparisontestbed
Intra-viewrecognition
[9]H.Iwama,M.Okumura,Y.Makihara,andY.Yagi,“Theou-isirgaitdatabasecomprisingthelargepopulationdatasetandperformanceevaluationofgaitrecognition,”IEEETIFS.[7]Sivapalan,D.Chen,S.Denman,S.Sridharan,andC.Fookes,“Histogramofweightedlocaldirectionsforgaitrecognition,”inCVPRW,2013.
SomeResults
Inter-viewrecognition
D.Muramatsu,A.Shiraishi,Y.Makihara,M.Uddin,andY.Yagi,“Gait-basedpersonrecognitionusingarbitraryviewtransformationmodel,”IEEETIP,2015.R.Mart´ın-F´elez andT.Xiang,“Gaitrecognitionbyranking,”inECCV,2012.
Conclusions• Wepresentoneofthefirstattemptstostudythedeepneural
networkbasedgaitrecognitionforhumanidentificationwithdistancemetriclearning
• Intheend-to-endframework,weleveragethecompetitiveGEIpresentationastheinputofnetworkwhileholisticallyexploittheSiameseneuralnetworktolearneffectivefeaturerepresentationsforhumanidentification
• Thecomprehensiveevaluationsshowthatweimpressivelyoutperformthestate-of-the-artsontheworld’slargestchallengegaitbenchmarkdataset
FutureWorks
• 3-DimensionalSiameseneuralnetwork• Quasi-periodicorsub-framegaitrecognition• Unconstrainedenvironment,likeilluminationchanges,darkillumination,clutteredbackground,motionblur,andimagecompressionnoise
• …
“High’st Queenofstate,GreatJunocomes;Iknowherbyhergait”—— TheTempest[Act4Scene1],Shakespeare
Anyquestions?