CMPT 888 – Human Activity Recognition
Transcript of CMPT 888 – Human Activity Recognition
![Page 1: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/1.jpg)
CMPT888–HumanActivityRecognition
GregMori
![Page 2: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/2.jpg)
Outline
• Introtoclass• Administrativedetails
![Page 3: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/3.jpg)
Overview
• Thisclassisaboutvision‐basedactionrecognition– Inputisimagesorvideos– Outputisdescriptionofwhatpeoplearedoingintheimages/videos
![Page 4: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/4.jpg)
ActionRecognitionExample
• Recognizehumanactionsfromrawvideodata
![Page 5: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/5.jpg)
Gatheringactiondata
• 3components:– detecthumans,track,recognizeaction
![Page 6: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/6.jpg)
6
ApplicationsI• Automatedvideosurveillance– Drawattentiontoactionsofinterest– Savehumanoperatortime
![Page 7: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/7.jpg)
ApplicationsII
• Collectdataonpedestrianbehaviour– CollaborationwithSaunierandSayed(UBCCivilEngineering)
![Page 8: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/8.jpg)
ApplicationsIII
Automaticallydetectfalls,near‐falls(withS.RobinovitchSFU)
![Page 9: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/9.jpg)
WhyuseComputerVision?
• Competingapproaches– Wearablesensors
– Manuallabour
• Non‐intrusive– Donotneedcooperativesubjects
• Inexpensive,nooperatorfatigue– Semi‐automatictechniques
![Page 10: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/10.jpg)
PROBLEMDEFINITION
![Page 11: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/11.jpg)
WhatisActionRecognition?
• Terminology– Whatisan“action”?
• Outputrepresentation– Whatdowewanttosayaboutanimage/video?
Unfortunately,neitherquestionhassatisfactoryansweryet
![Page 12: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/12.jpg)
Terminology
• Theterms“actionrecognition”,“activityrecognition”,“eventrecognition”,areusedinconsistently– Findingacommonlanguagefordescribingvideosisanopenproblem
![Page 13: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/13.jpg)
TerminologyExample
• “Action”isalow‐levelprimitivewithsemanticmeaning– E.g.walking,pointing,placinganobject
• “Activity”isahigher‐levelcombinationwithsometemporalrelations– E.g.takingmoneyoutfromATM,waitingforabus
• “Event”isacombinationofactivities,ofteninvolvingmultipleindividuals– E.g.asoccergame,atrafficaccident
• Thisiscontentious– Nostandard,rigorousdefinitionexists
![Page 14: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/14.jpg)
OutputRepresentation
• Thisimagecontainsamanwalking– Actionclassification/recognition
• Themanwalkingishere– Actiondetection
• Giventhisimagewhatisthedesiredoutput?
![Page 15: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/15.jpg)
OutputRepresentation
• Thisimagecontains5menwalking,4jogging,2running
• The5menwalkingarehere
• Thisisasoccergame
• Giventhisimagewhatisthedesiredoutput?
![Page 16: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/16.jpg)
OutputRepresentation
• Giventhisvideowhatisthedesiredoutput?
• Frames1‐20themanrantotheleft,thenframes21‐25heranawayfromthecamera
• Isthisanaccuratedescription?• Arelabelsandvideoframesin1‐1correspondence?
![Page 17: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/17.jpg)
ChallengesinRecognition
• Intra‐classvariation• Objectposevariation• Backgroundclutter• Occlusion• Lighting
![Page 18: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/18.jpg)
TRIMESTERPREVIEW
![Page 19: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/19.jpg)
Week2
• Preliminaries– Humandetection
– Backgroundsubtraction– Opticalflow
Dalal+TriggsCVPR05
![Page 20: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/20.jpg)
Weeks3‐4
• MotionTemplates
BobickandDavisPAMI01
Efrosetal.ICCV03
![Page 21: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/21.jpg)
Weeks5‐6
• Localfeaturevideorepresentations
Schuldtetal.ICPR04Dollaretal.VSPETS05
![Page 22: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/22.jpg)
Week7
• Unsupervisedandweaklysupervisedmethods
Laptevetal.CVPR08
![Page 23: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/23.jpg)
Week8
• Temporalmodels
?? ? ? ? ? ? ? ? ?
WangandMoriPAMI09
![Page 24: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/24.jpg)
Week9
• Humanposeestimationandposeretrieval
Yangetal.CVPR10
![Page 25: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/25.jpg)
Week10
• Discriminativemethods
Runright Walkleft Runright45
FathiandMoriCVPR08
![Page 26: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/26.jpg)
Week11• Humanactionsinstillimages
SLAGWangetal.CVPR06
![Page 27: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/27.jpg)
ADMINISTRIVIA
![Page 28: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/28.jpg)
CoursePlan
• Readresearchpapers– ForeachtopicIpresentimportantpapers
– Studentseachpresentarecentpaper– Wediscuss
• Doaproject– Gainin‐depthexperienceonaproblemandalgorithm
![Page 29: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/29.jpg)
Introductions
![Page 30: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/30.jpg)
Prerequisite
• Noformalprerequisites– Butitwouldbebestifyouknowsomecomputervision/imageprocessingandsomemachinelearning
• Youwillneedtodotheusualthings– Math(continuous),programming,reading,writing,presenting
• Askmeifyouareconcerned
![Page 31: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/31.jpg)
GradingScheme
• 10%Classparticipation– Participateindiscussionsaboutpapers,ask/answerquestions
• 10%Readingassignments– 1or2paperseachweek;subsetoftheonesIpresent
• 10%Paperpresentation– Choosefromlistofpapersonline
• 10%Assignment– Smallprogrammingassignmentonmotionanalysis
• 60%Project– Individualorinsmallgroups– Presentation,writtenreport
![Page 32: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/32.jpg)
ReadingAssignments
• Similartominipaperreview– Oneparagraphsummarizingpaper
– Criticaldiscussion(whatyoulike/don’tlike)– Questionsyouhave(formetoexplain)
• Duebeforestartoflectureviaemail– FirstonedueMonday
• Thesedetailsandlistofpapersareonline
![Page 33: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/33.jpg)
PaperPresentations
• Chooseonepaperthatinterestsyou– Fromlistonline/insyllabus
• 20minutepresentation– 10+minutesquestions/discussion– Feelfreetouseslidesprovidedbyauthors
![Page 34: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/34.jpg)
Assignment
• Shortprogrammingassignment– Backgroundsubtraction– Motion‐basedactionrecognition
• Outnextweek,due2weekslater
![Page 35: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/35.jpg)
Project
• Majorcomponentofcourse– Recognizeactions
• Implementexistingtechnique– Orvariantthereof– Canusesomethingyou’reworkingoninyourresearch• Mustrecognizeactions• Mustdosomethingthatdidn’texistbeforethiscourse
• Proposal,presentation,report
![Page 36: CMPT 888 – Human Activity Recognition](https://reader030.fdocuments.net/reader030/viewer/2022012113/61dcd64aeeb18b6c9e1e9c6d/html5/thumbnails/36.jpg)
CoursePlan
• Nextweek– Preliminaries
• Backgroundsubtraction,humandetection,motion
• Afterthat– Papers,papers,papers