CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20:...
Transcript of CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20:...
![Page 1: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/1.jpg)
Lecture0:Introduction
CSE252C:AdvancedComputerVision
ManmohanChandraker
CSE252C,SP20:ManmohanChandraker
![Page 2: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/2.jpg)
Virtualclassrooms
• VirtuallecturesonZoom
– Onlyhostsharesthescreen
– Keepvideoturnedoffincaseofbandwidthissues
– Microphonesmutedunlessspeakingtoreducenoise
– Butpleasedospeakup(remembertounmute!)
– Slidestobeuploadedonwebpagejustbeforeclass
– WhiteboardthroughcameraandiPadtowritethings
• LecturesrecordedanduploadonKaltura
– Availableunder“MyMedia”onCanvas
CSE252C,SP20:ManmohanChandraker
![Page 3: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/3.jpg)
Virtualclassrooms
• VirtualinteractionsonZoom
– Askandanswerplentyofquestions
– Trytohavein-classdiscussions
– “Raisehand”featureonZoomwhenyouwishtospeak
– InstructororTAscanunmute
– Postquestionsonchatwindow
– TAwillhelpkeeptrackofraisedhandsandchatwindow
– Ifinstructorcannotnotice,TAwillinterruptatlogicalpause
• Happytotryothersuggestions
CSE252C,SP20:ManmohanChandraker
![Page 4: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/4.jpg)
Enrollmentlogistics
• Waitlist
– Thereisalongwaitlistfortheclass
– Departmenthasnotagreedtoaddmorestudents
– Youarewelcometoattendlecturesevenifonwaitlist
– TolimitTAworkload,wecangradeonlyenrolledstudents
• Canvas
– Allenrolledandwaitlistedstudentsshouldhaveaccess
• AllannouncementswillbepostedonPiazza
– SendemailtoTAs(CCinstructor)ifdidnotgetPiazzainvite
CSE252C,SP20:ManmohanChandraker
![Page 5: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/5.jpg)
ComputerVision
CSE252C,SP20:ManmohanChandraker
![Page 6: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/6.jpg)
Definingcomputervision
Wall-E: Fact and Fiction (Minh Do, Princeton University)CSE252C,SP20:ManmohanChandraker
![Page 7: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/7.jpg)
Studyingcomputervision
Source:2Pmedia
• Visionisafundamentalinterfacetotheworld
CSE252C,SP20:ManmohanChandraker
![Page 8: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/8.jpg)
Studyingcomputervision
Source:2Pmedia
• Visionisafundamentalinterfacetotheworld
CSE252C,SP20:ManmohanChandraker
Humans
Machines
Virtual
(Cyborgs!)
![Page 9: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/9.jpg)
• Visionisafundamentalinterfacetotheworld
• Weareallusersofcomputervision
Studyingcomputervision
Transportation Design
Communication E-CommerceCSE252C,SP20:ManmohanChandraker
![Page 10: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/10.jpg)
Studyingcomputervision• Visionisafundamentalinterfacetotheworld
• Weareallusersofcomputervision
• Deepandattractivescientificproblems• Howdowerecognizeobjects?
• Whydonewbornbabiesrespondtoface-likeshapes?
[Farroni etal.,2005]
CSE252C,SP20:ManmohanChandraker
![Page 11: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/11.jpg)
Definingcomputervision
CSE252C,SP20:ManmohanChandraker
![Page 12: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/12.jpg)
Definingcomputervision
CSE252C,SP20:ManmohanChandraker
Ourproblemsbecamemorecomplex
![Page 13: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/13.jpg)
Definingcomputervision
CSE252C,SP20:ManmohanChandraker
Ourambitionsbecamehigher
![Page 14: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/14.jpg)
Definingcomputervision
CSE252C,SP20:ManmohanChandraker
Ourconceptschangedovertime
![Page 15: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/15.jpg)
Definingcomputervision
CSE252C,SP20:ManmohanChandraker
Ourtoolsadvancedovertime
[Mahony etal.,2019]
![Page 16: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/16.jpg)
Gainingperspectiveoncomputervision
CSE252C,SP20:ManmohanChandraker
ImportantforAutopilot:Donothitapolice car!
![Page 17: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/17.jpg)
Gainingperspectiveoncomputervision
CSE252C,SP20:ManmohanChandraker
ImportantforAutopilot:Donothitapolice car!
ImportantforRobots:Stayclearofhumans!
[Wired.com]
![Page 18: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/18.jpg)
Gainingperspectiveoncomputervision
CSE252C,SP20:ManmohanChandraker
ImportantforAutopilot:Donothitapolice car!
ImportantforRobots:Stayclearofhumans!
[Wired.com]
![Page 19: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/19.jpg)
Autonomousdriving
Source:WiredCSE252C,SP20:ManmohanChandraker
![Page 20: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/20.jpg)
AutonomousdrivingWhereisourcar?
StructurefromMotion
VisualSLAM
Whereareotheragents?
Objectdetection
3Dlocalization
Wherearesceneelements?
Semanticsegmentation
Whatisasafepath?
Behaviorprediction
Pathplanning
CSE252C,SP20:ManmohanChandraker
![Page 21: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/21.jpg)
Afewtopicsinthiscourse
• StructureandMotion
• FacesandHumans
• ObjectsandStuff
• MaterialandLighting
• BiasandPrivacy
CSE252C,SP20:ManmohanChandraker
![Page 22: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/22.jpg)
StructureandMotion
CSE252C,SP20:ManmohanChandraker [Agarwaletal.,ICCV2009]
![Page 23: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/23.jpg)
StructureandMotion
CSE252C,SP20:ManmohanChandraker[SongandChandraker,CVPR2015]
![Page 24: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/24.jpg)
FacesandHumans
CSE252C,SP20:ManmohanChandraker
[Blanz andVetter]
[Microsoftblog, 2018]
![Page 25: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/25.jpg)
FacesandHumans
CSE252C,SP20:ManmohanChandraker [Kanazawaetal.,CVPR2019]
![Page 26: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/26.jpg)
ObjectsandStuff
CSE252C,SP20:ManmohanChandraker [Choi,ICCV2015]
![Page 27: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/27.jpg)
ObjectsandStuff
CSE252C,SP20:ManmohanChandraker
[Wangetal.,CVPR2019]
[Leeetal.,ICCV2017]
![Page 28: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/28.jpg)
MaterialandLightingInputimage Augmented image
CSE252C,SP20:ManmohanChandraker [Lietal.,CVPR2020]
![Page 29: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/29.jpg)
MaterialandLighting
CSE252C,SP20:ManmohanChandraker [Li,Yeh,etal.,CVPR2020]
![Page 30: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/30.jpg)
BiasandPrivacy
CSE252C,SP20:ManmohanChandraker
Caucasian African-American East-Asian
Proportion in datasets: 80% Proportion in datasets: 10% Proportion in datasets: 5%
High accuracy Low accuracy Low accuracyTraining on biased data without domain adaptation
High accuracy High accuracy High accuracyUsing domain adaptation to address dataset bias
![Page 31: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/31.jpg)
BiasandPrivacy
CSE252C,SP20:ManmohanChandraker
Noadaptation
Withadaptation
Labelsrequire1.5hours perimage!
Sourcedomain:goodweather,with labels
Targetdomain:rainyweather,no labels
Trainonsource,apply ontarget
Trainonsource,adapt totarget
[Tsaietal.,ICCV2019]
![Page 32: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/32.jpg)
BiasandPrivacy
CSE252C,SP20:ManmohanChandraker
America
Collect
data
Label
data
Train
modelDeployExisting
solution
Newmarket
Newmarket
....
....
....
....
Asia
Collect
data
1week
Label
data
Adapt
model
1week
Deploy
Africa
Collect
data
Label
data
Adapt
modelDeploy
1week 1year 1week
1week 1week
![Page 33: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/33.jpg)
BiasandPrivacy
CSE252C,SP20:ManmohanChandraker [Xiaoetal.,AAAI2020]
![Page 34: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/34.jpg)
Overallgoalsforthecourse
• Introducethestate-of-the-artincomputervision
• Studyprinciplesthatmakethempossible
• Getunderstandingoftoolsthatdrivecomputervision
• Enableoneorallofseveralsuchoutcomes
– Pursuehigherstudiesincomputervision
– Joinindustrytodocutting-edgeworkinAI
– GainanappreciationofmodernAItechnologies
CSE252C,SP20:ManmohanChandraker
![Page 35: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/35.jpg)
CourseDetails
CSE252C,SP20:ManmohanChandraker
![Page 36: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/36.jpg)
Coursedetails• Eachclasswillcovertopicsincomputervision
• Examplesoftopics
• Correspondence
• Stereo
• Opticalflow
• Structurefrommotion
• Facerecognition
• Humanposeestimation
• Materialandlighting
• Semanticsegmentation
• Objectdetection
• Tracking
• Actionrecognition
• Domainadaptation
• Privacyandfairness
CSE252C,SP20:ManmohanChandraker
![Page 37: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/37.jpg)
Coursedetails
• Topicsstructuredintoafewmodules
– Background
– StructureandMotion
– FacesandHumans
– ObjectsandStuff
– MaterialandLighting
– AdaptationandBias
CSE252C,SP20:ManmohanChandraker
![Page 38: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/38.jpg)
Coursedetails
• “Lightning”presentations
– Fourstudentstopresentinoneclass
– Timelimit:5minutes
– Paperstobeassignedbyinstructor
– Orderofpresentation:alphabetic
• Sendpresentation1daybeforeclass
– Well-practicedandfluentpresentation
– Includenarrationifasynchronous
– Askandanswerquestionsafterpresentation
CSE252C,SP20:ManmohanChandraker
![Page 39: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/39.jpg)
Coursedetails
• Presentationformat(1slideforeach):
1.Motivationandproblemdescription
2.Priorwork
3.Methodoverview
4.Methodanalysis
5.Experiments
6.Futureworkanddiscussion
CSE252C,SP20:ManmohanChandraker
![Page 40: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/40.jpg)
Coursedetails• Classwebpage:
– http://cseweb.ucsd.edu/~mkchandraker/classes/CSE252C/Spring2020/
• Instructoremail:
• TAs:Zhengqin LiandYou-YiJau
– Emails:[email protected] [email protected]
• Grading
– 10%presentation
– 60%assignments
– 30%finalexam
• Aimistolearntogether,discussandhavefun!
CSE252C,SP20:ManmohanChandraker
![Page 41: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/41.jpg)
MyInterests
CSE252C,SP20:ManmohanChandraker
![Page 42: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/42.jpg)
Sceneunderstandingforself-drivingLearningtosimulate
Large-scale,real-time,monocularSFM.
Accuracycomparabletostereosystems.
Teacher
CNN
Student
CNN
ImageHint
loss
Student
Detector
Teacher
Detector
Distillation
loss
GT
loss
Distillationnetworksforfastandaccurateobjectdetection
DistillationforcompressedCNN(student) tomimicuncompressed
CNN(teacher),toachievegreateraccuracyatthesamespeed.
MonocularSFMLearningtopredictuncertainfuturebehavior
DESIRE:DeepStochasticIOCRNNEncoder-Decoder
• DeepCVAE(autoencoder) togeneratediversehypotheses.• RNNtorankpredictions basedonmotion,sceneandinteractions.• Deepinversereinforcementlearningforlong-termfuturerewards.
Reinforcementlearningforsimulations.
Ensuringdiversity andcoverage.
NeurIPS 2017
CVPR2017CVPR2015
ICLR2019
CSE252C,SP20:ManmohanChandraker
![Page 43: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/43.jpg)
Geometricandsemantic3Dreconstruction
WeaklysupervisedsemanticreconstructionMetriclearningforcorrespondence
OcclusionreasoningandlargetransformationsDeepsupervisionforocclusion-reasonedparts
CVPR2016
CVPR2017,PAMI2018
NeurIPS 2016,ECCV2018
ECCV2018,CVPR2019
Input
Occlusion-reasoned
top-viewlayout
CSE252C,SP20:ManmohanChandraker
![Page 44: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/44.jpg)
Physically-basedlearningforshapeandmaterial
Materialsand
global
illumination
Refractive
interfaces
Shapeand
motion
WarpNet ColorNet
SIGGRAPHAsia2018
WACV2018,2019
ICCV2017ICCV2017CSE252C,SP20:ManmohanChandraker
![Page 45: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/45.jpg)
UnsupervisedadaptationtonewdomainsReconstructionwithunaligneddataFacerecognitionforprofileinputs
FromrainytogoodweatherCarrecognitionacrosscameraandlightingconditions
CVPR2019
3DV2017
ICCV2017
CVPR2018CSE252C,SP20:ManmohanChandraker
![Page 46: CSE 252C: Advanced Computer Visioncseweb.ucsd.edu/~mkchandraker/classes/CSE252C/...CSE 252C, SP20: Manmohan Chandraker [Kanazawa et al., CVPR 2019] Objects and Stuff CSE 252C, SP20:](https://reader034.fdocuments.net/reader034/viewer/2022050103/5f41fabd6b87eb19d80695e3/html5/thumbnails/46.jpg)
Take-homemessage
• ComputervisionisakeybranchofAI
• Enablesseveralmodernapplicationsaroundus
• Alotofhighlyvisibleandhigh-impactactivity
• Hugeindustryinterest
• Thisisagreattimetostudycomputervision!
CSE252C,SP20:ManmohanChandraker