OPTO 6124 Perception Scott Stevenson Image Segmentation ......pareidolia meaning “wrong image”....
9
OPTO 6124 Perception Scott Stevenson Image Segmentation What is really behind so many perception demos? Perception demos show us that our visual understanding of the world involves a lot of “filling in” of information in order to reach knowledge of objects and their relationships. Often, visual information is sparse or ambiguous. The demos are surprising to us because we don’t often realize how much guesswork or “top down processes” are involved. The process of finding objects in an image is called Image Segmentation. Perception is a hard problem to solve! In 1966, Artificial Intelligence pioneer Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. (Szeliski 2009, Computer Vision) This turns out to be one of the hardest problems in AI. As an example, compare the ability of humans and computers to match faces across age. Andy Adler of Carleton University in Ottawa CA ran an experiment on humans and machines to see which did better at matching faces. Humans used a web interface to say “same person” or “different person” for a large set of faces. Several computer programs made the same comparisons This plot shows the error rates for humans and machines. Lower numbers mean better performance. Humans (data shown by the dots) generally made few errors Each solid curve is a different program, and the newest programs make fewer errors. Software is just now catching up with humans at this basic, everyday task.
Transcript of OPTO 6124 Perception Scott Stevenson Image Segmentation ......pareidolia meaning “wrong image”....
OPTO6124 Perception ScottStevensonImageSegmentationWhatisreallybehindsomanyperceptiondemos?Perceptiondemosshowusthatourvisualunderstandingoftheworldinvolvesalotof“fillingin”ofinformationinordertoreachknowledgeofobjectsandtheirrelationships.Often,visualinformationissparseorambiguous.Thedemosaresurprisingtousbecausewedon’toftenrealizehowmuchguessworkor“topdownprocesses”areinvolved.TheprocessoffindingobjectsinanimageiscalledImageSegmentation.Perceptionisahardproblemtosolve!In1966,ArtificialIntelligencepioneerMarvinMinskyatMITaskedhisundergraduatestudentGeraldJaySussmanto“spendthesummerlinkingacameratoacomputerandgettingthecomputertodescribewhatitsaw”.(Szeliski2009,ComputerVision)ThisturnsouttobeoneofthehardestproblemsinAI.Asanexample,comparetheabilityofhumansandcomputerstomatchfacesacrossage.AndyAdlerofCarletonUniversityinOttawaCArananexperimentonhumansandmachinestoseewhichdidbetteratmatchingfaces.
Humansusedawebinterfacetosay“sameperson”or“differentperson”foralargesetoffaces.Severalcomputerprogramsmadethesamecomparisons
Thisplotshowstheerrorratesforhumansandmachines.Lowernumbersmeanbetterperformance.Humans(datashownbythedots)generallymadefewerrorsEachsolidcurveisadifferentprogram,andthenewestprogramsmakefewererrors.Softwareisjustnowcatchingupwithhumansatthisbasic,everydaytask.
ImageSegmentation.Imagesegmentationreferstotheproblemofsortingrawimagedataintodistinctobjects.Seeing“things”insteadofjustareasofcolorandorientation.Incomputervision,thisisrecognizedasanextremelydifficultproblem.Forhumans,itisalmosteffortless.ConsiderthisimagerandomlypulledfromtheInternetusingthesearchphrase“kitchendrawerclutter”
https://oldworldgardenfarms.files.wordpress.com/2014/09/kitchen-drawer.jpgIfIaskyoutoreachfortheslottedspatula,afterafewmomentsofscanningyouwillidentifytheobject,thenorientyourhandtograbthehandle.Robotsonassemblylineshavetosolvethisproblemwhenreachingforapartfromabinfullofjumbleditems.Itisanextremelydifficultproblemforartificialintelligence.
Perceptiondoesn’talwaysgetitright:Thefollowingimagesdemonstratehowyourownvisualsystemsometimesmakes“errors”orhasconflictinganswersorhasdifficultywiththisproblem.Sometimes,weseefamiliarshapesinrandomconfigurations,aphenomenoncalledpareidoliameaning“wrongimage”.TheRorschachTestusespareidoliatoprobethepsycheofpatientsinpsychoanalysis.
Rorschach’soriginalimage3 Meow!(fromDr.Bedell,sourceunknown)
Googleimagesearch,“Jesusinatortilla”
Weseeorganizationinrepeatingpatterns.Thesecanshiftandreorganizedynamically.
Sourceunknown
Weseeshapesandignorebackgrounds,evenifthosebackgroundsareshapes.Zoominontheimage,lookbetweenthepillars.
ThisisasetofactualpillarsattheExploratoriuminSanFrancisco
Insituationswithverypoorinformation,itcantakealongtimetoreachanorganizedperception.Doyouseetheanimal?
“Dalmation”fromMarr1982
Sometimeswefillinmissinginformation,likeedgesthatareprobablythereeventhoughtheyhavenocontrastintheimage.Thesearecalled“illusorycontours.”Manypeopleseeasolidwhitetrianglewithdistinctedges,lyinginfrontoftheoutlinedtriangleanddiscs.Thesolidwhitetrianglelooksalittlebrighterthanthebackground.
“KanizsaTriangle”byGaetanoKanizsa,1955
Theinferencesmadeaboutanimagehaveastronginfluenceonhowmissinginformationgetsfilledin.Isthewireframecubeinfrontofblackdiscs?Orisitbehindasetofholes?
KanizsaNeckerCube,sourceunknown.
Impossiblefigures,likethosemadefamousbyMCEscher,pitlocalsolutionstoimagesegmentationagainstglobalsolutions.Inthisfigure,themissinginformationisshading,stereoscopic3D,andmotionparallax.Withoutthese,thebrainmakesabestguessatthethreedimensionalshapesbasedonedges.Theartisthascleverlycreatedambiguityfromoneendtoanotheroftheobject.
Sourceunknown.Fromagoogleimagesearchon“MultistablePerception”
