The Impact of Eye Tracking on Head Mounted / Near Eye ...€¦ · Thank You for your Attention!...

TheImpactofEyeTrackingonHeadMounted/NearEyeDisplay

Development

BálintIstvánKovács(01227520)

InstituteofComputerGraphicsandAlgorithmsTUWien,Austria

Outline

1.  Summary2.  Discussionoffundamentalhead-mounted/near-eyedisplay

concepts3.  Historicaldevelopmentsampling4.  Persistingissuesandchallenges5.  Keyeyetrackingterminology6.  Discussionofselectedpapers7.  Conclusion

BálintKovács-TheImpactofEyeTrackingonHM/NEDisplayDevelopment 2

1.Summary

ChallengesSizeconstraintsCostconstraintsComputationalcostDisplaytechnologynotyetwelladapted

GoalsImprovementofimagequalityRenderingoptimizationAdditional,morenaturaldepthcuesEnhancedinteraction


2.DiscussionoffundamentalHM/NEdisplayconcepts

AugmentedRealityVirtualenvironmentsenhancerealenvironments„See-through“display

VirtualRealityVirtualenvironmentsreplacerealenvironments

MixedReality„anywherebetweentheextremaofthevirtualitycontinuum”



OpticalsolutionsSemi-transparentmirrorsSee-throughbydefault

Display-onlysolutionsCompletelyblockingtheusersviewSee-throughpossiblewithcameras



DisplaytechnologyState-of-theartdisplaysEvolutionofdifferenttechnologies

CRT->LCD->OLEDLayeredpanelsMicrolensarrays

RenderingOptimization


3.Shorthistoricaldevelopmentsampling

EarlyExample1:[Sutherland1968]ARprojectviatrackedHMDMirror/prismsetupwithminiatureCRTsHiResultrasonicheadpositionsensorClippingdividerfordynamicperspectivedisplay



EarlyExample2:[Rolland1994]

Comparisonofsee-throughHMDsExcellentbaselineofkeyissues



CurrentExample1:[Lanman2013]

NElightfielddisplayAccurateaccommodation,convergence,andbinoculardisparitydepthcuesbinocularprototype+aGPU-acceleratedstereoscopiclightfieldrenderer.



CurrentExample2:[Dunn2017]AugmentedrealityusecaseSinglesee-through,varifocaldeformablemembranemirrorforeacheyeWidefieldofview(100°diagonal)Fastdepthswitching(from20cmtoinfinitywithin300ms)



CurrentExample3:Maimone2017AR/VRusecaseFresnelholographyDoublephaseamplitudeencodingFullcolor,highcontrastandlownoisehologramsHighresolution,trueper-pixelfocalcontrolRelativelywidefieldofview(80°)



CurrentExample4:Consumergradehardware

E.g.HTCVive,OculusRift,PSVR,WindowsMR,etc.AdoptionratedrivingforcefordevelopmentAffordable/enthusiastcostHighhardwaredemandResolution(screendooreffect)


4.PersistingChallengesinNE/HMDisplayDevelopment

Problem:Usuallyatrade-offbetweendifferentgoals


Challenges GoalsMissingnaturalvisionfeatures Accommodativedepthcues

Limitedperiphery WidefieldofviewScreendooreffect HighresolutionHighcomputationaldemand OptimizationHighhardwarecost Commoditysolutions

5.Keyeyetrackingterminology

TechnicalEyeTrackingTechniquesEyemovement/gazeanalysis

AnatomicalVisualattentionVisualpsychophysicsTaxonomyofeyemovements


5.Keyeyetrackingterminology

ForNE/HMdisplayapplicationsEye-gazevs.eye-poseDetectionrateOpticalaxisvs.visualaxisEye-box/exitpupilPupildetectionmethods


6.Discussionofselectedpapers

Example1:[Stengel2015]Contribution

Lowcost(commoditycameraswithdichroicmirrors)Featuretrackingapproach

LimitationsInterferencewitheyeglassesSmallscaleuserstudyandapplicationtests



Example2:[Pai2016]Contribution

CustomhardwareGazefocusdepthestimation

LimitationsLimitedfocusdepthaccuracyReducedimagequalitythroughcutlensesLimitedapplicationtests



Example3:[Plopsky2016]Contribution

MicrolensdisplayGazetrackingthrougheye-poseestimationAccountsforpixelsseenthoughmultiplelenses

LimitationsCalibrationquickbutinaccurateLimiteduserstudy



Example4:[Pohl2016a]Contribution

Off-the-shelfhardware(OculusDK2+PupilPro)Dynamicsamplingmap,combinedfromgazeandlensastigmatismdata

LimitationsCustomrenderingarchitectureneededwithperpixelsupersamplingcapabilities(doesnotintegrateinOpenGL45orDX12)



Example5:[Pohl2016b]Contribution

SamehardwareVisualfieldmappingthoughextensivecalibrationNorenderingininvisibleareas

LimitationsCalibrationtedious(lacksgenericmodel)



Example6:[Patney2016](andFOVE)FocusonfoveatedrenderingRapid,wellfundeddevelopment(NVIDIA/SMI)



Example7:VarjoBionicDisplayFoveatedprojectionCombiningtwodisplaysEyetrackingforgazedetections


6.ComparisonofDiscussedDevices


Hardware Cost Fromfactor Contribution Limitations

Lanman2013 proprietary high wearable accuratedepthcues (limitedinformation)

Maimone2017 proprietary high benchtop/wearable

HiRes,trueperpixelfocalcontrol

smalleye-box

Dunn2017 proprietary high benchtop deformablemembranemirrortech.

bulkyformfactor

Stengel2015 proprietary verylow wearable lowcosttrackingsolution interferencewitheyeglasses

Pai2016 off-the-shelf low wearable gazefocusdepthest. imagequality,usability

Plopsky2016 proprietary high wearable Microlensdisplaywitheye-poseestimation

Inaccuratecalibration

Pohl2016a off-the-shelf low wearable renderingoptimization customrenderingarchitecture

Pohl2016b off-the-shelf low wearable renderingoptimization calibration

Patney2016/Fove preproduction/dev-kit

low wearable industry-standardrenderingoptimization

perceptualvs.optimizedapproach

Varjo preproduction/dev-kit

mid wearable foveatedprojections (limitedinformation)

7.Conclusion

PersistingchallengesinHM/NEdisplaydevelopmentDisplayresolutionneedstoincreasetoretinaresolutionlevelMorenaturaldepthcuesneededforhigherfidelityIncreasedcomputationalcostEyetrackingdatahighlyvaluable

IssuesTrackinghardwareandsoftwareneedintegrationDetectionreliabilityCalibration,genericmodel


ThankYouforyourAttention!ReferencedPapers:[Duchowski2007]Duchowski,AndrewT."Eyetrackingmethodology."Theoryandpractice(2007).[Dunn2017]D.Dunnetal.,“WideFieldOfViewVarifocalNear-EyeDisplayUsingSee-ThroughDeformableMembraneMirrors,”IEEETransactionson

VisualizationandComputerGraphics,vol.23,no.4,pp.1322–1331,2017.[Lanman2013]D.LanmanandD.Luebke,“Near-eyeLightFieldDisplays,”ACMTrans.Graph.,vol.32,no.6,p.220:1–220:10,Nov.2013[Maimone2017]A.Maimone,A.Georgiou,andJ.S.Kollin,“HolographicNear-eyeDisplaysforVirtualandAugmentedReality,”ACMTrans.Graph.,vol.36,no.

4,p.85:1–85:16,Jul.2017.[Pai2016]Pai,YunSuen,BenjaminTag,BenjaminOutram,NoriyasuVontin,KazunoriSugiura,andKaiKunze."GazeSim:simulatingfoveatedrenderingusing

depthineyegazeforVR."InACMSIGGRAPH2016Posters,p.75.ACM,2016.[Patney2016]AnjulPatneyetal.“TowardsFoveatedRenderingforGaze-TrackedVirtualReality.”InACMTrans.Graph.,Vol.35,No.6,Article179,Publication

Date:November2016[Plopski2016]Plopski,A.,Kashima,W.,Taketomi,T.,Sandor,C.,&Kato,H.,"Eye-GazeTrackinginNear-EyeHead-MountedDisplays",ProceedingsoftheOzCHI,

2016[Pohl2016a]Pohl,D.etal."CombiningEyeTrackingwithOptimizationsforLensAstigmatisminmodernwide-angleHMDs"[Pohl2016b]Pohl,Daniel,XucongZhang,AndreasBulling,andOliverGrau."Conceptforusingeyetrackinginahead-mounteddisplaytoadaptrenderingtothe

user'scurrentvisualfield."InProceedingsofthe22ndACMConferenceonVirtualRealitySoftwareandTechnology,pp.323-324.ACM,2016.[Rolland1994]Rolland,JanickP.,RichardL.Holloway,andHenryFuchs."Acomparisonofopticalandvideosee-throughhead-mounteddisplays."SPIE

TelemanipulatorandTelepresenceTechnologies.Vol.2351.1994.[Stengel2015]M.Stengeletal.,"AnAffordableSolutionforBinocularEyeTrackingandCalibrationinHead-mountedDisplays,"inProceedingsofthe23rdACM

internationalconferenceonMultimedia(MM'15).ACM,NewYork,NY,USA,15-24.2015[Sutherland1968]I.E.Sutherland,“AHead-mountedThreeDimensionalDisplay,”inProceedingsoftheDecember9-11,1968,FallJointComputerConference,

PartI,NewYork,NY,USA,1968,pp.757–764.


The Impact of Eye Tracking on Head Mounted / Near Eye ...€¦ · Thank You for your Attention!...

Documents

Transcript of The Impact of Eye Tracking on Head Mounted / Near Eye ...€¦ · Thank You for your Attention!...