Augmented Reality Augmented Reality Augmented Reality (AR) blends virtual and real scenes in both...

Click here to load reader

download Augmented Reality Augmented Reality Augmented Reality (AR) blends virtual and real scenes in both time

of 33

  • date post

    02-Jun-2020
  • Category

    Documents

  • view

    7
  • download

    0

Embed Size (px)

Transcript of Augmented Reality Augmented Reality Augmented Reality (AR) blends virtual and real scenes in both...

  • Mauro Dalla Mura, Michele Zanin, Claudio Andreatta, Paul Chippendale,

    Fondazione Bruno Kessler, Trento, Italy

    Augmented Reality: Fusing the Real and

    Synthetic Worlds

  • 2 M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

    Morphological and Attribute Profiles Spectral/spatial analysis

    IEEE IGARSS’12

    What is Augmented Reality?

  • 3 M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

    Morphological and Attribute Profiles Spectral/spatial analysis

    IEEE IGARSS’12

    What is Augmented Reality?

    Can it be useful for the GRS community?

  • Outline

    1. Introduction to Augmented Reality

    2. AR and GRS: the proposed concept

    3. Augmenting the user’s view

    4. Augmenting geo-data

    5. Discussion and conclusion

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

  • Augmented Reality

    Augmented Reality (AR) blends virtual and real scenes in both time and space

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

  • Augmented Reality

    Augmented Reality (AR) blends virtual and real scenes in both time and space

    1950s 1966

    Nowadays, AR technology starts to run on consumer level mobile devices (e.g.,

    smartphones and tablets) in real-time

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

  • Augmented Reality

    Augmented Reality (AR) blends virtual and real scenes in both time and space

    1950s 1966 2000

    Nowadays, AR technology starts to run on consumer level mobile devices

    (e.g., smartphones and tablets) in real-time

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

  • Augmented Reality

    Augmented Reality (AR) blends virtual and real scenes in both time and space

    1950s 1966 2000 2008

    Nowadays, AR technology starts to run on consumer level mobile devices

    (e.g., smartphones and tablets) in real-time

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

  • Augmented Reality

    Augmented Reality (AR) blends virtual and real scenes in both time and space

    1950s 1966 2000 2008 2014

    Nowadays, AR technology starts to run on consumer level mobile devices

    (e.g., smartphones and tablets) in real-time

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

  • Augmented Reality

    Augmented Reality (AR) blends virtual and real scenes in both time and space

    1950s 1966 2000 2008 2014 . . .

    Nowadays, AR technology starts to run on consumer level mobile devices

    (e.g., smartphones and tablets) in real-time

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

  • AR applications

     Gaming

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

  • AR applications

     Gaming  Entertainment

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

  • AR applications

     Gaming  Entertainment  Sport

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

  • AR applications

     Gaming  Entertainment  Sport  Commercial

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

  • AR applications

     Gaming  Entertainment  Sport  Commercial

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

    Urban planning

  • AR applications

     Gaming  Entertainment  Sport  Commercial

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

    Urban planning  Tourism

  • AR applications

     Gaming  Entertainment  Sport  Commercial

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

    Urban planning  Tourism  Navigation

  • AR applications

     Gaming  Entertainment  Sport  Commercial

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

    Urban planning  Tourism  Navigation  . . .

    . . .

  • Augmented Reality for GRS Augmented Reality exploiting geo-data

     Availability of massive archives of geo-referenced data (e.g., remote sensing,

    geo-referenced social photo websites, GIS, geo-data)

    A possible exploitation of geo-data

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

    Augmentation of a 2D map superimposing the corresponding 3D model

     Effective integration of AR and geo-data is becoming feasible on consumer-

    level portable devices

  • Augmented Reality for GRS Available applications showing geo-data on smart-phones

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

    Layar (since Jun 2009) Peak.AR (since Oct 2009)

    Wikitude (since May 2009)

    SwissPeaks (since Nov 2009)

    Peaks Mixare (since Mar 2010)

     Device-world registration is usually not accurate (e.g., errors in horizontal orientation of 30 deg if using the on-board compass)

  • Photo-world registration Accurate alignment of the device and the

    surrounding panorama is necessary

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

    Captured Photograph DEM

    2D Rendered DEM

    Aligned Photograph

    Salient feature extraction

    Photo registration

     Camera position (ϕ0, λ0, h0) and pose (α, β, γ) should be estimated

     Vision based approaches operates by finding the best match between visual features and synthetic ones (from a 3D model)

     Hardware information (e.g., magnetometer, compass, GPS) can be used as initial clue of the pose

     After photo-world registration, each pixel in the image can be associated to geographic coordinates

    Panorama rendering

  • AR for GRS: the proposed concept

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

  • Reality augmentation

    Augmenting the user’s view

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

  • Augmenting the user’s view  Once registered, geo-data can

    be delivered to the user  Geographical toponyms  GIS layers  Topographic maps  3D building models  . . .

     New modality to query and access geo-data

     Fast interactive modality (e.g., touchscreen)

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

  • Augmenting the geoscientist’s view

     Support tool for field campaigns:  Real-time guidance for field surveys (e.g., ground truth collection)  In-situ validation of remote sensing data and products (e.g., land cover maps)

     Aid the interpretation of geo-referenced RS data  data not in the visible range  time series

    Forest regions from Corine land

    cover 2000 (25m resolution)

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu Topographic map

  • Reality augmentation

    Augmenting the geo-data archives

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

  • Augmenting the geo-data archives

    Users’ portable devices become

    ground-based sensors

    r c

    ϕ

    λ

    Digital Elevation Model

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

    position pose focal length

    (r, c)  (ϕ, λ)

    f(F, ϕ0, λ0, h0, α, β, γ, DEM)

  • Ground-based sensing

     Extract Information on the land cover

    November snowline

    >1785 m

    December snowline >1212 m

    Oct. 2007

    Nov. 2007

    Dec. 2007

    Snow cover map

    0

    0.2

    0.4

    0.6

    0.8

    1

    8 6

    0

    1 0

    6 0

    1 2

    6 0

    1 4

    6 0

    1 6

    6 0

    1 8

    6 0

    2 0

    6 0

    2 2

    6 0

    2 4

    6 0

    2 6

    6 0

    2 8

    6 0

    3 0

    6 0

    Altitude (m)

    Sn o

    w (

    p e

    rc e

    n ta

    ge )

    Sept Nov Jan

    M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 dallamura@fbk.eu

  • Ground-based sensors

     Extract Informatio