Augmented Reality in Transportation C Augmented Reality for...¢  Augmented Reality in...

download Augmented Reality in Transportation C Augmented Reality for...¢  Augmented Reality in Transportation

of 40

  • date post

    06-Jun-2020
  • Category

    Documents

  • view

    7
  • download

    0

Embed Size (px)

Transcript of Augmented Reality in Transportation C Augmented Reality for...¢  Augmented Reality in...

  • Augmented Reality in Transportation Construction FHWA Contract DTFH6117C00027: LEVERAGING AUGMENTED REALITY FOR HIGHWAY CONSTRUCTION

    Hoda Azari, Nondestructive Evaluation Research Program Manager

    IHEEP 2018 – Lincoln, Nebraska. September 25, 2018

    September • 2018

  • Augmented Reality Research Project

     FHWA Contract DTFH6117C00027

     Thorough review of state-of-the

    art augmented reality (AR)

    technologies and applications for

    Highway Construction

     Draft Report will be completed at

    the end of 2018

  • Augmented Reality (xR) Terminologies

    Reality Virtual Reality Augmented

    Reality Augmented

    Virtuality

    The Reality – Virtuality Continuum

    Mixed Reality (xR)

  • Augmented Reality (xR) Terminologies

    Reality Virtual Reality Augmented

    Reality Augmented

    Virtuality

    Mixed Reality (xR)

    Daqri Smart Glasses (AR)

    Google Daydream (VR)

  • Augmented Reality (xR) Terminologies

    Reality Virtual Reality Augmented

    Reality Augmented

    Virtuality

    Mixed Reality (xR)

    Typical AR Devices are closer to the ‘Reality’ end of the spectrum:

    • Primarily ‘See-through’ of reality • Some VR overlay

  • Augmented Reality Technologies

    Components of AR System:

     Sensory input

     Display device

     Haptic input (navigation/interaction)

     Tracking Systems

     Computing Device

     Media Representation

     Data Input / Storage

  • Augmented Reality Technologies

    Display devices fit into two categories

    Hand-held devices – tablets, smart phones

    Head-mounted displays (HMD) / Glasses

  • Augmented Reality Technologies

    Tracking Methods – Position and Orientation

     GNSS / Wifi networks / V2X/I2X (radio systems for CAV)

     Inertial sensors – track local movements of the device

     Optical sensors – track 2D video or 3D imagery of the real-world

     Sensor fusion – systems that combine all of the tracking methods

     Marker-based position initialization

  • Augmented Reality Technologies

     Hand-held devices and commercial AR applications

    Google Translate

    PokemonGo

  • Augmented Reality Technologies

     Hand-held devices and AR applications  Video see-through: real-world captured by camera on device

     GNSS positioning / Inertial tracking for orientation and movement

     Some devices use optical or 3D sensor tracking

     Initial registration typically achieved with markers / targets in scene

     Hand-helds on construction sites not necessarily new – AR could be add-on

    to existing uses of mobile devices

  • Augmented Reality Technologies

     Head-mounted displays  Optical see-through: real-world seen directly through lenses

     GNSS positioning / Inertial tracking for orientation and movement

     Some devices use optical or 3D sensor tracking

     Initial registration typically achieved with markers / targets in scene

  • AR Information Display Options

    Categories of information display that AR will support:

     Display what is not yet constructed • Compare design alternatives in context

    • Check relationships between existing/future elements

    • Site logistics, equipment movements

    • Preview complex installation procedures

    • Illustrate construction methods and sequencing

    • Opportunities for training on-site

    • Check traffic management/temporary structures

     Display what was intended to be constructed • Site inspection and validation

    • Code/standards/compliance checking

    • Checking quantities and work progress

    • Inspection training opportunities

    • Check traffic management/temporary structures

  • AR Information Display Options

    Categories of information display that AR will support:

     Display what is hidden from view • Buried utilities or structural components

    • Elements obstructed from the current view

     Display abstract information aligned with real-world context • Alignment information, easements, site boundaries, ROW boundaries

    • Environmental boundaries, such as flood levels or sea-level rise data

    • Sensitive areas such as archeological and historic sites

    • Meta-data tagged to associated real-world objects

    • Potential work-zone hazards

  • Augmented Reality Challenges

     Challenges with current systems

     Field of View (FOV) • Handheld devices limited to FOV displayed on screen held at arms length • Current HMD’s typically limit virtual display to 60 degrees FOV (see next slide)

  • Field of View (FOV)

    Right eye 135⁰Left eye 135⁰

    Binocular Field of View 120⁰

    Typical AR Display FOV 60⁰

  • Augmented Reality Challenges

     Challenges with current systems

     Occlusion

    • Current systems can only display overlaid virtual information in the foreground

    • Virtual elements that are ‘behind’ real- world elements are not masked out

    • Limits the immersive quality of the AR representation

    Calculating occlusion requires that the AR application have detailed 3D information of both the real-world scene and the virtual model

  • Augmented Reality Challenges

     Challenges with current systems

     Ruggedness / Durability • Current devices not designed for daily outdoor use, safety issues

     Performance / Portability • Display processing must be performed on-board the device

    • Some devices tethered to ‘wearable’ computing device – adds weight

    • Most applications require storage for ‘pre-loading’ of 3D assets onto the device

    • WiFi connectivity / streaming could allow assets to be loaded on demand – no examples of this yet (solvable in the near future with 5G?)

     Safety • Hand-held devices – use one or usually both hands and block view

    • HMD’s – can limit peripheral vision, and possibly audio

  • Augmented Reality Challenges

     Challenges with current hardware

     Tracking technologies • GNSS requires visibility of satellite network • Most systems require marker-based or manual registration • Optical / 3D sensors require ‘visible’ detail in environment • Bright / outdoor scenes are difficult for optical tracking

    These last two requirements could be key challenges with construction, especially in an outdoor work environment without details to track

     Display brightness • Current display technology in HMD’s is limited in ability to display over optical

    see-through devices in a bright outdoor environment

    Matching virtual display to real-world brightness problematic, especially in outdoor environment

  • Augmented Reality Software

    AR Ready ‘Cloud-based’ Platforms

    Autodesk • BIM360 Platform – Not currently supporting Geo-referencing • Forge– development platform

    Bentley • Bentley Connect

    Trimble • Trimble Connect

    AR Ready Developer Platforms • Apple ARkit, Google ARCore, Windows Mixed-Reality, WebAR

    Game Platforms - Unity most common in AR

  • Augmented Reality Workflows

     AR in 3D Model-based Workflows • AR should become a critical tool within existing BIM model workflows

    • Piggyback on BIM data management and 3D modeling efforts

    • Some current BIM platforms already supporting AR tools

     AR in Workflows for other data types

    • Leveraging other types of information (non-3D) will require new tools and workflows

    • Abstract data could be included in 3D model if geolocated correctly

  • ARTC Workshop

    Augmented Reality in Transportation Construction (ARTC)

     Held May 9th 2018, at Turner Fairbanks Highway Research Facility

     ~40 participants

    • FHWA / State DOT’s

    • University / Research

    • Contractors / Consultants

    • AR Vendors / R&D

     Goals:

    • Educate participants in AR

    • Identify new applications for AR

    • Prioritize future applications of AR

  • ARTC Workshop

    Conducted in Three Sessions

     1: Overview of AR Technologies • Current display technologies

    • Characteristics of AR systems

    • Challenges with AR tools for construction

     2: Vendor technology presentations and hands-on demonstrations

    • Participants were able to see and experience the AR tools

     3: Brainstorming and participant polling of application opportunities

    • Discussion and identification of potential AR applications

    • Documentation of challenges for AR applications in construction

    • Electronic ranking of top AR application concepts

  • ARTC Workshop

    Key themes emerged in the brainstorming session on AR

    application areas in construction

     1: Visualization of design information • Simplified 3D model components

    • Abstract 2D information such as alignments or