Virtual Reality and Augmented Reality in Assembly Applications · © Competence Centre for Virtual...

© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)

Prof. Dr.-Ing. Dipl.-Kfm. Christoph Runde Virtual Dimension Center (VDC) Fellbach Auberlenstr. 13 70736 Fellbach www.vdc-fellbach.de

Virtual Reality and Augmented Reality in Assembly Applications VDC Whitepaper


Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary

Content

challenges & opportunities

in assembly

assembly line planning

product evaluation

assembly work place

ergonomics

assembly process evaluation

assembly process support

summary

Content

Fig.: Daimler

car assembly line at Daimler



Assembly overview

in industrial production, assembly is the entirety of all processes for the

assembly of bodies with a geometrically determined shape (lengths, angles).

in addition to work preparation and parts production, assembly forms part of

the production system of an industrial company.

the essential partial operations of an assembly process are:

o connecting (screwing, nailing, welding, gluing, soldering, clipping)

o handling (grab, place, turn over, move, secure, control)

o check

o adjust (e.g. settings)

o auxiliary operations (eg cleaning, heating or cooling for press connections, deburring,

unpacking, sealing, oiling, ...)




Assembly Challenges

permanent market changes

need for transformation ability of

assembly systems

growing and falling lot numbers

competition leads to need for high

assembly efficiency

new products and shorter product

life cycles

mass customization, individualized

products with high variance

increasing work safety regulations

increasing age of work force

Assembly Opportunities

digital design as provider for 3D

planning and simulation data

hardware and software for 3D

simulation, 3D visualization and

virtual reality:

o new and cost-effective

solutions available

o increasing performance

o increasing acceptance




Assembly Planning Functional Requirements

collision checks

assembly tree for sequence planning

product variations for sequence planning

spatial assembly paths

expand and melt parts for clearance

checks

represent assembly tools

represent flexible parts

(cables, seals, hoses, …)

represent assembly aids within product

design

3D model intersections

forward and backward kinematics

space requirements for workers,

manufacturing resources, goods, stocks,

conveyors, factory building and

infrastructure

representation of arms and hands

visualization and documentation of

reaching distances

visualization and documentation of

area of visibility

link to MTM tables, to not-traceable

human operations, e.g. picking screws

ergonomic analysis

efficiency assessment

speech recognition to control mobile

AR solutions and to add new content

to them during assembly process




Assembly Line Planning

Fig: Fraunhofer IPA

body-in-white assembly

station planning using

virtual reality at Daimler

Use of Virtual Reality (VR)

combined layout of

o conveyors,

o robots and

oother delivery systems

robot programming, ensuring

o collision free synchronization of robots

o synchronization of robots with conveying

systems

oprocess efficiency

line production versus dock production

choice of conveying / transportation systems

space requirements of work places, scaffolds

interactive VR exploration of virtual assembly

line mock-up

Fig.: Fraunhofer IPA

Fig: Fraunhofer IPA

aircraft assembly

planning at Fairschild-

Dornier

electric stator assembly

line planning using VR at

Electrolux

line planning



Assembly Line Planning

Fig.: Volkswagen

brown field planning of

robots (coloured) using

augmented reality

Use of Augmented Reality (AR):

graphical overlay of new, digital 3D content

(assembly systems) onto existing / brown field

factory structure

graphical overlay of new, digital 3D products

onto existing / brown field factory structure

visual check for

o collisions

o spaces for transportation systems such as

forklifts

avoiding LASER scan of factory structure

photo documentation

Bild: TU München

Fig.: Volkswagen

brown field planning of

new conveying system

(coloured) using

augmented reality

collision check of new

(digital) car body within

existing conveying line

line planning



Product Evaluation

Fig.: OPTIS

virtual check of design

for assembly of an

aircraft interior by

exploration using a VR

powerwall

virtual check of design-for-assembly (DFA)

subjective evaluation of buildability due to

interactive exploration by experienced product

assembly expert

accessibility studies

o by vision

o by hands

o with tools

two-hand exploration is feasible

use of projection based VR,

however could lead to visual conflict

with own physical hands

user of head mounted display (HMD)

based VR and controllers or VR gloves

Fig.: ESI Group

Fig.: ESI Group

virtual check of two-

handed assembly

feasibility using a VR

powerwall

virtual check of

reachability with head-

mounted-display-based

(HMD-based) VR and two

3D controllers

product evaluation



Work Place Ergonomics

Fig.: OPTIS

evaluation of overhead

work process ergonomics

by an anthropometric

human model, controlled

by a VR user with MoCap

processing of assembly tasks within a virtual

environment

figuring out ergonomic process properties

approaches: o interaction with virtual environment through a

virtual avatar. This avatar is controlled

- either by a motion capturing (MoCap)

system or by

- programmed animation

The avatar may deliver objective and

documented ergonomic analysis data.

o direct interaction with virtual objects, also

through a MoCap system, however the obtained

results are depending on the user (subjective)

Fig.: ESI Group

Fig.: Plavis

evaluation of reachability

and bending ergonomics

using MoCap technology

within VR

evaluating work place

ergonomics of manual

assembly stations using

digital human models

within virtual reality

workplace ergonomics



Work Place Ergonomics

Fig.: Volkswagen

check of inter space and

reachability of a manual

assembly station in a

brown field environment

using a digital human

model (colored) in

augmented reality

Use of Augmented Reality:

graphical overlay of different work

space zones, e.g.

o red area: contains all elements within a

grasping distance, thus with no need

to bend forward

o other area: out of direct grasping distance,

thus walk and/or bend forward

the AR user may walk around the work

stations and inspect all cask and tools,

if necessary correct their positions

photo documentation

workplace ergonomics



Assembly Process Evaluation

Fig.: ESI Group

Virtual evaluation of tool

use in interactive VR

Use of assembly tools:

relevant aspect of assembly analysis

examples:

opower screwdrivers

o lifting appliances

check for accessibility

check for ergonomics using tools

co-working

Fig.: ESI Group

Virtual evaluation of tool

use with 3D digital

human models in VR

process evaluation




Fig.: Daimler AG)

Assembly path

visualization in VR

Assembly path identification:

identification of optimum assembly paths by

interactive assembly studies in VR:

recording part movement

identification of impossible paths

simple VR system: collision detection and

3D controllers (e.g. flystick, gesture

recognition)

technical approaches:

o direct interaction with virtual environment

o programming a virtual avatar

check of assembly process considering

flexible parts (push aside, stretch away

from interfering region)

Fig.: ESI Group

Fig.: ESI Group

Assembly path

generation with 3D

digital human model

Interactive exploration of

flexible parts during

assembly process

process evaluation




Fig.: Virtalis

Haptic assembly

evaluation with force

feedback in VR

Assembly path identification:

in addition to visual and/or acoustic collision

response, haptic force feedback may be used

intuitive 6DOF directing of physical handle will

control virtual object position and orientation

haptic system blocks in case of collision

two force feedback systems may be joined to

achieve 6DOF (force and torque) feedback

assembly path generation

3D prints of additive manufacturing may be

attached to the handle to simulate realistic

grasping (grasp type and grasp point) by

human hand

Fig.: Fraunhofer IPK

Grasp type and grasp

point investigation using

a 3D print attached to a

haptic device

process evaluation




Fig.: ESI Group

Virtual evaluation of

assembly sequencing

Assembly sequencing:

evaluating correct assembly sequencing

by variant comparison

in principle all aspect mentioned in this

presentation (line planning, product evaluation,

ergonomics) can be checked within this step

Virtual prototyping of assembly process:

for process optimization

for product optimization (e.g. assembly

support features)

standard times determination with MTM

(methods time measurement)

Fig.: Industriehansa / now Altran]

Fig.: Industriehansa / now Altran]

Guided assembly process

Identification of standard

working times for an

assembly operation by

the means of VR

process evaluation



Process Support in Assembly

Fig.: Testia/Airbus

Assembly positions and

instruction are projected

as AR onto aircraft body

assembly assistance systems using

augmented reality

showing next process step during assembly

itself

indicating tools and parameters (e.g. torques)

applicable technologies:

oAR projection

oAR smart glasses

oAR tablet PCs and AR smartphones Fig.: BMW

Fig.: Volkswagen

Perspective through AR

smart glasses, showing

assembly instructions,

part positions, tools and

tool parameters

Disassembly instruction

as an AR overlay shown

on a tablet PC

process support




assembly assistance systems: remote assistance with shared camera view

Fig.: Re’flekt

Assistance during assembly

process using remote

teleconferencing with a shared

camera view and augmented

reality support on a tablet PC

process support




Fig.: Metaio

Comparison of planned

versus as-built status of

weld studs at car

assembly using AR

Quality assurance in assembly using

augmented reality

comparison of planned versus as-built status

visual check of deviations

check of installation space and positions

photo documentation

technologies tested in practice:

oAR tablet PCs and AR smartphones

ooffline AR systems (screen and camera

detached)

Fig.: Daimler

Fig.: Testia / Airbus

Check of installation

space and part positions

after assembly using AR

Quality check of

assembled parts using

AR comparison on a

tablet PC

process support




Quality assurance in assembly

Fig.: Testia / Airbus

Quality check of assembled

parts: Augmented Reality

overlay shows assembled clip

as-planned and its status in

reality. The user visually

compares both and either

confirms of rejects the quality.

process support



Summary

diverse field, many assembly applications for VR and AR

many specific VR / AR tools, thus no single solution for all

digital 3D design data is a necessity

Main benefits of VR / AR use

virtual prototyping of assembly process, optimizing and supporting

o assembly lines,

o products (DFA),

o workplace ergonomics and

o assembly process

already in a early, digital planning stage

human-integrated manufacturing engineering, considering

o human factors

o subjective assessments and tacit knowledge

summary



Requirements on VR / AR products for assembly applications

CAD import

CAE import

collision detection

physics and sliding simulation

forward / backward kinematics

tolerance analysis

flexible components (cables, hoses, seals)

demonstrate (design-based) assembly aids

adding and melting of material for

clearance tests

priority graph for assembly order planning

present assembly sequence variations

identification of assembly paths

cuts through the 3D model

illustration of assembly tools

illustrate and document manual reach

distances

illustrate and document visible areas

illustration of arms / arm movements

anthropometric human model with

ergonomic analysis

consider space requirements for operating

equipment, products, people, materials

handling technology, building technology

links to MTM tables, for example to

detailed manual work which is hard of

impossible to track (e.g. separation of

screws)

evaluation not only of working hours,

but also of ergonomics, distances

AR engine for process support

speech recognition to easily supplement

documentation with spoken annotations

content authoring for training applications

summary



Literature

Deuse, J.; Busch, F.: Zeitwirtschaft in der

Montage. In: Bruno Lotter und Hans-Peter

Wiendahl (Hg.): Montage in der industriellen

Produktion. Ein Handbuch für die Praxis. 2. Aufl.

Berlin: Springer Vieweg

ESI: VR im Mittelpunkt der Produktionstechnik;

online unter: k https://www.virtual-reality-

magazin.de/vr-im-mittelpunkt-der-

produktionstechnik ; abgerufen am 13.5.2018

Heinig, M.: Nutzung von Virtuellen Technologien

für die Montageplanung von Unikaten.

Technischen Universität Hamburg-Harburg zur

Erlangung des akademischen Grades Doktor-

Ingenieur (Dr.-Ing.) genehmigte Dissertation,

Hamburg: 2015

Konold, P.; Reger, H. (2003): Praxis der

Montagetechnik. Produktdesign, Planung,

Systemgestaltung. 2., überarb. und erw. Aufl.

Wiesbaden: Vieweg

Lotter, B.; Wiendahl H.-P. (Hg.): Montage in der

industriellen Produktion. Ein Handbuch für die

Praxis. 2. Aufl. Berlin: Springer Vieweg

Patron, C.: Konzept für den Einsatz von

Augmented Reality in der Montageplanung.

Forschungsberichte IWB, Band 190, Herbert Utz

Verlag, München, 2004

Reinhart, G.; Zäh, M.F.; Patron, C.; Doil, F.; Alt,

T.; Meier, P.: Augmented Reality in der

Produktionsplanung. In: wt Werkstattstechnik

online 93 (2003), Nr. 9, S. 651-653

Roosen, G.: Virtuelle Technologien in der

Montageplanung ; online unter:

http://www.cadplace.de/SIM-VR/VR-

Produkte/Virtuelle-Technologien-in-der-

Montageplanung ; abgerufen am 13.5.2018

summary

https://www.virtual-reality-magazin.de/vr-im-mittelpunkt-der-produktionstechnik













http://www.cadplace.de/SIM-VR/VR-Produkte/Virtuelle-Technologien-in-der-Montageplanung















VDC members in VR / AR assembly applications

summary

http://www.cdmtech.de/

http://www.cmc-engineers.de/

http://www.daimler.com/

http://www.esi-group.com/

http://www.fh-swf.de/

http://www.festo.com/

http://www.iao.fraunhofer.de/

http://www.ipa.fraunhofer.de/

http://www.ipk.fraunhofer.de/

http://www.iav.com/

http://www.optis-world.com/

http://www.ptc.com/

http://www.re-flekt.com/

http://www.rif-ev.de/

http://www.salt-and-pepper.eu/

http://www.techviz.net/

http://www.virtalis.de/

http://www.vr-on.com/


Thank you very much for your attention If you are interested in further details of this topic, please refer to

Virtual Dimension Center (VDC) Fellbach Auberlenstr. 13 70736 Fellbach www.vdc-fellbach.de [email protected]

http://www.vdc-fellbach.de/



mailto:[email protected]



Virtual Reality and Augmented Reality in Assembly Applications · © Competence Centre for Virtual...

Documents

Transcript of Virtual Reality and Augmented Reality in Assembly Applications · © Competence Centre for Virtual...