Recording augmented reality experiences to capture design ... Recording augmented reality...

download Recording augmented reality experiences to capture design ... Recording augmented reality experiences

of 12

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Recording augmented reality experiences to capture design ... Recording augmented reality...

Int J Interact Des Manuf (2009) 3:189200DOI 10.1007/s12008-009-0074-8


Recording augmented reality experiences to capture designreviews

Jouke Verlinden Imre Horvth Tek-Jin Nam

Received: 28 September 2008 / Accepted: 23 June 2009 / Published online: 10 July 2009 The Author(s) 2009. This article is published with open access at

Abstract During design reviews, multiple stakeholdersconvene to reflect on the quality of intermediate results andto decide upon following steps. As prior research indicates,such design reviews are only partly a structured, rationalprocess; often aspects as trust, hidden agendas or lackingcommissioning skills influence this activity. Furthermore,a wide range of media is being used during these meet-ings, which are difficult to recollect in their context afterthe event. This research project attempts to improve designreviews in the domain of Industrial Design Engineering bytwo means: (1) by providing a specific prototyping and anno-tation device employing physical mockups, (2) by recordingboth communication between stakeholders and interactionwith the prototype to produce comprehensive coverage ofthe review session. Based on literature and additional casestudies, an analysis of the information streams is presented.Furthermore, an Interactive Augmented Prototyping solu-tion is devised. Early verifications show that the recordingresolution still requires a lot of fine-tuning, which will bethe primary focus prior to a comprehensive evaluation inpractice.

J. Verlinden (B) I. HorvthFaculty of Industrial Design Engineering,Delft University of Technology, Landbergstraat 15,2628 CE Delft, The Netherlandse-mail:

I. Horvthe-mail:

T.-J. NamDepartment of Industrial Design,KAIST, Gusung-Dong, Yusung-Gu,Daejeon, Republic of Koreae-mail:

Keywords Augmented reality Augmented prototyping Design review Recording Industrial design

1 Introduction

In conducting a number of empirical field studies we foundthat collective decision making and discussions during designreview meetings were essential in driving the design pro-cess. Even among the most pragmatic and functionalist think-ing people miscommunication occurs, leading to sub-optimaldesigns, unnecessary delays and distrust. The recording andrecollection of design review meetings is important; futuredecisions sometimes are based on the remembrance of priormeetings. Often, this administrative activity gets little atten-tion or is over formalized to record specific events of the workbut not its full discussion. Furthermore, meeting minutes canbe biased, as they are made from the point of view of onlyone of the stakeholders.

As can be found in the field of Interactive Design [8], a vastcollection of advanced prototyping techniques have emerged,including virtual and augmented solutions. Such techniquesengage intricate simulation and multi-sensory interactionprinciples for improving decision-making in product designand manufacturing. However, the primary focus of this rese-arch domain is of a functionalist viewe.g. to simulate andvalidate product tests. We believe other, softer views shouldget more emphasis in order to become truly interactive pro-totyping means that has a positive influence on the overalldesign process [37].

In this article, we introduce a new prototyping concept,based on augmented reality technology. The presented I/OPad is unique as it supports prototyping and recording inparallel. By employing this solution, physical prototypes areenriched with additional information (colour, features) that


190 J. Verlinden et al.

can be altered in situ, while all users can observe and interactwhen necessary. Based on extensive design case studies, weclaim that these are well supporting the needs and require-ments of stakeholders during design reviews in the field ofindustrial design.

This paper is structured as follows: first, literature is dis-cussed covering design reviews, augmented prototyping, andrelated recording systems. Then, the proposed recordingmethod is presented, which comprises both hardware andsoftware concepts. Next, an initial system is presented toexplore and validate the basic issues. The final section spec-ifies the following steps in developing and evaluating thissystem.

2 Backgrounds

2.1 Interactive Augmented Prototyping

The concept of Interactive Augmented Prototyping (IAP)employs augmented (mixed) reality technologies to com-bine virtual and physical prototypes. Enabling technologiesencompass display means, position sensing methods, inter-action techniques, and physical model manufacturing. Inparticular, the display for augmentation ranges from videomixing and see-through displays to spatial augmentedreality; in the latter case video projectors are used to castcomputer imagery directly on physical objects [2]. Althoughprojector-based display techniques have their limitations e.g.[12] it is easily accessible by most design studios. Humanfactors studies indicate that projector-based AR performsbetter than video-mixing head-mounted displays [23] andprovides more object-presence than VR-based techniquesas the virtual workbench [31]. Powerful interaction tech-niques can be shared in such systems. One of the first exam-ples of this technique was presented by Underkoffler andIshii [34], in which an urban planning scenario was fol-lowed. Physical wire frames represented blocks that couldbe placed arbitrarily on a plane. Real-time simulations wereprojected on the table, including reflections, shadows, andwind turbulence in its surroundings. Recent progress in thisfield allows sketching on arbitrary surfaces from variousdistances [4].

For the work presented in this paper, the projector-basedAR display will be used to merge digital modelling and sim-ulation with physical models, either made by hand or auto-matically by 3D printing or Rapid Prototyping processes. Themain advantage of IAP techniques lies in its ability to providenatural haptic/tactile feedback and its mix with the physicalenvironment. It constitutes an embodied interface, allowingnatural spatial reasoning and supports social interaction incollaborative settings [6]. Compared to traditional physicalprototyping techniques, possible advantages of Interactive

Augmented Prototyping are (1) the display of a new type ofinformation (e.g. the wind simulation in the example above),(2) increasing the intensity of particular type of information(e.g. material expression by including texture maps), and (3)increasing the richness of interacting or the sense of engage-ment with the artefact representation.

2.2 Case studies in design

In order to obtain insight in the possibilities and limitations ofcurrent prototyping practice in industrial design, the authorshave executed an empirically study of three design projectsin different sub domains: the design of a tractor, a hand-held oscilloscope, and the interior for a museum [35]. Ourobjective was to produce a deep and accurate account ofprototyping and modelling activities in a range of industrialdesign engineering domains, with a primary focus on productrepresentation and design reviews. Although the design pro-cesses differed in many respects, common issues originatedfrom mismatches between stakeholders, either because somedesign aspects are challenging to explicate (like aesthetics)or differences in values and attitudes.

From the case studies, it was quite apparent that designreview meetings were one of the most influential constituentsof the design process in which prototypes and other designrepresentations are essential. In our forerunning theory form-ing, notions from Critical Systems Thinking are applied.According to this framework design review meetings andthe employment of prototypes should be viewed from fourseparate paradigms, covering (1) the functionalist stance,which focuses on utilitarian aspects such as the efficiencyand quality of design results, (2) the interpretive, which isaimed at establish consensus and shared understanding, (3)the emancipatory, which considers power relations, e.g. howdecision making and trust is influenced, and (4) the postmod-ernist, which attempts to cultivate an entertaining, pluralisticand creative atmosphere. Examples and issues concerningthese were discussed in [37].

Based on the three case studies, a number of functions forInteractive Augmented Prototyping systems could be identi-fied, categorized in four usage scenarios: user studies, explo-ration, design review and presentations to customers/highermanagement. The full collection includes over 29 functions.The ones that refer to design reviews are summarized inTable 1. In our case studies, the review sessions followeda structured agenda, often supported by a slideshow that pre-sented solutions and design issues. In one of the case stud-ies, the power of the narrative, storytelling was stressed; asa means to address abstract design aspects without resortinginto concrete visual examples. Another important finding wasthe variety of roles a physical models play during a meet-ing with heterogeneous group of attendants., ranging from


Recording augmented reality experiences to capture design reviews 191

Table 1 Design review functions derived from three case studies

Internal discussion of design alternatives, capturing interaction and reflections (annotation)

Freehand sketching on surface (captured with author + timestamp for later use)

Present user studies: usage feedback, co-located events and subjective evaluations

Presentation of design alternatives, capturing interaction and reflections and possibly design decisions (annotation)

Presentations of design exploration scenarios to