Baya, ConceptDesign

INFORMATION TO USERS

This manuscript has been reproduced from the microfilm master. UMI

films the text directly from the original or copy submitted. Thus, some

thesis and dissertation copies are in typewriter face, while others may be

from any type of computer printer.

The quality of this reproduction is dependent upon the quality of the

copy submitted. Broken or indistinct print, colored or poor quality

illustrations and photographs, print bleedthrough, substandard margins,

and improper alignment can adversely affect reproduction.

In the unlikely event that the author did not send UMI a complete

manuscript and there are missing pages, these will be noted. Also, if

unauthorized copyright material had to be removed, a note will indicate

the deletion.

Oversize materials (e.g., maps, drawings, charts) are reproduced by

sectioning the original, beginning at the upper left-hand comer and

continuing from left to right in equal sections with smalI overlaps. Each

original is also photographed in one exposure and is included in reduced

form at the back of the book.

Photographs included in the original manuscript have been reproduced

xerographically in this copy. Higher quality 6' x 9" black and white

photographic prints are available for any photographs or illustrations

appearing in this copy for an additional charge. Contact UMI directly to

order.

UMI A Bell & Howell Information Company

300 North Zeeb Road. Ann Arbor MI 48106-1346 USA 313n61-4700 800/521-0600

INFORMATION HANDLING BEHAVIOR OF DESIGNERS DURING

CONCEPTUAL DESIGN: THREE EXPERIMENTS

A DISSERTATION

SUBMITIED TO THE DEPARTMENT OF MECHANICAL ENGINEERING

AND THE COMMITTEE ON GRADUATE STUDIES

OF STANFORD UNIVERSITY

IN PARTIAL FULLFILLMENT OF THE REQUIREMENTS

FOR THE DEGREE OF

DOCTOR OF PHILOSOPHY

IN

MECHANICAL ENGINEERING

Vinod Baya

September 1996

UMI Number: 9714076

Copyright 1996 by Baya, Vinod

All rights reserved.

UMI Microform 9714076 Copyright 1997, by UMI Company. All rights reserved.

This microform edition is protected against unauthorized copying under Title 17, United States Code.

UMI 300 North Zeeb Road Ann Arbor, MI 48103

© Copyright by Vinod Baya 1996 All Rights Reserved

ii

I certify that I have read this dissertation and that in my opinion it is fully adequate. in scope and qUality. as a dissertation for the degree of Doctor of Philosophy.

I certify that I have read this dissertation and that in my opinion it is fully adequate. in scope and quality. as a dissertation for the degree of Doctor ofP ·osophy.

I certify that I have read this dissertation and that in my opinion it is fully adequate. in scope and qUality. as a dissertation for the degree of Doctor of Philosophy.

~.

Approved for the University Committee on Graduate Studies:

iii

Abstract

This dissertation demonstrates an iterative Observe->Analyze->Intervene, design

research methodology to incrementally improve the understanding and the sup

port of information handling in the conceptual design process. It reports on two

observational studies, based on the verbal protocol method, and an information

management service.

The first observational study is a detailed analysis of the questioning behavior of

designers to understand information needs during a redesign task. This resulted

in the design information framework, which classifies information that should be

captured during a design process for effective reuse at a later date. This frame

work was used to develop an information management service called Dedal, a

tool for indexing, modeling and retrieving design information. Observations from

deployment and usage of Dedallead to the second observational study. This was

a study of the information handling behavior of individual designers doing short

conceptual design tasks. This study resulted in the Information Handling Frame

work (lliF).

The lliF is a framework for understanding information handling behavior during

conceptual design. Some key observations from the use of this framework are:

• designers move frequently between different types of information (on an average every 13 seconds),

• they handle information about upto 40 concepts in one minute, • their ability to work fluidly and with ease while handling all types of infor

mation is essential during the conceptual design process, and • the points of transition between different information types are critical from

the viewpoint of computational support.

Deeper understanding of the information handling behavior is explained by

means of detailed qualitative and quantitative results. The implications of these

results towards the improvement in the understanding of the conceptual design

process and the recommendations on the development of intuitive and integral

information handling services are discussed.

iv

Acknowledgements

I would like to thank Prof. Larry Leifer for his patient, generous and liberal guid

ance over the course of this dissertation. Larry, I thank you specially for sharpen

ing my communications and management skills and encouraging me to pursue

this line of research.

I would like to thank Prof. Mark Cutkosky, Prof. Sheri Sheppard and Dr. Cathe

rine Baudin for providing valuable feedback on the drafts of this dissertation.

Catherine, Thank you also for giving me an opportunity to work on Dedal.

I would like to acknowledge all the members of the Dead Designer Society: Ade

Mabogunje, David Cannon and Margot Brereton for many things, but specially

for friendship and support over all these years.

I would like to acknowledge students and staff at the Center for Design Research

for providing a very stimulating and amiable research environment. Noelle,

thanks for all the CDR-socials and the numerous other trips.

It would also like to thank my friends in the horse polo world, in and outside of

Stanford, who added a whole new dimension to my experience on the Farm.

Thank you Stanford Polo Club members and horses.

I would like to thank the AI Branch (now the Information Sciences Division) at

NASA Ames Research Center for supporting this research. Thanks also to

TUDelft and Xerox P ARC for providing data for one of experiments used in this

disserta tion.

Finally, I would like to thank my family in India for their patience, understanding

and support while I took my time completing this chapter of my life.

v

Table of Contents

Chapter 1. Understanding and Improving Engineering Design Practice .................................................. 1

1.1 Background: Engineering Design ................................. 2

1.1.1 Information Handling in Design ..... '" ...................... 2 1.1.2 Conceptual Design Focus .................................... 4 1.1.3 Research Objectives ......................................... 5

1.2 Research Methodology .......................................... 6

1.2.1 Grounding Research in Reality ............................... 6 1.2.2 Observation by Verbal Protocol .............................. 7 1.2.3 Improving Design Practice: Incremental Support ............... 8

1.3 Analysis of Verbal Protocol Data ................................. 9

1.3.1 Information Handling and Design Information Framework ..... 10 1.3.2 Information Reuse Study: Analysis Example .................. 12 1.3.3 Information Handling Study: Analysis Example ............... 12

1.4 Evolution of the Information Handling Perspective ................ 13

1.5 Key Results ................................................... 16

1.5.1 Key Results from Information Reuse Study ................... 16 1.5.2 Key Results from Dedal Development & Usage ................ 16 1.5.3 Key Results from Information Handling Study .. , ... " ........ 17

1.6 Knowledge Gained ............................................. 17

1.7 Thesis Organization ............................................ 18

Chapter 2. Design Theory & Methodology Research: Literature Re-vie'W' ............•.........•.............•........•...... 20

2.1 Design Theory and Methodology Research ....................... 21

2.1.1 Prescriptive Studies ........................................ 21 2.1.2 Descriptive (Empirical) Studies .............................. 24 2.1.3 Verbal (or Think-Aloud) Protocol Method .......... " ........ 26 2.1.4 Studies Using Verbal Protocol Method ....................... 28 2.1.5 Studies Using Other (Non-Verbal Protocol) Methods ........... 31

2.2 Design Process: An Information Handling Perspective ............. 32

vi

Chapter 3. Design Information Needs, Capture & Reuse ....... 35

3.1 Design Information ............................................ 36

3.1.1 Information Reuse Essential for Good Design Practice .......... 36 3.1.2 Design Information Capture is Difficult ...................... 37

3.1.3 Information Reuse Study Objectives ......................... 38

3.1.4 Questions Reflect Information Needs ......................... 39

3.2 Experimental Study: Verbal Protocol Method ..................... 39

3.2.1 Experimental Setup and Procedure .......................... 39

3.2.1.1 The Design Problem: Continuously Variable Damper .... 41

3.3 Analysis of Verbal Data ........................................ 42

3.3.1 Transcription ............................................. 42

3.3.2 Question Extraction ........................................ 43

3.3.3 Question Reformulation .................................... 43 3.3.4 Question Classification ..................................... 43

3.4 Question Classification Framework .............................. 44

3.4.1 Design Information Framework (DIF) ........................ 45

3.4.2 Descriptor ................................................ 45

3.4.3 Subject-Class .............................................. 45

3.4.4 Medium .................................................. 46

3.4.5 Level-of-Detail ............................................ 48

3.5 Analysis Examples ............................................. 48

3.6 Results ........................................................ 48

3.6.1 Results on Descriptors ..................................... 50

3.6.2 Results on Subject-Class .................................... 51

3.6.3 Results on Level-of-Detail .................................. 52

3.6.4 Results on Crossing Attributes .............................. 52

3.7 Discussion .................................................... 54

3.8 Recommendations for Information Reuse Service ................. 56

3.9 Summa.ry-...................................................... 56

Chapter 4. Dedal: An Information Management Service ........ 57

4.1 Introduction to Dedal ........................................... 58

4.1.1 Indexing Tool ............................................. 59

4.1.2 Modeling Tool ............................................ 60

4.1.3 Retrieval Tool. ............................................ 60

vii

4.2 Dedal Test Domains and Results ................................ 62

4.2.1 Damper Domain Results ................................... 63

4.2.2 Bioreactor Domain Results .................................. 65

4.3 Discussion on Dedal: Lessons Learned ........................... 66

Chapter 5. Information Handling Behavior Framework ........ 68

S.1 Introduction ................................................... 69

5.1.1 Experimental Setup and Procedure .......................... 69

5.2 Analysis Procedure ............................................. 70

5.2.1 Segmenting into Information Fragments ...................... 70 5.2.2 Classifying Information Fragments .......................... 71

5.3 Information Handling Framework (lHF) .......................... 72

5.3.1 Informational Activity ..................................... 74 5.3.2 Level of Abstraction ....................................... 76

5.3.3 Design Information Measure (dim) . .......................... 77

5.3.3.1 Definition of dim: Assumptions ....................... 78 5.3.3.2 Rules for measuring dims ............................. 79

5.4 Analysis Example .............................................. 80

5.5 Summary ...................................................... 82

Chapter 6. Understanding Information Handling Behavior: Quan-titative & Qualitative Results ............................. 83

6.1 Introduction ................................................... 84

6.2 Information Fragment Duration (deitat) .......•..•...•.........•. 85

6.2.1 deltat Variation over Time .................................. 85

6.2.2 Statistics on deltat . ......................................... 85 6.2.3 Discussion on deltat Results ................................. 86

6.3 Design Information Measure (dim) . ...........•......••....•..... 88

6.3.1 Information Handling Rate ......... , ....................... 88

6.3.2 Discussion on Design Information Measure Results ............ 91

6.4 Distribution of Time and Design Information ..................... 91

6.4.1 Informational Activities Distribution ......................... 91

6.4.2 Information Descriptor Distribution .......................... 92

6.4.3 Information Subject-Class Distribution ....................... 93 6.4.4 Information Level of Abstraction Distribution ................. 94

viii

6.4.5 Discussion on Time and Information Distribution .............. 95

6.5 State Transition Analysis: Statistics and Probabilities .............. 96

6.5.1 Informational Activity State Transition Analysis ............... 97 6.5.2 Information Descriptor State Transition Analysis .............. 97 6.5.3 Information Subject-Class State Transition Analysis ............ 99 6.5.4 Information Level-of-Abstraction State Transition Analysis ..... 99 6.5.5 Discussion on State transition Analysis ...................... 100

6.6 Multiple Attributes Analysis ................................... 101

6.6.1 Informational Activities and Information Descriptor .......... 101 6.6.2 Informational Activities and Information Subject-Class ........ 102 6.6.3 Informational Activities and Level-of-Abstraction ............ 102 6.6.4 Most Frequent Information Fragments ...................... 103

6.7 Summary ..................................................... 103

Chapter 7. Conclusions & Future Work ...................... 105

7.1 Concluding Remarks .......................................... 106

7.2 Contributions ................................................. 107

7.3 Discussion ................................................... 107

7.4 Future Work .................................................. 108

Bibliography .............................................. 110

Appendix A. Additional Material on Design Information Reuse Study ................................................. . 117

A.1 Design Experiment Material ................................... 117

A.I.1 Design Experiment Problem Statement ............. '" ..... 117 A.I.2 Description of ME210 ..................................... 120

A.2 Results on individual subjects ................................. 122

A.3 Analysis Examples ............................................ 123

Appendix B. Additional Material on Dedal Development ..... 126

B.1 Dedal Implementation Details ................................. 126

B.2 Dedal Interfaces for Indexing, Querying and Modeling ........... 127

B.3 Examples of Heuristics ........................................ 129

ix

B.4 References on Dedal papers .................................... 133

Appendix C. Additional Material on Design Information Handling Study ................................................. . 134

C.1 Instructions: Overview ........................................ 135

C.2 Design Exercise: Problem Statement Bike Lock 1 ................. 136

C.3 Design Exercise: Problem Statement Bike Lock 2 ................. 139

C.4 Design Expt. Problem Statement: Backpack Harness ........ _ ..... 142

Appendix D. Additional Results on Information Handling Study ................................................ 143

0.1 Additional Results on deltat . .................................. 143

0.1.1 deltat vs. ~e ............................................ 143 0.1.2 Distribution of deltat: histograms ........................... 148

0.2 Additional Results on Information Handling Rate ............... 149

0.2.1 Information handling rate over time ........................ 149 0.2.2 Distribution of information handling rate ................... 152 0.2.3 Dim rate vs. Verbal rate: Regression analysis ................ 153

0.3 Information Fragment Attributes Behavior Over Time ............ 153

0.4 Additional Results on Distribution of Time ..................... 158

0.5 Results on State Transition Probabilities ........................ 160

0.5.1 Informational activities state transition probabilities .......... 160 0.5.2 Information descriptor state transition probabilities .......... 161 0.5.3 Information subject-class state transition probabilities ........ 162 0.5.4 Information level-of-abstraction state transition probabilities .. 162

0.6 Additional Results from Multiple Attribute Analysis ............ 162

0.6.1 Information descriptor and informational activities ........... 163 0.6.2 Information subject-class and informational activities ......... 164 0.6.3 Information level-of-abstraction and informational activities ... 164 0.6.4 Information descriptor and subject-class .................... 165 0.6.5 Information descriptor and level-of-abstraction .............. 166 0.6.6 Information subject-class and level-of-abstraction ............ 166

Appendix E. References on Papers from this Dissertation ..... 167

E.1 ASME Conference: DTM-1992 .................................. 167

x

E.2 ASME Conference: DTM-1994 .•................................ 167

E.3 AS ME Conference: DTM-1995 .................................. 167

E.4 Chapter in Analysing Design Activity: 1996 ...................... 167

xi

List of Figures

Figure 1-1 Strategy for grounding research in reality ....................... 7

Figure 1-2 Strategy for incrementally improving design practice ............. 8

Figure 1-3 Template for classifying information fragments. . . . . . . . . . . . . . . .. 13

Figure 1-4 Evolution of the work presented in this dissertation.. . . . . . . . . . . .. 14

Figure 3-1 An information perspective on the design process. . . . . . . . . . . . . .. 37

Figure 3-2 Experimental setup for information reuse study .... " ........... 40

Figure 3-3 Analysis procedure for information reuse study ................. 42

Figure 3-4 Steps in the development of question classification framework. . .. 44

Figure 3-5 Template for interpreting question and information segment.. . . .. 46

Figure 3-6 Distribution of question deScriptor ............................ 50

Figure 3-7 Distribution of question subject-class. ........... . . . . . . . . . . . . .. 51

Figure 3-8 Distribution of question level of detail. .......... . . . . . . . . . . . . .. 52

Figure 4-1 Dedal system architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 58

Figure 4-2 Example of conceptual indexing ............................. " 60

Figure 4-3 Example of a model in Dedal. ................................ 61

Figure 4-4 Flow chart of the retrieval algorithm used by Dedal.. . . . . . . . . . . .. 61

Figure 4-5 Retrieval performance of conceptual indexing.. . . . . . . . . . . . . . . . .. 64

Figure 5-1 Analysis procedure for information handling study .............. 71

Figure 5-2 Template for classifying an information fragment. .............. 72

Figure 5-3 Evolution of the informational activity classification. . . . . . . . . . . .. 75

Figure 6-1 Information fragment duration vs. time for S1. ................ " 86

Figure 6-2 Distribution of information fragment duration ................ " 87

Figure 6-3 Information handling rate over time for 54 .................... " 89

Figure 6-4 Distribution of time and design information across informational activ-ities ...................................................... " 92

Figure 6-5 Distribution of time and design information across descriptor ... " 93

Figure 6-6 Distribution of time and design information across subject-class. " 94 Figure 6-7 Distribution of time and information across level-of-abstraction. " 95

Figure 6-8 Variation in level-of-abstraction over time for 54 .............. " 96

Figure 6-9 Informational activity state transition statistics ................ " 98 Figure 6-10 Information descriptor state transition statistics .............. " 98

Figure 6-11 Information subject-class state transition statistics ............ " 99

Figure 6-12 Information level-of-abstraction state transition statistics. . . . . .. 100

Figure A-I Distribution of question descriptor for RS1 and RS2. . . . . . . . . . .. 122

xii

Figure A-2 Distribution of question subject-class for RS1 and RS2. . . . . . . . .. 122

Figure A-3 Distribution of question level-of-detail for RS1 and RS2. . . . . . . .. 123

Figure B-1 Dedal's indexing and querying environment. ................. 127

Figure B-2 Dedal's modeling interface .................................. 128

Figure B-3 Dedal's answers window. .................................. 129

Figure D-1 Behavior of deltat over time for 51. . . . . . . . . . . . . . . . . . . . . . . . . .. 144

Figure 0-2 Behavior of deltat over time for 52. . . . . . . . . . . . . . . . . . . . . . . . . .. 144

Figure 0-3 Behavior of deltat over time for 53 ........................... 145


Figure 0-5 Behavior of deltat over time for 55 ........................... 146


Figure 0-7 deltat behavior over time for all exp-subjects .................. 147

Figure 0-8 Distribution of the information fragment duration ............. 148

Figure 0-9 Information handling rate over time for 51 ................... 149

Figure 0-10 Information handling rate over time for 52 .................. 150

Figure 0-11 Information handling rate over time for 53 .................. 150

Figure 0-12 Information handling rate over time for 54. . . . . . . . . . . . . . . . . .. 151

Figure 0-13 Information handling rate over time for 55 ............... . .. 151

Figure 0-14 Information handling rate over time for 56 ................... 152

Figure 0-15 Distribution of dim rate for all exp-subjects. ................. 152

Figure 0-16 Regression analysis between dim rate and verbal rate ......... 153

Figure 0-17 Behavior of informational activity over time for 51. . . . . . . . . . .. 154

Figure 0-18 Behavior of Information descriptor over time for 52. . . . . . . . . .. 155

Figure 0-19 Behavior of Information subject-class over time 53. . . . . . . . . . .. 156

Figure 0-20 Behavior of Information level-of-abstraction over time 54. . . . .. 157

Figure 0-21 Distribution of time across informational activity. . . . . . . . . . . .. 158

Figure 0-22 Distribution of time across information descriptor. ........... 158

Figure 0-23 Distribution of time across information subject-class.. . . . . . . . .. 159

Figure D-24 Distribution of time across information level-of-abstraction. ... 159

Figure 0-25 Information activity state transition probabilities.. . . . . . . . . . . .. 160

Figure 0-26 information descriptor state transition probabilities. . . . . . . . . .. 161

Figure 0-27 Information subject class state transition probabilities. ..... . .. 162

Figure D-28 Information level-of-abstraction state transition probabilities. .. 163

xiii

List of Tables

Table 1-1 Analysis procedure ........................................... 10

Table 1-2 Information handling and design information framework. ......... 11 Tabie 2-1 Empirical design research methods .............................. 25

Table 3-1 Framework to classify questions ................................. 45 Table 3-2 Question descriptors .......................................... 47

Table 3-3 Question subject-class ......................................... 47

Table 3-4 Question level-of-detail ........................................ 48

Table 3-5 Example of analysis procedure in information reuse study .......... 49

Table 3-6 Information reuse study: subjects background information .......... 50

Table 3-7 Crossing descriptor with subject-class ............................ 53

Table 3-8 Crossing descriptor with level-of-detail .......................... 54 Table 3-9 Crossing subject-class with level-of-detail.. ....................... 54 Table 4-1 Dedal test domains details ...................................... 63

Table 5-1 Details on experiments and experimental subject background ....... 70 Table 5-2 Information handling framework (ll-IF) .......................... 73

Table 5-3 Descriptors added to the design information framework ............ 74 Table 5-4 A classification for the Informational activities .................... 76 Table 5-5 Definition and interpretation for level-of-abstraction ............... 77 Table 5-6 Examples of dim measurement .................................. 80

Table 5-7 Examples of information handling behavior analysis ............... 80

Table 6-1 Information fragment duration (deltat) statistics ................... 87

Table 6-2 Information handling rates and verbal rates for all subjects ......... 90 Table 6-3 Distribution of information activities across descriptors. . ......... 101

Table 6-4 Distribution of informational activity across subject-class .......... 102

Table 6-5 Distribution of informational activities across level-of-abstraction ... 102

Table 6-6 Frequently encountered info-fragment attribute combinations ...... 103

Table 6-7 Summary of the key findings and recommendations .............. 104 Table A-I Example of analysis procedure in information reuse study ......... 124 Table 0-1 Distribution of descriptor across the informational activities ....... 163 Table 0-2 Distribution of subject-cass across the informational activities ..... 164 Table 0-3 Distribution of level-of-abstraction across informational activities .. 164 Table 0-4 Distribution of deScriptor across the subject-classes ............... 165 Table 0-5 Distribution of descriptor across the levels-of-abstraction. . ....... 166 Table 0-6 Distribution of subject-class across the levels-of-abstraction ....... 166

xiv

Glossary

Here are some common abbreviations and terms used in this dissertation:

OAl Observe ->Analyze -> Intervene iterative cycle

OIF Design Information Framework

IHF Information handling Framework

EDN Electronic Design Notebook

ms Information Reuse Study

IHS Information Handling Study

DS [DeScriptor, Subject] pair in information fragment classification

dim Design Information Measure (quantitative measure)

dpm dims per minute

wpm words per minute

deltat Information fragment duration in seconds

xv

Chapter 1

Understanding and Improving Engineering Design Practice

Design Research

TO

,

Develop Research Methods

< Goal > shortest path

T~ Understand Design Practice

I~

Deliver High Quality, Low Cost Products

~

TO

. Improve Design

Practice

~o

This chapter is a summary of the research work being reported in this dissertation. It discusses the evolution of this dissertation and the motivation for pursuing the research. It briefly describes the experimental approach and analysis procedure, developed to carry out two observational studies, namely information reuse study and information handling study. It lists the contributions that this dissertation is making to the knowledge in the field of design theory and methodology. It ends with a narration on the content of the remaining chapters.

Information Handling Behavior of Designers During Conceptual Design: Three Experiments 1

Chapter 1: Understanding and Improving Engineering Design Practice 2

1.1 Background: Engineering Design

Engineering design is a complex process involving people, multiple technical

and non-technical disciplines, material and non-material resources, computa

tional and non-computational tools and a variety of cognitive strategies. As a

result, studying, understanding and supporting this process is very challenging.

Given its complexity, there is a potential for substantial savings in cost, labor

and resources by improving engineering design practice. Improvement can be

achieved by supporting design practice with tools and services which accrue sav

ings over the course of the process. Economic pressures to shorten product

development cycles and improve market responsiveness have increased interest

in the study of design theory and methodology both in industry, in academia

and in design education. Developing methods to study and understand engi

neering design, and using this understanding to build services! to support this

process is essential for developing high quality products at low cost. This thesis

addresses the need for new methods required in design research for improving

design process understanding.

Design process can be studied from many perspectives such as, technical, com

putational, problem solving, organizational, social and others. Each perspective

has a potential to lead to a better understanding of the process and lead to

improved design practice. I have chosen to approach the engineering design pro

cess from an information handling viewpoint.

1.1.1 Information Handling in Design

In this dissertation information handling refers to activities and sub-processes

undertaken by the designer which involve design information2. For instance

1. The term services will be used in this thesis to refer to tools and(or) methods (computational or non-computational) that can support engineering design practice. 2. The terms design information and infonnntion will be used synonymously in this thesis unless stated otherwise.

Information Handling Behavior of Designers During Conceptual Design: Three Experiments


activities dealing with generating, capturing, accessing, transforming, indexing,

structuring and analyzing information with the intent to create an artifact, are

some examples of information handling. While it is possible to construe all

actions of a designer as information handling, for the purpose of this study I

only consider the actions pertinent to the design task on hand.

Engineering design is an information intensive process. Over the duration of a

design project designers handle a large amount of information. The information

is handled by means of numerous non-computational (paper notebooks, cata

logs, handbooks, telephone etc.) as well as computational tools (CAD /CAE/

CAM software, word processors, spreadsheets, e-mail etc.). The Information

exists in a variety of media such as: text, graphic, audio, video, CAD drawings

and others. This diversity imposes considerable burden of information manage

ment on designers. As a result, the efficiency and quality of the design process

depends heavily on how capable designers are in handling information.

To find out about the information requirements of engineering designers, [Court

et. a11993] conducted a questionnaire based survey of about 200 designers from

various industries located in the United Kingdom. They found that on average

designers spent 18% of their time searching for information, 23% dealing with

paperwork, 16% in meetings and only 43% of their time designing. Hales [1987]

performed a detailed analysis of all the activities of one project involving the

design of a high-pressure, high-temperature system for testing materials. He

was a participant observer! in this project over a period of 2.8 years and involv

ing 37 different participants. Hales found that only 47% of the total effort was

spent in doing design. The remaining 53% was spent in information retrieval,

planning, cost estimating, reviews, helping others and social contact. These two

studies strongly suggest that there is an opportunity to improve design practice

by supporting information intensive activities such as, information gathering,

1. An observational technique in which the observer is also a member (participant) of the design team.

Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS


documentation, communication, information management and others. A reduc

tion in information related activities will lead to an improvement in designers

capacity to do better designs.

All over the world, there is a steady movement towards electronic commerce.

Much of information that is available today in paper medium (design notebooks,

libraries, handbook, catalogues etc.) will be available in electronic formats over

computer networks. As a result, many transactions which today are paper based

will become electronic (group communications, interaction with vendors etc.).

This transition will significantly impact design practice in industry. Designers

will have easy access to a large amount of information, resulting in increased

information management burden. We need to make sure that services that are

made available to designers in the electronic workplace integrate smoothly with

their work environment, so that designers can handle information efficiently.

This will be possible if services are built based on a good understanding of the

information handling behavior of designers. Developing this understanding is

the core concern in this dissertation.

1.1.2 Conceptual Design Focus

Among the commonly agreed upon phases of the design process, conceptual, lay

out and detail [Cross 1990, Page 21], the conceptual phase should deserve special

attention from an information handling perspective. Conceptual phase refers to

the ideational stage of the design process where ideas are generated and analy

sed, and choices are made. In this stage, ideas and solutions are described

abstractly without quantitative detail. While conceptual activity takes place all

along the design process [Guindon 1990], it occurs more frequently in the early

stage of the process. It is this early stage that is of interest in this dissertation.

The conceptual phase happens to be the one where important decisions are

made [Wood and Antonsson 1987]. Also, it is the most informal, and the least

Infor=tion Handling Behavior of Designers During ConccpruaJ Design: Three Experiments


understood [Rabins et al. 1986]. It is also the stage for which there is little compu

tational support [Rouse and Boff 1987]. Decisions and information generated in

this phase have a big impact on the downstream design process and the overall

cost. It has been reported that upwards of 80% of the final cost are obligated dur

ing conceptual design [National Materials Advisory Board 1991]. This means

that understanding and supporting information handling in this phase is very

likely to lead to Significant betterment of design practice. This is why this disser

tation focuses on conceptual design.

1.1.3 Research Objectives

The primary objectives of this dissertation are to model the information han

dling behavior of designers during conceptual design, and develop

requirements to guide the evolution of information handling support services. A

secondary objective is to develop a research methodology for understanding and

improving engineering design practice. Several questions have guided this

inquiry:

1. What is design information?

2. Can design information be classified?

3. Are design information needs during conceptual work special?

4. Are some types of design information more important than others?

5. What are the activities designers perform with design information?

6. vVhat is the temporal distribution of design information processing?

7. Can design information be quantitatively measured?

8. What is the information content in different activities?

9. How does information processing rate change with the design process? and,

10. What do answers to the above questions inform us about services needed to support information handling in the design process?

Information Handling Behavior of Designers During Conceplual Design: Three ExperimenlS


1.2 Research Methodology

An underlying concern in this study is to build services which would support

and improve design practice. This required that the research methodology give

adequate attention to, Ca) understanding the design process and (b) development

of computational services. Three research strategies were used in this study to

maintain thls balance,

1. Grounding the research in reality: the research paradigm was grounded in reality by experimenter's involvement in real design practice.

2. Understanding the design process: design process was examined in detail by means of observational studies, and

3. Improving design practice: design process was incrementally supported by a prototype service and the performance of designers using the prototype service was assessed.

These three strategies are described in detail in the subsections below. In the dis

cussion observational study refers to the method adopted in this dissertation to

understand the design process. Two observational studies were conducted, one

to understand information reuse and the other to understand information handling

behavior.

1.2.1 Grounding Research in Reality

A special feature of the research presented in this dissertation is shown in

Figure 1-1. The figure is meant to indicate that the researcher should have a rich

involvement in real design activity during observational studies and service

development. The involvement in real design activity is motivated by the con

cern for assuring that formal observations from the experiments are consistent

with experiences from real design activity, and that during service development

there is constant feedback from an ongoing design project.

Results from observational studies are the basis for guiding service develop

ment, and feedback form service development to observational studies is only

Information Handling Behavior of Designers During Conccptual Design: Three ExperimcnlS

Chapter 1: Understanding and Improving Engineering Design Practice

Service Development -Dedal

Observational Studies (formal observation)

- Information Reuse - Information Handling Behavior

Real Design (informal observation)

- Variable Damper - Bioreactor

Figure 1-1 Strategy for grounding research in reality. Double arrow 1 is a means for assuring that formal observations from the experiments are consistent with informal observations made from real design activity. Double arrow 2 is a means for assuring that during service development there is constant feedback from an ongoing real design project. Single arrow 3 indicates that service development is based on results of observational studies.

7

through real design activity (since a service has to be incorporated in real design

activity before its performance can be examined in an observational study). Alto

gether this strategy provides a mechanism for grounding the research in reality.

During the evolution of this dissertation, which covers two observational studies

and development of a computational service (Dedal), I was a member of the two

design projects, Variable Damper and Bioreactor, as shown in Figure 1-1.

1.2.2 Observation by Verbal Protocol

Observational studies of the design process can be conducted using various

methods. Some of the popular methods are discussed and compared in Chapter

2. This study used the verbal protocol method. This method is popular in single

subject experiments and has been used widely by cognitive scientists [Newell

1968] and design researchers [Eastman 1970; Akin 1979 and Stauffer 1987b]. In

verbal protocol method, experimental subjects are asked to talk out loud while

they work on a design problem. In this manner the actions and intentions of the

subject become explicit in the verbal recording. The transcript of this recording



is the raw data for analysis.

There is debate in the research community regarding the appropriateness of this

method, since it is not understood in what manner and how much the act of ver

balizing affects the natural thought process and the behavior of the experimental

subject. Despite these limitations, verbal protocol is a popular method for single

subject studies because of its relative simplicity and tacit assurance that what is

said "is at least subjects own words". Procedures and guidelines have been

developed for collecting and analyzing such data so that the results obtained are

valid and free from bias [Ericsson and Simon 1993]. The appropriateness of this

method is discussed further in Chapter 2.

1.2.3 Improving Design Practice: Incremental Support

A strategy for improving design practice by incrementally supporting the design

process! is shown in Figure 1-2. This approach has been advocated and used by

Analyze Analyze

! Observe Intervene --.~~ Observe Intervene --.~~ Observe

Improved Design Practice -----.~~

Figure 1-2 Strategy for incrementally improving deSign practice. The design process is cyclically observed, analyzed and intervened while incrementally developing services to facilitate and improve design practice.

[Tang 1989 and Minneman 1991] in their research in improving collaborative

1. It is an assumption that incremental improvements in understanding and support of the design process will lead to improved design practice over time. This is not being proved in this thesis.

Information Handling Behavior of Designer.; During Conceptual Design: 1breo: ExperimenlS


design practice. The fundamental assertion of this methodology is that hypothe

sis free observation of design activity must precede the development of services

intended to support that activityl.

The process starts by conducting observational studies of design activity to col

lect raw data. The data is analyzed to gain an understanding of the design

activity and to develop frameworks and requirements for services which will

improve the observed design practice. Using these frameworks and require

ments, services are built and introduced into the original design process. The

introduction of the service into the design process is an intervention, since it

changes the original process itself. A second cycle of observational study is

needed to understand the changed process. The cycle of observe -> analyze ->

intervene (OAl) goes on, and the design process is incrementally supported,

restructured and improved.

1.3 Analysis of Verbal Protocol Data

A total of eight verbal protocol experiments were conducted. Two were con

ducted in the information reuse study and six in the information handling study.

The recordings from the experiments were analyzed using the procedure out

lined in Table 1-1. The purpose of the analysis was to reduce the raw data into a

form where the design behavior could be quantitatively analyzed.

During the experiments the verbalizations were recorded both on audio and

video tapes2. The transcription process transformed the audio and video data

into a transcript whi.:h would contain the raw data for analysis. Video data was

useful in numerous situations to resolve ambiguities and establish context dur

ing transcription. The segmentation process fragmented the raw data such that

1. The approach of deriving a hypothesis from collected data rather than confirming a pre-formulated hypothesis. 2. The video tapes had an audio track as well. The audio tapes were used as backup and for facilitating transcription activity.

Information Handling Behavior of Designers During Conceprual Design: Three Experiments


Table 1-1 Analysis procedure for transforming raw data into quantitative descriptions of the designer behavior.The analysis steps were similar for the two studies. The differences in the implementation of the steps are compared below .

Analysis Steps Information Reuse Study Information HandJing Study

conductexper.rrnen~: to record verbalizations to record verbalizations

transcribe and time stamp to get raw data to get raw data

segment transcription into questions into information fragmen~

rephrase segment rephrase question rephrase fragment

classify fragmen~ using design information fraDle- using information handling

work(D1F) framework (IHF)

quantitative analysis statistical, first -order statistical, correlation, regres-sion, state-transition, first-order

each chunk represented a unit of an interesting observation. In the information

reuse study this unit was a question and in the information handling study this

unit was an information fragment. In the next stage of the analysis these units

were classified. The framework used for classification was constructed as the

analysis progressed. It is important to point out here that the analysis was not

approached with a pre-existing framework. The framework evolved iteratively

as an outcome of the analysis procedure. The classifications were quantitatively

analyzed, the results of which are discussed later in this chapter. Each step of the

analysis procedure will be discussed in detail in Chapter 3 and Chapter 5.

1.3.1 Information Handling and Design Information Framework

Table 1-2 shows the information handling GIld the design information frame

work which resulted from analysis. This framework is a classification of the

activities designers undertake with information and a classification on design

information handled in those activities. Design information was classified using

four attributes: Descriptor, Subject-class, Medium and Level-of Detail. Four more

attributes were added to extend this framework to information handling. These

were, informational-activity, level-of-abstraction, information fragment duration

(deltat) and design information measure (dim). The attributes are briefly defined

below and the values these attributes can take are shown in Table 1-2.

Information Handling Behavior of Designers During Conccplunl Design: Three Experiment'

Chapter 1: Understanding and Improving enginesring Design Practice 11

Table 1-2 Information handling and design information framework.

Information HandIiDg Framework (IHF)

Informational Design Information Framework (DIF)

Activity Level of Abstraction Desaiptor Subject Class Medium Level of Detail

Generate Unlabeled Alternative Assembly Audio Conceptual

Access Labeled Assumption Component Video Configurational

Analyze Associative Comparison Connection Text Detail

Qualitative Construction Feature Graphic

Quantitative Location Requirement Gesture

Information Fragment Duration (sec) Operation Design-concept

Quantitative measure in seconds Performance Other

Rationale

Design Information Measure (dim) Relation

Quantitative measure: takes integer Requirement values (1,2,3, and so on) Miscellaneous

Informational-activity. This is the activity (generate, access or analyze) that the

designer performs with information.

Level-of-Abstraction. This represents the level at which the information is in it

evolution from abstract to concrete.

Design information measure. This is a quantitative measure of the amount of

semantic information in an information fragment.

Descriptor. This refers to the basic constitution or the nature of the information

in an information fragment.

Subject-Class. This is the class (assembly, component, connection etc.) to which the

subject1 of the sentence representing the information fragment belongs.

Medium. This is the physical form (text, graphic, audio, video, etc.) in which the

information fragment exists.

1. The word subject is being used in two contexts in this dissertation. One use is in the context of subjectclass and the other in the context of the experimentalSIlbjects. In most cases the context would be sufficient to impart the right meaning. In other cases the use will be explicitly stated to avoid any possible confusion.

Information Handling Behavior of De.~igners During Conceptual De.~ign: Three ExpcrimcnlS


Level of Detail. It is the phase of the design process (conceptual, configuration or

detail) to which the information fragment can be associated with.

1.3.2 Information Reuse Study: Analysis Example

A unit of interesting observation in the information reuse study was a question.

A question is any utterance by the experimental subject expressing a need for

information. Below is an example of a question from the protocol and its

analysis.

QUESTION FROM PROTOCOL:

To find in here where you figured out, or how you figured out your force requirements. basically the force requirements as they relate to the diameter of these plates.

QUESTION REPHRASED:

How does the force requirement relate to the diameter of the friction disks?

INTERPRETATION:

The designer needs DETAIL information regarding the RELATION between the FEATURES (force requirement) and the (diameter of friction disks) .

CLASSIFICATION:

Descriptor: RELATION Subject Class: Feature Level of Detail: DETAIL Medium: TEXT

1.3.3 Information Handling Study: Analysis Example

A unit of observation in the information handling study is an information frag

ment. An information fragment is a continuous period during which the

informational activity and, the descriptor of information, remains the same.

Figure 1-3 shows the template that was used for classifying each information

fragment.

An example of an information fragment and its classification is:

INFQRMA TION FRAGMENT:

I guess the idea about the strength of these lock is that the lock should be as strong as the bike ...

CLASSIFIED FRAGMENT:



(info-fragment :activity informational-activity :time (stan-time end-time) :descriptor descriptor :subjclass subject-class :subjects (subjects-list) :medium medium :levels (level-of-detail level-of-abstraction) :measures (design-information-measures:dim) )

Figure 1-3 Template for classifying information fragments in the information handling study. The figure shows the attributes that are given values to classify an information fragment. The fields take vcllues from the information handling framework (Table 1-2).

( info-fragment :activity GENERATE :time (00572007 00573300) :descriptor ASSUMPTION :subjclass FEATURE :subjects (Strength-or-Lock) : medium AUDIO :levels (CONCEPTUAL ASSOCIATIVE) : measures (1)

1.4 Evolution of the Information Handling Perspective

13

The evolution of this study is captured in Figure 1-4. This figure maps the evolu

tion of this dissertation on top of the methodology presented in Figure 1-2. Note

that this figure represents a rationalization of four years of research and while it

closely represent the temporal sequence of the work, many tasks were carried

out concurrently (Figure 1-1).

My early years of doctoral work were devoted towards a project called Genera

tion and Conservation of Design Knowledge (GCDK) [Leifer 1989]. The major

concerns of this project were to capture and document design knowledge/infor

mation so that it could be available for easy reuse at a later time. The concern for

capturing design information in an electronic medium tied this project to the

Information Handling Behavior of Designers During Conceptual Dcsign: Three Experiments


/-------( Information Classification \ \ Framework I ------_/

/-----, { Dedal Performance \ \. Results /

,,-----, { Information Handling \ \. Framework /

t t t ------------------------------Analyze Analyze Analyze

1~~'9':f~~~m /9'-'_,,\ /.~-'9~\

14

Information Support by Observe Support by Information Reuse Study Dedal ---II~~ Dedal Use Dedal - ...... ~ .. Handling

Study

Understanding of Information Handling

Figure 1-4 Evolution of the work presented in this dissertation. This covers two observational studies and development of a computational service called Deda!.

development of an Electronic Design Notebook™ (EDN'fM) . This notebook was

built on top of ~ text-graphic editor called vmacs™ [Lakin et al. 1989]1. The

hypothesis motivating this research was that design knowledge conservation

(capture, representation, reasoning and retrieval) is far less costly than re-inven

tion of comparable knowledge [Leifer 1991].

The first question faced was: Of all the information designers handle, what infor

mation do designers reuse? To find an answer to this question I conducted the

observational study on information reuse. Two experiments were conducted and

questioning behavior of the experimental subjects was analyzed. This resulted in

the design information framework (DIF) which classifies design information that

designers need during redesign2. The experiments and resulting framework are

discussed in Chapter 3.

Based on the results of the information reuse study, an information management

system called Dedal was developed3. Dedal is an information management tool

1. Electronic Design Notebook, EDN and vrnacs are trademarks of The Performance Graphics Company. 2. The experiments were carried out in collaboration with Catherine Baudin and Jody Underwood from NASA Ames Research Center and Ade Mabogunje from Stanford University.

Information Handling Behavior of ne.,igners During Conceptual Design: Three Experiment'


which integrates with an information capture medium such as an Electronic

Design Notebook to provide designers with computational services for indexing,

modeling and retrieval of design information. Dedal emphasizes managing infor

mation incrementally and in real time. It uses the framework in Table 1-2 to

implement a conceptual indexing scheme. Development and performance of

Dedal are discussed in detail in Chapter 4.

When evaluating the impact of Dedal on design practice, I learned that the activi

ties of indexing and modeling using Dedal were add-on activities, requiring

additional effort. This was a symptom of lack of adequate integration of Dedal

with the work habits of designers and the design process. I was faced with many

new questions. The critical question was: How can we support information han

dling so that it is intuitive and integrates smoothly with the design process? I

realized that there was very little understanding of the work habits of designers

relating to the dynamics of information handling, Le. information handling

behavior over time, activities performed with information, nature of information

handled in these activities, amount of information handled and others. I was con

vinced that knowledge of answers to these questions was critical to developing

services which were intuitive and integrated smoothly with design process.

In search of answers to the above questions I conducted the infonnation handling

study. In this study I conducted six experiments using the verbal protocol

method. Analysis of the experimental data resulted in the information handling

framework (llIF). To determine the information handling rates, I defined a quan

titative measure for design information using the framework in Table 1-2. This

measure is called design infonnation measure or simply dim. The experiments, anal

ysis method, dim and results of this study are discussed in Chapter 5 and

Chapter 6.

3. Dedal was built in collaboration with Catherine Baudin and lady Underwood from NASA Ames Research Center and Ade Mabogunje from Stanford University.



1.5 Key Results

Experiments involving human subjects are a challenge to monitor and control,

since results are easily influenced by variables such as human subjects, design

problem and experimenter. Verbal protocol analysis is tedious and time inten

sive, which makes it impractical to conduct and analyze a statistically significant

number of experiments. The analysis procedure is subjective and there are many

questions regarding the reliability of the results from such an analysis. Being con

scious of these shortcomings, I have built guidelines at each stage to maintain

objectivity and rigor in analysis. I have tried to explain some of the results

obtained either from prior understanding of the design process or from peculiari

ties of the experimental/ analytical method I used. I have used the remaining

results to push the edges of my understanding of the design process. Below I dis

cuss the key results from each of the observational studies and development of

Dedal.

1.5.1 Key Results from Information Reuse Study

The information reuse study resulted in some recommendations for developing

computational support for design information capture. Some key results are:

• the design information framework, Table 1-2, classifies the design information that should be captured for effective reuse.

• the framework was adequate to classify 80% of the questions encountered in the experiments

• information with descriptor performance, relation, construction and operation were encountered frequently (57%), and should get special attention during capture

• information with subject-class feature, component and requirement were encountered frequently (68%), and should get special attention during capture

1.5.2 Key Results from Dedal Development & Usage

Dedal was tested to measure its retrieval performance. Also it was deployed in a

Information Handling Behavior of Dcsigners During Conceptual nc.<ign: Three Experiment<


design project to evaluate real time usage. Here are some key results:

• retrieval performance of the conceptual indexing scheme implemented in Dedal (precision = 80%; recall = 60%) was better than keyword based information retrieval systems (precision = 20%; recall = 50%).

• information framework was adequate to capture information necessary for effective reuse

• the classification in the DIP was intuitive to designers, hence they were able to index, model and formulate queries

• indexing and modeling integrated poorly with the designer work habits, requiring designer to spend additional time indexing and modeling

1.5.3 Key Results from Information Handling Study

The information handling study gave a detailed quantitative insight into the

dynamics of information handling behavior of designers. Some key results are:

• designers spend between 2 to 35 seconds with one information type. On an average they changed the information they were handling every 13 seconds.

• the global information handling rate (dim rate) for the six experimental subjects ranged from 6.2 to 9.2 dims/minute (average being 8.1 dims/minute)

• the local information handling rate (dim rate for each of the information fragments) ranged between 2 to 40 dims/minute. About 40% of the variation in dim rate can be ascribed to the variations in verbal rate (words per minute spoken by experimental subjects)

• conceptual tasks are generation intensive. 55% of design time and 53% of design information are handled while generating new information

• more than 65% of time and design information were accounted for in descriptors [construction, alternative, operation, requirement]; in subject-class [assembly, component, design-concepts] and in non-quantitative level-of-detail

• regarding transitions, more than 82% were to and from generate for informational activity; 29% were to and from construction for descriptors; 54% were to and from [assembly, component] for subject-class and more than 90% were to and from non-quantitative level-of-abstraction

1.6 Knowledge Gained

This dissertation adds knowledge to the field of design research. It demonstrates

Information Handling Behavior of Designers During Conceplua[ Design: Three Expcrimcms


a quantitative experimental method for studying and understanding the engi

neering design activity. It illustrates a methodology for incrementally improving

design practice. It is based on observational studies that remain grounded in real

design. This dissertation makes three key contributions, it

• demonstrates a framework for characterizing the information handling behavior of designers. This framework effectively classifies 80% of the design information that should be captured during the design process for adequate reuse.

• demonstrates a computational implementation of the information framework that is intuitive for designers and works in real time.

• provides a quantitative as well as qualitative understanding of the information handling behavior of designers and makes recommendations on how this understanding can be incorporated in building services to improve design practice.

1.7 Thesis Organization

Chapter 2 surveys the literature in the field of design research. It looks at the

methods used in studying and understanding the design process. It discusses

the appropriateness of the verbal protocol approach as a design research

method. It also cites studies that have impacted the evolution of this thesis.

Chapter 3 describes the information reuse study. It describes the analysis proce

dure and the evolution of the design information framework (DIF).

Chapter 4 describes the computational service Dedal. It discusses how Dedal

was used in a design project and what lessons were learned from its deployment

and usage.

Chapter 5 describes the information handling behavior study. It describes the

analysis procedure and the development of the information handling framework

(lliF).

Chapter 6 describes the quantitative and qualitative results of the information

Information H:mdling Behavior of Designers During Conceptual Design: Three Experiments

Chapter 1: Understanding and Improving engineering Design Practice 19

handling study. It discusses the understanding gained and suggests recommen

dations for the design and implementation of information support services.

Chapter 7 discusses the conclusions of this study. It describes the key contribu

tions of this work and some directions for further research.

The bibliography provides a list of research papers, articles and books that have

influenced the evolution of this dissertation.

Appendix A is a collection of supporting material from the information reuse

study. It includes the design problem used in the experiment, samples of raw

data, examples of analysis and some results.

Appendix B is a collection of supporting material on Dedal implementation and

usage.

Appendix C is a collection of supporting material from the information handling

study. It includes the design problem used in the experiments, samples of raw

data and examples of analysis.

Appendix 0 is a collection of additional results from the information handling

study which are not included in the main body of the thesis.

Appendix E lists the key papers published in the past 4 years. These papers

reflect the evolution of this dissertation.


Chapter 2

Design Theory & Methodology Research: Literature Review

Design Research I..J/"""-~~-'~> Deliver High Quality, \. Goal ~,~--~," Low Cost Products

~ jl

TO TO

" \ shortest path

Develop Research ~ Improve Design Methods ______ Practice

~ Understand Design ~o Practice

This chapter reviews the literature in the field of design theory and methodology that has influenced the work reported in this dissertation. Prescriptive and descriptive (empirical) design research studies from the past few decades are discussed. Appropriateness of the verbal protocol method is discussed along with a review of some studies which have used this method. A few studies using non-verbal protocol method are also discussed. The chapter ends with a discussion of research that focuses on information handling issues in the design process


Chapter 2: Design Theory & Methodology Research: Uterature Review 21

2.1 Design Theory and Methodology Research

The objecjve of much of the research carried out in the field of design theory

and methodology is to develop a better understanding of the engineering design

process. This understanding can be used to develop new methods and services

to improve design practice in industry and design education at academic institu

tions. The complexity! of the design process offers numerous challenges in

accomplishing these objectives. There are many directions possible for conduct

ing design research, as is evident from the number of academic disciplines

involved in it, such as engineering, anthropology, sociology, management and

psychology, among others. As a result, design research is published in a large

variety of literature sources. In this chapter I will discuss the literature which has

impacted the evolution of the work reported in this dissertation.

Looking at design research from the last few decades, the studies that have been

conducted can be grouped under two approaches. They are either prescriptive

or descriptive [Tang 1989, page 25]. Prescriptive studies typically result in recom

mending a sequential procedure for carrying out design, whereas descriptive

studies aim to provide a description of how design is carried out. In the sections

below I will discuss studies associated with both of these approaches.

2.1.1 Prescriptive Studies

Prescriptive design studies result in laying out a method for doing design. These

methods are typically developed by reflecting and theorizing about the design

process. They suggest a systematic and algorithmic procedure that should be car

ried out sequentially. A sequential process ensures that decisions are made after

all necessary information has been generated, accessed and considered, so that

no important elements of the design problem are overlooked. These methods

1. The apparent complexity of the design process stems from the fact that it is not just a technical process or a problem solving process. It is a complex and poorly understood mix of technical, problem-solving, social, organizational, behavioral and creative processes, just to name a few.



emphasize the need for analytical work before generating new design concepts

and moving on to the next stage. Most prescriptive models allow for iterations

and feedback to take place between different stages.

Asimov [1962] prescribed a seven phase procedure to transform needs into phys

ical artifacts. These phases are:

1. feasibility study

2. preliminary design

3. detailed design

4. planning for production

5. planning for distribution

6. planning for consumption

7. planning for retirement

The first three phases form the design stage and the last four phases form the

production and marketing stages. Each of these phases involves problem solving

situations which include the stages of analysis, synthesis, evaluation, decision, opti

mization, revision and implementation. This problem solving pattern carried out

over the seven phases provides a methodology for transforming needs into phys

ical artifacts.

Jones [1963] prescribed a three stage design method. This method was a system

atic move from analysis to synthesis to evaluation. These stages are: [Cross, 1990,

page 24]:

Analysis: Listing of all design requirements and the reduction of these to

a complete set of logically related performance specifications.

Synthesis: Finding possible solutions for each individual performance

specification and building up complete design from these with least pos

sible compromise.

Evaluation: Evaluating the accuracy with which alternative designs fulfil

performance requirements for operation, manufacture and sales before

the final design is selected.


Chapter 2: Design Theory & Methodology Research: Literature Review 23

The emphasis in this procedure is on generating numerous alternatives and

choosing the best alternative based on extensive analysis and comparisons.

Alexander [19641 proposed a method of simplifying and structuring design prob

lems by listing all the requirements of the problems and analyzing the

interactions between them. These requirements could then be grouped into sets,

which lead to mostly independent sub-problems, which were easy to solve. Solv

ing these sub-problems and integrating their solution would lead to a solution

for the bigger problem. His method is usually seen as an iterative analysis - syn

thesis model of design. Analysis is the stage of decomposing the problem into

mostly independent smaller problems and synthesis is the stage of solving these

smaller problems and integrating the resulting solutions.

Archer [1963-641 prescribed a three phase design process based on the following

six activities:

1. Programming: establish crucial issues, propose a course of action.

2. Data Collection: collect, classify and store data.

3. Analysis: identify sub-problems; prepare performance specifications; reap-praise proposed program and estimate.

4. Synthesis: prepare outline design proposals.

5. Development: develop prototype designs; conduct validation studies

6. Communication: prepare manufacturing documentation.

The three phases based on the above activities are: analytical (activities of pro

gramming and data collection), creative (activities of analysis and synthesis) and

executive (activities of development and communication). He suggests:

"One of the special features of the process of designing is that the

analytical phase with which it begins requires objective observation and

inductive reasoning, while the creative phase at the heart of it requires

involvement, subjective judgement and deductive reasoning. Once the

crucial decisions are made, the design process continues with the execu

tion of working drawings, schedules etc., again in an objective and

Information Handling Behavior of Designers During Concepturu Design: Three Experiments

Chapter 2: Design Theory & Methodology Research: Uterature Review

descriptive mood. The design process is thus a creative sandwich. The

bread of objective and systematic analysis may be thick or thin, but the

creative act is always there in the middle. [Archer, 1964]

24

The most recent prescriptive design model is by Pahl and Beitz [1977 and 1984].

This model presents design process as iteratively going through four stages.

These stages are: clarification of task, conceptual design, embodiment design and detail

design. Each stage is accomplished by the following seven step procedure:

1. Clarify and define the task: Specifications

2. Determine functions and their structures: Function structure

3. Search for solution principles and their combinations: Principal structure

4. Divide into realizable modules: Module structure

5. Develop layouts of key modules: Preliminary layouts

6. Complete overall layout Definitive layouts

7. Prepare production and operating instructions: Product documents

This mode1lays out a systematic procedure for developing a good and complete

understanding of the problem, breaking it up into smaller problems, solving

these smaller problems and combining the solutions to obtain the overall solu

tion. This model has wide acceptance in the European design community.

It is not uncommon to see criticism for prescriptive methods. They are seen as

focusing on the problem rather than on the solution and therefore not reflecting

designer's manner of thinking. They also structure the design process into a set

of compartments with well defined boundaries. This is very contrary to how

most designers work and what many researchers have observed as will be evi

dent from the descriptive studies below.

2.1.2 Descriptive (Empirical) Studies

Unlike prescriptive studies which layout a sequential procedure, descriptive

studies aim to provide a description and understanding of what goes on during

the design process as a basis for improving design practice. These are typically

Information Handling Behllvior of Designers During Conceptual Design: Three ExperimcnL'


based on formal observations and an in-depth analysis of design activity. Much

of the recent design research falls under this category. Methods have been devel

oped to conduct experiments to collect raw data. The data is analyzed2 to gain

an understanding of certain design behavior. This understanding is subse

quently used to develop services to improve design practice. Waldron and

Waldron [1996] have presented a survey of the various methods that are in use

today to study design. Definitions, advantages and disadvantages of these meth

ods are summarized in Table 2-1.

Table 2-1 Empirical design research methods. The table lists brief definitions of some of the popular empirical methods and the design scenarios for which they are appropriate. Advantages and disadvantages for each of the methods is also discussed.

Appropriate Empiric:aI Method Scenario Advantages Disadvantages

Deposition: timely inter- single - targeted .:lata capture - interferes with design pro-views during the process designer - easier analysis cess

(can also be used with ver- - can study long tasks - incomplete data capture

bal protocol) - data is chronological

Verbal Protocol: designer single - access to designers thoughts - difficult and time consuming talks out loud during designer - comprehensive data capture analysis procedure

design to externalize think- - data is chronological - can only study short tasks

ing process - verbalizing may interfere with design process

Discussion protocol: natu- small team - no interference with design - difficult and time consuming raI communication among process analysis

team members - can study communication. - incomplete data capture negotiation and such - can only study short tasks

Retrospection: interviews single or - no interference with design - data collected is not chrono-or questionnaire after the small team process logical or real-time

design process - targeted data capture - partial data capture - easier analysis

Participant observer: small or - can study very long tasks - difficult and time consuming observer is also a member large team - no interference with design analysis

of the design team process - incomplete data capture - data is chronological

The studies reported in this dissertation in Chapter 3 and Chapter 6 were con

ducted using the verbal protocol method. Section 2.1.3 is devoted to describing

and discussing the appropriateness of the verbal protocol method. In subsequent

2. An analysis procedure is usually a by-product of these research techniques.

Information HllOdling Behavior of Designers During Conceptual Design: Three ExperimenlS


sections studies conducted using verbal protocol method and some studies con

ducted using other methods will be discussed.

2.1.3 Verbal (or Think-Aloud) Protocol Method

The verbal protocol method was developed in early sixties to study human prob

lem solving behavior [Newell and Simon 19721. In this method the experimental

subjects are requested to verbalize their thinking (think-aloud) as they work on

the task given to them. The verbalizations are recorded on audio and video

medium.3 These provides the raw data for analysis. This method was and is

most commonly used in cognitive science research. However in the last decade

this method has gained noticeable popularity within the design research commu

nity as is evident from the number of researchers, publications and workshops

focusing on this method [Stauffer, 1987; Ullman et al., 1988; Guindon,1990; Chris

tiaans et al., 1993; Cross et al., 19961.

The appropriateness of the verbal protocol method has been debated extensively

for and against by Ericsson and Simon [19931 and Nisbett and Wilson [1977]

respectively. Crutches [1994] emphasizes the value of the verbal protocol

method as a means for making data explicit for capture. He reports:

"Beyond the increased acceptance of verbal reports as behavioral data,

the unique advantages of verbal report data are increasingly apparent to

many people. In particular, researchers have emphasized the advantage

of verbal reports as protocol data, providing a sequence of observations

over time rather than just a single observation at the end of the process .

......... In addition, verbal protocols provide many more observations of a

phenomenon over a given time period than other methodologies, increas-

ing the information yield of studies .......... Verbal reports can provide

3. In its early usage, the verbal protocol was usually recorded on audio medium. Now days, because of advancements in video technologies, the protocol is also recorded on video medium. For studying the design process, video data is an essential ingredient for analysis since many actions which are non-verbal, and therefore cannot be recorded on an audio medium, get recorded on video medium resulting in a more complete capture of data.

Information HllIIdling Behavior of Designers Ouring Conceptual Design: Three Experiments


information difficult to obtain by other means- in particular, information

about types of knowledge accessed in task processing ..... " [Crutcher,

1994, In Psychological Science, page 242]

The important issues concerning verbal report data are:

27

• Validity: Does the information in the verbal report reflect the actual thinking process?

• Interference: Does the act of verbalizing alter the normal thinking process?

• Completeness: Does the method capture all relevant data?

From a cognitive viewpoint Ericsson and Simon [1993] have argued that:

"subjects can generate verbalizations, subordinate to task-driven cogni

tive processes (think aloud), without changing the sequence of their

thoughts, and slowing down only moderately due to the additional ver

balization." [Ericsson and Simon, 1993, page xxxii].

This means that as long as thinking is assumed to be a sequential process4, the

verbalizations are a valid reflection of the natural thought process. From the

viewpoint of information handling activity during a design process, the evolu

tion of the design is reflected in the sequence of actions performed by the

designers. Verbal protocol reports (captured on audio as well as video medium)

will provide an accurate capture of these sequences.

As stated by Ericsson and Simon above, verbalizing does cause the thought pro

cess to slow down moderately. However, the interference to the natural thought

process can be minimized by providing adequate instructions to the experimen

tal subject. Payne [1994] suggests that while it is not necessary for subjects to be

extensively traineds, they should not be requested to verbalize a particular type

of information. He reports:

" ...... Extensive training in verbalization is seldom needed or really

4. Verbal protocol analysis assumes that cOgnition, specially for tasks such as problem solving, is serial. 5. It is a common experimental procedure to train the subjects on a short trial problem to get them comfortable with thinking out loud.


Chapter 2: Design Theory & Methodology Research: Literature Review

desired ... The subject must remain focused on the task, with verbaliza

tion clearly subordinate .... Researchers should also avoid temptation of

trying to get the subject to give a coherent sequence of verbal reports by,

for instance, asking for verbalizations of only selected information (e.g.

reasons for behavior). Such efforts frequently compromise concurrent

verbalizations. The subject should be asked to verbalize all thoughts that

occur to him or her during the performance of the task without emphasis

on any particular type of information." [payne, 1994, in Psychological Sci

ence, page 246]

28

From the viewpoint of completeness, the verbal protocol method has a draw

back. Many representations and information that designers deal with during

design is non-verbal. Much of this information is not captured in the verbal por

tion of the protocol data. However video recordings can be used, as is common

now days, to capture some of the non-verbal information.

From the accounts above, we can conclude that verbal protocol method is appro

priate for collecting data for single designer working on short duration design

tasks. Experimental subjects should verbalize all of their thinking and should

not be requested to verbalize any specific kind of information. This is specially

critical if the focus is on observing the information related behavior as was the

case in the studies presented in this dissertation.

2.1.4 Studies Using Verbal Protocol Method

Akin [1978, 1979] used the verbal protocol method to study the architectural

design process. He was interested in understanding and analyzing the informa

tion processing activities of the designers and finding the relationships between

the design problem and the representations used. He observed that designers

shifted attention among different issues frequently, and that there was a relation

between the representations used and the issue being attended to. From his anal

ysis he concluded that designers exhibit eight recurring behavior patterns which

he called schemata. These schemata are:



1. instantiation: creating a new item or symbol

2. generalization: grouping related symbols together into a common category

3. inquiry: seeking new information, internally or externally

4. inference: generating new information based on internal knowledge

5. representation: making a visual image of an item

6. goal-definition: defining goals to pursue solutions

7. specification: specifying a partial solutions

8. integration: combining partial solutions

He also observed that design is a top-down, breadth-first process and that

designers seek solutions that adequately satisfy the requirements rather than

develop optimal solutions.

Ullman [1988] and Stauffer [1987, 1991] used protocol analysis method to study

five mechanical designers doing non-routine tasks. Based on their analysis, they

propose an information processing system model which describes the problem

solving process of a mechanical designer. It is called task/episode accumulation

model (TEA model). They state:

" ........ The key features of the model are (a) the design is constructed by

incrementally refining and patching the initial conceptual design, (b)

design alternatives are not considered outside the boundaries of design

episodes (which are short stretches of problem solving aimed at specific

goals), (c) the design process is controlled locally, primarily at the level of

individual episodes."[U11man et al. 1988 page 33]

The TEA model contains ten operators which are identified as the finest informa

tion operator. The design is accumulated gradually by the application of

operators in a meaningful sequences called episodes. Hence the name task/

episode accumulation model. The episodes are on an average 56 seconds long.

Guindon [1990] studied early stages of the software design process using verbal

protocol method. She analysed protocols from three experienced designers and

observed that, designers deviate from a strictly top-down approach in early

Information Handling Behavior of Designers During ConceplUal Design: Three Experiments


stages of the design and display an opportunistic behavior. She states:

" ........ Designers were observed interleaving decisions at various levels of abstraction in the solution decomposition....... The sudden discovery of new requirements and partial solutions triggered by data-driven rules and associations, the immediate developments of solutions for newly discovered requirements , and drifting through partial solutions are important causes of opportunistic design. II

30

Using the results of her study she suggests implications for training, methods

and computational environments to support early stages of the design process.

The interest in using protocol analysis to study design process has been grow

ing. However, much of the effort is isolated among many research groups. It is

therefore difficult to generalize, validate or compare results across researchers

and therefore incrementally grow the knowledge about the design process. To

address these concerns a protocol analysis workshop was organized at Delft,

The Netherlands in September 1994. All participants of the workshop were

requested to analyze data resulting from the same experiments so that analysis

procedure and conclusions could be compared. Cross et al. have summarized

the results of this workshop in a book [1996]. They concluded that use of proto

col method for studying the engineering design process is in its very early

stages. They state:

fl............ Protocol analysis is clearly not a universal cure for design research problems. Nevertheless, we feel that we can say that protocol analysis as a research technique for design has been 'validated' with some qualifications .....

The qualifications were concerned with the completeness of the data collected

and the process of carrying out the analysis. They caution that, (a) verbal proto

col method does not capture the non-verbal thought process going on in design

work and therefore should not be treated as capturing complete data, and (b) the

bias of experimental setup, researcher and experimental subject on the analysis

Information Handling Behavior of Designers Ouring Conceptual Design: Three ExperimenlS


procedure should be minimized.

2.1.5 Studies Using Other (Non-Verbal Protocol) Methods

Waldron and Waldron [1988] conducted a study using the retrospective method.

They studied a large design project on the design of an experimental walking

machine involving several designers over a few years. They constructed the

design process sequence retrospectively after the design process was complete.

This was based on interviews with the designers and records from different

stages of the design project. They found that

• experience of the design team members had a significant influence on the development of concepts.

• competing design concepts were pursued in parallel

• important decisions were based mainly on qualitative information and simpie models

• the separations between conceptual, layout and detailed design phases was not distinct, rather all phases were encountered all along

• the design team members changed roles among different conceptual, layout and detailed design throughout the design project

Tang [1989] used the discussion protocol method to study the use of shared

workspaces by a group doing conceptual design. Using techniques borrowed

from video interaction analysis he studied 8 groups of 3 or 4 designers work on

short term design tasks. He observed that the actions of the designers could be

grouped under the categories of list, draw and gesture. These actions were used

to accomplish functions of storing information, expressing ideas and mediating inter

action. He observed that a large portion of the actions are gestures which are not

adequately handled by the existing computational support tools. He recom

mends that tools to support workspace activity of teams working on conceptual

design tasks should consider:

• providing ways of conveying and supporting gestural information

• minimize the overhead encountered in storing information



- allowing intermixing of workspace actions and functions

-enabling all participants to share a common view of the workspace while providing simultaneous access and a sense of close proximity to it

- facilitating the participants' ability to coordinate their collaboration

[Tang 1989, page 101]

Minneman [1991] used discussion protocol method to study group engineering

design practice. He collected data from an industrial team working on design of

a photocopier sub-system as well as conducted short experiments to simulate

design communication between different divisions of a design firm (engineer

ing, manufacturing and marketing). He expressed his understanding in terms of

a framework and designers practices. These are:

1. Framework: a perspective for thinking about the content of design work., has two dimensions- facets (the things being worked on) and trajectories (a temporal sense to the work being done), and

2. Practices: actions for getting design work done. Four practices were observed to be crucial. These are negotiating understanding, conserving ambiguity, tailoring engineering communication for recipients, and manipulating mundane representations.

The framework highlights that participants have a personal views of the ongoing

activity and at any given time, design activity is doing communicative work on

multiple projects. The practices signify that, designers' activity is not something

shaped by externally imposed context, but rather as attending to and creating a

recognizable order in the ongoing social interaction. He argues that there is no

innate structure to designing, the structure in the activity is a product of the

designers taking a series of local actions to produce a result.

2.2 Design Process: An Information Handling Perspective

In the previous sections we looked at many studies which have attempted to

understand either the individual or the team design process. These studies

examined the process from a variety of perspectives such as: problem solving,

information processing, workspace activity and collaboration. The studies in this



section examine the design process from an information handling perspective.

They are aimed at understanding the information needs and information man

agement burden on designers.

Hales [1987] performed a detailed analysis of all the activities from one project

involving the design of a high-pressure, high-temperature system for testing

materials. He was a participant observer in this project over a period of 2.8 years

which involved 37 different participants. He found that only 47% of the total

observed effort was spent in doing design. The remaining 53% was spent in

information retrieval, planning, cost estimating, reviews, social contact and help

ing others. He observed that structured or prescriptive methods accounted for

less than 25% of the design effort. Also, over 50% of observed effort involved

either working alone or in pairs and about 30% in meetings of 2, 3 or 4 people.

This emphasizes that large portion of the design happens by individuals or

small teams. Based on a quantitative analysis he found that design process can

be characterized by a set of overlapping phases, each consisting of a particular

mix of procedural steps and other general activities.

Kuffner et al. [1991] conducted a verbal protocol study over six experiments to

determine the information requests of mechanical design engineers doing rede

sign. He developed a taxonomy of design information, based on the questions

and conjectures raised by the designers. Design information was categorized by

its nature (construction, location, operation, purpose) and topic (assembly, component,

interface, feature). These nature and topic are identified as the information that

should be captured, structured and made available in CAD environments for

facilitating reuse.

Court et al. [1993] conducted a questionnaire based survey of over 200 engineers

and designers from various industries (power generation, automotive, construc

tion, manufacturing, agriculture and utilities) to determine their information

requirements. This comprehensive survey covered industries of all sizes (small,


Chapter 2: Design Theory & Methodology Research: Literature Review 34

medium, large) and targeted designers in variety of roles (management, creative

design, routine design, etc.). They found that only about 43% of designers time is

spent doing design. The rest of the time is spent in meetings (16%), searching for

information (18%) and paperwork (23%). They also found that 40% of the

respondents did not use any formal operating procedures or methods. Their

study underscores the fact that designers spend a large portions of their time

handling and managing information, and that there is a lack of adequate compu

tational tools to alleviate the information management burden.

The above studies provide some understanding of information related behavior

of designers. More so, they bring out the need for developing a richer under

standing of the information handling behavior so that it can be supported by

novel services and methods. The following chapters will describe the studies

that were conducted using the verbal protocol method to develop an under

standing of the information handling behavior of designers as a basis for

developing requirements for information management frameworks and services.


Chapter 3

Design Information Needs, Capture & Reuse

/-------( Information Classification \ \ Framework J -------_/

/-----, ( Dedal Performance \ \. Results I

/-----, ( Information Handling \ \. Framework I

___ 1 __________ l __________ l __ _ Analyze Analyze Analyze

Is Development ;. ~eVise /. \ of Dedal Dedal

19901991 1991-1992 1992-1995

Information Support by Observe Support by ___ ~~ Information Reuse Study Dedal ---~~ Dedal Use Dedal Handling

Study


This chapter describes the experimental procedure, analysis method and results of an observational study conducted to understand the information needs of designers during the design process. Questions asked by the designers while engaged in design activity are captured to comprehend their information needs. A classification for these questions is developed. This classification furnishes the framework for design information that should be captured and structured during the design process to enable effective reuse in the future.

Information Handling Behavior of Designers During Conceprual Design: Three Experiments 35

Chapter 3: Design Information Needs, Capture & Reuse 36

3.1 Design Information

Engineering design is an information intensive process. All along the design pro

cess, designers need information to complete their tasks. They fulfill these

information needs by retrieving information from a variety of sources such as:

personal information databases, colleagues, handbooks, technical reports, ven

dors, suppliers and in some cases by library/information retrieval services.

From a survey of more than 200 industry designers from various industries

Court et. al. [19931 found that designers spend in excess of 18% of their time

searching for information. In a participant observer! study [Hales 1987] found

that 53% of the design effort was spent on non-design related activities. In his

study Hales considered information retrieval a non-design related activity.

These studies corroborate the understanding that timely availability of appropri

ate information in a usable format will significantly improve design practice.

3.1.1 Information Reuse Essential for Good Design Practice

It is well known that much of the design activity taking place in industry is rede

sign. A new design is either an improved version of an older design, or an

adaptation of a design from a different domain, or a novel combination of fea

tures from many other designs. In all these scenarios knowledge and experience

from previous designs is used in designing a new product. Figure 3-1 shows

how the process is cost and time effective if designers reuse design information

from past designs rather than re-engineering it. But, a prerequisite to effective

reuse is capturing information in reusable form. The design process does not nec

essarily lend itself naturally to capturing information in a reusable form. Allen

[1977] reported that, during the design process designers use information to pro

duce physical changes in the world. They consume and transform information to

produce a product which is information bearing, but this information is no

1. A study where the observer is a member of the design team being studied.


Chapter 3: Design Information Needs, Capture & Reuse

- past experience .- - reusable info -- technical documents Information Capture - structured info -colleagues

allows Reuse r= j~ and is , Cost and Time Effective

~'7

Old '- Satisfy '- -"'-

New Design Information Design Process Design

Information -/ Needs / 7 Information

o.

~ No Information Capture ..) requires Re-Engineering t

which is

- physical artifacts Expensive and Time Consuming - non-verbal info - - non-reusable info - CAD drawings -- past designs

Figure 3-1 An information perspective on the design process. The figure shows that old design information can be used in satisfying information needs during a design process resulting in new design information. Reusing old information will be cost and time effective compared to re-inventing it. Current design practices have less reuse (shaded region) and more re-engineering (unshaded region). Reuse can be increased by capturing information in reusable form.

37

longer in verbal form. Information in the form of physical or even software prod

uct artifact is difficult to reuse, since it does not capture decisions and the

rationale behind those decisions. Therefore, capturing information in a reusable

form is necessary to avoid expensive and time intensive re-engmeering.

3.1.2 Design Information Capture is Difficult

A good methodology for capturing design information is a prerequisite for effec

tive reuse. The capabilities of current design tools in capturing information is far

less than what is needed to realize the benefits of reuse. Information capture ser

vices available today integrate poorly with the design process, burdening

designers with additional tasks. Consequently these services are under-utilized

or not used. This is specially true for the conceptual design phase. Design infor-



mation capture is difficult because:

• design information is often informal, fragmented or incomplete and therefore difficult to represent in a computational medium

• the design process most often results in non-verbal information bearing product

• design information exists in many media

• effort is required to structure information for reuse

• it is not practical or wise to capture all information, therefore capture must be selective

• capture services cannot keep pace with the design process, i.e. rate of information capture is less than the rate of information generation (by a wide margin).

There is a need to develop innovative information capture services which inte

grate smoothly with the design process and capture information in a form

suitable for reuse. We can build such services if we have a better understanding

of the information needs of a designer. This chapter will focus on developing

this understanding.

3.1.3 Information Reuse Study Objectives

The objective of the information reuse study was to develop an understanding of

designer information needs and develop a framework and guidelines for devel

oping information capture and reuse services. Understanding provides

knowledge of the information that should be captured during design. It will also

inform the development of guidelines for structuring this information for reuse.

Kuffner [1991] conducted a study to analyze the information requests of mechan

ical engineers. He came up with a taxonomy which characterized information

based on its nature, topic and category. There are some similarities in the objec

tives and the approaches of his study and this study. These studies will be

compared in the discussion at the end of this chapter.



3.1.4 Questions Reflect Information Needs

It is natural for humans to express a need for information in the form of a ques

tion. For instance, if you needed to find the torque developed by a motor

manufactured by a vendor, you will most likely express your need in the form of

a question such as "How much torque does this motor develop when hooked up

to a car battery?" An answer to this question would fulfill your information

need. If we were to capture the questions that designers ask while engaged in

design, we would have a view into their information needs. The information

reuse study was designed to capture the questions that designers ask while

engaged in design.

3.2 Experimental Study: Verbal Protocol Method

I designed an observational study using the verbal protocol method to capture

the questions designers have when engaged in design activity. Experimental sub

jects were asked to redesign an existing product to a new set of requirements. I

chose the experimental method since it allowed me to:

• look at the design process from a reuse viewpoint

• deal with the complexity of the process by exercising adequate control

• simulate real design activity

• study design activity at a fine level of granularity

• make designers actions explicit for observation

I used the redesign scenario in the experiments since, redesign is a very common

scenario in industry and information reuse is prevalent during redesign.

3.2.1 Experimental Setup and Procedure

Figure 3-2 shows the experimental setup used to collect data. The subject

worked in a room fitted with two video cameras. One camera captured a wide

angle view of the workspace and the other captured a close-up of the desk. The



data was independently recorded on the two cameras and an audio recorder.

~ fi Workspace ... a Documents

./ / " / "

Figure 3-2 Experimental setup for information reuse study. Two video cameras capture activity in the workspace (camera 2) and the wide angle view of the subjects work area (camera 1). The subject has access to documentation from the first generation design, and to a member of the first generation design team. (expert). The expert sits in a separate room and is available on request.

40

At the start of an experiment the subject was given a small problem, to get him

comfortable with talking out loud and the experimental setup. After this he was

given the experiment's redesign problem. The subject was asked to complete the

redesign task in about six hours. The experimenter was out of the room monitor

ing the subject on video. If the subject was silent for an extended duration (15-20

seconds), the experimenter would remind the subject to talk out loud. The exper

iment was conducted over two sessions with an hour long break in between. The

subject was asked to summarize the status of his work before and after the break

to capture any thoughts that may have occurred during the break.

Subjects had access to design notebooks and technical reports from the first gen-



eration design. These contained detailed description of the first generation

functional prototype and its performance. One member of the original design

team, to be referred to as the expert, was also available to answer questions and

serve as an index to information in the documentation which totaled more than

a thousand pages. The subject using the expert as an index into the documenta

tion provided a natural way of making the questions explicit without being

obtrusive to the subject. The expert stayed outside the room but could be called

in by the subject any time. The expert stayed out of the room, so as to be unaware

of the information that the subject was encountering. This setup assured that the

expert answered any questions objectively without volunteering additional infor

mation. The instructions and the redesign problem used in the experiment are

attached in Appendix A.

3.2.1.1 The Design Problem: Continuously Variable Damper

The problem chosen was the redesign of a continuously variable car damper

(popularly known as shock absorber) to meet a new and revised set of require

ments. The design was complex enough that the subjects needed to develop a

good understanding of the first generation design before coming to a solution

and simple enough that a conceptual solution could be reached in about six

hours. Such a trade-off was essential to conduct an experiment in a laboratory.

The first generation design was done in a real world academic setting by a three

member design team. This project was done for a leading USA automobile man

ufacturer over a period of seven months [Baya 1990]. The design was carried out

from concept to laboratory testing of a working prototype. Three documents

spaced equally over that period were produced by the design team. Each docu

ment summarized the status of design at the time it was written. These

documents captured the evolution of the first generation design from early ide

ation stage to final machine drawings and test results of a functional prototype.

Information Handling Behavior of Designers During Conceptual Desigo: Three Experiments


3.3 Analysis of Verbal Data

The analysis required a procedure for transforming verbal data into a frame

work which reflects design information needs. This procedure is shown in

Figure 3-3. It involves six steps. The form of the data at input and output of each

Q

Design Problem Conduct Recorded Data Written Data

Experiment Transcribe

uestions with Context Reformulate Questions Extract

Questions Questions

Quantitative

Classify Classification Statistical Results

r Questions Analysis

Figure 3-3 Analysis procedure for information reuse study. The steps in the analysis reduce recorded data into quantitative description of the designer information needs.

of the steps is indicated. Altogether these steps reduce recorded data into a quan

titative description of the information activity of designer in this scenario. Each

of the steps is described below. The examples for each of the step are discussed

in section 3.5.

3.3.1 Transcription

The process of transcription yields written protocol data. The transcript captures

the words spoken by the subject as well as information about pauses, intonation,

focus of attention and context. Video tapes are used in this step to add action

and gesture data to the record. The resulting transcript has the details necessary

for carrying out the rest of the analysis. However there were occasions when

raw data was used in later stages of analysis to resolve ambiguities.

Information Handling Behavior of Designers During Conceptual Design: Three ExperimcnlS


3.3.2 Question Extraction

The written data resulting from transcription is used for extracting questions.

For the purpose of this study a question is an interrogative expression uttered by

the subject to seek information (e.g .. what is the diameter or X?) , or an utterance

which suggests a need for information, although it may not have been explicitly

asked (e.g .. I should find the diameter of X). In ambiguous cases, the subject was

consulted (after the fact) to determine if an utterance was a question or not.

Some key scenarios and rules in extracting questions were:

• explicit ask to expert e.g .. "How heavy is this device?"

• searching of documentation by self e.g .. "Where are the test results?"

• utterance to self e.g .. "I will need to find the thickness of this part."

• conjecture, e.g .. "This must be 12 inches."

• a question uttered twice within the same activity focus! was counted once.

• a question repeated in an unrelated context or in a different activity focus was counted again.

• a complex question was broken into a set of simple questions.

3.3.3 Question Reformulation

Questions as spoken by the subject in the transcript are often dependent on

nearby information to be intelligible. i.e. they have meaning within the context

of their location in the transcript. Reformulation was the required to fill in con

textual and missing information so that question classification could be done

objectively without consulting the transcript.

3.3.4 Question Classification

Before questions could be classified, there was a need to develop a framework

for doing so. Figure 3-4 shows some of the important steps involved in the devel

opment of the question classification framework. This framework is described in

1. An activity focus was defined by closeness in context, time or design goal of the designer.

Information Handling Behavior of Designers During Conccprual Ocsign: Three ExpcrimcnlS


Questions

+ step 1 Characterize ~

"1 \" ____

//" No / \"

/ / \ " / " ", " step2 ~ step 6

Collection of Question Characteristics Can all questions be classified?

~"'/~ Group into Categories

+ step 4 Label Categories --•• ~

~ Classify New

Questions

Yes

Rgure 3-4 Steps in the development of question classification framework. First step is divergent and generates a collections of terms to characterize questions. This collection is grouped, based on similarity in characteristics or meaning, into categories. These categories were labeled. If this set of categories were suitable to classify any question encountered in the data then it was the classification framework.

44

detail in section 3.4. Question classification used this framework on the reformu

lated questions. The classification was based on interpreting the questions with

respect to a template as a method of maintaining rigor and objectivity in classifi

cation. The template is shown in Figure 3-5.

3.4 Question Classification Framework

Table 3-1 shows the framework for classifying questions. In this framework a

question is characterized by four attributes namely DESCRIPTOR, SUBJECI'-cr.ASS,

MEDIUM and LEVEL-oF-DETAIL. It is classified by being given a label from each of

the four attributes. The values for each of the attributes are shown in Table 3-l.

They are not a list of all possible values. They represent what is observed in the

experimental data and suggest values significant for describing design informa

tionneeds ..



Table 3-1 Framework to classify questions. The same framework can be used to classify design information. This will also be called the Design Information Framework (DIF).

Descriptor subject.c:Jass Medium Level of Detail

Alternative Assembly Audio Conceptual

Construction Component Video Configurational

Location Connection Text Detail

Operation Feature Graphic

Performance Requirement Gesture

Rationale Design-concept

Relation Other

Requirement

Miscellaneous

3.4.1 Design Information Framework (DIF)

The framework that is used to classify design information is the same as the one

used to classify questions. This is because any segment of design information

can be considered to be an answer to an implicit question. Thus it is possible to

interpret design information segments in a template identical to that of a ques

tion as shown in Figure 3-5.

3.4.2 Descriptor

The descriptor of a question refers to the inherent character, the basic constitu

tion or the nature of design information that is sought by the question. The

descriptors are independent of the design problem or the domain of design.

Table 3-2 shows the descriptors, their definitions and examples.

3.4.3 Subject-Class

The subject-class of a question refers to the class to which the subject of the ques

tion belongs. The subject of a question is the subject of the sentence or the clause

making up the question. Like descriptors the subject-classes are independent of

the design domain. The subjects of the question are dependent on the design



I

Question Interpretation Template:

the subject is requesting, O£Oll) information (LEVEL-QF-DETAIL)

regarding the 0'Il£S of the question-subject which (DESCRIPTOR)

belongs to OSC£ (SUBJEcr-CLASS)

r---------------------------, Design Information Interpretation Template:

the design information segment is l£Oll) (LEVEL-OF-DETAIL)

l'lYES of the information-(DESCRIPTOR)

information regarding the

subject which belongs to --r=~l=:SC~'£=--=-=. (SUBJEcr-CLASS)

L ___________________________ ~

Questionlinformatjon classification based on above interpretation

Question Descriptor: Q!JYES Question Subject-Class: QSC£ Question Level-of-Detail: Q£OIl)

Information Descriptor: l'lYES Information Subject-Class: lSC£ Information Level-of-Detail: l.£OlD

,,-----------------------------------------------------------------_. Figure 3-5 Template for interpreting question and information segment. Question-subject and information-subject in the above templates refers to the subject of the sentence forming the question or the segment of design information respectively.

46

domain. Table 3-3 shows the different subject-classes, their definitions and

examples.

3.4.4 Medium

The medium of a question refers to the form of the information which would

adequately answer the question. While this is an important category for classify

ing design information, from a computational viewpoint, the verbal protocol



Table 3-2 Question descriptors refer to the basic constitution of the information sought by the question. It can take the values defined below in the deSCriptor column.

Descriptor Question seeks information regarding Example

Alternative choices or decisions What are the other ways of changing linear motion into rotaxy motion?

Construction topology, shape, dimensions, arrangement What does the damper design look properties like?

Location position or site Where do you want the maximum resistive force in the full ON state?

Operation manner of functioning How is the resistive force varied in the hydraulic concept?

Performance behavior, efficiency, evaluation/test What was the maximum temperature results at the solenoid during test?

Rationale purpose, reason, basis, explanation of Why did they choose the rotaxy fric-choices tion concept?

Relation inter-dependence, cause-effect, interac- What is the effect of temperature on tion the solenOid?

Requirement specifications, needs, constraints What was the limit on solenoid stroke length in the previous design?

Miscellaneous process details, plans, strategies, and any- What are the characteristics of the thing not included in the other descriptors suspension system?

Table 3-3 Question subject-class refers to the class to which the subject of a question belongs.The seven classes below cover the range of question subjects encountered in the data.

Subject-Class Subject or the question refers to Example Subject

Assembly an assembly, sub-assembly or a Is the shape of the volume of damper collection of components the damper a cylinder?

Component a part of the assembly, a single Does this end of the arm go top arm item to bottom?

Connection interface between two or more How is the disk attached to the attachment components hub?

Feature attribute of a assembly or com- Why should stroke length be stroke-ponent, physical properties less than 6 inches? length of

solenoid

Requirement specification or constraint of How heavy was the last design? weight the design

Design-concept design concept, partially- How does rotaxy friction con- rotaxy-fric-formed ideas cept develop resistive force? tlon-concept

Other anything not included above What are these four lines in the lines graph of the solenoid?

method skews the data heavily towards the audio medium. As a result I have

not made an attempt to draw any conclusions regarding the medium. Table 3-1

Information Handling Behavior of Designers During Conceptwll Design: Three Experiments


lists some possible values that medium can take. This is not a complete list. It is

a representative sample of the dominating media observed in the study.

3.4.5 Level-of-Detail

If we were to classify the information about a design on how detailed it is, then

the level-of-detail of the design information which answers a question is the

level-of-detail of the question. It is common perception that design requires

information at the following levels-of-detail: conceptual, configurational and detail.

These levels differ in the extent to which design information is specified. Table 3-

4 defines the levels-of-detail with examples.

Table 3-4 Question level-of-detail. refers to the specificity of information sought by the question.

Level of Detail Question seeks information at a Example

Conceptual qualitative, conceptual level What are the other design concept for the force generation mechanism?

Configurational layout, arrangements of components Where does the solenoid fit in in the damper?

Detail specific, quantitative How much resistive force should the damper have when the arm is here?

3.5 Analysis Examples

A few examples of analysis from transcription to classification will clarify the

procedure. Table 3-5 shows these examples. Note that the original verbalizations

are rarely complete sentences or grammatically correct. Question segments are

underlined.

3.6 Results

Table 3-6 summarizes the background of the experimental subjectsl . The first

experiment lasted about five and a half hours and the second about three and a

half hours. 160 question were extracted from the first and 80 from the second

1. The designers in the two experiments will be referred to as RSl and RS2 respectively.

Information Handling Behavior of Designers During ConceplUaJ Design: Three ExperimenlS


Table 3-5 Example of analysis procedure in information reuse study. The segments 7,8,9 and 10 cover a continuous period of about 50 seconds. The question from protocol is underlined and shown again in the second column. The rephrased question is also shown in the second column under the question from the protocol.

Question from Protocol

No. After 'framcription Question Rephrased After Reformulation Question Classification

7 All right, let'srz see. Is this a Is it a Desaiptor. Requirement (pause} This is a, that continuously variable continuously Subjclass: Assembly was design a continuously damper? variable damper in leveL oJDetaH: Conceptual. variable damper. 1a tb1a the design problem? [d~aign g~gbl~l A Is it a continuously

~gDtiDUg~alx ~~1abl~ variable damper in

rumroer' Yeah, okay. Okay, the design problem?

let's see.

S Now, I don't see why, do 00 I have to go with 00 I have to go Descriptor. Miscellaneous I really have to go, or what they came up with the design Subjclass: Design-Concept gg 1 baxe tg ~g ~1tb Kbat with? concept the first leveL of Detail: Conceptual. tb!iD{; !~r~gya aeaign design team came team} came up with" 00 I have to go with up with?

the deSign concept the first design team came up wi th?

9 It depends on what their, What were the What were the Desaiptor. Rattonaie on what they found out drawbacks of the drawbacks of the Subjclass: Design-Concept about the other other systems? design concepts leveL oj Detail: Detail possibilities. tibae dZ;:~ the first design tb~ g~A~~~ka gf tb~ What were the team looked at? other Systems? drawbacJcs of the

other design concepts the first design team looked at?

10 I think they had that in What were the What were the Desaiptor. Performance here. (opening d3, advantages to this? advantages of the Subjclass: Design·Concept flipping pages} When they

Wha t were the hydraulic concept? leveL oj Detail: Detail

were considering other systems, like, um. Okay, advantages of the

the following section hydraulic concept?

points out approaches, (reading} each of which we believe has the potential to develop into a reliable prototype. Each design should mmble (inaudible} into a whole class of possible solutions,frictional,elect rorheological, and moving fulcrum concept. (stop reading} (pause} Okay, wllat lrli:Z;:~ eD!: a~DtAg:Ii::i tg tbh'

one. A possible reason for this difference is the familiarity of the subjects with

the expert. RSI was familiar with the expert and felt comfortable asking question

whereas RS2 was not that familiar and preferred to look through documenta

tion, therefore not externalizing some questions. Another possibility is that RS2

being experienced, asked fewer but pertinent questions and also took about 2

hours less than RSI in coming to a solution.

Information Handling Behavior of Designers During Conceptual Design: Three Experiment'


Table 3-6 Information reuse study: subjects background information.

Design Experiment Familiar with No. of Questions Subject Experience Duration Expert? Extracted

RSl 2 years 5.5 hours Yes 160

RS2 10 years 3.5 hours No 80

The distribution of questions, in the classification, in the two experiments is not

significantly different. The results below are for questions from both the experi

ments. Appendix A has results for individual subjects, RSI and RS2, and a

comparison of any differences.

3.6.1 Results on Descriptors

Figure 3-6 shows the distribution of questions across the descriptors. Informa

tion regarding performance, relation and construction was frequently accessed. All

other descriptors are well represented. About one fifth (21%) of questions were

miscellaneous. Thus this rather sparse classification scheme dealt effectively with

about 80% of the questions encountered. However 21% is a large enough num-

Performance

Relation

Construction

Operation

Alternative

Rationale

Requirement

Location

Miscellaneous

Percent of Questions

21

30

Figure 3-6 Distribution of question deSCriptor. Information with descriptors performance, relation and construction was frequently needed. 21 % of the questions needed miscellaneous information, i.e. the classification works well for about 80% of the questions encountered.

ber and suggests that there is considerable interest in accessing information



about design process strategies, plans, organization, design methods, evaluation

methods and others. These are some of the characteristics of information

spanned by the miscellaneous descriptor.

The information in Figure 3-6 can guide decisions about what type of informa

tion to capture during the design process. The relative frequency of occurrence

of these descriptors can be used to set priorities in developing capture strategies.

3.6.2 Results on Subject-Class

Figure 3-7 shows the distribution of questions across the subject-classes. Informa

tion about feature (35%), components (20%) and assembly (12%) were frequently

asked for. We know that there are more features in a design than components,

more components then assemblies. While it is reasonable to see more questions

35

40 Percent of Questions

Figure 3-7 Distribution of question subject-class. Information on features and components is asked for frequently, but other classes are also well represented.

addressed to the class which has more members, the breakdown emphasizes the

concern for adequately representing all subject-classes in a capture tool.

There were a significant number of questions on the subject-class requirements

(13%) and design-concept (9%). This suggests that adequate attention should be

given to capturing information about requirements and design-concepts.

Information Handling Behavior of Designers During Conceplual Design: Three Experiments


3.6.3 Results on Level-of-Detail

Figure 3-8 indicates that more than half the questions (57%) were asked about

Detail 57

Conceptual

Configurational

o 10 20 30 40 50 60 70 Percent of Questions

Figure 3-8 Distribution of question level of detail. The nature of the design problem used in the experiment may have focused the designers attention to detail information.

the details of the design. This could partly be due to the nature of the design

problem used in the experiment. Since the design problem was reasonably com

plex and time available to the subjects limited, the subjects decided to go with

the original concept quite early in the design process and tried to achieve the

new requirements by making parametric changes. This channeled their interest

to the details of the design for the most part. However, about 28% of the ques

tions are addressed to the conceptual level-of-detail. This asserts the need for

capturing information from the conceptual stage.

3.6.4 Results on Crossing Attributes

Interesting results are obtained by crossing two attributes, descriptor with subject

class, descriptor with level-of-detail and subject-class with level-of-detail.

Descriptor with Subject-Class. Crossing descriptor with subject-class gives us

the knowledge of what type of information about the different subject-classes is

frequently needed. Table 3-7 highlights the combinations of descriptor and sub-

Infonnation Handling Behavior of Desigocrs During Conccptlllll Design: Three Experiments


ject-class encountered frequently. Information about relations of features, operation

Table 3-7 Crossing descriptor with subject-class. Highlighted numbers signify the most frequently asked for descriptor for a particular subject-class.

Subject-Class

Descriptor Assem. Comp. Conn. Feature

Alternative

Construction o o 33

Location o o o 5

Operation o 22

Performance 1 5 43

Rationale 5 20

Relation 3 o 2 38

Requirement 2 2 o o 5 o 10

Miscellaneous 2 8 o 14 10 6 11 51

SCL-Total 29 49 9 83 31 22 17 240

of components and assemblies was frequently asked for. Information about perfor

mance and construction of features was also significant.

Descriptor with Level of Detail: Table 3-8 crosses descriptor with level-of

detail. It can be seen that detailed information regarding the performance, relation

and construction was most frequently asked for. Configurational information was

frequently asked about operations, and conceptual information was often asked

about the alternatives.

Subject-Class vs. Level of Detail: Table 3-9 crosses subject-class against level-of

detail. It shows that detailed information about features and components was

needed often. Configurational information was frequently asked about assemblies

and components and conceptual information regarding the design-concepts and

requirements was needed.

The information such as in the table above can be used in"filtering and prioritiz

ing information during capture. It can also be used in ordering responses to



Table 3-8 Crossing descriptor with level-of-detail. Highlighted numbers show what types of deSCriptors were most needed at a level of detail.

Level of Detail

Descriptor Configuratiooal Detail

Alternative 1 7

Construction 11

Location 0

Operation 2

Performance 8

Rationale 6 4

Relation 9 3

Requirement 5 0 5

Miscellaneous 25 2 24

LOD-Total 66 37 137

Table 3-9 Crossing subject-class with level-at-detail. The highlighted subject-class was most frequently asked for at each level-at-detail.

Subject-Class

Assembly

Component

Connection

Feature

Requirement

Design-concept

Other

LOD-Total

Conceptual

5

14

o

10

queries during retrieval.

3.7 Discussion

Level of Detail

9

4

2 3

0 7

37 137

Descriptor Total

18

33

5

22

43

20

38

10

51

240

numbers show what

SCL-Total

29

49

9

83

31

22

17

240

The objective of this study was to develop an understanding of the information

needs of designers before developing services for supporting information cap

ture and reuse. The goal was not to generalize the results but to utilize the

design information framework, developed in this study, in implementing an

Information Handling Behavior of Designers During Conceptual Design: Three ExpcrimenlS


information management service. The next chapter will go into the details of the

development of this service, called Dedal.

The design information framework has been adequate in classifying information

that deSigners ask for while engaged in design. While it cannot be used to clas

sify all the information designers deal with, it provides an example of

information that should be captured for reuse. This framework will be applica

ble to a wide range of design domains.

A sample of two subjects is quite small to draw any generic conclusions. How

ever, this study does demonstrate experimentation as a powerful method to

develop design process understanding and provides an analysis method for

doing so. Quantitative results obtained in this study have been explained based

on the nature of the experiment, the design problem and current understanding

of the design process. What is unexplained has been used to improve under

standing and refine technique of conducting and analyzing protocol

experiments.

There are differences in the classification developed in this study and the taxon

omy of questions reported in [Kuffner 1991]. The taxonomy in [Kuffner 1991]

classifies a question (and a conjecture) based on its nature (construction, location,

operation, purpose) and topic (assembly, component, interface, feature). Some reasons

for the differences in the outcome of the studies can be attributed to the

following:

• The size of the design problem will influence the range of the question one would encounter. In [Kuffner 1991] the first generation design was done in an experimental setting over a period of 6 hours. In this case the first generation design was a real world design. The bigger scale of the design problem exposed new types of questions which may not have occurred in a simpler problem.

• Designers do not carry out design with a pre-determined set of questions. They raise questions as new information is needed. The questions are often triggered by exposure to information accessed by them. In [Kuffner 1991]

Information Handling Behavior of Designers During Conceptual Design: Three ExperimenL~


the documentation available to the subjects was in the form of machine drawings and specifications. In this study subjects had access to a more complete and extensive documentation. The completeness in documentation provides an opportunity to collect a more complete set of questions that designers can raise.

• The questioning behavior is not random. New questions are asked after reflecting on information received in answer to a question. So the type of questions asked will depend on the form in which the information is available to the designer. For instance, if a designer was given a machine drawing it would be very likely for him to ask questions about location and connections, however if the same information was given in a descriptive form, the line of questioning could be different. In this experiment the information was available to designers in many forms, making it possible to collect a more complete set of questions.

3.8 Recommendations for Information Reuse Service

Based on the results of the information reuse study, here are a few recommenda

tions for information capture and reuse service:

• it should be able to capture and represent the information described by the design information framework

• it should keep pace with the generation of information. i.e. information capture and indexing should happen during the design process and should not be a post-facto add on activity.

• it should be capable of dealing with information at various levels of detail.

• it should be capable of dealing with multi-media information.

3.9 Summary

I have discussed quantitative and qualitative results from an empirical study of

design information reuse during design. I have developed a method for analyz

ing the questioning behavior of designers and used it to develop a framework

for classifying and understanding design information needs. The framework

characterizes design information based on its descriptor, subject-class, level-of

detail and medium. This framework is the foundation for the development of a

service for information capture and reuse discussed in Chapter 4.


Chapter 4

Dedal: An Information Management Service

/-------- ,.-----, /-----, ( Information Classification \ { Dedal Performance , { Information Handling,

\ Framework / I \ Results / \ Framework / ..... _----- ------ ------

t t . . t --------- ----------- ---------

Analyze Analyze Analyze

/. D~m ;.~ ).2-'~\ of DadaI Dadal

19901991 1991-1992

Information Support by Observe Support by Information Reuse Study DadaJ • DadaJUsa DadaJ - Handling Understanding

Study of Information Handling

This chapter describes the information management utility called Dedal which embodies services for indexing, modeling and retrieving design information. It uses the design information framework described in Table 3-1 to implement a conceptual indexing method. This chapter will report on Dedal's retrieval performance and results obtained when it was deployed to support design activity in real time. Details regarding implementation of Dedal are discussed in AppendixB.


Chapter 4: Deda/: An Information Management Service 58

4.1 Introduction to Deda11

Using the requirements extracted in the information reuse study (section 3.8), a

service for managing design information was developed. This service is called

Dedal2. Dedal provides tools for indexing, modeling and retrieving design infor

mation in real time. Figure 4-1 shows the different components of Dedal and the

flow of information among these components.

CD en cu .0 cu -cu C en

"C ~

o ()

CD a: c C)

en CD C cu "0 CD

E -::l

USER

Graphical User Interface

Modeling Tool

Retrieval Heuristics

::iE - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - •

Figure 4-1 Dedal system architecture. The dark shaded area represents the information capture environment and the light shaded area show the components which make up Dedal. Dashed arrows indicate information transfer. Indexing tool is used to create the indices database, modeling tool to create a domain model, and retrieval tool to answer queries. The indices database and the domain model are used by all the tools. The information capture environment is the tool designers use to document design information. It is integrated with Dedal via the graphical user interface. The multimedia design records database represents design information captured by the information capture environment.

1. Dedal has been developed in collaboration with Catherine Baudin and Jody Underwood from NASA Ames Research Center and Ade Mabogunje from Stanford University. 2. The name Dedal comes from the greek mythological character Daedalus, the legendary builder of the Cretan labyrinth who makes wings to enable himself and his son Icarus to escape imprisonment. It is the characteristics of finding way in complex labyrinth that is a parallel to the function of Dedal.



The tools for indexing, modeling and retrieval are accessed via a graphical user

interface which integrates Dedal with an information capture environment. The

screenshots of these interfaces are shown in Appendix B. The information cap

ture environment would typically be a collection of tools that designers use to

carry out their day to day work such as: word processor, drawing/sketching pro

grams, spreadsheet, CAD /CAE tools, electronic mail and others.

Dedal deals with multimedia information and enables indexing and modeling to

take place in real time. It deals with information at all levels of detail and formal

ity. For more details on the development of Dedal and its architecture please see

[Baudin et.al. 1993a]. A copy of this paper is included in Appendix B. Dedal is

written in common lisp. The interfaces for dealing with audio and video informa

tion are written in the language C.

4.1.1 Indexing Tool

The indexing tool is used to generate the indices .database using a conceptual

indexing scheme. In conceptual indexing, an information segment is indexed by

describing the concept represented in it. Concepts are domain dependent enti

ties which embody the semantics of the domain. In Dedal the concept is

described by using the design information framework in Table 3-1, Le. informa

tion is indexed by the classification of the question it answers. Figure 4-2 shows

an example of an information segment and its conceptual index.

The size and granularity of an information segment can vary depending on the

type of document and the medium of information. Typically a segment is a para

graph of text or a single graphic. It is possible, and often necessary, to index the

same information segment with more than one index. This is specially true of

cases where the segment is rich enough to answer multiple questions. Since

Dedal is integrated with the information capture environment the designer only

specifies the descriptor, subject, and level-oj-detail of the information. The medium,



document, segment and paragraph-no. are automatically picked up through the

interface.

Indexing pattern on the information segment on page 18 of document DRD-spring-1993:

Descriptor: Rationale Subject: Solenoid Medium: text Level of Detail: Conceptual Document: DRD-spring-1993 Segment: 18 Paragraph no: 2

A question which will return the above index as answer:

Descriptor: Rationale Subject: Solenoid

'C'he solenoid was chosen since It Is able to generate upto 30fbs of force with the given power and ••••••••••

18

Figure 4-2 Example of conceptual indexing. Paragraph 2 on page 18 of the document DRD-spring-1993 answers the question: What was the rationale of the solenoid? Therefore the indexing pattern for this segment is as shown. The concept this index represents is solenoid, a domain dependent entity.

4.1.2 Modeling Tool

The modeling tool is used to generate and maintain a model of the design infor

mation. A model in Dedal is a means of defining relationships among the

domain concepts in a hierarchical manner as shown in Figure 4-3. These relation

ships are typically of the form: part-of, is-a, depends-on and others. The model

includes a representation of the artifact structure, requirements, alternatives and

decision points and it evolves incrementally with the design process. The simple

hierarchical structure of the model assures that designers themselves can create

them in real time. The modeling tool also helps maintain consistency of concept

names and relations and provides browsing and editing facilities ..

4.1.3 Retrieval Tool

The retrieval tool is used to get answers to queries. It uses the indices database,


Chapter 4: Deda/: An Information Management Service

disk-stack solenoid

/ part-of ~

/ part-of

" outer-clisks inner-clisks

Rgure 4-3 Example of a model in Dedal. The figure shows the simple hierarchical representation which relates domain concepts.part-of and is-a relations can be seen in this model.

61

the domain model and heuristics to direct user to appropriate locations in the

design records database. The algorithm used by the retrieval tool is shown in

Return Answers to User

Figure 4-4 Flow chart of the retrieval algorithm used by Dedal. The answer for a query is first searched in the model •.

Figure 4-4. A query is phrased by specifying the descriptor and subject(s)

(Figure 4-2). The retrieval algorithm first tries to answer the query from the

model. The model has information about decisions made and alternatives consid-

Information Handling Behavior of Designers During ConceplUal Design: Three ExperimenL~


ered. Simple questions about them can be answered from information in the

model. If this fails to find a relevant answer then the indices database is searched

for an index which matches the query. Failing this heuristics are used to expand

the search. Dedal uses three types of heuristics. They are:

1. Related descriptor heuristic: This heuristic uses the fact that the boundaries between the descriptors often overlap. Therefore, sometimes information about a descriptor of a subject can be found by searching for a information about a different descriptor of the same subject. An example of such a heuristic is:

if (question is rationale of X) then

(search for decision of X)

2. Proximity heuristic: This heuristic uses the fact that a document is a continuum of information. Often a certain type of information can be found near another type of information. An example of such a heuristic is:

if (question is function of X) and (X is part-of Y)

then (search for operation of Y)

3. Causal relation heuristic: This heuristic uses the knowledge of certain relations which are causal in nature to change the subject of the query. For instance:

if (question is rationale of X) and (X depends-on YJ then

(search for rationale of Y)

Many examples of the heuristics are included in Appendix B.

4.2 Dedal Test Domains and Results

To measure retrieval performance of conceptual indexing scheme and the real

time performance of Dedal, we deployed it to support design information man

agement in three domains listed in Table 4-1. Results obtained in each of the

domains are discussed below.

Information Handling Behavior of De..igners During Conceptual De..ign: Three Experiment.

Chapter 4: Dedal: An Information Management Service 63

Table 4-1 Dedal test domains details. I = indexing, M = modeling and R = retrieval.

Project Capture Designer No. Domain Duration Dedal Feature Thsted Platform Medium Indexes?

1 Damper 7mon R perfonnance Symbolics vmacsa Yes

2 Bioreactor-I 7mon Real-time I & M Unix Framemaker No

3 Bioreactor-II 2mon Real-time R Unix Framemaker No

a. vmacs™ is a trademark of perfonnance graphics company.

4.2.1 Damper Domain Results

The first testbed domain was the design of a continuously variable damper

[Baya et al. 1990}. This domain was used to assess the retrieval performance of

the conceptual indexing scheme implemented in Dedal. Design information was

captured in an electronic design notebook called vmacs [Lakin et al. 1989}. A

member of the design team used Dedal to index about 400 pages of documenta

tion with 340 indices. Forty distinct questions, the answers to which were

available in the documentation, were selected from the 240 questions extracted

in the information reuse study (Table 3-6). Note that questions to which the

answer is not in the documentation cannot be used to assess retrieval perfor

mance since no answers retrieved would be relevant. This will cause the results

to be biased towards poor performance.

Figure 4-5 shows the retrieval performance measured is this study. A compari

son was done with full-text boolean retrieval scheme, a popular information

retrieval method. Full-text boolean retrieval is a baseline reference point for com

parison. The retrieval performance is measured by two parameters, recall and

precision. Recall is the ratio of the relevant answers retrieved to the total number

of relevant answers present in the information database. Precision is the ratio of

the number of relevant answers retrieved to the total number of answers

retrieved. A perfect retrieval system would have recall and precision of 100%.

As is clear from Figure 4-5, conceptual indexing scheme of Dedal out performed


Chapter 4: Deda!: An Information Management Service

Recall = # of right answers retrieved total right answers present P

.. # of right answers retrieved recision =

total answers retrieved

100~------------------------------------~

~ • -- • -- • -- - -- - -- ... Oedalwith , retrieval heuristics

80

60 Boolean retrieval I as 0 CD a:

_-_0 - - - - - - - - - - _ - - - - _.£ _ - - -B Dedal with Jut

I I : retrieval h uristics 40 I

I

20 I I I

00 I

20 40 60 80 -

100

Precision

Figure 4-5 Retrieval performance of conceptual indexing. Oedal's retrieval engine based on conceptual indexing performed far better than full-text boolean retrieval. Oedal's recall can be improved by using heuristics to widen the search space. However, it does lead to some loss in precision.

64

the boolean retrieval scheme by a wide margin. The precision of boolean

retrieval scheme was measured at 10% and that of Dedal at 80% (for exact

match, i.e. when not using retrieval heuristics). The recall was comparable at

around 52%. Dedal's recall performance can be improved by using heuristics to

loosen the search but at the expense of some loss in precision. Nevertheless, the

gain in recall, about 40%, is far more than the loss in precision, about 17%. It is

important to note that boolean or full-text search requires no indexing effort

unlike conceptual indexing which requires effort in indexing. Clearly the gain in

performance justifies the effort required in conceptual indexing, but can this

effort be expected of designers is a separate question. Another question which

rises is what techniques can be used to automate/semi-automate conceptual

indexing?

Information Handling Behavior of Designers During ConceplUai Design: Three Experiments

Chapter 4: Deda!: An Information Management Service 65

4.2.2 Bioreactor Domain Results

The bioreactor domain was used to assess the real-time performance of Dedal.

Dedal was deployed to support indexing and modeling of design information in

real time, i.e design information was indexed and modeled during the design

process. The objective of the bioreactor project was to design an apparatus for

interactive control of a life sustaining environment for bacteria. The bioreactor

project took place in two phases. In the first phase (I) a bioreactor was designed

over a period of 7 months by a team of three designers. This phase resulted in a

working prototype. In the second phase (IT) a team of three different designers

redesigned a subsystem of the bioreactor to improve on its design. The indexed

and modeled documents from bioreactor phase I were made available in bioreac

tor phase II. This was an ideal situation to determine if the effort put into

capturing design information from bioreactor phase I would payoff during

phase II. The indexing and modeling activities in phase I were done by a partici

pant designer (not one of the three principal designers). Following are some of

the observations that were made:

Bioreactor Phase I:

• information framework was adequate for indexing information for reuse

• the information framework was intuitive since the designer was able to index, model and formulate queries without formal training

• it was possible to index and model incrementally with the design process (about 2 hrs per week of additional effort was required)

Bioreactor Phase II:

• information reuse was effective, the design team redesigned and prototyped a new design in two months, relying only on Dedal to learn about the previous design

• on-line access to information was very effective. It contributed to improved design efficiency and practice

Information Handling Behavior of Ocsigners During Conceplual Ocsign: Three Experiment'

Chapter 4: Oeda/: An Information Management Service 66

4.3 Discussion on Dedal: Lessons Learned

Development and deployment of Dedal was an important learning experience. It

convinced me that design information capture has to be an integral part of the

design process and cannot be demanded as an add-on activity. Some how, the

process of structuring and archiving design information for reuse has to be seen

as a requirement of the design process so that it happens incrementally and in

real time.

Dedal has a framework for classifying design information. This framework can

be used for modeling and indexing design information in real time. It also

allows for accurate retrieval. Nevertheless, it has lessons. Some of them are:

• the activity of indexing and modeling are add-on activities.

• the modeling process, although incremental, is static in nature. The user has the burden of monitoring and reorganizing concept names as they evolve, which is a considerable cognitive load.

• the payoff of indexing and modeling was seen mostly by the redesign team and not by the team which did the indexing and modeling. It is my belief that information capture would always remain an add-on activity unless the rewards of these activities were immediate.

• the integration with the information capture environment is only a rough draft. It should be carefully redesigned to empower user indexing and modeling.

• the definition of level-ofdetail was not very useful. Although it was specified during indexing, users did not use it during retrieval. Also the retrieval algorithm did not use this information to any significant advantage.

It was evident from these limitations that development of integral and intuitive

services for capturing and managing design information require a deeper under

standing of the dynamics of information handling behavior. We need to

understand the various activities designers perform with information and how

they change over time in characteristics and content. This understanding would

provide the clues necessary for implementing intuitive and integral services/

interfaces. To my knowledge no one has reported such an understanding. To

Information Handling Behavior of Designers During Conceptual De.~ign: Three Experiments

Chapter 4: Dedal: An Information Management Service 67

develop this deeper understanding I conducted the information handling behav

ior study, discussed in the next chapter.


Chapter 5

Information Handling Behavior Framework

/--------{ Information Classification \ \. Framework } ------_/

/-----'" { Dadal Performance \ \. Results /

,...-----, ( Information Handling \ \. Framework /

i i t ------------------------------Analyze Analyze Analyze

1~~I00'f:;r.rmt !~1_19~~ 1-- 995

Information Support by Observe Support by Information Reuse Study Dadal - ...... ~~ Dadal Usa Dedal - ...... ~ Handling

Study


This chapter describes the experimental procedure and the analysis method of an observational study conducted to understand the information handling behavior of designers engaged in conceptual design. Information handling is studied in the framework of activities designers perform with information and the characteristics of information handled in those activities. A quantitative measure for design information is defined and used to measure information handling rate. Many examples of the analysis procedure are included in this chapter.

Information Handling Behavior of Designers During Conceprual Design: Three Experiments 68

Chapter 5: Information Handling Behavior Framework 69

5.1 Introduction

The experience of developing and implementing Dedal (Chapter 4) resulted in

many unanswered questions regarding information handling. Some of the key

questions were:

1. What are the activities designers perform with design information?

2. What is the temporal distribution of design information processing?

3. Can design information be measured quantitatively?

4. What is the information content in different activities?

5. How does information processing rate change during the design process? and,

6. What do answers to the above questions inform us about developing intui-tive and integral tools to support information handling?

To find answers to these questions I conducted the information handling behavior

study using the verbal protocol method. The experimental approach was similar

to the one used in the information reuse study, reported in Chapter 3. The differ

ences were in the design problems used in the experiments and the analysis

method. To avoid repetition I would only bring out aspects of the experimental

approach and analysis procedure which are different from the description in sec

tions 3.2 and 3.3.

5.1.1 Experimental Setup and Procedure

A total of six experiments were conducted in this study. The experimental setup

is same as the one used in information handling study (shown in Figure 3-2).

Table 5-1 lists some details on the experimental subjects and the design prob

lems used. Three different design problems were used. They are included in

Appendix C. Two of the design problems dealt with design of a bike-lock and

the third one was about designing a harness for attaching a backpack to moun

tain bikes.



Table 5-1 Details on experiments and experimental subject background. The table lists the design time, design problem and the design experience of each of the experimental subjects.

Design TIme Design Experience (yrs)

Subject (minutes) Design Problem Academic Industrial

Sl 78.8 bike-Iockl 4 2

S2 67.5 bike-Iockl 3 2

S3 71.4 bike-Iock2 I 0

S4 74.4 bike-Iock2 2 4

S5 51.2 bike-Iock2 3 0

S6 119.5 back-pack 0 20

The first two design problems were based on the everyday activity of locking

and unlocking a bike. In short the design problem was to design an off the shelf

lock for bicycles which could be integrated with the bike and operated without

having to take it off the bike (as opposed to kryptonite locks or U locks which

are not integrated with the bike and are operated as an independent unit). This

design problem was chosen because: it required no special technical back

ground, it enabled the designers to work within a space of familiar vocabulary

and technical jargon and it was suitable for an hour long conceptual design exer

cise. The designers had access to a bike and a kryptonite lock.

5.2 Analysis Procedure

The objective of the analysis procedure was to transform raw data from the

experiments into quantitative descriptions of the information handling behavior.

The procedure is shown in Figure 5-1. Some of the steps, conduct experiment and

transcribe, are same as described in section 3.3. The other steps, segmentation and

classification are described in sub-sections below.

5.2.1 Segmenting into Information Fragments

An information fragment is a continuous period during which the informational


Chapter 5: Information Handling Behavior Framework

Design Problem Conduct Recorded Data Transcript

F Experiment Transcribe

Information Fragments

Segment into Information Fragments

Quantitative

Classify Classification Statistical Results

F Fragments Analysis

Figure 5-1 Analysis procedure for information handling study. This procedure transforms raw data into quantitative description of the information handling behavior.

71

activity (activity performed with information) and the descriptor of information

(defined in section 3.4.2) remains the same. Segmentation of the transcript into

information fragments was done by developing a set of rules to identify points

of transitions between fragments. These rules are:

1. Typical points of transition could be semantic such as: change in designer's train of thought, change in focus of attention or change of action.

2. Typical points of transition could also be syntactic such as: significant pause, change in the medium of information or utterance of certain phrases such as "and", "ok", "alright", "umm".

3. The level-ofabstraction during an information fragment remains the same.

4. The level-ofdetail during an information fragment remains the same.

5.2.2 Classifying Information Fragments

Before an information fragment could be classified, a framework for describing

the information handling behavior was needed. Section 5.3 goes into the details

of the development of this framework. For now it is sufficient to note that an

information fragment is classified by

1. specifying the informational activity associated with it (section 5.3.1),

Information Handling Behavior of Designers During ConceplUai Design: Three ExperimenlS


2. classifying the information in the fragment using a revised design information framework (Table 5-2), and

3. quantitatively measuring the design information content using design information measure (section 5.3.3).

Figure 5-2 shows the template used for this classification. Note that for frag-

(info-fragment :activity infonnational-activity :time (start-time end-time) :descriptor descriptor :subjclass subject-class :subjects (subject-list)

:medium medium :levels (level-of-detaillevel-of-abstraction)

:measures (design-in formation-measure ) )

Figure 5-2 Template for classifying an information fragment. This template captures values for nine attributes of each information fragment. These attributes describe the informational activity and the classification and content of design information in that fragment.

ments rich in content it may be necessary to encode it with two or more

classification.

5.3 Information Handling Framework (IHF)

Table 5-2 shows the information handling framework (lliF) which is used for

describing information handling behavior. This framework is an extension of the

design information framework (DIP) developed in the information reuse study (Table

3-1). Three new attributes were added to the DIP to create the lliF. These are

informational activity, lev~l-of-abstraction and design information measure. Each of

these are defined in detail in sub-sections below. The lliF adequately describes

the activities designers perform with information, and the characteristics and

content of the information handled in those activities. Monitoring design activity



within this framework over time will improve our understanding of the dynam

ics of information handling behavior.

Table 5-2 Information handling framework (IHF). This includes a revised design information framework (DIF) to classify design information. Three new attributes namely, informational activity, level-of-abstraction and design information measure are added to DIF to create the IHF. The values these attributes can take are shown below.

Information Handling Framework (lHF)

Design Information Framework (DIF) Informational Level of

Activity Abstraction Descriptor SubJect-Class Medium Level of Detail

Generate Unlabeled Alternative Assembly Audio Conceptual

Access Labeled Assumption Component Video Configurational

Analyze Associative Comparison Connection Text Detail

Qualitative Construction Feature Graphic

Quantitative Location Requirement Gesture

Operation Design-concept

Performance Other

Design Information Measure (dim) Rationale

Quantitative measure: takes Relation integer values (1,2,3, and so on) Requirement

Miscellaneous

Since the scenario of experiments and the focus of observation in this study was

different from the information reuse study it was necessary to revisit the devel

opment of the OIF using the approach described in Figure 3-4. It is encouraging

to report that the DIF of Table 3-1 required only minor modifications. Two new

values were added for the attribute descriptor. The other attributes remained

unchanged. All attributes making up the DIF are redefined below in the context

of information handling behavior.

Descriptor. This refers to the basic constitution or the nature of the information

in an information fragment. Two new descriptors were added. They are defined



in Table 5-3. (details on other descriptors are in Table 3-2).

Table 5-3 Descriptors added to the design information framework. Other values are defined in Table 3-2.

Example Information Fragment Descriptor Information regarding from Protocol Explanation

Assumption supposing the truth or validity in general there is probably Designer assumes

of a fact or taking for granted enough room around here that you information regarding could,inside the frame, inside space constraint this tube attach this pivot, slash sliding mechanism,

Comparison examining two or more con- (task of locking} maybe a little Designer compares

ceplS to determine similarities bi t more than 1IIIIIIIIIIl •• , maybe a the task of locking to little bit more than changing a that of changing tires

and dissimilarities tire is permissible

Subject-Oass. This is the class to which the subject of an information fragment

belongs (details in Table 3-3).

Medium. This is the physical form in which the information fragment exists

(details in section 3.4.4).

Level of Detail. It is the phase of the design process to which the information

fragment can be associated with. (details in section 3.4.5).

5.3.1 Informational Activity

The activities designers perform with information will be called informational

activities, in this thesis. It is possible to view all actions as being performed with

information. However, for the purpose of this study, only actions which are

explicitly captured in the protocol data and which need to be supported by com

putational services are of interest. Abstract and unobservable actions such as

thinking, observing, speculating, ideating are not relevant to this study. Figure 5-

3 shows the process of developing the informational activities classification. In

the initial stages this process was tightly coupled with the development of rules

for segmentation of the transcript. This is because some transition points are at

the start and end of an informational activity, and therefore without a classifica

tion on activities it is not possible to identify transition points consistently and

objectively.

Information Handling Behavior of Designers During Conceptual DC5ign: Three Experiments

Chapter 5: Information Handling Behavior Framework

< Information Fragment

t step 1 Characterize informational action ~

, I \.... • "-.....

> --

..- ----" Segmentation: \ ~ Identify transition points I ,_ .......

'-------/

//" No / \" ~

/ / \ " / " , I \ .... step 2 step 6

Collection of all informational actions Can all info-fragments be classified?

~""-/~ / Group inlo Infannational Activities /.tep 5

t step 4 Classify N.:,teP 5 Label Activities --•• ~ Info-Fragments

Yes

~

Figure 5-3 Evolution of the informational activity classification. The cloud on the top right hand side signifies that the process of segmentation was coupled with the evolution process (in the initial stages).

75

The resulting classification of informational activities is shown in Table 5-4. It

shows that informational actions can be grouped into three classes namely Gener

ate, Access and Analyze. This is a minimal view of the informational behavior of

designers. However, it was possible to classify all actions that were observed in

the experiment in these three activities. This classification is a suitable level of

granularity for the purpose of this study. Note that this classification is most suit

able for a single designer scenario. A team design scenario would require a

different classification.

To better understand this classification, assume that at any time in the design

process there is an information space associated with the design. At the begin

ning, this information space may be thought of as containing the design

requirements. As the designer works on the deSign, he/she makes changes to

the information space, either by adding information to it by generating/accessing



it or they transform information in the space by analyzing it.

Table 5-4 A classification for the Informational activities. Their definitions, associated actions and example. The list of associated actions is a representative list, not an exhaustive list.

Informational Defioition/lnterpretation and Example Information Activity Associated actioM Fragment from Protocol Explanation

Generate An action which adds new informa- so umm I wonder if we Actiqn: Talk

tion to the information space from could do a combination Designer generates an of a the 0 lock type

an unidentified source. Actions: arrangement and the alternative of combining two

~ting,dr.awing,tUUdng doughnut type arrangements

arrangement

Access An action which references informa- {reading from dl} Action; Bead

tion within or outside the informa- Locking mechanism Designer accesses a should be an off the

tion space from an identifiable shelf item requirement by reading it.

source. Actions: read, listen, recall, request

Analyze An action which changes an than you really have to ActioQi ~11I1Iti:

attribute of the information frag- work it {apply force} Designer analyzes the bending this tube to

ment Action: interpret, organize, get this thing out, performance on strength by

calculate, evaluate making a qualitative evaluation of strength

5.3.2 Level of Abstraction

Information in design evolves from abstract to concrete. Thus the ability to repre

sent and monitor this evolution is essential to studying information handling.

While level-ofdetail was defined in the DIF to address this concern, its granular

ity and definition was not adequate to empower structuring or retrieval of

information (section 4.3). Level-ofabstraction has been defined as a substitute for

level-ofdetail. It represents the evolution of concepts in the design process.

Table 5-5 shows the different level-ofabstractions used in this study along with

their definitions and interpretations. The definition and granularity of these lev

els is based on observations from the experiments and therefore are most

suitable to represent the information encountered in the conceptual design pro-



cess. The levels in the table are arranged in a decreasing order of abstractness.

Table 5-5 Definition and interpretation for level-at-abstraction. The levels are arranged in decreasing order of abstractness. going from top to bottom. Subject" in the table refers to the subject of the information fragment.

Level of Example Information Abstraction DefinitionlInterpretation Fragment from Protocol Explanation

unlabeled Referring to the subjectof an inforrna- so its not really a The designer makes no

tion fragment without a name, as in a wedge its a what do mention of the name for we call it. okay. the subject (Kreally not a

idea or a new concept what I know what it wedge") that helshe is looks like. is chat describing right?

labeled Referring to the subject of an inforrna- umm but the reason The subject under

tion fragment by a name, either by gen- they are chains. I consideration is think is that you can mentioned by name i.e.

erating a new name or reusing a previous have any size bike to Kchains"

name lock.

associative Referring to the subject of an infonna- where aaa it kind of The subject (Kit") of the

tion fragment in relations to or by asso- hooked underneath. fragment is associated hooked underneath the with the -Wheel- and

ciating with other concepts/subjects wheel or its also Kframe-underneath the frame.

qualitative When a subject or features of a subject Citadels are probably The feature Kstrength of

are qualitatively addressed, a lot stronger right citadel- is qualitatively here in the middle described as stronger

or or or

When operation or motion is qualita- (unlocking oper) and The operation of

tively described then you put the key unlocking is in turn it • its just qualitatively described spring loaded and its as a sequence of actions just opens up •.

quantitative When a subject or the features of a sub- it looks just by The feature "length- of a

ject are quantitatively described looking at that bike component is it would be limited quantitatively on a mountain bike to described, " 6 to 6 to B inches in 8 inches-length •.•

5.3.3 Design Information Measure (dim)

Measuring information is an old concept. It originated in the field of communica

tions engineering, wherein the quantity of an information message is inversely

related to the probability of receiving that message from a source [Shannon and

Weaver, 1949). This measure is related to the statistical properties of a communi

cation system and does not depend on the semantic content of the message.

[Kim, 1990 and Suh, 1990) suggests a definition of information content from a

cybernetic perspective. Here the quantity of an information message is related to

the effect of the message on a system's ability to achieve a goal. So the measure

of information content is with respect to a certain goal. To use this measure one

Information Haodling Behavior of Designers During Conceptual Design: Three Experimenrs


needs to define a goal and a probability distribution on information regarding

its potential to bring the system closer to the goal. Neither of these two mea

sures are relevant to the nature of analysis done in this study. This is because the

analysis aims at measuring semantic information and it is not possible to put a

probability distribution on semantic information with regard to a goal.

The design information measure (dim) is a quantitative measure of the amount of

semantic design information contained in an information fragment. The defini

tion is based on the information fragment classification shown in Figure 5-2.

Every unique [descriptor, subject] pair (to be referred to as a DS pair from now

on) in the classification constitute 1 dim of design information. Thus, if a frag

ment is classified by 1 descriptor and 3 subjects (in the subject-list), it forms 3

unique DS pairs and its content is 3 dims. Note that DS pairs represent the seman

tic content of an information fragment. A DS pair is interpreted as; the fragment

contains information regarding the D of S (Figure 3-5). For instance, if a frag

ment was classified as [operation, solenoid], the interpretation suggests that the

fragment has information regarding the operation of solenoid.

5.3.3.1 Definition of dim: Assumptions

Here are a few assumptions made in defining the measure dim. This will help

understand the rules for calculating this measure.

1. The content of an information fragment is encoded in its classification.

2. The measure takes contributions from two attributes in the classification namely: descriptor and subjects (refer to Figure 5-2).

3. Assumptions pertaining to descriptors:

• All descriptors encode equal amount of semantic information. i.e. [Dl, 5] has the same measure as [D2, 5], where Dl and D2 are two distinct descriptors and S is a subject.

• A fragment classified by more descriptors has more content, and the relationship is linear. i.e. all else remaining same, a fragment classified by 2 descriptors has twice the content of a fragment classified by 1 descriptor.



4. Assumptions pertaining to subjects:

• Subjects from all subject-classes encode equal amount of semantic information. i.e. [D, SI] where SI is a SCI is equal in content to [D, S2] where S2 is a SC2. SC stands for subject-class and SCI and SC2 are two distinct subject-classes (for e.g .. assembly and component).

• A fragment which refers more subjects has more semantic content, and the relationship is linear. i.e. all else remaining same a fragment classified by 2 subjects has twice the content of a fragment classified by 1 subject.

5. All values of level-ofabstraction encode equal amount of information, i.e. a labeled OS has the same content as a qualitative OS. Therefore, since each fragment has one level-ofabstraction (by definition), the dim measure is independent of the level-oj-abstraction.

6. All values of level-ofdetail encode equal amount of information, i.e. a conceptual OS has the same dim measure as a detail OS. Therefore, since each fragment has one level-ofdetail (by definition), the dim measure is independent of the level-ofdetail.

7. Since information in any medium can be represented as a OS pair, the measure is independent of the medium of information. i.e. A graphical fragment and a textual fragment with the same OS classification have equal content.

5.3.3.2 Rules for measuring dims

Once a fragment has been classified the rules below help us determine the dim

measure:

1. For fragments with 1 descriptor and 1 subject the measure is 1 dim.

2. For fragments with 1 descriptor and n subjects the measure is n dims.

3. For fragments with n descriptors, the measure depends on how many subjects are associated with each of the descriptor. If descriptor i has Si subjects then the measure is sum of all Si ' s dims.

Table 5-6 shows some examples on how the above rules were used in counting

Information Handling Behavior of Designers During Conceptual Design: Three Expcrimcnl5


dims for information fragments.

Table 5-6 Examples of dim measurement. Three examples are listed, each using a different rule to measure the information content.

Information Fragment CIassification

Example Information Number Fragment from Protocol Descriptor Subject-list oCdims Explanation

than you really have to Performance strength-of-tube 1 one unique DS pair: work it (apply force) 1. [perfomuuu:e, strength-of-tube] bending this tube to get so the measure is I dim this thing out

so like the lets assume we Alternative lock 2 two unique DS pairs: are gonoa attach it (lock) down-tube 1. [alternatiue, lock] and to the down tube. 2. [alternatiue, down·tube]

so the measure is 2 dims

I am trying to decide if Location bike-lock 1+2=3 This fragment has two classifications there is any apparent

Comparison near-wheel which make three unique DS pairs:

advantage to locking right l.llocation. bike·lock] near this wheel verses down-tube 2. [comparison, near-wheel] and locking on this part 3. [comparison. down-tube] here (down-tube) so the measure is 3 dims.

5.4 Analysis Example

In Table 5-7 below is an example of the analysis procedure from transcription to

the classification of the information fragment, for about 1 minute of verbal pro

tocol data. The data is shown in three different stages: after transcription, after

segmentation and after classification.

Table 5-7 Examples of information handling behavior analysis_ In the table below protocol data is shown at different stages of the analysis. The first and the last row show brief transcript before and after the segment analyzed below_ The rows form a continuum from the protocol, so only the start time for each fragment is shown. The end time is the start time of the next fragment.

After Start1ime Transcription Stamp After Segmentation After Classification

Transcript Sefore: (OO:56:47:22)_: OK I have spent er forty-five minutes or so now forty minutes doing that er so em ... (mutter) going to check every possible location here

Information Handling Behavior of D.:signers During Conceptual Design: Three Experiments


Table 5-7 Examples of information handling behavior analysis. In the table below protocol data is shown at different stages of the analysis. The first and the last row show brief transcript before and after the segment analyzed below. The rows form a continuum from the protocol, so only the start time for each fragment is shown. The end time is the start time of the next fragment.

After SfartThne 1hmscription Stamp After Segmentation After Classification alright so 00:56:56:20 alright so we do have (info-.fragment we do have one that comes up front :activity ACCESS one that :time (00565620 005659241 comes up :descrtptDr AL7ERNA'IlVE front em :subjclass DESIGN-CONCEPI' really a :subjects (front-position) little bit :medium VIDEO wary about :levels (CONCEP1UAL lABELED1 backpacks :measures (1]1 on em

em really a little bit (info-.fragment fronts of 00:56:59:24

bikes em wary about backpacks on :acttvity ANALY:ZE let's see em fronts of bikes em :tfme (00565924005710061

if we can :descrtptDr AL7ERNA'IlVE

em and :subjclass DESIGN-CONCEPT

there is :subjects (front-position)

that issue :mediumAUD10

of it being :levels (CONCEP1UAL lABELED1

off the :measures (111 side __ 00:57:10:06 let's see if we can em (info-.fragment

and there is that issue :acttvity ACCESS of it being off the :tfme (00571006 005723261 side __ :descrtptDr AL1ERNA1TVE

:subjclass DESIGN-CONCEPI' :subjects (side-position) :medium VIDEO :levels (CONCEP1UAL lABELED1 :measures (1]]

you know 00:57:23:26 you know from the (info-.fragment from the aesthetics standpoint :acttvity GENERA7E aesthetic everybody likes things :tfme (00572326 005730051 standpoint symmetric :descrtptDr REQUIREMENr everybody :subjclass ASSEMBLY likes :subjects (syrnmetry-for-lDoks1 things :medium AUDIO symmetric :levels (CONCEP1UAL lABELED1 and er this :measures (111 is not that

and er this is not that (info-.fragment big a pack 00:57:30:05

em my big a pack :acttvily ACCESS

initial :tfme (00573005 005736001

tests :descrtptDr CONSTRUC7!ON

indicated I :subjclass FEA1tTRE

probably :subjects (size..of-padc}

couldn't :medium AUDIO

have one :levels (CONFIGURATIONAL QUALITATIVE1

right like :measures (1]1

that 00:57:36:00 em my initial tests (info-.fragment indicated I probably :activity ACCESS couldn't have one right :time (00573600 005742141 like that (horizontal :descrtptDr AL1ERNA'IlVE on the back! :subjclass DESIGN-CONCEPT

:subjects (badc-horlzDntal] :medium VIDEO :levels (CONCEP1UAL UNlABELED1 :measures (1]1



Table 5-7 Examples of information handling behavior analysis. In the table below protocol data is shown at different stages of the analysis. The first and the last row show brief transcript before and after the segment analyzed below. The rows form a continuum from the protocol, so only the start time for each fragment is shown. The end time is the start time of the next fragment.

After StartThne 1hmscription Stamp After Segmentation After Classification

certainly I 00:57:42:14 certainly I would not (fnfo-.fragment would not do it this way :acttutty ANALY2E do it this {vertical in the back} :time (00574214 00574822) way I would :clescriptDr AL1ERNAnvE do it that :subjclass DESIGN-CONCEPr way em in :subjects (back.vertical} fact I :medlum VIDEO would em :levels (CONCEPItJAL UNLABELED) maybe do it :measures (I}) this way

00:57:48:22 I would do it that way (tnfo-.fragment {horizontal in the :acttutty GENERA1E back} em in fact I :time (00574822 00575124) would em maybe do it :clescriptDr COMPARISON this way {horizontal in :subjclass DESIGN-CONCEPr the back with opening :subjects (back·horizontal back-horlzontal·2) on the outside} :medium VIDEO

:levels (CONCEPItJAL UNLABELED) :measures (2})

Transcript After: (00:57:5l:24) : em __ • and em er so we have that as an option here em there would be a standard ca=ier ....

5.5 Summary

In this chapter I have described the observation method for conducting experi

ments and analysis method for transforming protocol data into a classification

which can be quantitatively analyzed. In the next chapter I will describe the

results of the quantitative analysis and the understanding gained from the infor

mation handling study.

Information Handling Beha\ior of Designers During Conceptual Design: Three Experiments

Chapter 6

Understanding Information Handling Behavior:

Quantitative & Qualitative Results

/-------. ( Information Classification \ \ Fmmewo~ I ------_/

/-----"" ( Dedal Performance \ \. Results /

/-----"" ( Information Handling \ \. Fmmewo~ /

t t J---~ --, ------------------------------Analyze Analyze Ana !yze

I \~rb~~~f"ent ;. \ 9~~~F /. \ / 1990-1991 \ 1991-1992 \ 1992 1995\

Information Support by Observe Support by ..... _ .. ~~ Information Reuse Study Dedal - ..... ~~ Dedal Use Dedal Handling

Study


This chapter reports on the quantitative and qualitative results of the information handling behavior study. Results on information fragment duration (deltat), design information measure (dim), distribution of time & information in the information fragment attributes (informational activity, descriptor, subject-class, level-ofabstraction), state transition analysis and multiple attribute analysis are reported. Implications of these results improve the understanding of the design process and the development of intuitive and integral information handling services.

[nfonnadon Handling Behavior of Designers During Conccprua/ Design: Three ExperimenlS 83

Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Results 84

6.1 Introduction

Analysis of protocol data is a long and arduous process. In this study it took an

average of about one hour to analyze one minute of raw data. This makes it

impractical to conduct a large number of experiments. However, the six experi

ments used in this study are a representative sample of design activity involving

an individual designer doing conceptual design. Therefore, although the results

may not generalize statistically they do raise our level of understanding about

the conceptual design process.

In this chapter, information fragment attributes (or simply attributes) will refer to

the categories used in classifying information fragments, which are: informational

activity, descriptor, subject-class and level-ofabstraction.l In the following sections I

will discuss results on information fragment duration (deltat), design informa

tion measure (dim), distribution of time & information in the attributes, state

transition analysis and multiple attribute analysis. At the end of each section the

gain in understanding about conceptual design and the implications for develop

ing information management too1s2 will be discussed.

Out of the six subjects the results for S33 differ considerably from that of the oth

ers. S3 appeared to be uncomfortable with the talking aloud protocol and had

little design experience. This could have contributed to the differences. How

ever, since the objective is not to generalize the results, I have chosen to include

the results from S3 in all the analysis. The results reported include contributions

from all six subjects. When reporting graphical results, in many cases results for

one subject chosen randomly will be shown in this chapter. Results for other sub

jects and some supporting data will be included in Appendix D.

1. Medium and level-of-detail are intentionally not included in this list. Results on medium are not considered since the nature of protocol experiment methodology biases the data towards the audio medium. Results on level-of-detar1 are not considered since it has been replaced by level-of-abstraction. 2. From the information management viewpoint the functionality of interest is similar to what Dedal was designed for, such as: information capture, information structuring (indexing) and information retrieval. 3. The six experimental subjects in this study will be referred to as 51-56 respectively.



6.2 Information Fragment Duration (deitat)

An information fragment is the unit of measurement in this study as described in

section 5.2.1 and 5.2.2. The duration of an information fragment, to be referred to

by deltat, is the difference between the end-time and the start-time for an infor

mation fragment. deltat is a measure of how frequently designers move between

different kinds of design information 1, and therefore informs us regarding the

dynamics of information handling.

6.2.1 deltat Variation over Time

Figure 6-1 shows the variation of deltat over time for 51. This graph for other sub

jects is similar in characteristics. Altogether, they show that designers

frequently and fluidly change the type of information they are handling, spend

ing between 2 to 35 seconds with one kind of information. The smooth curve in

Figure 6-1 is a polynomial fit to the raw data. It illustrates movements between

small and large information fragments at a level higher than the raw data. It

shows that the behavior is oscillatory, i.e. deltat is neither steadily increasing nor

decreasing.

6.2.2 Statistics on deltat

Table 6-1 shows the statistics on deltat for each of the subject. It lists the design

activity time and the number of information fragments handled by each of the

subject. A total of 1605 information fragments were extracted from 347 minutes

of data. The average deltat ranged from 11 to 19 seconds and the average over all

subjects was about 13 seconds. The distribution of deltat is shown in Figure 6-2.

The figure shows that the mode (most frequent) of deltat is about 6 seconds. The

variation in the mode across the subjects is small.

1. By definition of information fragment, when designers move from one fragment to another they change the information type (section 52.1).

Information Handling Behavior of Ocsigners During Conccplual Design: Three ExpcrimcnlS

Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Resuffs

- 30 rn f\ "'0 c: 0 (,) Q) rn -c: 0 ;: ~ :::J 20 C ~ -c: Q)

E C)

~ u. c:

"'" ~~ ~

\ j ~ ~ ,

~ 11:1 th ~

0 10 ;: as E

~~ .... .E c:

30 50 70 TIme (minutes)

Figure 6-1 Information fragment duration vs. time for 81. Note that deltat ranges from about 2 seconds to 35 seconds. The thick curve is a polynomial fit to the raw data showing the movements between short and long information fragments at a higher level than shown by the raw data. It shows an oscillatory behavior.

6.2.3 Discussion on deltat Results

86

I

!It:.t ~

90

The results on deltat inform us that designers move from one type of design

information to another on an average every 13 seconds. Most frequently they

spend about 6 seconds with anyone kind of information. These numbers pro

vide us with an understanding of the pace at which information handling is

taking place during conceptual design. They also explain why paper is the

medium for most conceptual work, as it poses no hindrance to designers in mov

ing from one information type to another. As was the case during the

experiments, using paper as a working medium, designers worked fluidly and


Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Resufts

-S1 -52 -53 ·---·54 - - 55 -S6

~--"'~-:"': ... : ... : ... '" ...

00~--~--~10~--~-~2~0~--~~==~3~0~~~ Duration of Information Fragment (deltat seconds)

Figure 6-2 Distribution of information fragment duration. The smooth curves shown here are 10 degree polynomial fit to histograms with bin width of 0.5. For clarity sake, the histograms are not shown here, they are shown in Appendix D.

87

Table 6-1 Information fragment duration (delta~ statistics. The design activity time listed below only includes the sum of the time for all information fragments and hence is less than the total design time listed in Table 5-1. Average deltat=(design time)/(# of info-fragments). Most frequent deltat is read from Figure 6-2 (the pOint of maxima on the curves).

Design1iJDe Number oCinfo- Average dellat Most Crequent Subject (minutes) fragments (seconds) dellat (seconds)

Sl 67.5 323 13 5 S2 49.9 262 11 6

S3 62.9 200 19 6

S4 44.7 220 12 5

S5 42.4 190 13 5

S6 79.6 410 12 6

TotallAvg. 347.0 1605 13 6

with ease. Akin [1978] also found that designers shift attention among different

design issues frequently, and that the design issue being attended to and the rep

resentation used are related.

Being able to work fluidly and with ease is essential during the conceptual

Information Handling Behavior of Designers During Conceptual Design: Three Experimcnl~

Chapter 6: Understanding Information Handling Behavior. Quantitative & Qualitative Results 88

design process. A service aiming to provide support for information manage

ment during the conceptual process should allow designers to work at a pace

suggested by the results on deltat. Next generation design tools should realize

this and build interfaces which are responsive and adapt to the designers pace.

A measure of deltat for emerging media may be useful index for estimating their

power or improvement.

6.3 Design Information Measure (dim)

The quantitative measure for the design information used in this study is called

design information measure. The unit of this measure is dim. Dim has been

defined in section 5.3.3. The knowledge of how much information designers han

dle will provide an insight into the burden faced by designers in managing this

information. Table 6-2 lists in column 2 & 3 the design activity time and the num

ber of dims handled by each of the subjects. A total of 2802 dims were handled in

347 minutes.

6.3.1 Information Handling Rate

The ratio of the amount of information handled (in dims) to the design time (in

minutes) is being called the information handling rate to be measured in dims

per minute (dpm). Dim rate can be calculated globally (total number of dims

divided by total design time) as well as locally for each information fragment

(number of dims in the info-fragment divided by deltat for the information frag

ment). Since the number of dims and deltat vary with the information fragments,

so does the local dim rate. Figure 6-3 shows the variation of dim rate over time

for 54. This graph for other subjects is similar in character. They show that the

local dim rate ranges from 2 to 40 dpm. This means that, based on the definition

of dim, the designers handle anywhere between 2 to 40 [descriptor, subject-class]

pairs in one minute. Table 6-2 shows, in column 4 & 5 the global dim rate and

the average local dim rate for each of the subjects.

Information Handling Behavior of Designers During Conceptual Design: Three Experimenl~

Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Resufts

E % -.cg

40

30

CIS 20 a: E c

20

o o

~~ ... I ,~ U

20

~ CI

~ I ~ ~ 1 ~ ~

" IIUIII If II V

40 60 80 TIme (minutes)

Figure 6-3 Information handling rate over time for 54. The dim rate ranges between 2 and 40 dpm. The thick curve is a polynomial fit and illustrates the variation in dim rate at a level higher than the raw data.

89

The global dim rate ranged from 6.2 to 9.2 dpm (average being 8.1 dpm) and the

average of the local dim rate ranged from 10.5 to 13.8 dpm (average being 12.6

dpm)l. The small range on the observed dim rate is an encouraging sign for the

definition of the dim measure. It suggests that the dim rate is measuring some

thing that is within a small range for most humans. It is a hypothesis that it most

closely relates to the information processing ability of humans. The dim rate

may correlate with some of the results from studies in cognitive psychology on

human information processing. However, before these correlations are investi

gated further it is important to measure the correlation with some obvious

factors.

1. The global average and the average of the local averages differ since the duration of information fragments is a variable. If all deltat were the same, i.e. the segmentation process fragmented the data at equal time intervals, than the global and the local averages would be equal.

lnfonnalion Handling Behavior of Designers During Conceptual Design: Three Experiments

Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Results

Table 6-2 Information handling rates and verbal rates for all subjects. dim = design information measure; dpm = dims per minute; wpm = words per minute; Dt = total design activity time in minutes; N =total number of dims handled; Global Dim Rate = N/Dt dpm; Global Verbal Rate = (Total words spoken)! Dt wpm; Cr = Coefficient of correlation between local dim rate and local verbal rate; Cd=Coefficient of determination

Dim Rate (dpm) Verbal rate (wpm)

Subject Dt(min) N(dim) Global Iocalav. Global Iocalav. Cr Sl 67.5 572 8.5 13.3 130.0 15l.2 0.51

S2 49.9 406 8.1 U.8 97.7 122.4 0.49

S3 62.9 387 6.2 10.5 77.6 104.4 0.88

S4 44.7 411 9.2 14.3 88.5 116.4 0.7

S5 42.4 337 8.0 11.6 91.1 105.6 0.52

S6 79.6 689 8.7 13.8 117.3 141.6 0.55

TotallAvg. 347.0 2802 8.1 12.55 100.4 123.6 0.69

90

Cd

0.26

0.24

0.77

0.49

0.27

0.30

0.39

Besides the information processing ability of humans, the dim rate can be a func

tion of variables such as the verbal rate (measured in words per minute, wpm),

design experience, design problem and possibly others. There certainly is some

correlation between the dim rate and the verbal rate of subject, since in the limit

ing case when there is no verbalization the dim rate would be zero since there

would be no basis to measure dim. Also humans talk at a rate within a range

when they need to be comprehended by others. This would suggest that the ver

bal rate would be within a small range. In that case, is the dim rate merely

measuring the verbal rate of the subjects?

Regression analysis was performed between dim rate and verbal rate to deter

mine the extent of correlation between them. Table 6-2 lists the global verbal rate

and average local verbal rate for each of the subject. It also lists the coefficient of

correlation! and the coefficient of determination2 between the local dim rate and

local verbal rate. On an average the coefficient of correlation was about 0.7 and

1. Coefficient of correlation (Cf ) provides the extent to which two variables are related [Runyon 1988. page 166]. Details on how Cf is calculated is given in Appendix D.

2. Coefficient of determination (CdJ provides an estimate on the proportion of variance in one variable due to the other. It indicates the proportion of total variation that is explained by the coefficient of correlation [Runyon 1988. page 210]. Details on how Cd is calculated is given in Appendix D.


Chapter 6: Understanding Information Handling Behavior. Quantitative & Qualitative Resuns 91

the coefficient of determination was about 0.4. This means that about 40% of the

variation in dim rate is accounted for by the variation in verbal rate. This level of

correlation is consistent with expectations.

6.3.2 Discussion on Design Information Measure Results

We now understand that designers in the experiments handled anywhere

between 2 to 40 [descriptor, subject-class] pairs in one minute. The average infor

mation handling rate was about 13 dims per minute (dpm). As was expected, the

information handling rate is correlated well with the verbal rate. However, ver

bal rate only accounts for about 40% of the variation in the dim rate. It is a

hypothesis that a share of the remaining contribution to dim rate can be corre

lated with the human information processing abilities [reference1. This is

suggested as possible future work.

Design tool creators can use the information about dim rate to design interfaces

which enable designers to deal with an adequate amount of design information

in a given time. A service should support information generation, access and

analysis at a pace suggested by the observed range on information handling rate.

6.4 .Distribution of Time and Design Information

Knowledge of deltat and dim rate gives an insight into the dynamics of informa

tion handling. However, knowledge of time spent and design information

handled among the different values of the information fragment attributes (the

values are listed in Table 5-2), provides an understanding of where designers

spend the major share of their time and information. This understanding can be

used to focus the development of support services.

6.4.1 Informational Activities Distribution

Figure 6-4 shows the distribution of time and design information across the



informational activities. It shows that subjects spent 55% of their time generating

information, 24% of their time accessing information and 21% analyzing informa

tion. Also, 53% of the design information was generated, while 25% was

accessed and 22% was analyzed. This suggests that conceptual design is mainly

a generative activity. The relative dominance of generative activity indicates that

unless support services can keep pace with generation of new information, they

will fail to support the conceptual design process.

Analyze

Access

Generate

o

"Ndim Dnme

20 40 Proportion (%)

55

60

Figure 6-4 Distribution of time and design information across informational activities. Ndim is the number of dims. More than half the time and information is dealt with in generate, suggesting that conceptual design is generation intensive.

6.4.2 Information Descriptor Distribution

Figure 6-5 shows the distribution of time and design information across the

information descriptors. Most of the time (65%) is shared by four descriptors:

construction (22%), alternative (15.5%), operation (15.1%), and requirement (12%).

Also, about 60% of the total information handled belonged to these four descrip

tors. This suggests that developers of information handling services should

identify the characteristics of these descriptors in the context of computational

support (representations, media and interfaces) and give adequate attention to


Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative ResuUs

Miscellaneous •• ..,

Requirement .iiiiii ••••• L---, Relation .iiiiiiiiiiiiii------J

Rationale ~iiiiiiiii. Performance .iii •••••

Operation .iii ••• i ••••••••• Location •••• ii~------.....J

~Ndim c::J TIme

Construction ••••••••••••••••• ---,

Comparison __ •• ijil------------~

Assumption .iiii~ Altemative ••••••••••• ---.

10 20

Proportion (%)

Figure 6-5 Distribution of time and design information across descriptor. Ndim is the number of dims. Major share of time and design information is grouped under the categories: construction, alternative, operation and requirement. Representations, media and interfaces which facilitate handling of these information types would cover a significant share of conceptual activity.

93

30

integrating these characteristics. For instance information about operation is often

best captured and described by a visual medium (3-D geometric model or video)

and information about alternatives by a decision tree or hypertext tool. Identify

ing a collection of such characteristics so that they can be made available to

designers in an integrated seamless fashion will go a long way in supporting

information handling.

6.4.3 Information Subject-Class Distribution

Figure 6-6 shows the distribution of time and design information across the sub

ject-classes. A major share of the time is spent in handling information associated

Information Handling Behavior of DesignetS During Conceptual Design: Three Experiments

Chapter 6: Understanding Information Handling Behawor: Quantitative & Qualitative Resufts 94

with assembly (29%). The distribution is quite even among component (17.4%),

design-concept (15.7%) andfeature (13.8%). About 75% of the information handled

belonged to these four subject-classes. Again information handling tools should

give adequate attention to representations, media and interfaces which make

handling information belonging to these subject-classes easier.

Other.--..,

Design-Concept •••••••• L-,

Requirement ••••••

Feature ••••••• ...,

Connection •••••••

"Ndim o Time

component ••••••••••••

Assembly ••••••••••••••• ...,

o 10 20

Proportion (%) 30

Figure 6-6 Distribution of time and design information across subject-class. Ndim is the number of dims. Assembly, component, design-concepts and feature account for a large share of time and information.

6.4.4 Information Level of Abstraction Distribution

Figure 6-7 shows the distribution of time and design information across the level

of-abstraction. All levels except quantitative are well represented. The distribu

tion of time is unlabeled (21%), labeled (28%), associative (22%), qualitative (22%)

and quantitative (7%). We know that computational tools are very good at repre

senting and manipulating quantitative information. However, 93% of the time

and 94% of the design information during conceptual design is dealt with in non

quantitative levels of abstraction. Over the past few years there has been

progress in dealing with associative and qualitative information [references].

However, support services are yet to become adept at manipulating unlabeled



Quantitative

Qualitative

Associative

Labeled

Unlabeled

Proportion (%)

"Ndim c:::J TIme

Figure 6-7 Distribution of time and information across level-of-abstraction. Ndim is the number of dims. The plot indicates that, most of the time and information is dealt with in non-quantitative levels of abstraction.

information and capturing its evolution over time.

95

Since level-of-abstraction can be ordered by the extent of detail represented

(abstract to concrete), it is valuable to look at the behavior of level-of-abstraction

over time. Figure 6-8 shows the movement of S4 between the levels of abstrac

tion over time. The graphs for other subjects are similar in characteristics.

They show that designers move from one level to another often without any

apparent pattern. It was also observed that in most cases a new concept started

out as unlabeled and matured to other levels, although not in any particular

order. This means that support for easily transitioning between the different

level-of-abstraction is essential to supporting the conceptual design process.

6.4.5 Discussion on Time and Information Distribution

The distribution of time and design information in each of the information frag-

Infonnation Handling Behavior of Designers During Conceptual Design: Three ExperimenlS


Quantitative

Qualitative

Associative

Labeled

Unlabeled

o 10 20 30 40 50 60 70

Time (minutes)

Figure 6-8 Variation in level-at-abstraction over time tor 84. It was observed that concepts often started out at unlabeled level and matured to other levels as they became more concrete. Movement between the different levels happens often. However no pattern is obvious.

96

80

ment attributes, informs us of the types of design information which are

frequently dealt with during conceptual design. This information can be used to

prioritize the functionalities of support services and in selecting representations,

media and interfaces for manipulating information. The challenge lies in identi

fying the characteristics of services to support the behavior observed, and

providing simple and well integrated mechanisms for moving between these

behaviors.

6.5 State Transition Analysis: Statistics and Probabilities

In the previous sections we have developed an understanding of how frequently

designers transition from one information type to another. To further this under

standing, it is important to look deeper at the points of transition. Points of

transition can be defined as the points where the designer changes from one



state! to another for any of the information fragment attributes. State transition

statistics2 will inform us about transitions which are more frequent and there

fore more critical from the viewpoint of developing support. On the other hand

state transition probabilities3 inform us about which transition is most likely

from a given state and can be used in developing adaptive interfaces. In this sec

tion results on state transition statistics will be reported. Results on state

transition probabilities are included in Appendix D. fu the figures below the

arrows represent a transition from and to the states connected by the arrow.

6.5.1 Informational Activity State Transition Analysis

Figure 6-9 shows the state transition statistics for informational activities. With

three distinct states, there are six possible combinations for transitions. Transi

tions between generate and analyze make up 51% and transitions between

generate and access make up 31%, of all the informational activity transitions.

Clearly a tool which makes it easier to move to and from generate will be better

at supporting information handling.

6.5.2 Information Descriptor State Transition Analysis

Figure 6-10 shows the state transition statistics for information descriptor. There are 11

distinct states, therefore 110 types of transitions are possible. For clarity sake, only the

frequent transitions are shown in Figure 6-10. It is evident that transitions to and from

construction are more common than others and altogether they make up 29% of the total

descriptor transitions. Transitions to and from alternative (11 %), operation (9%) and

1. State refers to a particular instance of each of the information fragment attributes as listed in Table 5-2. 2. State transition statistics are calculated by counting the number of transitions of a particular type. For comparison purposes results reported are normalized. So results for each transition type is reported as a ratio (expressed in percentage) of the number of the transition type to the total number of transitions observed. 3. State transition probabilities are calculated under the assumption that a next state is determined by only the current state and not any states prior to it (i.e. the memory of anything but one previous state is erased. This assumption is commonly made in developing Markov chain for a process with discrete states. Such an assumption is valid for a first order analysis.



8

Figure 6-9 Informational activity state transition statistics. All six possible transitions are shown. Transitions to and from generate make up 82% of all the observed transitions.

98

relation (6%) are also significant. An information handling tool should make it easier to

transition to and form construction, alternative, operation and relation type information.

Most transitions between any two states are between construction and alternative (6.6%).

Figure 6-10 Information descriptor state transition statistics Only the most frequent transitions of the 110 possible combinations are shown here. Transitions to and from construction, alternative, operation and relation make up bulk of the transitions. Transition between construction and alternative are most likely {6.6%}.


Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Resufts 99

6.5.3 Information Subject-Class State Transition Analysis

Figure 6-11 shows the state transition statistics for infonnation subject-class. There are

six distinct states, therefore 30 types of transitions are possible. Once again for clarity

sake only the frequent transitions are shown in Figure 6-11. Transitions to and from

assembly and component make up 54% of all the subject-class transitions. Transitions to

and from feature (19%) are also frequent. Most transitions between any two states are

between assembly and component (13.7%).

Figure 6-11 Information subject-class state transition statistics Only the most Significant of the 30 possible combinations are shown here. Transitions to and from assembly and component make up a large share of the total transitions. Transitions between assembly and component are most likely (13.7%).

6.5.4 Information Level-of-Abstraction State Transition Analysis

Figure 6-12 shows the state transition statistics for information level-of-abstrac

tion. There are 20 types of transitions possible for the five distinct states. Top 10

frequent transitions are shown in Figure 6-12. Transitions among the states unla

beled, labeled, associative and qualitative make up 93.5% of all the level-of

abstraction transitions. The frequency of transitions to and from each of the

states is unlabeled (25%), labeled (53%), associative (34%), qualitative (37%) and quan

titative (6.5%). A critical need for easy transitions between non-quantitative levels

Information Handling Behavior of Designers During ConceplUaI Design: Three ExperimenlS


of abstraction is evident. Most transitions between any two states are between

labeled and associative (19.5%).

8.1

Figure 6-12 Information level-of-abstraction state transition statistics Only the most significant transitions of the 20 possible combinations are shown here. Transitions to and from unlabeled, labeled, associative and qualitative make up a large share of the transitions.

6.5.5 Discussion on State transition Analysis

From this section and the previous sections we have come to understand that fre

quent transitions between different information types is fundamental to the

conceptual design process. State transition statistics deepen our knowledge of

the designers movements between different states for each of the information

fragment attributes. They identify transitions which frequently take place and

therefore are critical to support.

From the viewpoint of most tools and services, mechanisms for supporting tran

sitions are the common places contributing to difficulty in usability. Since most

computation tools are syntax driven a user has to explicitly specify a transition.

For instance if a user wants to draw a circle in the middle of drawing a line, he

or she has to select circle from the menu, then draw the circle and select line to

come back to drawing a line. Paper is popular in conceptual design since it offers

no hindrance in making such transitions. It is anticipated that the knowledge of

Information Handling Behavior of Designers During Conceprual De.ign: Three Experiments


the state transition statistics can be used in developing well integrated services,

which support the frequent transitions well and therefore will be user friendly.

6.6 Multiple Attributes Analysis

The behavior of one attribute at a time informed us about the relative impor

tance of different values of the attributes. However, deeper insight can be gained

by looking at behavior over multiple (two or more) attributes at a time. First

results over two attributes combined together are described followed by results

over all four attributes. Because of space considerations, only some of the combi

nations are discussed here. Appendix D has results for some of the other

combinations.

6.6.1 Informational Activities and Information Descriptor

Table 6-3 shows the distribution of informational activities across the informa

tion descriptors. The information generated most often was of descriptor

alternative (19%) and construction (17%), information accessed most often was of

descriptor construction (25%) and requirement (20%), and information analyzed

most often was of descriptor construction (26%) and performance (21%). Looking

at these combinations one can recommend that a successful support service

should make it easy to generate [alternative, construction] information, access [con

struction, requirement] information and analyze [construction, performance]

information.

Table 6-3 Distribution of information activities across deSCriptors. The numbers below show the proportion of the different information descriptor in each of the informational activities. Two of the most significant combinations for each informational activity are highlighted.

Assn Comp Loca Oper Perf Rati Rela Requ Mise

Gen 6 6 7 12 3 10 7 7 4

Ace 8 3 2 8 11 11 2 3 5

Ana 13 2 4 3 13 0 4 2

Information Handling Behavior of Designers During ConceptUllI Design: Three Experiments


6.6.2 Informational Activities and Information Subject-Class

Table 6-4 shows the distribution of informational activity across the different

subject-classes. It can be observed that most of the information generated was

about component (24%), assembly (17%), design-concept (17%) and feature (16%),

most information accessed and analyzed was about assembly (41% and 26%

respectively). Thus an information handling support service should make it easy

to generate [component, assembly, design-concept, feature] information, access assem

bly information and analyze assembly information.

Table 6-4 Distribution of informational activity across subject-class. The numbers below represent the proportion of the different subject-classes in each of the informational activity. The most frequent combination for each informational activity is highlighted.

Activity Conn. Feature Reqmt. D-conc. Other

Generate 13 16 8 17 5

Access 8 12 13 10 2

Analyze 10 17 14 16 2

6.6.3 Informational Activities and Level-of-Abstraction

Table 6-5 shows the distribution of informational activities across the informa

tion level-of-abstraction. Information is well distributed across all but

quantitative level-of-abstraction. Therefore a support service for conceptual

design should make it easy to generate, access and analyze information at unla

beled, labeled, associative and qualitative level-of-abstraction.

Table 6-5 Distribution of informational activities across level-of-abstraction. The numbers below represent the proportion of the different level-of-abstraction in each of the informational activity. The most frequent combinations for each informational activity are highlighted. It is clear that much of the concentration is in non-quantitative levels.

Activity

Generate

Access

Analyze

Infonnalion Handling Behavior of Designel'5 During Conceptual Design: Three ExperimenlS

Quantitative

3

10

8


6.6.4 Most Frequent Information Fragments

Table 6-6 shows the top nine most frequently encountered combination of the

information fragment attributes. The listing is in the order of the time spent in

each of the combinations. The table also lists the number of dims handled and

the number of information fragments in each of the combinations. About 4.4% of

the total time and 4.8% of the design information is handled in generating qualita

tive operations of an assembly. However the most number of information

fragments were in generating unlabeled alternative design-concepts. Looking at com

binations in this manner can be used in prioritizing the functionalities of support

services.

Table 6-6 Frequently encountered info-fragment attribute combinations. The number of fragments, time and the number of dims of information in each combination are shown along with their proportion form the total.

TIme ndim # of fragments Info- Level of

Activity Descriptor Subject-clas§ A ...... mon* %* % # %

Generate Operation Assembly Qualitative 4.4 4.8 44 2.7

Generate Alternative Design-concept Unlabeled 12.9 3.7 61 2.2 3.2

Access Requirement Assembly Labeled 7.4 2.1 59 2.1 25 1.6 Access Requirement Design-concept Unlabeled 6.1 1.8 35 1.2 32 2.0

Generate Relation Component Associative 5.1 1.5 62 2.2 27 1.7 Generate Operation Component Qualitative 4.6 1.3 51 1.8 26 1.6 Generate Construction Component Labeled 3.8 1.1 31 l.l 22 1.4 Generate Construction Component Associative 3.6 1.0 40 1.4 20 1.2 Access Performance RI Labeled 3.5 l.0 40 1.4 21 1.3

6.7 Summary

Table 6-7 summarizes the quantitative and qualitative findings presented in this

chapter. Altogether this raises our understanding of the information handling

behavior of designers during conceptual design tasks. This understanding has

made it possible to formulate some recommendations for developing successful

information handling support services.



Table 6-7 Summary of the key findings and recommendations. IA=lnformational activity; D=Descriptor; SC=Subject-Class; loA=level of abstraction ditn=design information measure; dptn=dims per minute;

Results on: Observation(s) RecoDUDendation~)

deltat average=13 sec; mode=6 sec; useful index for estimating power of range={2 to 35} seconds support

Dim Rate about 8 [D.SC] pairs per minute another measure to estimate power of mode=6 dpm; range={2 to 40} dpm support

Distribution IA generate (53%) emphasis on ease of generating new of dims information

D construction. alternative. operation emphasis on ease of handling these (51%) types of information

SC assembly, component. design-con- emphasis on ease of handling these cept(62%) types of information

LoA all levels except quantitative are well emphasis on ease of handling all lev-represented els of abstraction

State Transi- IA 82% of all transitions are to and from should be easy to move to and from tion Analysis generate generate

D construction (29%). alternative movement to and from these should (11 %). operation (9%) be the easiest

SC assembly and component (54%) movement to and from these should be the easiest

LoA non-quantitative levels (96%) easy transitions between non-quantita-tive levels

Multiple Attribute Most information is generated about Brings out combinations which pro-Analysis qualitative operation of an assembly vide recommendations at a deeper

and similar observations level

Information Handling Behavior of Designers During Conceprual Design: Three ExperimenlS

Chapter 7

Conclusions & Future Work

Design Research

TO

,r Develop Research

Methods

< Goal > shortest path

T~ Understand Design Practice

~~

Deliver High Quality, Low Cost Products

~

TO

. Improve Design

Practice

~o

This chapter revisits the salient points of this thesis. It presents concluding remarks on the information reuse study, the information handling study, and deployment of Dedal. It states the contributions that this thesis is making to the field of design research and presents some opportunities for further research.


Chapter 7: Conclusions & Future Work 106

7.1 Concluding Remarks

This thesis demonstrates a methodology for incrementally understanding and

supporting the conceptual design process. It does so by reporting on two obser

vational studies and one information management service. It shows that the

verbal protocol method is good for studying individual designers and describes

a procedure for analyzing protocol data to extract understanding of the design

activity and requirements for support services.

By developing and deploying Dedal, this ~sertation demonstrates how require

ments emerging from a protocol study can be used to develop successful

support services. It also presents an information handling framework for studying

the information handling behavior of designers. Using this framework it

describes the gain in understanding about the information handling behavior.

This dissertation has found that the ability to transition frequently between dif

ferent types of information is fundamental to the conceptual design process.

Information handling services available today are mainly syntax driven 1 and fail

to support transitions effectively. In such services, a transition has to be explic

itly specified by the designer, which results in breaking his or her rhythm. On

the other hand, paper presents no hindrance in making transitions, and there

fore is used extensively during conceptual design. This suggests that there is a

need to develop well integrated representations and interfaces which allow the

designer to transition easily between different information types. I hope that the

quantitative measures presented in this dissertation will be used to evaluate ser

vices in their ability to support smooth transitions.

Design is an information intensive activity. The rewards for capturing and struc

turing information during a design process for efficient reuse during or after the

1. An example of syntax driven is: if one wants to change from drawing a line to drawing a circle, he/she has to explicitly pick circle from a menu. When using paper no such change is needed.



design process are well known. The work reported in this dissertation, my own

design experience and collaboration with other researchers at the Center for

Design Research [Leifer 1990; Lakin et al. 1989; Mabogunje 19931 has resulted in

the following realizations:

• design information capture and structuring should happen during the design process as opposed to after the process,

• design information capture and structuring should be a natural extension of the design process instead of being an add on activity, and

• the rewards of doing information capture and structuring should be available as a natural feedback to the designers doing the capture and structuring, as opposed to only the next generation of designers or re-designers.

7.2 Contributions

This dissertation adds knowledge to the field of engineering design research. It

corroborates verbal protocol approach as a valuable method for studying, under

standing and incrementally improving engineering design activity.

Additionally, this dissertation makes three key contributions, it

• demonstrates a framework for characterizing the information handling behavior of designers. This framework effectively classifies information that should be captured during the design process for effective reuse,

• demonstrates a computational implementation of the design information framework that is intuitive for designE'fs and works in real time, and

• rigorously describes the information handling behavior during conceptual design using quantitative measures and makes recommendations on how these measures can be incorporated in building successful services to support information handling.

7.3 Discussion

The complexity of the design process presents many challenges in understand

ing and supporting it. Although this dissertation shows that protocol studies

have the potential for providing a detail and rich understanding, it also illus

trates that the process of conducting the experiments and analyzing the protocol



data is a long, tedious and a subjective process. As a result, it is often not practi

cal to conduct experiments on a statistically large sample for each of the

scenarios and perspectives that are observed in a design process. However,

because of the success of protocol studies the interest in using this approach is

growing. Therefore, like in any new research field, the design research commu

nity faces the challenge of establishing mechanisms for validation and

comparing results across researchers. This will enable new research to build on

top of results already available.

While observational studies provide a rich understanding and lead to require

ments for support services, embodying these requirements in a usable

computational implementation is a challenge and probably a subject of research

as well. I found being involved in real design during the course of conducting

this research very empowering in shaping my judgement and my ability to pick

representations, media and interfaces for computational implementations.

7.4 Future Work

This dissertation raises numerous opportu.'1ities for further research. A natural

extension, in the interest of the iterative methodology presented, would be to

use the recommendations stated in the Chapter 6 to develop support services for

information management. These services could be evaluated using the observa

tional methodology presented in this dissertation, to further refine and improve

information handling support. Few other directions for further research are:

• study information handling in the other phases of the design process, so that support for information management can be continued all along the design process

• investigate correlation between design information measure and cognitive models of human information processing

• investigate relationship between information media and information handling behavior



• use the iterative methodology to study, understand and support the design process from a viewpoint other then information handling behavior.

• use the information handling framework to analyze information handling behavior in other design scenarios such as team design

The work presented in this dissertation is only a beginning towards studying,

understanding and supporting information handling behavior during concep

tual design. I hope that the recommendations made in this dissertation will be

useful in building support services. I also hope that the frameworks and the

methodology reported will be employed to study other design scenarios and

other phases of the design process.


Bibliography 110

Bibliography

Akin Orner, ''How do Architects Design?", Artificial Intelligence and Pattern Recognition in Computer Aided Design, J. Latombe (Ed.), IFIP, North-Holland Publishing Company, 1978.

Akin, Orner, /I An Exploration of the Design Process", Design Methods and Theories, Vol. 13, No. 3-4, pp 115-119, 1979. Also published in Developments in Design Methodology, Nigel Cross (Ed.), Chichester, John Wiley & Sons, pp 33-56, 1984.

Alexander, Christopher, Notes on the Synthesis of Form, Harvard University Press, Cambridge, MA, 1964.

Antonsson, Erik K., "Development and Testing of Hypothesis in Engineering Design Research", Journal of Mechanisms, Transmissions, and Automation in Design, Vol. 109, pp. 153-154,1987.

Archer, L.B., "Systematic Methods for Designers", Design, Vols. 172-174, 176, 179, 181, 185, 188, April 1963 - August 1964. Also in Developments in Design Methodology, N. Cross (Ed.), Wiley, Chichester, 1984.

Asimov, Morris, Introduction to Design, Prentice Hall, Engelwood Cliffs, NJ, 1962.

Babbie, Earl R.,"The Practice of Social Research", Wadsworth Publishing Co., Belmont, CA, Fourth Edition, 1986.

Baudin, Catherine; Underwood, Jody G. and Baya, Vinod, "Using Device Models to Facilitate the Retrieval of Multimedia Design Information", In Proceedings of the 13th International Joint Conference on Artificial Intelligence, Chambery, France, pp. 1237-1243, August 29-September 2,1993a.

Baudin, Catherine; Kedar, Smadar; Underwood, Jody G. and Baya, Vinod; "Question-based Acquisition of Conceptual Indices for Multimedia Design Documentation", In Proceedings of the 11th National Conference on Artificial Intelligence AAAI-93, Washington D.C., pp 452-458, July 1993b.

Baya, Vinod; Stout, Greg and Vaska, Matt, "Continuously Variable Damper Project", Design Requirement Document, ME210 Fall, Winter and Spring quarter 1989-90, Technical Report, Design Division, Department of Mechanical Engineering, Stanford University, 1990.

Baya, Vinod; Gevins, Jody; Baudin, Catherine; Mabogunje, Ade; Toye, George and Leifer, Larry, "An Experimental Study of Design Information Reuse", In Proceedings of the 4th International Conference on Design Theory and Methodology,


Bibliography 111

ASME, Scottsdale, Arizona, pp.141-147, Sept. 13-16, 1992.

Baya, Vinod and Leifer, Larry J., II A Study of the Information Handling Behavior of Designers During Conceptual Design", In Proceedings of the 6th International Conference on Design Theory and Methodology, ASME, Minneapolis, Minnesota, pp. 153-160, Sept. 11-14, 1994a.

Baya, Vinod and Leifer, Larry J., ''Understanding Information Management in Conceptual Design", Chapter in Analysing Design Activity, Cross, N.; Christiaans, H. and Dorst, K. (Eds.), John Wiley & Sons, pp. 151-168, 1996.

Baya, Vinod and Leifer, Larry J., ''Understanding Design Information Handling Behavior Using Time and Information Measure", In Proceedings of the 9th International Conference on Design Theory and Methodology, ASME, Boston, Massachusetts, DE-Vol 83, Volume 2, pp. 555-562, Sept. 17-20, 1995.

Bemers-Lee, T.J. and Cailliau, R. ''World-Wide Web", Computing in High Energy Physics 1992 Conference, Anneey, France, 23-27 September 1992.

Blair, D.C. and Maron, M.E., "An Evaluation of Retrieval Effectiveness for a FuUText Document Retrieval System", Communications of ACM, 28,3, pp. 289-299, March 1985.

Blessing, L.T.M., IIA Process-Based Approach to Computer Supported Engineering Design", In Proceedings of the International Conference on Engineering Design (lCED-93), The Hague, pp.1393-1400, August 1993.

Chovan, John D. and Waldron, Majula B., liThe Use of Function and Form When Reading Mechanical Engineering Drawings by Experts and Non-experts", In Proceedings of the International Conference on Design Theory and Methodology (DTM-90), Chicago, illinois, pp. 137-144, Sept. 1990.

Conklin, Jeff and Begeman, M. L., "gffiIS: A Hypertext Tool for Exploratory Policy Discussion", In Proceedings of the 1988 Conference on Computer Supported Cooperative Work, Portland, Oregon 1988.

Court, Andrew W.; Culley, Stephen J. and McMahon, Christopher A.; "The Information Requirements of Engineering Designers", In Proceedings of the International Conference on Engineering Design, The Hague, pp. 1708-1716, August 17-19,1993.

Cross, Nigel, Engineering Design Methods, John Wiley & Sons, July 1990.

Crutcher, Robert J., "Telling What We Know: The Use of Verbal Report MethodolOgies in Psychological Research", In Psychological Science, Vol. 5, No.5, pp. 241-244, Sept. 1994.

Dixon, John, R., "On Research Methodology Towards a Scientific Theory of


Bibliography 112

Engineering Design", In Artificial Intelligence for Engineering Design, Analysis and Manufacturing (AIEDAM), Volume 1, Number 3, pp. 145-157, June 1988.

Dorst, Kees; Christiaans, H. and Cross, Nigel, (Eds.). Analyzing Design Activity, VViley,Clrichesrex,InK,1996.

Eastman, Olarles M., "On the Analysis of Intuitive Design Processes", Emerging Methods in Environmental Design and Planning, Gary T. Moore (Ed.), Cambridge, MIT Press, pp. 21-37, 1970.

Eder, Ernest, VV., ''Design Science - Meta-Science to Engineering Design", In Proceedings of the 2nd International Conference on Design Theory and Methodology, Chicago, lllinois, pp. 327-335, Sept. 1990.

Ericsson, K. A. and Simon, H. A., Protocol Analysis: Verbal Reports as Data, 2nd edition, Cambridge, MA, MIT Press, 1993.

Fulton, Robert E. and Craig, James, I., Report from a Workshop on "Information Framework Technology for Integrated Design/Engineering Systems", Sponsored by NSF and Georgia Institute of Technology, Callaway Gardens, Georgia, March 12-15,1989.

Greeno, James G., ''Understanding Concepts in Activity", Chapter in Discourse Comprehension, Edited by C.A. VVeaver, ill, S. Mannes and C.R. Fletcher, Lawrence Erlbaum Associates, Publishers, Hillsdale, NJ, pp. 65-95, 1995.

Gruber, Thomas; Baudin, Catherine; Boose, John and VVeber, Jay, "Design Rationale Capture as Knowledge Acquisition: Tradeoffs in the Design of interactive Tools", In Machine Learning: Proceedings of the Eighth International Workshop, San Mateo, CA, Birnbaum, L. and Collins, G (Ed.), Morgan Kaufman, pp.3-12,1991.

Guindon Raymonde, "Designing the Design Process: Exploiting Opportunistic Thoughts", Human Computer Interaction, Vol 5, pp. 305-344,1990.

Hales, Crispin, "Analysis of the Engineering Design Process in an Industrial Context", Ph.D. Thesis, Department of Engineering, University of Cambridge, 1987.

Hundal, M.S., "A Computer-Based Approach to Conceptual Design of Engineering Systems", In Preprints of NSF Engineering Design Research Conference, Amherst, Massachusetts, pp. 251-265, June 1989.

Kim, Steven, H., Designing Intelligence: A Framework for Smart Systems, Oxford University Press, 1990.

Kuffner, Tom A. and Ullman, David G., "The Information Requests of Mechanical Design Engineers", Design Studies, Vol. 12, No.1, pp. 42-50, 1991; also in Proceedings of 1990 Design Theory and Methodology Conference, ASME, pp.

Information Handling Behavior of Designers During Concepllllll Design: Thn:e Experiments

Bibliography 113

167-174, Chicago, September 1990.

Lakin, Fred, "Spatial Parsing for Visual Languages", Chapter in VISUAL LANGUAGES, edited by Shi-Kuo Chang, Tadao Ichikawa and Panos. A., Ugomenides Plenum Press, 233 Spring Street, New York, NY, 1986.

Lakin, Fred; Wambaugh, John; Leifer, Larry; Cannon, Dave and Sivard, Cecilia, "The Electronic Design Notebook: Performing Medium And Processing Medium", THE VISUAL COMPUTER: International Journal of Computer Graphics, edited by Makkuni, Ranjit, Springer Verlag publishers, 5(4),1989.

Leifer, Larry J., "Generation and Conservation of Design Knowledge", Original Proposal to the AI Branch, NASA Ames Research Center, Moffett Field, CA, 1989. Also Center for Design Research Technical Report CDR 1990.

Leifer, Larry J., "Concurrent Engineering Design and Management Knowledge Capture", Technical Report CDR-NASA-90-3, Center for Design Research, Stanford University, March 1990.

Leifer, Larry J., "Instrumenting the Design Process: For Real Time Text-Graphic Design Process Records", In Proceedings of the 1991 International Conference on Engineering Design, ICED-91, Zurich, pp. 314-321, August 27-29, 1991.

Mabogunje, Ade, "Harnessing Questions in Mechanical Design", Engineers Thesis, Department of Mechanical Engineering, Stanford University, July 1993.

Mauldin, Michael L.,"Retreival Performance in FERRET: A Conceptual Information Retrieval System", In Communications of ACM, 448,1, pp. 347-355, 1991.

Minneman, Scott L., "The Social Construction of a Technical Reality: Empirical Studies of Group Engineering Design", Ph.D. Thesis, Department of Mechanical Engineering, Stanford University, December 1991.

Minneman, Scott L. and Bly, Sara, "Experiences in the development of a Multiuser Drawing tool", Proceedings of the third Guelph Symposium on Computer Mediated Communication, Guelph, Ontario, pp. 154-167, May 1990.

National Materials Advisory Board, NMAB-455, "Enabling Technologies for Unified Life-Cycle Engineering of Structural Components", National Academy Press, Washington, D.C., 1991.

Newell, Allen, "On the Analysis of Human Problem Solving Protocols", Calcul et Formalization dans les Sciences de L 'Homme, J. c. Gardin and B. Joulin (Eds.), Paris, pp. 145-185, 1968.

Newell, Allen and Simon, Herbert A., Human Problem Solving, Englewoods Cliffs, NJ, Prentice Hall, 1972.

Infonnalion Handling Bchavior of Designcrs During Conceptual Design: Three Experiments

Bibliography 114

Newsome, Sandra L. and Spillers, William R., "Tools for Expert Designers: Supporting Conceptual Design", Design Theory '88, S. L. Newsome (Eds.), Springer-Verlag, pp. 49-55, 1989.

O'Shanghnessy, K. and Sturges, R. H. Jr., "A Systematic Approach to Conceptual Engineering Design", In Proceedings of the 4th International Conference on Design Theory and Methodology, ASME, Scottsdale, Arizona, pp. 283-291, Sept. 1992.

Pahl, G. and Beitz, W., Engineering Design, Design Council, London, 1984. (Original German version published in 1977)

Payne, John W., "Thinking Aloud: Insights Into Information Processing", In Psychological Science, Vol. 5, No.5, pp. 241-248, Sept. 1994.

Perlman, Gary, "Descriptive Models of Cognitive Aspects of the Engineering Design Process", Design Theory '88, S. L. Newsome (Eds.), Springer-Verlag, pp. 10-17, 1989.

Rabins, Michael; Ardayfio, David; Fenves, Steven; Seireg, Ali; Nadler, Gerald; Richardson, Herbert and Clark Howard, "Design Theory and Methodology: A New Discipline", Mechanical Engineering, pp. 23-27, August 1986.

Regan, Melissa and Sheppard, Sheri, "Interactive Multimedia Courseware and the Hands-on Learning Experience: An Assessment Study", Journal of Engineering Education, pp. 1-9, April 1996.

Roozenburg, N. and Cross, N., "Models of the Design Process - Integrating Across the Disciplines", In Proceedings of the International Conference on Engineering Design (lCED-91), Zurich pp. 186-193, August 1991.

Rouse, William B. and Boff, Kenneth R., "Designers, Tools and Environments: State of Knowledge, Unresolved Issues, and Potential Directions", System Design: Behavioral Perspectives on Designers, Tools, and Organizations, William B. Rouse and Kenneth R. Boff (Eds.), New York, North-Holland, pp. 43-63, 1987.

Runyon, Richard P. and Haber, Audrey, Fundamentals of Behavioral Statistics, Sixth edition, Random house, New York, 1988.

Schon, Donald A. and Bucciarelli, Louis L., "Design Theory and ~fethods - An Interdisciplinary Approach", Design Theory '88, S. L. Newsome (Eds.), SpringerVerlag, pp. 29-35, 1989.

Shannon, C.E. and Weaver, W, The Mathematical Theory of Communication, Urbana: University of lllinois, 1949.

Simon, Herbert A., The Sciences of the Artificial, 2nd Edition, MIT Press, Cambridge, Massachusetts, 1992 (originally published in 1969).


Bibliography 115

Sivard, Cecilia; Zweben, Monte; Cannon, David M.; Lakin, Fred; Leifer, Larry J. and Wambaugh, John, "Conservation of Design Knowledge", In Proceedings of the 1989 American Institute of Aeronautics and Astronautics, AJAA-89, Washington DC,1989.

Spillers, W.R. and Newsome, S.L.,"Engineering Design, Conceptual Design, and Design Theory: A Report", In Design Methodology and Relationships with Science, M. J. de Vries et al. (Eds.), Kluwer Academic Publisher, Netherlands, pp. 103-120,1993.

Stauffer, Larry A.," An Empirical Study on the Process of Mechanical Design"; Ph.D. thesis, Oregon State University, September 1987.

Stauffer, Larry A. and Ullman, David G.," A Comparison of The Results of Empirical Studies into the Mechanical Design Process", Design Studies, Vol 9, No 2, pp. 107-114, April 1988.

Stauffer, Larry A. and Ullman, David G.,"FundamentaI Processes of Mechanical Designers Based on Empirical Data", Journal of Engineering Design, Vol. 2, No.2, 1991.

Takeda, Hideaki; Hamada, Susumu; Tomiyama, Tetsuo and Yoshikawa, Hiroyuki, "A Cognitive Approach to the Analysis of Design Process", In Proceedings of the International Conference on Design Theory and Methodology (DTM-90), Chicago, lllinois, pp. 153-160, Sept. 1990.

Tang, John C. "Listing, Drawing, and Gesturing in Design: A Study of the Use of Shared Workspaces by Design Teams", Ph.D. thesis, Department of Mechanical Engineering, Stanford University, April 1989.

Tang, John, C. and Minneman, Scott L.,''VideoWhiteboard: Video Shadows to support remote collaboration", Proceedings of the Conference on Computer Human Interaction (CHI) '91, New Orleans, LA, pp. 315-322, April 28 - May 2, 1991.

Tong, Richard M.; Appelbaum, Lee A. and Askman, Victor N.,"A Knowledge Representation for Conceptual Information Retrieval", In International Journal of Intelligent Systems, Vol. 4, pp. 259-283, 1989.

Toye, George; Cutkosky, Mark R.; Leifer, Larry J.;Tenenbaum, J. M. and Glicksman, Jay, "SHARE: A Methodology and Environment for Collaborative Product Development", In Post-Proceedings of the IEEE Infrastructure for Collaborative Enterprise, 1993,

Ullman, David G.; Dietterich, Thomas G. and Stauffer, Larry A., "A Model of the Mechanical Design Process Based on Empirical Data", AI EDAM, 2(1), pp. 33-52, 1988.


Bibliography 116

Ullman, David G., IIDesign Histories: Archiving the Evolution of Products", In Proceedings of the DARPA WOrkshop on Manufacturing, Salt Lake City, Utah, February 5-6, 1991.

Valsiner, Jaan (Ed.), The Individual Subject and Scientific Psychology, Plenum Press, New York, 1986.

Waldron, Kenneth J. and Waldron, Manjula B.,"A time Sequence Study of Complex Mechanical System design", Design Studies, Vol. 9, No.2, April 1988, pp.95-106.

Waldron, M.B. and Waldron, K.J., ''Methods of Studying Mechanical Design", In Mechanical Design Theory and Methodology, Editors: Waldron, M.B. and Waldron, K.J., Springer-Verlag, New York, pp.21-34, 1996.

Wallace, K. M. and Hales, Crispin, "Detailed Analysis of an Engineering Design Project", In Proceedings of the 1987 International Conference on Engineering Design, ICED-87, Boston, MA, pp. 94-102, August 17-20,1987.

Wilson, Timothy D., "The Proper Protocol: Validity and Completeness of Verbal Reports", In Psychological Science, Vol. 5, No.5, pp. 249-252, Sept. 1994.

Wood, Kristin L. and Antonsson, Erik K., "Engineering Design - Computational Tools in the Synthesis Domain", The Study of the Design Process: In Proceedings of NSF Workshop on Design Theory and Methodology, Oakland, CA pp. 337-360, February 1987.

Information Handling Behavior of Designers During ConcePlual Design: Three ExperimcnlS

Appendix A: Additional Material on Design Information Reuse Study 117

Appendix A

Additional Material on Design Information Reuse

Study

A.I Design Experiment Material

• A.I.I Design Experiment Problem Statement

• A.I.2 Description of ME2IO

A.2 Results on individual subjects

A.3 Analysis Examples

A.1 Design Experiment Material

Enclosed is the design problem and a description of first generation design team (ME2IO), used in conducting the information reuse study experiments.

A.l.l Design Experiment Problem Statement


Appendix A: Additional Material on Design Information Reuse Study

Generation & Conservation of Design Knowledge CDR-Stanford University, NASA-ARC,

Information Reuse Experiment

Programmable Continuously Variable Damper

BACKGROUND:

118

The suspension system in cars determines ride handling and comfort characteristics of the car. The suspension system is a vibrational system with the three essential vibrational components, mass (of the car), spring (per wheel) and damper (per wheel). Softer suspension systems provide a comfortable ride as in luxury cars, whereas stiffer suspensions provide better handling of the road as in racing cars.

The softness/stiffness of a suspension system can be controlled by the design of the damper. Conventional dampers, which are based on gas or hydraulic fluids, produce a damping force (non conservative force) which is a function of the actuation velocity and the damping coefficient. Thus, for a given damper design and road conditions there is no way of controlling the damping force. Therefore, a trade-off is made between the comfort and handling capability of the car, when designing the damper.

The concept of semi-active suspensions, which has been around for over 20 years, is an attempt at improving the above mentioned trade-off. The semi-active nature is achieved by having the ability to continuously vary the damping force developed by the damper, and removing its dependence on the actuation velocity. A computer onboard the car gets inputs from sensors about the speed, steering, braking, road condition etc. and it computes the damping force for dampers at each of the wheel which would result in the best trade-off between comfort and handling. It then sends appropriate signals to the dampers for developing the required force.

One ME210 group in 1989-90 designed and tested a working prototype of a semiactive damper based on the rotary friction-disk concept.

The requirements that the design was to meet were the following:

Maximum resistive force in full ON state: greater than 500 lbs Minimum residual force in OFF state: less than Sibs Draw a maximum power of 72 watts (12 volts at 6 amps available from car

battery). Force response with less than 30-degree phase lag for a 10Hz sine wave input. Damper should have a stroke length of 6 inches (+- 3). Weight less than 12 lbs.

Information Handling Behavior of Designers During Conceptual Design: Three ExperimeDls


Size less than 66 cubic in. Cost comparable to conventional gas shocks. Should be able to operate in harsh environments seen by external car parts.

- temperature range (-40 to 120 degree F) - water proof - withstand dust/dirt.

Should be maintenance free. Should last longer than 100,000 miles.

REDESIGN PROBLEM:

Design firm A is designing a vehicle for use on a rough desert terrain as well as on regular tar roads. This requires the vehicle to have the ability to change its suspension system characteristics so as to adapt to a particular terrain. The adaptability can be achieved by having a continuously variable damper in the suspension system.

Based on some analysis it has been determined that the damper should develop damping force in the range of 10 lbs to 750 lbs. It has also been determined that the higher limit on the stoke length of the damper should be at least 10 (+-5 from mean position) inches. The vehicle will have a ground clearance more than what is designed in compact cars. This will allow more room and freedom in integrating the damper with the suspension system. The maximum volume available for the damper is 120 cubic inches. The desert has an ambient temperature in the range of 20 to 120 F.

The power available from the onboard battery is 12volts with a peak current of 10 amperes. The damper should be maintenance free.

Design a continuously variable damper which can be used on this vehicle.

Information Handling Behavior of Designers During Conccptual Design: Three Experiments


A.I.2 Description of ME210

Generation & Conservation of Design Knowledge CDR-Stanford University, NASA-ARC

Design Experiment

Description of ME210

120

This sheet of paper contains background information on the first generation design of a continuously variable damper which needs to be redesigned. The design was performed in the academic year 1989-90 at Stanford University by a group of three masters level graduate students. The project was part of the requirement for the depth series ME210 A, B and C. The project was sponsored by the electronics division of Ford Motor Corporation. At the end of the academic year the design team delivered a working prototype of a continuously variable damper along with complete documentation of the design and its performance as evaluated by laboratory experiments. The design team worked with a budget of about $13,750.

ME210 is a three quarter depth series in Automation in Machine Design. Approximately 30 students work in groups of two or three through out the series. In the beginning of ME210 A, students are given two short 2 week long projects. These projects are typically run as competition (eg. paper bike race) and students work on these projects from design to fabrication. These projects serve as warm-up exercises and means by which students can make appropriate choice of teammates for the corporate sponsored project which span the rest of A, B andC.

There are about 13-15 sponsored project in any year, and they vary in focus from process design to product design, automobiles to space applications etc. Each team has a coach who guides the team over the course of the project in issues of design, group dynamics, consultation etc. Most coaches are old ME210 students or industry designers with a good deal of design experience.

Design teams report their progress every two weeks to the teaching staff and the coaches. Guidelines for themes of the presentation are provided by the teaching staff to ensure that design teams cover all grounds. The presentation also serve as a means of getting feedback from other students in the class. Students are encouraged to use structured methods such as trade-off analysis, value analysis, how-why analysis as aids in making important decisions and scoping their problem. They are also advised to rapid prototype their ideas to get early answers to important questions and discover any surprises.

At the end of each quarter the design team summarizes its progress in the design requirement document. Thus, the three requirement document capture a



good overview of the design process undergone by the design team. In ME210 A the emphasis is on scoping the design problem and identifying the specifications (functional, physical and external) of the design. Teams also list a few promising concepts.

ME210B emphasizes selecting a final concept for the design and developing configurations and other details by rapid prototyping and other structured methods. The report contains documentation on any structured methods used, choices made and reasons for the choices etc.

ME210C deals with detail design, fabrication and testing of the prototype design. The performance of the design is evaluated under laboratory conditions. The teams are encouraged to make recommendation to their sponsors about ways in which the design can be improved.

The above breakdown is only qualitative and serves as a guideline for the teams, and is not strictly imposed. Each of the report has extensive supporting material in the appendix.

Besides the three reports, you will also have access to the personal design notebooks of the three designers, pictures of the prototype and a video of the laboratory setup for testing.

The goal of our experiment is to observe the reuse of design information and subsequently develop tools which would facilitate the reuse.

Information Handling Behavior of Designers During ConceplUaI Design: Three Experiments


A.2 Results on individual subj eets

Performance

Relation

Construction

Operation

Alternative

Rationale

Requirement

Location

Miscellaneous

~"~~""'-~"""""'""

.""""""""., [=:J

~,~'-,~ ~

""""

~

~

""""""""'" o 10 20

Proportion of Questions (%)

RS2

RS1

Figure A-1 Distribution of question descriptor for RS1 and RS2. a

30

Requirement k-oo~~'""C""O'-+~""'Il [=:J RS2

~ RS1 Assernblyk-oo~~'""C""O~~

Design-concept k-oo~~------I

Connection Io.:-...--..J

o 10 20 30 Proportion of Questions (%)

Figure A-2 Distribution of question subject-class for RS1 and RS2.

Information Handliog Behavior of Designers During Conceprual Design: Three Experiments

122

40


Conceptual t-c-~"T"'O~~-T-o~~

Configurational ~~~...,.--"'"

o 10 20 30 40

c:=J RS2

ISS:S] RS1

50 60 Proportion of Questions (%)

70

Figure A-3 Distribution of question level-of-detail for RS1 and RS2.

A.3 Analysis Examples

Below is an analysis example from about 17 minutes of the protocol data.

123

51: What have I, I made two comments just now. The lever and the heat. If I do the lever. then I may get a better force. And if I put the bub. if I separate the bub and the solenoid. then I may be able to deal with the heat problem. There's some reference material here on the solenoid. Now I'm looking for effect of temperature on the solenoid. Okay, there's some graphs here showing the relationships between the force and the stroke. so part of what I was looking for, force and stroke. Heat.

R2: what page is it?

51: This is E-2 in the appendix. Heat and current. There's current here. but I'ye not yet seen them put it together Temperature and force resistance factor. Okay ...

{a033} {vSO: 4B}

There are two ways to become aware of these dependencies between the heat and the force and stroke. The recommendations has a lot of that detail there. I'm not sure if I'll have been more sensitive to it without the initial questions, but it sure stands out now .. What do you use a transducer for?

R2: Is the transducer a part of the damper?

51: yeah, but I think you didn't do that.

El: we used it only for testing.

51: for testing, okay ... {aOS3} {reading} Ah. This is your bearing. I think I read they had some problem with bearings. But I can't remember. but at least it's good to know this document is there. No problem, it operates in good temperature. What's the C-card?? Adyanced technology. composite. what do you use it for. Probably for testing too. Composite materials.

El: Some other frictional material we looked at.



51: Ah. Looking for your index.

El: Index starts on like al bl cl.

{v54:22}

51: Okay. Now I'm thinking to myself, what was their friction material. Because that's another way of um, they had .3 coefficient of friction. That's pretty high. If I can go higher. like 5 or 4. that'll be good Get some force Qut of ~ I know what they used •• There's carbon, carbon there's asbestos,

R2: what page is that?

51: This is e23. Terrible coefficients of friction. point .14. 5tatic to dynamic ratio. I don't quite see where we'll be getting all those jerky motions El spoke ~. Oh okay. This funny looking torque curves .. So we want a, yeah. When the ratio is good, then the torque curve is very beautiful. Oh yes, El's a very smart guy ..

R2: What are you looking at now?

51: right now I'm still in the appendix. And I'm looking at the different frictional surfaces that they used. One thing I'd like to establish. was the key information that I want there really is something that has a high coefficient of friction and a good dynamic to static friction ratio. I'm not sure how critical that is, but if I can push up my coefficient of friction, that'll be good. So what else ..

{aI09} {v58:21}

I'm now looking at the original cost proposal. Just to see if there's something, I may have missed that ...

{aI25} {v59:49}

Right now I'm just looking at other materials. I don't think they will be ... {a13?} Okay, there's some test data here. It talks about levers. This is page P-56 ...

{a16l} {vl:03:0?}

Ah, good information here. They have something on the potential friction materials. Which says the carbon fiber will be a good option, but it's very expensive. It's 135 dol~ars and the kevlars are like 35 bucks ...

{al?5} {vl:04:1?}

Here's some information about the electrorheological fluid .. It also shows ~ examples of mono tube that uses electrorheological fluids. I wonder why it was eliminated ... Okay. So let's see .. At this point I'm, I know a bit much more about the problem ... {a205} So maybe I'll call El. Let's talk a little bit .. ~ happens if my leyers had a longer arm?

Table A-1 Example of analysis procedure in information reuse study.

QNo. Question from Protocol After Reformulation Question Classification

b52 If I do the lever, then I may If I use the lever will I get Descr/pror: Relation get a better force. a better force? Subjclass: Component

Level of Detail: Conceptual

b53 And if I put the hub, if I If I separate the hub and the Descrlpror.Relation separate the hub and the solenoid, will that impact Subjc/ass: Assembly solenoid, then I may be able positively on the heat problem? Level of Detail: Conceptual to deal with the heat problem.

Information Handling Behavior of Designers Ouring Conceptual Design: Three ExperimenlS


Table A-1 Example of analysis procedure in information reuse study.

QNo. Question from Protocol After Reformulation Question Classification

bs4 Now I'm looking for effect ?f What is the effect of Descriptor: RelatiDn temperature on the solenoid. temperature on thesolenoid? Subjclass: Component


bss Heat and current. There's What is the relation between Descriptor: RelatiDn current here, but I've not yet heat and current to the Subjclass: Feature seen them put it together. solenoid? Level of Detail: Conceptual

bs6 What do you use a transducer What do you use a transducer Descriptor: Rationale for? for? Subjclass: Componellt


bs7 I think I read they had some Did they have a problem with Descriptor: Performance problem with bearings. But I bearings? Subjclass: Cornponertt can't remember, Level of Detail: Conceptual

bs8 Advanced technology, What do you use composite for? Descriptor: Rationale composite, what do you use it Subjclass: Component for. Level of Detail: Conceptual

bs9 what was their friction what was their friction Descriptor: Construction material. material? Subjclass: Feature

Level ofDetoil: Detail

b60 If I can go higher, like .5 or If I go higher, like .5 or .4, Descriptor: Performance .4, that'll be good. Get some will I get some force out of Subjclass: Feature force out of that. that? Level of Detail: Detail

b61 I don't quite see where we'll Where will we be getting all Descriptor: OperatiDn be getting all those jerky those jerky motions E1 spoke Subjclass: Assembly motions El spoke about. about? Level of Detail: Configtua·

ttona!

b62 I'm looking at the different What different frictional Descriptor: Altemative frictional surfaces that they surfaces did they use? Subjclass: component used. Level of Detail: Conceptual

b63 One thing I'd like to Is it very important that the Descriptor: Requirement establish, was the key coeffiCient of friction be Subjclass: Component information that I want there high and dynamic to static Level of Detail: Conceptual really is something that has a friction ratio be equal? high coefficient of friction and a good dynamic to static friction ratio.

b64 two examples of mono tube that Why did they eliminate using Descriptor: Rationale uses electrorheological electro rheological fluids? Subjclass: Design-concept fluids. I wonder why it was Level of Detail: Conceptual eliminated •..

b6s What happens if my levers had What happens if my levers had Descriptor: Performance a longer arm. a longer arm? Subjclass: Feature

LevelofDetoil: Detail


Appendix 8: Additional Material on Deda! Development 126

Appendix B

B.1 Dedal Implementation Details

Additional Material on Dedal Development

B.2 Dedal Interfaces for Indexing, Querying and Modeling

B.3 Examples of Heuristics

B.4 References on Dedal Papers

B.l Dedal Implementation Details

• Dedal runs in the Unix environment under Sun as 4.1.3

• Framemaker is the information capture environment

• It is mostly written in Common Lisp

• Heuristics are represented in MRS (Meta Reasoning System). MRS is also is used to do the reasoning and backward chaining during retrieval

• Interface to audio and video players is written in C

• Dedal has also been ported to the Macintosh but has not been tested yet.

• A web based implementation of Dedal is under development. With this, the tasks of indexing, modeling and retrieval will be available over the world wide web.

Information Handling Behavior of Designers During Conceptual Design: Three ExpcrimcDlS

Appendix B: Additional Material on Dedal Development 127

B.2 Dedal Interfaces for Indexing, Querying and Modeling

All the interfaces shown in figures below are from the Unix implementation of

DedaI.

Figure 8-1 Oedal's indexing and querying environment. The same interface is used to indexing (this adds indices to the database) and to query (this searches the indices database).This interface allows users to perform concept based indexing, the scheme resulting from the information reuse study.


Appendix B: Additional Material on Dedal Development

Figure 8-2 Dedal's modeling interface. Users can relate concepts in a model by using this interface. New concepts can either be created here or during indexing.

Information Handling Behavior of Ocsigners During Conceptual Ocsign: Three Experiments

128

Appendix 8: Additional Materia! on Deda! Development

Figure 8-3 Dedal's answers window. Answers to a query are returned in a window shown above. The name of the document is linked to the document. Users can give feedback on whether a answer is relevant or not.

B.3 Examples of Heuristics

129

Here are some examples of heuristics encoded as rules in a lisp like syntax. A

symbol preceeded by a $ sign stands for a variable. $q is the variable for the

descriptor. The : if section of the rule checks for some conditions. If these con

ditions are true then it generates a new-question represented in the : then

part of the rule. The text following rule is the name of the rule.

(rule look-isa

Information Handling Behavior ot Designers During Conccptlllll Design: Thn:c Experiments

Appendix 8: Additional Material on Dedal Development

:if (and (question $q) (object $0) (a-kind-of $0 $p) (unprovable(is-generic-term $p»)

(new-que.tion $q $p nil look-isa»

(rule look-for-specializations :i£ (and

: then

(question $q) (object $0) (a-kind-of $p $0) (choice $dec $p»

(new-que.tion $q $p nil look-for-specializations»

(rule look-isa-for-attribute :i£ (and

: then

(question $q) (attribute $s $0) (isa $0 $p) (unprovable (is-generic-term $p»)

(new-question $q $p $s look-isa-for-attribute»

(rule 100k-for-dependent-attributes-1 :i£ (and

: then

(question $q) (or (question relation) (question rationale» (attribute $s $0) (dependent-on ($s of $0) ($sl of $01»)

(new-que.tion $q $01 $sl 100k-for-dependent-attributes-1»

(rule from-attribute-to-component-operation :weight 2 :i£ (and

: then

(question relation) (attribute $s $0»

(new-que.tion operation $0 nil from-attribute-to-component-operation»

(rule from-attribute-to-component :weight -5 :if (and

: then

(question $q) (attribute $s $0»

(new-que.tion $q $0 nil from-attribute-to-component»

(rule look-for-common-computed-from :if (and

(question $q) (attribute $s $0) (attribute $s2 $02) (attribute $s3 $03) (notsame $s $s2) (is-computed-from ($sl of $01) ($s of $0» (is-computed-from ($sl of $01) ($s2 of $02» (is-computed-from ($sl of $01) ($s2 of $02») : then

(new-que.tion $q $01 $sl look-for-common-computed-from»

(rule look-for-common-computed-from :if (and

(question $q) (attribute $s $0) (attribute $s2 $02)


130


(notsame $s $s2) (is-computed-from ($sl of $01) ($s of $0» (is-computed-from ($sl of $01) ($s2 of $02») : then

(new-qae.t!oD $q $01 $sl look-for-common-computed-from»

(rale look-relation-< :if (and

(question $q) (attribute $s $0) (is-computed-from ($sl of $01) ($s of $0»)

: then (Dew-qae.t!oD $q $01 $sl look-relation-<»

(rale look-related-attribute :if (and

: then

(question $q) (attribute $s $0) (isa $s $p»

(new-qae.tion $q $0 $p look-related-attribute»

(rale look-related-value :if (and

: then

(question $q) (attribute $s $0) (val ($s of $0) $v) (is-symbol $v»

(new-qae.tion $q $v nil look-related-value»

(rale look-parent-of-related-value :if (and

: then

(question $q) (attribute $s $0) (val ($s of $0) $v) (is-symbol $v) (isa $v $p»

(new-qae.tion $q $p nil look-parent-of-related-value»

(rale part-of :if (and

: then

(question location) (object $0) (subpart $p $0»

(new-qae.t!on description $p nil part-of configuration schematic»

(rale part-of-2 :if (and

: then

(question location) (object $0) (subpart $p $0»

(new-qae.t!on construction $p nil part-of configuration schematic»

(rale find-function-component :if (and

: then

(question function) (object $0) (subpart $p $0»

(new-qae.t!on operation $p nil find-function-component»

(rale find-function-connectorl :if (and

(question function)

Information Handling Behavior of Designers During ConcepruaJ Design: Three ExperimenlS

131

Appendix B: Additional Material on Deda/ Development

(object $0) (connects $0 $p1 $p2»)

: then (aew-qge.tion construction $p1 nil find-function-connector1)

(rule find-function-connector2 :if (and

: then

(question function) (object $0) (connects $0 $p1 $p2»)

(new-qge.tion construction $p2 nil find-function-connector2)


: then

(question function) (object $0) (connects $0 $p1 $p2»)

(aew-qge.tion operation $p1 nil find-function-connector3))


: then

(question function) (object $0) (connects $0 $p1 $p2))

(new-qge.tion operation $p2 nil find-function-connector4))

(rule basic-part-of :weight-1 :if (and

(or (question description) (question operation) (question construction)

(question analysis) (question performance»)

(question $q) (object $0)

(part-of $p $0) : then (new-qge.tion $q $p nil extended-part-of ))

(rule find-alternatives :if (and

: then

(question alternative) (object $0) (alternative $d $0) (alternative $d Salt) (notsame $0 $alt»)

(aew-qge.tion alternative Salt nil find-alternatives)

(rule find-alternatives-from-decision-point :if (and

: then

(question alternative) (object $0) (alternative $0 $alt))

(new-qge.tion alternative Salt nil find-alt-from-decision-point)

(rule find-alternatives-for-attribute-value :if (and

: then

(question alternative) (attribute $att $0) (decision $d $0 $att) (alternative $d $alt))

(new-qge.tion alternative Salt nil find-alt-from-attribute-value»)

Information Handling Behavior of Designers During ConcepltlaI Design: Three Experimenls

132


(rule describe-connectors-l :if (and

: then

(question description I (object $01 (connects $0 $pl $p211

(new-qa •• tioD description $pl nil describe-connectors-lll

(rul. describe-connectors-2 :if (and

: then

(question description I (object $01 (connects $0 $pl $p211

(Dew-qa •• tioD description $p2 nil describe-connectors-211

(rule from-attribute-to-requirement-l :if (and

:th.D

(question description I (attribute $s $01 (r-relate ($s of $01 $req $ type I I

(Dew-qa •• tioD requirement $0 $s from-attribute-to-reqmt-lll

(rul. from-attribute-to-requirement-2 :if (and

: then

(question descriptionl (attribute $s $01 (r-relate ($s of $01 $req $ type I I

(new-qa •• tion requirement $req nil from-attribute-to-reqmt-211

B.4 References on Dedal Papers

133

1. Baudin, Catherine; Underwood, Jody G. and Baya, Vmod, "Using Device Models to Facilitate the Retrieval of Multimedia Design Information", In Proceedings of the 13th International Joint Conference on Artificial Intelligence, Chambery, France, pp 1237-1243, August 29-September 2,1993a.

2. Baudin, Catherine; Kedar, Smadar; Underwood, Jody G. and Baya, Vmod; "Question-based Acquisition of Conceptual Indices for Multimedia Design Documentation", In Proceedings of the 11th National Conference on Artificial Intelligence AAAI-93, Washington D.C., pp 452-458, July 1993b.

3. Baudin, Catherine; Underwood, Jody G.; Baya, Vmod and Mabogunje, Ade, "Using Domain Concepts to Index Engineering Design Information", In Proceedings of the meeting of the Cognitive Science Society, Bloomington, Indiana, 1992.


Appendix C: Additional Material on Design Information Handling Study 134

Appendix C

C.1 Instructions and Overview

Additional Material on Design Information

Handling Study

C.2 Design Exercise Problem Statement: Bike Lock 1

C.3 Design Exercise Problem Statement: Bike Lock 2

C.4 Design Exercise Problem Statement: Backpack Harness

Information Handling Behavior of Designers Ouring Conceptual Design: Three Experiments

Appendix C: Additional Material on Design Information Handling Study

Dear Subject:

Center for Design Research Stanford University Design Experiment Date: May 26, 1993

e.l Instructions: Overview

135

Thank you for willing to participate in the design experience you are now starting. The whole experience is anticipated to take about 1.5 hours. The experience will be recorded on audio and video mediums.

I am interested in the manner in which you would be handling information during your design exercise. To get an explicit recording of your activities with information, I request you to talk out loud while you are doing the design. If there are noticeable periods of silence, I would very likely tap you on your shoulder to remind you to talk out loud.

We will be following the procedure below.

1. Warm-up Exercise: 10 min, This is a short exercise to give you an experience in talking out loud.

2. Design Exercise: 1 hour, This is the meat of the experience.

3. Debriefing: 10 min, Answering a small questionnaire.

Some stationery is provided for your use. Feel free to ask me for any clarifications or information during the course of the experience.

Thanking you,

Vinod Baya


Appendbc C: Additional Material on Design Information Handling Study

Center for Design Research Stanford University Design Experiment Date: May 26, 1993

C-2 Design Exercise: Problem Statement Bike Lock 1

136

Background: As you might have experienced, an everyday activity of locking/ unlocking a bicycle with a kryptonite lock is quite an arduous task. It involves seven steps (believe me I have counted) and takes a fairly long time (It takes me on an average 15 seconds to lock or unlock my bike). Imagine it taking you so long to lock/unlock your car! Wouldn't it be nice if there was a more userfriendly bike lock.

Problell1: Design a user-friendly locking mechanism for a bicycle which meets the following requirements.

1. The locking mechanism should be an off the shelf item which can be installed on a bicycle without much difficulty.

2. Once installed on a bike the mechanism should become an integral part of the bike.

3. It should be possible to perform the locking/unlocking operations quickly and in few steps.

4. 'the lock should have the strength equal to or better than that of a kryptor\ite lock.

5. 'the locking mechanism should have a reasonable cost.

6. 'the locking mechanism should be safe to use.

The following assumptions can apply to the design:

1. A. bike is considered locked if either the front or the rear wheel is not free to rotate.

2. 'the lock can be mounted anywhere on the frame of bike.

3. Use the bike provided as the sample of what the lock could be mounted on.

4. '!'he U lock provided can be assumed to be a representative of a kryptonite lock.



5. Design of an appropriate lock/key mechanism can be assumed to be available.

6. While it is sufficient if the bike can be locked in isolation, an ability to lock the bike to a fixed external frame (shown below) would be an added feature of the lock. If bike is locked to a fixed external frame, then the wheels can be free to rotate.

7. One possible solution that is widely used in India is sketched in an accompanying sheet. This is meant only to explain the requirements.

A detail quantitative design solution is not expected. An adequate solution would qualitatively describe the configuration and the operation of the mechanism.

3.5 feet

2.5 feet

Fixed external frame to which the bike should be locked



unlocked position ---II~ /

locked ~( position

locking mechanism

~ rearfork

~ rearwheel

Sample Solution of a Bike Lock which is installed on a Bike, commonly used in India


138


Center for Design Research Stanford University Design Experiment

Date: March 17, 1994

139

C.3 Design Exercise: Problem Statement Bike Lock 2

Background: As you might have experienced, an everyday activity of locking/ unlocking a bicycle with a kryptonite lock is quite an arduous task. It involves seven steps (believe me I have counted) and takes a fairly long time (It takes me on an average 15 seconds to lock or unlock my bike). Imagine it taking you so long to lock/unlock your car! Wouldn't it be nice if there was a more userfriendly bike lock.

Problem: Design a user-friendly locking mechanism for a bicycle which meets the following requirements.

1. The locking mechanism should be an off the shelf item which can be installed on a bicycle without much difficulty.

2. Once installed, the mechanism should become an integral part of the bike.

3. It should be possible to lock the bike in isolation as well as to a stationary frame. The shape of a typical frame is at the end of this statement. Being able to lock to just this frame would be sufficient.

4. It should not be possible to detach the locking mechaniSm from the bike when it is in the locked position (otherwise it wouldn't be a lock would it?)

5. It should be possible to perform the locking/unlocking operations quickly and in fewer steps.

6. Strength of the lock should be better than or equal to that of a kryptonite lock.

7. The mechanism should have a reasonable cost and be safe to use.

The following assumptions can apply to the design:

1. A bike is considered locked if either the front or the rear wheel is not free to rotate, or if the frame of the bike is not free to move.

2. The lock can be mounted anywhere on the frame of bike.

3. Use the bike provided as the sample of what the lock could be mounted on.

Information Handling Behavior of Designers During Conceptual Design: Three Experir!1C'nls


4. The U lock provided can be assumed to be a representative of a kryptonite lock.

5. Design of an appropriate lock/key mechanism can be assumed to be available.

6. One possible solution that is widely used in India is sketched in an accompanying sheet. This is meant only to explain the requirements. Note that this solution only locks the bike in isolation.

A detail quantitative design solution is not expected. An adequate solution would qualitatively describe the configuration and the operation of the mechanism.

3.5 feet

2.5 feet

Fixed external frame to which the bike should be locked

The sketch for possible solution is same as the one used in problem statement for bike lock 1 (page C-S).

Informnlion Handling Behavior of Designers During Conceptual Design: Three Experiments


A n a i y sin g Des i g n Act i v i t Y -The Delft Protocols Workshop

Instructions: Protocol Study Individual Design

Thank you for agreeing to participate in this research project. We are interested in the ways that designers work on conceptual design projects. This session is being videotaped for later analysis. I am going to ask you to undertake a short design project. Let me assure you that no commercial advantage will be taken of any design work or ideas that you produce.

Because we want to understand what you are thinking during the task, we want you to think aloud continually during the session. This would be as if you are talking to yourself but loud enough for me to hear. In order for you to get used to thinking aloud, I will give you a small problem which has nothing to do with the design task. Try to solve this problem while thinking aloud. This is only a training exercise, which last 10 minutes: so please do not worry if you do not solve the problem. Here is the problem I want you to work on.

Three missionaries and three cannibals are together on one side of a river. They have one rowing-boat, which can hold up to two people. They all know how to row. How can they all reach the other side of the river, given that for obvious reasons there must never be more cannibals than missionaries on either river-bank?

I am now going to ask you to undertake the design project. The time available is limited to two hours. In a moment I will give you a short written design brief for the project. Additional information which you may require during the project is available in an information file which I keep and have ac.:ess to. So, if you think that you need additional information during the project, please ask me for what you need to know. Please be specific about what you ask for. The information available in the file includes both technical and client information. Although access to the file is only available through me, please do not feel constrained about asking for any information that you feel you need during the project. I also have an outside helper who can give me additional information if it is not available in the file. This book TITLE is also available if you need it please ask me for it when you want it and say why you want it. Apart from providing you with information when you ask for it I will not interact with you in any other way. You should try to ignore my presence in the room.

Drawing materials - paper, pens and markers are available for your use. There is also a white board which you may use if you wish. The particular design project which we are asking you to undertake is concerned with a new product related to mountain bikes and backpacking. Therefore we also have in the room a mOlIDtain bike and a backpack, which you may use or refer to as you wish. Please work as you normally would on such a design assignment as this. But remember to keep thinking aloud.

Before I give you the design brief, are there any questions about the procedure? If there are no further questions we will start the session now. I remind you that the maximum time available to you is two hours. I will remind you of the time at 30 minutes and 15 minutes before the end of the session, if you are still working.

(After the designer has read the assignment the experimenter says:)

The bicycle here in in the room is not the same as the Batavus Buster.



C.4 Design Expt. Problem Statement: Backpack Harness

Assignment

HiAdventure Inc. is a fairly large US firm (some 2000 employees) making backpacks and other hiking gear. They have been very successful over the last ten years, and are well known nationwide for making some of the best external-frame backpacks around. Their best selling backpack, the midrange Hi Star, is also sold in Europe. In the last one and a half years, this European activity has suffered some setbacks in the market; in Europe internal-frame backpacks are gaining a larger and larger market share.

As a response, HiAdventure has hired a marketin-firin to look for new trends and opportunities for the European markeL On the basis of this marketing report, fiiAdventure has decided to develop an accessory for the HiStar:

A special carrying/fastening device that would enable you to fasten and carry the backpack on mountain bikes. The device would have to fit on most touring- and mountain bikes, and should fold down, or at any rate be stacked away easily. A quick survey has shown that there is nothing like this on the european market.

This idea is particularly interesting for HiAdventure, because the director, Mr. Christiansen, has a long-standing private association with one of the chief product managers at the Batavus bicycle company (one of the larger bicycle manufacturers in northern Europe, based in Holland). Mr. Christiansen sees this as a good opportunity to strike up a cooperation and profit from the European marketing capacities of Batavus.

The Batavus product manager, Mr. Lemmens, is very enthusiastic about putting a combinationproduct on the market, a mountain bike and a backpack that can be fastened to it. The idea is to base the combination-product on the Batavus Buster (a midrange rtlountain bike), and to sell it under the name Batavus HikeStar. The design department at Batavus has made a preliminary design for the carrying I fastening device, but both Mr. Christiansen and Mr. Lemmens are not very content with it. The user's test performed on a prototype also showed some serious shortcomings.

That is why they have hired you as a consultant to make a different proposal. Tomorrow there is going to be a meeting between Mr. Christiansen and Mr. Lemmens, scheduled as the last one before presenting the idea to the board of Bataeus. Before then, they need to have a clearer idea of the kind of product it is going to be, its feasibility and price.

You are hired by HiAdventure to make a concept design for the device, determining the layout of the product, concentrating on

- ease of use - a sporty, appealing form - demonstrate the technical feasibility of the design - make sure to stay within a reasonable price range

You are asked to produce annotated sketches explaining your concept design. Good Luck.

Information Handling Behavior of Designers During Conceptual Design: 11IRC ExperirncDls

Appendix 0: Additional Results on Information Handling Study 143

Appendix D

Additional Results on Information Handling Study

This appendix has results on the information handling study, in addition to

those discussed in Chapter 6. Some supporting data is included as well. In many

cases graphs for each of the exp-subjects are shown to illustrate the nature of the

raw data across all the exp-subjects and for comparison purposes. However, in

cases where the characteristics of the raw data is similar, data for only one exp

subject is shown because of space considerations. Results included are on:

• information fragment duration (deltat)

• information handling rate (dpm)

• information fragment attributes behavior over time

• distribution of time for each attribute

• state transition probabilities

• multiple attribute analysis

0.1 Additional Results on deltat

0.1.1 deltat vs. time

Figure 0-1 to Figure 0-6 show the variation of information fragment duration

(deltat) over time for all exp-subjects. The curve in each of the figures is a polyno

mial fit to the raw data. It shows the movement of the exp-subjects between small

and large information fragments at a level higher than shown by the raw data.

This curve for all subjects on a normalized time scale is shown in Figure 0-7.

Information Handling Behavior of Designers During Conceptual Design: 1luec Experiments

Appendix D: Additional Results on Information Handling Study

-III "C C a (J Q)

~ c a 1a .... :J C -c Q)

E C)

e LL c a :;:; as E .... a -c

20

-III "C C a (J Q)

~ c a

01

Cii :520 c -c Q)

E Cl e u. c a Cii E .... a C

N I ..... ~" '/ \

V' • r-..

:U "y.

I i

. 0 30 . 50_ .

Time (minutes)

Figure D-1 Behavior of deltat over time for 81

.

!

/ 11 ~

\ . . 35

l t ~ rr f'-1

)

~ 'J

55 Time (minutes)

Figure D-2 Behavior of deltat over time for 82.

" ,pi

70

I

/ ~~

~ V

!\ V

Information Handling Behavior of Designers During Cooceprual Desigo: 1luec Experiments

144

I

:-.~ r-.

90

~ N . i'.,

75

Appendix D: Additional Results on Information Handling Study 145

40.-----------------------------------------------------,

-(J)

"C c: o ~30 ~ c: o e :J c E20 CD E C)

e! u. c: o iii 1 0 E o :s

°5~~--~------~25~----~----~45~----~------~6~5------~~ Time (minutes)

Figure 0-3 Behavior of deltat over time for 83.

(i)30 "tJ c: 0 0 CD (J) -c: 0

~

\ :;20 c -c: CD

~ ~ E C)

~ CIS r

1\// .... u.

1'\ L../1 c:

~ 0210

~ :~~ iii E ~ r"- \ .... 0 1: V

1\

\

010 30 50 Time (minutes)

70

Figure 0-4 Behavior of de/tat over time for 84.

Information Handling Behavior of Designers During ConccplUai Design: Tbrcc Expcriments


i) 2 5 .J D n -2 ii 5 ) J

~ :n IS L. -2 ~

2

40

30

20

-....,

10

°0~------~------~2~0--------~------~4~0--------~------~60 Time (minutes)

Figure 0-5 Behavior of de/tat over time for S5.

en 30 "C c: 0 0 Q) (J) -c: 0

:e 20 :::J

i\ .

/ . Cl

'E I

Q)

E C)

~ u. c:

10 0 :a; E ....

1 V ~, .~l.~ / ,

I"- ~. " 0

:s I 65Ti . } (me (minutes

115

Figure 0-6 Behavior of de/tat over time for S6.

Information Handling Behavior of Designers During Conceptual Design: Three Experimcn~


I ..... ~ . /' S5 /'

I l'

I

______ ---....... S2 ,.-~;

_~_- ______ / I, I

\ / /---- I 'J \...,/ I

S1

0.6 0.8 Normalized Time

Figure D-7 de/tat behavior over time for all exp-subjects. The curves are polynomial fit to the raw data. The time on the x-axis is normalized to neutralize the effects of varying design times. The oscillations are more pronounced in some cases than others but clearly de/tat behavior is oscillatory, i.e. it is neither steadily increasing or decreasing. As one moved along the time axis, three regions stand out. Region 1 (O to 0.2): this is the region where designers developed an understanding of the design problem and a strategy to solving it. Region 2 (0.2 to 0.9): this is where the bulk of the problem solving occurred. Region 3 (0.9 to 1.0): this region started soon after exp-subjects were informed of the remaining time and it involved wrapping up the design and describing a solution.

Information Handling Behavior of Designers During ConceplWll Design: Three ExpcrimcnlS

1.0

147

II)

E Q)

E C) as .... -c: .Q as E .... a -c:

'0 .... Q) .c E :::s Z


0.1.2 Distribution of deltat: histograms

20 20

15 81 15 82

10 / /

10

5

~ -llmllJL 0 TTTn! rm

0 10 20 30 10 20 30 20

~ 15 c: Q)

83 E C)

84 15

as .... - 10 10 c: a ~ E .... 5 a 5

'E -a .... Q)

10 20 30 .c E :::s Z

15 15

/ ~

10 85 10 \ 86

5 5

ill ~~ ~.rrufl n n Ilnllflll 111I1Im-tt o~wwwwww~uuuwww~~~ww~~~ww~ 0

o 10 20 30 0 10 20 30 deltat (seconds)

Figure D-8 Distribution of the information fragment duration. The curves are polynomial fit to the histograms. Histograms are drawn using a bin width of 0.5.

Infonnation Handling Behavior of Designers During Conceptual Design: Three Experiments


0.2 Additional Results on Information Handling Rate

0.2.1 Information handling rate over time

Figure 0-9 to Figure 0-14 show the variation of information handling rate (dpm)

over time for all exp-subjects. The curve in each of the figures is a polynomial fit

to the raw data. It shows the movements between small and large information

fragments at a level higher than shown by the raw data.

40~--------------------------------------------------~

E a. "C -'s20 as II: E C

\

r\

-- ~ V--'-~-«<1"Ir

~U i~ N

~~O----~-----'3~O----~-----5~IQ~----~----~7~O----~----~90 Time (minutes)

Figure 0-9 Information handling rate over time for 51

Infonnation Handling Behavior of Designers During Concepwal Design: 1'It= ExperimenlS


40

E a.

"C -.!20 ta II: E o

o ~

[

"\ ",

"" " f

U . /

~ " ....

I~

/ ~ 'A

25 .45 TIme (minutes)


150

. / ~

1A r' .~

~~

I~. V rl V

65

40r-------,-----------------~----------------------------_,

E a.

"C -.!20 ta II: E o

!

I~ o~------~----~~------~------~------~------~~----~--~

Time (minutes)


Infonnation Handling Behavior of Designers During Conccpwal Design: Three Experiments


E c.. "0 -

40

Q) 20 ca a: E (5

o

•

'f

~4~'~o,l ., '>, j 1",

I/ I V V~

I 2U 4U Time (minutes)

Figure D-12 Information handling rate over time for 54.

151

T

~ r-

r. "'\ I V-

('-~.

ij

60 80

40~------------------------------------~------------------~

E Co

"0 -~ I

ok---------~-------~20--------~------~4nrU--------~-------~tO

Time (minutes)

Figure D-13 Information handling rate over time for 55

Information Handling Behavior of Designers During Conc:eplWll Design: Three Experiments


40r-------------------~------------------------r_~------_,

E a.

1:1 -'s20 as a: E

C5

Figure 0-14 Information handling rate over time for 56.

0.2.2 Distribution of information handling rate

en E Q)

E C) as ... u.. c: .Q Cii E ... 0 :s "0 ... Q) .c E :::J Z

15 -51 -52 -53 •••• 54

. ·55 -56

10

5

0 I

Dim Rate (dpm)

Figure 0-15 Distribution of dim rate for all exp-subjects. The curves are polynoml~1 fit to histograms now shown here. The average most frequent information hdndling rate (the average of the maxima point of each curve) is 6.1 dpm.

Infomuuion Handling Behavior of Designers During Conceprual Design: Three ExperimenlS

Appendix 0: Additional Results on Information Handling Study

0.2.3 Dim rate vs. Verbal rate: Regression analysis

28

~24 E c.. ~20 CD 16 a: 16 (ij -e ~ 12

~S1

-S2 b:--AS3

"'--"54 +- +S5 -S6

Ok---~~--~~--------~~------~~--~----~----~~

dim rate (dims/min)

Figure 0-16 Regression analysis between dim rate and verbal rate. Only a few data points are shown for clarity sake.The lines are linear regression fit to the raw data. Except for 53 the coefficient of correlation is within a narrow range

153

D.3 Information Fragment Attributes Behavior Over Time

Figure 0-17, Figure 0-18, Figure 0-19 and Figure 0-20 show the behavior of

informational activity, information descriptor, information subject-class and

information level-of-abstraction over time respectively.



CI) rJ) CI)

~ rJ) 1ii CI)

"-"iii CJ CI) c: CJ c: < < CI)

(!)

o CD

o ,...

_ .....

rn CD

0_ III ~

C ·e -CD E

oi= ...

o t')

o ""

0 -

Figure 0-17 Behavior of informational activity over time for 81. The figure illustrates that designers move frequently between the different informational activities all along the conceptual design process. No predictable pattern is obvious. The behavior for other exp-subjects is similar in characteristics.

Information Handling Behavior of Designers During Conc:epruw Design: Three Experiments

154


r - g

- ~



.... Q) ~

5 a. Q) o c: o ()

I

c: C)

en Q)

Cl

c Q)

E ~ ·s 0-Q)

ex:

: ::

c: .2 '0 Q) c: c: o U

f

-c: Q) c: o a. E o u

; • •

~

>. :c E Q) en en «

0

"

0 ..,

0 ~

g

o N

Figure 0-19 Behavior of Information subject-class over time 53. The figure illustrates that designers frequently move between the different information subject classes all along the conceptual design process. No predictable pattern is obvious. The behavior for other exp-subjects is similar in characteristics.

Information Handling Behavior of Designers During Concepnml Design: Three Experiments

156

(i) Q) -:l c: g Q)

E i=


r-----------------------------~----------~~=======r--------_.~

; I : [ :

I

~ ~ : ~ I ;=:

(J) (J) (J) "0 "0 > > > (J) (J)

E E ~ CD (J)

U .0 .c :;= (ij (\l (\l c 0 --l C tU :l Ul :l a Ul :::> a «

Figure D-20 Behavior ot Intormation level-ot-abstraction over time 54. The figure illustrates that designers move between the different levels ot abstraction often all along the conceptual design process. It was also observed that concepts start at the unlabeled level and mature to other levels as they become more concrete. No predictable pattern is obvious.


o ,..

0 tn

0 ...

0 t')

0 C\I

5!

157

-Ul (J)

'S c

I (J)

E i=


D.4 Additional Results on Distribution of Time

Analyze

Access

Generate

o 20 40 Proportion of Total Design Time (%)

G--E>S1 G-EJS2 1!r--6.S3 x---~S4

+-+55 ~S6

60

Figure 0-21 Distribution of time across informational activity. The distribution across the different exp-subjects is quite consistent. For all exp-subjects, more than half the time is spent generating information.

Miscellaneous

Requirement

Relation

Rationale

Performance

Operation

Location

Construction

Comparison

Assumption

Altemative

---::::::«

o 20 Proportion of Total Design Time (%)

G--E>S1 G-EJS2 1!r---6.S3 X··_J( S4

+- +55 ~S6

Figure 0-22 Distribution of time across information descriptor. The distribution is quite comparable across the exp-subjects. For all exp-subjects, information regarding I construction, operation, alternative and requirement dominate.


158

40


Other

Design-Concept

Requirement

Feature

Connection

Component

Assembly

o

G---eSl G-£lS2 /!r--AS3 X---i<S4 +- +S5 ~S6

... -... -:~---::-:-:---:-:----~ -. :=1'"-)(

20 40 Proportion of Total Design TIme (%)

Figure 0-23 Distribution of time across information subject-class. The figure shows that the distribution is reasonably consistent for most classes across the exp-subjects. It differs for classes connection and requirement.

Quantitative

Qualitative

Associative

Labeled

Unlabeled

o 20 40

Proportion of Total Design TIme (%)

G---e51 G-£l52 /!r--A53 X---K 54 + ·+55 ~56

Figure 0-24 Distribution of time across information level-af-abstraction. The distribution differs across the exp-subjects.The differences are well distributed across all levels of abstraction.

Information Handling Behavior of Designers During Conceptual Design: Thn:c Experiments

159


D.5 Results on State Transition Probabilities

State transition probabilities inform. us about the likelihood of a transition from

one state to another. These probabilities are calculated under the assumption

that the next state is determined by only the current state and not any states

prior to it (i.e. the memory of anything but one previous state is erased. This

assumption is commonly made in developing Markov chain for a process with

discrete states. This assumption is being made in obtaining the results below.

0.5.1 Informational activities state transition probabilities

22

Figure D-25 Information activity state transition probabilities. The figure shows that from generate, transition to analyze is most likely, and from access or analyze transition to generate is most likely.



D.5.2 Information descriptor state transition probabilities

Figure 0-26 information descriptor state transition probabilities. For clarity sake only the transitions with probability greater than 0.19 are shown here. From most states transition to construction is most likely. From construction, transition to alternative or operation is most likely.

Information Handling Behavior of Designers During Conceprual Design: Three ExperiDll:ms

161

Appendix 0: Additional Resufts on Information Handling Study

0.5.3 Information subject-class state transition probabilities

Figure 0-27 Information subject class state transition probabilities. For clarity sake only transition with probability greater than 0.2 are shown here. Transition to assembly are most likely from most other states.

162

0.5.4 Information level-of-abstraction state transition probabilities

Shown in Figure 0-28.

D.6 Additional Results from Multiple Attribute Analysis

Table 0-4 to Table 0-6 show various combinations resulting from multiple

attribute analysis. In all the tables, significant combinations are highlighted. The

numbers in the table are the fraction (in percentage) of information fragments in

the particular combination to the total number of information fragments.



Figure 0-28 Information level-of-abstraction state transition probabilities. Transitions with probabilities greater than 0.25 are shown here. No particular transition particularly stands out here.

D.6.1 Information descriptor and informational activities

Table 0-1 Distribution of descriptor across the informational activities.

163

The numbers below represents the proportion of informational activities in each of the descriptors. Much of the information regarding the descriptors is generated. In case of performance most of the information is analyzed and most of the requirements are accessed.

Descriptor Access

Alternative 12

Assumption 17 8

Comparison 11 20

Construction 28 27

Location 32 II

Operation 21 24

Perfonnance 30 • _. --. ,.'. - ,. - 1

: ____ X, _. ~J Rationale 8

Relation 16

Requirement 25

Miscellaneous 12



0.6.2 Information subject-class and informational activities

Table 0-2 Distribution of subject-cass across the informational activities. The numbers below represent the proportion of informational activities in each of the subject classes. For all classes, much of the concentration is in the activity generate. However, a significant number of info-fragments in assembly were accessed and a good number of info-fragments in requirement were accessed and analyzed.

Subject Class Analyze

Assembly 23

Component 18

Connection 20

Feature

Requirement

Design concept

Other 15 15

0.6.3 Information level-of-abstraction and informational activities

Table 0-3 Distribution of level-of-abstraction across informational activities. The numbers below represent the proportion of informational activities in each of the level of abstraction. For all levels, large share of information is generated and nearly equal amount of quantitative information fragments were in each of the informational activities.

Level of Abstraction Access Analyze

Unlabeled 24 20

Labeled 29 23

Associative 16 20

Qualitative

Quantitative

Infnrmation Handling Behavior of Designers During ConcepwaI Design: Three Experiments

Appendix 0: Additional Results on Information Handling Study 165

0.6.4 Information descriptor and subject-class

Table 0-4 Distribution of descriptor across the subject-classes. The numbers below represent the proportion of subject-classes in each of the information descriptor. Most alternatives were about design-concepts. Most assumptions were about assembly, feature and requirements. Most performance were about requirements. Most relations were about components. Most requirements were about assembly.

Descriptor Assem. Comp. Conn. Feature Reqmt. Other

Alternative 8 16 2

Assumption 7 8 8

Comparison 14 2

Construction 4 o Location 0 6

Operation 2 2 11 2

Performance

Rationale 17

-' .. ", ... '. '. ~"I 2 . . . - - ~ ~ '- - . - .

Relation 7 19 2

Requirement 7 11 15 7 o Miscellaneous 10 o 7 3 13



0.6.5 Information descriptor and level-of-abstraction

Table 0-5 Distribution of descriptor across the levels-of-abstraction. The numbers below represent the proportion of level-of-abstractions in each of the information descriptor. most alternatives were unlabeled or labeled. Most assumptions, comparison, performance were labeled. most operations were qualitative.

Descriptor Associative Qualitative Quantitative

Alternative 14 6 2

Assumption 18 13 6

Comparison 15 20

Construction 12 16 14

Location 9

Operation 5 2

Performance 7

Rationale 13 o Relation 20 6

Requirement 4 6

Miscellaneous 3 3 5

0.6.6 Information subject-class and level-of-abstraction

Table 0-6 Distribution of subject-class across the levels-of-abstraction. The numbers below represent the proportion of level-of-abstractions in each of the information subject-class. Most assembly information is labeled. Most components and connections are associated. Most requirements are labeled and most design-concepts were unlabeled.

Subject Class

Assembly

Component

Connection

Feature

Requirement

Design concept

Other

Associative

18

5

IO 13

9 9

Information Handling Behavior of Designers During ConceplUlll Design: Three Experiments

Quantitative

3

2

2

Appendix IE: References on Papers from this Dissertation 167

Appendix E

References on Papers from this Dissertation

E.l ASME Conference: DTM-1992

Baya, Vinod; Gevins, Jody; Baudin, Catherine; Mabogunje, Ade; Toye, George and Leifer, Leifer; "An Experimental Study of Design Information Reuse", In Proceedings of the 4th International Conference on Design Theory and Methodology, ASME, Scottsdale, Arizona, pp. 141-147, Sept. 13-16, 1992.

E.2 ASME Conference: DTM-1994

Baya, Vinod and Leifer, Larry J.; "A Study of the Information Handling Behavior of Designers During Conceptual Design", In Proceedings of the 6th International Conference on Design Theory and Methodology, ASME, Minneapolis, Minnesota, pp. 153-160, Sept. 11-14,1994.

E.3 ASME Conference: DTM-1995

Baya, Vmod and Leifer, Larry J.; "Understanding Design Information Handling Behavior Using TIme and Information Measure", In Proceedings of the 7th International Conference on Design Theory and Methodology, ASME, Boston, Massachusetts, Sept. 1995.

E.4 Chapter in Analysing Design Activity: 1996

Baya, Vmod and Leifer, Larry J., "Understanding Information Management in Conceptual Design", Chapter in Analysing Design Activity, Cross, N.; Christiaans, H. and Dorst, K. (Eds.), John Wiley & Sons, pp. 151-168, 1996.

Information Handling Bebavior of Desigoers During Conceptual Design: Three Experiments

Baya, ConceptDesign

Documents

Transcript of Baya, ConceptDesign