Post on 16-Feb-2016
description
INFORMATION TO USERS
This manuscript has been reproduced from the microfilm master. UMI
films the text directly from the original or copy submitted. Thus, some
thesis and dissertation copies are in typewriter face, while others may be
from any type of computer printer.
The quality of this reproduction is dependent upon the quality of the
copy submitted. Broken or indistinct print, colored or poor quality
illustrations and photographs, print bleedthrough, substandard margins,
and improper alignment can adversely affect reproduction.
In the unlikely event that the author did not send UMI a complete
manuscript and there are missing pages, these will be noted. Also, if
unauthorized copyright material had to be removed, a note will indicate
the deletion.
Oversize materials (e.g., maps, drawings, charts) are reproduced by
sectioning the original, beginning at the upper left-hand comer and
continuing from left to right in equal sections with smalI overlaps. Each
original is also photographed in one exposure and is included in reduced
form at the back of the book.
Photographs included in the original manuscript have been reproduced
xerographically in this copy. Higher quality 6' x 9" black and white
photographic prints are available for any photographs or illustrations
appearing in this copy for an additional charge. Contact UMI directly to
order.
UMI A Bell & Howell Information Company
300 North Zeeb Road. Ann Arbor MI 48106-1346 USA 313n61-4700 800/521-0600
INFORMATION HANDLING BEHAVIOR OF DESIGNERS DURING
CONCEPTUAL DESIGN: THREE EXPERIMENTS
A DISSERTATION
SUBMITIED TO THE DEPARTMENT OF MECHANICAL ENGINEERING
AND THE COMMITTEE ON GRADUATE STUDIES
OF STANFORD UNIVERSITY
IN PARTIAL FULLFILLMENT OF THE REQUIREMENTS
FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
IN
MECHANICAL ENGINEERING
Vinod Baya
September 1996
UMI Number: 9714076
Copyright 1996 by Baya, Vinod
All rights reserved.
UMI Microform 9714076 Copyright 1997, by UMI Company. All rights reserved.
This microform edition is protected against unauthorized copying under Title 17, United States Code.
UMI 300 North Zeeb Road Ann Arbor, MI 48103
© Copyright by Vinod Baya 1996 All Rights Reserved
ii
I certify that I have read this dissertation and that in my opinion it is fully adequate. in scope and qUality. as a dissertation for the degree of Doctor of Philosophy.
I certify that I have read this dissertation and that in my opinion it is fully adequate. in scope and quality. as a dissertation for the degree of Doctor ofP ·osophy.
I certify that I have read this dissertation and that in my opinion it is fully adequate. in scope and qUality. as a dissertation for the degree of Doctor of Philosophy.
~.
Approved for the University Committee on Graduate Studies:
iii
Abstract
This dissertation demonstrates an iterative Observe->Analyze->Intervene, design
research methodology to incrementally improve the understanding and the sup
port of information handling in the conceptual design process. It reports on two
observational studies, based on the verbal protocol method, and an information
management service.
The first observational study is a detailed analysis of the questioning behavior of
designers to understand information needs during a redesign task. This resulted
in the design information framework, which classifies information that should be
captured during a design process for effective reuse at a later date. This frame
work was used to develop an information management service called Dedal, a
tool for indexing, modeling and retrieving design information. Observations from
deployment and usage of Dedallead to the second observational study. This was
a study of the information handling behavior of individual designers doing short
conceptual design tasks. This study resulted in the Information Handling Frame
work (lliF).
The lliF is a framework for understanding information handling behavior during
conceptual design. Some key observations from the use of this framework are:
• designers move frequently between different types of information (on an average every 13 seconds),
• they handle information about upto 40 concepts in one minute, • their ability to work fluidly and with ease while handling all types of infor
mation is essential during the conceptual design process, and • the points of transition between different information types are critical from
the viewpoint of computational support.
Deeper understanding of the information handling behavior is explained by
means of detailed qualitative and quantitative results. The implications of these
results towards the improvement in the understanding of the conceptual design
process and the recommendations on the development of intuitive and integral
information handling services are discussed.
iv
Acknowledgements
I would like to thank Prof. Larry Leifer for his patient, generous and liberal guid
ance over the course of this dissertation. Larry, I thank you specially for sharpen
ing my communications and management skills and encouraging me to pursue
this line of research.
I would like to thank Prof. Mark Cutkosky, Prof. Sheri Sheppard and Dr. Cathe
rine Baudin for providing valuable feedback on the drafts of this dissertation.
Catherine, Thank you also for giving me an opportunity to work on Dedal.
I would like to acknowledge all the members of the Dead Designer Society: Ade
Mabogunje, David Cannon and Margot Brereton for many things, but specially
for friendship and support over all these years.
I would like to acknowledge students and staff at the Center for Design Research
for providing a very stimulating and amiable research environment. Noelle,
thanks for all the CDR-socials and the numerous other trips.
It would also like to thank my friends in the horse polo world, in and outside of
Stanford, who added a whole new dimension to my experience on the Farm.
Thank you Stanford Polo Club members and horses.
I would like to thank the AI Branch (now the Information Sciences Division) at
NASA Ames Research Center for supporting this research. Thanks also to
TUDelft and Xerox P ARC for providing data for one of experiments used in this
disserta tion.
Finally, I would like to thank my family in India for their patience, understanding
and support while I took my time completing this chapter of my life.
v
Table of Contents
Chapter 1. Understanding and Improving Engineering Design Practice .................................................. 1
1.1 Background: Engineering Design ................................. 2
1.1.1 Information Handling in Design ..... '" ...................... 2 1.1.2 Conceptual Design Focus .................................... 4 1.1.3 Research Objectives ......................................... 5
1.2 Research Methodology .......................................... 6
1.2.1 Grounding Research in Reality ............................... 6 1.2.2 Observation by Verbal Protocol .............................. 7 1.2.3 Improving Design Practice: Incremental Support ............... 8
1.3 Analysis of Verbal Protocol Data ................................. 9
1.3.1 Information Handling and Design Information Framework ..... 10 1.3.2 Information Reuse Study: Analysis Example .................. 12 1.3.3 Information Handling Study: Analysis Example ............... 12
1.4 Evolution of the Information Handling Perspective ................ 13
1.5 Key Results ................................................... 16
1.5.1 Key Results from Information Reuse Study ................... 16 1.5.2 Key Results from Dedal Development & Usage ................ 16 1.5.3 Key Results from Information Handling Study .. , ... " ........ 17
1.6 Knowledge Gained ............................................. 17
1.7 Thesis Organization ............................................ 18
Chapter 2. Design Theory & Methodology Research: Literature Re-vie'W' ............•.........•.............•........•...... 20
2.1 Design Theory and Methodology Research ....................... 21
2.1.1 Prescriptive Studies ........................................ 21 2.1.2 Descriptive (Empirical) Studies .............................. 24 2.1.3 Verbal (or Think-Aloud) Protocol Method .......... " ........ 26 2.1.4 Studies Using Verbal Protocol Method ....................... 28 2.1.5 Studies Using Other (Non-Verbal Protocol) Methods ........... 31
2.2 Design Process: An Information Handling Perspective ............. 32
vi
Chapter 3. Design Information Needs, Capture & Reuse ....... 35
3.1 Design Information ............................................ 36
3.1.1 Information Reuse Essential for Good Design Practice .......... 36 3.1.2 Design Information Capture is Difficult ...................... 37
3.1.3 Information Reuse Study Objectives ......................... 38
3.1.4 Questions Reflect Information Needs ......................... 39
3.2 Experimental Study: Verbal Protocol Method ..................... 39
3.2.1 Experimental Setup and Procedure .......................... 39
3.2.1.1 The Design Problem: Continuously Variable Damper .... 41
3.3 Analysis of Verbal Data ........................................ 42
3.3.1 Transcription ............................................. 42
3.3.2 Question Extraction ........................................ 43
3.3.3 Question Reformulation .................................... 43 3.3.4 Question Classification ..................................... 43
3.4 Question Classification Framework .............................. 44
3.4.1 Design Information Framework (DIF) ........................ 45
3.4.2 Descriptor ................................................ 45
3.4.3 Subject-Class .............................................. 45
3.4.4 Medium .................................................. 46
3.4.5 Level-of-Detail ............................................ 48
3.5 Analysis Examples ............................................. 48
3.6 Results ........................................................ 48
3.6.1 Results on Descriptors ..................................... 50
3.6.2 Results on Subject-Class .................................... 51
3.6.3 Results on Level-of-Detail .................................. 52
3.6.4 Results on Crossing Attributes .............................. 52
3.7 Discussion .................................................... 54
3.8 Recommendations for Information Reuse Service ................. 56
3.9 Summa.ry-...................................................... 56
Chapter 4. Dedal: An Information Management Service ........ 57
4.1 Introduction to Dedal ........................................... 58
4.1.1 Indexing Tool ............................................. 59
4.1.2 Modeling Tool ............................................ 60
4.1.3 Retrieval Tool. ............................................ 60
vii
4.2 Dedal Test Domains and Results ................................ 62
4.2.1 Damper Domain Results ................................... 63
4.2.2 Bioreactor Domain Results .................................. 65
4.3 Discussion on Dedal: Lessons Learned ........................... 66
Chapter 5. Information Handling Behavior Framework ........ 68
S.1 Introduction ................................................... 69
5.1.1 Experimental Setup and Procedure .......................... 69
5.2 Analysis Procedure ............................................. 70
5.2.1 Segmenting into Information Fragments ...................... 70 5.2.2 Classifying Information Fragments .......................... 71
5.3 Information Handling Framework (lHF) .......................... 72
5.3.1 Informational Activity ..................................... 74 5.3.2 Level of Abstraction ....................................... 76
5.3.3 Design Information Measure (dim) . .......................... 77
5.3.3.1 Definition of dim: Assumptions ....................... 78 5.3.3.2 Rules for measuring dims ............................. 79
5.4 Analysis Example .............................................. 80
5.5 Summary ...................................................... 82
Chapter 6. Understanding Information Handling Behavior: Quan-titative & Qualitative Results ............................. 83
6.1 Introduction ................................................... 84
6.2 Information Fragment Duration (deitat) .......•..•...•.........•. 85
6.2.1 deltat Variation over Time .................................. 85
6.2.2 Statistics on deltat . ......................................... 85 6.2.3 Discussion on deltat Results ................................. 86
6.3 Design Information Measure (dim) . ...........•......••....•..... 88
6.3.1 Information Handling Rate ......... , ....................... 88
6.3.2 Discussion on Design Information Measure Results ............ 91
6.4 Distribution of Time and Design Information ..................... 91
6.4.1 Informational Activities Distribution ......................... 91
6.4.2 Information Descriptor Distribution .......................... 92
6.4.3 Information Subject-Class Distribution ....................... 93 6.4.4 Information Level of Abstraction Distribution ................. 94
viii
6.4.5 Discussion on Time and Information Distribution .............. 95
6.5 State Transition Analysis: Statistics and Probabilities .............. 96
6.5.1 Informational Activity State Transition Analysis ............... 97 6.5.2 Information Descriptor State Transition Analysis .............. 97 6.5.3 Information Subject-Class State Transition Analysis ............ 99 6.5.4 Information Level-of-Abstraction State Transition Analysis ..... 99 6.5.5 Discussion on State transition Analysis ...................... 100
6.6 Multiple Attributes Analysis ................................... 101
6.6.1 Informational Activities and Information Descriptor .......... 101 6.6.2 Informational Activities and Information Subject-Class ........ 102 6.6.3 Informational Activities and Level-of-Abstraction ............ 102 6.6.4 Most Frequent Information Fragments ...................... 103
6.7 Summary ..................................................... 103
Chapter 7. Conclusions & Future Work ...................... 105
7.1 Concluding Remarks .......................................... 106
7.2 Contributions ................................................. 107
7.3 Discussion ................................................... 107
7.4 Future Work .................................................. 108
Bibliography .............................................. 110
Appendix A. Additional Material on Design Information Reuse Study ................................................. . 117
A.1 Design Experiment Material ................................... 117
A.I.1 Design Experiment Problem Statement ............. '" ..... 117 A.I.2 Description of ME210 ..................................... 120
A.2 Results on individual subjects ................................. 122
A.3 Analysis Examples ............................................ 123
Appendix B. Additional Material on Dedal Development ..... 126
B.1 Dedal Implementation Details ................................. 126
B.2 Dedal Interfaces for Indexing, Querying and Modeling ........... 127
B.3 Examples of Heuristics ........................................ 129
ix
B.4 References on Dedal papers .................................... 133
Appendix C. Additional Material on Design Information Handling Study ................................................. . 134
C.1 Instructions: Overview ........................................ 135
C.2 Design Exercise: Problem Statement Bike Lock 1 ................. 136
C.3 Design Exercise: Problem Statement Bike Lock 2 ................. 139
C.4 Design Expt. Problem Statement: Backpack Harness ........ _ ..... 142
Appendix D. Additional Results on Information Handling Study ................................................ 143
0.1 Additional Results on deltat . .................................. 143
0.1.1 deltat vs. ~e ............................................ 143 0.1.2 Distribution of deltat: histograms ........................... 148
0.2 Additional Results on Information Handling Rate ............... 149
0.2.1 Information handling rate over time ........................ 149 0.2.2 Distribution of information handling rate ................... 152 0.2.3 Dim rate vs. Verbal rate: Regression analysis ................ 153
0.3 Information Fragment Attributes Behavior Over Time ............ 153
0.4 Additional Results on Distribution of Time ..................... 158
0.5 Results on State Transition Probabilities ........................ 160
0.5.1 Informational activities state transition probabilities .......... 160 0.5.2 Information descriptor state transition probabilities .......... 161 0.5.3 Information subject-class state transition probabilities ........ 162 0.5.4 Information level-of-abstraction state transition probabilities .. 162
0.6 Additional Results from Multiple Attribute Analysis ............ 162
0.6.1 Information descriptor and informational activities ........... 163 0.6.2 Information subject-class and informational activities ......... 164 0.6.3 Information level-of-abstraction and informational activities ... 164 0.6.4 Information descriptor and subject-class .................... 165 0.6.5 Information descriptor and level-of-abstraction .............. 166 0.6.6 Information subject-class and level-of-abstraction ............ 166
Appendix E. References on Papers from this Dissertation ..... 167
E.1 ASME Conference: DTM-1992 .................................. 167
x
E.2 ASME Conference: DTM-1994 .•................................ 167
E.3 AS ME Conference: DTM-1995 .................................. 167
E.4 Chapter in Analysing Design Activity: 1996 ...................... 167
xi
List of Figures
Figure 1-1 Strategy for grounding research in reality ....................... 7
Figure 1-2 Strategy for incrementally improving design practice ............. 8
Figure 1-3 Template for classifying information fragments. . . . . . . . . . . . . . . .. 13
Figure 1-4 Evolution of the work presented in this dissertation.. . . . . . . . . . . .. 14
Figure 3-1 An information perspective on the design process. . . . . . . . . . . . . .. 37
Figure 3-2 Experimental setup for information reuse study .... " ........... 40
Figure 3-3 Analysis procedure for information reuse study ................. 42
Figure 3-4 Steps in the development of question classification framework. . .. 44
Figure 3-5 Template for interpreting question and information segment.. . . .. 46
Figure 3-6 Distribution of question deScriptor ............................ 50
Figure 3-7 Distribution of question subject-class. ........... . . . . . . . . . . . . .. 51
Figure 3-8 Distribution of question level of detail. .......... . . . . . . . . . . . . .. 52
Figure 4-1 Dedal system architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 58
Figure 4-2 Example of conceptual indexing ............................. " 60
Figure 4-3 Example of a model in Dedal. ................................ 61
Figure 4-4 Flow chart of the retrieval algorithm used by Dedal.. . . . . . . . . . . .. 61
Figure 4-5 Retrieval performance of conceptual indexing.. . . . . . . . . . . . . . . . .. 64
Figure 5-1 Analysis procedure for information handling study .............. 71
Figure 5-2 Template for classifying an information fragment. .............. 72
Figure 5-3 Evolution of the informational activity classification. . . . . . . . . . . .. 75
Figure 6-1 Information fragment duration vs. time for S1. ................ " 86
Figure 6-2 Distribution of information fragment duration ................ " 87
Figure 6-3 Information handling rate over time for 54 .................... " 89
Figure 6-4 Distribution of time and design information across informational activ-ities ...................................................... " 92
Figure 6-5 Distribution of time and design information across descriptor ... " 93
Figure 6-6 Distribution of time and design information across subject-class. " 94 Figure 6-7 Distribution of time and information across level-of-abstraction. " 95
Figure 6-8 Variation in level-of-abstraction over time for 54 .............. " 96
Figure 6-9 Informational activity state transition statistics ................ " 98 Figure 6-10 Information descriptor state transition statistics .............. " 98
Figure 6-11 Information subject-class state transition statistics ............ " 99
Figure 6-12 Information level-of-abstraction state transition statistics. . . . . .. 100
Figure A-I Distribution of question descriptor for RS1 and RS2. . . . . . . . . . .. 122
xii
Figure A-2 Distribution of question subject-class for RS1 and RS2. . . . . . . . .. 122
Figure A-3 Distribution of question level-of-detail for RS1 and RS2. . . . . . . .. 123
Figure B-1 Dedal's indexing and querying environment. ................. 127
Figure B-2 Dedal's modeling interface .................................. 128
Figure B-3 Dedal's answers window. .................................. 129
Figure D-1 Behavior of deltat over time for 51. . . . . . . . . . . . . . . . . . . . . . . . . .. 144
Figure 0-2 Behavior of deltat over time for 52. . . . . . . . . . . . . . . . . . . . . . . . . .. 144
Figure 0-3 Behavior of deltat over time for 53 ........................... 145
Figure 0-4 Behavior of deltat over time for 54. . . . . . . . . . . . . . . . . . . . . . . . . .. 145
Figure 0-5 Behavior of deltat over time for 55 ........................... 146
Figure 0-6 Behavior of deltat over time for 56. . . . . . . . . . . . . . . . . . . . . . . . . .. 146
Figure 0-7 deltat behavior over time for all exp-subjects .................. 147
Figure 0-8 Distribution of the information fragment duration ............. 148
Figure 0-9 Information handling rate over time for 51 ................... 149
Figure 0-10 Information handling rate over time for 52 .................. 150
Figure 0-11 Information handling rate over time for 53 .................. 150
Figure 0-12 Information handling rate over time for 54. . . . . . . . . . . . . . . . . .. 151
Figure 0-13 Information handling rate over time for 55 ............... . .. 151
Figure 0-14 Information handling rate over time for 56 ................... 152
Figure 0-15 Distribution of dim rate for all exp-subjects. ................. 152
Figure 0-16 Regression analysis between dim rate and verbal rate ......... 153
Figure 0-17 Behavior of informational activity over time for 51. . . . . . . . . . .. 154
Figure 0-18 Behavior of Information descriptor over time for 52. . . . . . . . . .. 155
Figure 0-19 Behavior of Information subject-class over time 53. . . . . . . . . . .. 156
Figure 0-20 Behavior of Information level-of-abstraction over time 54. . . . .. 157
Figure 0-21 Distribution of time across informational activity. . . . . . . . . . . .. 158
Figure 0-22 Distribution of time across information descriptor. ........... 158
Figure 0-23 Distribution of time across information subject-class.. . . . . . . . .. 159
Figure D-24 Distribution of time across information level-of-abstraction. ... 159
Figure 0-25 Information activity state transition probabilities.. . . . . . . . . . . .. 160
Figure 0-26 information descriptor state transition probabilities. . . . . . . . . .. 161
Figure 0-27 Information subject class state transition probabilities. ..... . .. 162
Figure D-28 Information level-of-abstraction state transition probabilities. .. 163
xiii
List of Tables
Table 1-1 Analysis procedure ........................................... 10
Table 1-2 Information handling and design information framework. ......... 11 Tabie 2-1 Empirical design research methods .............................. 25
Table 3-1 Framework to classify questions ................................. 45 Table 3-2 Question descriptors .......................................... 47
Table 3-3 Question subject-class ......................................... 47
Table 3-4 Question level-of-detail ........................................ 48
Table 3-5 Example of analysis procedure in information reuse study .......... 49
Table 3-6 Information reuse study: subjects background information .......... 50
Table 3-7 Crossing descriptor with subject-class ............................ 53
Table 3-8 Crossing descriptor with level-of-detail .......................... 54 Table 3-9 Crossing subject-class with level-of-detail.. ....................... 54 Table 4-1 Dedal test domains details ...................................... 63
Table 5-1 Details on experiments and experimental subject background ....... 70 Table 5-2 Information handling framework (ll-IF) .......................... 73
Table 5-3 Descriptors added to the design information framework ............ 74 Table 5-4 A classification for the Informational activities .................... 76 Table 5-5 Definition and interpretation for level-of-abstraction ............... 77 Table 5-6 Examples of dim measurement .................................. 80
Table 5-7 Examples of information handling behavior analysis ............... 80
Table 6-1 Information fragment duration (deltat) statistics ................... 87
Table 6-2 Information handling rates and verbal rates for all subjects ......... 90 Table 6-3 Distribution of information activities across descriptors. . ......... 101
Table 6-4 Distribution of informational activity across subject-class .......... 102
Table 6-5 Distribution of informational activities across level-of-abstraction ... 102
Table 6-6 Frequently encountered info-fragment attribute combinations ...... 103
Table 6-7 Summary of the key findings and recommendations .............. 104 Table A-I Example of analysis procedure in information reuse study ......... 124 Table 0-1 Distribution of descriptor across the informational activities ....... 163 Table 0-2 Distribution of subject-cass across the informational activities ..... 164 Table 0-3 Distribution of level-of-abstraction across informational activities .. 164 Table 0-4 Distribution of deScriptor across the subject-classes ............... 165 Table 0-5 Distribution of descriptor across the levels-of-abstraction. . ....... 166 Table 0-6 Distribution of subject-class across the levels-of-abstraction ....... 166
xiv
Glossary
Here are some common abbreviations and terms used in this dissertation:
OAl Observe ->Analyze -> Intervene iterative cycle
OIF Design Information Framework
IHF Information handling Framework
EDN Electronic Design Notebook
ms Information Reuse Study
IHS Information Handling Study
DS [DeScriptor, Subject] pair in information fragment classification
dim Design Information Measure (quantitative measure)
dpm dims per minute
wpm words per minute
deltat Information fragment duration in seconds
xv
Chapter 1
Understanding and Improving Engineering Design Practice
Design Research
TO
,
Develop Research Methods
< Goal > shortest path
T~ Understand Design Practice
I~
Deliver High Quality, Low Cost Products
~
TO
. Improve Design
Practice
~o
This chapter is a summary of the research work being reported in this dissertation. It discusses the evolution of this dissertation and the motivation for pursuing the research. It briefly describes the experimental approach and analysis procedure, developed to carry out two observational studies, namely information reuse study and information handling study. It lists the contributions that this dissertation is making to the knowledge in the field of design theory and methodology. It ends with a narration on the content of the remaining chapters.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments 1
Chapter 1: Understanding and Improving Engineering Design Practice 2
1.1 Background: Engineering Design
Engineering design is a complex process involving people, multiple technical
and non-technical disciplines, material and non-material resources, computa
tional and non-computational tools and a variety of cognitive strategies. As a
result, studying, understanding and supporting this process is very challenging.
Given its complexity, there is a potential for substantial savings in cost, labor
and resources by improving engineering design practice. Improvement can be
achieved by supporting design practice with tools and services which accrue sav
ings over the course of the process. Economic pressures to shorten product
development cycles and improve market responsiveness have increased interest
in the study of design theory and methodology both in industry, in academia
and in design education. Developing methods to study and understand engi
neering design, and using this understanding to build services! to support this
process is essential for developing high quality products at low cost. This thesis
addresses the need for new methods required in design research for improving
design process understanding.
Design process can be studied from many perspectives such as, technical, com
putational, problem solving, organizational, social and others. Each perspective
has a potential to lead to a better understanding of the process and lead to
improved design practice. I have chosen to approach the engineering design pro
cess from an information handling viewpoint.
1.1.1 Information Handling in Design
In this dissertation information handling refers to activities and sub-processes
undertaken by the designer which involve design information2. For instance
1. The term services will be used in this thesis to refer to tools and(or) methods (computational or non-computational) that can support engineering design practice. 2. The terms design information and infonnntion will be used synonymously in this thesis unless stated otherwise.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 1: Understanding and Improving Engineering Design Practice 3
activities dealing with generating, capturing, accessing, transforming, indexing,
structuring and analyzing information with the intent to create an artifact, are
some examples of information handling. While it is possible to construe all
actions of a designer as information handling, for the purpose of this study I
only consider the actions pertinent to the design task on hand.
Engineering design is an information intensive process. Over the duration of a
design project designers handle a large amount of information. The information
is handled by means of numerous non-computational (paper notebooks, cata
logs, handbooks, telephone etc.) as well as computational tools (CAD /CAE/
CAM software, word processors, spreadsheets, e-mail etc.). The Information
exists in a variety of media such as: text, graphic, audio, video, CAD drawings
and others. This diversity imposes considerable burden of information manage
ment on designers. As a result, the efficiency and quality of the design process
depends heavily on how capable designers are in handling information.
To find out about the information requirements of engineering designers, [Court
et. a11993] conducted a questionnaire based survey of about 200 designers from
various industries located in the United Kingdom. They found that on average
designers spent 18% of their time searching for information, 23% dealing with
paperwork, 16% in meetings and only 43% of their time designing. Hales [1987]
performed a detailed analysis of all the activities of one project involving the
design of a high-pressure, high-temperature system for testing materials. He
was a participant observer! in this project over a period of 2.8 years and involv
ing 37 different participants. Hales found that only 47% of the total effort was
spent in doing design. The remaining 53% was spent in information retrieval,
planning, cost estimating, reviews, helping others and social contact. These two
studies strongly suggest that there is an opportunity to improve design practice
by supporting information intensive activities such as, information gathering,
1. An observational technique in which the observer is also a member (participant) of the design team.
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
Chapter 1: Understanding and Improving Engineering Design Practice 4
documentation, communication, information management and others. A reduc
tion in information related activities will lead to an improvement in designers
capacity to do better designs.
All over the world, there is a steady movement towards electronic commerce.
Much of information that is available today in paper medium (design notebooks,
libraries, handbook, catalogues etc.) will be available in electronic formats over
computer networks. As a result, many transactions which today are paper based
will become electronic (group communications, interaction with vendors etc.).
This transition will significantly impact design practice in industry. Designers
will have easy access to a large amount of information, resulting in increased
information management burden. We need to make sure that services that are
made available to designers in the electronic workplace integrate smoothly with
their work environment, so that designers can handle information efficiently.
This will be possible if services are built based on a good understanding of the
information handling behavior of designers. Developing this understanding is
the core concern in this dissertation.
1.1.2 Conceptual Design Focus
Among the commonly agreed upon phases of the design process, conceptual, lay
out and detail [Cross 1990, Page 21], the conceptual phase should deserve special
attention from an information handling perspective. Conceptual phase refers to
the ideational stage of the design process where ideas are generated and analy
sed, and choices are made. In this stage, ideas and solutions are described
abstractly without quantitative detail. While conceptual activity takes place all
along the design process [Guindon 1990], it occurs more frequently in the early
stage of the process. It is this early stage that is of interest in this dissertation.
The conceptual phase happens to be the one where important decisions are
made [Wood and Antonsson 1987]. Also, it is the most informal, and the least
Infor=tion Handling Behavior of Designers During ConccpruaJ Design: Three Experiments
Chapter 1: Understanding and Improving Engineering Design Practice 5
understood [Rabins et al. 1986]. It is also the stage for which there is little compu
tational support [Rouse and Boff 1987]. Decisions and information generated in
this phase have a big impact on the downstream design process and the overall
cost. It has been reported that upwards of 80% of the final cost are obligated dur
ing conceptual design [National Materials Advisory Board 1991]. This means
that understanding and supporting information handling in this phase is very
likely to lead to Significant betterment of design practice. This is why this disser
tation focuses on conceptual design.
1.1.3 Research Objectives
The primary objectives of this dissertation are to model the information han
dling behavior of designers during conceptual design, and develop
requirements to guide the evolution of information handling support services. A
secondary objective is to develop a research methodology for understanding and
improving engineering design practice. Several questions have guided this
inquiry:
1. What is design information?
2. Can design information be classified?
3. Are design information needs during conceptual work special?
4. Are some types of design information more important than others?
5. What are the activities designers perform with design information?
6. vVhat is the temporal distribution of design information processing?
7. Can design information be quantitatively measured?
8. What is the information content in different activities?
9. How does information processing rate change with the design process? and,
10. What do answers to the above questions inform us about services needed to support information handling in the design process?
Information Handling Behavior of Designers During Conceplual Design: Three ExperimenlS
Chapter 1: Understanding and Improving Engineering Design Practice 6
1.2 Research Methodology
An underlying concern in this study is to build services which would support
and improve design practice. This required that the research methodology give
adequate attention to, Ca) understanding the design process and (b) development
of computational services. Three research strategies were used in this study to
maintain thls balance,
1. Grounding the research in reality: the research paradigm was grounded in reality by experimenter's involvement in real design practice.
2. Understanding the design process: design process was examined in detail by means of observational studies, and
3. Improving design practice: design process was incrementally supported by a prototype service and the performance of designers using the prototype service was assessed.
These three strategies are described in detail in the subsections below. In the dis
cussion observational study refers to the method adopted in this dissertation to
understand the design process. Two observational studies were conducted, one
to understand information reuse and the other to understand information handling
behavior.
1.2.1 Grounding Research in Reality
A special feature of the research presented in this dissertation is shown in
Figure 1-1. The figure is meant to indicate that the researcher should have a rich
involvement in real design activity during observational studies and service
development. The involvement in real design activity is motivated by the con
cern for assuring that formal observations from the experiments are consistent
with experiences from real design activity, and that during service development
there is constant feedback from an ongoing design project.
Results from observational studies are the basis for guiding service develop
ment, and feedback form service development to observational studies is only
Information Handling Behavior of Designers During Conccptual Design: Three ExperimcnlS
Chapter 1: Understanding and Improving Engineering Design Practice
Service Development -Dedal
Observational Studies (formal observation)
- Information Reuse - Information Handling Behavior
Real Design (informal observation)
- Variable Damper - Bioreactor
Figure 1-1 Strategy for grounding research in reality. Double arrow 1 is a means for assuring that formal observations from the experiments are consistent with informal observations made from real design activity. Double arrow 2 is a means for assuring that during service development there is constant feedback from an ongoing real design project. Single arrow 3 indicates that service development is based on results of observational studies.
7
through real design activity (since a service has to be incorporated in real design
activity before its performance can be examined in an observational study). Alto
gether this strategy provides a mechanism for grounding the research in reality.
During the evolution of this dissertation, which covers two observational studies
and development of a computational service (Dedal), I was a member of the two
design projects, Variable Damper and Bioreactor, as shown in Figure 1-1.
1.2.2 Observation by Verbal Protocol
Observational studies of the design process can be conducted using various
methods. Some of the popular methods are discussed and compared in Chapter
2. This study used the verbal protocol method. This method is popular in single
subject experiments and has been used widely by cognitive scientists [Newell
1968] and design researchers [Eastman 1970; Akin 1979 and Stauffer 1987b]. In
verbal protocol method, experimental subjects are asked to talk out loud while
they work on a design problem. In this manner the actions and intentions of the
subject become explicit in the verbal recording. The transcript of this recording
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 1: Understanding and Improving Engineering Design Practice 8
is the raw data for analysis.
There is debate in the research community regarding the appropriateness of this
method, since it is not understood in what manner and how much the act of ver
balizing affects the natural thought process and the behavior of the experimental
subject. Despite these limitations, verbal protocol is a popular method for single
subject studies because of its relative simplicity and tacit assurance that what is
said "is at least subjects own words". Procedures and guidelines have been
developed for collecting and analyzing such data so that the results obtained are
valid and free from bias [Ericsson and Simon 1993]. The appropriateness of this
method is discussed further in Chapter 2.
1.2.3 Improving Design Practice: Incremental Support
A strategy for improving design practice by incrementally supporting the design
process! is shown in Figure 1-2. This approach has been advocated and used by
Analyze Analyze
! Observe Intervene --.~~ Observe Intervene --.~~ Observe
Improved Design Practice -----.~~
Figure 1-2 Strategy for incrementally improving deSign practice. The design process is cyclically observed, analyzed and intervened while incrementally developing services to facilitate and improve design practice.
[Tang 1989 and Minneman 1991] in their research in improving collaborative
1. It is an assumption that incremental improvements in understanding and support of the design process will lead to improved design practice over time. This is not being proved in this thesis.
Information Handling Behavior of Designer.; During Conceptual Design: 1breo: ExperimenlS
Chapter 1: Understanding and Improving Engineering Design Practice 9
design practice. The fundamental assertion of this methodology is that hypothe
sis free observation of design activity must precede the development of services
intended to support that activityl.
The process starts by conducting observational studies of design activity to col
lect raw data. The data is analyzed to gain an understanding of the design
activity and to develop frameworks and requirements for services which will
improve the observed design practice. Using these frameworks and require
ments, services are built and introduced into the original design process. The
introduction of the service into the design process is an intervention, since it
changes the original process itself. A second cycle of observational study is
needed to understand the changed process. The cycle of observe -> analyze ->
intervene (OAl) goes on, and the design process is incrementally supported,
restructured and improved.
1.3 Analysis of Verbal Protocol Data
A total of eight verbal protocol experiments were conducted. Two were con
ducted in the information reuse study and six in the information handling study.
The recordings from the experiments were analyzed using the procedure out
lined in Table 1-1. The purpose of the analysis was to reduce the raw data into a
form where the design behavior could be quantitatively analyzed.
During the experiments the verbalizations were recorded both on audio and
video tapes2. The transcription process transformed the audio and video data
into a transcript whi.:h would contain the raw data for analysis. Video data was
useful in numerous situations to resolve ambiguities and establish context dur
ing transcription. The segmentation process fragmented the raw data such that
1. The approach of deriving a hypothesis from collected data rather than confirming a pre-formulated hypothesis. 2. The video tapes had an audio track as well. The audio tapes were used as backup and for facilitating transcription activity.
Information Handling Behavior of Designers During Conceprual Design: Three Experiments
Chapter 1: Understanding and Improving Engineering Design Practice 10
Table 1-1 Analysis procedure for transforming raw data into quantitative descriptions of the designer behavior.The analysis steps were similar for the two studies. The differences in the implementation of the steps are compared below .
Analysis Steps Information Reuse Study Information HandJing Study
conductexper.rrnen~: to record verbalizations to record verbalizations
transcribe and time stamp to get raw data to get raw data
segment transcription into questions into information fragmen~
rephrase segment rephrase question rephrase fragment
classify fragmen~ using design information fraDle- using information handling
work(D1F) framework (IHF)
quantitative analysis statistical, first -order statistical, correlation, regres-sion, state-transition, first-order
each chunk represented a unit of an interesting observation. In the information
reuse study this unit was a question and in the information handling study this
unit was an information fragment. In the next stage of the analysis these units
were classified. The framework used for classification was constructed as the
analysis progressed. It is important to point out here that the analysis was not
approached with a pre-existing framework. The framework evolved iteratively
as an outcome of the analysis procedure. The classifications were quantitatively
analyzed, the results of which are discussed later in this chapter. Each step of the
analysis procedure will be discussed in detail in Chapter 3 and Chapter 5.
1.3.1 Information Handling and Design Information Framework
Table 1-2 shows the information handling GIld the design information frame
work which resulted from analysis. This framework is a classification of the
activities designers undertake with information and a classification on design
information handled in those activities. Design information was classified using
four attributes: Descriptor, Subject-class, Medium and Level-of Detail. Four more
attributes were added to extend this framework to information handling. These
were, informational-activity, level-of-abstraction, information fragment duration
(deltat) and design information measure (dim). The attributes are briefly defined
below and the values these attributes can take are shown in Table 1-2.
Information Handling Behavior of Designers During Conccplunl Design: Three Experiment'
Chapter 1: Understanding and Improving enginesring Design Practice 11
Table 1-2 Information handling and design information framework.
Information HandIiDg Framework (IHF)
Informational Design Information Framework (DIF)
Activity Level of Abstraction Desaiptor Subject Class Medium Level of Detail
Generate Unlabeled Alternative Assembly Audio Conceptual
Access Labeled Assumption Component Video Configurational
Analyze Associative Comparison Connection Text Detail
Qualitative Construction Feature Graphic
Quantitative Location Requirement Gesture
Information Fragment Duration (sec) Operation Design-concept
Quantitative measure in seconds Performance Other
Rationale
Design Information Measure (dim) Relation
Quantitative measure: takes integer Requirement values (1,2,3, and so on) Miscellaneous
Informational-activity. This is the activity (generate, access or analyze) that the
designer performs with information.
Level-of-Abstraction. This represents the level at which the information is in it
evolution from abstract to concrete.
Design information measure. This is a quantitative measure of the amount of
semantic information in an information fragment.
Descriptor. This refers to the basic constitution or the nature of the information
in an information fragment.
Subject-Class. This is the class (assembly, component, connection etc.) to which the
subject1 of the sentence representing the information fragment belongs.
Medium. This is the physical form (text, graphic, audio, video, etc.) in which the
information fragment exists.
1. The word subject is being used in two contexts in this dissertation. One use is in the context of subjectclass and the other in the context of the experimentalSIlbjects. In most cases the context would be sufficient to impart the right meaning. In other cases the use will be explicitly stated to avoid any possible confusion.
Information Handling Behavior of De.~igners During Conceptual De.~ign: Three ExpcrimcnlS
Chapter 1: Understanding and Improving Engineering Design Practice 12
Level of Detail. It is the phase of the design process (conceptual, configuration or
detail) to which the information fragment can be associated with.
1.3.2 Information Reuse Study: Analysis Example
A unit of interesting observation in the information reuse study was a question.
A question is any utterance by the experimental subject expressing a need for
information. Below is an example of a question from the protocol and its
analysis.
QUESTION FROM PROTOCOL:
To find in here where you figured out, or how you figured out your force requirements. basically the force requirements as they relate to the diameter of these plates.
QUESTION REPHRASED:
How does the force requirement relate to the diameter of the friction disks?
INTERPRETATION:
The designer needs DETAIL information regarding the RELATION between the FEATURES (force requirement) and the (diameter of friction disks) .
CLASSIFICATION:
Descriptor: RELATION Subject Class: Feature Level of Detail: DETAIL Medium: TEXT
1.3.3 Information Handling Study: Analysis Example
A unit of observation in the information handling study is an information frag
ment. An information fragment is a continuous period during which the
informational activity and, the descriptor of information, remains the same.
Figure 1-3 shows the template that was used for classifying each information
fragment.
An example of an information fragment and its classification is:
INFQRMA TION FRAGMENT:
I guess the idea about the strength of these lock is that the lock should be as strong as the bike ...
CLASSIFIED FRAGMENT:
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 1: Understanding and Improving Engineering Design Practice
(info-fragment :activity informational-activity :time (stan-time end-time) :descriptor descriptor :subjclass subject-class :subjects (subjects-list) :medium medium :levels (level-of-detail level-of-abstraction) :measures (design-information-measures:dim) )
Figure 1-3 Template for classifying information fragments in the information handling study. The figure shows the attributes that are given values to classify an information fragment. The fields take vcllues from the information handling framework (Table 1-2).
( info-fragment :activity GENERATE :time (00572007 00573300) :descriptor ASSUMPTION :subjclass FEATURE :subjects (Strength-or-Lock) : medium AUDIO :levels (CONCEPTUAL ASSOCIATIVE) : measures (1)
1.4 Evolution of the Information Handling Perspective
13
The evolution of this study is captured in Figure 1-4. This figure maps the evolu
tion of this dissertation on top of the methodology presented in Figure 1-2. Note
that this figure represents a rationalization of four years of research and while it
closely represent the temporal sequence of the work, many tasks were carried
out concurrently (Figure 1-1).
My early years of doctoral work were devoted towards a project called Genera
tion and Conservation of Design Knowledge (GCDK) [Leifer 1989]. The major
concerns of this project were to capture and document design knowledge/infor
mation so that it could be available for easy reuse at a later time. The concern for
capturing design information in an electronic medium tied this project to the
Information Handling Behavior of Designers During Conceptual Dcsign: Three Experiments
Chapter 1: Understanding and Improving Engineering Design Practice
/-------( Information Classification \ \ Framework I ------_/
/-----, { Dedal Performance \ \. Results /
,,-----, { Information Handling \ \. Framework /
t t t ------------------------------Analyze Analyze Analyze
1~~'9':f~~~m /9'-'_,,\ /.~-'9~\
14
Information Support by Observe Support by Information Reuse Study Dedal ---II~~ Dedal Use Dedal - ...... ~ .. Handling
Study
Understanding of Information Handling
Figure 1-4 Evolution of the work presented in this dissertation. This covers two observational studies and development of a computational service called Deda!.
development of an Electronic Design Notebook™ (EDN'fM) . This notebook was
built on top of ~ text-graphic editor called vmacs™ [Lakin et al. 1989]1. The
hypothesis motivating this research was that design knowledge conservation
(capture, representation, reasoning and retrieval) is far less costly than re-inven
tion of comparable knowledge [Leifer 1991].
The first question faced was: Of all the information designers handle, what infor
mation do designers reuse? To find an answer to this question I conducted the
observational study on information reuse. Two experiments were conducted and
questioning behavior of the experimental subjects was analyzed. This resulted in
the design information framework (DIF) which classifies design information that
designers need during redesign2. The experiments and resulting framework are
discussed in Chapter 3.
Based on the results of the information reuse study, an information management
system called Dedal was developed3. Dedal is an information management tool
1. Electronic Design Notebook, EDN and vrnacs are trademarks of The Performance Graphics Company. 2. The experiments were carried out in collaboration with Catherine Baudin and Jody Underwood from NASA Ames Research Center and Ade Mabogunje from Stanford University.
Information Handling Behavior of ne.,igners During Conceptual Design: Three Experiment'
Chapter 1: Understanding and Improving Engineering Design Practice 15
which integrates with an information capture medium such as an Electronic
Design Notebook to provide designers with computational services for indexing,
modeling and retrieval of design information. Dedal emphasizes managing infor
mation incrementally and in real time. It uses the framework in Table 1-2 to
implement a conceptual indexing scheme. Development and performance of
Dedal are discussed in detail in Chapter 4.
When evaluating the impact of Dedal on design practice, I learned that the activi
ties of indexing and modeling using Dedal were add-on activities, requiring
additional effort. This was a symptom of lack of adequate integration of Dedal
with the work habits of designers and the design process. I was faced with many
new questions. The critical question was: How can we support information han
dling so that it is intuitive and integrates smoothly with the design process? I
realized that there was very little understanding of the work habits of designers
relating to the dynamics of information handling, Le. information handling
behavior over time, activities performed with information, nature of information
handled in these activities, amount of information handled and others. I was con
vinced that knowledge of answers to these questions was critical to developing
services which were intuitive and integrated smoothly with design process.
In search of answers to the above questions I conducted the infonnation handling
study. In this study I conducted six experiments using the verbal protocol
method. Analysis of the experimental data resulted in the information handling
framework (llIF). To determine the information handling rates, I defined a quan
titative measure for design information using the framework in Table 1-2. This
measure is called design infonnation measure or simply dim. The experiments, anal
ysis method, dim and results of this study are discussed in Chapter 5 and
Chapter 6.
3. Dedal was built in collaboration with Catherine Baudin and lady Underwood from NASA Ames Research Center and Ade Mabogunje from Stanford University.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 1: Understanding and Improving Engineering Design Practice 16
1.5 Key Results
Experiments involving human subjects are a challenge to monitor and control,
since results are easily influenced by variables such as human subjects, design
problem and experimenter. Verbal protocol analysis is tedious and time inten
sive, which makes it impractical to conduct and analyze a statistically significant
number of experiments. The analysis procedure is subjective and there are many
questions regarding the reliability of the results from such an analysis. Being con
scious of these shortcomings, I have built guidelines at each stage to maintain
objectivity and rigor in analysis. I have tried to explain some of the results
obtained either from prior understanding of the design process or from peculiari
ties of the experimental/ analytical method I used. I have used the remaining
results to push the edges of my understanding of the design process. Below I dis
cuss the key results from each of the observational studies and development of
Dedal.
1.5.1 Key Results from Information Reuse Study
The information reuse study resulted in some recommendations for developing
computational support for design information capture. Some key results are:
• the design information framework, Table 1-2, classifies the design information that should be captured for effective reuse.
• the framework was adequate to classify 80% of the questions encountered in the experiments
• information with descriptor performance, relation, construction and operation were encountered frequently (57%), and should get special attention during capture
• information with subject-class feature, component and requirement were encountered frequently (68%), and should get special attention during capture
1.5.2 Key Results from Dedal Development & Usage
Dedal was tested to measure its retrieval performance. Also it was deployed in a
Information Handling Behavior of Dcsigners During Conceptual nc.<ign: Three Experiment<
Chapter 1: Understanding and Improving Engineering Design Practice 17
design project to evaluate real time usage. Here are some key results:
• retrieval performance of the conceptual indexing scheme implemented in Dedal (precision = 80%; recall = 60%) was better than keyword based information retrieval systems (precision = 20%; recall = 50%).
• information framework was adequate to capture information necessary for effective reuse
• the classification in the DIP was intuitive to designers, hence they were able to index, model and formulate queries
• indexing and modeling integrated poorly with the designer work habits, requiring designer to spend additional time indexing and modeling
1.5.3 Key Results from Information Handling Study
The information handling study gave a detailed quantitative insight into the
dynamics of information handling behavior of designers. Some key results are:
• designers spend between 2 to 35 seconds with one information type. On an average they changed the information they were handling every 13 seconds.
• the global information handling rate (dim rate) for the six experimental subjects ranged from 6.2 to 9.2 dims/minute (average being 8.1 dims/minute)
• the local information handling rate (dim rate for each of the information fragments) ranged between 2 to 40 dims/minute. About 40% of the variation in dim rate can be ascribed to the variations in verbal rate (words per minute spoken by experimental subjects)
• conceptual tasks are generation intensive. 55% of design time and 53% of design information are handled while generating new information
• more than 65% of time and design information were accounted for in descriptors [construction, alternative, operation, requirement]; in subject-class [assembly, component, design-concepts] and in non-quantitative level-of-detail
• regarding transitions, more than 82% were to and from generate for informational activity; 29% were to and from construction for descriptors; 54% were to and from [assembly, component] for subject-class and more than 90% were to and from non-quantitative level-of-abstraction
1.6 Knowledge Gained
This dissertation adds knowledge to the field of design research. It demonstrates
Information Handling Behavior of Designers During Conceplua[ Design: Three Expcrimcms
Chapter 1: Understanding and Improving Engineering Design Practice 18
a quantitative experimental method for studying and understanding the engi
neering design activity. It illustrates a methodology for incrementally improving
design practice. It is based on observational studies that remain grounded in real
design. This dissertation makes three key contributions, it
• demonstrates a framework for characterizing the information handling behavior of designers. This framework effectively classifies 80% of the design information that should be captured during the design process for adequate reuse.
• demonstrates a computational implementation of the information framework that is intuitive for designers and works in real time.
• provides a quantitative as well as qualitative understanding of the information handling behavior of designers and makes recommendations on how this understanding can be incorporated in building services to improve design practice.
1.7 Thesis Organization
Chapter 2 surveys the literature in the field of design research. It looks at the
methods used in studying and understanding the design process. It discusses
the appropriateness of the verbal protocol approach as a design research
method. It also cites studies that have impacted the evolution of this thesis.
Chapter 3 describes the information reuse study. It describes the analysis proce
dure and the evolution of the design information framework (DIF).
Chapter 4 describes the computational service Dedal. It discusses how Dedal
was used in a design project and what lessons were learned from its deployment
and usage.
Chapter 5 describes the information handling behavior study. It describes the
analysis procedure and the development of the information handling framework
(lliF).
Chapter 6 describes the quantitative and qualitative results of the information
Information H:mdling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 1: Understanding and Improving engineering Design Practice 19
handling study. It discusses the understanding gained and suggests recommen
dations for the design and implementation of information support services.
Chapter 7 discusses the conclusions of this study. It describes the key contribu
tions of this work and some directions for further research.
The bibliography provides a list of research papers, articles and books that have
influenced the evolution of this dissertation.
Appendix A is a collection of supporting material from the information reuse
study. It includes the design problem used in the experiment, samples of raw
data, examples of analysis and some results.
Appendix B is a collection of supporting material on Dedal implementation and
usage.
Appendix C is a collection of supporting material from the information handling
study. It includes the design problem used in the experiments, samples of raw
data and examples of analysis.
Appendix 0 is a collection of additional results from the information handling
study which are not included in the main body of the thesis.
Appendix E lists the key papers published in the past 4 years. These papers
reflect the evolution of this dissertation.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 2
Design Theory & Methodology Research: Literature Review
Design Research I..J/"""-~~-'~> Deliver High Quality, \. Goal ~,~--~," Low Cost Products
~ jl
TO TO
" \ shortest path
Develop Research ~ Improve Design Methods ______ Practice
~ Understand Design ~o Practice
This chapter reviews the literature in the field of design theory and methodology that has influenced the work reported in this dissertation. Prescriptive and descriptive (empirical) design research studies from the past few decades are discussed. Appropriateness of the verbal protocol method is discussed along with a review of some studies which have used this method. A few studies using non-verbal protocol method are also discussed. The chapter ends with a discussion of research that focuses on information handling issues in the design process
Information Handling Behavior of Designers During Conceptual Design: Three Experiments 20
Chapter 2: Design Theory & Methodology Research: Uterature Review 21
2.1 Design Theory and Methodology Research
The objecjve of much of the research carried out in the field of design theory
and methodology is to develop a better understanding of the engineering design
process. This understanding can be used to develop new methods and services
to improve design practice in industry and design education at academic institu
tions. The complexity! of the design process offers numerous challenges in
accomplishing these objectives. There are many directions possible for conduct
ing design research, as is evident from the number of academic disciplines
involved in it, such as engineering, anthropology, sociology, management and
psychology, among others. As a result, design research is published in a large
variety of literature sources. In this chapter I will discuss the literature which has
impacted the evolution of the work reported in this dissertation.
Looking at design research from the last few decades, the studies that have been
conducted can be grouped under two approaches. They are either prescriptive
or descriptive [Tang 1989, page 25]. Prescriptive studies typically result in recom
mending a sequential procedure for carrying out design, whereas descriptive
studies aim to provide a description of how design is carried out. In the sections
below I will discuss studies associated with both of these approaches.
2.1.1 Prescriptive Studies
Prescriptive design studies result in laying out a method for doing design. These
methods are typically developed by reflecting and theorizing about the design
process. They suggest a systematic and algorithmic procedure that should be car
ried out sequentially. A sequential process ensures that decisions are made after
all necessary information has been generated, accessed and considered, so that
no important elements of the design problem are overlooked. These methods
1. The apparent complexity of the design process stems from the fact that it is not just a technical process or a problem solving process. It is a complex and poorly understood mix of technical, problem-solving, social, organizational, behavioral and creative processes, just to name a few.
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
Chapter 2: Design Theory & Methodology Research: Uterature Review 22
emphasize the need for analytical work before generating new design concepts
and moving on to the next stage. Most prescriptive models allow for iterations
and feedback to take place between different stages.
Asimov [1962] prescribed a seven phase procedure to transform needs into phys
ical artifacts. These phases are:
1. feasibility study
2. preliminary design
3. detailed design
4. planning for production
5. planning for distribution
6. planning for consumption
7. planning for retirement
The first three phases form the design stage and the last four phases form the
production and marketing stages. Each of these phases involves problem solving
situations which include the stages of analysis, synthesis, evaluation, decision, opti
mization, revision and implementation. This problem solving pattern carried out
over the seven phases provides a methodology for transforming needs into phys
ical artifacts.
Jones [1963] prescribed a three stage design method. This method was a system
atic move from analysis to synthesis to evaluation. These stages are: [Cross, 1990,
page 24]:
Analysis: Listing of all design requirements and the reduction of these to
a complete set of logically related performance specifications.
Synthesis: Finding possible solutions for each individual performance
specification and building up complete design from these with least pos
sible compromise.
Evaluation: Evaluating the accuracy with which alternative designs fulfil
performance requirements for operation, manufacture and sales before
the final design is selected.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 2: Design Theory & Methodology Research: Literature Review 23
The emphasis in this procedure is on generating numerous alternatives and
choosing the best alternative based on extensive analysis and comparisons.
Alexander [19641 proposed a method of simplifying and structuring design prob
lems by listing all the requirements of the problems and analyzing the
interactions between them. These requirements could then be grouped into sets,
which lead to mostly independent sub-problems, which were easy to solve. Solv
ing these sub-problems and integrating their solution would lead to a solution
for the bigger problem. His method is usually seen as an iterative analysis - syn
thesis model of design. Analysis is the stage of decomposing the problem into
mostly independent smaller problems and synthesis is the stage of solving these
smaller problems and integrating the resulting solutions.
Archer [1963-641 prescribed a three phase design process based on the following
six activities:
1. Programming: establish crucial issues, propose a course of action.
2. Data Collection: collect, classify and store data.
3. Analysis: identify sub-problems; prepare performance specifications; reap-praise proposed program and estimate.
4. Synthesis: prepare outline design proposals.
5. Development: develop prototype designs; conduct validation studies
6. Communication: prepare manufacturing documentation.
The three phases based on the above activities are: analytical (activities of pro
gramming and data collection), creative (activities of analysis and synthesis) and
executive (activities of development and communication). He suggests:
"One of the special features of the process of designing is that the
analytical phase with which it begins requires objective observation and
inductive reasoning, while the creative phase at the heart of it requires
involvement, subjective judgement and deductive reasoning. Once the
crucial decisions are made, the design process continues with the execu
tion of working drawings, schedules etc., again in an objective and
Information Handling Behavior of Designers During Concepturu Design: Three Experiments
Chapter 2: Design Theory & Methodology Research: Uterature Review
descriptive mood. The design process is thus a creative sandwich. The
bread of objective and systematic analysis may be thick or thin, but the
creative act is always there in the middle. [Archer, 1964]
24
The most recent prescriptive design model is by Pahl and Beitz [1977 and 1984].
This model presents design process as iteratively going through four stages.
These stages are: clarification of task, conceptual design, embodiment design and detail
design. Each stage is accomplished by the following seven step procedure:
1. Clarify and define the task: Specifications
2. Determine functions and their structures: Function structure
3. Search for solution principles and their combinations: Principal structure
4. Divide into realizable modules: Module structure
5. Develop layouts of key modules: Preliminary layouts
6. Complete overall layout Definitive layouts
7. Prepare production and operating instructions: Product documents
This mode1lays out a systematic procedure for developing a good and complete
understanding of the problem, breaking it up into smaller problems, solving
these smaller problems and combining the solutions to obtain the overall solu
tion. This model has wide acceptance in the European design community.
It is not uncommon to see criticism for prescriptive methods. They are seen as
focusing on the problem rather than on the solution and therefore not reflecting
designer's manner of thinking. They also structure the design process into a set
of compartments with well defined boundaries. This is very contrary to how
most designers work and what many researchers have observed as will be evi
dent from the descriptive studies below.
2.1.2 Descriptive (Empirical) Studies
Unlike prescriptive studies which layout a sequential procedure, descriptive
studies aim to provide a description and understanding of what goes on during
the design process as a basis for improving design practice. These are typically
Information Handling Behllvior of Designers During Conceptual Design: Three ExperimcnL'
Chapter 2: Design Theory & Methodology Research: Uterature Review 25
based on formal observations and an in-depth analysis of design activity. Much
of the recent design research falls under this category. Methods have been devel
oped to conduct experiments to collect raw data. The data is analyzed2 to gain
an understanding of certain design behavior. This understanding is subse
quently used to develop services to improve design practice. Waldron and
Waldron [1996] have presented a survey of the various methods that are in use
today to study design. Definitions, advantages and disadvantages of these meth
ods are summarized in Table 2-1.
Table 2-1 Empirical design research methods. The table lists brief definitions of some of the popular empirical methods and the design scenarios for which they are appropriate. Advantages and disadvantages for each of the methods is also discussed.
Appropriate Empiric:aI Method Scenario Advantages Disadvantages
Deposition: timely inter- single - targeted .:lata capture - interferes with design pro-views during the process designer - easier analysis cess
(can also be used with ver- - can study long tasks - incomplete data capture
bal protocol) - data is chronological
Verbal Protocol: designer single - access to designers thoughts - difficult and time consuming talks out loud during designer - comprehensive data capture analysis procedure
design to externalize think- - data is chronological - can only study short tasks
ing process - verbalizing may interfere with design process
Discussion protocol: natu- small team - no interference with design - difficult and time consuming raI communication among process analysis
team members - can study communication. - incomplete data capture negotiation and such - can only study short tasks
Retrospection: interviews single or - no interference with design - data collected is not chrono-or questionnaire after the small team process logical or real-time
design process - targeted data capture - partial data capture - easier analysis
Participant observer: small or - can study very long tasks - difficult and time consuming observer is also a member large team - no interference with design analysis
of the design team process - incomplete data capture - data is chronological
The studies reported in this dissertation in Chapter 3 and Chapter 6 were con
ducted using the verbal protocol method. Section 2.1.3 is devoted to describing
and discussing the appropriateness of the verbal protocol method. In subsequent
2. An analysis procedure is usually a by-product of these research techniques.
Information HllOdling Behavior of Designers During Conceptual Design: Three ExperimenlS
Chapter 2: Design Theory & Methodology Research: Uterature Review 26
sections studies conducted using verbal protocol method and some studies con
ducted using other methods will be discussed.
2.1.3 Verbal (or Think-Aloud) Protocol Method
The verbal protocol method was developed in early sixties to study human prob
lem solving behavior [Newell and Simon 19721. In this method the experimental
subjects are requested to verbalize their thinking (think-aloud) as they work on
the task given to them. The verbalizations are recorded on audio and video
medium.3 These provides the raw data for analysis. This method was and is
most commonly used in cognitive science research. However in the last decade
this method has gained noticeable popularity within the design research commu
nity as is evident from the number of researchers, publications and workshops
focusing on this method [Stauffer, 1987; Ullman et al., 1988; Guindon,1990; Chris
tiaans et al., 1993; Cross et al., 19961.
The appropriateness of the verbal protocol method has been debated extensively
for and against by Ericsson and Simon [19931 and Nisbett and Wilson [1977]
respectively. Crutches [1994] emphasizes the value of the verbal protocol
method as a means for making data explicit for capture. He reports:
"Beyond the increased acceptance of verbal reports as behavioral data,
the unique advantages of verbal report data are increasingly apparent to
many people. In particular, researchers have emphasized the advantage
of verbal reports as protocol data, providing a sequence of observations
over time rather than just a single observation at the end of the process .
......... In addition, verbal protocols provide many more observations of a
phenomenon over a given time period than other methodologies, increas-
ing the information yield of studies .......... Verbal reports can provide
3. In its early usage, the verbal protocol was usually recorded on audio medium. Now days, because of advancements in video technologies, the protocol is also recorded on video medium. For studying the design process, video data is an essential ingredient for analysis since many actions which are non-verbal, and therefore cannot be recorded on an audio medium, get recorded on video medium resulting in a more complete capture of data.
Information HllIIdling Behavior of Designers Ouring Conceptual Design: Three Experiments
Chapter 2: Design Theory & Methodology Research: Uterature Review
information difficult to obtain by other means- in particular, information
about types of knowledge accessed in task processing ..... " [Crutcher,
1994, In Psychological Science, page 242]
The important issues concerning verbal report data are:
27
• Validity: Does the information in the verbal report reflect the actual thinking process?
• Interference: Does the act of verbalizing alter the normal thinking process?
• Completeness: Does the method capture all relevant data?
From a cognitive viewpoint Ericsson and Simon [1993] have argued that:
"subjects can generate verbalizations, subordinate to task-driven cogni
tive processes (think aloud), without changing the sequence of their
thoughts, and slowing down only moderately due to the additional ver
balization." [Ericsson and Simon, 1993, page xxxii].
This means that as long as thinking is assumed to be a sequential process4, the
verbalizations are a valid reflection of the natural thought process. From the
viewpoint of information handling activity during a design process, the evolu
tion of the design is reflected in the sequence of actions performed by the
designers. Verbal protocol reports (captured on audio as well as video medium)
will provide an accurate capture of these sequences.
As stated by Ericsson and Simon above, verbalizing does cause the thought pro
cess to slow down moderately. However, the interference to the natural thought
process can be minimized by providing adequate instructions to the experimen
tal subject. Payne [1994] suggests that while it is not necessary for subjects to be
extensively traineds, they should not be requested to verbalize a particular type
of information. He reports:
" ...... Extensive training in verbalization is seldom needed or really
4. Verbal protocol analysis assumes that cOgnition, specially for tasks such as problem solving, is serial. 5. It is a common experimental procedure to train the subjects on a short trial problem to get them comfortable with thinking out loud.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 2: Design Theory & Methodology Research: Literature Review
desired ... The subject must remain focused on the task, with verbaliza
tion clearly subordinate .... Researchers should also avoid temptation of
trying to get the subject to give a coherent sequence of verbal reports by,
for instance, asking for verbalizations of only selected information (e.g.
reasons for behavior). Such efforts frequently compromise concurrent
verbalizations. The subject should be asked to verbalize all thoughts that
occur to him or her during the performance of the task without emphasis
on any particular type of information." [payne, 1994, in Psychological Sci
ence, page 246]
28
From the viewpoint of completeness, the verbal protocol method has a draw
back. Many representations and information that designers deal with during
design is non-verbal. Much of this information is not captured in the verbal por
tion of the protocol data. However video recordings can be used, as is common
now days, to capture some of the non-verbal information.
From the accounts above, we can conclude that verbal protocol method is appro
priate for collecting data for single designer working on short duration design
tasks. Experimental subjects should verbalize all of their thinking and should
not be requested to verbalize any specific kind of information. This is specially
critical if the focus is on observing the information related behavior as was the
case in the studies presented in this dissertation.
2.1.4 Studies Using Verbal Protocol Method
Akin [1978, 1979] used the verbal protocol method to study the architectural
design process. He was interested in understanding and analyzing the informa
tion processing activities of the designers and finding the relationships between
the design problem and the representations used. He observed that designers
shifted attention among different issues frequently, and that there was a relation
between the representations used and the issue being attended to. From his anal
ysis he concluded that designers exhibit eight recurring behavior patterns which
he called schemata. These schemata are:
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
Chapter 2: Design Theory & Methodology Research: Uterature Review 29
1. instantiation: creating a new item or symbol
2. generalization: grouping related symbols together into a common category
3. inquiry: seeking new information, internally or externally
4. inference: generating new information based on internal knowledge
5. representation: making a visual image of an item
6. goal-definition: defining goals to pursue solutions
7. specification: specifying a partial solutions
8. integration: combining partial solutions
He also observed that design is a top-down, breadth-first process and that
designers seek solutions that adequately satisfy the requirements rather than
develop optimal solutions.
Ullman [1988] and Stauffer [1987, 1991] used protocol analysis method to study
five mechanical designers doing non-routine tasks. Based on their analysis, they
propose an information processing system model which describes the problem
solving process of a mechanical designer. It is called task/episode accumulation
model (TEA model). They state:
" ........ The key features of the model are (a) the design is constructed by
incrementally refining and patching the initial conceptual design, (b)
design alternatives are not considered outside the boundaries of design
episodes (which are short stretches of problem solving aimed at specific
goals), (c) the design process is controlled locally, primarily at the level of
individual episodes."[U11man et al. 1988 page 33]
The TEA model contains ten operators which are identified as the finest informa
tion operator. The design is accumulated gradually by the application of
operators in a meaningful sequences called episodes. Hence the name task/
episode accumulation model. The episodes are on an average 56 seconds long.
Guindon [1990] studied early stages of the software design process using verbal
protocol method. She analysed protocols from three experienced designers and
observed that, designers deviate from a strictly top-down approach in early
Information Handling Behavior of Designers During ConceplUal Design: Three Experiments
Chapter 2: Design Theory & Methodology Research: Uterature Review
stages of the design and display an opportunistic behavior. She states:
" ........ Designers were observed interleaving decisions at various levels of abstraction in the solution decomposition....... The sudden discovery of new requirements and partial solutions triggered by data-driven rules and associations, the immediate developments of solutions for newly discovered requirements , and drifting through partial solutions are important causes of opportunistic design. II
30
Using the results of her study she suggests implications for training, methods
and computational environments to support early stages of the design process.
The interest in using protocol analysis to study design process has been grow
ing. However, much of the effort is isolated among many research groups. It is
therefore difficult to generalize, validate or compare results across researchers
and therefore incrementally grow the knowledge about the design process. To
address these concerns a protocol analysis workshop was organized at Delft,
The Netherlands in September 1994. All participants of the workshop were
requested to analyze data resulting from the same experiments so that analysis
procedure and conclusions could be compared. Cross et al. have summarized
the results of this workshop in a book [1996]. They concluded that use of proto
col method for studying the engineering design process is in its very early
stages. They state:
fl............ Protocol analysis is clearly not a universal cure for design research problems. Nevertheless, we feel that we can say that protocol analysis as a research technique for design has been 'validated' with some qualifications .....
The qualifications were concerned with the completeness of the data collected
and the process of carrying out the analysis. They caution that, (a) verbal proto
col method does not capture the non-verbal thought process going on in design
work and therefore should not be treated as capturing complete data, and (b) the
bias of experimental setup, researcher and experimental subject on the analysis
Information Handling Behavior of Designers Ouring Conceptual Design: Three ExperimenlS
Chapter 2: Design Theory & Methodology Research: Uterature Review 31
procedure should be minimized.
2.1.5 Studies Using Other (Non-Verbal Protocol) Methods
Waldron and Waldron [1988] conducted a study using the retrospective method.
They studied a large design project on the design of an experimental walking
machine involving several designers over a few years. They constructed the
design process sequence retrospectively after the design process was complete.
This was based on interviews with the designers and records from different
stages of the design project. They found that
• experience of the design team members had a significant influence on the development of concepts.
• competing design concepts were pursued in parallel
• important decisions were based mainly on qualitative information and simpie models
• the separations between conceptual, layout and detailed design phases was not distinct, rather all phases were encountered all along
• the design team members changed roles among different conceptual, layout and detailed design throughout the design project
Tang [1989] used the discussion protocol method to study the use of shared
workspaces by a group doing conceptual design. Using techniques borrowed
from video interaction analysis he studied 8 groups of 3 or 4 designers work on
short term design tasks. He observed that the actions of the designers could be
grouped under the categories of list, draw and gesture. These actions were used
to accomplish functions of storing information, expressing ideas and mediating inter
action. He observed that a large portion of the actions are gestures which are not
adequately handled by the existing computational support tools. He recom
mends that tools to support workspace activity of teams working on conceptual
design tasks should consider:
• providing ways of conveying and supporting gestural information
• minimize the overhead encountered in storing information
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 2: Design Theory & Methodology Research: Uterature Review 32
- allowing intermixing of workspace actions and functions
-enabling all participants to share a common view of the workspace while providing simultaneous access and a sense of close proximity to it
- facilitating the participants' ability to coordinate their collaboration
[Tang 1989, page 101]
Minneman [1991] used discussion protocol method to study group engineering
design practice. He collected data from an industrial team working on design of
a photocopier sub-system as well as conducted short experiments to simulate
design communication between different divisions of a design firm (engineer
ing, manufacturing and marketing). He expressed his understanding in terms of
a framework and designers practices. These are:
1. Framework: a perspective for thinking about the content of design work., has two dimensions- facets (the things being worked on) and trajectories (a temporal sense to the work being done), and
2. Practices: actions for getting design work done. Four practices were observed to be crucial. These are negotiating understanding, conserving ambiguity, tailoring engineering communication for recipients, and manipulating mundane representations.
The framework highlights that participants have a personal views of the ongoing
activity and at any given time, design activity is doing communicative work on
multiple projects. The practices signify that, designers' activity is not something
shaped by externally imposed context, but rather as attending to and creating a
recognizable order in the ongoing social interaction. He argues that there is no
innate structure to designing, the structure in the activity is a product of the
designers taking a series of local actions to produce a result.
2.2 Design Process: An Information Handling Perspective
In the previous sections we looked at many studies which have attempted to
understand either the individual or the team design process. These studies
examined the process from a variety of perspectives such as: problem solving,
information processing, workspace activity and collaboration. The studies in this
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 2: Design Theory & Methodology Research: Uterature Review 33
section examine the design process from an information handling perspective.
They are aimed at understanding the information needs and information man
agement burden on designers.
Hales [1987] performed a detailed analysis of all the activities from one project
involving the design of a high-pressure, high-temperature system for testing
materials. He was a participant observer in this project over a period of 2.8 years
which involved 37 different participants. He found that only 47% of the total
observed effort was spent in doing design. The remaining 53% was spent in
information retrieval, planning, cost estimating, reviews, social contact and help
ing others. He observed that structured or prescriptive methods accounted for
less than 25% of the design effort. Also, over 50% of observed effort involved
either working alone or in pairs and about 30% in meetings of 2, 3 or 4 people.
This emphasizes that large portion of the design happens by individuals or
small teams. Based on a quantitative analysis he found that design process can
be characterized by a set of overlapping phases, each consisting of a particular
mix of procedural steps and other general activities.
Kuffner et al. [1991] conducted a verbal protocol study over six experiments to
determine the information requests of mechanical design engineers doing rede
sign. He developed a taxonomy of design information, based on the questions
and conjectures raised by the designers. Design information was categorized by
its nature (construction, location, operation, purpose) and topic (assembly, component,
interface, feature). These nature and topic are identified as the information that
should be captured, structured and made available in CAD environments for
facilitating reuse.
Court et al. [1993] conducted a questionnaire based survey of over 200 engineers
and designers from various industries (power generation, automotive, construc
tion, manufacturing, agriculture and utilities) to determine their information
requirements. This comprehensive survey covered industries of all sizes (small,
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 2: Design Theory & Methodology Research: Literature Review 34
medium, large) and targeted designers in variety of roles (management, creative
design, routine design, etc.). They found that only about 43% of designers time is
spent doing design. The rest of the time is spent in meetings (16%), searching for
information (18%) and paperwork (23%). They also found that 40% of the
respondents did not use any formal operating procedures or methods. Their
study underscores the fact that designers spend a large portions of their time
handling and managing information, and that there is a lack of adequate compu
tational tools to alleviate the information management burden.
The above studies provide some understanding of information related behavior
of designers. More so, they bring out the need for developing a richer under
standing of the information handling behavior so that it can be supported by
novel services and methods. The following chapters will describe the studies
that were conducted using the verbal protocol method to develop an under
standing of the information handling behavior of designers as a basis for
developing requirements for information management frameworks and services.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 3
Design Information Needs, Capture & Reuse
/-------( Information Classification \ \ Framework J -------_/
/-----, ( Dedal Performance \ \. Results I
/-----, ( Information Handling \ \. Framework I
___ 1 __________ l __________ l __ _ Analyze Analyze Analyze
Is Development ;. ~eVise /. \ of Dedal Dedal
19901991 1991-1992 1992-1995
Information Support by Observe Support by ___ ~~ Information Reuse Study Dedal ---~~ Dedal Use Dedal Handling
Study
Understanding of Information Handling
This chapter describes the experimental procedure, analysis method and results of an observational study conducted to understand the information needs of designers during the design process. Questions asked by the designers while engaged in design activity are captured to comprehend their information needs. A classification for these questions is developed. This classification furnishes the framework for design information that should be captured and structured during the design process to enable effective reuse in the future.
Information Handling Behavior of Designers During Conceprual Design: Three Experiments 35
Chapter 3: Design Information Needs, Capture & Reuse 36
3.1 Design Information
Engineering design is an information intensive process. All along the design pro
cess, designers need information to complete their tasks. They fulfill these
information needs by retrieving information from a variety of sources such as:
personal information databases, colleagues, handbooks, technical reports, ven
dors, suppliers and in some cases by library/information retrieval services.
From a survey of more than 200 industry designers from various industries
Court et. al. [19931 found that designers spend in excess of 18% of their time
searching for information. In a participant observer! study [Hales 1987] found
that 53% of the design effort was spent on non-design related activities. In his
study Hales considered information retrieval a non-design related activity.
These studies corroborate the understanding that timely availability of appropri
ate information in a usable format will significantly improve design practice.
3.1.1 Information Reuse Essential for Good Design Practice
It is well known that much of the design activity taking place in industry is rede
sign. A new design is either an improved version of an older design, or an
adaptation of a design from a different domain, or a novel combination of fea
tures from many other designs. In all these scenarios knowledge and experience
from previous designs is used in designing a new product. Figure 3-1 shows
how the process is cost and time effective if designers reuse design information
from past designs rather than re-engineering it. But, a prerequisite to effective
reuse is capturing information in reusable form. The design process does not nec
essarily lend itself naturally to capturing information in a reusable form. Allen
[1977] reported that, during the design process designers use information to pro
duce physical changes in the world. They consume and transform information to
produce a product which is information bearing, but this information is no
1. A study where the observer is a member of the design team being studied.
Information Handling Behavior of Designers During ConceplUal Design: Three Experiments
Chapter 3: Design Information Needs, Capture & Reuse
- past experience .- - reusable info -- technical documents Information Capture - structured info -colleagues
allows Reuse r= j~ and is , Cost and Time Effective
~'7
Old '- Satisfy '- -"'-
New Design Information Design Process Design
Information -/ Needs / 7 Information
o.
~ No Information Capture ..) requires Re-Engineering t
which is
- physical artifacts Expensive and Time Consuming - non-verbal info - - non-reusable info - CAD drawings -- past designs
Figure 3-1 An information perspective on the design process. The figure shows that old design information can be used in satisfying information needs during a design process resulting in new design information. Reusing old information will be cost and time effective compared to re-inventing it. Current design practices have less reuse (shaded region) and more re-engineering (unshaded region). Reuse can be increased by capturing information in reusable form.
37
longer in verbal form. Information in the form of physical or even software prod
uct artifact is difficult to reuse, since it does not capture decisions and the
rationale behind those decisions. Therefore, capturing information in a reusable
form is necessary to avoid expensive and time intensive re-engmeering.
3.1.2 Design Information Capture is Difficult
A good methodology for capturing design information is a prerequisite for effec
tive reuse. The capabilities of current design tools in capturing information is far
less than what is needed to realize the benefits of reuse. Information capture ser
vices available today integrate poorly with the design process, burdening
designers with additional tasks. Consequently these services are under-utilized
or not used. This is specially true for the conceptual design phase. Design infor-
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 3: Design Information Needs, Capture & Reuse 38
mation capture is difficult because:
• design information is often informal, fragmented or incomplete and therefore difficult to represent in a computational medium
• the design process most often results in non-verbal information bearing product
• design information exists in many media
• effort is required to structure information for reuse
• it is not practical or wise to capture all information, therefore capture must be selective
• capture services cannot keep pace with the design process, i.e. rate of information capture is less than the rate of information generation (by a wide margin).
There is a need to develop innovative information capture services which inte
grate smoothly with the design process and capture information in a form
suitable for reuse. We can build such services if we have a better understanding
of the information needs of a designer. This chapter will focus on developing
this understanding.
3.1.3 Information Reuse Study Objectives
The objective of the information reuse study was to develop an understanding of
designer information needs and develop a framework and guidelines for devel
oping information capture and reuse services. Understanding provides
knowledge of the information that should be captured during design. It will also
inform the development of guidelines for structuring this information for reuse.
Kuffner [1991] conducted a study to analyze the information requests of mechan
ical engineers. He came up with a taxonomy which characterized information
based on its nature, topic and category. There are some similarities in the objec
tives and the approaches of his study and this study. These studies will be
compared in the discussion at the end of this chapter.
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
Chapter 3: Design Information Needs, Capture & Reuse 39
3.1.4 Questions Reflect Information Needs
It is natural for humans to express a need for information in the form of a ques
tion. For instance, if you needed to find the torque developed by a motor
manufactured by a vendor, you will most likely express your need in the form of
a question such as "How much torque does this motor develop when hooked up
to a car battery?" An answer to this question would fulfill your information
need. If we were to capture the questions that designers ask while engaged in
design, we would have a view into their information needs. The information
reuse study was designed to capture the questions that designers ask while
engaged in design.
3.2 Experimental Study: Verbal Protocol Method
I designed an observational study using the verbal protocol method to capture
the questions designers have when engaged in design activity. Experimental sub
jects were asked to redesign an existing product to a new set of requirements. I
chose the experimental method since it allowed me to:
• look at the design process from a reuse viewpoint
• deal with the complexity of the process by exercising adequate control
• simulate real design activity
• study design activity at a fine level of granularity
• make designers actions explicit for observation
I used the redesign scenario in the experiments since, redesign is a very common
scenario in industry and information reuse is prevalent during redesign.
3.2.1 Experimental Setup and Procedure
Figure 3-2 shows the experimental setup used to collect data. The subject
worked in a room fitted with two video cameras. One camera captured a wide
angle view of the workspace and the other captured a close-up of the desk. The
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 3: Design Information Needs, Capture & Reuse
data was independently recorded on the two cameras and an audio recorder.
~ fi Workspace ... a Documents
./ / " / "
Figure 3-2 Experimental setup for information reuse study. Two video cameras capture activity in the workspace (camera 2) and the wide angle view of the subjects work area (camera 1). The subject has access to documentation from the first generation design, and to a member of the first generation design team. (expert). The expert sits in a separate room and is available on request.
40
At the start of an experiment the subject was given a small problem, to get him
comfortable with talking out loud and the experimental setup. After this he was
given the experiment's redesign problem. The subject was asked to complete the
redesign task in about six hours. The experimenter was out of the room monitor
ing the subject on video. If the subject was silent for an extended duration (15-20
seconds), the experimenter would remind the subject to talk out loud. The exper
iment was conducted over two sessions with an hour long break in between. The
subject was asked to summarize the status of his work before and after the break
to capture any thoughts that may have occurred during the break.
Subjects had access to design notebooks and technical reports from the first gen-
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 3: Design Information Needs, Capture & Reuse 41
eration design. These contained detailed description of the first generation
functional prototype and its performance. One member of the original design
team, to be referred to as the expert, was also available to answer questions and
serve as an index to information in the documentation which totaled more than
a thousand pages. The subject using the expert as an index into the documenta
tion provided a natural way of making the questions explicit without being
obtrusive to the subject. The expert stayed outside the room but could be called
in by the subject any time. The expert stayed out of the room, so as to be unaware
of the information that the subject was encountering. This setup assured that the
expert answered any questions objectively without volunteering additional infor
mation. The instructions and the redesign problem used in the experiment are
attached in Appendix A.
3.2.1.1 The Design Problem: Continuously Variable Damper
The problem chosen was the redesign of a continuously variable car damper
(popularly known as shock absorber) to meet a new and revised set of require
ments. The design was complex enough that the subjects needed to develop a
good understanding of the first generation design before coming to a solution
and simple enough that a conceptual solution could be reached in about six
hours. Such a trade-off was essential to conduct an experiment in a laboratory.
The first generation design was done in a real world academic setting by a three
member design team. This project was done for a leading USA automobile man
ufacturer over a period of seven months [Baya 1990]. The design was carried out
from concept to laboratory testing of a working prototype. Three documents
spaced equally over that period were produced by the design team. Each docu
ment summarized the status of design at the time it was written. These
documents captured the evolution of the first generation design from early ide
ation stage to final machine drawings and test results of a functional prototype.
Information Handling Behavior of Designers During Conceptual Desigo: Three Experiments
Chapter 3: Design Information Needs, Capture & Reuse 42
3.3 Analysis of Verbal Data
The analysis required a procedure for transforming verbal data into a frame
work which reflects design information needs. This procedure is shown in
Figure 3-3. It involves six steps. The form of the data at input and output of each
Q
Design Problem Conduct Recorded Data Written Data
Experiment Transcribe
uestions with Context Reformulate Questions Extract
Questions Questions
Quantitative
Classify Classification Statistical Results
r Questions Analysis
Figure 3-3 Analysis procedure for information reuse study. The steps in the analysis reduce recorded data into quantitative description of the designer information needs.
of the steps is indicated. Altogether these steps reduce recorded data into a quan
titative description of the information activity of designer in this scenario. Each
of the steps is described below. The examples for each of the step are discussed
in section 3.5.
3.3.1 Transcription
The process of transcription yields written protocol data. The transcript captures
the words spoken by the subject as well as information about pauses, intonation,
focus of attention and context. Video tapes are used in this step to add action
and gesture data to the record. The resulting transcript has the details necessary
for carrying out the rest of the analysis. However there were occasions when
raw data was used in later stages of analysis to resolve ambiguities.
Information Handling Behavior of Designers During Conceptual Design: Three ExperimcnlS
Chapter 3: Design Information Needs, Capture & Reuse 43
3.3.2 Question Extraction
The written data resulting from transcription is used for extracting questions.
For the purpose of this study a question is an interrogative expression uttered by
the subject to seek information (e.g .. what is the diameter or X?) , or an utterance
which suggests a need for information, although it may not have been explicitly
asked (e.g .. I should find the diameter of X). In ambiguous cases, the subject was
consulted (after the fact) to determine if an utterance was a question or not.
Some key scenarios and rules in extracting questions were:
• explicit ask to expert e.g .. "How heavy is this device?"
• searching of documentation by self e.g .. "Where are the test results?"
• utterance to self e.g .. "I will need to find the thickness of this part."
• conjecture, e.g .. "This must be 12 inches."
• a question uttered twice within the same activity focus! was counted once.
• a question repeated in an unrelated context or in a different activity focus was counted again.
• a complex question was broken into a set of simple questions.
3.3.3 Question Reformulation
Questions as spoken by the subject in the transcript are often dependent on
nearby information to be intelligible. i.e. they have meaning within the context
of their location in the transcript. Reformulation was the required to fill in con
textual and missing information so that question classification could be done
objectively without consulting the transcript.
3.3.4 Question Classification
Before questions could be classified, there was a need to develop a framework
for doing so. Figure 3-4 shows some of the important steps involved in the devel
opment of the question classification framework. This framework is described in
1. An activity focus was defined by closeness in context, time or design goal of the designer.
Information Handling Behavior of Designers During Conccprual Ocsign: Three ExpcrimcnlS
Chapter 3: Design Information Needs, Capture & Reuse
Questions
+ step 1 Characterize ~
"1 \" ____
//" No / \"
/ / \ " / " ", " step2 ~ step 6
Collection of Question Characteristics Can all questions be classified?
~"'/~ Group into Categories
+ step 4 Label Categories --•• ~
~ Classify New
Questions
Yes
Rgure 3-4 Steps in the development of question classification framework. First step is divergent and generates a collections of terms to characterize questions. This collection is grouped, based on similarity in characteristics or meaning, into categories. These categories were labeled. If this set of categories were suitable to classify any question encountered in the data then it was the classification framework.
44
detail in section 3.4. Question classification used this framework on the reformu
lated questions. The classification was based on interpreting the questions with
respect to a template as a method of maintaining rigor and objectivity in classifi
cation. The template is shown in Figure 3-5.
3.4 Question Classification Framework
Table 3-1 shows the framework for classifying questions. In this framework a
question is characterized by four attributes namely DESCRIPTOR, SUBJECI'-cr.ASS,
MEDIUM and LEVEL-oF-DETAIL. It is classified by being given a label from each of
the four attributes. The values for each of the attributes are shown in Table 3-l.
They are not a list of all possible values. They represent what is observed in the
experimental data and suggest values significant for describing design informa
tionneeds ..
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 3: Design Information Needs, Capture & Reuse 45
Table 3-1 Framework to classify questions. The same framework can be used to classify design information. This will also be called the Design Information Framework (DIF).
Descriptor subject.c:Jass Medium Level of Detail
Alternative Assembly Audio Conceptual
Construction Component Video Configurational
Location Connection Text Detail
Operation Feature Graphic
Performance Requirement Gesture
Rationale Design-concept
Relation Other
Requirement
Miscellaneous
3.4.1 Design Information Framework (DIF)
The framework that is used to classify design information is the same as the one
used to classify questions. This is because any segment of design information
can be considered to be an answer to an implicit question. Thus it is possible to
interpret design information segments in a template identical to that of a ques
tion as shown in Figure 3-5.
3.4.2 Descriptor
The descriptor of a question refers to the inherent character, the basic constitu
tion or the nature of design information that is sought by the question. The
descriptors are independent of the design problem or the domain of design.
Table 3-2 shows the descriptors, their definitions and examples.
3.4.3 Subject-Class
The subject-class of a question refers to the class to which the subject of the ques
tion belongs. The subject of a question is the subject of the sentence or the clause
making up the question. Like descriptors the subject-classes are independent of
the design domain. The subjects of the question are dependent on the design
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 3: Design Information Needs, Capture & Reuse
I
Question Interpretation Template:
the subject is requesting, O£Oll) information (LEVEL-QF-DETAIL)
regarding the 0'Il£S of the question-subject which (DESCRIPTOR)
belongs to OSC£ (SUBJEcr-CLASS)
r---------------------------, Design Information Interpretation Template:
the design information segment is l£Oll) (LEVEL-OF-DETAIL)
l'lYES of the information-(DESCRIPTOR)
information regarding the
subject which belongs to --r=~l=:SC~'£=--=-=. (SUBJEcr-CLASS)
L ___________________________ ~
Questionlinformatjon classification based on above interpretation
Question Descriptor: Q!JYES Question Subject-Class: QSC£ Question Level-of-Detail: Q£OIl)
Information Descriptor: l'lYES Information Subject-Class: lSC£ Information Level-of-Detail: l.£OlD
,,-----------------------------------------------------------------_. Figure 3-5 Template for interpreting question and information segment. Question-subject and information-subject in the above templates refers to the subject of the sentence forming the question or the segment of design information respectively.
46
domain. Table 3-3 shows the different subject-classes, their definitions and
examples.
3.4.4 Medium
The medium of a question refers to the form of the information which would
adequately answer the question. While this is an important category for classify
ing design information, from a computational viewpoint, the verbal protocol
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 3: Design Information Needs, Capture & Reuse 47
Table 3-2 Question descriptors refer to the basic constitution of the information sought by the question. It can take the values defined below in the deSCriptor column.
Descriptor Question seeks information regarding Example
Alternative choices or decisions What are the other ways of changing linear motion into rotaxy motion?
Construction topology, shape, dimensions, arrangement What does the damper design look properties like?
Location position or site Where do you want the maximum resistive force in the full ON state?
Operation manner of functioning How is the resistive force varied in the hydraulic concept?
Performance behavior, efficiency, evaluation/test What was the maximum temperature results at the solenoid during test?
Rationale purpose, reason, basis, explanation of Why did they choose the rotaxy fric-choices tion concept?
Relation inter-dependence, cause-effect, interac- What is the effect of temperature on tion the solenOid?
Requirement specifications, needs, constraints What was the limit on solenoid stroke length in the previous design?
Miscellaneous process details, plans, strategies, and any- What are the characteristics of the thing not included in the other descriptors suspension system?
Table 3-3 Question subject-class refers to the class to which the subject of a question belongs.The seven classes below cover the range of question subjects encountered in the data.
Subject-Class Subject or the question refers to Example Subject
Assembly an assembly, sub-assembly or a Is the shape of the volume of damper collection of components the damper a cylinder?
Component a part of the assembly, a single Does this end of the arm go top arm item to bottom?
Connection interface between two or more How is the disk attached to the attachment components hub?
Feature attribute of a assembly or com- Why should stroke length be stroke-ponent, physical properties less than 6 inches? length of
solenoid
Requirement specification or constraint of How heavy was the last design? weight the design
Design-concept design concept, partially- How does rotaxy friction con- rotaxy-fric-formed ideas cept develop resistive force? tlon-concept
Other anything not included above What are these four lines in the lines graph of the solenoid?
method skews the data heavily towards the audio medium. As a result I have
not made an attempt to draw any conclusions regarding the medium. Table 3-1
Information Handling Behavior of Designers During Conceptwll Design: Three Experiments
Chapter 3: Design Information Needs, Capture & Reuse 48
lists some possible values that medium can take. This is not a complete list. It is
a representative sample of the dominating media observed in the study.
3.4.5 Level-of-Detail
If we were to classify the information about a design on how detailed it is, then
the level-of-detail of the design information which answers a question is the
level-of-detail of the question. It is common perception that design requires
information at the following levels-of-detail: conceptual, configurational and detail.
These levels differ in the extent to which design information is specified. Table 3-
4 defines the levels-of-detail with examples.
Table 3-4 Question level-of-detail. refers to the specificity of information sought by the question.
Level of Detail Question seeks information at a Example
Conceptual qualitative, conceptual level What are the other design concept for the force generation mechanism?
Configurational layout, arrangements of components Where does the solenoid fit in in the damper?
Detail specific, quantitative How much resistive force should the damper have when the arm is here?
3.5 Analysis Examples
A few examples of analysis from transcription to classification will clarify the
procedure. Table 3-5 shows these examples. Note that the original verbalizations
are rarely complete sentences or grammatically correct. Question segments are
underlined.
3.6 Results
Table 3-6 summarizes the background of the experimental subjectsl . The first
experiment lasted about five and a half hours and the second about three and a
half hours. 160 question were extracted from the first and 80 from the second
1. The designers in the two experiments will be referred to as RSl and RS2 respectively.
Information Handling Behavior of Designers During ConceplUaJ Design: Three ExperimenlS
Chapter 3: Design Information Needs, Capture & Reuse 49
Table 3-5 Example of analysis procedure in information reuse study. The segments 7,8,9 and 10 cover a continuous period of about 50 seconds. The question from protocol is underlined and shown again in the second column. The rephrased question is also shown in the second column under the question from the protocol.
Question from Protocol
No. After 'framcription Question Rephrased After Reformulation Question Classification
7 All right, let'srz see. Is this a Is it a Desaiptor. Requirement (pause} This is a, that continuously variable continuously Subjclass: Assembly was design a continuously damper? variable damper in leveL oJDetaH: Conceptual. variable damper. 1a tb1a the design problem? [d~aign g~gbl~l A Is it a continuously
~gDtiDUg~alx ~~1abl~ variable damper in
rumroer' Yeah, okay. Okay, the design problem?
let's see.
S Now, I don't see why, do 00 I have to go with 00 I have to go Descriptor. Miscellaneous I really have to go, or what they came up with the design Subjclass: Design-Concept gg 1 baxe tg ~g ~1tb Kbat with? concept the first leveL of Detail: Conceptual. tb!iD{; !~r~gya aeaign design team came team} came up with" 00 I have to go with up with?
the deSign concept the first design team came up wi th?
9 It depends on what their, What were the What were the Desaiptor. Rattonaie on what they found out drawbacks of the drawbacks of the Subjclass: Design-Concept about the other other systems? design concepts leveL oj Detail: Detail possibilities. tibae dZ;:~ the first design tb~ g~A~~~ka gf tb~ What were the team looked at? other Systems? drawbacJcs of the
other design concepts the first design team looked at?
10 I think they had that in What were the What were the Desaiptor. Performance here. (opening d3, advantages to this? advantages of the Subjclass: Design·Concept flipping pages} When they
Wha t were the hydraulic concept? leveL oj Detail: Detail
were considering other systems, like, um. Okay, advantages of the
the following section hydraulic concept?
points out approaches, (reading} each of which we believe has the potential to develop into a reliable prototype. Each design should mmble (inaudible} into a whole class of possible solutions,frictional,elect rorheological, and moving fulcrum concept. (stop reading} (pause} Okay, wllat lrli:Z;:~ eD!: a~DtAg:Ii::i tg tbh'
one. A possible reason for this difference is the familiarity of the subjects with
the expert. RSI was familiar with the expert and felt comfortable asking question
whereas RS2 was not that familiar and preferred to look through documenta
tion, therefore not externalizing some questions. Another possibility is that RS2
being experienced, asked fewer but pertinent questions and also took about 2
hours less than RSI in coming to a solution.
Information Handling Behavior of Designers During Conceptual Design: Three Experiment'
Chapter 3: Design Information Needs, Capture & Reuse 50
Table 3-6 Information reuse study: subjects background information.
Design Experiment Familiar with No. of Questions Subject Experience Duration Expert? Extracted
RSl 2 years 5.5 hours Yes 160
RS2 10 years 3.5 hours No 80
The distribution of questions, in the classification, in the two experiments is not
significantly different. The results below are for questions from both the experi
ments. Appendix A has results for individual subjects, RSI and RS2, and a
comparison of any differences.
3.6.1 Results on Descriptors
Figure 3-6 shows the distribution of questions across the descriptors. Informa
tion regarding performance, relation and construction was frequently accessed. All
other descriptors are well represented. About one fifth (21%) of questions were
miscellaneous. Thus this rather sparse classification scheme dealt effectively with
about 80% of the questions encountered. However 21% is a large enough num-
Performance
Relation
Construction
Operation
Alternative
Rationale
Requirement
Location
Miscellaneous
Percent of Questions
21
30
Figure 3-6 Distribution of question deSCriptor. Information with descriptors performance, relation and construction was frequently needed. 21 % of the questions needed miscellaneous information, i.e. the classification works well for about 80% of the questions encountered.
ber and suggests that there is considerable interest in accessing information
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
Chapter 3: Design Information Needs, Capture & Reuse 51
about design process strategies, plans, organization, design methods, evaluation
methods and others. These are some of the characteristics of information
spanned by the miscellaneous descriptor.
The information in Figure 3-6 can guide decisions about what type of informa
tion to capture during the design process. The relative frequency of occurrence
of these descriptors can be used to set priorities in developing capture strategies.
3.6.2 Results on Subject-Class
Figure 3-7 shows the distribution of questions across the subject-classes. Informa
tion about feature (35%), components (20%) and assembly (12%) were frequently
asked for. We know that there are more features in a design than components,
more components then assemblies. While it is reasonable to see more questions
35
40 Percent of Questions
Figure 3-7 Distribution of question subject-class. Information on features and components is asked for frequently, but other classes are also well represented.
addressed to the class which has more members, the breakdown emphasizes the
concern for adequately representing all subject-classes in a capture tool.
There were a significant number of questions on the subject-class requirements
(13%) and design-concept (9%). This suggests that adequate attention should be
given to capturing information about requirements and design-concepts.
Information Handling Behavior of Designers During Conceplual Design: Three Experiments
Chapter 3: Design Information Needs, Capture & Reuse 52
3.6.3 Results on Level-of-Detail
Figure 3-8 indicates that more than half the questions (57%) were asked about
Detail 57
Conceptual
Configurational
o 10 20 30 40 50 60 70 Percent of Questions
Figure 3-8 Distribution of question level of detail. The nature of the design problem used in the experiment may have focused the designers attention to detail information.
the details of the design. This could partly be due to the nature of the design
problem used in the experiment. Since the design problem was reasonably com
plex and time available to the subjects limited, the subjects decided to go with
the original concept quite early in the design process and tried to achieve the
new requirements by making parametric changes. This channeled their interest
to the details of the design for the most part. However, about 28% of the ques
tions are addressed to the conceptual level-of-detail. This asserts the need for
capturing information from the conceptual stage.
3.6.4 Results on Crossing Attributes
Interesting results are obtained by crossing two attributes, descriptor with subject
class, descriptor with level-of-detail and subject-class with level-of-detail.
Descriptor with Subject-Class. Crossing descriptor with subject-class gives us
the knowledge of what type of information about the different subject-classes is
frequently needed. Table 3-7 highlights the combinations of descriptor and sub-
Infonnation Handling Behavior of Desigocrs During Conccptlllll Design: Three Experiments
Chapter 3: Design Information Needs, Capture & Reuse 53
ject-class encountered frequently. Information about relations of features, operation
Table 3-7 Crossing descriptor with subject-class. Highlighted numbers signify the most frequently asked for descriptor for a particular subject-class.
Subject-Class
Descriptor Assem. Comp. Conn. Feature
Alternative
Construction o o 33
Location o o o 5
Operation o 22
Performance 1 5 43
Rationale 5 20
Relation 3 o 2 38
Requirement 2 2 o o 5 o 10
Miscellaneous 2 8 o 14 10 6 11 51
SCL-Total 29 49 9 83 31 22 17 240
of components and assemblies was frequently asked for. Information about perfor
mance and construction of features was also significant.
Descriptor with Level of Detail: Table 3-8 crosses descriptor with level-of
detail. It can be seen that detailed information regarding the performance, relation
and construction was most frequently asked for. Configurational information was
frequently asked about operations, and conceptual information was often asked
about the alternatives.
Subject-Class vs. Level of Detail: Table 3-9 crosses subject-class against level-of
detail. It shows that detailed information about features and components was
needed often. Configurational information was frequently asked about assemblies
and components and conceptual information regarding the design-concepts and
requirements was needed.
The information such as in the table above can be used in"filtering and prioritiz
ing information during capture. It can also be used in ordering responses to
Information Handling Behavior of Designers During Conceprual Design: Three Experiments
Chapter 3: Design Information Needs, Capture & Reuse 54
Table 3-8 Crossing descriptor with level-of-detail. Highlighted numbers show what types of deSCriptors were most needed at a level of detail.
Level of Detail
Descriptor Configuratiooal Detail
Alternative 1 7
Construction 11
Location 0
Operation 2
Performance 8
Rationale 6 4
Relation 9 3
Requirement 5 0 5
Miscellaneous 25 2 24
LOD-Total 66 37 137
Table 3-9 Crossing subject-class with level-at-detail. The highlighted subject-class was most frequently asked for at each level-at-detail.
Subject-Class
Assembly
Component
Connection
Feature
Requirement
Design-concept
Other
LOD-Total
Conceptual
5
14
o
10
queries during retrieval.
3.7 Discussion
Level of Detail
9
4
2 3
0 7
37 137
Descriptor Total
18
33
5
22
43
20
38
10
51
240
numbers show what
SCL-Total
29
49
9
83
31
22
17
240
The objective of this study was to develop an understanding of the information
needs of designers before developing services for supporting information cap
ture and reuse. The goal was not to generalize the results but to utilize the
design information framework, developed in this study, in implementing an
Information Handling Behavior of Designers During Conceptual Design: Three ExpcrimenlS
Chapter 3: Design Information Needs, Capture & Reuse 55
information management service. The next chapter will go into the details of the
development of this service, called Dedal.
The design information framework has been adequate in classifying information
that deSigners ask for while engaged in design. While it cannot be used to clas
sify all the information designers deal with, it provides an example of
information that should be captured for reuse. This framework will be applica
ble to a wide range of design domains.
A sample of two subjects is quite small to draw any generic conclusions. How
ever, this study does demonstrate experimentation as a powerful method to
develop design process understanding and provides an analysis method for
doing so. Quantitative results obtained in this study have been explained based
on the nature of the experiment, the design problem and current understanding
of the design process. What is unexplained has been used to improve under
standing and refine technique of conducting and analyzing protocol
experiments.
There are differences in the classification developed in this study and the taxon
omy of questions reported in [Kuffner 1991]. The taxonomy in [Kuffner 1991]
classifies a question (and a conjecture) based on its nature (construction, location,
operation, purpose) and topic (assembly, component, interface, feature). Some reasons
for the differences in the outcome of the studies can be attributed to the
following:
• The size of the design problem will influence the range of the question one would encounter. In [Kuffner 1991] the first generation design was done in an experimental setting over a period of 6 hours. In this case the first generation design was a real world design. The bigger scale of the design problem exposed new types of questions which may not have occurred in a simpler problem.
• Designers do not carry out design with a pre-determined set of questions. They raise questions as new information is needed. The questions are often triggered by exposure to information accessed by them. In [Kuffner 1991]
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenL~
Chapter 3: Design Information Needs, Capture & Reuse 56
the documentation available to the subjects was in the form of machine drawings and specifications. In this study subjects had access to a more complete and extensive documentation. The completeness in documentation provides an opportunity to collect a more complete set of questions that designers can raise.
• The questioning behavior is not random. New questions are asked after reflecting on information received in answer to a question. So the type of questions asked will depend on the form in which the information is available to the designer. For instance, if a designer was given a machine drawing it would be very likely for him to ask questions about location and connections, however if the same information was given in a descriptive form, the line of questioning could be different. In this experiment the information was available to designers in many forms, making it possible to collect a more complete set of questions.
3.8 Recommendations for Information Reuse Service
Based on the results of the information reuse study, here are a few recommenda
tions for information capture and reuse service:
• it should be able to capture and represent the information described by the design information framework
• it should keep pace with the generation of information. i.e. information capture and indexing should happen during the design process and should not be a post-facto add on activity.
• it should be capable of dealing with information at various levels of detail.
• it should be capable of dealing with multi-media information.
3.9 Summary
I have discussed quantitative and qualitative results from an empirical study of
design information reuse during design. I have developed a method for analyz
ing the questioning behavior of designers and used it to develop a framework
for classifying and understanding design information needs. The framework
characterizes design information based on its descriptor, subject-class, level-of
detail and medium. This framework is the foundation for the development of a
service for information capture and reuse discussed in Chapter 4.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 4
Dedal: An Information Management Service
/-------- ,.-----, /-----, ( Information Classification \ { Dedal Performance , { Information Handling,
\ Framework / I \ Results / \ Framework / ..... _----- ------ ------
t t . . t --------- ----------- ---------
Analyze Analyze Analyze
/. D~m ;.~ ).2-'~\ of DadaI Dadal
19901991 1991-1992
Information Support by Observe Support by Information Reuse Study DadaJ • DadaJUsa DadaJ - Handling Understanding
Study of Information Handling
This chapter describes the information management utility called Dedal which embodies services for indexing, modeling and retrieving design information. It uses the design information framework described in Table 3-1 to implement a conceptual indexing method. This chapter will report on Dedal's retrieval performance and results obtained when it was deployed to support design activity in real time. Details regarding implementation of Dedal are discussed in AppendixB.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments 57
Chapter 4: Deda/: An Information Management Service 58
4.1 Introduction to Deda11
Using the requirements extracted in the information reuse study (section 3.8), a
service for managing design information was developed. This service is called
Dedal2. Dedal provides tools for indexing, modeling and retrieving design infor
mation in real time. Figure 4-1 shows the different components of Dedal and the
flow of information among these components.
CD en cu .0 cu -cu C en
"C ~
o ()
CD a: c C)
en CD C cu "0 CD
E -::l
USER
Graphical User Interface
Modeling Tool
Retrieval Heuristics
::iE - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - •
Figure 4-1 Dedal system architecture. The dark shaded area represents the information capture environment and the light shaded area show the components which make up Dedal. Dashed arrows indicate information transfer. Indexing tool is used to create the indices database, modeling tool to create a domain model, and retrieval tool to answer queries. The indices database and the domain model are used by all the tools. The information capture environment is the tool designers use to document design information. It is integrated with Dedal via the graphical user interface. The multimedia design records database represents design information captured by the information capture environment.
1. Dedal has been developed in collaboration with Catherine Baudin and Jody Underwood from NASA Ames Research Center and Ade Mabogunje from Stanford University. 2. The name Dedal comes from the greek mythological character Daedalus, the legendary builder of the Cretan labyrinth who makes wings to enable himself and his son Icarus to escape imprisonment. It is the characteristics of finding way in complex labyrinth that is a parallel to the function of Dedal.
Information Handling Behavior of Designers During Conceptual Design: Three ExpcrimenlS
Chapter 4: Deda/: An Information Management Service 59
The tools for indexing, modeling and retrieval are accessed via a graphical user
interface which integrates Dedal with an information capture environment. The
screenshots of these interfaces are shown in Appendix B. The information cap
ture environment would typically be a collection of tools that designers use to
carry out their day to day work such as: word processor, drawing/sketching pro
grams, spreadsheet, CAD /CAE tools, electronic mail and others.
Dedal deals with multimedia information and enables indexing and modeling to
take place in real time. It deals with information at all levels of detail and formal
ity. For more details on the development of Dedal and its architecture please see
[Baudin et.al. 1993a]. A copy of this paper is included in Appendix B. Dedal is
written in common lisp. The interfaces for dealing with audio and video informa
tion are written in the language C.
4.1.1 Indexing Tool
The indexing tool is used to generate the indices .database using a conceptual
indexing scheme. In conceptual indexing, an information segment is indexed by
describing the concept represented in it. Concepts are domain dependent enti
ties which embody the semantics of the domain. In Dedal the concept is
described by using the design information framework in Table 3-1, Le. informa
tion is indexed by the classification of the question it answers. Figure 4-2 shows
an example of an information segment and its conceptual index.
The size and granularity of an information segment can vary depending on the
type of document and the medium of information. Typically a segment is a para
graph of text or a single graphic. It is possible, and often necessary, to index the
same information segment with more than one index. This is specially true of
cases where the segment is rich enough to answer multiple questions. Since
Dedal is integrated with the information capture environment the designer only
specifies the descriptor, subject, and level-oj-detail of the information. The medium,
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
Chapter 4: Deda/: An Information Management Service 60
document, segment and paragraph-no. are automatically picked up through the
interface.
Indexing pattern on the information segment on page 18 of document DRD-spring-1993:
Descriptor: Rationale Subject: Solenoid Medium: text Level of Detail: Conceptual Document: DRD-spring-1993 Segment: 18 Paragraph no: 2
A question which will return the above index as answer:
Descriptor: Rationale Subject: Solenoid
'C'he solenoid was chosen since It Is able to generate upto 30fbs of force with the given power and ••••••••••
18
Figure 4-2 Example of conceptual indexing. Paragraph 2 on page 18 of the document DRD-spring-1993 answers the question: What was the rationale of the solenoid? Therefore the indexing pattern for this segment is as shown. The concept this index represents is solenoid, a domain dependent entity.
4.1.2 Modeling Tool
The modeling tool is used to generate and maintain a model of the design infor
mation. A model in Dedal is a means of defining relationships among the
domain concepts in a hierarchical manner as shown in Figure 4-3. These relation
ships are typically of the form: part-of, is-a, depends-on and others. The model
includes a representation of the artifact structure, requirements, alternatives and
decision points and it evolves incrementally with the design process. The simple
hierarchical structure of the model assures that designers themselves can create
them in real time. The modeling tool also helps maintain consistency of concept
names and relations and provides browsing and editing facilities ..
4.1.3 Retrieval Tool
The retrieval tool is used to get answers to queries. It uses the indices database,
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 4: Deda/: An Information Management Service
disk-stack solenoid
/ part-of ~
/ part-of
" outer-clisks inner-clisks
Rgure 4-3 Example of a model in Dedal. The figure shows the simple hierarchical representation which relates domain concepts.part-of and is-a relations can be seen in this model.
61
the domain model and heuristics to direct user to appropriate locations in the
design records database. The algorithm used by the retrieval tool is shown in
Return Answers to User
Figure 4-4 Flow chart of the retrieval algorithm used by Dedal. The answer for a query is first searched in the model •.
Figure 4-4. A query is phrased by specifying the descriptor and subject(s)
(Figure 4-2). The retrieval algorithm first tries to answer the query from the
model. The model has information about decisions made and alternatives consid-
Information Handling Behavior of Designers During ConceplUal Design: Three ExperimenL~
Chapter 4: Deda/: An Information Management Service 62
ered. Simple questions about them can be answered from information in the
model. If this fails to find a relevant answer then the indices database is searched
for an index which matches the query. Failing this heuristics are used to expand
the search. Dedal uses three types of heuristics. They are:
1. Related descriptor heuristic: This heuristic uses the fact that the boundaries between the descriptors often overlap. Therefore, sometimes information about a descriptor of a subject can be found by searching for a information about a different descriptor of the same subject. An example of such a heuristic is:
if (question is rationale of X) then
(search for decision of X)
2. Proximity heuristic: This heuristic uses the fact that a document is a continuum of information. Often a certain type of information can be found near another type of information. An example of such a heuristic is:
if (question is function of X) and (X is part-of Y)
then (search for operation of Y)
3. Causal relation heuristic: This heuristic uses the knowledge of certain relations which are causal in nature to change the subject of the query. For instance:
if (question is rationale of X) and (X depends-on YJ then
(search for rationale of Y)
Many examples of the heuristics are included in Appendix B.
4.2 Dedal Test Domains and Results
To measure retrieval performance of conceptual indexing scheme and the real
time performance of Dedal, we deployed it to support design information man
agement in three domains listed in Table 4-1. Results obtained in each of the
domains are discussed below.
Information Handling Behavior of De..igners During Conceptual De..ign: Three Experiment.
Chapter 4: Dedal: An Information Management Service 63
Table 4-1 Dedal test domains details. I = indexing, M = modeling and R = retrieval.
Project Capture Designer No. Domain Duration Dedal Feature Thsted Platform Medium Indexes?
1 Damper 7mon R perfonnance Symbolics vmacsa Yes
2 Bioreactor-I 7mon Real-time I & M Unix Framemaker No
3 Bioreactor-II 2mon Real-time R Unix Framemaker No
a. vmacs™ is a trademark of perfonnance graphics company.
4.2.1 Damper Domain Results
The first testbed domain was the design of a continuously variable damper
[Baya et al. 1990}. This domain was used to assess the retrieval performance of
the conceptual indexing scheme implemented in Dedal. Design information was
captured in an electronic design notebook called vmacs [Lakin et al. 1989}. A
member of the design team used Dedal to index about 400 pages of documenta
tion with 340 indices. Forty distinct questions, the answers to which were
available in the documentation, were selected from the 240 questions extracted
in the information reuse study (Table 3-6). Note that questions to which the
answer is not in the documentation cannot be used to assess retrieval perfor
mance since no answers retrieved would be relevant. This will cause the results
to be biased towards poor performance.
Figure 4-5 shows the retrieval performance measured is this study. A compari
son was done with full-text boolean retrieval scheme, a popular information
retrieval method. Full-text boolean retrieval is a baseline reference point for com
parison. The retrieval performance is measured by two parameters, recall and
precision. Recall is the ratio of the relevant answers retrieved to the total number
of relevant answers present in the information database. Precision is the ratio of
the number of relevant answers retrieved to the total number of answers
retrieved. A perfect retrieval system would have recall and precision of 100%.
As is clear from Figure 4-5, conceptual indexing scheme of Dedal out performed
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 4: Deda!: An Information Management Service
Recall = # of right answers retrieved total right answers present P
.. # of right answers retrieved recision =
total answers retrieved
100~------------------------------------~
~ • -- • -- • -- - -- - -- ... Oedalwith , retrieval heuristics
80
60 Boolean retrieval I as 0 CD a:
_-_0 - - - - - - - - - - _ - - - - _.£ _ - - -B Dedal with Jut
I I : retrieval h uristics 40 I
I
20 I I I
00 I
20 40 60 80 -
100
Precision
Figure 4-5 Retrieval performance of conceptual indexing. Oedal's retrieval engine based on conceptual indexing performed far better than full-text boolean retrieval. Oedal's recall can be improved by using heuristics to widen the search space. However, it does lead to some loss in precision.
64
the boolean retrieval scheme by a wide margin. The precision of boolean
retrieval scheme was measured at 10% and that of Dedal at 80% (for exact
match, i.e. when not using retrieval heuristics). The recall was comparable at
around 52%. Dedal's recall performance can be improved by using heuristics to
loosen the search but at the expense of some loss in precision. Nevertheless, the
gain in recall, about 40%, is far more than the loss in precision, about 17%. It is
important to note that boolean or full-text search requires no indexing effort
unlike conceptual indexing which requires effort in indexing. Clearly the gain in
performance justifies the effort required in conceptual indexing, but can this
effort be expected of designers is a separate question. Another question which
rises is what techniques can be used to automate/semi-automate conceptual
indexing?
Information Handling Behavior of Designers During ConceplUai Design: Three Experiments
Chapter 4: Deda!: An Information Management Service 65
4.2.2 Bioreactor Domain Results
The bioreactor domain was used to assess the real-time performance of Dedal.
Dedal was deployed to support indexing and modeling of design information in
real time, i.e design information was indexed and modeled during the design
process. The objective of the bioreactor project was to design an apparatus for
interactive control of a life sustaining environment for bacteria. The bioreactor
project took place in two phases. In the first phase (I) a bioreactor was designed
over a period of 7 months by a team of three designers. This phase resulted in a
working prototype. In the second phase (IT) a team of three different designers
redesigned a subsystem of the bioreactor to improve on its design. The indexed
and modeled documents from bioreactor phase I were made available in bioreac
tor phase II. This was an ideal situation to determine if the effort put into
capturing design information from bioreactor phase I would payoff during
phase II. The indexing and modeling activities in phase I were done by a partici
pant designer (not one of the three principal designers). Following are some of
the observations that were made:
Bioreactor Phase I:
• information framework was adequate for indexing information for reuse
• the information framework was intuitive since the designer was able to index, model and formulate queries without formal training
• it was possible to index and model incrementally with the design process (about 2 hrs per week of additional effort was required)
Bioreactor Phase II:
• information reuse was effective, the design team redesigned and prototyped a new design in two months, relying only on Dedal to learn about the previous design
• on-line access to information was very effective. It contributed to improved design efficiency and practice
Information Handling Behavior of Ocsigners During Conceplual Ocsign: Three Experiment'
Chapter 4: Oeda/: An Information Management Service 66
4.3 Discussion on Dedal: Lessons Learned
Development and deployment of Dedal was an important learning experience. It
convinced me that design information capture has to be an integral part of the
design process and cannot be demanded as an add-on activity. Some how, the
process of structuring and archiving design information for reuse has to be seen
as a requirement of the design process so that it happens incrementally and in
real time.
Dedal has a framework for classifying design information. This framework can
be used for modeling and indexing design information in real time. It also
allows for accurate retrieval. Nevertheless, it has lessons. Some of them are:
• the activity of indexing and modeling are add-on activities.
• the modeling process, although incremental, is static in nature. The user has the burden of monitoring and reorganizing concept names as they evolve, which is a considerable cognitive load.
• the payoff of indexing and modeling was seen mostly by the redesign team and not by the team which did the indexing and modeling. It is my belief that information capture would always remain an add-on activity unless the rewards of these activities were immediate.
• the integration with the information capture environment is only a rough draft. It should be carefully redesigned to empower user indexing and modeling.
• the definition of level-ofdetail was not very useful. Although it was specified during indexing, users did not use it during retrieval. Also the retrieval algorithm did not use this information to any significant advantage.
It was evident from these limitations that development of integral and intuitive
services for capturing and managing design information require a deeper under
standing of the dynamics of information handling behavior. We need to
understand the various activities designers perform with information and how
they change over time in characteristics and content. This understanding would
provide the clues necessary for implementing intuitive and integral services/
interfaces. To my knowledge no one has reported such an understanding. To
Information Handling Behavior of Designers During Conceptual De.~ign: Three Experiments
Chapter 4: Dedal: An Information Management Service 67
develop this deeper understanding I conducted the information handling behav
ior study, discussed in the next chapter.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 5
Information Handling Behavior Framework
/--------{ Information Classification \ \. Framework } ------_/
/-----'" { Dadal Performance \ \. Results /
,...-----, ( Information Handling \ \. Framework /
i i t ------------------------------Analyze Analyze Analyze
1~~I00'f:;r.rmt !~1_19~~ 1-- 995
Information Support by Observe Support by Information Reuse Study Dadal - ...... ~~ Dadal Usa Dedal - ...... ~ Handling
Study
Understanding of Information Handling
This chapter describes the experimental procedure and the analysis method of an observational study conducted to understand the information handling behavior of designers engaged in conceptual design. Information handling is studied in the framework of activities designers perform with information and the characteristics of information handled in those activities. A quantitative measure for design information is defined and used to measure information handling rate. Many examples of the analysis procedure are included in this chapter.
Information Handling Behavior of Designers During Conceprual Design: Three Experiments 68
Chapter 5: Information Handling Behavior Framework 69
5.1 Introduction
The experience of developing and implementing Dedal (Chapter 4) resulted in
many unanswered questions regarding information handling. Some of the key
questions were:
1. What are the activities designers perform with design information?
2. What is the temporal distribution of design information processing?
3. Can design information be measured quantitatively?
4. What is the information content in different activities?
5. How does information processing rate change during the design process? and,
6. What do answers to the above questions inform us about developing intui-tive and integral tools to support information handling?
To find answers to these questions I conducted the information handling behavior
study using the verbal protocol method. The experimental approach was similar
to the one used in the information reuse study, reported in Chapter 3. The differ
ences were in the design problems used in the experiments and the analysis
method. To avoid repetition I would only bring out aspects of the experimental
approach and analysis procedure which are different from the description in sec
tions 3.2 and 3.3.
5.1.1 Experimental Setup and Procedure
A total of six experiments were conducted in this study. The experimental setup
is same as the one used in information handling study (shown in Figure 3-2).
Table 5-1 lists some details on the experimental subjects and the design prob
lems used. Three different design problems were used. They are included in
Appendix C. Two of the design problems dealt with design of a bike-lock and
the third one was about designing a harness for attaching a backpack to moun
tain bikes.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 5: Information Handling Behavior Framework 70
Table 5-1 Details on experiments and experimental subject background. The table lists the design time, design problem and the design experience of each of the experimental subjects.
Design TIme Design Experience (yrs)
Subject (minutes) Design Problem Academic Industrial
Sl 78.8 bike-Iockl 4 2
S2 67.5 bike-Iockl 3 2
S3 71.4 bike-Iock2 I 0
S4 74.4 bike-Iock2 2 4
S5 51.2 bike-Iock2 3 0
S6 119.5 back-pack 0 20
The first two design problems were based on the everyday activity of locking
and unlocking a bike. In short the design problem was to design an off the shelf
lock for bicycles which could be integrated with the bike and operated without
having to take it off the bike (as opposed to kryptonite locks or U locks which
are not integrated with the bike and are operated as an independent unit). This
design problem was chosen because: it required no special technical back
ground, it enabled the designers to work within a space of familiar vocabulary
and technical jargon and it was suitable for an hour long conceptual design exer
cise. The designers had access to a bike and a kryptonite lock.
5.2 Analysis Procedure
The objective of the analysis procedure was to transform raw data from the
experiments into quantitative descriptions of the information handling behavior.
The procedure is shown in Figure 5-1. Some of the steps, conduct experiment and
transcribe, are same as described in section 3.3. The other steps, segmentation and
classification are described in sub-sections below.
5.2.1 Segmenting into Information Fragments
An information fragment is a continuous period during which the informational
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 5: Information Handling Behavior Framework
Design Problem Conduct Recorded Data Transcript
F Experiment Transcribe
Information Fragments
Segment into Information Fragments
Quantitative
Classify Classification Statistical Results
F Fragments Analysis
Figure 5-1 Analysis procedure for information handling study. This procedure transforms raw data into quantitative description of the information handling behavior.
71
activity (activity performed with information) and the descriptor of information
(defined in section 3.4.2) remains the same. Segmentation of the transcript into
information fragments was done by developing a set of rules to identify points
of transitions between fragments. These rules are:
1. Typical points of transition could be semantic such as: change in designer's train of thought, change in focus of attention or change of action.
2. Typical points of transition could also be syntactic such as: significant pause, change in the medium of information or utterance of certain phrases such as "and", "ok", "alright", "umm".
3. The level-ofabstraction during an information fragment remains the same.
4. The level-ofdetail during an information fragment remains the same.
5.2.2 Classifying Information Fragments
Before an information fragment could be classified, a framework for describing
the information handling behavior was needed. Section 5.3 goes into the details
of the development of this framework. For now it is sufficient to note that an
information fragment is classified by
1. specifying the informational activity associated with it (section 5.3.1),
Information Handling Behavior of Designers During ConceplUai Design: Three ExperimenlS
Chapter 5: Information Handling Behavior Framework 72
2. classifying the information in the fragment using a revised design information framework (Table 5-2), and
3. quantitatively measuring the design information content using design information measure (section 5.3.3).
Figure 5-2 shows the template used for this classification. Note that for frag-
(info-fragment :activity infonnational-activity :time (start-time end-time) :descriptor descriptor :subjclass subject-class :subjects (subject-list)
:medium medium :levels (level-of-detaillevel-of-abstraction)
:measures (design-in formation-measure ) )
Figure 5-2 Template for classifying an information fragment. This template captures values for nine attributes of each information fragment. These attributes describe the informational activity and the classification and content of design information in that fragment.
ments rich in content it may be necessary to encode it with two or more
classification.
5.3 Information Handling Framework (IHF)
Table 5-2 shows the information handling framework (lliF) which is used for
describing information handling behavior. This framework is an extension of the
design information framework (DIP) developed in the information reuse study (Table
3-1). Three new attributes were added to the DIP to create the lliF. These are
informational activity, lev~l-of-abstraction and design information measure. Each of
these are defined in detail in sub-sections below. The lliF adequately describes
the activities designers perform with information, and the characteristics and
content of the information handled in those activities. Monitoring design activity
Information Handling Behavior of Designers During Conceprual Design: Three Experiments
Chapter 5: Information Handling Behavior Framework 73
within this framework over time will improve our understanding of the dynam
ics of information handling behavior.
Table 5-2 Information handling framework (IHF). This includes a revised design information framework (DIF) to classify design information. Three new attributes namely, informational activity, level-of-abstraction and design information measure are added to DIF to create the IHF. The values these attributes can take are shown below.
Information Handling Framework (lHF)
Design Information Framework (DIF) Informational Level of
Activity Abstraction Descriptor SubJect-Class Medium Level of Detail
Generate Unlabeled Alternative Assembly Audio Conceptual
Access Labeled Assumption Component Video Configurational
Analyze Associative Comparison Connection Text Detail
Qualitative Construction Feature Graphic
Quantitative Location Requirement Gesture
Operation Design-concept
Performance Other
Design Information Measure (dim) Rationale
Quantitative measure: takes Relation integer values (1,2,3, and so on) Requirement
Miscellaneous
Since the scenario of experiments and the focus of observation in this study was
different from the information reuse study it was necessary to revisit the devel
opment of the OIF using the approach described in Figure 3-4. It is encouraging
to report that the DIF of Table 3-1 required only minor modifications. Two new
values were added for the attribute descriptor. The other attributes remained
unchanged. All attributes making up the DIF are redefined below in the context
of information handling behavior.
Descriptor. This refers to the basic constitution or the nature of the information
in an information fragment. Two new descriptors were added. They are defined
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 5: Information Handling Behavior Framework 74
in Table 5-3. (details on other descriptors are in Table 3-2).
Table 5-3 Descriptors added to the design information framework. Other values are defined in Table 3-2.
Example Information Fragment Descriptor Information regarding from Protocol Explanation
Assumption supposing the truth or validity in general there is probably Designer assumes
of a fact or taking for granted enough room around here that you information regarding could,inside the frame, inside space constraint this tube attach this pivot, slash sliding mechanism,
Comparison examining two or more con- (task of locking} maybe a little Designer compares
ceplS to determine similarities bi t more than 1IIIIIIIIIIl •• , maybe a the task of locking to little bit more than changing a that of changing tires
and dissimilarities tire is permissible
Subject-Oass. This is the class to which the subject of an information fragment
belongs (details in Table 3-3).
Medium. This is the physical form in which the information fragment exists
(details in section 3.4.4).
Level of Detail. It is the phase of the design process to which the information
fragment can be associated with. (details in section 3.4.5).
5.3.1 Informational Activity
The activities designers perform with information will be called informational
activities, in this thesis. It is possible to view all actions as being performed with
information. However, for the purpose of this study, only actions which are
explicitly captured in the protocol data and which need to be supported by com
putational services are of interest. Abstract and unobservable actions such as
thinking, observing, speculating, ideating are not relevant to this study. Figure 5-
3 shows the process of developing the informational activities classification. In
the initial stages this process was tightly coupled with the development of rules
for segmentation of the transcript. This is because some transition points are at
the start and end of an informational activity, and therefore without a classifica
tion on activities it is not possible to identify transition points consistently and
objectively.
Information Handling Behavior of Designers During Conceptual DC5ign: Three Experiments
Chapter 5: Information Handling Behavior Framework
< Information Fragment
t step 1 Characterize informational action ~
, I \.... • "-.....
> --
..- ----" Segmentation: \ ~ Identify transition points I ,_ .......
'-------/
//" No / \" ~
/ / \ " / " , I \ .... step 2 step 6
Collection of all informational actions Can all info-fragments be classified?
~""-/~ / Group inlo Infannational Activities /.tep 5
t step 4 Classify N.:,teP 5 Label Activities --•• ~ Info-Fragments
Yes
~
Figure 5-3 Evolution of the informational activity classification. The cloud on the top right hand side signifies that the process of segmentation was coupled with the evolution process (in the initial stages).
75
The resulting classification of informational activities is shown in Table 5-4. It
shows that informational actions can be grouped into three classes namely Gener
ate, Access and Analyze. This is a minimal view of the informational behavior of
designers. However, it was possible to classify all actions that were observed in
the experiment in these three activities. This classification is a suitable level of
granularity for the purpose of this study. Note that this classification is most suit
able for a single designer scenario. A team design scenario would require a
different classification.
To better understand this classification, assume that at any time in the design
process there is an information space associated with the design. At the begin
ning, this information space may be thought of as containing the design
requirements. As the designer works on the deSign, he/she makes changes to
the information space, either by adding information to it by generating/accessing
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 5: Information Handling Behavior Framework 76
it or they transform information in the space by analyzing it.
Table 5-4 A classification for the Informational activities. Their definitions, associated actions and example. The list of associated actions is a representative list, not an exhaustive list.
Informational Defioition/lnterpretation and Example Information Activity Associated actioM Fragment from Protocol Explanation
Generate An action which adds new informa- so umm I wonder if we Actiqn: Talk
tion to the information space from could do a combination Designer generates an of a the 0 lock type
an unidentified source. Actions: arrangement and the alternative of combining two
~ting,dr.awing,tUUdng doughnut type arrangements
arrangement
Access An action which references informa- {reading from dl} Action; Bead
tion within or outside the informa- Locking mechanism Designer accesses a should be an off the
tion space from an identifiable shelf item requirement by reading it.
source. Actions: read, listen, recall, request
Analyze An action which changes an than you really have to ActioQi ~11I1Iti:
attribute of the information frag- work it {apply force} Designer analyzes the bending this tube to
ment Action: interpret, organize, get this thing out, performance on strength by
calculate, evaluate making a qualitative evaluation of strength
5.3.2 Level of Abstraction
Information in design evolves from abstract to concrete. Thus the ability to repre
sent and monitor this evolution is essential to studying information handling.
While level-ofdetail was defined in the DIF to address this concern, its granular
ity and definition was not adequate to empower structuring or retrieval of
information (section 4.3). Level-ofabstraction has been defined as a substitute for
level-ofdetail. It represents the evolution of concepts in the design process.
Table 5-5 shows the different level-ofabstractions used in this study along with
their definitions and interpretations. The definition and granularity of these lev
els is based on observations from the experiments and therefore are most
suitable to represent the information encountered in the conceptual design pro-
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 5: Information Handling Behavior Framework 77
cess. The levels in the table are arranged in a decreasing order of abstractness.
Table 5-5 Definition and interpretation for level-at-abstraction. The levels are arranged in decreasing order of abstractness. going from top to bottom. Subject" in the table refers to the subject of the information fragment.
Level of Example Information Abstraction DefinitionlInterpretation Fragment from Protocol Explanation
unlabeled Referring to the subjectof an inforrna- so its not really a The designer makes no
tion fragment without a name, as in a wedge its a what do mention of the name for we call it. okay. the subject (Kreally not a
idea or a new concept what I know what it wedge") that helshe is looks like. is chat describing right?
labeled Referring to the subject of an inforrna- umm but the reason The subject under
tion fragment by a name, either by gen- they are chains. I consideration is think is that you can mentioned by name i.e.
erating a new name or reusing a previous have any size bike to Kchains"
name lock.
associative Referring to the subject of an infonna- where aaa it kind of The subject (Kit") of the
tion fragment in relations to or by asso- hooked underneath. fragment is associated hooked underneath the with the -Wheel- and
ciating with other concepts/subjects wheel or its also Kframe-underneath the frame.
qualitative When a subject or features of a subject Citadels are probably The feature Kstrength of
are qualitatively addressed, a lot stronger right citadel- is qualitatively here in the middle described as stronger
or or or
When operation or motion is qualita- (unlocking oper) and The operation of
tively described then you put the key unlocking is in turn it • its just qualitatively described spring loaded and its as a sequence of actions just opens up •.
quantitative When a subject or the features of a sub- it looks just by The feature "length- of a
ject are quantitatively described looking at that bike component is it would be limited quantitatively on a mountain bike to described, " 6 to 6 to B inches in 8 inches-length •.•
5.3.3 Design Information Measure (dim)
Measuring information is an old concept. It originated in the field of communica
tions engineering, wherein the quantity of an information message is inversely
related to the probability of receiving that message from a source [Shannon and
Weaver, 1949). This measure is related to the statistical properties of a communi
cation system and does not depend on the semantic content of the message.
[Kim, 1990 and Suh, 1990) suggests a definition of information content from a
cybernetic perspective. Here the quantity of an information message is related to
the effect of the message on a system's ability to achieve a goal. So the measure
of information content is with respect to a certain goal. To use this measure one
Information Haodling Behavior of Designers During Conceptual Design: Three Experimenrs
Chapter 5: Information Handling Behavior Framework 78
needs to define a goal and a probability distribution on information regarding
its potential to bring the system closer to the goal. Neither of these two mea
sures are relevant to the nature of analysis done in this study. This is because the
analysis aims at measuring semantic information and it is not possible to put a
probability distribution on semantic information with regard to a goal.
The design information measure (dim) is a quantitative measure of the amount of
semantic design information contained in an information fragment. The defini
tion is based on the information fragment classification shown in Figure 5-2.
Every unique [descriptor, subject] pair (to be referred to as a DS pair from now
on) in the classification constitute 1 dim of design information. Thus, if a frag
ment is classified by 1 descriptor and 3 subjects (in the subject-list), it forms 3
unique DS pairs and its content is 3 dims. Note that DS pairs represent the seman
tic content of an information fragment. A DS pair is interpreted as; the fragment
contains information regarding the D of S (Figure 3-5). For instance, if a frag
ment was classified as [operation, solenoid], the interpretation suggests that the
fragment has information regarding the operation of solenoid.
5.3.3.1 Definition of dim: Assumptions
Here are a few assumptions made in defining the measure dim. This will help
understand the rules for calculating this measure.
1. The content of an information fragment is encoded in its classification.
2. The measure takes contributions from two attributes in the classification namely: descriptor and subjects (refer to Figure 5-2).
3. Assumptions pertaining to descriptors:
• All descriptors encode equal amount of semantic information. i.e. [Dl, 5] has the same measure as [D2, 5], where Dl and D2 are two distinct descriptors and S is a subject.
• A fragment classified by more descriptors has more content, and the relationship is linear. i.e. all else remaining same, a fragment classified by 2 descriptors has twice the content of a fragment classified by 1 descriptor.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 5: Information Handling Behavior Framework 79
4. Assumptions pertaining to subjects:
• Subjects from all subject-classes encode equal amount of semantic information. i.e. [D, SI] where SI is a SCI is equal in content to [D, S2] where S2 is a SC2. SC stands for subject-class and SCI and SC2 are two distinct subject-classes (for e.g .. assembly and component).
• A fragment which refers more subjects has more semantic content, and the relationship is linear. i.e. all else remaining same a fragment classified by 2 subjects has twice the content of a fragment classified by 1 subject.
5. All values of level-ofabstraction encode equal amount of information, i.e. a labeled OS has the same content as a qualitative OS. Therefore, since each fragment has one level-ofabstraction (by definition), the dim measure is independent of the level-oj-abstraction.
6. All values of level-ofdetail encode equal amount of information, i.e. a conceptual OS has the same dim measure as a detail OS. Therefore, since each fragment has one level-ofdetail (by definition), the dim measure is independent of the level-ofdetail.
7. Since information in any medium can be represented as a OS pair, the measure is independent of the medium of information. i.e. A graphical fragment and a textual fragment with the same OS classification have equal content.
5.3.3.2 Rules for measuring dims
Once a fragment has been classified the rules below help us determine the dim
measure:
1. For fragments with 1 descriptor and 1 subject the measure is 1 dim.
2. For fragments with 1 descriptor and n subjects the measure is n dims.
3. For fragments with n descriptors, the measure depends on how many subjects are associated with each of the descriptor. If descriptor i has Si subjects then the measure is sum of all Si ' s dims.
Table 5-6 shows some examples on how the above rules were used in counting
Information Handling Behavior of Designers During Conceptual Design: Three Expcrimcnl5
Chapter 5: Information Handling Behavior Framework 80
dims for information fragments.
Table 5-6 Examples of dim measurement. Three examples are listed, each using a different rule to measure the information content.
Information Fragment CIassification
Example Information Number Fragment from Protocol Descriptor Subject-list oCdims Explanation
than you really have to Performance strength-of-tube 1 one unique DS pair: work it (apply force) 1. [perfomuuu:e, strength-of-tube] bending this tube to get so the measure is I dim this thing out
so like the lets assume we Alternative lock 2 two unique DS pairs: are gonoa attach it (lock) down-tube 1. [alternatiue, lock] and to the down tube. 2. [alternatiue, down·tube]
so the measure is 2 dims
I am trying to decide if Location bike-lock 1+2=3 This fragment has two classifications there is any apparent
Comparison near-wheel which make three unique DS pairs:
advantage to locking right l.llocation. bike·lock] near this wheel verses down-tube 2. [comparison, near-wheel] and locking on this part 3. [comparison. down-tube] here (down-tube) so the measure is 3 dims.
5.4 Analysis Example
In Table 5-7 below is an example of the analysis procedure from transcription to
the classification of the information fragment, for about 1 minute of verbal pro
tocol data. The data is shown in three different stages: after transcription, after
segmentation and after classification.
Table 5-7 Examples of information handling behavior analysis_ In the table below protocol data is shown at different stages of the analysis. The first and the last row show brief transcript before and after the segment analyzed below_ The rows form a continuum from the protocol, so only the start time for each fragment is shown. The end time is the start time of the next fragment.
After Start1ime Transcription Stamp After Segmentation After Classification
Transcript Sefore: (OO:56:47:22)_: OK I have spent er forty-five minutes or so now forty minutes doing that er so em ... (mutter) going to check every possible location here
Information Handling Behavior of D.:signers During Conceptual Design: Three Experiments
Chapter 5: Information Handling Behavior Framework 81
Table 5-7 Examples of information handling behavior analysis. In the table below protocol data is shown at different stages of the analysis. The first and the last row show brief transcript before and after the segment analyzed below. The rows form a continuum from the protocol, so only the start time for each fragment is shown. The end time is the start time of the next fragment.
After SfartThne 1hmscription Stamp After Segmentation After Classification alright so 00:56:56:20 alright so we do have (info-.fragment we do have one that comes up front :activity ACCESS one that :time (00565620 005659241 comes up :descrtptDr AL7ERNA'IlVE front em :subjclass DESIGN-CONCEPI' really a :subjects (front-position) little bit :medium VIDEO wary about :levels (CONCEP1UAL lABELED1 backpacks :measures (1]1 on em
em really a little bit (info-.fragment fronts of 00:56:59:24
bikes em wary about backpacks on :acttvity ANALY:ZE let's see em fronts of bikes em :tfme (00565924005710061
if we can :descrtptDr AL7ERNA'IlVE
em and :subjclass DESIGN-CONCEPT
there is :subjects (front-position)
that issue :mediumAUD10
of it being :levels (CONCEP1UAL lABELED1
off the :measures (111 side __ 00:57:10:06 let's see if we can em (info-.fragment
and there is that issue :acttvity ACCESS of it being off the :tfme (00571006 005723261 side __ :descrtptDr AL1ERNA1TVE
:subjclass DESIGN-CONCEPI' :subjects (side-position) :medium VIDEO :levels (CONCEP1UAL lABELED1 :measures (1]]
you know 00:57:23:26 you know from the (info-.fragment from the aesthetics standpoint :acttvity GENERA7E aesthetic everybody likes things :tfme (00572326 005730051 standpoint symmetric :descrtptDr REQUIREMENr everybody :subjclass ASSEMBLY likes :subjects (syrnmetry-for-lDoks1 things :medium AUDIO symmetric :levels (CONCEP1UAL lABELED1 and er this :measures (111 is not that
and er this is not that (info-.fragment big a pack 00:57:30:05
em my big a pack :acttvily ACCESS
initial :tfme (00573005 005736001
tests :descrtptDr CONSTRUC7!ON
indicated I :subjclass FEA1tTRE
probably :subjects (size..of-padc}
couldn't :medium AUDIO
have one :levels (CONFIGURATIONAL QUALITATIVE1
right like :measures (1]1
that 00:57:36:00 em my initial tests (info-.fragment indicated I probably :activity ACCESS couldn't have one right :time (00573600 005742141 like that (horizontal :descrtptDr AL1ERNA'IlVE on the back! :subjclass DESIGN-CONCEPT
:subjects (badc-horlzDntal] :medium VIDEO :levels (CONCEP1UAL UNlABELED1 :measures (1]1
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 5: Information Handling Behavior Framework 82
Table 5-7 Examples of information handling behavior analysis. In the table below protocol data is shown at different stages of the analysis. The first and the last row show brief transcript before and after the segment analyzed below. The rows form a continuum from the protocol, so only the start time for each fragment is shown. The end time is the start time of the next fragment.
After StartThne 1hmscription Stamp After Segmentation After Classification
certainly I 00:57:42:14 certainly I would not (fnfo-.fragment would not do it this way :acttutty ANALY2E do it this {vertical in the back} :time (00574214 00574822) way I would :clescriptDr AL1ERNAnvE do it that :subjclass DESIGN-CONCEPr way em in :subjects (back.vertical} fact I :medlum VIDEO would em :levels (CONCEPItJAL UNLABELED) maybe do it :measures (I}) this way
00:57:48:22 I would do it that way (tnfo-.fragment {horizontal in the :acttutty GENERA1E back} em in fact I :time (00574822 00575124) would em maybe do it :clescriptDr COMPARISON this way {horizontal in :subjclass DESIGN-CONCEPr the back with opening :subjects (back·horizontal back-horlzontal·2) on the outside} :medium VIDEO
:levels (CONCEPItJAL UNLABELED) :measures (2})
Transcript After: (00:57:5l:24) : em __ • and em er so we have that as an option here em there would be a standard ca=ier ....
5.5 Summary
In this chapter I have described the observation method for conducting experi
ments and analysis method for transforming protocol data into a classification
which can be quantitatively analyzed. In the next chapter I will describe the
results of the quantitative analysis and the understanding gained from the infor
mation handling study.
Information Handling Beha\ior of Designers During Conceptual Design: Three Experiments
Chapter 6
Understanding Information Handling Behavior:
Quantitative & Qualitative Results
/-------. ( Information Classification \ \ Fmmewo~ I ------_/
/-----"" ( Dedal Performance \ \. Results /
/-----"" ( Information Handling \ \. Fmmewo~ /
t t J---~ --, ------------------------------Analyze Analyze Ana !yze
I \~rb~~~f"ent ;. \ 9~~~F /. \ / 1990-1991 \ 1991-1992 \ 1992 1995\
Information Support by Observe Support by ..... _ .. ~~ Information Reuse Study Dedal - ..... ~~ Dedal Use Dedal Handling
Study
Understanding of Information Handling
This chapter reports on the quantitative and qualitative results of the information handling behavior study. Results on information fragment duration (deltat), design information measure (dim), distribution of time & information in the information fragment attributes (informational activity, descriptor, subject-class, level-ofabstraction), state transition analysis and multiple attribute analysis are reported. Implications of these results improve the understanding of the design process and the development of intuitive and integral information handling services.
[nfonnadon Handling Behavior of Designers During Conccprua/ Design: Three ExperimenlS 83
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Results 84
6.1 Introduction
Analysis of protocol data is a long and arduous process. In this study it took an
average of about one hour to analyze one minute of raw data. This makes it
impractical to conduct a large number of experiments. However, the six experi
ments used in this study are a representative sample of design activity involving
an individual designer doing conceptual design. Therefore, although the results
may not generalize statistically they do raise our level of understanding about
the conceptual design process.
In this chapter, information fragment attributes (or simply attributes) will refer to
the categories used in classifying information fragments, which are: informational
activity, descriptor, subject-class and level-ofabstraction.l In the following sections I
will discuss results on information fragment duration (deltat), design informa
tion measure (dim), distribution of time & information in the attributes, state
transition analysis and multiple attribute analysis. At the end of each section the
gain in understanding about conceptual design and the implications for develop
ing information management too1s2 will be discussed.
Out of the six subjects the results for S33 differ considerably from that of the oth
ers. S3 appeared to be uncomfortable with the talking aloud protocol and had
little design experience. This could have contributed to the differences. How
ever, since the objective is not to generalize the results, I have chosen to include
the results from S3 in all the analysis. The results reported include contributions
from all six subjects. When reporting graphical results, in many cases results for
one subject chosen randomly will be shown in this chapter. Results for other sub
jects and some supporting data will be included in Appendix D.
1. Medium and level-of-detail are intentionally not included in this list. Results on medium are not considered since the nature of protocol experiment methodology biases the data towards the audio medium. Results on level-of-detar1 are not considered since it has been replaced by level-of-abstraction. 2. From the information management viewpoint the functionality of interest is similar to what Dedal was designed for, such as: information capture, information structuring (indexing) and information retrieval. 3. The six experimental subjects in this study will be referred to as 51-56 respectively.
Information Handling Behavior of Designers During ConceplUal Design: Three Experiments
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Results 85
6.2 Information Fragment Duration (deitat)
An information fragment is the unit of measurement in this study as described in
section 5.2.1 and 5.2.2. The duration of an information fragment, to be referred to
by deltat, is the difference between the end-time and the start-time for an infor
mation fragment. deltat is a measure of how frequently designers move between
different kinds of design information 1, and therefore informs us regarding the
dynamics of information handling.
6.2.1 deltat Variation over Time
Figure 6-1 shows the variation of deltat over time for 51. This graph for other sub
jects is similar in characteristics. Altogether, they show that designers
frequently and fluidly change the type of information they are handling, spend
ing between 2 to 35 seconds with one kind of information. The smooth curve in
Figure 6-1 is a polynomial fit to the raw data. It illustrates movements between
small and large information fragments at a level higher than the raw data. It
shows that the behavior is oscillatory, i.e. deltat is neither steadily increasing nor
decreasing.
6.2.2 Statistics on deltat
Table 6-1 shows the statistics on deltat for each of the subject. It lists the design
activity time and the number of information fragments handled by each of the
subject. A total of 1605 information fragments were extracted from 347 minutes
of data. The average deltat ranged from 11 to 19 seconds and the average over all
subjects was about 13 seconds. The distribution of deltat is shown in Figure 6-2.
The figure shows that the mode (most frequent) of deltat is about 6 seconds. The
variation in the mode across the subjects is small.
1. By definition of information fragment, when designers move from one fragment to another they change the information type (section 52.1).
Information Handling Behavior of Ocsigners During Conccplual Design: Three ExpcrimcnlS
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Resuffs
- 30 rn f\ "'0 c: 0 (,) Q) rn -c: 0 ;: ~ :::J 20 C ~ -c: Q)
E C)
~ u. c:
"'" ~~ ~
\ j ~ ~ ,
~ 11:1 th ~
0 10 ;: as E
~~ .... .E c:
30 50 70 TIme (minutes)
Figure 6-1 Information fragment duration vs. time for 81. Note that deltat ranges from about 2 seconds to 35 seconds. The thick curve is a polynomial fit to the raw data showing the movements between short and long information fragments at a higher level than shown by the raw data. It shows an oscillatory behavior.
6.2.3 Discussion on deltat Results
86
I
!It:.t ~
90
The results on deltat inform us that designers move from one type of design
information to another on an average every 13 seconds. Most frequently they
spend about 6 seconds with anyone kind of information. These numbers pro
vide us with an understanding of the pace at which information handling is
taking place during conceptual design. They also explain why paper is the
medium for most conceptual work, as it poses no hindrance to designers in mov
ing from one information type to another. As was the case during the
experiments, using paper as a working medium, designers worked fluidly and
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Resufts
-S1 -52 -53 ·---·54 - - 55 -S6
~--"'~-:"': ... : ... : ... '" ...
00~--~--~10~--~-~2~0~--~~==~3~0~~~ Duration of Information Fragment (deltat seconds)
Figure 6-2 Distribution of information fragment duration. The smooth curves shown here are 10 degree polynomial fit to histograms with bin width of 0.5. For clarity sake, the histograms are not shown here, they are shown in Appendix D.
87
Table 6-1 Information fragment duration (delta~ statistics. The design activity time listed below only includes the sum of the time for all information fragments and hence is less than the total design time listed in Table 5-1. Average deltat=(design time)/(# of info-fragments). Most frequent deltat is read from Figure 6-2 (the pOint of maxima on the curves).
Design1iJDe Number oCinfo- Average dellat Most Crequent Subject (minutes) fragments (seconds) dellat (seconds)
Sl 67.5 323 13 5 S2 49.9 262 11 6
S3 62.9 200 19 6
S4 44.7 220 12 5
S5 42.4 190 13 5
S6 79.6 410 12 6
TotallAvg. 347.0 1605 13 6
with ease. Akin [1978] also found that designers shift attention among different
design issues frequently, and that the design issue being attended to and the rep
resentation used are related.
Being able to work fluidly and with ease is essential during the conceptual
Information Handling Behavior of Designers During Conceptual Design: Three Experimcnl~
Chapter 6: Understanding Information Handling Behavior. Quantitative & Qualitative Results 88
design process. A service aiming to provide support for information manage
ment during the conceptual process should allow designers to work at a pace
suggested by the results on deltat. Next generation design tools should realize
this and build interfaces which are responsive and adapt to the designers pace.
A measure of deltat for emerging media may be useful index for estimating their
power or improvement.
6.3 Design Information Measure (dim)
The quantitative measure for the design information used in this study is called
design information measure. The unit of this measure is dim. Dim has been
defined in section 5.3.3. The knowledge of how much information designers han
dle will provide an insight into the burden faced by designers in managing this
information. Table 6-2 lists in column 2 & 3 the design activity time and the num
ber of dims handled by each of the subjects. A total of 2802 dims were handled in
347 minutes.
6.3.1 Information Handling Rate
The ratio of the amount of information handled (in dims) to the design time (in
minutes) is being called the information handling rate to be measured in dims
per minute (dpm). Dim rate can be calculated globally (total number of dims
divided by total design time) as well as locally for each information fragment
(number of dims in the info-fragment divided by deltat for the information frag
ment). Since the number of dims and deltat vary with the information fragments,
so does the local dim rate. Figure 6-3 shows the variation of dim rate over time
for 54. This graph for other subjects is similar in character. They show that the
local dim rate ranges from 2 to 40 dpm. This means that, based on the definition
of dim, the designers handle anywhere between 2 to 40 [descriptor, subject-class]
pairs in one minute. Table 6-2 shows, in column 4 & 5 the global dim rate and
the average local dim rate for each of the subjects.
Information Handling Behavior of Designers During Conceptual Design: Three Experimenl~
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Resufts
E % -.cg
40
30
CIS 20 a: E c
20
o o
~~ ... I ,~ U
20
~ CI
~ I ~ ~ 1 ~ ~
" IIUIII If II V
40 60 80 TIme (minutes)
Figure 6-3 Information handling rate over time for 54. The dim rate ranges between 2 and 40 dpm. The thick curve is a polynomial fit and illustrates the variation in dim rate at a level higher than the raw data.
89
The global dim rate ranged from 6.2 to 9.2 dpm (average being 8.1 dpm) and the
average of the local dim rate ranged from 10.5 to 13.8 dpm (average being 12.6
dpm)l. The small range on the observed dim rate is an encouraging sign for the
definition of the dim measure. It suggests that the dim rate is measuring some
thing that is within a small range for most humans. It is a hypothesis that it most
closely relates to the information processing ability of humans. The dim rate
may correlate with some of the results from studies in cognitive psychology on
human information processing. However, before these correlations are investi
gated further it is important to measure the correlation with some obvious
factors.
1. The global average and the average of the local averages differ since the duration of information fragments is a variable. If all deltat were the same, i.e. the segmentation process fragmented the data at equal time intervals, than the global and the local averages would be equal.
lnfonnalion Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Results
Table 6-2 Information handling rates and verbal rates for all subjects. dim = design information measure; dpm = dims per minute; wpm = words per minute; Dt = total design activity time in minutes; N =total number of dims handled; Global Dim Rate = N/Dt dpm; Global Verbal Rate = (Total words spoken)! Dt wpm; Cr = Coefficient of correlation between local dim rate and local verbal rate; Cd=Coefficient of determination
Dim Rate (dpm) Verbal rate (wpm)
Subject Dt(min) N(dim) Global Iocalav. Global Iocalav. Cr Sl 67.5 572 8.5 13.3 130.0 15l.2 0.51
S2 49.9 406 8.1 U.8 97.7 122.4 0.49
S3 62.9 387 6.2 10.5 77.6 104.4 0.88
S4 44.7 411 9.2 14.3 88.5 116.4 0.7
S5 42.4 337 8.0 11.6 91.1 105.6 0.52
S6 79.6 689 8.7 13.8 117.3 141.6 0.55
TotallAvg. 347.0 2802 8.1 12.55 100.4 123.6 0.69
90
Cd
0.26
0.24
0.77
0.49
0.27
0.30
0.39
Besides the information processing ability of humans, the dim rate can be a func
tion of variables such as the verbal rate (measured in words per minute, wpm),
design experience, design problem and possibly others. There certainly is some
correlation between the dim rate and the verbal rate of subject, since in the limit
ing case when there is no verbalization the dim rate would be zero since there
would be no basis to measure dim. Also humans talk at a rate within a range
when they need to be comprehended by others. This would suggest that the ver
bal rate would be within a small range. In that case, is the dim rate merely
measuring the verbal rate of the subjects?
Regression analysis was performed between dim rate and verbal rate to deter
mine the extent of correlation between them. Table 6-2 lists the global verbal rate
and average local verbal rate for each of the subject. It also lists the coefficient of
correlation! and the coefficient of determination2 between the local dim rate and
local verbal rate. On an average the coefficient of correlation was about 0.7 and
1. Coefficient of correlation (Cf ) provides the extent to which two variables are related [Runyon 1988. page 166]. Details on how Cf is calculated is given in Appendix D.
2. Coefficient of determination (CdJ provides an estimate on the proportion of variance in one variable due to the other. It indicates the proportion of total variation that is explained by the coefficient of correlation [Runyon 1988. page 210]. Details on how Cd is calculated is given in Appendix D.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 6: Understanding Information Handling Behavior. Quantitative & Qualitative Resuns 91
the coefficient of determination was about 0.4. This means that about 40% of the
variation in dim rate is accounted for by the variation in verbal rate. This level of
correlation is consistent with expectations.
6.3.2 Discussion on Design Information Measure Results
We now understand that designers in the experiments handled anywhere
between 2 to 40 [descriptor, subject-class] pairs in one minute. The average infor
mation handling rate was about 13 dims per minute (dpm). As was expected, the
information handling rate is correlated well with the verbal rate. However, ver
bal rate only accounts for about 40% of the variation in the dim rate. It is a
hypothesis that a share of the remaining contribution to dim rate can be corre
lated with the human information processing abilities [reference1. This is
suggested as possible future work.
Design tool creators can use the information about dim rate to design interfaces
which enable designers to deal with an adequate amount of design information
in a given time. A service should support information generation, access and
analysis at a pace suggested by the observed range on information handling rate.
6.4 .Distribution of Time and Design Information
Knowledge of deltat and dim rate gives an insight into the dynamics of informa
tion handling. However, knowledge of time spent and design information
handled among the different values of the information fragment attributes (the
values are listed in Table 5-2), provides an understanding of where designers
spend the major share of their time and information. This understanding can be
used to focus the development of support services.
6.4.1 Informational Activities Distribution
Figure 6-4 shows the distribution of time and design information across the
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Results 92
informational activities. It shows that subjects spent 55% of their time generating
information, 24% of their time accessing information and 21% analyzing informa
tion. Also, 53% of the design information was generated, while 25% was
accessed and 22% was analyzed. This suggests that conceptual design is mainly
a generative activity. The relative dominance of generative activity indicates that
unless support services can keep pace with generation of new information, they
will fail to support the conceptual design process.
Analyze
Access
Generate
o
"Ndim Dnme
20 40 Proportion (%)
55
60
Figure 6-4 Distribution of time and design information across informational activities. Ndim is the number of dims. More than half the time and information is dealt with in generate, suggesting that conceptual design is generation intensive.
6.4.2 Information Descriptor Distribution
Figure 6-5 shows the distribution of time and design information across the
information descriptors. Most of the time (65%) is shared by four descriptors:
construction (22%), alternative (15.5%), operation (15.1%), and requirement (12%).
Also, about 60% of the total information handled belonged to these four descrip
tors. This suggests that developers of information handling services should
identify the characteristics of these descriptors in the context of computational
support (representations, media and interfaces) and give adequate attention to
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative ResuUs
Miscellaneous •• ..,
Requirement .iiiiii ••••• L---, Relation .iiiiiiiiiiiiii------J
Rationale ~iiiiiiiii. Performance .iii •••••
Operation .iii ••• i ••••••••• Location •••• ii~------.....J
~Ndim c::J TIme
Construction ••••••••••••••••• ---,
Comparison __ •• ijil------------~
Assumption .iiii~ Altemative ••••••••••• ---.
10 20
Proportion (%)
Figure 6-5 Distribution of time and design information across descriptor. Ndim is the number of dims. Major share of time and design information is grouped under the categories: construction, alternative, operation and requirement. Representations, media and interfaces which facilitate handling of these information types would cover a significant share of conceptual activity.
93
30
integrating these characteristics. For instance information about operation is often
best captured and described by a visual medium (3-D geometric model or video)
and information about alternatives by a decision tree or hypertext tool. Identify
ing a collection of such characteristics so that they can be made available to
designers in an integrated seamless fashion will go a long way in supporting
information handling.
6.4.3 Information Subject-Class Distribution
Figure 6-6 shows the distribution of time and design information across the sub
ject-classes. A major share of the time is spent in handling information associated
Information Handling Behavior of DesignetS During Conceptual Design: Three Experiments
Chapter 6: Understanding Information Handling Behawor: Quantitative & Qualitative Resufts 94
with assembly (29%). The distribution is quite even among component (17.4%),
design-concept (15.7%) andfeature (13.8%). About 75% of the information handled
belonged to these four subject-classes. Again information handling tools should
give adequate attention to representations, media and interfaces which make
handling information belonging to these subject-classes easier.
Other.--..,
Design-Concept •••••••• L-,
Requirement ••••••
Feature ••••••• ...,
Connection •••••••
"Ndim o Time
component ••••••••••••
Assembly ••••••••••••••• ...,
o 10 20
Proportion (%) 30
Figure 6-6 Distribution of time and design information across subject-class. Ndim is the number of dims. Assembly, component, design-concepts and feature account for a large share of time and information.
6.4.4 Information Level of Abstraction Distribution
Figure 6-7 shows the distribution of time and design information across the level
of-abstraction. All levels except quantitative are well represented. The distribu
tion of time is unlabeled (21%), labeled (28%), associative (22%), qualitative (22%)
and quantitative (7%). We know that computational tools are very good at repre
senting and manipulating quantitative information. However, 93% of the time
and 94% of the design information during conceptual design is dealt with in non
quantitative levels of abstraction. Over the past few years there has been
progress in dealing with associative and qualitative information [references].
However, support services are yet to become adept at manipulating unlabeled
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Results
Quantitative
Qualitative
Associative
Labeled
Unlabeled
Proportion (%)
"Ndim c:::J TIme
Figure 6-7 Distribution of time and information across level-of-abstraction. Ndim is the number of dims. The plot indicates that, most of the time and information is dealt with in non-quantitative levels of abstraction.
information and capturing its evolution over time.
95
Since level-of-abstraction can be ordered by the extent of detail represented
(abstract to concrete), it is valuable to look at the behavior of level-of-abstraction
over time. Figure 6-8 shows the movement of S4 between the levels of abstrac
tion over time. The graphs for other subjects are similar in characteristics.
They show that designers move from one level to another often without any
apparent pattern. It was also observed that in most cases a new concept started
out as unlabeled and matured to other levels, although not in any particular
order. This means that support for easily transitioning between the different
level-of-abstraction is essential to supporting the conceptual design process.
6.4.5 Discussion on Time and Information Distribution
The distribution of time and design information in each of the information frag-
Infonnation Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Results
Quantitative
Qualitative
Associative
Labeled
Unlabeled
o 10 20 30 40 50 60 70
Time (minutes)
Figure 6-8 Variation in level-at-abstraction over time tor 84. It was observed that concepts often started out at unlabeled level and matured to other levels as they became more concrete. Movement between the different levels happens often. However no pattern is obvious.
96
80
ment attributes, informs us of the types of design information which are
frequently dealt with during conceptual design. This information can be used to
prioritize the functionalities of support services and in selecting representations,
media and interfaces for manipulating information. The challenge lies in identi
fying the characteristics of services to support the behavior observed, and
providing simple and well integrated mechanisms for moving between these
behaviors.
6.5 State Transition Analysis: Statistics and Probabilities
In the previous sections we have developed an understanding of how frequently
designers transition from one information type to another. To further this under
standing, it is important to look deeper at the points of transition. Points of
transition can be defined as the points where the designer changes from one
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Results 97
state! to another for any of the information fragment attributes. State transition
statistics2 will inform us about transitions which are more frequent and there
fore more critical from the viewpoint of developing support. On the other hand
state transition probabilities3 inform us about which transition is most likely
from a given state and can be used in developing adaptive interfaces. In this sec
tion results on state transition statistics will be reported. Results on state
transition probabilities are included in Appendix D. fu the figures below the
arrows represent a transition from and to the states connected by the arrow.
6.5.1 Informational Activity State Transition Analysis
Figure 6-9 shows the state transition statistics for informational activities. With
three distinct states, there are six possible combinations for transitions. Transi
tions between generate and analyze make up 51% and transitions between
generate and access make up 31%, of all the informational activity transitions.
Clearly a tool which makes it easier to move to and from generate will be better
at supporting information handling.
6.5.2 Information Descriptor State Transition Analysis
Figure 6-10 shows the state transition statistics for information descriptor. There are 11
distinct states, therefore 110 types of transitions are possible. For clarity sake, only the
frequent transitions are shown in Figure 6-10. It is evident that transitions to and from
construction are more common than others and altogether they make up 29% of the total
descriptor transitions. Transitions to and from alternative (11 %), operation (9%) and
1. State refers to a particular instance of each of the information fragment attributes as listed in Table 5-2. 2. State transition statistics are calculated by counting the number of transitions of a particular type. For comparison purposes results reported are normalized. So results for each transition type is reported as a ratio (expressed in percentage) of the number of the transition type to the total number of transitions observed. 3. State transition probabilities are calculated under the assumption that a next state is determined by only the current state and not any states prior to it (i.e. the memory of anything but one previous state is erased. This assumption is commonly made in developing Markov chain for a process with discrete states. Such an assumption is valid for a first order analysis.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Results
8
Figure 6-9 Informational activity state transition statistics. All six possible transitions are shown. Transitions to and from generate make up 82% of all the observed transitions.
98
relation (6%) are also significant. An information handling tool should make it easier to
transition to and form construction, alternative, operation and relation type information.
Most transitions between any two states are between construction and alternative (6.6%).
Figure 6-10 Information descriptor state transition statistics Only the most frequent transitions of the 110 possible combinations are shown here. Transitions to and from construction, alternative, operation and relation make up bulk of the transitions. Transition between construction and alternative are most likely {6.6%}.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Resufts 99
6.5.3 Information Subject-Class State Transition Analysis
Figure 6-11 shows the state transition statistics for infonnation subject-class. There are
six distinct states, therefore 30 types of transitions are possible. Once again for clarity
sake only the frequent transitions are shown in Figure 6-11. Transitions to and from
assembly and component make up 54% of all the subject-class transitions. Transitions to
and from feature (19%) are also frequent. Most transitions between any two states are
between assembly and component (13.7%).
Figure 6-11 Information subject-class state transition statistics Only the most Significant of the 30 possible combinations are shown here. Transitions to and from assembly and component make up a large share of the total transitions. Transitions between assembly and component are most likely (13.7%).
6.5.4 Information Level-of-Abstraction State Transition Analysis
Figure 6-12 shows the state transition statistics for information level-of-abstrac
tion. There are 20 types of transitions possible for the five distinct states. Top 10
frequent transitions are shown in Figure 6-12. Transitions among the states unla
beled, labeled, associative and qualitative make up 93.5% of all the level-of
abstraction transitions. The frequency of transitions to and from each of the
states is unlabeled (25%), labeled (53%), associative (34%), qualitative (37%) and quan
titative (6.5%). A critical need for easy transitions between non-quantitative levels
Information Handling Behavior of Designers During ConceplUaI Design: Three ExperimenlS
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Results 100
of abstraction is evident. Most transitions between any two states are between
labeled and associative (19.5%).
8.1
Figure 6-12 Information level-of-abstraction state transition statistics Only the most significant transitions of the 20 possible combinations are shown here. Transitions to and from unlabeled, labeled, associative and qualitative make up a large share of the transitions.
6.5.5 Discussion on State transition Analysis
From this section and the previous sections we have come to understand that fre
quent transitions between different information types is fundamental to the
conceptual design process. State transition statistics deepen our knowledge of
the designers movements between different states for each of the information
fragment attributes. They identify transitions which frequently take place and
therefore are critical to support.
From the viewpoint of most tools and services, mechanisms for supporting tran
sitions are the common places contributing to difficulty in usability. Since most
computation tools are syntax driven a user has to explicitly specify a transition.
For instance if a user wants to draw a circle in the middle of drawing a line, he
or she has to select circle from the menu, then draw the circle and select line to
come back to drawing a line. Paper is popular in conceptual design since it offers
no hindrance in making such transitions. It is anticipated that the knowledge of
Information Handling Behavior of Designers During Conceprual De.ign: Three Experiments
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Results 101
the state transition statistics can be used in developing well integrated services,
which support the frequent transitions well and therefore will be user friendly.
6.6 Multiple Attributes Analysis
The behavior of one attribute at a time informed us about the relative impor
tance of different values of the attributes. However, deeper insight can be gained
by looking at behavior over multiple (two or more) attributes at a time. First
results over two attributes combined together are described followed by results
over all four attributes. Because of space considerations, only some of the combi
nations are discussed here. Appendix D has results for some of the other
combinations.
6.6.1 Informational Activities and Information Descriptor
Table 6-3 shows the distribution of informational activities across the informa
tion descriptors. The information generated most often was of descriptor
alternative (19%) and construction (17%), information accessed most often was of
descriptor construction (25%) and requirement (20%), and information analyzed
most often was of descriptor construction (26%) and performance (21%). Looking
at these combinations one can recommend that a successful support service
should make it easy to generate [alternative, construction] information, access [con
struction, requirement] information and analyze [construction, performance]
information.
Table 6-3 Distribution of information activities across deSCriptors. The numbers below show the proportion of the different information descriptor in each of the informational activities. Two of the most significant combinations for each informational activity are highlighted.
Assn Comp Loca Oper Perf Rati Rela Requ Mise
Gen 6 6 7 12 3 10 7 7 4
Ace 8 3 2 8 11 11 2 3 5
Ana 13 2 4 3 13 0 4 2
Information Handling Behavior of Designers During ConceptUllI Design: Three Experiments
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Results 102
6.6.2 Informational Activities and Information Subject-Class
Table 6-4 shows the distribution of informational activity across the different
subject-classes. It can be observed that most of the information generated was
about component (24%), assembly (17%), design-concept (17%) and feature (16%),
most information accessed and analyzed was about assembly (41% and 26%
respectively). Thus an information handling support service should make it easy
to generate [component, assembly, design-concept, feature] information, access assem
bly information and analyze assembly information.
Table 6-4 Distribution of informational activity across subject-class. The numbers below represent the proportion of the different subject-classes in each of the informational activity. The most frequent combination for each informational activity is highlighted.
Activity Conn. Feature Reqmt. D-conc. Other
Generate 13 16 8 17 5
Access 8 12 13 10 2
Analyze 10 17 14 16 2
6.6.3 Informational Activities and Level-of-Abstraction
Table 6-5 shows the distribution of informational activities across the informa
tion level-of-abstraction. Information is well distributed across all but
quantitative level-of-abstraction. Therefore a support service for conceptual
design should make it easy to generate, access and analyze information at unla
beled, labeled, associative and qualitative level-of-abstraction.
Table 6-5 Distribution of informational activities across level-of-abstraction. The numbers below represent the proportion of the different level-of-abstraction in each of the informational activity. The most frequent combinations for each informational activity are highlighted. It is clear that much of the concentration is in non-quantitative levels.
Activity
Generate
Access
Analyze
Infonnalion Handling Behavior of Designel'5 During Conceptual Design: Three ExperimenlS
Quantitative
3
10
8
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Results 103
6.6.4 Most Frequent Information Fragments
Table 6-6 shows the top nine most frequently encountered combination of the
information fragment attributes. The listing is in the order of the time spent in
each of the combinations. The table also lists the number of dims handled and
the number of information fragments in each of the combinations. About 4.4% of
the total time and 4.8% of the design information is handled in generating qualita
tive operations of an assembly. However the most number of information
fragments were in generating unlabeled alternative design-concepts. Looking at com
binations in this manner can be used in prioritizing the functionalities of support
services.
Table 6-6 Frequently encountered info-fragment attribute combinations. The number of fragments, time and the number of dims of information in each combination are shown along with their proportion form the total.
TIme ndim # of fragments Info- Level of
Activity Descriptor Subject-clas§ A ...... mon* %* % # %
Generate Operation Assembly Qualitative 4.4 4.8 44 2.7
Generate Alternative Design-concept Unlabeled 12.9 3.7 61 2.2 3.2
Access Requirement Assembly Labeled 7.4 2.1 59 2.1 25 1.6 Access Requirement Design-concept Unlabeled 6.1 1.8 35 1.2 32 2.0
Generate Relation Component Associative 5.1 1.5 62 2.2 27 1.7 Generate Operation Component Qualitative 4.6 1.3 51 1.8 26 1.6 Generate Construction Component Labeled 3.8 1.1 31 l.l 22 1.4 Generate Construction Component Associative 3.6 1.0 40 1.4 20 1.2 Access Performance RI Labeled 3.5 l.0 40 1.4 21 1.3
6.7 Summary
Table 6-7 summarizes the quantitative and qualitative findings presented in this
chapter. Altogether this raises our understanding of the information handling
behavior of designers during conceptual design tasks. This understanding has
made it possible to formulate some recommendations for developing successful
information handling support services.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Results 104
Table 6-7 Summary of the key findings and recommendations. IA=lnformational activity; D=Descriptor; SC=Subject-Class; loA=level of abstraction ditn=design information measure; dptn=dims per minute;
Results on: Observation(s) RecoDUDendation~)
deltat average=13 sec; mode=6 sec; useful index for estimating power of range={2 to 35} seconds support
Dim Rate about 8 [D.SC] pairs per minute another measure to estimate power of mode=6 dpm; range={2 to 40} dpm support
Distribution IA generate (53%) emphasis on ease of generating new of dims information
D construction. alternative. operation emphasis on ease of handling these (51%) types of information
SC assembly, component. design-con- emphasis on ease of handling these cept(62%) types of information
LoA all levels except quantitative are well emphasis on ease of handling all lev-represented els of abstraction
State Transi- IA 82% of all transitions are to and from should be easy to move to and from tion Analysis generate generate
D construction (29%). alternative movement to and from these should (11 %). operation (9%) be the easiest
SC assembly and component (54%) movement to and from these should be the easiest
LoA non-quantitative levels (96%) easy transitions between non-quantita-tive levels
Multiple Attribute Most information is generated about Brings out combinations which pro-Analysis qualitative operation of an assembly vide recommendations at a deeper
and similar observations level
Information Handling Behavior of Designers During Conceprual Design: Three ExperimenlS
Chapter 7
Conclusions & Future Work
Design Research
TO
,r Develop Research
Methods
< Goal > shortest path
T~ Understand Design Practice
~~
Deliver High Quality, Low Cost Products
~
TO
. Improve Design
Practice
~o
This chapter revisits the salient points of this thesis. It presents concluding remarks on the information reuse study, the information handling study, and deployment of Dedal. It states the contributions that this thesis is making to the field of design research and presents some opportunities for further research.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments 105
Chapter 7: Conclusions & Future Work 106
7.1 Concluding Remarks
This thesis demonstrates a methodology for incrementally understanding and
supporting the conceptual design process. It does so by reporting on two obser
vational studies and one information management service. It shows that the
verbal protocol method is good for studying individual designers and describes
a procedure for analyzing protocol data to extract understanding of the design
activity and requirements for support services.
By developing and deploying Dedal, this ~sertation demonstrates how require
ments emerging from a protocol study can be used to develop successful
support services. It also presents an information handling framework for studying
the information handling behavior of designers. Using this framework it
describes the gain in understanding about the information handling behavior.
This dissertation has found that the ability to transition frequently between dif
ferent types of information is fundamental to the conceptual design process.
Information handling services available today are mainly syntax driven 1 and fail
to support transitions effectively. In such services, a transition has to be explic
itly specified by the designer, which results in breaking his or her rhythm. On
the other hand, paper presents no hindrance in making transitions, and there
fore is used extensively during conceptual design. This suggests that there is a
need to develop well integrated representations and interfaces which allow the
designer to transition easily between different information types. I hope that the
quantitative measures presented in this dissertation will be used to evaluate ser
vices in their ability to support smooth transitions.
Design is an information intensive activity. The rewards for capturing and struc
turing information during a design process for efficient reuse during or after the
1. An example of syntax driven is: if one wants to change from drawing a line to drawing a circle, he/she has to explicitly pick circle from a menu. When using paper no such change is needed.
Information Handling Behavior of Designers During Conceprual Design: Three Experiments
Chapter 7: Conclusions & Future Work 107
design process are well known. The work reported in this dissertation, my own
design experience and collaboration with other researchers at the Center for
Design Research [Leifer 1990; Lakin et al. 1989; Mabogunje 19931 has resulted in
the following realizations:
• design information capture and structuring should happen during the design process as opposed to after the process,
• design information capture and structuring should be a natural extension of the design process instead of being an add on activity, and
• the rewards of doing information capture and structuring should be available as a natural feedback to the designers doing the capture and structuring, as opposed to only the next generation of designers or re-designers.
7.2 Contributions
This dissertation adds knowledge to the field of engineering design research. It
corroborates verbal protocol approach as a valuable method for studying, under
standing and incrementally improving engineering design activity.
Additionally, this dissertation makes three key contributions, it
• demonstrates a framework for characterizing the information handling behavior of designers. This framework effectively classifies information that should be captured during the design process for effective reuse,
• demonstrates a computational implementation of the design information framework that is intuitive for designE'fs and works in real time, and
• rigorously describes the information handling behavior during conceptual design using quantitative measures and makes recommendations on how these measures can be incorporated in building successful services to support information handling.
7.3 Discussion
The complexity of the design process presents many challenges in understand
ing and supporting it. Although this dissertation shows that protocol studies
have the potential for providing a detail and rich understanding, it also illus
trates that the process of conducting the experiments and analyzing the protocol
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 7: Conclusions & Future Work 108
data is a long, tedious and a subjective process. As a result, it is often not practi
cal to conduct experiments on a statistically large sample for each of the
scenarios and perspectives that are observed in a design process. However,
because of the success of protocol studies the interest in using this approach is
growing. Therefore, like in any new research field, the design research commu
nity faces the challenge of establishing mechanisms for validation and
comparing results across researchers. This will enable new research to build on
top of results already available.
While observational studies provide a rich understanding and lead to require
ments for support services, embodying these requirements in a usable
computational implementation is a challenge and probably a subject of research
as well. I found being involved in real design during the course of conducting
this research very empowering in shaping my judgement and my ability to pick
representations, media and interfaces for computational implementations.
7.4 Future Work
This dissertation raises numerous opportu.'1ities for further research. A natural
extension, in the interest of the iterative methodology presented, would be to
use the recommendations stated in the Chapter 6 to develop support services for
information management. These services could be evaluated using the observa
tional methodology presented in this dissertation, to further refine and improve
information handling support. Few other directions for further research are:
• study information handling in the other phases of the design process, so that support for information management can be continued all along the design process
• investigate correlation between design information measure and cognitive models of human information processing
• investigate relationship between information media and information handling behavior
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 7: Conclusions & Future Work 109
• use the iterative methodology to study, understand and support the design process from a viewpoint other then information handling behavior.
• use the information handling framework to analyze information handling behavior in other design scenarios such as team design
The work presented in this dissertation is only a beginning towards studying,
understanding and supporting information handling behavior during concep
tual design. I hope that the recommendations made in this dissertation will be
useful in building support services. I also hope that the frameworks and the
methodology reported will be employed to study other design scenarios and
other phases of the design process.
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
Bibliography 110
Bibliography
Akin Orner, ''How do Architects Design?", Artificial Intelligence and Pattern Recognition in Computer Aided Design, J. Latombe (Ed.), IFIP, North-Holland Publishing Company, 1978.
Akin, Orner, /I An Exploration of the Design Process", Design Methods and Theories, Vol. 13, No. 3-4, pp 115-119, 1979. Also published in Developments in Design Methodology, Nigel Cross (Ed.), Chichester, John Wiley & Sons, pp 33-56, 1984.
Alexander, Christopher, Notes on the Synthesis of Form, Harvard University Press, Cambridge, MA, 1964.
Antonsson, Erik K., "Development and Testing of Hypothesis in Engineering Design Research", Journal of Mechanisms, Transmissions, and Automation in Design, Vol. 109, pp. 153-154,1987.
Archer, L.B., "Systematic Methods for Designers", Design, Vols. 172-174, 176, 179, 181, 185, 188, April 1963 - August 1964. Also in Developments in Design Methodology, N. Cross (Ed.), Wiley, Chichester, 1984.
Asimov, Morris, Introduction to Design, Prentice Hall, Engelwood Cliffs, NJ, 1962.
Babbie, Earl R.,"The Practice of Social Research", Wadsworth Publishing Co., Belmont, CA, Fourth Edition, 1986.
Baudin, Catherine; Underwood, Jody G. and Baya, Vinod, "Using Device Models to Facilitate the Retrieval of Multimedia Design Information", In Proceedings of the 13th International Joint Conference on Artificial Intelligence, Chambery, France, pp. 1237-1243, August 29-September 2,1993a.
Baudin, Catherine; Kedar, Smadar; Underwood, Jody G. and Baya, Vinod; "Question-based Acquisition of Conceptual Indices for Multimedia Design Documentation", In Proceedings of the 11th National Conference on Artificial Intelligence AAAI-93, Washington D.C., pp 452-458, July 1993b.
Baya, Vinod; Stout, Greg and Vaska, Matt, "Continuously Variable Damper Project", Design Requirement Document, ME210 Fall, Winter and Spring quarter 1989-90, Technical Report, Design Division, Department of Mechanical Engineering, Stanford University, 1990.
Baya, Vinod; Gevins, Jody; Baudin, Catherine; Mabogunje, Ade; Toye, George and Leifer, Larry, "An Experimental Study of Design Information Reuse", In Proceedings of the 4th International Conference on Design Theory and Methodology,
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Bibliography 111
ASME, Scottsdale, Arizona, pp.141-147, Sept. 13-16, 1992.
Baya, Vinod and Leifer, Larry J., II A Study of the Information Handling Behavior of Designers During Conceptual Design", In Proceedings of the 6th International Conference on Design Theory and Methodology, ASME, Minneapolis, Minnesota, pp. 153-160, Sept. 11-14, 1994a.
Baya, Vinod and Leifer, Larry J., ''Understanding Information Management in Conceptual Design", Chapter in Analysing Design Activity, Cross, N.; Christiaans, H. and Dorst, K. (Eds.), John Wiley & Sons, pp. 151-168, 1996.
Baya, Vinod and Leifer, Larry J., ''Understanding Design Information Handling Behavior Using Time and Information Measure", In Proceedings of the 9th International Conference on Design Theory and Methodology, ASME, Boston, Massachusetts, DE-Vol 83, Volume 2, pp. 555-562, Sept. 17-20, 1995.
Bemers-Lee, T.J. and Cailliau, R. ''World-Wide Web", Computing in High Energy Physics 1992 Conference, Anneey, France, 23-27 September 1992.
Blair, D.C. and Maron, M.E., "An Evaluation of Retrieval Effectiveness for a FuUText Document Retrieval System", Communications of ACM, 28,3, pp. 289-299, March 1985.
Blessing, L.T.M., IIA Process-Based Approach to Computer Supported Engineering Design", In Proceedings of the International Conference on Engineering Design (lCED-93), The Hague, pp.1393-1400, August 1993.
Chovan, John D. and Waldron, Majula B., liThe Use of Function and Form When Reading Mechanical Engineering Drawings by Experts and Non-experts", In Proceedings of the International Conference on Design Theory and Methodology (DTM-90), Chicago, illinois, pp. 137-144, Sept. 1990.
Conklin, Jeff and Begeman, M. L., "gffiIS: A Hypertext Tool for Exploratory Policy Discussion", In Proceedings of the 1988 Conference on Computer Supported Cooperative Work, Portland, Oregon 1988.
Court, Andrew W.; Culley, Stephen J. and McMahon, Christopher A.; "The Information Requirements of Engineering Designers", In Proceedings of the International Conference on Engineering Design, The Hague, pp. 1708-1716, August 17-19,1993.
Cross, Nigel, Engineering Design Methods, John Wiley & Sons, July 1990.
Crutcher, Robert J., "Telling What We Know: The Use of Verbal Report MethodolOgies in Psychological Research", In Psychological Science, Vol. 5, No.5, pp. 241-244, Sept. 1994.
Dixon, John, R., "On Research Methodology Towards a Scientific Theory of
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Bibliography 112
Engineering Design", In Artificial Intelligence for Engineering Design, Analysis and Manufacturing (AIEDAM), Volume 1, Number 3, pp. 145-157, June 1988.
Dorst, Kees; Christiaans, H. and Cross, Nigel, (Eds.). Analyzing Design Activity, VViley,Clrichesrex,InK,1996.
Eastman, Olarles M., "On the Analysis of Intuitive Design Processes", Emerging Methods in Environmental Design and Planning, Gary T. Moore (Ed.), Cambridge, MIT Press, pp. 21-37, 1970.
Eder, Ernest, VV., ''Design Science - Meta-Science to Engineering Design", In Proceedings of the 2nd International Conference on Design Theory and Methodology, Chicago, lllinois, pp. 327-335, Sept. 1990.
Ericsson, K. A. and Simon, H. A., Protocol Analysis: Verbal Reports as Data, 2nd edition, Cambridge, MA, MIT Press, 1993.
Fulton, Robert E. and Craig, James, I., Report from a Workshop on "Information Framework Technology for Integrated Design/Engineering Systems", Sponsored by NSF and Georgia Institute of Technology, Callaway Gardens, Georgia, March 12-15,1989.
Greeno, James G., ''Understanding Concepts in Activity", Chapter in Discourse Comprehension, Edited by C.A. VVeaver, ill, S. Mannes and C.R. Fletcher, Lawrence Erlbaum Associates, Publishers, Hillsdale, NJ, pp. 65-95, 1995.
Gruber, Thomas; Baudin, Catherine; Boose, John and VVeber, Jay, "Design Rationale Capture as Knowledge Acquisition: Tradeoffs in the Design of interactive Tools", In Machine Learning: Proceedings of the Eighth International Workshop, San Mateo, CA, Birnbaum, L. and Collins, G (Ed.), Morgan Kaufman, pp.3-12,1991.
Guindon Raymonde, "Designing the Design Process: Exploiting Opportunistic Thoughts", Human Computer Interaction, Vol 5, pp. 305-344,1990.
Hales, Crispin, "Analysis of the Engineering Design Process in an Industrial Context", Ph.D. Thesis, Department of Engineering, University of Cambridge, 1987.
Hundal, M.S., "A Computer-Based Approach to Conceptual Design of Engineering Systems", In Preprints of NSF Engineering Design Research Conference, Amherst, Massachusetts, pp. 251-265, June 1989.
Kim, Steven, H., Designing Intelligence: A Framework for Smart Systems, Oxford University Press, 1990.
Kuffner, Tom A. and Ullman, David G., "The Information Requests of Mechanical Design Engineers", Design Studies, Vol. 12, No.1, pp. 42-50, 1991; also in Proceedings of 1990 Design Theory and Methodology Conference, ASME, pp.
Information Handling Behavior of Designers During Concepllllll Design: Thn:e Experiments
Bibliography 113
167-174, Chicago, September 1990.
Lakin, Fred, "Spatial Parsing for Visual Languages", Chapter in VISUAL LANGUAGES, edited by Shi-Kuo Chang, Tadao Ichikawa and Panos. A., Ugomenides Plenum Press, 233 Spring Street, New York, NY, 1986.
Lakin, Fred; Wambaugh, John; Leifer, Larry; Cannon, Dave and Sivard, Cecilia, "The Electronic Design Notebook: Performing Medium And Processing Medium", THE VISUAL COMPUTER: International Journal of Computer Graphics, edited by Makkuni, Ranjit, Springer Verlag publishers, 5(4),1989.
Leifer, Larry J., "Generation and Conservation of Design Knowledge", Original Proposal to the AI Branch, NASA Ames Research Center, Moffett Field, CA, 1989. Also Center for Design Research Technical Report CDR 1990.
Leifer, Larry J., "Concurrent Engineering Design and Management Knowledge Capture", Technical Report CDR-NASA-90-3, Center for Design Research, Stanford University, March 1990.
Leifer, Larry J., "Instrumenting the Design Process: For Real Time Text-Graphic Design Process Records", In Proceedings of the 1991 International Conference on Engineering Design, ICED-91, Zurich, pp. 314-321, August 27-29, 1991.
Mabogunje, Ade, "Harnessing Questions in Mechanical Design", Engineers Thesis, Department of Mechanical Engineering, Stanford University, July 1993.
Mauldin, Michael L.,"Retreival Performance in FERRET: A Conceptual Information Retrieval System", In Communications of ACM, 448,1, pp. 347-355, 1991.
Minneman, Scott L., "The Social Construction of a Technical Reality: Empirical Studies of Group Engineering Design", Ph.D. Thesis, Department of Mechanical Engineering, Stanford University, December 1991.
Minneman, Scott L. and Bly, Sara, "Experiences in the development of a Multiuser Drawing tool", Proceedings of the third Guelph Symposium on Computer Mediated Communication, Guelph, Ontario, pp. 154-167, May 1990.
National Materials Advisory Board, NMAB-455, "Enabling Technologies for Unified Life-Cycle Engineering of Structural Components", National Academy Press, Washington, D.C., 1991.
Newell, Allen, "On the Analysis of Human Problem Solving Protocols", Calcul et Formalization dans les Sciences de L 'Homme, J. c. Gardin and B. Joulin (Eds.), Paris, pp. 145-185, 1968.
Newell, Allen and Simon, Herbert A., Human Problem Solving, Englewoods Cliffs, NJ, Prentice Hall, 1972.
Infonnalion Handling Bchavior of Designcrs During Conceptual Design: Three Experiments
Bibliography 114
Newsome, Sandra L. and Spillers, William R., "Tools for Expert Designers: Supporting Conceptual Design", Design Theory '88, S. L. Newsome (Eds.), Springer-Verlag, pp. 49-55, 1989.
O'Shanghnessy, K. and Sturges, R. H. Jr., "A Systematic Approach to Conceptual Engineering Design", In Proceedings of the 4th International Conference on Design Theory and Methodology, ASME, Scottsdale, Arizona, pp. 283-291, Sept. 1992.
Pahl, G. and Beitz, W., Engineering Design, Design Council, London, 1984. (Original German version published in 1977)
Payne, John W., "Thinking Aloud: Insights Into Information Processing", In Psychological Science, Vol. 5, No.5, pp. 241-248, Sept. 1994.
Perlman, Gary, "Descriptive Models of Cognitive Aspects of the Engineering Design Process", Design Theory '88, S. L. Newsome (Eds.), Springer-Verlag, pp. 10-17, 1989.
Rabins, Michael; Ardayfio, David; Fenves, Steven; Seireg, Ali; Nadler, Gerald; Richardson, Herbert and Clark Howard, "Design Theory and Methodology: A New Discipline", Mechanical Engineering, pp. 23-27, August 1986.
Regan, Melissa and Sheppard, Sheri, "Interactive Multimedia Courseware and the Hands-on Learning Experience: An Assessment Study", Journal of Engineering Education, pp. 1-9, April 1996.
Roozenburg, N. and Cross, N., "Models of the Design Process - Integrating Across the Disciplines", In Proceedings of the International Conference on Engineering Design (lCED-91), Zurich pp. 186-193, August 1991.
Rouse, William B. and Boff, Kenneth R., "Designers, Tools and Environments: State of Knowledge, Unresolved Issues, and Potential Directions", System Design: Behavioral Perspectives on Designers, Tools, and Organizations, William B. Rouse and Kenneth R. Boff (Eds.), New York, North-Holland, pp. 43-63, 1987.
Runyon, Richard P. and Haber, Audrey, Fundamentals of Behavioral Statistics, Sixth edition, Random house, New York, 1988.
Schon, Donald A. and Bucciarelli, Louis L., "Design Theory and ~fethods - An Interdisciplinary Approach", Design Theory '88, S. L. Newsome (Eds.), SpringerVerlag, pp. 29-35, 1989.
Shannon, C.E. and Weaver, W, The Mathematical Theory of Communication, Urbana: University of lllinois, 1949.
Simon, Herbert A., The Sciences of the Artificial, 2nd Edition, MIT Press, Cambridge, Massachusetts, 1992 (originally published in 1969).
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Bibliography 115
Sivard, Cecilia; Zweben, Monte; Cannon, David M.; Lakin, Fred; Leifer, Larry J. and Wambaugh, John, "Conservation of Design Knowledge", In Proceedings of the 1989 American Institute of Aeronautics and Astronautics, AJAA-89, Washington DC,1989.
Spillers, W.R. and Newsome, S.L.,"Engineering Design, Conceptual Design, and Design Theory: A Report", In Design Methodology and Relationships with Science, M. J. de Vries et al. (Eds.), Kluwer Academic Publisher, Netherlands, pp. 103-120,1993.
Stauffer, Larry A.," An Empirical Study on the Process of Mechanical Design"; Ph.D. thesis, Oregon State University, September 1987.
Stauffer, Larry A. and Ullman, David G.," A Comparison of The Results of Empirical Studies into the Mechanical Design Process", Design Studies, Vol 9, No 2, pp. 107-114, April 1988.
Stauffer, Larry A. and Ullman, David G.,"FundamentaI Processes of Mechanical Designers Based on Empirical Data", Journal of Engineering Design, Vol. 2, No.2, 1991.
Takeda, Hideaki; Hamada, Susumu; Tomiyama, Tetsuo and Yoshikawa, Hiroyuki, "A Cognitive Approach to the Analysis of Design Process", In Proceedings of the International Conference on Design Theory and Methodology (DTM-90), Chicago, lllinois, pp. 153-160, Sept. 1990.
Tang, John C. "Listing, Drawing, and Gesturing in Design: A Study of the Use of Shared Workspaces by Design Teams", Ph.D. thesis, Department of Mechanical Engineering, Stanford University, April 1989.
Tang, John, C. and Minneman, Scott L.,''VideoWhiteboard: Video Shadows to support remote collaboration", Proceedings of the Conference on Computer Human Interaction (CHI) '91, New Orleans, LA, pp. 315-322, April 28 - May 2, 1991.
Tong, Richard M.; Appelbaum, Lee A. and Askman, Victor N.,"A Knowledge Representation for Conceptual Information Retrieval", In International Journal of Intelligent Systems, Vol. 4, pp. 259-283, 1989.
Toye, George; Cutkosky, Mark R.; Leifer, Larry J.;Tenenbaum, J. M. and Glicksman, Jay, "SHARE: A Methodology and Environment for Collaborative Product Development", In Post-Proceedings of the IEEE Infrastructure for Collaborative Enterprise, 1993,
Ullman, David G.; Dietterich, Thomas G. and Stauffer, Larry A., "A Model of the Mechanical Design Process Based on Empirical Data", AI EDAM, 2(1), pp. 33-52, 1988.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Bibliography 116
Ullman, David G., IIDesign Histories: Archiving the Evolution of Products", In Proceedings of the DARPA WOrkshop on Manufacturing, Salt Lake City, Utah, February 5-6, 1991.
Valsiner, Jaan (Ed.), The Individual Subject and Scientific Psychology, Plenum Press, New York, 1986.
Waldron, Kenneth J. and Waldron, Manjula B.,"A time Sequence Study of Complex Mechanical System design", Design Studies, Vol. 9, No.2, April 1988, pp.95-106.
Waldron, M.B. and Waldron, K.J., ''Methods of Studying Mechanical Design", In Mechanical Design Theory and Methodology, Editors: Waldron, M.B. and Waldron, K.J., Springer-Verlag, New York, pp.21-34, 1996.
Wallace, K. M. and Hales, Crispin, "Detailed Analysis of an Engineering Design Project", In Proceedings of the 1987 International Conference on Engineering Design, ICED-87, Boston, MA, pp. 94-102, August 17-20,1987.
Wilson, Timothy D., "The Proper Protocol: Validity and Completeness of Verbal Reports", In Psychological Science, Vol. 5, No.5, pp. 249-252, Sept. 1994.
Wood, Kristin L. and Antonsson, Erik K., "Engineering Design - Computational Tools in the Synthesis Domain", The Study of the Design Process: In Proceedings of NSF Workshop on Design Theory and Methodology, Oakland, CA pp. 337-360, February 1987.
Information Handling Behavior of Designers During ConcePlual Design: Three ExperimcnlS
Appendix A: Additional Material on Design Information Reuse Study 117
Appendix A
Additional Material on Design Information Reuse
Study
A.I Design Experiment Material
• A.I.I Design Experiment Problem Statement
• A.I.2 Description of ME2IO
A.2 Results on individual subjects
A.3 Analysis Examples
A.1 Design Experiment Material
Enclosed is the design problem and a description of first generation design team (ME2IO), used in conducting the information reuse study experiments.
A.l.l Design Experiment Problem Statement
Information Handling Behavior of Designers During Conceptual Design: Three ExpcrimenlS
Appendix A: Additional Material on Design Information Reuse Study
Generation & Conservation of Design Knowledge CDR-Stanford University, NASA-ARC,
Information Reuse Experiment
Programmable Continuously Variable Damper
BACKGROUND:
118
The suspension system in cars determines ride handling and comfort characteristics of the car. The suspension system is a vibrational system with the three essential vibrational components, mass (of the car), spring (per wheel) and damper (per wheel). Softer suspension systems provide a comfortable ride as in luxury cars, whereas stiffer suspensions provide better handling of the road as in racing cars.
The softness/stiffness of a suspension system can be controlled by the design of the damper. Conventional dampers, which are based on gas or hydraulic fluids, produce a damping force (non conservative force) which is a function of the actuation velocity and the damping coefficient. Thus, for a given damper design and road conditions there is no way of controlling the damping force. Therefore, a trade-off is made between the comfort and handling capability of the car, when designing the damper.
The concept of semi-active suspensions, which has been around for over 20 years, is an attempt at improving the above mentioned trade-off. The semi-active nature is achieved by having the ability to continuously vary the damping force developed by the damper, and removing its dependence on the actuation velocity. A computer onboard the car gets inputs from sensors about the speed, steering, braking, road condition etc. and it computes the damping force for dampers at each of the wheel which would result in the best trade-off between comfort and handling. It then sends appropriate signals to the dampers for developing the required force.
One ME210 group in 1989-90 designed and tested a working prototype of a semiactive damper based on the rotary friction-disk concept.
The requirements that the design was to meet were the following:
Maximum resistive force in full ON state: greater than 500 lbs Minimum residual force in OFF state: less than Sibs Draw a maximum power of 72 watts (12 volts at 6 amps available from car
battery). Force response with less than 30-degree phase lag for a 10Hz sine wave input. Damper should have a stroke length of 6 inches (+- 3). Weight less than 12 lbs.
Information Handling Behavior of Designers During Conceptual Design: Three ExperimeDls
Appendix A: Additional Material on Design Information Reuse Study 119
Size less than 66 cubic in. Cost comparable to conventional gas shocks. Should be able to operate in harsh environments seen by external car parts.
- temperature range (-40 to 120 degree F) - water proof - withstand dust/dirt.
Should be maintenance free. Should last longer than 100,000 miles.
REDESIGN PROBLEM:
Design firm A is designing a vehicle for use on a rough desert terrain as well as on regular tar roads. This requires the vehicle to have the ability to change its suspension system characteristics so as to adapt to a particular terrain. The adaptability can be achieved by having a continuously variable damper in the suspension system.
Based on some analysis it has been determined that the damper should develop damping force in the range of 10 lbs to 750 lbs. It has also been determined that the higher limit on the stoke length of the damper should be at least 10 (+-5 from mean position) inches. The vehicle will have a ground clearance more than what is designed in compact cars. This will allow more room and freedom in integrating the damper with the suspension system. The maximum volume available for the damper is 120 cubic inches. The desert has an ambient temperature in the range of 20 to 120 F.
The power available from the onboard battery is 12volts with a peak current of 10 amperes. The damper should be maintenance free.
Design a continuously variable damper which can be used on this vehicle.
Information Handling Behavior of Designers During Conccptual Design: Three Experiments
Appendix A: Additional Material on Design Information Reuse Study
A.I.2 Description of ME210
Generation & Conservation of Design Knowledge CDR-Stanford University, NASA-ARC
Design Experiment
Description of ME210
120
This sheet of paper contains background information on the first generation design of a continuously variable damper which needs to be redesigned. The design was performed in the academic year 1989-90 at Stanford University by a group of three masters level graduate students. The project was part of the requirement for the depth series ME210 A, B and C. The project was sponsored by the electronics division of Ford Motor Corporation. At the end of the academic year the design team delivered a working prototype of a continuously variable damper along with complete documentation of the design and its performance as evaluated by laboratory experiments. The design team worked with a budget of about $13,750.
ME210 is a three quarter depth series in Automation in Machine Design. Approximately 30 students work in groups of two or three through out the series. In the beginning of ME210 A, students are given two short 2 week long projects. These projects are typically run as competition (eg. paper bike race) and students work on these projects from design to fabrication. These projects serve as warm-up exercises and means by which students can make appropriate choice of teammates for the corporate sponsored project which span the rest of A, B andC.
There are about 13-15 sponsored project in any year, and they vary in focus from process design to product design, automobiles to space applications etc. Each team has a coach who guides the team over the course of the project in issues of design, group dynamics, consultation etc. Most coaches are old ME210 students or industry designers with a good deal of design experience.
Design teams report their progress every two weeks to the teaching staff and the coaches. Guidelines for themes of the presentation are provided by the teaching staff to ensure that design teams cover all grounds. The presentation also serve as a means of getting feedback from other students in the class. Students are encouraged to use structured methods such as trade-off analysis, value analysis, how-why analysis as aids in making important decisions and scoping their problem. They are also advised to rapid prototype their ideas to get early answers to important questions and discover any surprises.
At the end of each quarter the design team summarizes its progress in the design requirement document. Thus, the three requirement document capture a
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Appendix A: Additional Material on Design Information Reuse Study 121
good overview of the design process undergone by the design team. In ME210 A the emphasis is on scoping the design problem and identifying the specifications (functional, physical and external) of the design. Teams also list a few promising concepts.
ME210B emphasizes selecting a final concept for the design and developing configurations and other details by rapid prototyping and other structured methods. The report contains documentation on any structured methods used, choices made and reasons for the choices etc.
ME210C deals with detail design, fabrication and testing of the prototype design. The performance of the design is evaluated under laboratory conditions. The teams are encouraged to make recommendation to their sponsors about ways in which the design can be improved.
The above breakdown is only qualitative and serves as a guideline for the teams, and is not strictly imposed. Each of the report has extensive supporting material in the appendix.
Besides the three reports, you will also have access to the personal design notebooks of the three designers, pictures of the prototype and a video of the laboratory setup for testing.
The goal of our experiment is to observe the reuse of design information and subsequently develop tools which would facilitate the reuse.
Information Handling Behavior of Designers During ConceplUaI Design: Three Experiments
Appendix A: Additional Material on Design Information Reuse Study
A.2 Results on individual subj eets
Performance
Relation
Construction
Operation
Alternative
Rationale
Requirement
Location
Miscellaneous
~"~~""'-~"""""'""
.""""""""., [=:J
~,~'-,~ ~
""""
~
~
""""""""'" o 10 20
Proportion of Questions (%)
RS2
RS1
Figure A-1 Distribution of question descriptor for RS1 and RS2. a
30
Requirement k-oo~~'""C""O'-+~""'Il [=:J RS2
~ RS1 Assernblyk-oo~~'""C""O~~
Design-concept k-oo~~------I
Connection Io.:-...--..J
o 10 20 30 Proportion of Questions (%)
Figure A-2 Distribution of question subject-class for RS1 and RS2.
Information Handliog Behavior of Designers During Conceprual Design: Three Experiments
122
40
Appendix A: Additional Material on Design Information Reuse Study
Conceptual t-c-~"T"'O~~-T-o~~
Configurational ~~~...,.--"'"
o 10 20 30 40
c:=J RS2
ISS:S] RS1
50 60 Proportion of Questions (%)
70
Figure A-3 Distribution of question level-of-detail for RS1 and RS2.
A.3 Analysis Examples
Below is an analysis example from about 17 minutes of the protocol data.
123
51: What have I, I made two comments just now. The lever and the heat. If I do the lever. then I may get a better force. And if I put the bub. if I separate the bub and the solenoid. then I may be able to deal with the heat problem. There's some reference material here on the solenoid. Now I'm looking for effect of temperature on the solenoid. Okay, there's some graphs here showing the relationships between the force and the stroke. so part of what I was looking for, force and stroke. Heat.
R2: what page is it?
51: This is E-2 in the appendix. Heat and current. There's current here. but I'ye not yet seen them put it together Temperature and force resistance factor. Okay ...
{a033} {vSO: 4B}
There are two ways to become aware of these dependencies between the heat and the force and stroke. The recommendations has a lot of that detail there. I'm not sure if I'll have been more sensitive to it without the initial questions, but it sure stands out now .. What do you use a transducer for?
R2: Is the transducer a part of the damper?
51: yeah, but I think you didn't do that.
El: we used it only for testing.
51: for testing, okay ... {aOS3} {reading} Ah. This is your bearing. I think I read they had some problem with bearings. But I can't remember. but at least it's good to know this document is there. No problem, it operates in good temperature. What's the C-card?? Adyanced technology. composite. what do you use it for. Probably for testing too. Composite materials.
El: Some other frictional material we looked at.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Appendix A: Additional Material on Design Information Reuse Study 124
51: Ah. Looking for your index.
El: Index starts on like al bl cl.
{v54:22}
51: Okay. Now I'm thinking to myself, what was their friction material. Because that's another way of um, they had .3 coefficient of friction. That's pretty high. If I can go higher. like 5 or 4. that'll be good Get some force Qut of ~ I know what they used •• There's carbon, carbon there's asbestos,
R2: what page is that?
51: This is e23. Terrible coefficients of friction. point .14. 5tatic to dynamic ratio. I don't quite see where we'll be getting all those jerky motions El spoke ~. Oh okay. This funny looking torque curves .. So we want a, yeah. When the ratio is good, then the torque curve is very beautiful. Oh yes, El's a very smart guy ..
R2: What are you looking at now?
51: right now I'm still in the appendix. And I'm looking at the different frictional surfaces that they used. One thing I'd like to establish. was the key information that I want there really is something that has a high coefficient of friction and a good dynamic to static friction ratio. I'm not sure how critical that is, but if I can push up my coefficient of friction, that'll be good. So what else ..
{aI09} {v58:21}
I'm now looking at the original cost proposal. Just to see if there's something, I may have missed that ...
{aI25} {v59:49}
Right now I'm just looking at other materials. I don't think they will be ... {a13?} Okay, there's some test data here. It talks about levers. This is page P-56 ...
{a16l} {vl:03:0?}
Ah, good information here. They have something on the potential friction materials. Which says the carbon fiber will be a good option, but it's very expensive. It's 135 dol~ars and the kevlars are like 35 bucks ...
{al?5} {vl:04:1?}
Here's some information about the electrorheological fluid .. It also shows ~ examples of mono tube that uses electrorheological fluids. I wonder why it was eliminated ... Okay. So let's see .. At this point I'm, I know a bit much more about the problem ... {a205} So maybe I'll call El. Let's talk a little bit .. ~ happens if my leyers had a longer arm?
Table A-1 Example of analysis procedure in information reuse study.
QNo. Question from Protocol After Reformulation Question Classification
b52 If I do the lever, then I may If I use the lever will I get Descr/pror: Relation get a better force. a better force? Subjclass: Component
Level of Detail: Conceptual
b53 And if I put the hub, if I If I separate the hub and the Descrlpror.Relation separate the hub and the solenoid, will that impact Subjc/ass: Assembly solenoid, then I may be able positively on the heat problem? Level of Detail: Conceptual to deal with the heat problem.
Information Handling Behavior of Designers Ouring Conceptual Design: Three ExperimenlS
Appendix A: Additional Material on Design Information Reuse Study 125
Table A-1 Example of analysis procedure in information reuse study.
QNo. Question from Protocol After Reformulation Question Classification
bs4 Now I'm looking for effect ?f What is the effect of Descriptor: RelatiDn temperature on the solenoid. temperature on thesolenoid? Subjclass: Component
Level of Detail: Conceptual
bss Heat and current. There's What is the relation between Descriptor: RelatiDn current here, but I've not yet heat and current to the Subjclass: Feature seen them put it together. solenoid? Level of Detail: Conceptual
bs6 What do you use a transducer What do you use a transducer Descriptor: Rationale for? for? Subjclass: Componellt
Level of Detail: Conceptual
bs7 I think I read they had some Did they have a problem with Descriptor: Performance problem with bearings. But I bearings? Subjclass: Cornponertt can't remember, Level of Detail: Conceptual
bs8 Advanced technology, What do you use composite for? Descriptor: Rationale composite, what do you use it Subjclass: Component for. Level of Detail: Conceptual
bs9 what was their friction what was their friction Descriptor: Construction material. material? Subjclass: Feature
Level ofDetoil: Detail
b60 If I can go higher, like .5 or If I go higher, like .5 or .4, Descriptor: Performance .4, that'll be good. Get some will I get some force out of Subjclass: Feature force out of that. that? Level of Detail: Detail
b61 I don't quite see where we'll Where will we be getting all Descriptor: OperatiDn be getting all those jerky those jerky motions E1 spoke Subjclass: Assembly motions El spoke about. about? Level of Detail: Configtua·
ttona!
b62 I'm looking at the different What different frictional Descriptor: Altemative frictional surfaces that they surfaces did they use? Subjclass: component used. Level of Detail: Conceptual
b63 One thing I'd like to Is it very important that the Descriptor: Requirement establish, was the key coeffiCient of friction be Subjclass: Component information that I want there high and dynamic to static Level of Detail: Conceptual really is something that has a friction ratio be equal? high coefficient of friction and a good dynamic to static friction ratio.
b64 two examples of mono tube that Why did they eliminate using Descriptor: Rationale uses electrorheological electro rheological fluids? Subjclass: Design-concept fluids. I wonder why it was Level of Detail: Conceptual eliminated •..
b6s What happens if my levers had What happens if my levers had Descriptor: Performance a longer arm. a longer arm? Subjclass: Feature
LevelofDetoil: Detail
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Appendix 8: Additional Material on Deda! Development 126
Appendix B
B.1 Dedal Implementation Details
Additional Material on Dedal Development
B.2 Dedal Interfaces for Indexing, Querying and Modeling
B.3 Examples of Heuristics
B.4 References on Dedal Papers
B.l Dedal Implementation Details
• Dedal runs in the Unix environment under Sun as 4.1.3
• Framemaker is the information capture environment
• It is mostly written in Common Lisp
• Heuristics are represented in MRS (Meta Reasoning System). MRS is also is used to do the reasoning and backward chaining during retrieval
• Interface to audio and video players is written in C
• Dedal has also been ported to the Macintosh but has not been tested yet.
• A web based implementation of Dedal is under development. With this, the tasks of indexing, modeling and retrieval will be available over the world wide web.
Information Handling Behavior of Designers During Conceptual Design: Three ExpcrimcDlS
Appendix B: Additional Material on Dedal Development 127
B.2 Dedal Interfaces for Indexing, Querying and Modeling
All the interfaces shown in figures below are from the Unix implementation of
DedaI.
Figure 8-1 Oedal's indexing and querying environment. The same interface is used to indexing (this adds indices to the database) and to query (this searches the indices database).This interface allows users to perform concept based indexing, the scheme resulting from the information reuse study.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Appendix B: Additional Material on Dedal Development
Figure 8-2 Dedal's modeling interface. Users can relate concepts in a model by using this interface. New concepts can either be created here or during indexing.
Information Handling Behavior of Ocsigners During Conceptual Ocsign: Three Experiments
128
Appendix 8: Additional Materia! on Deda! Development
Figure 8-3 Dedal's answers window. Answers to a query are returned in a window shown above. The name of the document is linked to the document. Users can give feedback on whether a answer is relevant or not.
B.3 Examples of Heuristics
129
Here are some examples of heuristics encoded as rules in a lisp like syntax. A
symbol preceeded by a $ sign stands for a variable. $q is the variable for the
descriptor. The : if section of the rule checks for some conditions. If these con
ditions are true then it generates a new-question represented in the : then
part of the rule. The text following rule is the name of the rule.
(rule look-isa
Information Handling Behavior ot Designers During Conccptlllll Design: Thn:c Experiments
Appendix 8: Additional Material on Dedal Development
:if (and (question $q) (object $0) (a-kind-of $0 $p) (unprovable(is-generic-term $p»)
(new-que.tion $q $p nil look-isa»
(rule look-for-specializations :i£ (and
: then
(question $q) (object $0) (a-kind-of $p $0) (choice $dec $p»
(new-que.tion $q $p nil look-for-specializations»
(rule look-isa-for-attribute :i£ (and
: then
(question $q) (attribute $s $0) (isa $0 $p) (unprovable (is-generic-term $p»)
(new-question $q $p $s look-isa-for-attribute»
(rule 100k-for-dependent-attributes-1 :i£ (and
: then
(question $q) (or (question relation) (question rationale» (attribute $s $0) (dependent-on ($s of $0) ($sl of $01»)
(new-que.tion $q $01 $sl 100k-for-dependent-attributes-1»
(rule from-attribute-to-component-operation :weight 2 :i£ (and
: then
(question relation) (attribute $s $0»
(new-que.tion operation $0 nil from-attribute-to-component-operation»
(rule from-attribute-to-component :weight -5 :if (and
: then
(question $q) (attribute $s $0»
(new-que.tion $q $0 nil from-attribute-to-component»
(rule look-for-common-computed-from :if (and
(question $q) (attribute $s $0) (attribute $s2 $02) (attribute $s3 $03) (notsame $s $s2) (is-computed-from ($sl of $01) ($s of $0» (is-computed-from ($sl of $01) ($s2 of $02» (is-computed-from ($sl of $01) ($s2 of $02») : then
(new-que.tion $q $01 $sl look-for-common-computed-from»
(rule look-for-common-computed-from :if (and
(question $q) (attribute $s $0) (attribute $s2 $02)
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
130
Appendix B: Additional Material on Dedal Development
(notsame $s $s2) (is-computed-from ($sl of $01) ($s of $0» (is-computed-from ($sl of $01) ($s2 of $02») : then
(new-qae.t!oD $q $01 $sl look-for-common-computed-from»
(rale look-relation-< :if (and
(question $q) (attribute $s $0) (is-computed-from ($sl of $01) ($s of $0»)
: then (Dew-qae.t!oD $q $01 $sl look-relation-<»
(rale look-related-attribute :if (and
: then
(question $q) (attribute $s $0) (isa $s $p»
(new-qae.tion $q $0 $p look-related-attribute»
(rale look-related-value :if (and
: then
(question $q) (attribute $s $0) (val ($s of $0) $v) (is-symbol $v»
(new-qae.tion $q $v nil look-related-value»
(rale look-parent-of-related-value :if (and
: then
(question $q) (attribute $s $0) (val ($s of $0) $v) (is-symbol $v) (isa $v $p»
(new-qae.tion $q $p nil look-parent-of-related-value»
(rale part-of :if (and
: then
(question location) (object $0) (subpart $p $0»
(new-qae.t!on description $p nil part-of configuration schematic»
(rale part-of-2 :if (and
: then
(question location) (object $0) (subpart $p $0»
(new-qae.t!on construction $p nil part-of configuration schematic»
(rale find-function-component :if (and
: then
(question function) (object $0) (subpart $p $0»
(new-qae.t!on operation $p nil find-function-component»
(rale find-function-connectorl :if (and
(question function)
Information Handling Behavior of Designers During ConcepruaJ Design: Three ExperimenlS
131
Appendix B: Additional Material on Deda/ Development
(object $0) (connects $0 $p1 $p2»)
: then (aew-qge.tion construction $p1 nil find-function-connector1)
(rule find-function-connector2 :if (and
: then
(question function) (object $0) (connects $0 $p1 $p2»)
(new-qge.tion construction $p2 nil find-function-connector2)
(rule find-function-connector3 :if (and
: then
(question function) (object $0) (connects $0 $p1 $p2»)
(aew-qge.tion operation $p1 nil find-function-connector3))
(rule find-function-connector4 :if (and
: then
(question function) (object $0) (connects $0 $p1 $p2))
(new-qge.tion operation $p2 nil find-function-connector4))
(rule basic-part-of :weight-1 :if (and
(or (question description) (question operation) (question construction)
(question analysis) (question performance»)
(question $q) (object $0)
(part-of $p $0) : then (new-qge.tion $q $p nil extended-part-of ))
(rule find-alternatives :if (and
: then
(question alternative) (object $0) (alternative $d $0) (alternative $d Salt) (notsame $0 $alt»)
(aew-qge.tion alternative Salt nil find-alternatives)
(rule find-alternatives-from-decision-point :if (and
: then
(question alternative) (object $0) (alternative $0 $alt))
(new-qge.tion alternative Salt nil find-alt-from-decision-point)
(rule find-alternatives-for-attribute-value :if (and
: then
(question alternative) (attribute $att $0) (decision $d $0 $att) (alternative $d $alt))
(new-qge.tion alternative Salt nil find-alt-from-attribute-value»)
Information Handling Behavior of Designers During ConcepltlaI Design: Three Experimenls
132
Appendix B: Additional Material on Dedal Development
(rule describe-connectors-l :if (and
: then
(question description I (object $01 (connects $0 $pl $p211
(new-qa •• tioD description $pl nil describe-connectors-lll
(rul. describe-connectors-2 :if (and
: then
(question description I (object $01 (connects $0 $pl $p211
(Dew-qa •• tioD description $p2 nil describe-connectors-211
(rule from-attribute-to-requirement-l :if (and
:th.D
(question description I (attribute $s $01 (r-relate ($s of $01 $req $ type I I
(Dew-qa •• tioD requirement $0 $s from-attribute-to-reqmt-lll
(rul. from-attribute-to-requirement-2 :if (and
: then
(question descriptionl (attribute $s $01 (r-relate ($s of $01 $req $ type I I
(new-qa •• tion requirement $req nil from-attribute-to-reqmt-211
B.4 References on Dedal Papers
133
1. Baudin, Catherine; Underwood, Jody G. and Baya, Vmod, "Using Device Models to Facilitate the Retrieval of Multimedia Design Information", In Proceedings of the 13th International Joint Conference on Artificial Intelligence, Chambery, France, pp 1237-1243, August 29-September 2,1993a.
2. Baudin, Catherine; Kedar, Smadar; Underwood, Jody G. and Baya, Vmod; "Question-based Acquisition of Conceptual Indices for Multimedia Design Documentation", In Proceedings of the 11th National Conference on Artificial Intelligence AAAI-93, Washington D.C., pp 452-458, July 1993b.
3. Baudin, Catherine; Underwood, Jody G.; Baya, Vmod and Mabogunje, Ade, "Using Domain Concepts to Index Engineering Design Information", In Proceedings of the meeting of the Cognitive Science Society, Bloomington, Indiana, 1992.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Appendix C: Additional Material on Design Information Handling Study 134
Appendix C
C.1 Instructions and Overview
Additional Material on Design Information
Handling Study
C.2 Design Exercise Problem Statement: Bike Lock 1
C.3 Design Exercise Problem Statement: Bike Lock 2
C.4 Design Exercise Problem Statement: Backpack Harness
Information Handling Behavior of Designers Ouring Conceptual Design: Three Experiments
Appendix C: Additional Material on Design Information Handling Study
Dear Subject:
Center for Design Research Stanford University Design Experiment Date: May 26, 1993
e.l Instructions: Overview
135
Thank you for willing to participate in the design experience you are now starting. The whole experience is anticipated to take about 1.5 hours. The experience will be recorded on audio and video mediums.
I am interested in the manner in which you would be handling information during your design exercise. To get an explicit recording of your activities with information, I request you to talk out loud while you are doing the design. If there are noticeable periods of silence, I would very likely tap you on your shoulder to remind you to talk out loud.
We will be following the procedure below.
1. Warm-up Exercise: 10 min, This is a short exercise to give you an experience in talking out loud.
2. Design Exercise: 1 hour, This is the meat of the experience.
3. Debriefing: 10 min, Answering a small questionnaire.
Some stationery is provided for your use. Feel free to ask me for any clarifications or information during the course of the experience.
Thanking you,
Vinod Baya
Information Handling Behavior of Designers During Conceptual Design: Three ExperimcnlS
Appendbc C: Additional Material on Design Information Handling Study
Center for Design Research Stanford University Design Experiment Date: May 26, 1993
C-2 Design Exercise: Problem Statement Bike Lock 1
136
Background: As you might have experienced, an everyday activity of locking/ unlocking a bicycle with a kryptonite lock is quite an arduous task. It involves seven steps (believe me I have counted) and takes a fairly long time (It takes me on an average 15 seconds to lock or unlock my bike). Imagine it taking you so long to lock/unlock your car! Wouldn't it be nice if there was a more userfriendly bike lock.
Problell1: Design a user-friendly locking mechanism for a bicycle which meets the following requirements.
1. The locking mechanism should be an off the shelf item which can be installed on a bicycle without much difficulty.
2. Once installed on a bike the mechanism should become an integral part of the bike.
3. It should be possible to perform the locking/unlocking operations quickly and in few steps.
4. 'the lock should have the strength equal to or better than that of a kryptor\ite lock.
5. 'the locking mechanism should have a reasonable cost.
6. 'the locking mechanism should be safe to use.
The following assumptions can apply to the design:
1. A. bike is considered locked if either the front or the rear wheel is not free to rotate.
2. 'the lock can be mounted anywhere on the frame of bike.
3. Use the bike provided as the sample of what the lock could be mounted on.
4. '!'he U lock provided can be assumed to be a representative of a kryptonite lock.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Appendix C: Additional Material on Design Information Handling Study 137
5. Design of an appropriate lock/key mechanism can be assumed to be available.
6. While it is sufficient if the bike can be locked in isolation, an ability to lock the bike to a fixed external frame (shown below) would be an added feature of the lock. If bike is locked to a fixed external frame, then the wheels can be free to rotate.
7. One possible solution that is widely used in India is sketched in an accompanying sheet. This is meant only to explain the requirements.
A detail quantitative design solution is not expected. An adequate solution would qualitatively describe the configuration and the operation of the mechanism.
3.5 feet
2.5 feet
Fixed external frame to which the bike should be locked
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
Appendix C: Additional Material on Design Information Handling Study
unlocked position ---II~ /
locked ~( position
locking mechanism
~ rearfork
~ rearwheel
Sample Solution of a Bike Lock which is installed on a Bike, commonly used in India
Information Handling Behavior of Designers During Conceptual Design: Three ExperimcnlS
138
Appendix C: Additional Material on Design Information Handling Study
Center for Design Research Stanford University Design Experiment
Date: March 17, 1994
139
C.3 Design Exercise: Problem Statement Bike Lock 2
Background: As you might have experienced, an everyday activity of locking/ unlocking a bicycle with a kryptonite lock is quite an arduous task. It involves seven steps (believe me I have counted) and takes a fairly long time (It takes me on an average 15 seconds to lock or unlock my bike). Imagine it taking you so long to lock/unlock your car! Wouldn't it be nice if there was a more userfriendly bike lock.
Problem: Design a user-friendly locking mechanism for a bicycle which meets the following requirements.
1. The locking mechanism should be an off the shelf item which can be installed on a bicycle without much difficulty.
2. Once installed, the mechanism should become an integral part of the bike.
3. It should be possible to lock the bike in isolation as well as to a stationary frame. The shape of a typical frame is at the end of this statement. Being able to lock to just this frame would be sufficient.
4. It should not be possible to detach the locking mechaniSm from the bike when it is in the locked position (otherwise it wouldn't be a lock would it?)
5. It should be possible to perform the locking/unlocking operations quickly and in fewer steps.
6. Strength of the lock should be better than or equal to that of a kryptonite lock.
7. The mechanism should have a reasonable cost and be safe to use.
The following assumptions can apply to the design:
1. A bike is considered locked if either the front or the rear wheel is not free to rotate, or if the frame of the bike is not free to move.
2. The lock can be mounted anywhere on the frame of bike.
3. Use the bike provided as the sample of what the lock could be mounted on.
Information Handling Behavior of Designers During Conceptual Design: Three Experir!1C'nls
Appendix C: Additional Material on Design Information Handling Study 140
4. The U lock provided can be assumed to be a representative of a kryptonite lock.
5. Design of an appropriate lock/key mechanism can be assumed to be available.
6. One possible solution that is widely used in India is sketched in an accompanying sheet. This is meant only to explain the requirements. Note that this solution only locks the bike in isolation.
A detail quantitative design solution is not expected. An adequate solution would qualitatively describe the configuration and the operation of the mechanism.
3.5 feet
2.5 feet
Fixed external frame to which the bike should be locked
The sketch for possible solution is same as the one used in problem statement for bike lock 1 (page C-S).
Informnlion Handling Behavior of Designers During Conceptual Design: Three Experiments
Appendix C: Additional Material on Design Information Handling Study 141
A n a i y sin g Des i g n Act i v i t Y -The Delft Protocols Workshop
Instructions: Protocol Study Individual Design
Thank you for agreeing to participate in this research project. We are interested in the ways that designers work on conceptual design projects. This session is being videotaped for later analysis. I am going to ask you to undertake a short design project. Let me assure you that no commercial advantage will be taken of any design work or ideas that you produce.
Because we want to understand what you are thinking during the task, we want you to think aloud continually during the session. This would be as if you are talking to yourself but loud enough for me to hear. In order for you to get used to thinking aloud, I will give you a small problem which has nothing to do with the design task. Try to solve this problem while thinking aloud. This is only a training exercise, which last 10 minutes: so please do not worry if you do not solve the problem. Here is the problem I want you to work on.
Three missionaries and three cannibals are together on one side of a river. They have one rowing-boat, which can hold up to two people. They all know how to row. How can they all reach the other side of the river, given that for obvious reasons there must never be more cannibals than missionaries on either river-bank?
I am now going to ask you to undertake the design project. The time available is limited to two hours. In a moment I will give you a short written design brief for the project. Additional information which you may require during the project is available in an information file which I keep and have ac.:ess to. So, if you think that you need additional information during the project, please ask me for what you need to know. Please be specific about what you ask for. The information available in the file includes both technical and client information. Although access to the file is only available through me, please do not feel constrained about asking for any information that you feel you need during the project. I also have an outside helper who can give me additional information if it is not available in the file. This book TITLE is also available if you need it please ask me for it when you want it and say why you want it. Apart from providing you with information when you ask for it I will not interact with you in any other way. You should try to ignore my presence in the room.
Drawing materials - paper, pens and markers are available for your use. There is also a white board which you may use if you wish. The particular design project which we are asking you to undertake is concerned with a new product related to mountain bikes and backpacking. Therefore we also have in the room a mOlIDtain bike and a backpack, which you may use or refer to as you wish. Please work as you normally would on such a design assignment as this. But remember to keep thinking aloud.
Before I give you the design brief, are there any questions about the procedure? If there are no further questions we will start the session now. I remind you that the maximum time available to you is two hours. I will remind you of the time at 30 minutes and 15 minutes before the end of the session, if you are still working.
(After the designer has read the assignment the experimenter says:)
The bicycle here in in the room is not the same as the Batavus Buster.
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
Appendix C: Additional Material on Design Information Handling Study 142
C.4 Design Expt. Problem Statement: Backpack Harness
Assignment
HiAdventure Inc. is a fairly large US firm (some 2000 employees) making backpacks and other hiking gear. They have been very successful over the last ten years, and are well known nationwide for making some of the best external-frame backpacks around. Their best selling backpack, the midrange Hi Star, is also sold in Europe. In the last one and a half years, this European activity has suffered some setbacks in the market; in Europe internal-frame backpacks are gaining a larger and larger market share.
As a response, HiAdventure has hired a marketin-firin to look for new trends and opportunities for the European markeL On the basis of this marketing report, fiiAdventure has decided to develop an accessory for the HiStar:
A special carrying/fastening device that would enable you to fasten and carry the backpack on mountain bikes. The device would have to fit on most touring- and mountain bikes, and should fold down, or at any rate be stacked away easily. A quick survey has shown that there is nothing like this on the european market.
This idea is particularly interesting for HiAdventure, because the director, Mr. Christiansen, has a long-standing private association with one of the chief product managers at the Batavus bicycle company (one of the larger bicycle manufacturers in northern Europe, based in Holland). Mr. Christiansen sees this as a good opportunity to strike up a cooperation and profit from the European marketing capacities of Batavus.
The Batavus product manager, Mr. Lemmens, is very enthusiastic about putting a combinationproduct on the market, a mountain bike and a backpack that can be fastened to it. The idea is to base the combination-product on the Batavus Buster (a midrange rtlountain bike), and to sell it under the name Batavus HikeStar. The design department at Batavus has made a preliminary design for the carrying I fastening device, but both Mr. Christiansen and Mr. Lemmens are not very content with it. The user's test performed on a prototype also showed some serious shortcomings.
That is why they have hired you as a consultant to make a different proposal. Tomorrow there is going to be a meeting between Mr. Christiansen and Mr. Lemmens, scheduled as the last one before presenting the idea to the board of Bataeus. Before then, they need to have a clearer idea of the kind of product it is going to be, its feasibility and price.
You are hired by HiAdventure to make a concept design for the device, determining the layout of the product, concentrating on
- ease of use - a sporty, appealing form - demonstrate the technical feasibility of the design - make sure to stay within a reasonable price range
You are asked to produce annotated sketches explaining your concept design. Good Luck.
Information Handling Behavior of Designers During Conceptual Design: 11IRC ExperirncDls
Appendix 0: Additional Results on Information Handling Study 143
Appendix D
Additional Results on Information Handling Study
This appendix has results on the information handling study, in addition to
those discussed in Chapter 6. Some supporting data is included as well. In many
cases graphs for each of the exp-subjects are shown to illustrate the nature of the
raw data across all the exp-subjects and for comparison purposes. However, in
cases where the characteristics of the raw data is similar, data for only one exp
subject is shown because of space considerations. Results included are on:
• information fragment duration (deltat)
• information handling rate (dpm)
• information fragment attributes behavior over time
• distribution of time for each attribute
• state transition probabilities
• multiple attribute analysis
0.1 Additional Results on deltat
0.1.1 deltat vs. time
Figure 0-1 to Figure 0-6 show the variation of information fragment duration
(deltat) over time for all exp-subjects. The curve in each of the figures is a polyno
mial fit to the raw data. It shows the movement of the exp-subjects between small
and large information fragments at a level higher than shown by the raw data.
This curve for all subjects on a normalized time scale is shown in Figure 0-7.
Information Handling Behavior of Designers During Conceptual Design: 1luec Experiments
Appendix D: Additional Results on Information Handling Study
-III "C C a (J Q)
~ c a 1a .... :J C -c Q)
E C)
e LL c a :;:; as E .... a -c
20
-III "C C a (J Q)
~ c a
01
Cii :520 c -c Q)
E Cl e u. c a Cii E .... a C
N I ..... ~" '/ \
V' • r-..
:U "y.
I i
. 0 30 . 50_ .
Time (minutes)
Figure D-1 Behavior of deltat over time for 81
.
!
/ 11 ~
\ . . 35
l t ~ rr f'-1
)
~ 'J
55 Time (minutes)
Figure D-2 Behavior of deltat over time for 82.
" ,pi
70
I
/ ~~
~ V
!\ V
Information Handling Behavior of Designers During Cooceprual Desigo: 1luec Experiments
144
I
:-.~ r-.
90
~ N . i'.,
75
Appendix D: Additional Results on Information Handling Study 145
40.-----------------------------------------------------,
-(J)
"C c: o ~30 ~ c: o e :J c E20 CD E C)
e! u. c: o iii 1 0 E o :s
°5~~--~------~25~----~----~45~----~------~6~5------~~ Time (minutes)
Figure 0-3 Behavior of deltat over time for 83.
(i)30 "tJ c: 0 0 CD (J) -c: 0
~
\ :;20 c -c: CD
~ ~ E C)
~ CIS r
1\// .... u.
1'\ L../1 c:
~ 0210
~ :~~ iii E ~ r"- \ .... 0 1: V
1\
\
010 30 50 Time (minutes)
70
Figure 0-4 Behavior of de/tat over time for 84.
Information Handling Behavior of Designers During ConccplUai Design: Tbrcc Expcriments
Appendix D: Additional Results on Information Handling Study 146
i) 2 5 .J D n -2 ii 5 ) J
~ :n IS L. -2 ~
2
40
30
20
-....,
10
°0~------~------~2~0--------~------~4~0--------~------~60 Time (minutes)
Figure 0-5 Behavior of de/tat over time for S5.
en 30 "C c: 0 0 Q) (J) -c: 0
:e 20 :::J
i\ .
/ . Cl
'E I
Q)
E C)
~ u. c:
10 0 :a; E ....
1 V ~, .~l.~ / ,
I"- ~. " 0
:s I 65Ti . } (me (minutes
115
Figure 0-6 Behavior of de/tat over time for S6.
Information Handling Behavior of Designers During Conceptual Design: Three Experimcn~
Appendix D: Additional Results on Information Handling Study
I ..... ~ . /' S5 /'
I l'
I
______ ---....... S2 ,.-~;
_~_- ______ / I, I
\ / /---- I 'J \...,/ I
S1
0.6 0.8 Normalized Time
Figure D-7 de/tat behavior over time for all exp-subjects. The curves are polynomial fit to the raw data. The time on the x-axis is normalized to neutralize the effects of varying design times. The oscillations are more pronounced in some cases than others but clearly de/tat behavior is oscillatory, i.e. it is neither steadily increasing or decreasing. As one moved along the time axis, three regions stand out. Region 1 (O to 0.2): this is the region where designers developed an understanding of the design problem and a strategy to solving it. Region 2 (0.2 to 0.9): this is where the bulk of the problem solving occurred. Region 3 (0.9 to 1.0): this region started soon after exp-subjects were informed of the remaining time and it involved wrapping up the design and describing a solution.
Information Handling Behavior of Designers During ConceplWll Design: Three ExpcrimcnlS
1.0
147
II)
E Q)
E C) as .... -c: .Q as E .... a -c:
'0 .... Q) .c E :::s Z
Appendix D: Additional Results on Information Handling Study 148
0.1.2 Distribution of deltat: histograms
20 20
15 81 15 82
10 / /
10
5
~ -llmllJL 0 TTTn! rm
0 10 20 30 10 20 30 20
~ 15 c: Q)
83 E C)
84 15
as .... - 10 10 c: a ~ E .... 5 a 5
'E -a .... Q)
10 20 30 .c E :::s Z
15 15
/ ~
10 85 10 \ 86
5 5
ill ~~ ~.rrufl n n Ilnllflll 111I1Im-tt o~wwwwww~uuuwww~~~ww~~~ww~ 0
o 10 20 30 0 10 20 30 deltat (seconds)
Figure D-8 Distribution of the information fragment duration. The curves are polynomial fit to the histograms. Histograms are drawn using a bin width of 0.5.
Infonnation Handling Behavior of Designers During Conceptual Design: Three Experiments
Appendix D: Additional Results on Information Handling Study 149
0.2 Additional Results on Information Handling Rate
0.2.1 Information handling rate over time
Figure 0-9 to Figure 0-14 show the variation of information handling rate (dpm)
over time for all exp-subjects. The curve in each of the figures is a polynomial fit
to the raw data. It shows the movements between small and large information
fragments at a level higher than shown by the raw data.
40~--------------------------------------------------~
E a. "C -'s20 as II: E C
\
r\
-- ~ V--'-~-«<1"Ir
~U i~ N
~~O----~-----'3~O----~-----5~IQ~----~----~7~O----~----~90 Time (minutes)
Figure 0-9 Information handling rate over time for 51
Infonnation Handling Behavior of Designers During Concepwal Design: 1'It= ExperimenlS
Appendix D: Additional Results on Information Handling Study
40
E a.
"C -.!20 ta II: E o
o ~
[
"\ ",
"" " f
U . /
~ " ....
I~
/ ~ 'A
25 .45 TIme (minutes)
Figure 0-10 Information handling rate over time for 52
150
. / ~
1A r' .~
~~
I~. V rl V
65
40r-------,-----------------~----------------------------_,
E a.
"C -.!20 ta II: E o
!
I~ o~------~----~~------~------~------~------~~----~--~
Time (minutes)
Figure 0-11 Information handling rate over time for 53
Infonnation Handling Behavior of Designers During Conccpwal Design: Three Experiments
Appendix D: Additional Results on Information Handling Study
E c.. "0 -
40
Q) 20 ca a: E (5
o
•
'f
~4~'~o,l ., '>, j 1",
I/ I V V~
I 2U 4U Time (minutes)
Figure D-12 Information handling rate over time for 54.
151
T
~ r-
r. "'\ I V-
('-~.
ij
60 80
40~------------------------------------~------------------~
E Co
"0 -~ I
ok---------~-------~20--------~------~4nrU--------~-------~tO
Time (minutes)
Figure D-13 Information handling rate over time for 55
Information Handling Behavior of Designers During Conc:eplWll Design: Three Experiments
Appendix D: Additional Results on Information Handling Study 152
40r-------------------~------------------------r_~------_,
E a.
1:1 -'s20 as a: E
C5
Figure 0-14 Information handling rate over time for 56.
0.2.2 Distribution of information handling rate
en E Q)
E C) as ... u.. c: .Q Cii E ... 0 :s "0 ... Q) .c E :::J Z
15 -51 -52 -53 •••• 54
. ·55 -56
10
5
0 I
Dim Rate (dpm)
Figure 0-15 Distribution of dim rate for all exp-subjects. The curves are polynoml~1 fit to histograms now shown here. The average most frequent information hdndling rate (the average of the maxima point of each curve) is 6.1 dpm.
Infomuuion Handling Behavior of Designers During Conceprual Design: Three ExperimenlS
Appendix 0: Additional Results on Information Handling Study
0.2.3 Dim rate vs. Verbal rate: Regression analysis
28
~24 E c.. ~20 CD 16 a: 16 (ij -e ~ 12
~S1
-S2 b:--AS3
"'--"54 +- +S5 -S6
Ok---~~--~~--------~~------~~--~----~----~~
dim rate (dims/min)
Figure 0-16 Regression analysis between dim rate and verbal rate. Only a few data points are shown for clarity sake.The lines are linear regression fit to the raw data. Except for 53 the coefficient of correlation is within a narrow range
153
D.3 Information Fragment Attributes Behavior Over Time
Figure 0-17, Figure 0-18, Figure 0-19 and Figure 0-20 show the behavior of
informational activity, information descriptor, information subject-class and
information level-of-abstraction over time respectively.
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
Appendix 0: Additional Results on Information Handling Study
CI) rJ) CI)
~ rJ) 1ii CI)
"-"iii CJ CI) c: CJ c: < < CI)
(!)
o CD
o ,...
_ .....
rn CD
0_ III ~
C ·e -CD E
oi= ...
o t')
o ""
0 -
Figure 0-17 Behavior of informational activity over time for 81. The figure illustrates that designers move frequently between the different informational activities all along the conceptual design process. No predictable pattern is obvious. The behavior for other exp-subjects is similar in characteristics.
Information Handling Behavior of Designers During Conc:epruw Design: Three Experiments
154
Appendix D: Additional Results on Information Handling Study 155
r - g
- ~
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
Appendix 0: Additional Results on Information Handling Study
.... Q) ~
5 a. Q) o c: o ()
I
c: C)
en Q)
Cl
c Q)
E ~ ·s 0-Q)
ex:
: ::
c: .2 '0 Q) c: c: o U
f
-c: Q) c: o a. E o u
; • •
~
>. :c E Q) en en «
0
"
0 ..,
0 ~
g
o N
Figure 0-19 Behavior of Information subject-class over time 53. The figure illustrates that designers frequently move between the different information subject classes all along the conceptual design process. No predictable pattern is obvious. The behavior for other exp-subjects is similar in characteristics.
Information Handling Behavior of Designers During Concepnml Design: Three Experiments
156
(i) Q) -:l c: g Q)
E i=
Appendix D: Additional Results on Information Handling Study
r-----------------------------~----------~~=======r--------_.~
; I : [ :
I
~ ~ : ~ I ;=:
(J) (J) (J) "0 "0 > > > (J) (J)
E E ~ CD (J)
U .0 .c :;= (ij (\l (\l c 0 --l C tU :l Ul :l a Ul :::> a «
Figure D-20 Behavior ot Intormation level-ot-abstraction over time 54. The figure illustrates that designers move between the different levels ot abstraction often all along the conceptual design process. It was also observed that concepts start at the unlabeled level and mature to other levels as they become more concrete. No predictable pattern is obvious.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
o ,..
0 tn
0 ...
0 t')
0 C\I
5!
157
-Ul (J)
'S c
I (J)
E i=
Appendix 0: Additional Results on Information Handling Study
D.4 Additional Results on Distribution of Time
Analyze
Access
Generate
o 20 40 Proportion of Total Design Time (%)
G--E>S1 G-EJS2 1!r--6.S3 x---~S4
+-+55 ~S6
60
Figure 0-21 Distribution of time across informational activity. The distribution across the different exp-subjects is quite consistent. For all exp-subjects, more than half the time is spent generating information.
Miscellaneous
Requirement
Relation
Rationale
Performance
Operation
Location
Construction
Comparison
Assumption
Altemative
---::::::«
o 20 Proportion of Total Design Time (%)
G--E>S1 G-EJS2 1!r---6.S3 X··_J( S4
+- +55 ~S6
Figure 0-22 Distribution of time across information descriptor. The distribution is quite comparable across the exp-subjects. For all exp-subjects, information regarding I construction, operation, alternative and requirement dominate.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
158
40
Appendix D: Additional Results on Information Handling Study
Other
Design-Concept
Requirement
Feature
Connection
Component
Assembly
o
G---eSl G-£lS2 /!r--AS3 X---i<S4 +- +S5 ~S6
... -... -:~---::-:-:---:-:----~ -. :=1'"-)(
20 40 Proportion of Total Design TIme (%)
Figure 0-23 Distribution of time across information subject-class. The figure shows that the distribution is reasonably consistent for most classes across the exp-subjects. It differs for classes connection and requirement.
Quantitative
Qualitative
Associative
Labeled
Unlabeled
o 20 40
Proportion of Total Design TIme (%)
G---e51 G-£l52 /!r--A53 X---K 54 + ·+55 ~56
Figure 0-24 Distribution of time across information level-af-abstraction. The distribution differs across the exp-subjects.The differences are well distributed across all levels of abstraction.
Information Handling Behavior of Designers During Conceptual Design: Thn:c Experiments
159
Appendix D: Additional Results on Information Handling Study 160
D.5 Results on State Transition Probabilities
State transition probabilities inform. us about the likelihood of a transition from
one state to another. These probabilities are calculated under the assumption
that the next state is determined by only the current state and not any states
prior to it (i.e. the memory of anything but one previous state is erased. This
assumption is commonly made in developing Markov chain for a process with
discrete states. This assumption is being made in obtaining the results below.
0.5.1 Informational activities state transition probabilities
22
Figure D-25 Information activity state transition probabilities. The figure shows that from generate, transition to analyze is most likely, and from access or analyze transition to generate is most likely.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Appendix D: Additional Results on Information Handling Study
D.5.2 Information descriptor state transition probabilities
Figure 0-26 information descriptor state transition probabilities. For clarity sake only the transitions with probability greater than 0.19 are shown here. From most states transition to construction is most likely. From construction, transition to alternative or operation is most likely.
Information Handling Behavior of Designers During Conceprual Design: Three ExperiDll:ms
161
Appendix 0: Additional Resufts on Information Handling Study
0.5.3 Information subject-class state transition probabilities
Figure 0-27 Information subject class state transition probabilities. For clarity sake only transition with probability greater than 0.2 are shown here. Transition to assembly are most likely from most other states.
162
0.5.4 Information level-of-abstraction state transition probabilities
Shown in Figure 0-28.
D.6 Additional Results from Multiple Attribute Analysis
Table 0-4 to Table 0-6 show various combinations resulting from multiple
attribute analysis. In all the tables, significant combinations are highlighted. The
numbers in the table are the fraction (in percentage) of information fragments in
the particular combination to the total number of information fragments.
Information Handling Behavior of Designers During Conceprual Design: Three Experiments
Appendix D: Additional Results on Information Handling Study
Figure 0-28 Information level-of-abstraction state transition probabilities. Transitions with probabilities greater than 0.25 are shown here. No particular transition particularly stands out here.
D.6.1 Information descriptor and informational activities
Table 0-1 Distribution of descriptor across the informational activities.
163
The numbers below represents the proportion of informational activities in each of the descriptors. Much of the information regarding the descriptors is generated. In case of performance most of the information is analyzed and most of the requirements are accessed.
Descriptor Access
Alternative 12
Assumption 17 8
Comparison 11 20
Construction 28 27
Location 32 II
Operation 21 24
Perfonnance 30 • _. --. ,.'. - ,. - 1
: ____ X, _. ~J Rationale 8
Relation 16
Requirement 25
Miscellaneous 12
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
Appendix D: Additional Results on Information Handling Study 164
0.6.2 Information subject-class and informational activities
Table 0-2 Distribution of subject-cass across the informational activities. The numbers below represent the proportion of informational activities in each of the subject classes. For all classes, much of the concentration is in the activity generate. However, a significant number of info-fragments in assembly were accessed and a good number of info-fragments in requirement were accessed and analyzed.
Subject Class Analyze
Assembly 23
Component 18
Connection 20
Feature
Requirement
Design concept
Other 15 15
0.6.3 Information level-of-abstraction and informational activities
Table 0-3 Distribution of level-of-abstraction across informational activities. The numbers below represent the proportion of informational activities in each of the level of abstraction. For all levels, large share of information is generated and nearly equal amount of quantitative information fragments were in each of the informational activities.
Level of Abstraction Access Analyze
Unlabeled 24 20
Labeled 29 23
Associative 16 20
Qualitative
Quantitative
Infnrmation Handling Behavior of Designers During ConcepwaI Design: Three Experiments
Appendix 0: Additional Results on Information Handling Study 165
0.6.4 Information descriptor and subject-class
Table 0-4 Distribution of descriptor across the subject-classes. The numbers below represent the proportion of subject-classes in each of the information descriptor. Most alternatives were about design-concepts. Most assumptions were about assembly, feature and requirements. Most performance were about requirements. Most relations were about components. Most requirements were about assembly.
Descriptor Assem. Comp. Conn. Feature Reqmt. Other
Alternative 8 16 2
Assumption 7 8 8
Comparison 14 2
Construction 4 o Location 0 6
Operation 2 2 11 2
Performance
Rationale 17
-' .. ", ... '. '. ~"I 2 . . . - - ~ ~ '- - . - .
Relation 7 19 2
Requirement 7 11 15 7 o Miscellaneous 10 o 7 3 13
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
Appendix D: Additional Results on Information Handling Study 166
0.6.5 Information descriptor and level-of-abstraction
Table 0-5 Distribution of descriptor across the levels-of-abstraction. The numbers below represent the proportion of level-of-abstractions in each of the information descriptor. most alternatives were unlabeled or labeled. Most assumptions, comparison, performance were labeled. most operations were qualitative.
Descriptor Associative Qualitative Quantitative
Alternative 14 6 2
Assumption 18 13 6
Comparison 15 20
Construction 12 16 14
Location 9
Operation 5 2
Performance 7
Rationale 13 o Relation 20 6
Requirement 4 6
Miscellaneous 3 3 5
0.6.6 Information subject-class and level-of-abstraction
Table 0-6 Distribution of subject-class across the levels-of-abstraction. The numbers below represent the proportion of level-of-abstractions in each of the information subject-class. Most assembly information is labeled. Most components and connections are associated. Most requirements are labeled and most design-concepts were unlabeled.
Subject Class
Assembly
Component
Connection
Feature
Requirement
Design concept
Other
Associative
18
5
IO 13
9 9
Information Handling Behavior of Designers During ConceplUlll Design: Three Experiments
Quantitative
3
2
2
Appendix IE: References on Papers from this Dissertation 167
Appendix E
References on Papers from this Dissertation
E.l ASME Conference: DTM-1992
Baya, Vinod; Gevins, Jody; Baudin, Catherine; Mabogunje, Ade; Toye, George and Leifer, Leifer; "An Experimental Study of Design Information Reuse", In Proceedings of the 4th International Conference on Design Theory and Methodology, ASME, Scottsdale, Arizona, pp. 141-147, Sept. 13-16, 1992.
E.2 ASME Conference: DTM-1994
Baya, Vinod and Leifer, Larry J.; "A Study of the Information Handling Behavior of Designers During Conceptual Design", In Proceedings of the 6th International Conference on Design Theory and Methodology, ASME, Minneapolis, Minnesota, pp. 153-160, Sept. 11-14,1994.
E.3 ASME Conference: DTM-1995
Baya, Vmod and Leifer, Larry J.; "Understanding Design Information Handling Behavior Using TIme and Information Measure", In Proceedings of the 7th International Conference on Design Theory and Methodology, ASME, Boston, Massachusetts, Sept. 1995.
E.4 Chapter in Analysing Design Activity: 1996
Baya, Vmod and Leifer, Larry J., "Understanding Information Management in Conceptual Design", Chapter in Analysing Design Activity, Cross, N.; Christiaans, H. and Dorst, K. (Eds.), John Wiley & Sons, pp. 151-168, 1996.
Information Handling Bebavior of Desigoers During Conceptual Design: Three Experiments