Rigorous Definition of Robustness Analysis

A Rigorous Definition of Robustness AnalysisAuthor(s): H.-Y. Wong and J. RosenheadSource: The Journal of the Operational Research Society, Vol. 51, No. 2 (Feb., 2000), pp. 176-182Published by: Palgrave Macmillan Journals on behalf of the Operational Research SocietyStable URL: http://www.jstor.org/stable/254258 .Accessed: 14/01/2015 04:13

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

Palgrave Macmillan Journals and Operational Research Society are collaborating with JSTOR to digitize,preserve and extend access to The Journal of the Operational Research Society.

http://www.jstor.org

This content downloaded from 202.94.83.200 on Wed, 14 Jan 2015 04:13:42 AMAll use subject to JSTOR Terms and Conditions

journal of the Operational Research Society (2000) 51, 176-182 y(?!2000 Operational Research Society Ltd. All rights reserved. 0160-5682/00 $15.00 http://www.stockton-press.co.ukljors

A rigorous definition of Robustness Analysis H-Y Wong* and J Rosenhead London School of Economics

Robustness Analysis (RA) is a problem structuring method for flexible planning. This paper presents a revised definition of RA which is based on earlier presentations of RA as well as new results derived from empirical research. This paper first identifies the basic concepts that are necessary for a rigorous definition of RA and relationships between these concepts. On this foundation, this paper constructs new definitions of robustness score and discusses their uses.

Keywords: methodology; planning; problem structuring; soft OR

Introduction

Robustness Analysis (RA), is an approach for the evalua- tion of alternative initial strategic decisions, which was first proposed by Gupta and Rosenhead4. Further development in the theory and methodology of RA was presented in Rosenhead. 1-3 Reported applications in the past included the location of production plants,4 sequential development of a chemical plant,5 the location of medical facilities for a new town,6 personal educational career decisions,7 and the regional planning of health care provision decision in Ottawa.8 More recent methodological advances have included the amalgamation of RA and scenario planning for the planning of HIV/AIDS health provision.9

The applications of RA, and the literature on it, are less extensive than those of several of its counterparts in the field of problem structuring methods. One consequence is that its conceptual foundations have not been subjected to the same degree of critical scrutiny. The result is that areas of ambiguity have persisted, which have also impeded the further development of the approach. 10 The principal aim of this paper is to present a reformulation of the foundations of RA on the basis of precise definitions of all the relevant terms. An advantage of these more secure foundations is that their explicitness clarifies the opportunities for further developments, and some resulting extensions of the methodology are also presented in this paper.

It will be helpful at this stage to clarify the status of the mathematical formulations which will be presented in later sections. The starting point is to recognise that RA falls under the heading of a decision-aiding rather than a decision-analytic approach, with both terms taken in a

Correspondence: Dr H- Y Wong, Department of Operational Research, London School of Economics, Hot ghton Street, London WC24 2AE, UK. E-mail: hywacompuserve.com

broad sense. By declining the task of identifying an optimal decision, decision-aiding side-steps the practical difficulties of identifying individual preferences, aggregating utilities and estimating probabilities. This simplification of the analytic task is accompanied, however, by a corresponding loss of analytic power. Decision-aiding therefore throws the burden of forming these judgements and trade-offs back to the decision makers. It can only be effective as an assis- tance to decision making if the elicited information is so structured that the problem, otherwise insurmountable through complexity and uncertainty, is rendered tractable. As decision aiding approaches do not attempt to represent or substitute for the act of decision, the explicit assump- tions or axioms of rational choice required for quantitative approaches such as decision analysis are not needed." "2 Nor is there any scope for algebraic formulations of performance measures designed for algorithmic optimisation. The employment of mathematical formalisms in this paper has the quite different and limited functions, firstly by ensuring that the representations of the problem situation to be employed within RA are unambiguously defined; and secondly, by enabling developments of and operations on these concepts to be explained in a compact manner.

The first part of this paper identifies basic concepts that are necessary to lay down a more secure foundation for RA. The second part specifies the relationships between these concepts. The following sections articulate an extended view of the key relationship among these concepts, namely the attainability of future configurations from initial commitment sets, and derives alternative expressions for computing robustness scores. These extend the range of aspects of option maintenance and closure which the methodology can address. Finally there is a discussion of the significance of adopting this revised view.

The research reported in this paper is complemented by work on the development of a new structured and partici-


HKY Wong and J Rosenhead-A rigorous definition of robustness analysis 177

pative methodology for RA, termed Robustness Analysis Methodology (RAM), which is presented elsewhere.'0

Defining the basic concepts in Robustness Analysis In this section, basic concepts of RA are defined. These concepts, which are necessarily simplified abstractions of features of real life situations, provide a framework for RA and may also offer a clearer understanding of the problem situation being confronted.

Systems and the environment

RA deals with decision problems where the results of decisions taken will lead to changes to the system within its environment. We define a system as a group of interact- ing, interrelated, or interdependent elements forming a complex whole. Environment is defined as the circum- stances or conditions under which the system operates and cannot be controlled. Because only some of these circum- stances or conditions will affect the existence and development of the system within the environment, all aspects of the environment need not to be considered. The relatively loose definition of the terms proposed here will enable us to capture a wide range of situations.

Decision and implementation periods

A decision period is a period of time during which a decision needs to be made from a number of available commitments (to be defined later). An implementation period is a period of time during which the chosen commitments are implemented. Once these commitments are implemented, which usually takes place towards the beginning of the implementation period, no further changes will be introduced to the system until the next implementation period. Each decision period is succeeded by an implementation period. The first decision period is termed the initial decision period. The planning horizon lies beyond the last implementation period. The durations of decision periods and implementation periods are variable, with the former typically short compared with the latter. Figure 1 provides a

I , I X I15 I ". 1 Planning Horizon t 1 2 3

O Pecision period Imrplermentation period

Figure 1 Example of a situation with three decision and implementation periods.

graphical representation of a hypothetical situation with three decision and implementation periods.

Commitments

A commitment is defined as a single unique, indivisible action that will cause specific changes to the system. It is assumed that the implementation of the same actions in different periods can have quite different consequences. Therefore, similar actions that can be implemented in different periods are treated as different commitments. In most cases, the changes introduced by the implementation of a commitment will be irreversible (though the changes made by some commitments may be cancelled by adopting specific further commitments). We use

z,.t

to represent a commitment i that is implementable in period t. For convenience, we put all the commitments that are implementable in period t in the set

z Zt -

In general, a plan consists of commitments to be implemented at each period up to the planning horizon. The actual decision on the implementation of period t commitments will be made in decision period t; decisions on the implementation of commitments in subsequent periods are not required at this stage. In practice, decisions are only made on the commitments that need to be implemented immediately.

Commitment sets

A commitment set is defined as a unique feasible combination of commitments. A commitment set i that is implementable in period t is denoted by

Si, t I

A period t commitment set only contains commitments that can be implemented during that implementation period, so

St t C Zt.


178 Journal of the Operational Research Society Vol. 51, No. 2

It is assumed that if a commitment set Sit is chosen for implementation at decision period t, all of its constituent commitments must be implemented within implementation period t.

Although we can consider commitment sets that may be implemented at different implementation periods, in RA, a significant emphasis is placed on the possible commitment sets at the first decision period. These are termed initial commitment sets, denoted by

Si,. There are potentially a very large number of possible combinations of commitments at each decision period. The nature and content of the problem often makes certain combinations of these commitments of particular interest. Ways of identifying and designing combinations of commitments that are likely to be of interest are suggested elsewhere. ' n

Configurations

A configuration is defined as a form, shape or pattern that the system under consideration may exhibit at the planning horizon as a result of the implementation of a combination of commitments. We use

CJ to refer to configuration j. Potentially there are many possible configurations at the planning horizon. Issues regarding the identification of configurations are addressed elsewhere. l ?

More than one combination of commitments can result in the same configuration (this is because otherwise identical actions occurring in different periods are designated as different commitments). Each such combination of commitments is defined as a configuration composition. That is, a configuration composition is the union of a particular sequence of commitment sets which results in a given configuration.

We use the notation

Cijk where

Cj,k C_ ZI U .* U Zt to refer to the kth configuration composition of configuration ].

Future scenarios and perspectives

A future scenario is defined as a description of the environment at the planning horizon after a hypothesised or predicted chain of events. Since there can be many such projections and the fulture occurrence of these events is unknowable, it is possible to identifN many fulture scenarios.

The defining characteristic of a future scenario is that its properties cannot be controlled. In general, a configuration will behave differently under different future scenarios. We use the subscriptf to denote a particular future scenario.

A perspective is defined as the set of views or value judgements of a distinctive group of people who will be affected by the performance of the configuration in the environment. In general, there will be more than one such group, and each group will have its own perspective. The desirability of the performance of a configuration will vary between these perspectives. Therefore, it is necessary to take into account the groups' perspectives when choosing an initial commitment set for implementation. We use the subscript p to denote a particular perspective.

Methods for the identification of perspectives and the generation of future scenarios have been suggested.10

Desirable configurations

Different configurations can be expected to perform differently in the same environment at the planning horizon. A range of relevant valuation criteria can be used for valuing the anticipated performance of these configurations at the planning horizon. These criteria, which can be quantitative and qualitative, will be scalar measures and may include monetary values, throughput measures, perceived image, and political acceptability, etc.

We define

VALi(C1) as the performance valuation of configuration C1 using valuation criterion i. A satisficing level, li, for criterion i determines those configurations that are considered as desirable on this criterion. If there are n valuation criteria, we use the vector

L=[11 12 .

. l]

to indicate the satisficing levels for the n valuation criteria. For simplicity, we will first consider situations with a

single future scenario and a single perspective. A desirable configuration is defined as a configuration with anticipated performance valuation at or above all the satisficing levels L, that is,

VALi(Cj) -> li Vi.

This definition assumes implicitly that higher values for each configuration valuation are preferred; equivalent inequalities may be substituted to handle situations where lower valuations are desired. Although the use of satisficing levels should have broad application, alternative mechan- ism for identifying desirable configurations may be employed.

We define WL as the set of desirable configurations that satisfy L. This is in keeping with the fact that RA is


H-Y Wong and J Rosenhead-A rigorous definition of robustness analysis 179

founded on the identification of acceptable, rather than optimal, solutions. 1"2

We now extend this definition to cover situations with multiple perspectives and multiple future scenarios. We define

VALq1'p(C1) as the performance valuation of the configuration C1 using valuation criterion i under the conditions of future scenario f from perspective p. The satisficing level on criterion i for future scenario f and perspective p is denoted by

and the corresponding set of satisficing levels (across criteria i) by

We further define

WL,P

as the set of desirable configurations that satisfies Lf P. That is

Cj E WLy., if and only if

'VALi, 'p(Cj) >-- ii7f.p Vi-

Procedures for the elicitation of valuation criteria, valuations and satisficing levels under multiple future scenarios and perspectives have been developed.'0

Attainability

The preceding section has defined the fundamental concepts in Robustness Analysis. This section will define the relationship of attainability between initial commitment sets and configurations. In general, a configuration is said to be attainable by an initial commitment set if, together with a set of future commitments, it can result in achieving this configuration at the planning horizon. We distinguish two types of attainability, exact and supe-rfuous, using the definitions of commitment sets and configurations already established.

Exact attainability

If the period I commitments in a configuration composition Ci k are identical to those in an initial commitment set S, I, that is,

Si,C = Cj,knZ, the configuration Cj is said to be exactly attainable by Si,,. This exact attainability relationship between Si and C1

remains true if there exist other configuration compositions of CJ that also satisfy the above condition. We define

A(Si 1. WJP as the set of desirable configurations that are exactly attainable by Sij.

Stuperfluous attainability Previous work on RA has implicitly used this concept of exact attainability.2> The assumption has been that the implementation of a commitment that is not an integral part of a particular configuration composition will result in the non-attainability of the corresponding configuration. However, there exists an alternative view of attainability that may sometimes be appropriate: a configuration is attainable by an initial commitment set that includes all the commitments necessary to be made as well as additional ones that are not required by the configuration concerned. In this case, we say that the configuration is superfluously attainable by the commitment set. Therefore, if the commitment set Si, contains more period 1 commitments than those required by configuration composition Cjk, that is,

Si,I D cj,k n z,Q configuration C1 is said to be superfluously attainable by Si, I

It should be noted that in this paper, the symbols C and D will be used to define proper subsets which exclude the condition of equality.

We define A+(Si1, WLf ) as the set of desirable configurations that are superfluously attainable by the commitment set Si I

The desirability of a configuration may be taken to be unaffected by whether or not it is attained exactly or superfluously, since a superfluous commitment is, by definition, an independent and unrelated element.

Superfluous attainability will occur where early investments are made which are redundant for the particular configuration under consideration. However, the formal definition can produce counter-intuitive effects in cases where commitments which are commutative in effect are interchanged between the initial and a later time period. Here an apparent superfluous attainability may be signalled for one of the arrangements, where in the event there will be no superfluous resources. Such cases should be disre- garded-though to eliminate them formally from the definitions would require a quite disproportionate extension of the notation.

Characteristic of exact and superfluous attainability The distinctive characteristic of exact attainability between a particular initial commitment set and configuration is that



each of the commitments in the commitment set must form an element of the configuration. Conversely, superfluous attainability is characterised by the existence in the commitment set of 'redundant' commitments, which do not feature in the specific configuration. Where a configuration can only be constructed by a single configuration composition, the conditions for exact and superfluous attainability are mutually exclusive: a configuration that is exactly attainable by an initial commitment set cannot be superfluously attainable by the same commitment set.

Where a configuration can be achieved by implementing different configuration compositions, it is in principle possible that a configuration can be attained both exactly and superfluously by the same commitment set. Suppose that configuration C1 can be achieved by implementing either configuration composition Cijk or Cj k. Then if both

Si, I jZkn Z and Si DC, knZl were to hold then C, could be attained exactly and superfluously by Si',. In this rather remote eventuality, which could only obtain if the effect of implementation of time dependent commitments is not commutative, it would be necessary to assess any distorting effect on robustness calculations.

The general characteristics of the attainability relationships may be illustrated graphically. Figure 2 illustrates a hypothetical problem with a number of initial commitment sets (left hand side of diagram), desirable configurations (right hand side of diagram) and the possible attainability relationships between them (a solid line indicates exact attainability while a dashed-line indicates superfluous attainability). This figure illustrates that a configuration may be attained exactly by one group of commitment sets and superfluously by another. Similarly, a single

C2

S~~~~~~~~~~ C5

CG

Cg

Co

CFio

Figure 2 Attainability relationships between commitment sets and desirable configurations of a hypothetical problem.

commitment set may attain different configurations exactly and superfluously. The number of links of both kinds between a given initial commitment set and desirable configurations will vary depending on the number of desirable configurations as determined by the satisficing levels set for each future scenario and/or perspective. As these levels are reduced, the number of attainable configurations tend to rise.

The concept of superfluous attainability is a new conceptual development. It draws attention to the possibility of situations where initial commitment sets can increase the number of attainable configurations, at the expense of committing potentially redundant commitments. The types of robustness scores defined in the next section are based on the two kinds of attainability presented in this subsection.

Robustness score

This section presents two alternative definitions of robustness score: exact and superfluous. These definitions are based on the definitions of attainability presented in the previous section. A further concept that of aggregate robustness score will also be defined.

Exact robtustness score

The exact robustness score of an initial commitment set is defined as the ratio of the number of desirable configurations that are exactly attainable by a commitment set to the total number of desirable configurations. We define


H-Y Wong and J Rosenhead-A rigorous definition of robustness analysis 181

r(Si 1, WL ), the exact robustness score of the commitment set Sl 1, as

r(Si 1, WL) = jA(Sl, WIf)I ]~P)- I WL11 I This definition of robustness score will yield the same results as the earlier definition of RA.

Superfluous robustness score

The superfluous robustness score of an initial commitment set is defined analogously as the ratio of the number of desirable configurations that are superfluously attainable by a commitment set to the total number of desirable configurations. We define r?(Si 1, WLf ), the superfluous robustness score of the commitment set Si, 1, as

rJA(S1,I, WL,)I r+(Si, 1? WL p) jW ~fp IWLf I

Aggregate robustness score

We define the aggregate robustness score of an initial commitment set as the ratio of the number of desirable configurations that are either exactly or superfluously attainable by this commitment set to the total number of desirable configurations. We denote

r* (Si, 1, WLf

as the aggregate robustness score of the commitment set Si,1, which is given by

r*(Si,1 WL) =A(S 1, WiLf5) )UA(S,1, W ) I

Lf'p ~I WL,1I

= r(Si 1, WLf ) + r (Si,1, WLf)

The definition of aggregate robustness score provides a combined measure of the attainability relationships between initial commitment sets and configurations.

Characteristics and interpretation of robustness scores The exact, superfluous or aggregate robustness score of an initial commitment set can range from zero to one. If the initial commitment set Si cannot exactly attain any of the desirable configurations then the exact robustness score is zero. Should no desirable configurations exist at satisficing levels Lfp, the exact robustness score is undefined. The same logic applies to both superfluous and aggregate robustness scores. As the number of desirable configurations that are exactly or superfluously attainable by a commitment set approaches the total number of desirable configurations at the planning horizon, its aggregate robustness score approaches one.

In these respects, as in others, both exact and superfluous robustness scores share the broad characteristics of robustness identified in previous publications. Either or both may be employed, according to circumstance, to assess the extent of useful flexibility which will remain following alternative initial decisions. In accordance with the decision-aiding status of RA, these flexibility assessments are intended as contributions to the informational background to choice, not as sole determinants of proposed choice. In virtually every case, there will be other factors relevant to a decision-making group or process, not least the shorter-run effects of alternative commitments. However, in general, it is to be expected that the robustness scores should be more influential in situations where uncertainty levels are high.

The discrimination between exact, superfluous and aggregate robustness which the definitions in this paper introduce provide additional information content. Exact robustness captures the essence of robustness as previously described, though on a sounder conceptual footing. These definitions make it clear that this essence was to provide an indicator of the degree of useful flexibility left open, in a particular sense. This is that options (configurations) are to be considered as 'left open' only if all the initial prerequisite actions are taken, and if there are no components of the initial commitments made which would be irrelevant to that option. Aggregate robustness by contrast adopts a more relaxed posture on 'openness' (or attainability as we have termed it). An option is regarded as maintained open by an initial commitment set so long as none of its prerequisites has been omitted, regardless of the extent to which the option may fail to put all of the initial resource investments to work. Superfluous robustness may be seen as the differ- ence between these two indicators. It therefore provides a measure of the degree of extra flexibility which the legit- imisation of possibly redundant investments can provide. It can therefore contribute to an assessment of the trade-off between possibly redundant initial resource investments, and the extra flexibility which they might buy. The incorporation of such redundancy into the robustness framework allows it to represent an important source of flexibility that enables the system to respond to future uncertainties.' 3-15

Discussion

The new definitions of Robustness Analysis, through the use of set notations, describe clearly and explicitly the relationships among the concepts of commitments, commitment sets, configurations, attainability and robustness. In particular, the notion of attainability, using commitment as the common element, permits the explicit and logical definition of robustness. Collectively, the ideas presented in this paper widen the vocabulary available to capture the prevailing complexity.

The introduction of a commitment-centric approach to problem structuring in RA helps to clarify the relationship



between initial commitment sets and permits the development of the new concept of attainability. An added advantage of this approach is that the process of interrogating the situation and eliciting information from decision-makers in terms of elementary commitments in itself is a contribution to achieving greater flexibility.16

Previous definitions of RA offered no explicit provision for handling problems with multiple criteria, even though the idea has been suggested previously.3 This imposed a considerable limitation on its applicability to problems where configurations need be assessed from a variety of angles. The new definitions presented in this paper permit the incorporation of multiple criteria, both quantitative and qualitative, to evaluate performance of configurations. Desirable configurations are then identified based on the satisficing levels set for each criterion.

Another significant development of the theory of RA is the differentiation made between the mutually exclusive exact and superfluous attainability relationships. Exact attainability only considers combinations of possible commitments which are compatible with a given configuration, with combinations containing elements beyond those required by that configuration excluded. The introduction of the concept of superfluous attainability relaxes this assumption. It identifies the increment in flexibility provided by the additional configurations which can be attained as a result. This criterion is therefore appropriate where the decision-making environment does not rule out the possible redundancy of elements of future options.

Although greater differentiation of robustness, in its exact, superfluous and aggregate forms, is now available to indicate the flexibility offered by each decision in practice, the use of these criteria will need to take account of the decision making climate. The comparison of exact and aggregate robustness scores can be expected to be useful only under situations where the additional investment of potentially redundant commitments is likely to be acceptable, for instance, when resource constraints are not tight, or when the turbulence of the environment is such as to make committing extra resources to 'buy' additional flexibility seems an insurance policy worth considering. Superfluous and aggregate robustness may be unsuitable if converse conditions apply, and the decision makers conse- quently respond negatively to any form of redundancy.

Conclusions

Complex problems often involve a considerable number of decision makers who do not necessarily share similar views and objectives. By neglecting such diversity, RA in its earlier formulation offered no guidance for handling uncertainties about guiding values, 7 an effective disqualification from an important class of decision problems. The extensions to RA presented in this paper formalise the calcula-

tion of robustness scores based on the alternative perspectives held by different decision making groups. The process of eliciting and analysing these contrasting perspectives can itself be expected to improve mutual understanding, so making negotiated accommodations more likely to merge, particularly in political situations or inter-organisational problems.'8

References

1 Rosenhead JV (1980). Planning under uncertainty: I. the inflexibility of methodologies. J Opl Res Soc 31: 203-216.

2 Rosenhead JV (1980). Planning under uncertainty: II. a methodology for robustness analysis. J Opl Res Soc 31: 331-341.

3 Rosenhead JV (ed) (1989). Robustness analysis: keeping your options open. In: Rational Analvsis for a Problematic World. Problem Structuring Methods for Coomplexity, Uncertaintv and Conflict. Wiley: Chichester.

4 Gupta SK and Rosenhead JV (1968). Robustness in sequential investment decisions. Mgmt Sci 15: B 18-B29.

5 Caplin DA and Kombluth JSH (1975). Multiobjective investment planning under uncertainty. OMEGA 3: 423-441.

6 Llewelyn Davies Kinhill Pty Ltd (1976). Health Services for a New City, Albury-Wodonga Development Corporation, Austra- lia.

7 Rosenhead JV (1978). An education in robustness. J Opl Res Soc 29: 105-111.

8 Best G, Parston G and Rosenhead JV (1986). Robustness in practice the regional planning of health services. J Opl Res Soc 37: 463-478.

9 Rizakou E (1995). Scenario-Robustness Methodology: An Approach to Flexible Planning under Uncertainty with an Application to AIDS-Related Resource Allocation, PhD Thesis, Department of Operational Research, London School of Economics.

10 Wong HY (1998). Making Flexible Planning Decisions: A Clarification and Elaboration of the Theory and Methodology of Robustness Analysis, PhD Thesis, Department of Opera- tional Research, London School of Economics.

11 Watson SR and Buede DM (1987). Decision Synthesis: The Principles and Practice of Decision Analvsis. University Press: Cambridge.

12 Bell DE, Raiffa H and Tversky A (eds) (1988). Descriptive, normative and prescriptive interactions in decision making. In: Decision Making: Descriptive, Normative and Prescriptive Interactions. University Press: Cambridge.

13 Stigler G (1939). Production and distribution in the short run. J of Political Econ 47: 305-327.

14 Eppink DJ (1978). Planning for strategic flexibility. Long Range Planning 11: 9-15.

15 Collingridge D and James P (1991). Inflexible energy policies in a rapidly-changing market. Long Range Planning 24: 101-107.

16 Mandelbaum M (1978). Flexibility and Decision Making: an Exploration and Unification. PhD Thesis. University of Toronto.

17 Friend J and Hickling A (1987). Planning linder Pressulre: the Strategic Choice Approach. Pergamon: Oxford.

18 Churchill J (1990). Complexity and strategic decision-making. In: Eden C and Radford J (eds). Tackling Strategic Problems. Sage Publications Ltd: London.

Received March 1996; accepted September 1999 after two revisions


Article Contentsp. [176]p. 177p. 178p. 179p. 180p. 181p. 182

Issue Table of ContentsThe Journal of the Operational Research Society, Vol. 51, No. 2 (Feb., 2000), pp. 135-249Front Matter50th Anniversary PaperA Commentary on Stringer J (1967): Operational Research for "Multi-Organisations" [p. 135]

Strategic PaperEnergy Modelling Platforms for Policy and Strategy Support [pp. 136 - 144]

Case-Oriented PapersTowards a General Condition Based Maintenance Model for a Stochastic Dynamic System [pp. 145 - 155]Optimisation of Sinking Funds for Major Repairs in a Housing Association [pp. 156 - 161]A Simulation Analysis of Arrest Costs [pp. 162 - 167]An Exploration into the Influence of Psychological Profiles upon Group Decision Making [pp. 168 - 175]

Theoretical PapersA Rigorous Definition of Robustness Analysis [pp. 176 - 182]Project Finance: A Multi-Criteria Approach to Arbitration [pp. 183 - 197]An Approximation Method for General Closed Queuing Networks with Fork/Join Mechanisms [pp. 198 - 208]Linear Programming with Interval Coefficients [pp. 209 - 220]An Option-Pricing Look at the Introduction of Private Labels [pp. 221 - 230]

Technical NotesEfficient Formulation of Electric Utility Resource Planning Models [pp. 231 - 236]A Note on the Sampling Distribution of the Information Content of the Priority Vector of a Consistent Pairwise Comparison Judgment Matrix of AHP [pp. 237 - 240]

ViewpointDiagramming Conventions in System Dynamics [pp. 241 - 245]

Book Selectionuntitled [p. 246]untitled [pp. 246 - 247]untitled [pp. 247 - 248]untitled [pp. 248 - 249]

Back Matter

Rigorous Definition of Robustness Analysis

Documents

Transcript of Rigorous Definition of Robustness Analysis