Abstract :

Constructing a qualitative model of somedevice usually proceeds as a cycle of model formulationand model debugging . The latter is driven by discrep-ancies between the behaviour predicted by the modeland the actual device behaviour. This paper describeshow the elimination of one type of discrepancy, incor-rectly predicted derivatives, can be supported . It pro-vides an analysis of the knowledge that is required forsuccesfully adapting the model. Then an implementedprocedure is described for supporting the eliminationof incorrect derivatives . Heuristics are employed togenerate plausible hypotheses, and in interaction withthe modeler one model adaptation is selected .

1 IntroductionA qualitative model is hardly ever constructed in onego . Usually the modeling process proceeds incremen-tally . It consists of cycles of model simulation andmodel debugging. Figure 1 depicts the cyclic natureof model construction . An initial model is formulated

Supporting Qualitative Model Construction :Eliminating incorrectly predicted derivatives*

Figure 1 : The incremental nature of qualitative modelconstruction

and the qualitative simulator generates a behaviourprediction . Usually the behaviour predicted by the

'The research presented in this paper was partially sup-ported by the Foundation for Computer Science in the Nether-lands (SION) with financial support from the Netherlands Or-ganisation for Scientific Research (NWO), project number 612-322-307. We thank Bob Wielinga, Jaap Kamps, and Kees deKoning for their insightful comments .

Cis Schut

Bert BredewegDepartment of Social Science Informatics (S .W.I .)

University of AmsterdamRoetersstraat 15

1018 WB Amsterdam (The Netherlands)Telephone: +31-20-525 6765, Telefax: +31-20-525 6896

E-mail : {schut,bert}@swi.psy.uva.n l

model is not completely in line with the real devicebehaviour. For example, the model predicts that acertain temperature increases, whereas in the real de-vice it decreases. The modeler is faced with the task offinding out which modeling error(s) caused this incor-rect prediction, and to change the model in order toresolve the discrepancy. We call the process of elimi-nating discrepancies between predicted behaviour andexpected behaviour, model adaptation . Our aim is tosupport a modeler in this process. The paper focuseson one specific type of discrepancy: incorrect deriva-tives of quantities . We assume that a modeler identi-fies one or more derivatives that are incorrect. Thenwe describe how the support system finds, in interac-tion with the modeler, the model modifications thateliminate the incorrect derivatives. The contributionsof the paper are threefold:

" It describes the modeling knowledge that is re-quired for eliminating incorrect derivatives

" It gives an analysis of the reasoning process ineliminating incorrect derivatives

" It demonstrates how this process can be supported

First, section 2 briefly explains the knowledge rep-resentation that is employed in the qualitative rea-soner that we use. Then, section 3 gives an exampleof the model adaptation problem and discusses themodeladaptation task in a general way. Section 4 dis-cusses the specific case of adapting amodel in order toeliminate a discrepant derivative . Section 5 explainshow the occurrence of multiple discrepant derivativesis treated. Finally, section 6 discusses some open-endsand relates model adaptation to model-based diagno-sis .

2

Knowledge RepresentationThe process of constructing a qualitative model forsome device is, among others, determined by the par-ticular qualitative reasoning formalism . The model

primitives of the formalism dictate what can be ex-pressed and how. The formalism we use, GARP (see[1]), employs model fragments to represent pieces ofgeneric domain knowledge, and a case model for rep-resenting the particular device . Model fragments areexpressed as a set of conditions and consequences .Both the case model and the model fragments aremade up of the following set of model primitives : en-tities represent physical objects (e.g . a container) ;attributes represent properties of objects (e.g . con-nected-to) ; quantities represent variable properties ofobjects (e.g . temperature) that have a qualitativevalue and a derivative; dependencies represent howdifferent quantities are qualitatively related (e.g . aninequality) .In order to predict the behaviour of some device, its

structure and initial conditions have to be modeled ina case model . The qualitative simulator searches thelibrary of model fragments and instantiates the oneswhose conditions match the case model. This resultsin a set of instantiated dependencies that are used tocompute the values and derivatives of all quantities,yielding a description of the device behaviour during aperiod of time : a qualitative state. Changes from onestate to another are derived by applying transforma-tion rules. A sequence of states represents the devicebehaviour over time .

3

Model AdaptationFirst, we will discuss the model adaptation task in ageneral sense in order to illustrate which notions areimportant . The next section describes in detail howmodel adaptation proceeds in the case of incorrectlypredicted derivatives . As an example of the modeladaptation task, consider figure 2 . The drawing rep-resents the condensor part of a refrigerator . A com-pressor pumps substance into the condensor, and thesubstance flows out through the throttle-valve . Due tothe increased amount of substance in the condensor,the pressure rises, and as a consequence, the tempera-ture of the substance rises too. Now suppose that themodeler has constructed an incorrect model for whichthe behaviour is predicted. The table in figure 2 givesthe expected and the predicted derivatives of Pressureand Temperature for a state in which the compressorstarts working. Pressure is predicted to be decreasing,and Temperature is predicted to be steady, whereasthey both should be increasing . The modeler's task isto identify what is wrong with the model, and how ithas to be modified . The first step is to find out whichelements in the model affect the derivatives of Pressureand Temperature . These determinants are depicted inthe left side of figure 3. Next, the modeler has to de-termine how the set of determinants is incorrect andwhich modifications to the model might resolve theincorrect prediction . For example, the derivative ofthe Amount could be incorrect. If it were decreasing,

ExpansionRate

Throttle-Valve Compressor

Figure 2: The condensor with predicted and expectedderivatives

the derivative of Pressure would be correctly predictedto increase as well . Likewise, adding a positive influ-ence of CompressionRate on the Temperature wouldresolve the incorrectly predicted derivative of the lat-ter. This shows that a selection has to be made fromthe various ways of modifying the model. The rightside of figure 3 shows the model that should be at-tained . From all possible candidatemodel adaptationsthe modeler has to select the one that realizes threethings : it must change the negative proportionalitybetween the Amount and the Pressure into a positiveone, it must add a positive proportionality from Pres-sure to Temperature, and it must remove the influenceof the ExpansionRate on Pressure .This example illustrates four notions that are impor-

tant for model adaptation in general :

Discrepancies. In general, discrepancies are charac-terized by the model primitive they concern, andby their type . Three types of discrepancies can bedistinguished: i) inappropriate: a model primitiveis instantiated in the model where it should nothave been (e.g . the influence of ExpansionRate onPressure) ; ii) incorrect : the primitive should in-deed be instantiated but have another value (e.g .Pressure being negative proportional to Amountshould be positive proportional) ; iii) missing, itis not instantiated where it should have been (e.g .the proportionality between Pressure and Temper-ature) .

Current model state. The

predicted

behaviourprovides the instantiated current model state. Itallows the modeler to identify which elements in

Predicted ( ExpectedDerivatives:

Pressure I decreasing ( increasing

Temperature I ( increasing

Current Model

Legend:

the model were responsible for predicting a partic-ular primitive instance . In the condensor examplethe determinants of the derivative of Pressure are:the influence of ExpansionRate, and the propor-tionality with Amount .

Modeling knowledge. Modeling knowledge can bethought of as a meta-view on the reasoning mech-anism of the qualitative reasoner . For example, itcomprises knowledge about the effect of adding acertain primitive to the model, knowledge of thereasons that might explain why a certain expectedprimitive is not in the model, and so on. The rele-vant modeling knowledge differs for each combina-tion of primitive type and discrepancy type. Thenext section provides an account of the modelingknowledge that is used for eliminating incorrectderivatives .

Model modifications. A model modification is aset of one or more alterations in the model (in-cluding additions), that together eliminate the discrepancy. A model modification is the result of themodel adaptation process .

The process of model adaptation consists of fourmain reasoning steps: discrepancy detection, determi-nant collection, hypothesis generation, and hypothesisselection . In the discrepancy detection step the differ-ences between predicted and expected behaviour areidentified . Determinant collection is the process ofidentifying which elements in the model are responsi-ble for determining the discrepant primitive. It takesthe discrepancy and the current model state as in-put and generates the relevant determinants as out-put. Hypothesis generation is the process of findingthe candidate model modifications (hypothesis) thatmight be applied in order to resolve the discrepancy.The set of determinants and the modeling knowledgeare used to come up with such candidate model mod-ifications . Finally, one model modification has to beselected from the alternatives that were generated .

I- : Negative InfluenceI+ :Positive InfluenceQ- :Negative ProportionalityQ+ : Positive Proportionality

=0 Value equalto zeroO+ : Derivative is plus

Figure 3 : Current Model State and Desired Model State

Desired Model

Note that modeling errors may propagate. For ex-ample, an incorrect derivative might be caused by anincorrect value of another quantity, which might becaused by a missing causal dependency, which finallycan be traced to an omission in a model fragment .This illustrates that the cause of some discrepancymight be a discrepancy itself, possibly in another typeof primitive. As a consequence, another discrepancy isdiscovered that requires model adaptation . This cango further until finally a discrepancy can be relatedto a change in a model fragment or the case model.Since the modeling knowledge used in model adap-tation differs for each type of primitive, the modeladaptation task can be decomposed naturally alongthe lines of the primitives involved in the discrepan-cies . Each combination of primitive type and discrep-ancy type can be dealt with as an independent partof the model adaptation task . The rest of the paperdiscusses one such combination : incorrect derivatives .

4

Eliminating Incorrect Derivatives

After discussing the model adaptation task in a moregeneral way, we can now turn to the specific task ofadapting the model in order to eliminate incorrectderivatives .

4.1

Detecting incorrect derivativesIt is assumed that the modeler is capable of assessingthe appropriateness of the model's behaviour predic-tion . Hence, the modeler identifies the discrepanciesbetween predicted and expected behaviour.' Discrep-ancies are stated in terms of the derivative value thatis incorrect, and the derivative value that is expected .A predicted derivative value is one of: +, 0, or - . Anexpected derivative value is also one of +, 0 or -, butthree ranges may be specified as well : [-, 0], [0, +], or[-, 0, +] . If a range is indicated it means that themodeler expects different states in which the only dif-ference is the derivative value of the particular quan-

IThis step could be largely automated if an accurate descrip-tion of expected device behaviour were available.

tity. In the condensor example the two discrepan-cies can be expressed as : discrepancy(incorrect, deriva-tive(Pressure), +) and discrepancy(incorrect, deriva-tive(Temperature), +), expressing that the type of dis-crepancy is incorrect, the model primitive involved isthe derivative of Pressure and Temperature respec-tively, and the expected derivative is in both cases +.In this exampletwodiscrepancies are identified . Thereis a possibility that both discrepancies are caused bythe same modeling error. Therefore, a distinction hasto be made between situations in which only one dis-crepant derivative is identified, and situations whereseveral are identified . This section discusses the caseof a single discrepant derivative and the next sectionadresses the case of multiple discrepant derivatives .

4.2

Collecting determinants of derivatives

When the discrepant derivatives have been identified,the next step is to collect those elements in the modelthat have contributed in determining its value. Theseelements can be collected from the current modelstate, the set of instantiated model primitives in thestate that exhibits the incorrect derivative . The modelelements that have an effect on a quantity's derivativeare the dependencies that hold for that quantity. How-ever, not all types dependencies have an effect on thederivative of a quantity . Only three types of depen-dencies are used to compute derivatives :

Causal dependencies . The common way in whichderivatives are determined is by one or more causaldependencies with other quantities . The causaldependencies occurring in the formalism we use,are influences, I+, I-, and proportionalities, Q+,Q- ( see also [4]) .

Constraints . If various causal dependencies inter-act then the relative effects cannot be determinedfrom the dependencies themselves . Sometimesthere is knowledge available about such relativeeffects . Constraints are the vehicle for modelingknowledge about relative effects . In addition, theycan be used for pruning behaviours that are not ofinterest to the modeler . We use five types of con-straints : >, >, =, <, <, each of them defined withrespect to two quantities .

Assignments . If only certain behaviours of a deviceare of interest, c .q . only behaviours where somequantity has a fixed derivative, an assignment canbe used . Also if it is exogenously determined thata quantity has one particular derivative through-out the device behaviour then it is assigned thisvalue. Quantities can be assigned one particularderivative, +, 0, or -, or they can be assigned arange : [0, +], [0, -], resulting in the prediction ofdifferent states with different derivatives for theparticular quantity . Assignments are defined as

inequalities of derivatives with respect to zero :

The effect of a particular type of dependency is notonly determined by the type of that dependency butalso by the value or derivative of the quantity that ex-erts an effect through that dependency. Therefore, alldeterminants (causal dependencies, constraints, andassignments) that are present in the current modelstate, are annotated with the effect they have on theparticular quantity's derivative . This is possible sincethe values and derivatives ofthe quantities involved ina dependency, are also available in the current modelstate.Annotating a determinant with its effect differs for

each type of determinant. For an assignment it is de-termined which derivatives it allows and consequentlywhich are excluded . For example, if quantity A isassigned derivative +, then - and 0 are excluded .Constraints are annotated in a similar way: the typeof constraint and the derivative of the constrainingquantity determine which derivatives are allowed andwhich are excluded . For example, suppose there holdsan equality between the derivatives of A and B andthe derivative of B is + . This means that for thederivative of A only + is allowed and hence - and 0are excluded . Causal dependencies are annotated ina slightly different way because their effects are qual-itatively added . They are annoted according to thedirection of influence (Doi) they have . The directionof influence is determined on the basis of the type ofcausal dependency and the value or derivative of thedetermining quantity. For example, if A is positivelyproportional to B and B has derivative +, the Doi ofthe dependency is + . Table 1 shows how the directionof influence for each causal dependency is determined .In the condensor example the determinants of Pres-sure are:

Q-(Pressure, Amount) with Doi = - becausebAmount = +

" 1-(Pressure, ExpansionRate) with Doi = 0 be-cause ExpansionRate = 0

4.3 Generating candidate model modifica-tions

Having collected all determinants and their effects, thenext step is to generate candidate model modificationsthat resolve the discrepancy. The idea is that each ofthe identified determinants may involve a modeling er-ror, or propagates the effect of another modeling error(the determinant may also be correct, of course). Be-sides the possibility of errors in the model, it may alsobe incomplete : determinants may be missing. Whichdeterminants may be missing and howeach individualdeterminant may be modified, constitutes one type ofthe modeling knowledge mentioned in section 3.

There is a limited number of ways in which a de-terminant can involve a modeling error, or in whichit propagates one . As a consequence, the number ofways in which each determinant can be modified, islimited as well . This makes it possible to enumeratein advance all different modifications for each type ofdeterminant . In addition, each modification can beannotated with the effect that it has on the quantityinvolved, similar to how determinants are annotated .Modifications are expressed implicitly as the inverse

of a discrepancy of another model primitive . For ex-ample, if a modification entails that a positive in-fluence is replaced by a negative influence, this isexpressed as : discrepancy(incorrect(I -}- (A, B)), I -(A, B)) . The reason for expressing a modification inthis way, is that, as already explained in section 3,modeling errors propagate . Therefore, one discrep-ancy may be explained by another discrepancy untilone can be explained by an error in the case modelor in the model fragments . For example, replacing apositive by a negative influence may resolve an incor-rectly predicted derivative but introduces the prob-lem of why the positive influence was present in themodel : some model fragment may have been definedincorrectly, or it should not have been included in themodel whereas another (specifying a negative influ-ence) should have been included, and so on . For ourpurposes, we assume that the new problem is adressedby another part of the model adaptation task .As an example of a set of determinant modifications,

take the determinant Q-(Pressure, Amount) from thecondensor system . This determinant may be incor-rect or inappropriate . The incorrectness may concernthe derivative of the Amount or the dependency itself.Suppose the annotation of the determinant, its Doi,is + in the current model state . Then the possiblemodifications are :" incorrect 6Amount, should be - (new Doi = -)" incorrect 6Amount, should be 0 (new Doi = 0)

" incorrect Q-(Amount, Pressure), should be Q+(new Doi = +)

" incorrect Q-(Amount, Pressure), should be I+(new Doi depends on value of Amount)

" incorrect Q-(Amount, Pressure), should be I-(new Doi depends on value of Amount)

Table 1 : Determining the Direction of Influence (Doi

" inappropriate Q-(Amount, Pressure), (giving noDoi)

" correct Q-(Amount, Pressure), (Doi remains -)

As another example, suppose that one of the deter-minants of a quantity A is the constraint that thederivatives of A and B are equal, and also supposethat the derivative of B is 0 . In this case the errorsare :

" incorrect 6B, should be -, realizes -

" incorrect 6B, should be +, realizes +

" incorrect bA = 6B, should be >, realizes [0, +]

" incorrect SA = bB, should be >, realizes +

" incorrect bA = 6B, should be <, realizes -

" incorrect bA = bB, should be <, realizes [-, 0]

" inappropriate bA = bB, realizes no derivative

" correct SA = 6B, realizes 0

In this fashion it is possible to enumerate all differ-ent ways in which each determinant may be modified,annotated with the effect that the modification has .Recall that annotations are for causal dependencies interms of the direction of influence, for constraints andassignments they are in terms of the derivative theyrealize . For constraints and assignments the sets ofpossible modifications can now be pruned : some mod-ifications can be discarded already, since they are notcompatible with the expected derivative . 2 Becausethe modifications are annotated it can be checkedwhether the annotation excludes the expected deriva-tive . If it does, that particular alternative is no longerconsidered . An example may clarify this . Supposean assignment is present in the current model state,realizing derivative + . Possible modifications are : in-correct assignment (annotation [0, +]) ; incorrect as-signment (0) ; incorrect assignment ([-, 0]) ; incorrectassignmentH;correct assignment (+) . Suppose thatthe expected derivative is - . Now the first two mod-ifications and the last one are not compatible with

2The reason why this can only be done for constraints andassignments, is that they determine a derivative in an absoluteway, unlike causal dependencies whose effects are qualitativelyadded.

Dependency Type: I- I+ Q- Q+value V of

determining quantity Doi Doiderivative D of

determining quantity Doi DoiV<0 + - D=- + -V=p ~' 0 0 D=0 0 0V>0 - + D=+ ~-++

the expected derivative : they need not be consideredin modifying the model because they can impossiblylead to a model realizing the expected derivative .Having for each determinant a set of possible modi-

fications, hypotheses are generated by picking for eachdeterminant one of its possible modifications. Such aset of alternatives forms a potential hypothesis. Sinceall modifications for each determinant are enumeratedin advance, eventually a combination is formed that isthe desired model adaptation . Here it becomes clearwhy correctness of a determinant is included as oneof the possible "modifications" of that determinant .Explicitly including correctness of some determinantmakes it possible to generate hypotheses where, forexample, only one determinant is modified .Besides picking one alternative for each determinant,

it is also possible to extend hypotheses by includingone or more missing determinants in the hypothesis .Thirteen types of annotated determinants might bemissing: a causal dependency with a Doi of either-, 0, or +; a constraint or an assignment realizing+, [0, +], -, [-, 0], or - . Based on the same argumentas above, the alternatives for missing constraints orassignments that exclude the expected derivative, arenot considered .A set of modifications of determinants, possibly ex-

tended with errors of missing determinants, is only ahypothesis if the conjunction of modifications leads toa model that correctly predicts the expected deriva-tive . How can this be checked? First, recall that eachmodification has been annotated with its effect on thequantity involved . Thus the set of modifications (andadditions) gives a complete picture of the situationwhere a model including these modifications wouldbe simulated. Second, an additional type of modelingknowledge can be employed . This knowledge capturesthe fact that a certain derivative is only predicted ifparticular combinations of determinants are present ina model. These combinations can be represented asso-called ideal model states . An ideal model state spec-ifies which annotated determinants have to be present,and which must not be present in the model so thatthe model predicts a certain derivative . For example,one way to realize an expected derivative of -~, is tohave two causal dependencies, one with Doi = -, andone with Doi = +, and an assignment realizing + : theeffects of the causal dependencies are ambiguous butthe assignment excludes derivatives - and 0 . Thistype of knowledge is based on the semantics of the de-pendencies and on the manner in which the simulatoruses them to compute derivatives .

It is possible to enumerate all ideal model states thatrealize each particular derivative or range of deriva-tives. Table 2 gives all ideal model states for someexpected derivatives.' The upper row designates the

'For purposes of conciseness, the ideal model states for ex-pected derivative - and [-,0] are left out of the figure . They

expected derivative . The first two columns specify thetypes of determinants that may occur in the modeland their effects . The following columns of the tablerepresent each an ideal model state for some expectedderivative . The entries of a column specify whichannotated determinants must be present, marked 3,which must not be present, marked 0, and which mayor may not be present in that particular ideal modelstate, not marked . For example, take in table 2 thesecond column for expected derivative 0 (see j) . Nowstart reading from above: derivative 0 is realized by amodel that has no causal dependencies with Doi = +,that has a caisal dependency with Doi = 0, that hasno causal dependencies with Doi = -, that has noconstraints and no alignments realizing + or - .Now the set of modifications with their effects can

be checked against the ideal model states . A set ofmodifications is a hypothesis only if it matches one ofthe ideal model states for the expected derivative . Ifnot, it can be rejected as a hypothesis . In the conden-sor example, a hypothesis for the incorrect derivativeof Pressure is that the derivative of the Amount is in-correct (should be 0), and that a causal dependencywith Doi = + is missing. Another hypothesis is thatthe proportionality with Amount is incorrect (shouldbe positive) . Note that both hypotheses contain adiscrepancy concerning the same determinant, thatis, Q-(Pressure, Amount) . However, the former con-cerns the determining derivative, bAmount, the latterconcerns the dependency .

Selecting a model modification

Obviously, this way of generating hypotheses leads tolarge numbers of hypotheses, many of which are im-plausible. The number of hypotheses can be reducedby asking the modeler to verify whether some modifi-cation of a determinant is accepted or not . If the mod-eler accepts it, then for that determinant the propermodification has been found, ruling out many combi-nations that include the other, invalid modifications.If the modeler does not accept that modification, it ismarked as an invalid modification, which still rules outmany candidate hypotheses . But for effective supportthe interaction with the modeler should be minimized.In order to do so, the plausible hypotheses should begenerated first . This calls for the formulation of a setof heuristics that operationalize plausibility of an er-ror . The effect of applying a heuristic is that someof the ways in which each determinant may be mod-ified, are initially not considered . Only if no solu-tion is found, then the heuristic is discarded and thepreviously skipped modifications are considered . Bybuilding several of these filters on top of each other,the most plausible hypotheses are generated beforehypotheses that are less plausible.

correspond to the mirrored situations of expected derivative +and [0, +] respectively.

The first heuristic is based on the observation thatmodeling errors may propagate: a quantity earlier inthe causal structure may have an incorrect value orderivative . Therefore, many hypotheses are excludedif a determiner can be rendered incorrect . This givesrise to the following heuristic :

" Prefer modifications of determining valuesand derivatives over others .

A major presupposition that can serve as a basis forsome heuristics, is that the modeler is not constructinga model in a randomly fashion . This means that itis safe to assume that the larger part of the modelis correct. As a consequence, hypotheses involvingfew modifications are more plausible than hypothesesinvolving many modifications. These considerationsare captured in the heuristic:

" Prefer simpler hypotheses over more com-plex hypotheses .

The same presupposition allows us to order the hy-pothesis types according to the probability of theiroccurrence . Recall that hypotheses may specify thatsomething is missing, inappropriate, or incorrect. Itis unlikely that some determinant is first specified andlater on appears to be inappropriate . Similarly, it ismore probable that some specified determinant ap-pears to be incorrect than that the determinant iscompletely forgotten . Then again, forgetting some-thing is more likely than specifying it inappropriately.With respect to modifications of type incorrect a fur-ther refinement can be made . The most common wayof specifying some dependency incorrect is the so-called sign fault, e.g . specifying a positive proportion-ality instead of a negative one . Therefore, within thecategory of incorrect modifications, sign faults should

Table 2 : Ideal Model States

be preferred over more radical errors . This gives thefollowing heuristics :

" Prefer modifications of type incorrect overmodifications of type missing and inappro-priate .

" Prefer modifications concerning incorrectsign over modifications of other incorrect-ness .

" Prefer modifications of type missing overmodifications of type inappropriate .

As a second guideline in formulating heuristics, thedifferent types of determinants can be ranked accord-ing to the probability that they are involved in mod-eling errors . This ranking is based on the differ-ent aspects that are represented by the dependencies .Causal dependencies, for instance, represent generalfeatures of the domain knowledge. Constraints, how-ever, usually represent features of a device, or of anumber of physical objects in a particular configu-ration . Assignments are used to exclude behavioursthat are not of interest . It can now be argued thata modeler is less prone to make errors in the domainknowledge than in device-specific knowledge. Also itis more likely that mistakes were made in trying to ex-clude some specific behaviourthan that device-specificaspects were incorrectly modeled. Hence, the follow-ing heuristics are used :

" Prefer modifications involving assignmentsover modifications involving causal depen-dencies or constraints.

" Prefer modifications involving constraintsover modifications involving causal depen-dencies.

11 Ideal Model States for Expected Derivative

_Determinant Annotationcausal + ( r/ 3 3 3 0 3 3 3 3 3 3 ~~ 3 3 3

dependencies 0 33 3 0 0 3 3 3 3 3 3 I~' 3 3 3

+ 0 3 I U q q Uq riUUII!--UI 0

10,+1 in) )UMMM©©M~I©constraints 0 0 0 0 -',~ U 4 U

1-1 0 U ~~MUM

-InU

U U I U U U U l U UU

( 4 U

+

10, +1 ~MMOMIMMMMMM©©IO©0

assignments 0 In U U U i©.-u.--.i4 4 i

-, 0 '1 U U /J Uimmmmm©0mi U U Vii

In the condensor example, the model adaptationprocess proceeds as follows. First, hypotheses are gen-erated where only one change is involved (prefer sim-pler hypotheses). In addition, the set of heuristicsdetermines that only errors of type incorrect and con-cerning determining values and derivatives are con-sidered . This gives: {incorrect bAmount, Doi = +}as the only hypothesis . The modeler is asked to ver-ify whether bAmount is incorrect. Since that is notthe case, the set of modifications for the determi-nant Q-(Amount, Pressure) reduced : this modifica-tion of the derivative of Amount is ruled out . Newhypotheses have to be generated after relaxing the setof heuristics : now incorrect assignments are consid-ered too. No hypothesis can be found so again theheuristics are relaxed so that now also incorrect con-straints are considered . This still gives no hypoth-esis . Further relaxation makes that incorrect causaldependencies are considered, and even more specificonly sign-faults . This gives the hypothesis : {incorrectQ-(Amount, Pressure), should be Q+(Amount, Pres-sure), Doi = +} . Asking the modeler reveals that thisis the right model adaptation . In a similar fashion itis discovered that for the Temperature a causal depen-dency with effect sf plus is missing.This procedure does not discover (yet) that the other

causal dependency, from ExpansionRate to Pressure isinappropriate. The reason is that in this state of thepredicted behaviour, the influence is not harmful. Inother behaviour states, however, the harmful effectsof this dependency may become apparent, and as aconsequence its inappropriateness is only discoveredlater.

5 Eliminating multiple discrepantderivatives

If we assume that one modeling error causes multiplediscrepancies, then the error must have been propa-gated through a branching path . Therefore, identi-fying such an error if multiple discrepancies are ob-served, has to proceed the other way around. Becausean error can only be propagated along the paths ofcausally related quantities', the network of causal de-pendencies can be used to identify candidates thatmight explain different discrepancies . More specific,we use the notion of causal units (see also [2]) to findsuch candidates . Causal units are based on the ideathat the set of quantities can be divided into clus-ters that influence each other, but are independent ofother quantities . A causal unit starts with one or morequantities that influence aquantity (I+ or I-), and isfurther comprised of proportionally related quantities .

4 Although the knowledge representation formalism that weuse, allows enforcingderivatives in amodel without using causaldependencies (namely by constraints) doing so is consideredimproper modeling since it violates the causal view on how dy-namic behaviour comes about.

It ends with a quantity that has no causal effect onany other quantity by means of a proportionality. Acausal unit is essentially a graph that may have recur-sive loops, more than one starting point and more thanone terminal node (note that a causal unit can consistof one or more causal paths) . Causal units can be gen-erated from the predicted behaviour. In the condensorexample only one causal unit exists : [ExpansionRate,CompressionRate] -+ Amount -+ Pressure .One modeling error mayonly propagate to errors in

the derivative of several quantities, if they occur inthe same causal unit . Therefore, the first step is tofind a causal unit in which all discrepant quantitiesoccur. If such a causal unit is found then it is checkedwhether there is one discrepant quantity that precedesall other discrepant quantities . If there is one, thenit will probably have caused the discrepancies of theother quantities . This is depicted in figure 4 in case 1:quantity Cmay have caused quantities G and Kto bediscrepant . Therefore it is selected and amodel adap-tation is sought in the fashion as described in section4. Even if no single discrepant quantity precedes all

Case 1

Case2

© : Quantity with discrepant derivative

Figure 4: Causal unit with multiple incorrect deriva-tives

others, there might still be a single modeling error. Ifso, it must concern the quantities in the causal unitthat precede all discrepant quantities (recall that themodeler might not have indicated all discrepancies) .This stuation is depicted in the right part of figure 4as case 2. The error might concern quantities A, B, Cand D. The modeler is asked to verify the correctnessof the derivative of the first quantity(ies) precedingall discrepant quantities (D). If the derivative is in-correct, it becomes input for model adaptation . If itis correct, however, then multiple errors must existand as a consequence, incorrect derivative eliminationmust be repeatedly performed. Assuming now thattwo errors exist, the same process is repeated withthe set of discrepant quantities divided in two inde-pendent groups . Because quantity L is the only one

in the first group, it is input for the first model adap-tation procedure. In the other group are quantities Hand Kthat both might be explained by an error in F.

If the derivative of F is incorrect, it is input for thesecond model adaptation procedure. If it is correct,the process is repeated and there appear to be threemodeling errors .So far the case in which one causal unit is found

in which all discrepant quantities appear. If no suchcausal unit can be found, then different causal unitsare sought that cover as many discrepant quantities aspossible . The same process as above is repeated foreach causal unit .In the condensor example there is only one causal

unit and it only contains Pressure. Therefore, modeladaptations for Pressure and Temperature are selectedconsecutively.

6

Discussion and ConclusionsIn this paper we have shown how to support the pro-cess of eliminating incorrectly predicted derivativesby adapting the model on which the prediction wasbased. It is assumed that the modeler indicates one ormore incorrect derivatives, and the derivative(s) thatare expected . The elements in the model that affecta derivative are identified and annotated with theireffect . Hypotheses, sets of modifications and addi-tions of determinants, are generated that eliminate theincorrect derivative and realize the expected deriva-tive . They are based on explicit knowledge of howeach determinant may be wrong, and on knowledge ofwhich combinations of determinants realize a partic-ular derivative . Hypotheses are generated in order ofplausibility by using a set of heuristics . These candi-dates for model adaptation are presented to the mod-eler who decides whether the hypothesis is acceptedor not. In this interaction eventually one model adap-tation is selected .The support system has been implemented and

tested on a number of deliberately corrupted models .In all cases it came up with model adaptations thatwould realize the expected derivative . These modelmodifications were not always the inverse of the orig-inal model corruption . Sometimes a particular de-terminant was not modified or added . The reason isthat, as we also saw in the condensor example, someadaptations are not necessary for realizing a particu-lar derivative in a particular state. Their effects maybecome harmful in other states where another depen-dency structure may hold . From our tests it appearedthat the procedure tends to become slow when six ormore determinants are involved . In the models we areworking with, however, the number of determinantsseldomly exceeds four .At first glance, model adaptation seems to consist of

a diagnosis task that determines how the discrepan-cies have arisen, and a repair task, that resolves the

underlying modeling error. Since diagnosis has beenstudied extensively in AI ([5]), the obvious approachwould be to apply existing diagnosis techniques tomodel adaptation . However, closer examination im-mediately reveals that this is not feasible . The reasonis that in device diagnosis the assumption is made thatdevice structure is available and modeled correctly.The task amounts to identifying one or more compo-nents that function incorrectly . In model adaptationthe correct model structure is not known, in fact thegoal of model adaptation is to contribute to the for-*nulation of a correct model. That means that often,a missing "component", or an inappropriate "compo-nent" explains the symptom. An undesirable effect isthat many more hypotheses can be generated. Thisdifference in starting points also blurs the distinctionbetween a diagnosis and a repair . In device diagnosisone or more components are identified as faulty anda repair is simply its replacement. This implies thatthe fault has to be identified in one of the componentsthat are present. Having identified the component(s)the repair is immediately known : replace that compo-nent . In contrast with this, in model adaptation thefault may lie in the fact that some component (e.g . acausal dependency), many times several components,are missing or are erroneous. Therefore, it is oftennot possible to mark one or several determinants asthe cause of the discrepancy. However, viewed fromthe perspective of model construction, this is not aproblem. Apparently the subtasks of diagnosis andrepair are interwoven and executed in parallel . A di-agnosis is determined by considering modifications forexisting determinants and considering inappropriateand missing "components" . Our paper demonstratesthat model adaptation can effectively be supported.Although potentially the number of possible modeladaptations can become very large, especially whenthere are many determinants of a derivative, we haveshown how to deal with this . A set of heuristics is usedfor generating hypotheses in a step-wise fashion: themost plausible hypotheses are generated before lessplausible hypotheses .

Other related work lies in the area of qualitativemodeling . The prevailing research in this area isbased on compositional modeling [3] . An importantassumption in this approach is that all relevant do-main knowledge has been represented in model frag-ments. Different viewpoints on the same phenomenonare represented in different model fragments. Here themodeling problem is to select the right set of modelfragments. This is solved by incorporating the as-sumptions that underlie the different viewpoints in themodel fragments and use these as handles in the se-lection process. Our starting points are complemen-tary to compositional modeling : our focus is on thesituation where the domain knowledge has not beenmodeled completely, i .e . where the modeler still has

to define a set of model fragments . Later, when a par-ticular domain has been modeled substantially, thetechniques for selecting model fragments become ap-plicable . Our aim is to provide a modeler with a setof tools for defining the domain knowledge in termsof model fragments . If a modeler has such a set oftools at his disposal then the whole domain model-ing trajectory is supported . Some of our work in thisarea can be found in [2] . It describes a technique foreliminating incorrect derivatives as well, but applies ifadditional assumptions on the state of the model, arevalid . In [6] we describe how the specification of thedomain and device knowledge in terms of our qualita-tive reasoning formalism, is supported .

References[1] B. Bredeweg . Expertise in Qualitative Prediction of

Behaviour. PhD thesis, University of Amsterdam,1992 .

[2] B . Bredeweg and C . Schut . Reducing ambiguity bylearning assembly behaviour . In Proc . of the 13th IJ-CAI'g3, pages 980-985 . Chambery, France, August,1993 .B . Falkenhainer and K . D . Forbus . Compositionalmodeling : finding the right model for the job . Arti-ficial Intelligence, 51:95-143, 1991 .

[4] K . D . Forbus . Qualitative process theory. ArtificialIntelligence, 24:85-168, 1984.

[5] W. Hamscher, L . Console, and J . de Kleer . Readings inModel-based Diagnosis . Morgan Kaufmann, San Ma-teo, CA, 1992 .C . Schut and B. Bredeweg . Supporting qualitativemodel specification . In Proc . of the Second Inter-national Conference on Intelligent Systems Engineer-ing, pages 37-42 . Technical University of Hamburg-Harburg, Germany, 1994 .