Week 4

CSM10Intelligent

Information Systems

Reasoning in the real world

Case study: MYCIN

Coursework project and report

CSM10 Spring Semester 2007

Intelligent Information Systems

Professor Ian Wells

The journey so far ...

• introduction to the module

• what is intelligence?

• cognitive processes

• how we perceive, remember, solve puzzles

• how a computer can solve problems

• representing knowledge using semantic networks and production rules

• reasoning in the real world

• case study (MYCIN), uncertainty, chess, problems

3

Intelligence

Perception & cognitive processes

Knowledge representation

Real-world reasoning

1

2

3

4

Inside an expert system:

MYCIN

A case study of one of the first major expert

systems and an analysis of its performance when

compared with humans at solving complex problems

Once upon a time ...

• it is a dark night in 1972

• you are on holiday in the USA

• you have to be admitted to hospital

• the doctor prescribes antibiotics

• what should you do .... ?

• be very worried ... !!

6

Treatment of infection

• _____ % acceptable

• _____ % questionable

• _____ % clearly irrational

• (Roberts & Visconti: Am J Hosp Pharm 1972)

7

History of MYCIN

• diagnosis of bacteraemias

• suggestions for suitable therapy

• PhD thesis - Edward Shortliffe (Stanford 1974)

• project lasting 10 years + 5 more PhDs

• numerous offspring e.g. EMYCIN, PUFF, CLOT

• only Oncocin was ever used routinely

• foundation for much of today’s KBS technology

8

5

6

7

8

MYCIN

• if ... then production rules

• backward chaining inference

• management of uncertainty

• explanation & justification

• complex but bounded domain

• concrete and assessable output

9

MYCIN rule - English

if: the gramstain of the organism is negative

and: the aerobicity of the organism is anaerobic

and: the morphology of the organism is rod

then: the genus of the organism is bacteroides

with a certainty factor of 0.6

10

MYCIN rule - LISP

($AND(SAME CNTXT GRAM GRAMNEG)

(SAME CNTXT MORPH ROD)

(SAME CNTXT AIR AEROBIC))

(CONCLUDE CNTXT CLASS BACTEROIDES TALLY 0.6)

11

MYCIN rule - Prolog

rule(150, Context, genus, bacteroides,0.6) :-

same(Context, gramstain, gramneg),

same(Context, aerobicity, anaerobic),

same(Context, morphology, rod).

12

9

10

11

12

Certainty factors

• every deduced or requested value has a Cf

• tally of a rule is lowest Cf of its components

• Cf for rule x tally = Cf of new value deduced

• Cf combined = Cf1 + Cf2 x (1 - Cf1)

• if Cf = 1 all alternatives are deleted

13

MYCIN structure

Consultation Program

Dynamic Knowledge

about Patient

Context Tree

Values of Clinical

Parameters

Acquisition details

Static Knowledge

about Disease and Therapy

Properties of Contexts and

Clinical Parameters

Production Rules

Tables and Lists

Explanation Program

Clinical Users

Knowledge Acquisition Program

Infectious Diseases Experts

14

MYCIN objects

Context

PersonCulture

OrganismDrug

Operation

ClinicalParameter

NameSite

MorphologyDoseDate

Value

JonesBloodCoccus

100mg bd20 Jan 2004

Properties

Value formatText

Askable

Certainty Factor

15

MYCIN context tree

Patient-1

Culture-1

Organism-1

Therapy-1

Culture-2

Organism-2

Drug-1

Organism-1

Drug-2

Operation-1

Drug-3

16

13

14

15

16

Objects

Control

Relationships

MYCIN analysis

17

Meta-rules

Backward chaining

Rule categories

Production rules

Certainty factors

Context tree

Clinical parameters

Contexts

MYCIN on trial

• blinded evaluation by Yu et al (1979)

• ten selected disease case histories

• 8 doctors + original + MYCIN

• 10 x 10 conclusions sent to 8 experts

• evaluators not aware MYCIN included

18

MYCIN’s performance

60%

62.5%

60%

57.5%

55%

42.5%

45%

30%

57.5%

65%19

Dept. of Infectious Diseases fellow

University faculty member 1

University faculty member 2

University faculty member 3

University faculty member 4

University faculty member 5

Hospital house officer

Hospital medical student

Actual therapy prescribed

MYCIN

MYCIN’s family tree

20

DENDRAL

PUFFGRAVIDA

CLOTONCOCIN

GUIDON

NEOMYCIN

MYCIN

EMYCIN

17

18

19

20

ONCOCIN

21

• Stanford Oncology Clinic

• routine operation from 1981 to 1987

• 30 to 60 protocols of 40 to 60 pages each

• multiple drugs - up to ten per treatment

• 80% of plans accepted by experts

• used as an advisor by less specialised doctors

• enter their plans for comments; integrated into system

• eventually replaced by newer systems (Protégé, EON)

MYCIN revisited ...

rule 9 states that: the context has a genus neisseria

provided that: the gramstain is gramneg

and: the morphology is coccus

or ...

rule(9, Context, genus, neisseria) :-

same(Context, gramstain, gramneg),

same(Context, morphology, coccus).

trace(Context, Parameter, Value) :-

know(Context, Parameter, Value);

deduce(Context, Parameter, Value);

ask(Context, Parameter, Value).

23

PROSE1 - A RECONSTRUCTION OF MYCIN IN PROLOG============================================

Enter <start.> when ] ? prompt appears

] ?start.

m - menu - details of available routines h - help - instructions for use of Prose system s - select knowledge base f - full consultation of expert system b - abbreviated consultation only k - list the knowledge acquired l - list all rules in memory p - return to Primos - ek - edit knowledge base ep - edit Prose file tn - turn on trace mode tf - turn off trace mode pr - return to Prolog command level

24

21

22

23

24

> h

PROSE.1 - a reconstruction of the essential features of MYCIN

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

‘full’ gives a complete ‘MYCIN-like’ consultation establishing patient,

infection and culture before proceeding to identify the organism

‘brief’ moves directly to the identification of the organism

Confidence Factors -------------------------once the essential patient data has been entered, an optional ‘confidencefactor’ in the range -1 (definitely not) to 1 (definitely true) can beadded to the evidence supplied (a Cf of 1 is assumed if omitted)

eg. > gramneg .75

N.B. once ‘start.’ has been entered, no . is required at the end of each line

25

> s

> choose from : bugs - identification of organisms using MYCIN rules

bugs

> knowledge base loaded ...

source: subset of MYCIN rules from Bramer/Shortliffe

goal: to determine the genus of an organism

> f

------------------------- patient1 -------------------------

[1] Surname of patient : Smith

[2] Sex : Male

[3] Age : 37

26

------------------------- infection1 -------------------------

[4] Enter the date at which the infection first appeared : 23/8/89

The most recent culture associated with infection1

will be referred to as :

------------------------- culture1 -------------------------

[5] Enter the date on which the culture was obtained : 25/8/89

The first significant organism from culture1

will be referred to as :

------------------------- organism1 -------------------------

[6] Enter the gramstain of organism1 ... gramneg

[7] Enter the morphology of organism1 ... rod

27

[8] Enter the aerobicity of organism1 ... anaerobic

Rule 35 succeeds with a confidence factor of 0.6

[9] Enter the site of organism1 ... ?

Please enter one of the following : unknown armpit blood csf urine

blood

[10] Enter the portal of organism1 ... gi .6

Rule 200 succeeds with a confidence factor of 0.54

28

25

26

27

28

PROSE.1 concludes that the genus of organism1 is bacteroides

with a confidence of 81.5 percent

> k

> knowledge acquired during the consultation session:

the gramstain of organism1 is gramneg with a Cf of 1 the morphology of organism1 is rod with a Cf of 1 the aerobicity of organism1 is anaerobic with a Cf of 1 the site of organism1 is blood with a Cf of 1 the portal of organism1 is gi with a Cf of 0.6 the genus of organism1 is bacteroides with a Cf of 0.816

> p> returning to Primos ...

OK,

29

PROSE expert system shell

(Ahmad & Wells 1985)

User input

ConclusionsExplanations

Actions

Strategic layerStrategic rules

Foreign clauses & rules

StaticFacts

DynamicFacts

Deductive layerDeductive rules

Management of uncertainty

Unknowns Deductions

30

PROSE applications

• Clinical Biochemistry and medical knowledge

• original design aim and achievement

• front end to WASP

• Wallingford Storm Sewer Project

• allowed non-expert engineers to use the system

• intelligent tolerancing module for Medusa

• Prime CAD/CAM program

• replaced numerous rule books and tables

• BAC Journeyman project

• something to do with things that fly and go bang

31

Book reviews

Recommendations for further study ... or just for enjoyment!

29

30

31

32

Book review

• alternative basic text

• more technical approach than Negnevitsky

• updated version of early classic!

Book review

• how to apply cognitive psychology to software development

• well recommended for all hard-core coders!

• easy to read and good to dip into from time to time

Expert system shellsIntroduction to Penny

Expert system shells

• an expert system with the knowledge removed

• performance and design can be improved in the process

• first example was EMYCIN

• CLIPS is an expert system language - not a shell

• Penny is an expert system shell written in 4D

• based on production rules and Markov algorithm

36

33

34

35

36

Expert system shell for 4D

Download from www.4dcoop.com37

38

39

Why change from CLIPS?

• look at www.ghg.net/clips/CLIPS.html

• CLIPS is free, well documented & powerful but ...

• still a text-based system

• not easy to use in a real-world setting

• Windows versions not very stable

• MSc students wanted something more modern!

40

37

38

39

40

Advantanges of Penny over CLIPS

• written specifically for this course!

• 4D is a modern and open environment

• plenty of scope for GUI and visual effects

• internal working easier to understand and modify

• teaches how to represent knowledge in a database

• 4D skills more likely to prove useful in the future

• Penny shell can be used ‘as is’ for code-shy groups

41

Disadvantages

• Penny is still evolving

• shortcomings will be taken into account

• 4D has a non-trivial learning curve

• has worked well for those who persevere

• documentation on Penny needs improving

• Penny is a ‘grace and favour’ undertaking

42

Development of Penny

• watch the web site for new versions

• please feed back bugs and RFCs via link

• be patient as Penny matures - but enjoy!

• learn to think creatively in the context provided

• keep in contact with Lo Farnan or myself

43

Design of Penny

• based on classic production rules

• if condition (left hand side)

• then action (right hand side)

• combined with semantic network

• if condition then go to node

• inference uses Markov algorithm (G & Riley)

44

41

42

43

44

Inference• the rule cycle always starts at rule one

• rules are always tested in strict rule number order

• if a rule fires (condition evaluates to true) then its action(s) are executed and rule cycle starts again at rule one

• if a rule action is ‘exit’ then the session terminates gracefully

• if no rules fire then the session terminates with a warning (this should normally be avoided)

• a rule will normally only fire once and will be skipped in subsequent cycles

45

Rules

• each rule contains a single condition and one or more actions (but see below)

• multiple conditions handled by salience (rule execution order)

• ‘headless’ rules are allowed in order to construct semantic networks

• rule sets are provided to determine which rules and facts are to be used in a given session

46

Rule properties• name - used to identify rule especially in ‘goto’ actions

• number - order in which rules are tested during a cycle

• set ID - rule set to which rule belongs

• description - free text description of rule

• creation date - date on which rule was created

• created by - name of person who first created the rule

• revision notes - free text history of changes to the rule

• fact name - name of fact to be tested in rule condition

• operator - used to test fact (named above) against the stored value

• value - stored value against which fact is tested

• value type - type of above

• (string, value-list, text, integer, real, date, time, boolean)47

Rule conditions• equal to - test named fact against stored value

• greater than - test named fact against stored value

• greater than or equal to - test named fact against stored value

• less than - test named fact against stored value

• less than or equal to - test named fact against stored value

• true - test named fact against stored value

• false - test named fact against stored value

• headless - no test performed, rule can only be accessed from a goto action

48

45

46

47

48

Rule actions• each rule must have at least one action

• all actions are executed in the order specified

• assert - set the named fact to the stored value

• askfor - ask for a value and set the named fact to the value supplied

• goto - go to the named rule (which must be headless)

• exit - terminate the session (at least one rule should have this action)

• execute - run the named 4D method (hook for special functions or actions)

• display - display the stored comment

• write - write the stored comment to a text file

• open - open the named document

• play - play the named sound

• run - show the named video clip49

Control options

• buttons in dialog window to control session

• help - display free text comment to provide assistance in answering a question

• why - display free text comment explaining why a question is being asked

• how - display trace of session showing how it has arrived at this rule

• back - roll back the session to the previous rule fired and state of dynamic facts

50

Facts

• facts are addressed by name in the same way as rules

• facts also belong to rule set to determine which will be used in a given session

• static facts are stored in the database and are loaded at the start of a session

• dynamic facts are always cleared at the start of each session

• semi-dynamic facts are ‘remembered’ between successive sessions unless manually cleared

51

Fact properties• name - used to identify fact in rule conditions and actions

• set ID - rule set to which the fact belongs

• description - free text description of the fact

• prompt - help text to assist with answering question associated with fact

• creation date - date on which the fact was created

• created by - name of person who first created the fact

• revision notes - free text history of changes to the fact definition

• value - stored value which is assigned to the fact when session is loaded

• value type - type of the fact

• (must match that set by the rule/rules which test the fact)52

49

50

51

52

Groups and subjects

Week 4 update

Ideal group

• administrator

• domain expert

• one or two programmers

• one or two knowledge engineers

54

Subjects from 2000 to 2004

55

Bank loans

Career advice

Holiday destinations

Video production

Plant classification

Mobile phone calling plan selection

Advisory system for configuring PCs

Safety precautions for sub-aqua diving

Mobile phone purchase advice

Child care advice

Digital camera selection

Overseas travel advice

Car purchase advice

Strategy game assistant

Car fault advisor

University selection

Cricket umpiring

Lifestyle and diet

Video rentals

PC product pricing

Football team strategy

Intelligent travel planner for New Zealand

MSc student time advisor and organiser

Historical tour planner for Europe

Intelligent plant identifier for amateur use

House purchase advisor for Guildford area

Subjects for 2005 to 2007

56

Web site development

MSc course selection

Music selection

Cancer likelihood

Diabetes diagnosis/advice

Intelligent examination setter

Student lifestyle advisor

Investment capital advisor

London Underground guide

Accommodation selection

Laptop selection

MSc course selection

Restaurant selection

A - gastronomic advice

B -

C -

D -

E -

53

54

55

56

Reasoning in the real world

Moving on from puzzles ...

Habituation

“Einstellung - habituation - creates a

mechanised state of mind; a blind attitude

towards problems; one does not look at the

problem on its own merits but is led by a

mechanical application of a used method”

58

From puzzles to the real world

• problem space concepts

• lack of definition

• experts and novices

• knowledge vs intelligence vs wisdom

• algorithms vs heuristics

• encoding and representation

59

Algorithms and heuristics

• an algorithm is a method or procedure that will always solve the problem

• but it may take a VERY long time

• a heuristic is a ‘rule of thumb’ that may well solve the problem quickly but may also fail to provide a solution

• more often than not it will succeed and save considerable time and effort

60

57

58

59

60

Chess as a problem space

• complex but bounded

• well defined rules and problem space

• large pool of experts and novices

• numerous computer simulations

• what makes an expert?

61

Kasparov vs Big Blue in 1997

Kasparov lost by 2.5 games to 3.562

Kramnik vs Deep Fritz in 2002

Kramnik drew 4 games all after a tight finish63

Kasparov vs X3D Fritz in 2003

Kasparov drew 2 games all64

61

62

63

64

Studies with chess• de Groot (1960s) and Simon (1970s)

• consider 30 moves and up to six turns

• board positions from actual games

• masters 91% but novices 41%

• no difference with random positions

• reconstruction and chunking

• chess masters store 100,000 patterns

• and have studied for 10 years

• heuristics (Holding & Reynolds 1982)65

General Problem Solver

• Allen Newell and Herb Simon

• General Problem Solver (1963)

• Human Problem Solving (1972)

• problem-space theory

• means-ends analysis

• progressive deepening

66

Problem-space theory

• problem has initial state and goal state

• number of intermediate states and alternative paths

• mental operators used to move from one state to the next taking account of legal constraints

• knowledge and heuristics to search space

• limitations of cognitive system include capacity of working memory and the speed of retrieval from long term storage

67

Means-ends analysis

• note the difference between the current state

and the goal state

• create an achievable intermediate state (sub-

goal) that will reduce this difference

• select and apply a suitable operator to move to

the intermediate state

68

65

66

67

68

Problem solving

Start

GoalSub-goals

69

Tower of Hanoi problem

70

Physics problem

A block of mass M is dropped from a height h onto a spring of force constant K.

What is, neglecting friction, the maximum distance the spring will be compressed?

71

Physics solutions

• novices vs experts

• surface features - novices

• deep structure - experts

• analysis phase - longer for experts

• forward chaining - experts

• backward chaining - novices

72

69

70

71

72

Deductive reasoning

• impossible for the conclusion to be false if the premises are true

• modus ponens

• if A then B ... given A then infer B

• modus tollens

• if A then B ... if A is false infer B is false

• affirmation of consequent and denial of antecedent

73

Deductive reasoning examples

If it is raining I take an umbrella

It is raining

Therefore I take an umbrella

If it is raining I take an umbrella

If I take an umbrella I lose it

It is raining

Therefore I lose my umbrella

74

Wason’s four card problem (1971)

If a card has a vowel on one side

Then it has an even number on the other side

Which card(s) need to be turned over to prove the rule?

E K 2 7

75

Wason’s four card problem -2

If a student is drinking beer

Then the student must be over 18 years old

Which card(s) need to be turned over to prove the rule?

Beer Coke Age22

Age16

76

73

74

75

76

Inductive reasoning

• improbable that conclusion is false if premises are true

• general conclusion from premises referring to specific instances

• base rate rule

• probability of class membership increases with number of members

• conjunction rule

• probability of a proposition cannot be less if it is combined with another

77

Inductive reasoning example 1

• Linda is 31 years old and single; she is outgoing and

outspoken; she has a degree from Exeter University; she is very concerned with issues of discrimination

• which is more likely?

A - Linda works for a firm of management consultants

B - Linda is an ardent feminist and works for a firm of

management consultants

78

Inductive reasoning example 2

• Linda is 31 years old and single; she is outgoing and outspoken; she has a degree from Exeter University; she is very concerned with issues of discrimination

• does the probability of C increase given B?

A - Linda works for a firm of management consultants

B - Linda is a member of a club where 90% of the members are management consultants

C - Linda is a management consultant

Hilgard pp 328 to 33279

Reasoning strategies

• from facts to conclusions

• forward chaining

• bottom up

• from hypotheses to proof

• backward chaining

• top down

80

77

78

79

80

Perceptual model

Environment

Top downExpected features

DeductiveBackward chaining

Bottom upFeature analysisInductiveForward chaining

Reasoning in the real world

Neisser’s cyclic model of perception81

• work forwards to find what solutions follow from facts

• planning, monitoring, control

• present to future

• antecedent to consequent

• data driven, bottom up reasoning

• better for breadth-first search

• antecedents determine search

• not easy to provide explanations

Forward chaining

82

Backward chaining

• work backwards to find facts that support hypothesis

• diagnosis (medical e.g. MYCIN or general)

• present to past

• consequent to antecedent

• goal-driven top-down reasoning

• depth-first search facilitated

• consequents determine search

• easy to provide explanations

83

Next week ...

• filling the gaps ...

• frames, cases and other new directions

• uncertainty ...

• the real world is like this

• projects - support and feedback

84

81

82

83

84