8/12/2019 Brasil 96

1/6

A Methodology for Implementing Hybrid Expert Systems

L.M. Brasil, F.M. de Azevedo, R.Garcia and J.M. Barreto.Dept. of Electrical Engineering,

Federal University of Santa Catarina

Biomedical Engineering Research Group (GPEB)

University Campus, Florianpolis, Brazil, 88040-900

Phone (48)231-9594 Fax(48) 231-9770 E-Mail: lourdes@gpeb.ufsc.br

Abstract - This paper deals with a new methodology for

developing an Expert System (ES). It has the ability of learning

to extract knowledge from a poor knowledge base using the

learning by example paradigms. The choice of a poor knowledge

base was motivated by the fact that in this case it is easier to

have consistence in putting together the several pieces of

knowledge. So, the problems attached to knowledge elicitation

are simplified. The implementation leads to a Hybrid Expert

System (HES). This system consists of a Neural Network based

Expert System (NNES) and a Rule Based Expert System(RBES). The main idea is that if the knowledge engineer has

conditions to obtain some basic rules, and a set of examples,

from the domain expert then it is possible to define the basic

structure of the NNES using those basic rules. Then the NNES

can be refined using the set of examples. In this stage structural

changes of the network are allowed by the learning algorithm.

Rules can be deduced after this refinement. Then it can be used

to form a RBES and an Explanatory Expert System (EES). The

methodology developed to HES is intended to be used in

implementing Decision Support Systems in the Medical Area.

I -INTRODUCTION

At present, one of the most known products of the

Artificial Intelligence (AI) is ES. It has proved its efficiency

independently of the implementation paradigm adopted:

symbolic manipulation or connectionist.

There are several problems in building an ES. One of them

is the process of Knowledge Acquisition (KA), which

comprehend knowledge extraction from a domain expert and

the choice of the model for the knowledge representation to

code human reasoning [1]. Knowledge elicitation stage

consumes time mainly because normally human beings, even

knowing how to solve a problem, have difficulty in

explaining how they reached the solution of a specific

problem by themself. Moreover, there is the problem of

knowledge representation of a domain expert. There are

many forms to represent it. When symbolic manipulation is

used, production rules are one of the most common

knowledge representation schemes used. It is simple and

direct, but it relies on a rich knowledge base. Moreover, it is

necessary to update it constantly. In case of connecionist

paradigm, the problems have been basically the choice of

data to be fed to the input of neural network (NN), the

number of neurons in the hidden layer and the kind of

topology of the NN used. Finally, only in some special cases,

there are difficulties in obtaining the explanation on how the

network arrived to a conclusion.

So, a HES is proposed to deal with the problems

mentioned above. This system consists of a NNES, a RBES

and, an EES [2].

The paper presents the architecture and operation of the

HES, the system methodology, the learning algorithm, and

discusses the proposed approach.

II- PROPOSED SYSTEM

The proposed architecture for ES includes a NNES, a

RBES, and an EES.

NNES has been used to implement ES as an alternative

manner to RBES. Artificial Neural Networks (ANN) are

made through a big number of units. These units own some

properties like natural neurons. Therefore, every unit presents

several inputs, some excitatories and others inhibitories.

Moreover, these units take values of each input and generate

an output that is function of the inputs. So, a network ischaracterized by the units (neurons) and by the way the

neurons are connected (topology). Moreover, algorithms are

used to change weights of the connections (learning rules).

Thus, these three aspects constitute the connectionist

paradigm of the artificial intelligence [3]. The

implementation of ES this way are called NNES. These

systems are generally developed using a static network with

feedforward topology trained by a backpropagation-like

learning algorithm [4]. So, while the basic network represents

relations among concepts and connections by way of

inferring something through it, the set of examples will refine

NNES. In this last process, the algorithm provides

modifications not only in the weights of connections, but alsoin the network structure. It uses this topology and it generates

and/or eliminates connections that had not been in basic

rules. Moreover, it can also occasionally generate more

concepts that were not in the basic rules. Therefore, the

system translates as rules the basic rules that the expert was

not able to extract. The basic rules after extraction suffers a

treatment due to the kind of variables applied as input of the

network, where they represent different types of concepts, as

quantitative, linguistic, boolean or a combination of them

8/12/2019 Brasil 96

2/6

[2,4,5]. Moreover, a structural modification of the ANN

consists in determining the number of neurons of the hidden

layer using a Genetic Algorithm (GA) [6].

RBES broadly depends on formal logic as a way of

explicit knowledge representation. Our system has two kinds

of data: basic rules and set of examples. The basic rules are

used to create the initial NNES which is refined through the

examples. The refined NNES is then translated in a RBES

from which explanations can be obtained. The model

proposed is through fuzzy logic. The theory of fuzzy logic

provides a good mathematical framework to represent

imprecise knowledge as in our case [2,5,7].

Finally, the EES of the HES is derived from RBES. It

compares the answers given by the NNES and by the RBES.

If the two answers are equals the EES is triggered and it will

give an explanation. Otherwise, it will state the impossibility

of reaching the goal and it will suggest how to obtain a

suitable solution [2].

III -METHODOLOGY

One of the difficulties in eliciting the knowledge of domain

experts is when one wishes to obtain an adequate set of rules.

Firstly, in many fields experts are not capable of realizing or

articulating which knowledge they use in solving their

problems. Secondly, different experts have often different

explanations to their decisions and sometimes even their

decisions disagree. [3].

On the other hand, a NNES needs only a set of examples to

represent the problem considered. However, as the

knowledge is often distributed in the network connections,

explaining how the NNES reached a conclusion is very

difficulty except if the knowledge representation is localized.Or, explaining the ES reasoning is of capital importance,

mainly when the user disagrees with the ES suggestion and

also during the tuning phase. This is extremely important in

our case, where physicians are the potential users. So, the

proposed methodology deals with a hybrid scheme, which

explores the intrinsic suitable characteristics of each

approach (Fig.1).

KNOWLEDGE ACQUISITION

INTEGRATED SYSTEM

EXPLANATORY SYSTEM

RBES

REFINED NNES

NNES

SET OF

EXAMPLES

BASIC

RULES

Fig.1 - Block diagram of the general system.

A.Knowledge Acquisition

The KA task consists on extracting knowledge of the

domain expert (i.e., a physician expert). In our case, the main

goal is to minimize the intrinsic difficulties of the KA

process. To do so, we try to obtain all possible rules from the

domain expert in a short time and also a set of examples.

The basic rules can be improved capturing the

uncertainties associated with the human cognitive processes.

The model proposed uses fuzzy logic [5].

Basic rules are translated by AND/OR graphs, so that they

define the NN basic structure of the NNES. In other words,

an AND/OR graph, which represent concepts and

connections, indicates the neurons number in the input and

output layers. The graph also shows the existence of

intermediate concepts and their connections which they

translate in the intermediate layer of the NN, according to

Fig.2.

R6

R6: If Q Then X.

R5: If P Then X.

R4: If G And H Then Q.

R3: If E And F Then Q.

R2: If C And D Then P.

R1: If A And B Then P.

Basic Rules:

R4R3

R5

R2R1

X

QP

HGFEDCBA

Fig.2- Organization of the rules base as a network.

Basic rules consist of an if-part, that indicates the

antecedent and it expresses, in our case, the symptoms of a

disease, while a then-part deals with the consequent and it

expresses the possible diagnostics [8]. Moreover, each of

these rules presents a membership degree. It corresponds tothe value that will be put in the inputs of the basic NN after

the treatment of the semantic variables.

B.Neural Model

A NNES structure that has conditions to receive several

kinds of semantic variables, i.e., boolean, linguistic and,

quantitative inputs, is considered (Fig.3). These inputs have

been treated as a unique kind of variable of fuzzy type. It is

believed that this sort of structure deals with modeling a

structure by a possible simpler way and also consuming less

learning time than a traditional structure.

Quantitative

Boolean

Linguistic

STATEMENTS

Neural

Inputs

(Neural

Outputs)

DecisionsNeural

Network

Learning

Algorithm

Interface

to Fuzzy

Fig.3 - State Variable Kinds

8/12/2019 Brasil 96

3/6

The mathematics model of the neuron is given by,

X(t)= n-dimensional input vector

X(t) = [x1(t), x2(t),..., xi(t),...,xn(t)]T n (1)

y(t)= scalar output of each neuron

y(t) 1

N: nonlinear mapping function, X Y, x(t) |y(t)

where: X:

+

n

and Y:

+

(2)

This mapping can be noted as N and so:

y(t) = N [X(t) n] 1 (3)

Mathematically, the neural nonlinear mapping function N

can be divided into two parts: a function called confluence

[5] and a nonlinear activation operation. The confluence

function is the name given to a general function having as

arguments the synaptic weights and inputs. A particular case

widely use is the inner product.

This mapping yields a scalar output u(t) which is a

measure of the similarity between the neural input vector X(t)

and the knowledge stored in the synaptic weight vector W(t).So, u(t) and W(t) are given by,

W(t) = [w0(t), w1(t),..., wi(t),..., wn(t)]T n+1 (4)

u(t) 1

Redefining X(t) to include the bias x0(t) we have:

u(t) = X(t) W(t) (5)

The nonlinear activation function maps the confluence

value u(t) [-,] to a bounded neural output. Then, the nonlinear

activation operator transforms the signal u(t) into a bounded

neural output y(t), that is,

y(t) = [u(t)] (6)

y(t) = [W(t) X(t)] 1 (7)

Applying the equations (1), (4), (5), and (7) to a multilayer

NN (e.g.,three layers), we have:

Y(t) = N3[N2[N1[X(t)n]]] m (8)

Y(t) = 3[W3(t) 2[W2(t) 1[W1(t) X(t)]]] (9)

Where i is non-linear activation operator, is the

confluence operator, and W1(t), W2(t), and W3(t) are the

synaptic weight vectors for the input, hidden and output

layers, respectively.

If we express the neural input signals in terms of their

membership functions each over the interval [0,1], rather

than in their absolute amplitudes, then we can perform

mathematical operation on these signals using logical

operations such as AND/OR, according to [5].

Let us express the inputs x1 and x2 over [0,1]. Then we

define the generalized AND (T-norm) as a T mapping

function and generalized OR (T-conorm) as a S mapping

function [7]:

T: [0,1] x [0,1] [0,1]

S: [0,1] x [0,1] [0,1], given by:

y1= [x1AND x2] [x1T x2] = T[x1, x2] (10)

y2= [x1OR x2] [x1S x2] = S[x1, x2] (11)

Then, in (5), by replacing the -operation by the T-

operation, and the -operation by the S-operation, we get

u(t) = Si

n

=0[wi(t) T xi(t)] [0,1] (12)

and

y(t) = [u(t)] [0, 1] (13)

C. Other Stages of the HES

After the basic NNES is obtained, the set of examples

serves to validate the neural network structure. In worst case,

the network does not represent the knowledge of the

problem. Therefore, it becomes evident that the basic rulesextracted from the expert are not sufficient, as expected. So,

these same examples are used by the learning algorithm to

refine the NN. This algorithm can change, generate and/or

eliminate connections, or it can still generate or eliminate

neurons in the hidden layer. After the refinement of the

network, a new discussion is made with the domain expert to

validate the modifications in the basic structure of the

network. Thus, a new set of examples is obtained to test

again the network. In the case that it has well performed, it is

supposed the network represents the proposed goal.

After the NNES is refined, a reverse process is followed

toward the inferring of the if-then rules together with their

membership degrees. So, a RBES is implemented. Following,

the RBES serves as basis for developing other system, the

EES, that is supposed able to explain why the NNES reached

a conclusion.

IV- LEARNING ALGORITHM

The learning algorithm developed for NNES is inspired on

the traditional backpropagation algorithm. Nevertheless, it

presents some differences:

- Optimization of the hidden layer is supported by GA.

- Incorporation of logic operators AND/OR in place of theweighted sum.

Therefore, the conventional backpropagation learning

algorithm is altered in function of these modifications. Some

works that integrate these topics are [2,4,5,6,8,9,10].

A.Learning Algorithm Stages

Summarizing, the first stage in the implementation of the

learning algorithm is considered as the assembly of the NN

basic structure. It is based in function of a AND/OR graph, in

8/12/2019 Brasil 96

4/6

other words, the hidden layer consists of AND nodes and the

output layer of OR nodes. So, the logic operators AND/ OR

are incorporated in place of the weighted sum in the network

neurons [5].

In next stage, the learning algorithm uses examples, which

are extracted from the domain expert, such as modifying, not

only the weight of each connection, but also the network

structure. In this last case, connections are included or

excluded among neurons, as well as neurons can be included

or excluded of the hidden layer by the application of the GA

[6].

B. Genetic Algorithm

GA are based on the work of Holland [11] where he was

inspired by the evolution of a population subject to

reproduction, mutations and crossover in a selective

environment. So, following this idea, we choose GA to

optimize [6] the size of the hidden layer and determining

weights to be set to zero. This can be justified by the

following main facts: it allows to avoid local optimum andprovides near-global optimization solutions and they are easy

to implement. Nevertheless, when it is applied with this goal,

in hidden layer of a NN, we must take care of respecting a

maximum and a minimum number of neurons of this layer. In

fact, too many neurons generally have as effect a decrease of

the generalization capabilities of the network and implies a

long learning phase. On the other hand, too few neurons can

be unable to learn, with the desired precision, the task. So,

there is an intermediate number of neurons that must be put

in the hidden layer, to avoid the problems mentioned above.

Then, the network must be sufficiently rich to solve a

problem and as it must also be adequately simple to solve a

problem well as it must not consume longer time of training.The parameter called as momentum, which it has the

objective to give higher speed of training to network, can still

be added in this same algorithm [12].

V- RESULTS AND DISCUSSIONS

An HES, including a RBES and a NNES, is discussed

under the aspects of KA, where the treatment of imprecision

is a possibility of explaining the reasoning to attain a

conclusion. The KA phase is based on learning by example

paradigms and the intrinsic capabilities of the NNES are

exploited. Soon afterwards, that knowledge can be

transferred to the RBES that uses fuzzy logic to deal with

imprecision. This methodology has proved, in the

preliminary studies performed, very promising by leading to

an easier KA phase than expected if the KA was performed

using symbolic techniques alone. The hybridism, on the other

hand, allows to complement the NNES with explanation of

reasoning facilities, that in most cases are difficult to obtain

with a NNES.

In future we intend to validate the approach with a

concrete example. This example will be probably the

construction of a Medical Decision Support System to

classify epileptic crises. This system will have about 15 to 30

rules combined with their membership degrees. These data

will be elicited by physician experts, mainly at the University

Hospital of Federal University of Santa Catarina.

ACKNOWLEDGMENT

The first author acknowledges the CAPES (Coordination

Foundation of High Level Personnel Improving) and the last

author the CNPq (National Counsil for Scientific and

Technological Development) - RHAE Program - for the

material support in the development of this work.

REFERENCES

[1] L.M. Brasil, "Aquisio de Conhecimento Aplicada aoDiagnstico de Epilepsia", M.Sc.Thesis in Electrical

Engineering, Federal University of Santa Catarina,

Florianpolis, UFSC, 1994.

[2] L. M. Brasil, F.M. de Azevedo, R. Garcia and J.M. Barreto

Cooperation of Symbolic and Connectionist Expert System

Techniques to Overcome Difficulties. In: Proceedings of 2nd

Neural Networks Brazilian Congress, Brazil: Curitiba,

pp.177-182, 1995.

[3] F.M. de Azevedo, "Contribution to the Study of Neural

Networks in Dynamical Expert Systems", Ph.D. Thesis,

Institut d'Informatique, FUNDP, Belgium, 1993.

[4] S. Mitra and S.K. Pal, "Logical operation based fuzzy MLP

for classification and rule generation", Neural Networks,vol.7, no.2, pp.353-373, 1994.

[5] M.M. Gupta and D. H. Rao, "On the principles of fuzzy neural

networks", Fuzzy Sets and Systems, vol.61, no.1, pp.1-18,

1994.

[6] R. G. Ojeda, F.M. Azevedo and J.M. Barreto, "Genetic

algorithms in the optimal choice of neural networks for signal

processing", in 38th. Midwest Symposium on Circuits and

Systems, Rio de Janeiro, Brazil. (In Press), 1995.

[7] H.J. Zimmermann, Fuzzy Set Theory - and Its Applications,

Massachusetts, USA: Kluwer Academic Publishes, 1991.

[8] L.M. Fu, "Knowledge-based connectionism for revising domain

theories", IEEE Transactions on Systems, Man, and

Cybernetics, vol.23, no.1, pp.173-182, 1993.[9] H. Ishibuchi, R. Fujioka and H. Tanaka, "Neural networks that

learn from fuzzy if-then rules", IEEE Transactions on Fuzzy

Systems, vol.1, no.2, pp.85-97, 1993.

[10] R. Sun, "Rule-based structure identification in an adaptive-

network-based fuzzy inference system", IEEE Transactions on

Fuzzy Systems, vol.2, no.1, pp.64-73, 1994.

[11] J.H. Holland, Adaptation in Natural and Artificial Systems,

Cambridge, Massachusetts: MIT Press, 1992.

[12] J.M. Zurada, Introduction to Artificial Neural Systems, St.

Paul, MN: West Publishing Company, 1992.

8/12/2019 Brasil 96

5/6

8/12/2019 Brasil 96

6/6