Integration of Symbolic and Connectionist AI techniques fine...Engineering Systems, 11th...

PhD Curriculum Introduction Hybrid Architectures Rule Engine Results and Future Works

Integration of Symbolic and ConnectionistAI techniques

Decision Support Systems for biochemical processes

Davide Sottara

Seminari III anno dottorato - XXII ciclo


Outline

1 PhD Curriculum

2 IntroductionCase Study : Water TreatmentState of the Art

3 Hybrid ArchitecturesProposed ArchitectureHybrid Modules

4 Rule EngineInference under UncertaintyRule Language and Engine

5 Results and Future Works


Curriculum

Tutor:Prof. P. Mello

Co-Tutor:Ing. L. Luccarini

Cooperations :ENEA - ACS PROT IDR WaterResource Management Section(Jan 07 - Dec 09)

University of Newcastle / JBoss(Feb 09 - Jun 09)

Other:Track Co-Chair at RULEML09 “Rulesand Uncertainty”


Publications

Journal Papers

G. L. Bragadin, G. Colombini, L. Luccarini, M. Mancini, P. Mello, M. Montali,and D. Sottara.Formal verification of wastewater treatment processes using events detected fromcontinuous signals by means of artificial neural networks. Case study: SBR plant.Environmental Modelling and Software (IF 2.659).Article in Press.

P. Mello, M. Proctor, and D. Sottara.A configurable RETE-OO engine for reasoning with different types of imperfectinformation.IEEE Transactions on Knowledge and Data Engineering (TKDE) - Special Issueon Rule Representation, Interchange and Reasoning in Distributed,Heterogeneous Environments (IF 2.236).Article in Press.


Conference Acts I

Sottara D., P.Mello, L.Luccarini, and G.Colombini.

Controllo e gestione intelligente degli impianti di depurazione.In Europa del Recupero : le ricerche, le tecnologie, gli strumenti e i casi studio per una cultura dellaresponsabilitÃ ambientale, pages 156 – 161, S.Arcangelo di Romagna (RN) – ITA, 5-8 Novembre 2008.Maggioli Editore (ITALY).

D.Sottara, L.Luccarini, and P.Mello.

Strumenti di IA per il controllo e la diagnosi dei processi biologici negli impianti a fanghi attivi.In Europa del recupero : le ricerche, le tecnologie, gli strumenti e i casi studio per una cultura dellaresponsabilitÃ ambientale, pages 150 – 155, S.Arcangelo di Romagna (RN) – ITA, 5-8 Novembre 2008.Maggioli Editore (ITALY).

L. Luccarini, P. Mello, D. Sottara, and A. Spagni.

Artificial Intelligence based rules for event recognition and control applied to SBR systems.In Conference Proceedings of the 4th Sequencing Batch Reactor Conference, pages 155 – 158, ROMA –ITA, 7-10 April, 2008. s.n.

M. Nickles and D. Sottara.

Approaches to Uncertain or Imprecise Rules - A survey.In G. Governatori, J. Hall, and A. Paschke, editors, Rule Interchange and Applications, InternationalSymposium, RuleML 2009, Las Vegas, Nevada, USA, November 5-7, 2009. Proceedings, volume 5858 ofLecture Notes in Computer Science, pages 323–336. Springer, 2009.

D. Sottara and P. Mello.

Modelling radial basis functions with rational logic rules.In E. Corchado, A. Abraham, and W. Pedrycz, editors, Hybrid Artificial Intelligence Systems, ThirdInternational Workshop, HAIS 2008, Burgos, Spain, September 24-26, 2008. Proceedings, volume 5271 ofLecture Notes in Computer Science, pages 337–344. Springer, 2008.


Conference Acts II

D. Sottara, L. Luccarini, P. Mello, S. Grilli, M. Mancini, and G.L. Bragadin.

Tecniche di intelligenza artificiale per la gestione e il controllo di impianti di depurazione. caso di studio:SBR in scala pilota alimentato con refluo reale.In Luciano Morselli, editor, Ambiente: tecnologie, controlli e certificazioni per il recupero e la valorizzazionedi materiali ed energie. ECOMONDO X Fiera Internazionale del Recupero di Materia ed Energia e delloSviluppo Sostenibile. Rimini. 8-11 novembre 2006, volume 1, pages 106 – 111. Maggioli Editore (ITALY),2006.

D. Sottara, L. Luccarini, and P. Mello.

AI techniques for Waste Water Treatment Plant control. Case study: Denitrification in a pilot-scale SBR.In B. Apolloni, R. J. Howlett, and L. C. Jain, editors, Knowledge-Based Intelligent Information andEngineering Systems, 11th International Conference, KES 2007, XVII Italian Workshop on Neural Networks,Vietri sul Mare, Italy, September 12-14, 2007. Proceedings, Part I, volume 4692 of Lecture Notes inComputer Science, pages 639–646. Springer, 2007.

D. Sottara, P. Mello, and M. Proctor.

Adding uncertainty to a RETE-OO inference engine.In N. Bassiliades, G. Governatori, and A. Paschke, editors, Rule Representation, Interchange and Reasoningon the Web, International Symposium, RuleML 2008, Orlando, FL, USA, October 30-31, 2008.Proceedings, volume 5321 of Lecture Notes in Computer Science, pages 104–118. Springer, 2008.

D. Sottara, G. Colombini, L. Luccarini, and P. Mello.

A Pool of Experts to evaluate the evolution of biological processes in SBR plants.In E. Corchado, X. Wu, E. Oja, Á. Herrero, and B. Baruque, editors, Hybrid Artificial Intelligence Systems,4th International Conference, HAIS 2009, Salamanca, Spain, June 10-12, 2009. Proceedings, volume 5572of Lecture Notes in Computer Science, pages 368–375. Springer, 2009.


Conference Acts III

D. Sottara, L. Luccarini, G.L. Bragadin, M.L. Mancini, P. Mello, and M. Montali.

Process quality assessment in automatic management of wastewater treatment plants using formalverification.In International Symposium on Sanitary and Environmental Engineering-SIDISA 08 -Proceedings, volume 1,pages 152/1 – 152/8, ROMA – ITA, 24-27 june 2008 2009. ANDIS.

D. Sottara, A. Manservisi, P. Mello, G. Colombini, and L. Luccarini.

A CEP-based SOA for the management of wastewater treatment plants.In EESMS 2009. IEEE Workshop on Environmental, Energy, and Structural Monitoring Systems, 2009.Proceedings, pages 58–65, 25/09/ 2009.

D. Sottara, P. Mello, L. Luccarini, G. Colombini, and A. Manservisi.

Controllo intelligente in linea per una gestione efficiente e sostenibile degli impianti di trattamento reflui.Caso di studio: SBR in scala pilota.In Ecodesign per il pianeta: soluzioni per un ambiente pulito e per una nuova economia, pages 655 – 660,S.Arcangelo di Romagna (RN) – ITA, 28-31 Ottobre 2009. Maggioli Editore (ITALY).

D. Sottara, P. Mello, and M. Proctor.

Towards modelling defeasible reasoning with imperfection in production rule systems.In G. Governatori, J. Hall, and A. Paschke, editors, Rule Interchange and Applications, InternationalSymposium, RuleML 2009, Las Vegas, Nevada, USA, November 5-7, 2009. Proceedings, volume 5858 ofLecture Notes in Computer Science, pages 345–352. Springer, 2009.

N. Wulff and D. Sottara.

Fuzzy reasoning with a RETE-OO Rule Engine.In G. Governatori, J. Hall, and A. Paschke, editors, Rule Interchange and Applications, InternationalSymposium, RuleML 2009, Las Vegas, Nevada, USA, November 5-7, 2009. Proceedings, volume 5858 ofLecture Notes in Computer Science, pages 337–344. Springer, 2009.


Case Study : Water Treatment

Outline

1 PhD Curriculum







Sequencing Batch Reactors

Single treatment tank

Cyclic process : Reactions sequential in time

pH, redox potential orp, dissolved oxygen DO probes



Intelligent Management of Complex Systems

Control

Process Optimization

Greater efficiencyMoney/Energy savings

Diagnosis

Anomaly Prevention

Fault Isolation

Automatic Intervention

Plant

Probes Actuators

Detection Reaction

Diagnosis Prevention

Support Intervention

Operator



Reaction Completion

Change in signal trends are correlated to completed reactions

0 50 100 150 200 250 300 350−2.5

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

Load Anox Aero Set

Draw

Idle

Denitrification Nitrification


State of the Art

Outline

1 PhD Curriculum






State of the Art

A few considerations...

The problem is complex

Models are not applicable

Decision Support Systems are more suitable

No single AI technology is optimal


State of the Art

Integrated Intelligent Remote Control

PlantProbes Actuators

PID

ControllerDatabase

User Interface

DSS

Integrated RemoteControl System

Combines differenttechnologies

Remote Access

Diagnosis & FaultDetection

Optimal Set-pointControl

Operator Support

Benefits

Full KB system

Reactive / Proactive

Limitations

Monolithic architecture

Coordination?


State of the Art

Integrated Intelligent Remote Control


PID

ControllerDatabase

User Interface

DSSDN FRO

Analysis Techniques

Different approaches

Data Mining

Neural Networks

Fuzzy Logic

Rule BasedSystems

Ontologies

Benefits

Full KB system

Reactive / Proactive

Limitations

Monolithic architecture

Coordination?


Proposed Architecture

Outline

1 PhD Curriculum







Complex Event-Driven SOAs

CED-SOA

A Service-Oriented Architecture for Complex Event Processing

Services may interact producing or consuming events

Looser couplingReactiveness

Services aggregate events

Implementation is hidden

The middleware delivers events

Producers need not know Consumers, if any



Towards Complex Achitectures


Ctrl + PIDDatabase

User Interface

DSSDN FRO





Ctrl + PIDDatabase

User Interface

DSS

DN FRO





Ctrl + PIDDatabase

User Interface

DSS

D N F R O





Ctrl + PIDDatabase

User Interface

DSS

ORFN

D





Database

User Interface

DSS

ORFN

D

Controller





Database

User Interface

DSS

ORFN

D

ControllerAcquisition





ORFN

D


Store

DW

I/O

UI





ORFN

D


Store

DW

I/O

UISecurityAdmin

Registry . . .





ORFN

D


Store

DW

I/O

UISecurityAdmin

Registry . . .

Scheduler





ORFN

D


Store

DW

I/O

UISecurityAdmin

Registry . . .

Scheduler

Rule





ORFN

D


Store

DW

I/O

UISecurityAdmin

Registry . . .

Scheduler

Rule

Router



Event-Processing Networks : (Loose) Interactions

Chart Statistics Scheduler

Predict

Probes Denoise Analysis Policy Control Actuators

Trace

Storage Router



Events : A typical scenario


Predict


Trace

Storage Router

Raw





Predict


Trace

Storage Router

Sample





Predict


Trace

Storage Router

Trend

Stage

Estimate





Predict


Trace

Storage Router

Switch





Predict


Trace

Storage Router

Phase





Predict


Trace

Storage Router

Switch


Hybrid Modules

Outline

1 PhD Curriculum






Hybrid Modules

AI Modules


Predict


Trace

Storage Router

Num NumR

SOMR

SOMFFR

P R

R

R


Hybrid Modules

Hybrid Systems

Combine benefits of Soft and Hard Computing

Hard Computing (HCS)

Encode Knowledge

Self-Explanatory

Reason

Soft Computing (SCS)

Learn

Flexible

Evaluate

Complementary

Problem : Integration

The output of SCS is Uncertain and unsuitable for HCS


Hybrid Modules

An Ontology for Uncertainty (W3C)

Uncertainty

Nature Derivation Type Model

AleatoryEpisthemic

SubjectiveObjective

IncompletenessVagueness

InconsistencyRandomness

Ambiguity

FuzzySetsRoughSets

RandomSetsBelief

Probability

more. . .


Hybrid Modules


Uncertainty


AleatoryEpisthemic

SubjectiveObjective



Ambiguity

FuzzySetsRoughSets

RandomSetsBelief

Probability

more. . .

Uncertainty / Confidence Factors


Hybrid Modules


Uncertainty


AleatoryEpisthemic

SubjectiveObjective



Ambiguity

FuzzySetsRoughSets

RandomSetsBelief

Probability

more. . .

Uncertainty / Frequentist Probability


Hybrid Modules


Uncertainty


AleatoryEpisthemic

SubjectiveObjective



Ambiguity

FuzzySetsRoughSets

RandomSetsBelief

Probability

more. . .

Uncertainty / Bayesian Probability


Hybrid Modules


Uncertainty


AleatoryEpisthemic

SubjectiveObjective



Ambiguity

FuzzySetsRoughSets

RandomSetsBelief

Probability

more. . .

Vagueness / Fuzzy Logic


Hybrid Modules

Rule examples : Analysis

r u l e "Step Up"// f u z z y r u l e

when$ f : F e a t u r e s ( d e l t a T i s "short"

and d e l t a Y i s "high"and l e f t D e r i s "flat"and cenDer i s "steepPositive" )

thenTrendChange t c = new TrendChange ( $f ,

"step_up" ,d r o o l s . getConsequenceDegree ( ) ) ;

d e l i v e r E v e n t ( t c ) ;end


Hybrid Modules

Rule examples : Prediction

r u l e "Predict NO3"// c a s c a d e d h y b r i d

when$s : Sample ( $ i d : i d )$n : Neura lNet ( $out : output == "no3" )

theni n s e r t (new Value ( $ id , $out , $n . e v a l ( $s ) ) ;

end

r u l e "Validate"// f u n c t i o n−embedding h y b r i d

when$s : Sample ( $ i d : i d )$v : Value ( i d == $id , t y p e == "no3" )e x i s t s SOM Neuron ( t h i s s i m i l a r $s )

theni n s e r t (new E s t i m a t e ( $s , $v ,

d r o o l s . getConsequenceDegree ( ) ) ;end


Hybrid Modules

Rule examples : Policy

r u l e "EoD"// time−aware r u l e w i t h f u z z y−>p r o b a b i l i t y mappingi m p l i c a t i o n @[ k ind=”fuz2prob ” , p r i o r = ” i d e n t i t y ” ]

when$ f : CurrPhase ( name == "anox" )

and @[ k ind=”Luk” ] // c o n f i g a t t r i b u t e s$m : TrendChange ( s i g n a l == "pH" , t y p e == "max" )

and$k : TrendChange ( s i g n a l == "orp" , t y p e == "knee_down" )

andTrendChange ( t h i s == $m, t h i s ov e r l a p s $k )

thenEndOfReact eod = new EndOfReact ( "denitro" ,

d r o o l s . getConsequenceDegree ( ) ;d e l i v e r E v e n t ( eod ) ;

end


Hybrid Modules

Rule examples : Policy II

r u l e "Switch"// i n j e c t i n g r u l e

when$ f : CurrPhase ( name == "anox" )$eod : EndOfReact ( r e a c t i o n == "denitro" )

thenSwitch sw = new Switch (+1); // n e x t phasei n j e c t (new Tuple ( sw ) , "holds" ) ;

i n s e r t ( sw ) ;end


Hybrid Modules

Rule examples : Policy III

r u l e "Safe_Switch"// m u l t i−premise , g r a d u a l r u l e

when$s : Swi tch ( t h i s holds )and @[ kind=”prod” ]/∗ p ( S )∗ r (−S ) > p(−S )∗ r ( S ) ∗/imp l i e s (

Switch ( t h i s == @[ c r i s p ] $s ,t h i s neg holds and t h i s neg cost "falseN" )

Switch ( t h i s == @[ c r i s p ] $s ,t h i s holds and t h i s neg cost "falseP" )

)then

s c h e d u l e ( $s ,TMAX ∗ (1− d r o o l s . getConsequenceDegree ( ) ) ) ;

end


Outline

1 PhD Curriculum






DROOLS

JBoss Drools

Business Rule Management System

Production Rules : RETE-based

Open Source

Modular

Expert : Object-Oriented Rule engineFlow : Support for Workflows

Fusion : Support for EventsGuvnor : Remote rule Repository


Inference under Uncertainty

Outline

1 PhD Curriculum







Generalized Inference

〈P(x),P(X )→C (Y )〉C (y)

Classic Modus Ponens

Premise and Implication entail Consequence




〈Φ(...,Aj(x)/εj ,... ),P(X )→C (Y )〉C (y)

Premise

Atomic constraints areevaluatedGeneral, pluggableEvaluatorsA Degree is returned




〈Φ(...,Aj(x)/εj ,... )/εP ,P(X )→C (Y )〉C (y)

Premise

Atomic constraints areevaluatedGeneral, pluggableEvaluatorsA Degree is returned

Premise

Atoms are aggregated informulasusing generalized logicConnectivesevaluated by Operators




〈P(x)/εP , →(X ,Y )/ε→〉C (y)

Implication

Implication has a Degreeoften given a priori




〈P(x)/εP , →(X ,Y )/ε→〉C (y)/εC

Implication

Implication has a Degreeoften given a priori

Modus Ponens

MP computes the Degreeof the Consequence




〈P1,→1〉C1/εC1

,...,〈Pn,→n〉Cn/εCn

C (y)/εC

Merging multiple sources

Multiple premises for the same conclusionSolve conflictsHandle missing values


Rule Language and Engine

Outline

1 PhD Curriculum







Language extensions

Custom Evaluators

Before : limited support for boolean functions

After : integration with external modules

Adapter interfacesDegrees carry more information

Formulas

Before : conjunction, quantifiers, NaF

After : support for all standard connectives

Configuration Attributes

Before : parameters passed to custom evaluators only

After : granular configuration

Compile-time : choose implementationRun-time : configure propagation behaviour



Language extensions : Example

r u l e "Rule"// custom : i m p l i c a t i o n s and MPi m p l i c a t i o n @[ degree =”0.75” ]d e d u c t i o n @[ kind=”min” ]

when$o1 : Type ( $ f 1 : f i e l d 1

/∗ custom : e x t e r n a l e v a l u a t o r ∗/== @[ id=”i1 ” , kind=”externa l ” , params=”...” ]

"val" )or @[ kind=”max” ] // custom : o p e r a t o r s

$o2 : AnotherType (f i e l d 3 == 0ˆˆ // custom : o p e r a t o r sf i e l d 3 == @[ c r i s p ] $ f 1 ) // custom : b e h a v i o u r

then/∗ consequence d e g r e e ∗/. . . = d r o o l s . getConsequenceDegree ( ) ;



Engine Extension

Global additions

Evaluations and Degrees

Centralized Factory

Builds and converts degrees and operators

Improved RETE Network

Additional Nodes

Enabled NodeOperator Nodes

Including Implication and Modus Ponens

Augmented Alpha and Beta nodes



Extended Engine : Example

#

thisenabled

#0

$o1

Type

#1

$f1

field1== val

#2

⊗1

#3

⊗3

#4

#

thisenabled

#5

$o2

AnotherType

#6

field3== 0

#7

#

6=2

#9

⊗1

#10

⊗3

#11

#

field3== $f1

#8

∨2#12

⊗1

#13

→0#14

⇒2#15 #Rule

1


Outline

1 PhD Curriculum






Results and Future Works

Results

Design and development of a configurable RETE engine

Added support for different non-boolean logics

Development of strongly coupled hybrid systems

Technology transfer : application of modern technologies toWWTPs

Future Developments

Release the engine as an official module (“Drools Chance”)

Integration of Rule-Based Systems and Ontologies

Application to different domains



Results

Design and development of a configurable RETE engine

Added support for different non-boolean logics

Development of strongly coupled hybrid systems

Technology transfer : application of modern technologies toWWTPs

Future Developments

Release the engine as an official module (“Drools Chance”)

Integration of Rule-Based Systems and Ontologies

Application to different domains

PhD CurriculumIntroductionCase Study : Water TreatmentState of the Art

Hybrid ArchitecturesProposed ArchitectureHybrid Modules

Rule EngineInference under UncertaintyRule Language and Engine


Integration of Symbolic and Connectionist AI techniques fine...Engineering Systems, 11th...

Documents

Transcript of Integration of Symbolic and Connectionist AI techniques fine...Engineering Systems, 11th...