Post on 12-May-2015
description
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
Bayesia Expert Knowledge Elicitation Environment - BEKEE
An innovative Brainstorming Tool
Dr. Lionel JOUFFE
February 2012
1
1
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission2
All models are wrong; the practical question is how wrong do they have to be to not be useful (Box&Draper 87)
MODELING BY BRAINSTORMINGMODELING BY BRAINSTORMING
2
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission3
Why?
There is a clear need for Decision Support Systems
Every Decision Maker is faced to complex decisions
Human Beings are not so good at taking rational
decision under uncertainty
3
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission4
How?
Explicit Knowledge
Tacit Knowledge
Data is not always available for automatically learning a Decision Support
System with Data Mining algorithms
But experts have gathered invaluable Tacit Knowledge through
their Experience
We need to convert this Tacit Knowledge into Explicit
Knowledge and use it for building models
4
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission5
What?
3,6
3,65
3,7
3,75
3,8
3,85
3,9
3,95
A priori Flowery Feminine Original Tenacious Fruity
Bayesian Belief Networks (BBNs) are ideal models for Expert Knowledge Modeling
Graphical Representation Powerful Probabilistic Engines
What-if scenarios
Drivers analysis
5
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission6
BBNs are made of Two Distinct Parts
Qualitative part: the Structure
Directed Acyclic Graph (DAG), i.e. no directed loop
Nodes represent the variables
Each node has a set of exclusive states (e.g.: Poor, Good)
Arcs represent the direct probabilistic relationships between the variables (possibly causal)
6
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission7
BBNs are made of Two Distinct Parts
Quantitative part: the Parameters
Probability tables are associated to each node
MARGINAL PROBABILITY DISTRIBUTIONHalf of the products are of Good quality
40% of the Brand Images are Poor
The size of the Conditional Probability Tables grows
exponentially with respect to the number of Parents
CONDITIONAL PROBABILITY DISTRIBUTIONThere are 60% of chance that the Perceived
Quality is Good for Poor Quality products with Good Brand Image
7
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission8
BBNs are Powerful Observational Inference Engines ...
We get some evidence on the states of a subset of variables: Hard positive and negative evidence, Likelihood, Probability distributions, Mean values
We take these findings into account in a rigorous way to update our belief on the states of all the other variables
Probability distributions on their values
Multi-Directional Inference (Simulation and/or Diagnosis)
The evidence on Perceived Quality (a new
probability distribution) allows to update the probability distribution of
Brand Image (Diagnosis) and Satisfaction (Simulation)
Prior Distribution Posterior Distribution
8
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission9
We DO some state modification on a subset of variables: Hard positive and negative actions, Likelihood, Probability distributions, Mean values
We take these actions into account in a rigorous way to update our belief on the states of all the descendant variables
Simulation of the effects of these actions
Probability distributions on their values
Simulating a new population made of 85% of Good
Perceived Quality products rather than focusing on a sub-population made of
such products
Prior Distribution Posterior Distribution
... and Powerful Causal Inference Engines
9
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission10
BBN Modeling by Brainstorming
Brain Storming Sessions with this group of Experts to manually build the BBN, conceptual dimension per conceptual dimension
Clear definition of the BBN’s objective(s) (e.g.: Improvement of the Product/Service Quality,
improvement of the Purchase Intent, improvement of the Company’s performance, ...)
Identification of the conceptual dimensions that are linked to these objectives
(e.g.: Human resources, Management, Production, Marketing, ...)
Definition of the group of experts that will fully cover all the dimensions (and the different geographical zones),
with a small redundancy for allowing fruitful expert debates
10
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The StructureThe Qualitative Part
11
One sub-network per Conceptual Dimension
11
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission12
BAYESIA ExpertKnowledge Elicitation Environment
Interactive
Batch
12
Each expert gives his/her belief on the probability distributions
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The ParametersThe Quantitative Part
13
Probabilities do not have to be exact to be useful
BIASES
Cognitive (Plausibility
, Control, A
vailability,
Anchoring)
Emotional (Mood, Motivation)Group (Anchoring, Herding)
Facilitator (can be biased toward charismatic experts or toward the last expressed opinion)
☛ Bayesia Expert Knowledge Elicitation Environment for reducing these biases, improving traceability, gathering all the useful
knowledge, ....
13
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission14
Interactive Sessions
14
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission15
Batch Sessions
15
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Expert Management
* Available on subscription only16
The Expert Editor allows defining:The Expert’s name, its Credibility (that will be use globally during
the consensus computation), her/his Picture, a Comment to describe her/his area of expertise. The last field contains the number of assessments realized by the expert on the current
network
Group of experts can be Imported and Exported
Communication with the BEKEE web server*
Allows generating a Bayesian network by using the assessments of the selected experts only
16
Posting a Question to the Server
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission17
Selecting a cell in the probability table activates the Assessment
button for assessing the question corresponding to the selected line,
i.e. what is the marginal probability distribution of Mobility over the 3
defined states?
The Assessment Editor allows the Facilitator manually adding,
deleting and modifying Experts’ Assessments.
The Post Assessment button is used by the Facilitator to send the question to the BayesiaLab’s secured website for
an online assessment
17
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
https://www.bayesialab.com/expertise2/
18
18
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Web Tool
19
19
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission20
Interactive Session
Waiting for a question send by the Facilitator
Pressing Play allows participating to the interactive session
Click the Lock to fix that probability
Confidence level of the expert used for weighting the assessment
Comment field for explaining,detailing the assessment
20
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission21
Interactive SessionNode with Parents
The context variables in the BBN
21
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Facilitator’s tool
22
Pressing OK makes BayesiaLab harvesting the assessments
Once the Expert validates her/his assessment, this assessment is sent to the BayesiaLab’s
server and the Facilitator’s listener is automatically updated
This listener allows following the status of the Experts’
assessments
22
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Facilitator’s tool
Sorting the assessments by state probabilities can be used for:- detecting Experts’ misunderstanding
- Knowledge sharing, especially by making the 2 “extremes” Experts debate
If some useful knowledge comes out from the debate, the Facilitator can post again the question for new Expert Assessments. Each Expert will then be allowed to update her/his assessment online (each Experts’ webpage is initialized with the
information she/he set in the previous round)
23
The content of this editor is sortable by each column just by clicking on the corresponding
header.It is sorted here in the ascending order on the
probabilities assessed for the state Weak
23
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Consensus
24
Once the assessments validated, a Mathematical consensus is computed by using the Experts’
credibility and their assessment’s confidence. This automatic consensus can be manually modified by the Facilitator to set a Behavioral consensus, i.e.
one issued after a fruitful expert debate
Hovering over this icon returns the minimum and the maximum assessments, and the
number of assessments
A small icon is associated to each probability for graphically
representing the consensus degree. That icon goes from full transparency, when all
the votes are identical, to no transparency at all, when the assessment range is 1 (one expert set
0% and another one set 100%)
A Consensus icon is also associated to the nodes for indicating the global consensus over all the
distributions. The darker the icon is, the lower the global consensus is
24
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Consensus
This information panel contains:- the number of rows (probability distributions) that have Experts assessments- the total number of assessments that have been set in the probability table
- the number of Experts that have assessed at least one probability distribution in the table - a measure of the global disagreement that takes into account the deviations from the
mathematical consensus- the maximum disagreement corresponding to the greatest difference between two
assessments in the probability table
25
Pressing the “I” key while hovering over the expert icon allows displaying the
information panel below
25
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Assessment Report
26
This report first gives information on the Experts, then returns a sorted list of the nodes wrt the
global disagreements, and another one wrt the maximal disagreements.
Finally, for each node, a summary contains all the global information on the assessments of the
(Conditional) Probability Table
Right clicking on the Expert Icon in the lower left corner of the Graph
window allows generating an HTML report
26
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Graph Report
27
The information given by the Assessment and Graph reports is useful for the Model Verification. High divergences can be due to state inversion, fuzzy definitions of the variables and/or their
states, different contexts
The Graph report allows generating HTML Conditional Probability Tables. These tables comes with the consensual probability distributions and the
maximum divergences.
From white (0) to blue (100) for the probabilities
Colors are associated to each cell
From green (0) to red (50) for the divergences
27
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Batch Session
28
In-person meetings are essential for building the qualitative part of the models.
Probability elicitation is time consuming and that quantitative part can be too long to allow the interactive elicitation of all the parameters during the meetings.
Batch sessions allow then each expert to remotely:
Complete the parameter elicitation process
Verify the assessed probabilities
The Facilitator can select the nodes for which the probability distributions
have to be assessed and/or verified
Warning are generated for the distributions that are greater than
30% threshold
28
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Web Tool
29
This expert has assessed 3 distributions out of 12
The Play button allows participating to the batch
session
The pie chart represents that completion rate
Nodes to assess or verify
29
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Web Tool
30
30
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Exportation of a Bayesian Network per Expert
31
This exportation tool allows the creation of one Bayesian Belief Network per Expert.
The parameters (probabilities) are those assessed by the Expert. For probabilities not assessed by the Expert, the model is based on the consensual probabilities, either the mathematical one, or
the behavioral one entered by the Facilitator
31
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Exportation of the Probability Assessments
32
Generation of a CSV file with all the assessed probabilities, one line per cell/
probability
Context in terms of states Assessed Node
32
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Exportation of the Expert Assessments
33
Generation of a CSV file with all the assessments of the Experts, one
line per cell/probability.
1/number of states of the assessed variable
33
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Analysis of the Expert Assessments
34
The Expert Assessment file can be analyzed with the unsupervised learning algorithms of BayesiaLab for finding the direct probabilistic relationships that hold
between the Experts’ assessments
Each node represents the discretized probabilities assessed by the Expert
34
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Automatic Segmentation of the Experts
Based on the obtained Expert Segments, one Bayesian network per segment can be generated (by using the Expert Editor). This can be useful for analyzing the sensibility of the model, but also to get specific networks (depending on the geographical localization
for example)
35
Based on the obtained network, Experts can be clustered into homogeneous groups by using the BayesiaLab’s Variable Clustering algorithm
Dendrogram corresponding to that segmentation Each color corresponds to a
cluster.
The real experts behind those anonymized experts have indeed 3 different profiles (functionally and
geographically)
35
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Parameter Sensibility Analysis
36
The Assessment Sensitivity Analysis tool allows measuring
the uncertainty associated to the consensus
Generation of a set of networks by randomly selecting Experts’ assessments
Measurement of the uncertainty associated to each probability distribution
The probability of Strong goes from 30% to 86%
36
©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Plan
Modeling by Brainstorming
Bayesia ExpertKnowledge Elicitation Environment
Contact
37
6 rue Léonard de Vinci BP0119
53001 LAVAL CedexFRANCE
Dr. Lionel JOUFFEPresident / CEO
Tel.: +33(0)243 49 75 58Skype: +33(0)970 44 64 28Mobile: +33(0)607 25 70 05Fax: +33(0)243 49 75 83
37