Lecture 09 Value of Information - College of …Lecture Outline 1 Introduction to Value of...
Transcript of Lecture 09 Value of Information - College of …Lecture Outline 1 Introduction to Value of...
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Lecture 09Value of Information
Jitesh H. Panchal
ME 597: Decision Making for Engineering Systems Design
Design Engineering Lab @ Purdue (DELP)School of Mechanical Engineering
Purdue University, West Lafayette, INhttp://engineering.purdue.edu/delp
October 07, 2014c©Jitesh H. Panchal Lecture 09 1 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Lecture Outline
1 Introduction to Value of Information
2 Expected Value of Perfect Information
3 Expected Value of Imperfect Information
4 Application of Value of Information in Engineering Design
Clemen, R. T. (1996). Making Hard Decisions: An Introduction to Decision Analysis. Belmont, CA, Wadsworth Publishing Company.Chapter 12.
c©Jitesh H. Panchal Lecture 09 2 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Illustrative Example - Investing in the Stock Market
Investor has three choices:
High-risk stock
Low-risk stock
Savings account
Stocks have a brokerage feeof $200
Payoff on stocks depends onwhether the market goes up,remains same, or goes down.
Savings account
Up (0.5)
Same (0.3)
Down (0.2)
High-Risk
Stock
Low-Risk
Stock
Up (0.5)
Same (0.3)
Down (0.2)
1500
100
-1000
1000
200
-100
500
Figure: 12.1 on Page 436 (Clemen)
c©Jitesh H. Panchal Lecture 09 3 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Basic Questions
What does it mean for an expert to provide perfect information?
How does probability relate to the idea of information?
What is an appropriate basis on which to evaluate information in adecision situation?
c©Jitesh H. Panchal Lecture 09 4 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
The notion of “Perfect Information”
An expert’s information is said to be perfect if it is always correct.
1 When state S will occur, the expert always says so. For example,P(Expert says “Market Up” | Market really Does Go Up) = 1
Since the probability must add to one,P(Expert says “Market will stay the same or fail” | Market really Does
Go Up) = 0
2 Also, the expert must never say that the state S will occur if any otherstate (S̄) will occur.
P(Expert says “Market will go up” | Market really will stay the Sameor Fail) = 0
c©Jitesh H. Panchal Lecture 09 5 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Flipping the Probabilities Using Bayes’ Theorem
P(AB) = P(A|B)P(B) = P(B|A)P(A)
P(B|A) =P(A|B)P(B)
P(A)
P(Market UP|Exp says UP) =P(Exp says UP|Market UP)P(Market UP)
P(Exp says UP)
c©Jitesh H. Panchal Lecture 09 6 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Flipping the Probabilities Using Bayes’ Theorem
P(Market UP|Exp says UP) =P(Exp says UP|Market UP)P(Market UP)
P(Exp says UP)
The denominator can be written as the sum of:
P(Exp says UP) =
P(Exp says UP|Market UP)P(Market UP)
+
P(Exp says UP|Market NOT UP)P(Market NOT UP)
We can check that
P(Market UP|Exp says UP) = 1
c©Jitesh H. Panchal Lecture 09 7 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
The Expected Value of Information – Key Concepts (1)
If the decision maker will make the same decision regardless of what the newinformation is, then the information has no value!
1 In the example, the best decision without the information is to invest.2 If the expert says that the market will go up, then the decision still
remains the same → no value.3 If the expert says that market will go down, then the decision would
change. So, information has value.4 This is ex-post value!
c©Jitesh H. Panchal Lecture 09 8 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
The Expected Value of Information – Key Concepts (2)
Need to define the value of information before hiring the expert (i.e., beforegetting the information).
1 Worst case scenario: Decision remains the same even after hiring theexpert ⇒ Zero value of information.
2 Better scenarios: Expected value increases ⇒ Positive value ofinformation
3 Best case scenario: Perfect Information (resolving all uncertainty; Experttells us exactly what will happen) ⇒ Maximum value of information (i.e.,Expected Value of Perfect Information)
c©Jitesh H. Panchal Lecture 09 9 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Expected Value of Perfect Information
An expert (clairvoyant) who could reveal exactly what the market would do.
Note: The uncertainty in the problem is resolved before the investmentdecision is made. Once the decision is made, the market must still go thoughits performance. It is simply that the investor knows exactly what theperformance will be.
c©Jitesh H. Panchal Lecture 09 10 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Expected Value of Perfect Information
Market
up (0.5)
Up (0.5)
Same (0.3)
Down (0.2)
High-Risk
Stock
Low-Risk
Stock
Up (0.5)
Same (0.3)
Down (0.2)
1500
100
-1000
1000
200
-100
500
(EMV = 540)
(EMV = 580)
High Risk Stock
Low Risk Stock
Savings Account
1500
1000
500
100
200
500
-1000
-100
500
Savings account
Market down
(0.2)
Market flat (0.3)
Consult
Clairvoyant
(EMV=1000)
High Risk Stock
Low Risk Stock
Savings Account
High Risk Stock
Low Risk Stock
Savings Account
Figure: 12.3 on Page 440 (Clemen)c©Jitesh H. Panchal Lecture 09 11 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Expected Value of Perfect Information
Ex-ante nature: The decision maker has not yet consulted the clairvoyant. Heis considering whether or not to consult!
There is 50% chance that the clairvoyant would say that the market will go up.
c©Jitesh H. Panchal Lecture 09 12 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Expected Value of “Imperfect” Information
Suppose that the expert’s track record shows that if the market actually willrise, he says:
“Up” 80% of the time
“Flat” 10% of the time
“Down” 10% of the time
True Market StateEconomist’s Prediction Up Flat Down“Up” 0.80 0.15 0.20“Flat” 0.10 0.70 0.20“Down” 0.10 0.15 0.60
1.00 1.00 1.00
c©Jitesh H. Panchal Lecture 09 13 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Decision Tree for the Investment Example
Economist says
“Market up” (?)
High-Risk Stock (EMV = 580)
Low-Risk Stock (EMV = 540)
High Risk Stock
Low Risk Stock
Savings Account
Savings account (EMV = 500)
Economist’s
forecast
Economist says
“Market flat” (?)
Economist says
“Market Down” (?) High Risk Stock
Low Risk Stock
Savings Account
High Risk Stock
Low Risk Stock
Savings Account
500
Down (?)
Up(?)Same(?)
1000
200
-100
Market Activity
Down (?)
Up(?)Same(?)
1500
100
-1000
Down (?)
Up(?)Same(?)
1500
100
-1000
Down (?)
Up(?)Same(?)
1000
200
-100
Down (?)
Up(?)Same(?)
1500
100
-1000
Down (?)
Up(?)Same(?)
1000
200
-100
500
500
Consult
Economist
Figure: 12.5 on Page 443 (Clemen)
c©Jitesh H. Panchal Lecture 09 14 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Using Bayes’ Theorem to Flip Probabilities
P(Market Up|Economist Says “Up”)
= P(Up|“Up”)
=P(“Up”|Up)P(Up)
P(“Up”|Up)P(Up) + P(“Up”|Flat)P(Flat) + P(“Up”|Down)P(Down)
=0.8(0.5)
0.8(0.5) + 0.15(0.3) + 0.2(0.2)
=0.4000.485
= 0.8247
c©Jitesh H. Panchal Lecture 09 15 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Flipping the Probability Tree
“Market up” (0.80)
Market up
(0.5)
Market
Flat (0.3)
Market
Down
(0.2)
Actual Market
Performance
Economist’s
Forecast
“Market Flat” (0.10)
“Market Down” (0.10)
“Market up” (0.15)
“Market Flat” (0.70)
“Market Down” (0.15)
“Market up” (0.20)
“Market Flat” (0.20)
“Market Down” (0.60)
Market up (?)
“Market
up” (?)
“Market
Flat” (?)
“Market
Down” (?)
Economist’s
Forecast
Actual Market
Performance
Market Flat (?)
Market Down (?)
Market up (?)
Market Flat (?)
Market Down (?)
Market up (?)
Market Flat (?)
Market Down (?)
Figure: 12.7 on Page 444 (Clemen)
c©Jitesh H. Panchal Lecture 09 16 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Posterior Probabilities For Market Trends
Posterior Probability forEconomist’s Prediction Market Up Market Flat Market Down“Up” 0.8247 0.0928 0.0825 1.0000“Flat” 0.1667 0.7000 0.1333 1.0000“Down” 0.2325 0.2093 0.5581 1.0000
c©Jitesh H. Panchal Lecture 09 17 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Completed Decision Tree
Economist says
“Market up”
(0.485)
High-Risk Stock (EMV = 580)
Low-Risk Stock (EMV = 540)
High Risk Stock
Low Risk Stock
Savings Account
Savings account (EMV = 500)
Economist’s
forecast
Economist says
“Market flat”
(0.300)
Economist says
“Market Down”
(0.215)
High Risk Stock
Low Risk Stock
Savings Account
High Risk Stock
Low Risk Stock
Savings Account
500
Down (0.0825)
Up(0.8247)Same(0.0928)
1000
200
-100
Down (0.0825)
Up(0.8247)Same(0.0928)
1500
100
-1000
Down (0.1333)
Up(0.1667)Same(0.7000)
1500
100
-1000
Down (0.1333)
Up (0.1667)Same(0.7000)
1000
200
-100
Down (0.5581)
Up(0.2325)Same(0.2093)
1500
100
-1000
Down (0.5581)
Up(0.2325)Same(0.2093)
1000
200
-100
500
500
Consult
Economist
(EMV=822)
EMV=1164
EMV=835
EMV=187
EMV=293
EMV=-188
EMV=219
Figure: 12.8 on Page 446 (Clemen)
c©Jitesh H. Panchal Lecture 09 18 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Application of Value of Information in Engineering Design
Design is a decision making process.
Design process = Network of decisions.
c©Jitesh H. Panchal Lecture 09 19 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Is Further Refinement Necessary? A Conceptual Example
How much refinement of a simulation
model is appropriate for design?
R
R LR L
T
Material Alternatives
Decision
Material 1
Simulation model predicts
material strength
Objective: Maximize
the pressure that the
vessel can withstand
Selected
Material
Alternative
Material 2
Should this model be refined further?
• Improved accuracy does not imply
improved usefulness in design !!!
• The appropriateness depends on
the overall design problem
(constraints, objectives, and
variable bounds)
Inferences
Material 1
Material 2Predicted Strength
Predicted Strength
Strength (MPa)
[150 175] MPa
[190 225] MPa
c©Jitesh H. Panchal Lecture 09 20 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Model Refinement Using Value-of-Information
c©Jitesh H. Panchal Lecture 09 21 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Application of Value of Information in Engineering Design
All models are wrong, some models are useful! [George Box]
Designers care about usefulness of a model. So, the question is:
How much refinement of a simulation model is appropriate for design?
c©Jitesh H. Panchal Lecture 09 22 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
Model Refinement Using Value-of-Information
Value of Information = Improvement in the outcome of a design decision dueto added information
Decision
Design
Alternatives
(e.g., options for
particle sizes)
a1a2a3…
an
Simulation Results for
System Behavior
(y)
(y, ai)
Designers’
Preferences
a0
+ Additional Information
from Model Refinement (d)
ad(yr, ad)Selected
Alternative
Build
& Test
Achieved
Utility
(yr, a0)
c©Jitesh H. Panchal Lecture 09 23 / 24
Introduction to Value of InformationExpected Value of Perfect Information
Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design
References
1 Clemen, R. T. (1996). Making Hard Decisions: An Introduction toDecision Analysis. Belmont, CA, Wadsworth Publishing Company.Chapter 12.
2 Panchal, J. H., Paredis, C. J. J., Allen, J. K., and Mistree, F., 2008, “AValue-of-Information Based Approach to Simulation Model Refinement,”Engineering Optimization, Vol. 40, No. 3, pp. 223 - 251.
3 Panchal, J.H., Paredis, C. J. J., Allen, J. K., and Mistree, F., 2009,“Managing Design-Process Complexity: A Value-of-Information BasedApproach for Scale and Decision Decoupling,” ASME Journal ofComputing and Information Sciences in Engineering, Vol. 9, No. 2,(021005)1-12.
c©Jitesh H. Panchal Lecture 09 24 / 24