Changes in Probabilistic Causal Models
description
Transcript of Changes in Probabilistic Causal Models
Changes in Probabilistic Causal Models
or: Do we stink at probability or what?
James Baker
PSY446
Participation Mostly NU undergrads Some NU grad students Early data collection error / internet preteens filtered
67 data records 53 finished samples
13 of type 0 14 of type 1 13 of type 2 13 of type 3
Gametype 0 (control)
A: Display "33%" Prob: .35 Price: $3 B: Display "66%" Prob: .68 Price: $6 C: Display "97%" Prob: .95 Price: $10 No changes
Responses to gametype 0 I can get 20 right if I always choose the most expensive... I didn't really know how the computer was operating. I just
figured that if the probability was high I would follow that suggestion.
I felt like 'B' was more than 66 percent, so after trying the other two, I stuck with B.
I chose to go with query B every time, because, on average, this would return 22 correct answers. Query A would do the same. And, I think, query C would return about 19, on average. Unless my math is wrong (which is very possible)...consistently choosing A or B is the best bet.
so i guess i got more 'correct' answers when i just played many times with low probablility of winning- but again- it's a little too late in the day for me to actually strategize.
Responses to gametype 0 I think the computer worked on the probability given,
but when you moved to a higher cost value, it decreased the likelihood that the probability given would work at a lower cost value. (Therefore, if you ask at $3, then move to $6, $3 will be less likely to work in subsequent rounds.) I mostly stayed with the $6 option, only moving to $10 when I wanted a correct answer and $3 when I wanted $6 to be more likely, which seemed to work.
I spent nearly all my money on the 97% question. i would stick with c until it stopped being correct,
then move to b for a while until it stopped working then go back to c, etc
Results-0
> 90% A’s: 3 > 75% B’s: 4 > 70% C’s: 4 30% < B and C < 60%: 2
Conclusion: Tendency to pick one favorite
Gametype 1
A: Display "30%" Prob: .32 Price: $3 B: Display "40%" Prob: .22 Price: $4 C: Display "50%" Prob: .52 Price: $5 Equally good choices B ramps from .22 to .92 Hypothesis: People will like C the best,
might detect something off about B.
Responses to gametype 1 If the computer was trying to pick up on my strategy, it would be
pretty easy to learn the strategy. I picked C every time. Although it was the most expensive option, it had the highest probability of a right answer. So I picked it each time.
i would stick with c until it stopped being correct, then move to b for a while until it stopped working then go to a, then start over
totaly random Honestly, I have no idea how the computer was operating. I think
asking the same thing, then changing to something less likely, then going back to the original query, makes it more likely to get the correct answer, so I continued to follow that strategy.
Probablility shouldn't necessarily play into what results occur because it is still a tossup and 40% still seemed to produce positive results about 50 % of the time..
Responses to gametype 1
I figured it was random, so I always picked A which was had the highest return rate.
I didn't make any changes; selecting C everytime seemed to be working.
It seems that 50% of a chance would be better odds despite the cost.
i figured i would last the longest if i kept choosing C The odds for this game were that I would get twenty
queries right, so I always went with the most expensive query because it was quicker.
Results-1
> 95% A’s: 2 > 70% B’s: 1 > 70% C’s: 6 Mixed: 4 Among mixed: 57 A, 61 B, 77 C
Conclusion: We like round numbers?
Gametype 2
A: Display "35%" Prob: .37 Price: $3 B: Display "65%" Prob: .67 Price: $5 C: Display "95%" Prob: 1.0 Price: $7 C starts as best choice C ramps from 1.0 to .3 Hypothesis: People will start on C, then
change to B/A when C gets worse.
Responses to gametype 2 Same logic as last time (divided $200 by the cost/query and then
multiplied by the odds of that query returning a "correct answer"). I predicted 27 correct answers for choice C, which would have been the most.
considering that the correct answer changed (I haven't figured out the pattern yet) when I stopped getting right answers I would switch.
Option C maximizes expected profit in the long run. Hence, I believe in following it should return in the best outcome in the long-run.
I just tried the 95% to play it safe. i didn't change my strategy at all, i just kept going for what i
thought was the most consistent, c every time. stick with highest odds
Responses to gametype 2 I think the computer worked on the probability given, but when
you moved to a higher cost value, it decreased the likelihood that the probability given would work at a lower cost value. (Therefore, if you ask at $3, then move to $5, $3 will be less likely to work in subsequent rounds.) I mostly stayed with the $5 option, since it was a better than 50% chance and it cost an average amount.
just played C as long as I could (in writing, it seemed like the best deal)
The correct answer was not returned at anywhere near 95%, so I don't know how I'm supposed to guess which query to go with.
I stayed with the highest probability option It seemed that the computer started returning wrong answers for
C more than 5% of the time, even when I would vary my answers.
Results-2
>90% B: 1 >75% C: 10 50 < C < 60: 2 The last two started with strings of C
Conclusion: Very slow to change mental model from samples
Gametype 3
A: Display "30%" Prob: .10 Price: $1 B: Display "60%" Prob: .35 Price: $6 C: Display "90%" Prob: .70 Price: $10 A appears to be best, but isn’t A ramps from .1 to .6 Hypothesis: People will start on A, discover
it’s bad, switch to C/B, move around a bit, perhaps find A again when it’s good
Responses to gametype 3 I stuck mostly with the cheapest one this time becuse it only cost
$1. I was able to score pretty high by being able to ask such cheap questions.
When the $1 started winning I just kept going with those. I assumed that the probabilities set out in the beginning of the task were going to be followed so I just tried to mix up the type of answer I chose and was more patient with A and B.
Same strategy as always. Loyally played the odds. I only predicted 60 correct answers...but I guess I got rewarded for hanging in there. Vegas here I come!
i chose a alot since it was so cheap i thought that there might have been different questions
eventually- since there weren't i decided to stick with the most likely 'correct' answer from the computer.
Responses to gametype 3
the cheapest query gave the best odds by about a factor of 3. I got even more right than the odds predicted.
tried A, but it failed 10 in a row; so then I tested C, and I was convined it was legit, so I plkayed it utnil the end, when I could only afford B
I tried the riskier ones when I got low on money. seemed as though ninety percent wasn't actually
appropriate.. it seemed more like 66%
Results
> 80% A: 5 > 90% B: 1 > 70% C: 0 Mixed: 7 Of A’s, 2 were ALL As and 3 had other
selections in the middle section.
Conclusion: More churn with bad outcomes, again slow to change model of A
Aggregate Results
Tendency to pick one favorite We like round numbers? Very slow to change mental model from
samples More churn with bad outcomes, again slow to
change model of A Disconnect between comments and
unconscious trends?
Confounding Factors
Display of percentages reinforcing static mental model
Fin
James Baker
Backup Gametype 0 20 choices: 0 A's, 0 B's, 20 C's. .00% A, .00% B, 1.00% C. 55 choices: 50 A's, 0 B's, 5 C's. .91% A, .00% B, .09% C. 20 choices: 0 A's, 0 B's, 20 C's. .00% A, .00% B, 1.00% C. 32 choices: 3 A's, 25 B's, 4 C's. .09% A, .78% B, .12% C. 30 choices: 6 A's, 15 B's, 9 C's. .20% A, .50% B, .30% C. 24 choices: 4 A's, 3 B's, 17 C's. .17% A, .12% B, .71% C. 35 choices: 5 A's, 29 B's, 1 C's. .14% A, .83% B, .03% C. 20 choices: 0 A's, 0 B's, 20 C's. .00% A, .00% B, 1.00% C. 33 choices: 0 A's, 33 B's, 0 C's. .00% A, 1.00% B, .00% C. 27 choices: 1 A's, 16 B's, 10 C's. .04% A, .59% B, .37% C. 60 choices: 55 A's, 4 B's, 1 C's. .92% A, .07% B, .02% C. 33 choices: 0 A's, 33 B's, 0 C's. .00% A, 1.00% B, .00% C. 66 choices: 66 A's, 0 B's, 0 C's. 1.00% A, .00% B, .00% C.
Backup Gametype 1 42 choices: 3 A's, 4 B's, 35 C's. .07% A, .10% B, .83% C. 40 choices: 0 A's, 0 B's, 40 C's. .00% A, .00% B, 1.00% C. 47 choices: 7 A's, 21 B's, 19 C's. .15% A, .45% B, .40% C. 51 choices: 20 A's, 15 B's, 16 C's. .39% A, .29% B, .31% C. 49 choices: 5 A's, 35 B's, 9 C's. .10% A, .71% B, .18% C. 47 choices: 16 A's, 3 B's, 28 C's. .34% A, .06% B, .60% C. 50 choices: 14 A's, 22 B's, 14 C's. .28% A, .44% B, .28% C. 65 choices: 63 A's, 0 B's, 2 C's. .97% A, .00% B, .03% C. 40 choices: 0 A's, 0 B's, 40 C's. .00% A, .00% B, 1.00% C. 40 choices: 0 A's, 0 B's, 40 C's. .00% A, .00% B, 1.00% C. 40 choices: 0 A's, 0 B's, 40 C's. .00% A, .00% B, 1.00% C. 41 choices: 2 A's, 2 B's, 37 C's. .05% A, .05% B, .90% C. 40 choices: 0 A's, 0 B's, 40 C's. .00% A, .00% B, 1.00% C. 66 choices: 66 A's, 0 B's, 0 C's. 1.00% A, .00% B, .00% C.
Backup Gametype 2 29 choices: 1 A's, 0 B's, 28 C's. .03% A, .00% B, .97% C. 35 choices: 7 A's, 9 B's, 19 C's. .20% A, .26% B, .54% C. 29 choices: 1 A's, 0 B's, 28 C's. .03% A, .00% B, .97% C. 29 choices: 0 A's, 2 B's, 27 C's. .00% A, .07% B, .93% C. 29 choices: 1 A's, 0 B's, 28 C's. .03% A, .00% B, .97% C. 29 choices: 1 A's, 0 B's, 28 C's. .03% A, .00% B, .97% C. 29 choices: 1 A's, 0 B's, 28 C's. .03% A, .00% B, .97% C. 31 choices: 2 A's, 5 B's, 24 C's. .06% A, .16% B, .77% C. 40 choices: 2 A's, 37 B's, 1 C's. .05% A, .93% B, .03% C. 29 choices: 1 A's, 0 B's, 28 C's. .03% A, .00% B, .97% C. 29 choices: 1 A's, 0 B's, 28 C's. .03% A, .00% B, .97% C. 29 choices: 1 A's, 0 B's, 28 C's. .03% A, .00% B, .97% C. 33 choices: 2 A's, 12 B's, 19 C's. .06% A, .36% B, .58% C.
Backup Gametype 3 115 choices: 98 A's, 17 B's, 0 C's. .85% A, .15% B, .00% C. 38 choices: 12 A's, 18 B's, 8 C's. .32% A, .47% B, .21% C. 200 choices: 200 A's, 0 B's, 0 C's. 1.0% A, .00% B, .00% C. 35 choices: 2 A's, 33 B's, 0 C's. .06% A, .94% B, .00% C. 119 choices: 106 A's, 9 B's, 4 C's. .89% A, .08% B, .03% C. 84 choices: 68 A's, 7 B's, 9 C's. .81% A, .08% B, .11% C. 39 choices: 18 A's, 7 B's, 14 C's. .46% A, .18% B, .36% C. 32 choices: 4 A's, 21 B's, 7 C's. .12% A, .66% B, .22% C. 44 choices: 24 A's, 6 B's, 14 C's. .55% A, .14% B, .32% C. 200 choices: 200 A's, 0 B's, 0 C's. 1.0% A, .00% B, .00% C. 33 choices: 10 A's, 10 B's, 13 C's. .30% A, .30% B, .39% C. 28 choices: 8 A's, 2 B's, 18 C's. .29% A, .07% B, .64% C. 26 choices: 4 A's, 6 B's, 16 C's. .15% A, .23% B, .62% C.
Backup Gametype 0 C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C, A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,C,C,C,C,C, C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C, B,B,B,A,A,A,B,B,B,C,C,C,C,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B, C,C,C,C,C,C,B,B,B,B,B,B,B,B,B,B,B,A,A,A,A,A,B,B,B,B,C,C,C,A, C,C,C,C,C,C,C,C,C,C,B,B,C,C,C,C,C,C,C,A,A,A,B,A, B,A,A,A,B,B,B,B,B,B,B,B,B,B,B,C,A,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,A, C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C, B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B, B,B,C,C,C,C,B,B,B,B,B,B,B,B,B,B,B,B,C,C,C,C,B,B,C,C,A, A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,B,A,A,A,A,A,A,A,A,A,B,C,B,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,B,A, B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B, A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
Backup Gametype 1 A,B,C,C,C,C,B,A,B,C,A,C,B,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C, C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C, C,C,C,C,B,C,A,C,C,B,B,A,C,C,C,B,A,A,A,C,C,C,C,C,C,C,B,B,B,A,C,B,A,C,B,B,B,B,B,B,B,B,B,B,B,
B,B, B,A,A,A,A,A,A,A,A,A,A,A,A,A,B,B,B,B,B,B,B,C,C,C,C,C,C,C,C,A,A,A,A,A,B,B,B,B,B,C,C,C,C,C,C,C
,B,B,A,A,C, A,A,A,B,B,B,B,C,C,C,C,C,C,C,C,C,A,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,
B,B,A A,A,A,A,A,C,C,C,C,C,C,B,B,A,A,A,A,A,A,A,A,A,A,A,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,
C,B, B,B,B,B,B,B,B,B,B,B,C,C,C,C,C,C,B,B,B,A,A,A,B,B,B,B,C,C,C,C,A,A,A,A,A,A,A,A,A,C,C,B,B,B,B,B
,A,C,C,A, C,C,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A, C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C, C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C, C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C, C,A,A,B,B,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C, C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C, A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
Backup Gametype 2 C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A, C,C,C,C,C,C,C,A,A,C,A,C,C,C,C,C,B,B,B,B,B,A,A,A,A,C,C,C,C,C,C,B,B,B,B
, C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A, C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,B,C,C,C,C,C,C,C,C,B, C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A, C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A, C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A, C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,B,B,A,B,B,A,C,C,B,C,C, B,B,B,B,B,B,B,B,B,B,B,A,B,B,B,B,B,B,B,C,B,B,B,A,B,B,B,B,B,B,B,B,B,B,B,B
,B,B,B,B, C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A, C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A, C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A, C,C,C,A,C,C,C,C,C,B,B,B,C,C,C,B,B,B,B,B,B,C,C,C,B,C,C,C,B,C,C,B,A,
Backup Gametype 3 A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,B,B,
B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
B,C,A,A,A,B,B,B,A,B,B,C,C,C,C,B,B,B,B,B,A,A,A,A,A,A,A,A,B,B,B,B,B,B,B,C,C,C, A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,A,A, B,B,B,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,C,C,C,C,B,B,B,B,B,B,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,C,C,C,C,A,A,B,B,A,A,A,A,C,C,A,B,B,B,A,C,A,A,A,A,A,A,C,B,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,C,B,A,
C,B,B,A,C,C,C,C,C,B,A,C,C,C,C,C,C,B,B,B,A,C,C,A,A,A,A,A,A,A,A,A,B,A,A,A,A,A,A, C,B,C,C,C,C,C,C,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,A,A,A,A, B,B,C,C,C,C,C,B,B,A,A,A,A,A,A,A,A,A,A,A,A,A,A,C,C,C,C,C,C,C,C,C,B,B,A,A,A,A,A,A,A,A,A,A, A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,C,C,C,B,B,B,B,B,B,B,C,C,C,C,C,C,C,C,C,C,B,B,B, C,B,B,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A,A,A,A,A,A,A,A, C,B,B,C,C,C,B,C,A,C,B,C,A,C,C,B,C,C,C,C,C,C,C,B,A,A