SOME PROBLEMS RELATED TO ACCUMULATION OF KNOWLEDGE IN CONTEMPORARY SCIENTIFIC PRACTICE....

SCIENCE AND KNOWLEDGE MANAGEMENT:

SOME PROBLEMS RELATED TO ACCUMULATION OF KNOWLEDGE IN CONTEMPORARY SCIENTIFIC PRACTICE

Goran S. Milovanović

PhD CandidateDepartment of Psychology

Faculty of PhilosophyUniversity of Belgrade

HUMBOLDT-KOLLEGWISSENSCHAFT UND BILDUNG IM WANDEL/ SCIENCE AND EDUCATION IN TRANSITION

Belgrade, October 28-30, 2010.

1



THE ACCUMULATION OFKNOWLEDGE

2



THE SCIENTIFIC PRODUCTION:POWER LAW FOR CITATIONS

3

1 or 2 references (~30%)

Less than 10% receives100 or more references

Note: data set is for ilustrative purposes only. Source: Google Scholar, 17. October 2010.

Citations

Freq

uenc

y



SCIENCE AND KNOWLEDGE MANAGEMENT:“AN ENGINEERING APPROACH” TO SCIENTIFIC KNOWLEDGE

How does this affect our “perception of science”: are we really able to manage all this information effectively?

HOW MANY RESEARCH PAPERS ARE YOU ABLE TO READ IN THE COURSE OF ONE WORKING WEEK?

HOW MANY OF THESE PAPERS WILL YOU CONSIDER RELEVANT?

HOW MANY WILL YOU CITE?

4



INCOMEASUREABLEMETHODOLOGIES AS SOURCES OF

INCREASED PRODUCTION

5



INCOMEASUREABLE METHODOLOGIES AND EMPIRICAL GROUNDING

6

T h e o r y A

C1A

Theoretical Concepts of A:

C2A C3A

Empirical scopes of the concepts of the Theory A

ES(C1A) ES(C2A) ES(C3A)

Note: the illustration presents a rude simplification, obviously.




7

C1A

Empirical scope of C1A

ES(C1A)

e1 e2 e3 e4 e5 e6 e7 e8

m1 m2 m8. . . Methods used to empiricallyground C1A in ES(C1A)

Empirical phenomena:ES(C1A) is the manifestationset of C1A.

Concept




8

C1A

ES(C1A)

e1 e2 e3

m1m2m3

C1A

ES(C1A)

e1 e2 e3

mAmBmC

Your Lab My Lab

PROBLEM!

When we communicate on C1A,do we report on the same

phenomenon?




9

IT HAPPENS ALL THE TIME.



CASE STUDY:MODEL SELECTION INCHOICE UNDER RISK

10



MODELING CHOICE UNDER RISK

11

Ticket A Ticket B

win EUR 50with 25%

win EUR 25with 75%

win EUR 500with 12%

lose EUR 50with 88%

What do you prefer: A or B?




NORMATIVE THEORY (axiomatized, von Neumann & Morgnestern, 1948, following Bernoulli’s early analysis of choice under risk from 1738).

PROBLEM: Normative theory does not predict all choices correctly – there are systematic behavioral violations from normative predictions; anomalies of rational choice).

Consequence: the development of ...

BEHAVIORAL THEORIES (many different models of choice under risk; most famous and most commonly used is Tversky and Kahneman’s Prospect Theory, Kahneman & Tversky, 1979, Tversky & Kahneman, 1992).

12




Two methods are prevalent in model selection in this field:

DETERMINATION OF CERTAINTY EQUIVALENTS (CE) OF RISKY GAMBLES

• Ask a participant to determine a certain amount of value that he or she would accept in exchange for the offered risky ticket.

13

win EUR 50with 25%

win EUR 25with 75%

EUR 30

Accept




Two methods are prevalent in model selection in this field:

14

Ticket A Ticket B

win EUR 50with 25%

win EUR 25with 75%

win EUR 500with 12%

lose EUR 50with 88%

THEORIES OF CHOICE UNDER RISK MUST BE ABLE TO PREDICT BOTH CE AND CHOICE DATA

CHOICE EXPERIMENTS




Model selection in this field: direct hypothesis testing based on experimental data is very rare.

1. Tversky & Kahneman (1992): CE dataset, estimation of their model (Cummulative Prospect Theory); no direct comparisons against Expected Utiliy (or any other) model.

This CE dataset exhibits properties that cannot be explained in the normative framework of Expected Utility.

15





2. Gonzales & Wu (1999): CE dataset, estimation of the Prospect Theory model; no direct comparisons with other models.

This CE dataset exhibits properties that cannot be explained in the normative framework of Expected Utility.

16





3. Hay & Orme (1994): Choice dataset, 100 pairwise choices; likelihood ratio tests -direct comparison of Expected Utility against various alternatives.

Conclusion: there is no superiority over Expected Utility. 39% of subjects are best fitted by Expected Utility model; in 61% of cases, other models fit better.

No single behavioral model is consistently superior to normative theory.

17





4. Harrison & Rutstrom (2006): Choice dataset, join estimation of Expected Utility and Prospect Theory, including modeling of the proportion of choices better explained by the former or the later theory.

Conclusion: there is no representative cognitive agent for a single theory; some choices are better explained by Expected Utility while others are better explained by Prospect Theory.

18



HOW MANY THEORIES OF CHOICE UNDER RISK?

Different methods (that were expected to provide convergent results) resulted in different conclusions.

In the meanwhile, the number of theories of choice under risk keeps on growing.How many explanations? Here we go:

Prospect Theory Yarri’s Dual ModelCummulative Prospect Theory Subjective Expected Utility TheoryDissapointment Aversion Theory Prospective Reference TheoryQuadratic Utility Regret Theory (w/wo independence)Weighted Utility Theory TAX ModelPerceived Relative Argument Model

There’s more on the market!

19



HOW MANY EMPIRICAL PHENOMENA TO EXPLAIN IN CHOICE UNDER RISK?

While the number of theories and models increases...

Birnbaum, 2008 (Psych.Review): There are 11 (eleven) new paradoxes that the normative theory (Expected Utility) cannot explain.

Guess what happens with the number of publications in this field...

20




WHAT IS HAPPENING IN THIS FIELD?

21



INCOMEASUREABLEMETHODOLOGIES

ANDHYPOTHESIS TESTING

22



INCOMEASURABLE METHODOLOGIES

The problem seems obvious: full hypothesis testing procedures are very rarely employed in this field.

CONSEQUENCES:

• The establishment of empirical phenomena that are not properly positioned in the context of other phenomena.

• The establishment of many models of choice under risk that sometimes work better and sometimes worse, probably as a function of what particular experimental design was used to test them.

23




All models of choice under risk are “naturally” expected to predict both CE and CHOICE data.

CE datasets most often exhibit properties that are not consistent with the normative theory.

CHOICE datasets most often support the normative theory or do not allow for empirical demarcation of models.

How is this possible?

24




A. INADEQUATE SAMPLING OF EXPERIMENTAL DESIGNS

Experimental designs in CE/CHOICE methodologies combine:

Levels of VALUE: 10, 20, 50, 100, 125, 150 EUR etc.Levels of PROBABILITY: .01 .05 .10 .25 .50 .75 .95 .99 etc.

1. The choice of levels on both factors in CE/CHOICE designs is arbitrary. No one provides a clear motivation for the design they use!

2. Experimental designs are very complicated. The number of all possible combinations of values and probabilities is huge. An experiment encompassing all possible choices is practically impossible.

3. Why there are no attempts to sample the universe of stimuli properly and work with random designs involving a large number of subjects?

25




B. THE STORY OF A COMPLEX MODEL MAKING SPECIFIC PREDICTIONS

1. One develops a complex model of a (prima facia simple) cognitive process.

2. Then, one’s model makes a very specific empirical prediction.

3. Then, one tests that specific prediction in an experiment specifically designed to test that prediction and gains empirical support for his/her model.

That empirical finding and the method used to establish it remain without context in a set of all possible experimental designs and their possible outcomes.

26




27

C1

ES(C1A)

e1 e2

Your Lab

My Lab

This can go on forever.C1m1

mA mBm2

Your publication

Our publication




28

SOCIAL, ECONOMIC AND PSYCHOLOGICAL PHENOMENA ARE

VERY COMPLICATED, SAMPLING FROM COMPLEX DESIGNS IS VERY

DIFFICULT ETC, ETC.




29

YES, I KNOW IT’S DIFFICULT.



30

THAT IS WHAT SCIENCE IS ABOUT: SOLVING

NOTORIOUSLY DIFFICULT PROBLEMS.



31

The golden plate attached to the Voyager space probe.You know, the E.T.s might find it one day.

THANK YOU FOR YOUR ATTENTION!



32

SCIENCE AND KNOWLEDGE MANAGEMENT:SOME PROBLEMS RELATED TO ACCUMULATION OF KNOWLEDGE IN CONTEMPORARY SCIENTIFIC PRACTICE

Goran S. MilovanovićUniversity of BelgradeContact: [email protected]: +381 69 16 86 298

mailto:[email protected]�

SOME PROBLEMS RELATED TO ACCUMULATION OF KNOWLEDGE IN CONTEMPORARY SCIENTIFIC PRACTICE....

Education

Transcript of SOME PROBLEMS RELATED TO ACCUMULATION OF KNOWLEDGE IN CONTEMPORARY SCIENTIFIC PRACTICE....