Who wants to be a millionaire

- Shyon K.Lam, David M.Pennock, Dan Cosley, Steve Lawrence

Archana Gupta

Temple University

Problem

Given a large number of documents that might contain the answers to a question how do we select the right answer.

How do we maximize money earned during the game.

Making the right decision of when to end the game.

Motivation

In the game millionaire player is asked questions and a right answer doubles the money won. Incorrect answer ends the game and reduces money won to last correctly answered ‘milestone’ question.

Three lifelines : phone a friend, poll the audience, eliminate two incorrect answers.

We want to device a system through which profits can be maximized.

Method

635 questions, seven difficulty levels. For exploration we use 3 random 90-question

samples and one random 180-question sample.

Two components :

Question answering module

Decision making module

Question answering module (QA) Exploit the fact that

question words associated with the answer tend to appear and repeat in multiple documents that contain the answer.

Search engines used include Google, all the web, MSN search, Alta Vista.

Also exclude .pdf files from results.

QA methods

Naïve approach: counting => response to question was the answer that produced highest no of search results and the opposite for inverted questions (not in question).

Word proximity measures: gives more credit to question words that appear closer to answer words in the document =>radius parameter (between 0 & 1).

Noun-phrase proximity.

Combining Strategies and Search Engines

On experimenting to get 70 % accuracy weights assigned were ±0.05 of wn=0.40, wp=0.15, wpp=0.45. However choosing weight manually is difficult and prone to over fitting. Hence we introduced a confidence ratio Xs which is ratio between second best score and best score for each strategy. Let T=Σ(1-xi

4) over all strategies. Weight for strategy S is ws = (1-x4

s)/T. This assigns higher weights to more confident scoring methods.

Decision making module Decides when to use a lifeline and

when to quit playing. It constructs a decision tree that

encodes probabilities and utilities at every possible future state of the game.

Maximize utility =>risk aversion u(x) = 1 – e -(x/k) k >0 risk averse, k->∞ neutral Chance node probability is 1 – x α , x is ratio

between second highest and highest score from QA module, is exp fn to convert confidence ratio to probabilities.

When a lifeline is used new probabilities are found.

Results and Conclusions

Google combined with Alta Vista gave 75%

accuracy on the sample questions.

Effects of alpha and k

DM module: discussion

Web does not perform well for 1.common sense questions, how many legs does fish have? 2.Multiple plausible answers, what does ‘I’ stand for in ‘IBM’? 3.Polysemy 4.Non textual knowledge, which city is in Russia? 5.Alternative representations

Humans generally answer the first few questions correctly because the answers are so obvious that it is difficult to find on the web. So another way to experiment would be the same program but with a handicap (able to answer first N questions correctly leaving lifeline for later.

Applications and Suggestions

It can be used for normal QA system and combined with DM module it can be used in systems like SAT.

Some timing mechanism can be incorporated to include the fastest finger round.

Some sort of an AI system could be built for the common sense questions.

Better ways to assign weights to different strategies.