الرحیم الرحمن الله بسم

KDD CUP ’10 DESCRIPTION AND RULES

Ehsan Khoddam MohammadiM.J.MahzoonKoosha K.Moogahi

The Challenge

This year's challenge asks you to predict student performance on mathematical problems from logs of student interaction with Intelligent Tutoring Systems

The students solve problems in the tutor and each interaction between the student and computer is logged as a transaction

Data Set

Development Data Sets: 3 data sets Carnegie Learning Algebra system 2005-

2006 Carnegie Learning Algebra system 2006-

2007 Bridge to Algebra system, 2006-2007

Challenge Data sets : 2 data sets Same from above systems

Data Set, Development

Four text files in each Data set: Train: All attributes are given,”Correct

First Attempt” is given Test: Some attributes are given,

“Correct First Attempt” should determine Master: same as Test file but all

attributes are given except “Correct First Attempt”

Submission: has same name as archive file, Rows which the judge wants their “Correct First Attempt” is determined here.

Attributes

19 attributes available All are given in Train file, 11

attributes is missed in Test file Four key attributes:

Problem Step Knowledge component Opportunity

Four Key Attributes(1/2)

Problem: task for a student involves multiple steps

Step: Observable part of solution, last one is “answer”. Other steps are “intermediate” Transaction: attempt to step:

Hint Incorrect Correct

Four Key Attributes(1/2)

Knowledge Component: a relevant concept or skill, to perform each step correctly

Opportunity: count for a given knowledge component increases by 1 each time the student encounters a step that requires this knowledge component

All cursed twenty attributes1. Row: 3902. Anon Student Id: 0BrbPbwCMz3. Problem Hierarchy: Unit ES_04, Section

ES_04-154. Problem Name: LIT59 5. Problem View: the total number of times the

student encountered the problem so far. 6. Step Name: the only unique identifier for a

step is the pair of problem_name and step_name.

b+r*(x+y) = v-s R5C1 GraphChoice1

1. Step Start Time: the starting time of the step. Can be null 2005-10-05 12:45:31.0

2. First Transaction Time: the time of the first transaction toward the step.

2005-10-05 12:45:46.0

3. Correct Transaction Time: the time of the correct attempt toward the step, if there was one.

4. Step End Time: the time of the last transaction toward the step.

5. Step Duration (sec): the elapsed time of the step in seconds, Can be null (if step start time is null).

15

6. Correct Step Duration (sec):the step duration if the first attempt for the step was correct.

15

7. Error Step Duration (sec): the step duration if the first attempt for the step was an error (incorrect attempt or hint request).

8. Correct First Attempt: the tutor's evaluation of the student's first attempt on the step—1 if correct, 0 if an error.

1. Incorrects: total number of incorrect attempts by the student on the step

2. Hints: total number of hints requested by the student for the step

3. Corrects: total correct attempts by the student for the step. (Only increases if the step is encountered more than once.)

4. KC(KC Model Name): the identified skills that are used in a problem, where available. A step can have multiple KCs assigned to it. Multiple KCs for a step are separated by ~~ (two tildes). Since opportunity describes practice by knowledge component, the corresponding opportunities are similarly separated by ~~.

Using small numbers~~Find X, positive slope~~Using difficult numbers

5. Opportunity(KC Model Name): a count that increases by one each time the student encounters a step with the listed knowledge component. Steps with multiple KCs will have multiple opportunity numbers separated by ~~.

36~~33~~33

Ignored Attributes in Test file Step Start Time First Transaction Time Correct Transaction Time Step End Time Step Duration (sec) Correct Step Duration (sec) Error Step Duration (sec) Correct First Attempt Incorrects Hints Corrects

Key observations

Conceptual Relationship: The data matrix is sparse, need to exploit relationships among problems

strong Temporal Dimension to the data: students improve over the course of the school year

Causal Relationships: problems is determined in part by student choices or past success history

Evaluation

performance at providing Correct First Attempt values

RMSE Both Challenge data sets have equal

weight in evaluation

Challenge duration

The challenge is 2 months in duration (April 1 - June 1, 2010). Final submissions must be received by June 1 11:59pm EDT (-4 GMT).