Conduct an

experiment to

investigate a

situation using

statistical methods

Level 2 AS91265

3 credits

MCS: 14 Nov 2012

What is an Experiment?Experiment: A study in which a researcher attempts to understand the

effect that a variable (the explanatory variable) may have on some

phenomenon (the response) by controlling the conditions of the study.

The researcher controls the conditions by allocating individuals to

groups and allocating the value of the explanatory variable to be

received by each group. A value of the explanatory variable is called a

treatment.

In a well-designed experiment, the allocation of subjects to groups is

done using randomisation. Randomisation attempts to make the

characteristics of each group very similar so that if each group was

given the same treatment, the groups should respond in a similar way,

on average.

• Give an example for each word or phrase in red, using the wall-sit

experiment that we did.

What is an Experiment? Pt 2Experiments usually have a control group, a group that receives no

treatment or receives an existing or established treatment. This allows

any differences in the response, on average, between the control group

and the treatment group(s) to be visible.

When the groups are similar in all ways apart from the treatment

received, then any observed differences in the response (if large

enough) among the groups, on average, is said to be caused by the

treatment.

Example: in an experiment to test

the effect of a fertilizer, the

experiment can be controlled by

assigning the treatments to

randomly selected plots of land.

This mitigates the effect of

variations in soil composition on

the growth of the plants and allows

us to see if the fertilizer works.

Paired Comparison ExperimentsThese comparisons are before (baseline) vs after treatment or

However, the data must be kept paired, as it is not independent.

Analysis:

• A link graph on the same scale.

• Followed by a dot plot of differences.

• Compare what you see (direction of arrows in link graph,

clusters, variation, averages in dot plot of differences)

Conclusion:

• Descriptive, "the data suggests that (running for 5 minutes)

caused (heart rates to increase) on average".

• Can calculate the average of the differences (medians, means)

for some quantification (how much did they increase by?)

Link Graphs• Represent the finding with a

link graph on the same scale.

• Make sure the data is kept as a pair,

joined with an arrow.

Compare what you see

(direction/gradient of arrows in link graph):

• The link graph shows that the vast majority of

people improved on their second attempt. We see

this as only one person scored fewer baskets on

the second attempt (arrow pointing down) and

only two people made no improvement (arrow

pointing straight across). The remaining 8 people

all improved (arrow pointing up).

• This suggests to us that the treatment (advice on

throwing a basketball) appears to improve the

number of baskets obtained out of 20 attempts.

Attempt

1

Attempt

2

Baskets Baskets

8 8

8 10

9 11

10 13

12 11

12 13

12 14

13 13

13 15

14 17

15 19

Difference

-1 0 1 2 3 4 5

Difference in Basketball shot attempts Dot Plot

Dot Plot of DifferencesA dot plot of differences: Attempt 1 Attempt 2

Baskets Baskets Difference

8 8 0

8 10 2

9 11 2

10 13 3

12 11 -1

12 13 1

12 14 2

13 13 0

13 15 2

14 17 3

15 19 4

median 2

mean 1.64

mode 2

LQ 0

UQ 3

IQR 3

Compare what you see (clusters, variation, averages):

• The differences are reasonably normally distributed,

centred around 2, which is the median of the

differences. The mean is 1.64, showing a slight skew

to the left.

• There are no significant clusters, but the mode is at a

difference of 2 baskets.

• The interquartile range shows the middle 50% of data

is between 0 and 3 extra baskets.

• This suggests to us that the treatment causes a

typical improvement of 2 extra successful baskets.

V DRINK ExperimentCounting the total number of Heart beats before and after the

treatment

1. What do you think this experiment is testing?

2. What is the treatment? What is the response variable?

3. What do you think the outcome will be? (Hypothesis)The scores will be written on the board.

Analyse the results by:

• Constructing a link graph.

• Constructing a dot plot of differences.

• Generate summary statistics

• Compare what you see

Write a conclusion:

• “The data suggests that…”

PPDAC Cycle

roblem

• Understand and define the situation that we are trying to investigate.

• Identify what variables will be investigated.

• Make sure you need an experiment to investigate the relationship between them.

• What you would need to look for in the data?

• Make a Hypothesis – predict the results of the experiment.

lan

• Link to relevant knowledge about the context

• Explain how/why the explanatory variable will be changed (could be categorical, eg control/ treatment)

• Explain how/why the response variable will be measured

• Describe how the data will be collected and recorded

• Identify factors that might affect the results of the experiment. ie Any related variables and the possible effects of these (sources of variation/bias)

• List and justify the steps to be taken to carry out the experiment

ata• When carrying out experiments, make notes

about the data collection and experimental

process. These notes will be useful in

reflection of the process in the report write up.

• What else could you investigate that would help you

understand the situation better?

Or consider why your experiment did not give you the

expected results.

• You should be combining ideas of how a well run experiment

gives you good data, allowing you to answer the

investigation question knowing that the effects you observe

can be attributed to the variable you manipulated.

• Make sure that you clean the data if necessary.

Problem and Planning PracticePick an investigation:

1. Does eating chocolate increase your heart rate?

2. Does the surface that you run on affect how fast you can run?

Define the Problem that you will investigate :

• Define the exact problem

• What variables will be investigated?

• What is your hypothesis? (ie what do you think that you will find?)

Write up a Plan for doing the experiment

• What do you know about the context?

• Define the exact explanatory variable

• Define the response variable and how it will be measured

• How will the data be recorded?

• What other factors/variables/bias might affect the results?

• Describe in full how the experiment will be conducted.

• Justify all of your decisions

nalysis

Paired Comparisons:

• A link graph on the same scale.

• Followed by a dot plot of differences.

• Compare what you see (direction of arrows in link graph,

clusters, variation, averages in dot plot of differences)

Independent Groups:

• Comparative dot plots/box plots on the same scale.

• Generate summary statistics (medians, means, IQR)

• Compare what you see (differences, clusters, variation)

onclusion• Descriptive: "the data suggests that using

(the treatment) caused (the students to

jump further) on average".

Paired Comparison:

• Can calculate the average of the

differences (medians, means) for some

quantification (how much did they

increase by?)

Independent Groups:

• Can calculate the difference of averages

(medians, means) for some quantification

(how much further?)

onclusion• What did you find out from the experiment?

• What does the data suggest?

• Consider what other questions could be

investigated that would give more insight

into the experimental situation.

• Discuss the impact of other sources of

variation on the experimental data and

explain how this can be seen in the data.

• Reflect on how well the experiment went

and aspects that could be changed so that

the question could be investigated better.

Making a Conclusion

• Repetition is key part of good experiment design.

• Ideally, we would repeat the experiment many times (or with many subjects) to confirm the findings.

Type of

experimentPaired comparison (dependent)

Comparison of two (or more)

independent groups

Questions

Does doing something improve

something? What’s the difference or

change if I do something? What’s the

effect of doing this?

Is this better than this? Does doing this

give better results than doing this?

Does it matter if I do this?

Conditions

Same group, before and after,

measuring change/difference/

improvement in variable

[linking two measurements of one

variable from the same unit/person]

Two different groups, comparing one

variable across two (or more)

independent measures/conditions [treatments]

What are you

manipulating?

The in between, the change in

conditions e.g. the fact that you

exercise, drinking caffeine, watching a

scary movie

The treatment(s) - what each group

gets done to them (or for a control

group, nothing)

Exploratory

data analysis

Link-graph, dot plots of differences.

Derive variable for increase/difference.

Conclusion about change/improvement.

Comparative dot plots and box plots,

with summary statistics.

Can explore data further and look at

variation – shape, distance.

What not to

do?Don’t: Separate the two measurements

and compare (breaking the link)Don’t: Compare two groups that are

not independent

Examples

Heart rate before compared to after

exercise, how much do heart rates

increase by?

Can people who stretch perform an

exercise longer than people who don’t?

Wall-Sit ExperimentWe are going to conduct an experiment to see:

Does stretching before a wall-sit increase the length of time

that it can be done for?

For this experiment, we are going to divide the class into two

groups – the control group and the treatment group.

Control group: does exercise without any prior stretching.

Treatment group: stretches before doing exercise.

• How can we ensure that there is no bias in

the allocation of groups?

• Should we let people pick their own group?

• How will we know if the treatment works?

Wall-Sit ExperimentA “wall-sit” is done like this: stand with your back against a

wall and lower yourself until your thighs are parallel to the

floor. Lift your dominant foot about 5cm off the ground.

The experiment ends when the subject puts their foot back on

the ground.

• How do we eliminate other sources of variation?

• What kinds of variables could we measure in

this experiment?

The treatment group should take some time

to stretch, especially their quads.

Both groups should start the exercise at the

same time.

Wall-Sit Experiment• How can we quantify the effect of the treatment?

• How could we display the data?

Gender Excellent Above

Average

Average Below

Average

Poor

Male >102 s 102 - 76 s 75 - 58 s 57 - 30 s <30 s

Female >60 s 60 - 46 s 45 - 36 s 35 - 20 s <20 s

Normative data for the Wall Squat Test

The following table is the USA norms for 16 to 19 year olds.