Conduct an
experiment to
investigate a
situation using
statistical methods
Level 2 AS91265
3 credits
MCS: 14 Nov 2012
What is an Experiment?Experiment: A study in which a researcher attempts to understand the
effect that a variable (the explanatory variable) may have on some
phenomenon (the response) by controlling the conditions of the study.
The researcher controls the conditions by allocating individuals to
groups and allocating the value of the explanatory variable to be
received by each group. A value of the explanatory variable is called a
treatment.
In a well-designed experiment, the allocation of subjects to groups is
done using randomisation. Randomisation attempts to make the
characteristics of each group very similar so that if each group was
given the same treatment, the groups should respond in a similar way,
on average.
• Give an example for each word or phrase in red, using the wall-sit
experiment that we did.
What is an Experiment? Pt 2Experiments usually have a control group, a group that receives no
treatment or receives an existing or established treatment. This allows
any differences in the response, on average, between the control group
and the treatment group(s) to be visible.
When the groups are similar in all ways apart from the treatment
received, then any observed differences in the response (if large
enough) among the groups, on average, is said to be caused by the
treatment.
Example: in an experiment to test
the effect of a fertilizer, the
experiment can be controlled by
assigning the treatments to
randomly selected plots of land.
This mitigates the effect of
variations in soil composition on
the growth of the plants and allows
us to see if the fertilizer works.
Paired Comparison ExperimentsThese comparisons are before (baseline) vs after treatment or
However, the data must be kept paired, as it is not independent.
Analysis:
• A link graph on the same scale.
• Followed by a dot plot of differences.
• Compare what you see (direction of arrows in link graph,
clusters, variation, averages in dot plot of differences)
Conclusion:
• Descriptive, "the data suggests that (running for 5 minutes)
caused (heart rates to increase) on average".
• Can calculate the average of the differences (medians, means)
for some quantification (how much did they increase by?)
Link Graphs• Represent the finding with a
link graph on the same scale.
• Make sure the data is kept as a pair,
joined with an arrow.
Compare what you see
(direction/gradient of arrows in link graph):
• The link graph shows that the vast majority of
people improved on their second attempt. We see
this as only one person scored fewer baskets on
the second attempt (arrow pointing down) and
only two people made no improvement (arrow
pointing straight across). The remaining 8 people
all improved (arrow pointing up).
• This suggests to us that the treatment (advice on
throwing a basketball) appears to improve the
number of baskets obtained out of 20 attempts.
Attempt
1
Attempt
2
Baskets Baskets
8 8
8 10
9 11
10 13
12 11
12 13
12 14
13 13
13 15
14 17
15 19
Difference
-1 0 1 2 3 4 5
Difference in Basketball shot attempts Dot Plot
Dot Plot of DifferencesA dot plot of differences: Attempt 1 Attempt 2
Baskets Baskets Difference
8 8 0
8 10 2
9 11 2
10 13 3
12 11 -1
12 13 1
12 14 2
13 13 0
13 15 2
14 17 3
15 19 4
median 2
mean 1.64
mode 2
LQ 0
UQ 3
IQR 3
Compare what you see (clusters, variation, averages):
• The differences are reasonably normally distributed,
centred around 2, which is the median of the
differences. The mean is 1.64, showing a slight skew
to the left.
• There are no significant clusters, but the mode is at a
difference of 2 baskets.
• The interquartile range shows the middle 50% of data
is between 0 and 3 extra baskets.
• This suggests to us that the treatment causes a
typical improvement of 2 extra successful baskets.
V DRINK ExperimentCounting the total number of Heart beats before and after the
treatment
1. What do you think this experiment is testing?
2. What is the treatment? What is the response variable?
3. What do you think the outcome will be? (Hypothesis)The scores will be written on the board.
Analyse the results by:
• Constructing a link graph.
• Constructing a dot plot of differences.
• Generate summary statistics
• Compare what you see
Write a conclusion:
• “The data suggests that…”
PPDAC Cycle
roblem
• Understand and define the situation that we are trying to investigate.
• Identify what variables will be investigated.
• Make sure you need an experiment to investigate the relationship between them.
• What you would need to look for in the data?
• Make a Hypothesis – predict the results of the experiment.
lan
• Link to relevant knowledge about the context
• Explain how/why the explanatory variable will be changed (could be categorical, eg control/ treatment)
• Explain how/why the response variable will be measured
• Describe how the data will be collected and recorded
• Identify factors that might affect the results of the experiment. ie Any related variables and the possible effects of these (sources of variation/bias)
• List and justify the steps to be taken to carry out the experiment
ata• When carrying out experiments, make notes
about the data collection and experimental
process. These notes will be useful in
reflection of the process in the report write up.
• What else could you investigate that would help you
understand the situation better?
Or consider why your experiment did not give you the
expected results.
• You should be combining ideas of how a well run experiment
gives you good data, allowing you to answer the
investigation question knowing that the effects you observe
can be attributed to the variable you manipulated.
• Make sure that you clean the data if necessary.
Problem and Planning PracticePick an investigation:
1. Does eating chocolate increase your heart rate?
2. Does the surface that you run on affect how fast you can run?
Define the Problem that you will investigate :
• Define the exact problem
• What variables will be investigated?
• What is your hypothesis? (ie what do you think that you will find?)
Write up a Plan for doing the experiment
• What do you know about the context?
• Define the exact explanatory variable
• Define the response variable and how it will be measured
• How will the data be recorded?
• What other factors/variables/bias might affect the results?
• Describe in full how the experiment will be conducted.
• Justify all of your decisions
nalysis
Paired Comparisons:
• A link graph on the same scale.
• Followed by a dot plot of differences.
• Compare what you see (direction of arrows in link graph,
clusters, variation, averages in dot plot of differences)
Independent Groups:
• Comparative dot plots/box plots on the same scale.
• Generate summary statistics (medians, means, IQR)
• Compare what you see (differences, clusters, variation)
onclusion• Descriptive: "the data suggests that using
(the treatment) caused (the students to
jump further) on average".
Paired Comparison:
• Can calculate the average of the
differences (medians, means) for some
quantification (how much did they
increase by?)
Independent Groups:
• Can calculate the difference of averages
(medians, means) for some quantification
(how much further?)
onclusion• What did you find out from the experiment?
• What does the data suggest?
• Consider what other questions could be
investigated that would give more insight
into the experimental situation.
• Discuss the impact of other sources of
variation on the experimental data and
explain how this can be seen in the data.
• Reflect on how well the experiment went
and aspects that could be changed so that
the question could be investigated better.
Making a Conclusion
• Repetition is key part of good experiment design.
• Ideally, we would repeat the experiment many times (or with many subjects) to confirm the findings.
Type of
experimentPaired comparison (dependent)
Comparison of two (or more)
independent groups
Questions
Does doing something improve
something? What’s the difference or
change if I do something? What’s the
effect of doing this?
Is this better than this? Does doing this
give better results than doing this?
Does it matter if I do this?
Conditions
Same group, before and after,
measuring change/difference/
improvement in variable
[linking two measurements of one
variable from the same unit/person]
Two different groups, comparing one
variable across two (or more)
independent measures/conditions [treatments]
What are you
manipulating?
The in between, the change in
conditions e.g. the fact that you
exercise, drinking caffeine, watching a
scary movie
The treatment(s) - what each group
gets done to them (or for a control
group, nothing)
Exploratory
data analysis
Link-graph, dot plots of differences.
Derive variable for increase/difference.
Conclusion about change/improvement.
Comparative dot plots and box plots,
with summary statistics.
Can explore data further and look at
variation – shape, distance.
What not to
do?Don’t: Separate the two measurements
and compare (breaking the link)Don’t: Compare two groups that are
not independent
Examples
Heart rate before compared to after
exercise, how much do heart rates
increase by?
Can people who stretch perform an
exercise longer than people who don’t?
Wall-Sit ExperimentWe are going to conduct an experiment to see:
Does stretching before a wall-sit increase the length of time
that it can be done for?
For this experiment, we are going to divide the class into two
groups – the control group and the treatment group.
Control group: does exercise without any prior stretching.
Treatment group: stretches before doing exercise.
• How can we ensure that there is no bias in
the allocation of groups?
• Should we let people pick their own group?
• How will we know if the treatment works?
Wall-Sit ExperimentA “wall-sit” is done like this: stand with your back against a
wall and lower yourself until your thighs are parallel to the
floor. Lift your dominant foot about 5cm off the ground.
The experiment ends when the subject puts their foot back on
the ground.
• How do we eliminate other sources of variation?
• What kinds of variables could we measure in
this experiment?
The treatment group should take some time
to stretch, especially their quads.
Both groups should start the exercise at the
same time.
Wall-Sit Experiment• How can we quantify the effect of the treatment?
• How could we display the data?
Gender Excellent Above
Average
Average Below
Average
Poor
Male >102 s 102 - 76 s 75 - 58 s 57 - 30 s <30 s
Female >60 s 60 - 46 s 45 - 36 s 35 - 20 s <20 s
Normative data for the Wall Squat Test
The following table is the USA norms for 16 to 19 year olds.
Top Related