STT 315 Acknowledgement: Author is indebted to Dr. Ashok Sinha, Dr. Jennifer Kaplan and Dr....
-
Upload
shannon-richardson -
Category
Documents
-
view
219 -
download
0
Transcript of STT 315 Acknowledgement: Author is indebted to Dr. Ashok Sinha, Dr. Jennifer Kaplan and Dr....
STT 315
Acknowledgement: Author is indebted to Dr. Ashok Sinha, Dr. Jennifer Kaplan and Dr. Parthanil Roy for allowing him to use/edit some of their slides.
What is the goal ? Till now we have been studying about one variable. Topics covered:
Histogram, Box-plot; Pie-chart; about center; shape and spread.
This lecture talks about two or more variables, we will mainly talk about how to display and measure relationship between two variables.
Exploring relationships (or association) between two quantitative variables
by drawing a picture (known as scatterplot), and
Using a quantitative summary (known as correlation coefficient or simply correlation).
2
Example: Height and Weight How is weight of an individual related to his/her height?
• Typically, one can expect a taller person to be heavier.
• Is it supported by the data? If yes, how to determine this “association”?
Price increase of onion and it’s demand in market ?
Supply and Price of a commodity in market ? Less is the supply more is the price increase.
3
What is a scatterplot?
• A scatterplot is a diagram which is used to display values of two quantitative variables from a data-set.
• The data is displayed as a collection of points, each having the value of one variable determining the position on the horizontal axis and the value of the other variable determining the position on the vertical axis.
4
Example 1: Scatterplot of height and weight
5
Example 2: Scatterplot of hours watching TV and test scores
6
Looking at Scatterplots
We look at the following features of a scatterplot:-• Direction (positive or negative)• Form (linear, curved)• Strength (of the relationship)• Unusual Features.
When we describe histograms we mention
• Shape • Center• Spread• Outliers 7
Asking Questions on a Scatterplot
• Are test scores higher or lower when the TV watching is longer? Direction (positive or negative association).
• Does the cloud of points seem to show a linear pattern, a curved pattern, or no pattern at all? Form.
• If there is a pattern, how strong does the relationship look? Strength.
• Are there any unusual features? (2 or more groups or outliers).
8
Positive and Negative Associations
• Positive association means for most of the data-points, a higher value of one variable corresponds to a higher value of the other variable and a lower value of one variable corresponds to a lower value of the other variable.
• Negative association means for most of the data-points, a higher value of one variable corresponds to a lower value of the other variable and vice-versa.
9
This association is: A. positive B. negative.
10
This association is: A. positive B. negative.
11
Curved Scatterplot
• When the plot shows a clear curved pattern, we shall call it a curved scatterplot.
12
Linear Scatterplot• Unless we see a curve, we shall call the scatterplot
linear.• We shall soon learn how to quantify the strength of
the linear form of a scatterplot.
13
Which one has stronger linear association?
A.left one,B.right one.
Because, in the right graph the points are closer to a straight line.
14
Which one has stronger linear association?
A.left one,B.right one.
Hard to say– we need a measure of linear association.
15
Unusual Feature: Presence of Outlier
• This scatterplot clearly has an outlier.
16
Unusual Feature: Two Subgroups
• This scatterplot clearly has two subgroups.
17
Explanatory and Response Variables
• The main variable of interest (the one which we would like to predict) is called the response variable.
• The other variable is called the explanatory variable or the predictor variable.
• Typically we plot the explanatory variable along the horizonatal axis (x-axis) and the response variable along the vertical axis (y-axis).
18
Example: Scatterplot of height and weight
In this case, we are trying to predict the weight based on the height of a person. Therefore • weight is the response variable, and • height is the explanatory variable.
19
How to measure linear association?• Use correlation coefficient or simply correlation.• Correlation is a value to describe the strength of the
linear association between two quantitative variables.
• Suppose x and y are two variables. Let zx and zy denote the z-scores of x and y respectively. Then correlation is defined as:
.1
1 yx zz
nr
• We shall use TI 83/84 Plus to compute correlation coefficient (to be discussed later).
20
21
Correlation is unit-freeBecause correlation is calculated using standardized scores
it is free of unit (i.e. does not have any unit); does not change if the data are rescaled.
In particular, this means that correlation does not depend on the unit of the two quantitative variables.
For example, if you are computing the correlation between the heights and weights of a bunch of individuals, it does not matter if the heights are measured in inches or cms and if the weights are measured in lbs or kgs.
22
Properties of Correlation• Correlation is unit-free.
• Correlation does not change if the data are rescaled.
• It is a number between -1 and 1.
• The sign of the correlation indicates the direction of the linear association (if the association is positive then so is the correlation and if the association is negative then so is the correlation).
• The closer the correlation is to -1 or 1, the stronger is the linear association.
• Correlations near 0 indicate weak linear association.
23
Words of Warning about Correlation• Correlation measures linear association between two
quantitative variables.• Correlation measures only the strength of the linear
association.• If correlation between two variables is 0, it only means that
they are not linearly associated. They may still be nonlinearly associated.
• To measure the strength of linear association only the value of correlation matters.A correlation of -0.8 is a stronger linear association compared
to a correlation value 0.7.The negative and positive signs of correlation only indicate
direction of association.• Presence of outlier(s) may severely influence correlation.• High correlation value may not always imply causation.
24
Check before calculation of correlation
• Are the variables quantitative?
• Is the form of the scatter plot straight enough (so that a linear relationship makes sense)?
• Have we removed the outliers? Or else, the value of the correlation can get distorted dramatically.
25
Regression line andprediction
26
Explanatory and Response Variables
Above scatter plot indicates a linear relationship between height and weight. Suppose an individual is 68 in tall. How can we predict his weight?
• The main variable of interest (the one which we would like to predict) is called the response variable (denoted by y).
• The other variable is called the explanatory variable or the predictor variable (denoted by x).
Here height is the predictor (or explanatory variable) and weight is the response variable.
27
What is Linear Regression?• When the scatter plot looks roughly
linear, we may model the relationship between the variables with “best-fitted” line (known as regression line): y = b0 + b1 x.
• b1 (the coefficient of x) is called the slope of the regression line.
• b0 is called the intercept of the
regression line. • We estimate the slope (b1) and the
intercept (b0).• Next given the value of x, we plug in
that value in the regression line equation to predict y.
This procedure is called linear regression.
28
Conditions for Linear Regression
• Quantitative Variables Condition: both variables have to be quantitative.
• Straight Enough Condition: the scatter plot must appear to have moderate linear association.
• Outlier Condition: there should not be any outliers.
29
Example of Linear RegressionSuppose
• x = amount of protein (in gm) in a burger (explanatory variable),
• y = amount of fat in (in gm) the burger (response variable).
Goal: Express the relationship of x and y using a line (the regression line): y = b0 + b1x.
Questions:1. How to find b1 (slope) and b0 (intercept)?2. How will it help in prediction?
30
Formulae of b0 and b1
.1
x
y
s
srb
.10 xbyb
Intercept: b0 = (mean of y) – b1×(mean of x)
i.e.
• Although there are many lines that can describe the relationship, there is a way to find “the line that fits best”.
• For the best fitted line: Slope: b1 = (correlation)×(std.dev. of y)/(std.dev. of x)
i.e.
31
• If we are given the summary statistics, i.e. mean, standard deviations of x and y and their correlations, then we plug in those values in the formulae to find b0 and b1.
• If we are given the actual data (not the summary), then we need to compute all those summary values.
• However given the data TI 83/84 Plus can find the equation of regression line. But be careful, because TI 83/84 writes the
regression equation as y = ax + b.So a = slope (= b1), and b = intercept (= b0).
Computation of b0 and b1
32
Example 1If in a linear regression problem, the correlation between the variables is 0.9 and the standard deviations of the explanatory (x) and the response (y) variables are 0.75 and 0.25 respectively, and the means of the explanatory and the response variables are 10 and 5 respectively, calculate the regression line.
• Estimate of slope: b1 = 0.9 × (0.25/0.75) = 0.3.• Estimate of intercept: b0 = 5 - 0.3 × 10 = 2.• So the estimated regression line is: y = 0.3x + 2.
33
Example 2
Fat (g) Sodium (mg) Calories
19 920 410
31 1500 580
34 1310 590
35 860 570
39 1180 640
39 940 680
43 1260 660
Fat (in gm), Sodium (in mg) and Calorie content in 7 burgers are given above.
Using TI 83/84 Plus for regression• Press [STAT].• Choose 1: Edit.• Type the Fat data under L1, Sodium under L2 and Calories under
L3.Suppose (L1) Fat is the predictor and (L2) Sodium is the response.• Press [STAT] again and select CALC using right-arrow.• Select 4: LinReg(ax+b) (LinReg(ax+b) appears on screen).• Type [2nd] and [1] (that will put L1 on screen).• Type , and then [2nd] and [2] (that will put ,L2 on screen).• Press [ENTER].• This will produce a (slope), b (intercept), r2 and r (correlation
coefficient).
34
Tips: Using TI 83/84 Plus
• Caution: After LinReg(ax+b) you must first put the predictor (explanatory) variable, and then the response variable.
• Note that the values of r and r2 will not show up if the diagnostic is not switched on in the TI 83/84 Plus calculator.
• To get the diagnostic switched on:1. Press [2nd] and [0] (that will choose CATALOG).2. Select using arrow keys DiagnosticOn.3. Press [ENTER] and [ENTER] again.4. This will switch the diagnostic on.
35
Tips: Using TI 83/84 Plus• To delete one particular list variable (say L2):
1. Press [STAT].2. Choose 1: Edit.3. Select the variable L2 using the arrow keys.4. Press [CLEAR] followed by [ENTER].
• To delete all stored data:a) Press [2nd] and [+] (that will choose MEM).b) Select 4: ClrAllLists.c) Press [ENTER] and [ENTER] again.d) This will clear all the stored data in the lists.
36
37
Fat vs. SodiumFat vs Sodium
600
800
1000
1200
1400
1600
15 20 25 30 35 40 45
Fat (g)
So
diu
m (
mg
)
Using TI 83/84 Plus we get: r = 0.199 r2 = 0.0396 a = 6.08 (= b1) b = 930.02 (= b0)
• Correlation is approximately 0.2, which indicates that linear association is very weak (positive) between fat and sodium.
• Scatter plot supports the small value of r.• Regression line: y = 6.08x + 930.02.
38
Fat vs. Calories
Using TI 83/84 Plus we get: r = 0.961 r2 = 0.923 a = 11.06 (= b1) b = 210.95 (= b0)
• Correlation 0.96 indicates that there is a very strong positive linear relation between fat and calories.
• Scatter plot supports the high positive value of r.• Regression line: y = 11.06x + 210.95.
Fat vs Calories
350
400
450
500
550
600
650
700
15 20 25 30 35 40 45
Fat (g)
Cal
ori
es
39
CountryPercent with Cell Phone
Life Expectancy (years)
Turkey 85.7% 71.96
France 92.5% 80.98
Uzbekistan 46.1% 71.96
China 47.4% 73.47
Malawi 11.9% 50.03
Brazil 75.8% 71.99
Israel 123.1% 80.73
Switzerland 115.5% 80.85
Bolivia 49.4% 66.89
Georgia 59.7% 76.72
Cyprus 93.8% 77.49
Spain 122.6% 80.05
Indonesia 58.5% 70.76
Botswana 74.6% 61.85
U.S. 87.9% 78.11
Example 3
40
Example 3: Scatter plot with regression line
%Cell Phone vs Life Expectancy
y = 0.21x + 56.91R = 0.7848
R2 = 0.6159
45
50
55
60
65
70
75
80
85
0 20 40 60 80 100 120 140
% Cell Phone
Lif
e E
xp
ec
tan
cy
Possibleoutliers
41
Example 3: Scatter plot with regression line
% Cell Phones vs Life Expectancy (without outliers)
y = 0.13x + 64.7R = 0.802
R2 = 0.6437
45
50
55
60
65
70
75
80
85
0 20 40 60 80 100 120 140
% Cell Phones
Life
Exp
ecta
ncy
42
Predicted values and residuals
• Let the regression line be y = b0 + b1x.• Suppose (x0, y0) is an observed data.• Then the predicted value of y given x = x0 is
.ˆ 010 xbby
.ˆ0 yye
• Residual (denoted by e) measures how much the predicted value deviates from the observed value of y.
• So if the observed value of y is y0 then
residual = (observed value - predicted value)
• Residuals are the errors due to prediction using the regression line.
43
Example 1 revisitedIf in a linear regression problem, the correlation between the variables is 0.9 and the standard deviations of the x and y are 0.75 and 0.25 respectively, and the means of the x and y are 10 and 5 respectively. Calculate the residual when x = 20 and y = 7.
• Estimate of slope: b1 = 0.9 × (0.25/0.75) = 0.3.
• Estimate of intercept: b0 = 5 - 0.3 × 10 = 2.
• So the estimated regression line is: y = 0.3x + 2.
• The predicted value of y when x = 20, is = 0.3 × 20 + 2 = 8.
• The corresponding residual is = (observed y) – (predicted y) = 7 – 8 = -1.
44
Evaluating regression
The fit of a regression line can be evaluated with
R2 (the coefficient of determination),
se (standard deviation of residuals).
45
R2 (the coefficient of determination)
R2 is the fraction of total sample variation in y explained by the regression model.
Some properties of R2 : R2 = (correlation)2 = r2. 0 ≤ R2 ≤ 1. R2 close to 0 implies weak linear relationship (and
also not a good fit of the regression line). R2 close to 1 implies strong linear relationship (and
also a very good fit of the regression line).
46
R2 (the coefficient of determination)• For instance, if R2 = 0.54, then 54% of the total sample variation in
y is explained by the regression model. It indicates a moderate fit of the regression line.On the scatter plot the points will not be very close to the
regression line.
• If R2 = 0.96, then 96% of the total sample variation in y is explained by the regression model. It indicates a very good fit of the regression line. On the scatter plot the points will be very close to the regression line.
• On the other hand, if R2 = 0.19, then only 19% of the total sample variation in y is explained by the regression model, which indicates a very bad fit of the regression line. The scatter plot will show either a curved pattern, or the points
will be clustered showing no pattern.
47
se (standard deviation of residuals)
se is the standard deviation of the residuals.
In case there is no ambiguity, we often just write s instead of se.
Smaller the se better the model fit. Larger the se worse the model fit.
Remember that residuals are the errors due to prediction using the regression line.
So larger value of se implies that there is more spread in the residuals, as a result there is more error in the prediction. Hence the observations are not close to the regression line.
On the other hand, smaller value of se indicates that the observations are closer to the regression line, implying a better fit.
48
Summary: Direction of linear association
• The sign of correlation (r) indicates the direction of the linear association between x and y.
• Notice that the slope (b1) of regression line has the same sign as that of correlation (r). Hence the sign of slope also indicates the direction of the linear association between x and y.
• Positive sign of correlation (and slope) implies positive linear association.
• Negative sign of correlation (and slope) implies negative linear association.
49
Summary: Strength of linear association
• The value of correlation (r) [ignoring the sign] gives us the strength of the linear association.
• Also the value of R2 gives us the strength of the linear association.
• Values close to 1 implies strong linear association.• Values close to 0 implies weak or no linear
association.• You cannot determine strength of linear association
from the value of slope (b1) of regression line.
50
Correlation (r) and R2
• If you know the value of correlation, you can compute R2 = (correlation)2.
• But knowing the value of R2 alone is not sufficient to compute correlation, because we cannot get the information of sign of correlation.
• However if we know the value of R2 and also the sign of slope (b1) we can compute correlation as follows:
r = (sign of b1) √ R2.
51
Lurking variable
• High correlation between x and y may not always mean causation.
• Sometimes there is a lurking variable which is highly correlated to both x and y and as a result we obtain a high correlation between x and y.
52
Example: Lurking variable• A study at the University of Pennsylvania Medical Center
(published in the May 13, 1999 issue of Nature) concluded:-
young children who sleep with the light on are much more likely to develop eye problems in later life.
• However, a later study at The Ohio State University did not find that infants sleeping with the light on caused the development of eye problems.
• The second study (done at OSU) did find a strong link between parental myopia and the development of child myopia, also noting that myopic parents were more likely to leave a light on in their children's bedroom. So parental myopia is a lurking variable here.
Reference: http://en.wikipedia.org/wiki/Correlation_does_not_imply_causation
53
Choose the best description of the scatter plot
A. Moderate, negative, linear associationB. Strong, curved, associationC. Moderate, positive, linear associationD. Strong, negative, non-linear associationE. Weak, positive, linear association
54
Match the following values of correlation coefficients for the data shown in this scatter plots.
A. r = -0.67
B. r = -0.10
C. r = 0.71
D. r = 0.96
E. r = 1.00
Fig. 1
Fig. 3
Fig. 2