Visualization - Statistical Methods

Sarah Filippi, University of Oxford

20 October 2015Michaelmas Term 2015

First step

The starting point of ALL good statistical data analysis beginswith graphical plots and summary statistics of the data

ALWAYS, ALWAYS, ALWAYS, PLOT YOUR DATA!!!

Graphics reveal data, communicate complex ideas anddependencies with clarity, precision and efficiency.

Graphical excellence

Excellent graphics:

I show the data

I induce the viewer to think about the substance

I avoid bias

I make large complex data sets coherent

I encourage data exploration and debate

Categorical Data

Let’s start by not using one common graph type:

There is no data that can be displayed in a pie chartthat cannot be displayed BETTER in some other type ofchart.J. W. Tukey

And let’s not even think about 3D and exploded pie charts.

What’s the matter with pie charts:

I people are not good at interpreting areas

I small and large slices are relatively distorted

I zero is often a very meaningful number but gets los

I very hard to compare two pie charts

Barplots are usually a much better choice: barplot(height)

Suppose we have a few ordinal or categorical variables: theinteresting questions are then how they vary together. Here is across-tabulation on the caffeine consumption (in mg/day) ofwomen in a maternity ward by marital status. (A contingencytable.)

0 1-150 151-300 300+Married 652 1537 598 242Prev.married 36 46 38 21Single 218 327 106 67

The next two slides show two graphical representations, a set ofbarplots (aka bar charts), and a mosaic plot.Different versions of these plots and other plots for categoricaldata can be found in package vcd.

Married Prev. married Single

00−150150−300300+

020

040

060

080

010

0012

0014

00

Married Prev. married Single

00−150150−300300+

0.0

0.2

0.4

0.6

0.8

1.0

A special case of a mosaic plot is sometimes called a spineplot.

0

0

Married Prev. married

00−

150

150−

300

0.0

0.2

0.4

0.6

0.8

1.0

Barplots

Barplots should not (!) be used to compare distributions of dataacross groups.

Box-and-whisker Plots (boxplots)

2000

2500

3000

3500

4000

Median

1st quartile

3rd quartile

Lower Whisker

Upper Whisker

Outliers

There are about as many variations as software designers.

Parallel box plots are often useful to show the differences betweensubgroups of the data.

●

●●

●

●

●

●

●●●●

(0,100] (100,1000] (1000,1e+04] (1e+04,1e+05]

050

100

150

GDP

Infant mortality

Violin plotsViolin plots replace the representation in a boxplot by avariable-width box determined by a density estimate.

See e.g. the help for function vioplot() in the package of thatname or the help for function panel.violin() in packagelattice.

050

100

150

(0,100] (100,1000] (1000,1e+04] (1e+04,1e+05]

●

●

●

●

infant.mortality

(0,100]

(100,1000]

(1000,1e+04]

(1e+04,1e+05]

0 50 100 150

●

●

●

●

●●● ●

● ● ●● ●●●

Histograms

I Very convenient to study the shape of the distribution of thedata.

I We can choose a set of breakpoints covering the data, andcount how many points fall into each interval.

I Warning: some software plot the counts or the proportions orpercentages.

A true histogram has the area of each bar proportional to thecount, and total area one. This matters if the breaks are notequally spaced. See function truehist() in package MASS.

How do we choose the number and position of the breaks?

hist(data, prob = FALSE, breaks=breaks)

truehist(data, x0)

Histogram of infant.mortality

infant.mortality

Fre

quen

cy

0 50 100 150

020

4060

80

Histogram of infant.mortality

infant.mortality

Den

sity

0 50 100 150

0.00

00.

005

0.01

00.

015

0.02

0

Histogram of infant.mortality

infant.mortality

Fre

quen

cy

0 50 100 150

05

1015

2025

3035

Histogram of infant.mortality

infant.mortality

Den

sity

0 50 100 150

0.00

00.

010

0.02

00.

030

1 2 3 4 5

0.0

0.2

0.4

0.6

0.8

duration

1 2 3 4 5

0.0

0.1

0.2

0.3

0.4

0.5

duration

Duration of Old Faithful eruptions

Density Plots

I Histograms are density plots: the tops of the bars is apiecewise-constant estimator of the underlying pdf.

I We can use smooth estimates of density. Examples:I Kernel density estimates

f̂(x) =1

n

n∑i=1

Kh(x− xi)

where Kh is a kernel and h is the bandwidth.

density() – check arguments bw, from, to...

I Splines or losplines: a spline is a piecewise polynomial functionwhich has smooth properties at the places where thepolynomial pieces connect.

logspline() in package polspline

kernel density

infant.mortality

Den

sity

0 50 100 150

0.00

00.

005

0.01

00.

015

0.02

00.

025

0.03

00.

035

logspline

infant.mortality

Den

sity

0 50 100 150

0.00

00.

005

0.01

00.

015

0.02

00.

025

0.03

00.

035

Rugs

We can add rug below the x axes to highlight position of data. Seethe help for rug() and also jitter().

Possible to have semi-transparent grey rugs by specifying the color:col=rgb(0,0,0,0.25).

1 2 3 4 5

0.0

0.5

1.0

1.5

kernel density

duration

1 2 3 4 5

0.0

0.5

1.0

1.5

logspline

duration

Scatterplots

The canonical plot of two continuous variables is a scatterplot.

Example from UN dataset:plot(infant.mortality ∼ gdp, UN, cex = 0.5)

plot(infant.mortality ∼ gdp, UN, log = "xy",...)

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

0 10000 20000 30000 40000

050

100

150

gdp

infa

nt.m

orta

lity

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

50 100 200 500 2000 5000 20000

25

1020

5010

020

0

gdp

infa

nt.m

orta

lity

Using scatterplot() from package car.

50 100 200 500 1000 2000 5000 10000 20000 50000

25

1020

5010

020

0

gdp

infa

nt.m

orta

lity

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

Tonga

Iraq

Afghanistan

Bosnia

Sao.Tome

Sudan

Gabon

Liberia

Korea.Dem.Peoples.Rep French.Guiana

Smoother

I A fitted regression line and a smooth line have beenautomatically added.

I Such smooth curves often help to highlight trends in ascatterplot, but they can also be deceptive. Smoothers arethings we will return to, but see the functionsloess.smooth() and smooth.spline().

I With the scatterplot() function, outliers are automaticallylabelled. It is often best to do this manually with theidentify() function.

It is often useful to visually convey confidence in your plots.

●●

●

●

●

●

●

●

●

●●

● ●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

0

100

200

300

10 15 20 25 30 35mpg

hp

Scatterplot with color by types

5000 10000 15000 20000 25000

2030

4050

6070

80

income

pres

tige

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

type

bc prof wc

scatterplot(prestige ∼ income|type, data=Prestige,

smoother=FALSE, reg.line=FALSE)

Scatterplot Matrices (or pairs plots)

Sepal.Length

2.0 2.5 3.0 3.5 4.0

●●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●● ●

●●

●●

●●

●

●●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●●

●●

●●

●●

●

●●●

●●

●

●

●

●

●●●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●●

●

●●●

●

●●●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●●

●●

●●

●●

●

●●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●●

●●

●●

●●

●

●●●

●●

●

●

●

●

●● ●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●●

●

●●

●

●

●●●

●●

●

●

0.5 1.0 1.5 2.0 2.5

4.5

5.5

6.5

7.5

●●●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●● ●●●

●●

●●

●

●●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●●

●●

●●●

●

●

●●●

●●

●

●

●

●

●●●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●●

●

●●

●

●

●●●

●●

●

●

2.0

2.5

3.0

3.5

4.0

●

●

●●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●●

●

●●●

●●

●

●●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●●

● ●

●

●

●

●

●

●●

●

●

●

●

●● ●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●● ●

●

●●

●

●

●

●

●Sepal.Width

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●●

●

●●

●

●●

●

●●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●●

●

●●

●

●●

● ●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●●

●

●●●

●●

●

●●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●●

● ●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●●

●

●

●

●●

● ●●

●

●●

●

●

●

●

●

●●●● ●

●● ●● ● ●●

●● ●

●●●

●●

●●

●

●●

●● ●●●● ●● ●●

● ●●●●

●●●●●

●●

● ●●

●●

●

●

●●●

●

●

●●

●●

●

●

●●●

●

●

●

●

●●

● ●●

●

●

●●●

●

●

● ●●

●●●

●●

●

●

●●● ●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●●●●

●●

●

●

●

●

●

●●

●●

●●

●●

●

●

●

●

●●

●

●●

●●

●●

●●

●●

●

●● ●● ●

●●●● ● ●●

●● ●

●●●

●●

●●

●

●●

● ●●●●● ● ●●●● ●●●

●●● ●●

●

●●

● ●●

●●

●

●

●●●

●

●

●●

●●

●

●

●●●

●

●

●

●

●●

●●●

●

●

●●●

●

●

● ●●

●●●

●●

●

●

● ●●●

●

●

●

●

●●

●

●

●

●

●●

●●

●

● ●●

●

●●

●

●

●

●

●

●●

● ●

●●

●●

●

●

●

●

●●

●

●●

●●

●●

●●

●●

●

Petal.Length

12

34

56

7

●●●●●

●●●●●●●●

●●●●●

●●

●●

●

●●● ●●●●● ●●●●●●

●●●

●●●●

●

●●●●●

●●●

●

●●●

●

●

●●

●●

●

●

●●●

●

●

●

●

●●

●●●

●

●

●●●

●

●

●●●

●●●

●●

●

●

●●●●

●

●

●

●

●●

●

●

●

●

●●

●●

●

● ●●

●

●●

●

●

●

●

●

●●

●●

●●

●●

●

●

●

●

●●

●

●●

●●

●●

●●

●●

●

4.5 5.5 6.5 7.5

0.5

1.0

1.5

2.0

2.5

●●●● ●

●●

●●●

●●●●

●

●●● ●●

●

●

●

●

● ●

●

●●●●

●

●●●● ●

●● ●

●●●

●

●●

●● ●●

●● ●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●●

● ●

●

●

●●●

●

●●

●●

●●●●

●

●

●

●●● ●

●

●

●

●

●

●

●●

●●●

●

●●

●●

●●

●

●●

●

●

● ●

●

●

●●●

●

●

●●

●

●●

●●

●●

●

●●

●

●

●

●

●●

●

●

●● ●● ●

●●●●

●●●

●●●

●●● ●●

●

●

●

●

●●

●

●●●●

●

●●●● ●

●● ●

●●●

●

●●

●● ●●

●●●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●●

●

●

●●●

●

●●

●●

● ●●●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●●●

●

●●

●●

●●

●

●●

●

●

●●

●

●

●● ●

●

●

●●

●

●●

●●

●●

●

●●

●

●

●

●

●●

●

●

1 2 3 4 5 6 7

●●●●●

●●●●●●●

●●●

●●●●●

●

●

●

●

●●

●

●●●●

●

●●●●●●

●●●●●

●

●●

●●●●

●● ●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●●● ●

●

●

●●

●

●

●●●

●

●●●●

●

●

●

●●●●

●

●

●

●

●

●

●●

●●●

●

●●

●●

●●

●

●●

●

●

● ●

●

●

●●●

●

●

●●

●

●●

●●

●●

●

●●

●

●

●

●

●●

●

●

Petal.Width

Anderson's Iris Data −− 3 species

pairs(iris[1:4], bg = c("red", "green3",

"blue")[unclass(iris$Species)])

Image or contours

The functions image or contour are useful to explore threedimensional data or to illustrate distributions in two dimensions.

−4 −2 0 2 4

−4

−2

02

4

0.02

0.04

0.06

0.08

0.1 0.12

0.1

4

−4 −2 0 2 4−

4−

20

24

Aspect ration of plotThe aspect ratio of a plot is very important:Cleveland/McGill recommended an average slope of about 45◦ asthe eye is most sensitive to departures from 45◦.

●●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

● ●●

●

●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●●

●

●

●

●

●

●

●

−3 −2 −1 0 1 2 3

−6

−4

−2

02

4

Normal Q−Q Plot

Theoretical Quantiles

Sam

ple

Qua

ntile

s

●●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●●●

●

●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●

−3 −1 0 1 2 3−

6−

4−

20

24

Normal Q−Q Plot

Theoretical Quantiles

Sam

ple

Qua

ntile

s

Arranging Several Plots on Single Page

par(mfrow=c(2,3))

for(i in 1:6) { plot(1:10) }

●

●

●

●

●

●

●

●

●

●

2 4 6 8 10

24

68

10

Index

1:10

●

●

●

●

●

●

●

●

●

●

2 4 6 8 10

24

68

10

Index1:

10

●

●

●

●

●

●

●

●

●

●

2 4 6 8 10

24

68

10

Index

1:10

●

●

●

●

●

●

●

●

●

●

2 4 6 8 10

24

68

10

Index

1:10

●

●

●

●

●

●

●

●

●

●

2 4 6 8 10

24

68

10

Index

1:10

●

●

●

●

●

●

●

●

●

●

2 4 6 8 10

24

68

10

Index

1:10

The layout function allows to divide the plotting device intovariable numbers of rows and columns with the column-widths andthe row-heights specified in the respective arguments.

nf <- layout(matrix(c(1,2,3,3), 2, 2, byrow=TRUE),

c(3,7), c(5,5),respect=TRUE)

for(i in 1:3) { plot(1:10) }

●

●

●

●

●

●

●

●

●

●

2 4 6 8

24

68

10

Index

1:10

●

●

●

●

●

●

●

●

●

●

2 4 6 8 10

24

68

10

Index

1:10

●

●

●

●

●

●

●

●

●

●

2 4 6 8 10

24

68

10

Index

1:10

Make your graphical display easy to understand

I Add labels to your axis (with appropriate font and font size)

I Control the scale of the axes using the commands xlim andylim. Also check the command axes.

I Add clear captions

I Use appropriate colors

I ....

Save a figure in a pdf file

I recommend you to always save your figure in pdf as it makes iteasier to include in LaTeX. This can be done in R using thefollowing command line:

pdf("filename.pdf")

...

dev.off()

You can also specify the size of the figure with the options width

and height – the measures are in inches.

To watch at home...

TED talk on The beauty of data visualization:

David McCandless turns complex data sets (like worldwide militaryspending, media buzz, Facebook status updates) into beautiful,simple diagrams that tease out unseen patterns and connections.Good design, he suggests, is the best way to navigate informationglut – and it may just change the way we see the world.

Link: http://www.ted.com/talks/david_mccandless_the_

beauty_of_data_visualization