04 reports

If you’re using a laptop, start installing latex, from the instructions on the website

Thursday, 2 September 2010

Office hours: before class.

Lab access: you should now have it

Hadley Wickham

Stat405Statistical reports

1. More subsetting.

2. Missing values.

3. Statistical reports: data, code, graphics & written report

Saving results

# Prints to screen

diamonds[diamonds$x > 10, ]

# Saves to new data frame

big <- diamonds[diamonds$x > 10, ]

# Overwrites existing data frame. Dangerous!

diamonds <- diamonds[diamonds$x < 10,]

diamonds <- diamonds[1, 1]diamonds

# Uh oh!

rm(diamonds)str(diamonds)

# Phew!

Your turn

Create a logical vector that selects diamonds with equal x & y. Create a new dataset that only contains these values.

Create a logical vector that selects diamonds with incorrect/unusual x, y, or z values. Create a new dataset that omits these values. (Hint: do this one variable at a time)

equal_dim <- diamonds$x == diamonds$yequal <- diamonds[equal_dim, ]

y_big <- diamonds$y > 10z_big <- diamonds$z > 6

x_zero <- diamonds$x == 0 y_zero <- diamonds$y == 0z_zero <- diamonds$z == 0zeros <- x_zero | y_zero | z_zero

bad <- y_big | z_big | zerosgood <- diamonds[!bad, ]

Missing values

Typically removing the entire row because of one error is overkill. Better to selectively replace problem values with missing values.

In R, missing values are indicated by NA

Data errors

Expression Guess Actual

5 + NA

NA / 2

sum(c(5, NA))

mean(c(5, NA)

NA < 3

NA == 3

NA == NA

NA behaviour

Missing values propagate

Use is.na() to check for missing values

Many functions (e.g. sum and mean) have na.rm argument to remove missing values prior to computation.

# Can use subsetting + <- to change individual # values

diamonds$x[diamonds$x == 0] <- NAdiamonds$y[diamonds$y == 0] <- NAdiamonds$z[diamonds$z == 0] <- NA

y_big <- !is.na(diamonds$y) & diamonds$y > 10diamonds$y[y_big] <- diamonds$y[y_big] / 10z_big <- !is.na(diamonds$z) & diamonds$z > 6diamonds$z[z_big] <- diamonds$z[z_big] / 10

What happens if you don’t remove missing values? Why?

Your turn

Statistical reports

Regardless of whether you go into academia or industry, you need to be able to present your findings.

And you should be able to do more than just present them, you should be able to reproduce them.

Data (.csv)+

Code (.r)+

Graphics (.png, .pdf)+

Written report (.tex)

In one directory

Set your working directory to specify where files will be saved by default.

From the terminal (linux or mac): the working directory is the directory you’re in when you start R

On windows: File | Change dir.

On the mac: ⌘-D

Working directory

DataSo far we’ve just used built in datasets

Next week we’ll learn how to use external data

Workflow

At the end of each interactive session, you want a summary of everything you did

Two options:

Save everything that you did with savehistory(filename.r) then remove the unimportant bits

Build up the important bits as you go

Up to you - I prefer the second

R editor

Linux: gedit(copy and paste - see website)

Windows: File | New Script(press F5 to send line)

Mac: File | New document (press command-enter to send)

Code is communication!

Code presentationUse comments (#) to describe what you are doing and to create scannable headings in your code

Every comma should be followed by a space, and every mathematical operator (+, -, =, *, / etc) should be surrounded by spaces. Parentheses do not need spaces

Lines should be at most 80 characters. If you have to break up a line, indent the following piece

qplot(table,depth,data=diamonds)qplot(table,depth,data=diamonds)+xlim(50,70)+ylim(50,70)qplot(table-depth,data=diamonds,geom="histogram")qplot(table/depth,data=diamonds,geom="histogram",binwidth=0.01)+xlim(0.8,1.2)

# Table and depth -------------------------

qplot(table, depth, data = diamonds)qplot(table, depth, data = diamonds) + xlim(50, 70) + ylim(50, 70)

# Is there a linear relationship?qplot(table - depth, data = diamonds, geom = "histogram")

# This bin width seems the most revealing qplot(table / depth, data = diamonds, geom = "histogram", binwidth = 0.01) + xlim(0.8, 1.2)# Also tried: 0.05, 0.005, 0.002

# Table and depth -------------------------

qplot(table, depth, data = diamonds)qplot(table, depth, data = diamonds) + xlim(50, 70) + ylim(50, 70)

# Is there a linear relationship?qplot(table - depth, data = diamonds, geom = "histogram")

# This bin width seems the most revealingqplot(table / depth, data = diamonds, geom = "histogram", binwidth = 0.01) + xlim(0.8, 1.2)# Also tried: 0.05, 0.005, 0.002

Graphics

Saving graphics# Uses size on screen:ggsave("my-plot.pdf")ggsave("my-plot.png")

# Specify sizeggsave("my-plot.pdf", width = 6, height = 6)

# Saves file in working directory# (where you started R from)

PDF PNG

Vector based (can zoom in infinitely)

Raster based(made up of pixels)

Good for most plots

Good for plots with thousands of

points

Your turn

Recreate some of the graphics from previous lectures and save them.

Experiment with the scale and height and width settings.

Modify the template to include them.

Written report

We are going to use the open source document typesetting system called latex to produce our reports.

This is widespread in statistics - if you ever write a journal article, you will probably write it in latex.

(Not so useful if you’re not in grad school)

Edit-Compile-Preview

Edit: a text document with special formatting

Compile: to produce a pdf

Preview: with a pdf viewer

See web page for system specifics.

Template

Sections

Images

Figures and cross-references

Verbatim input (for code)

Your turn# Get the sample reportwget http://had.co.nz/stat405/\resources/sample-report.zip unzip sample-report.zip

cd sample-reportgedit template.tex &pdflatex template.texevince template.pdf# Experiment!

Your turn

If not on linux, follow the instructions on the class website.

If you feel comfortable, start on homework 2.

Homework

04 reports

Documents

Transcript of 04 reports

3.1 Accessing the Reports · 4/20/2011 · Update # 2011-01 Revised: 04/20/11 Business Implementation Guide to Time on Aid page 3-1 Reports 3. Reports. Several reports are produced

Report WTR 04-16 Annual OCWA Reports for 2015

GAO-04-98 Climate Change: Selected Nations' Reports on

Consumer Reports Kitchen Planning and Buying Guide 2015-04

Audited Consolidated Financial Statements & Reports Required … · · 2013-04-08Audited Consolidated Financial Statements & Reports Required by Government Auditing Standards and

Housekeeping • CEO Report • Legal Counsel Report • Chair Reports · 2018-04-04 · • CEO Report • Legal Counsel Report • Chair Reports ... • Regulatory Affairs and Marketing

Reports Monitoring 2002 04

FAIRA TRANSPARENCY REPORTS 7416 140th Pl NE, Redmond, … · 2017-04-04 · 608 State St. South Kirkland, WA 98033 hello@faira.com +1.800.571.FAIR FAIRA TRANSPARENCY REPORTS 7416

EPAP member reports - Bevan - 2012-04

Agenda Reports Pack (Public) 11/04/2012, 19

Receive Monthly Reports 03-04-14

Kulbir Dhanju’s Market Reports · Warning: These Advanced Market Reports are Dangerous to operate without lesson ... anju 04-725-0107 anju 04-725-0107. anju 04-725-0107 anju 04-725-0107.

Policies and Procedures: American National Standards and Technical Reports · 2018-04-01 · 3 The Importance of Technical Reports Technical reports are educational or tutorial. They

A+LS V3.5 Sample Reports - Online Education Programs & …€¦ · · 2018-04-14REPORT WIZARD OVERVIEW 6 Old Reports (**) HTML/CSV Reports Curriculum Authoring A+® District Driven

Reports and Technical Reports All Technical Reports Collection · 2016-07-04 · Reports and Technical Reports All Technical Reports Collection 2010-04-14 Lifecycle Information of

04 Unit 03 Reports-KC

Essentials Usage Reports · 2020-05-04 · Essentials Usage Reports 1.What are the Essentials Usage Reports? These are new weekly reports you’ll receive every Monday showing how

Understanding Turnitin Originality Reports - Viv’z Blogvivrolfe.com/.../uploads/2016/04/Understanding-Turnitin-Reports.pdf · Understanding Turnitin Originality Reports ... The

04/2016 - Determining Pre Sentence Reports

ONTARIO POWER GENERATION REPORTS 2008 FINANCIAL RESULTS Reports... · 2019-04-12 · ONTARIO POWER GENERATION REPORTS 2008 FINANCIAL RESULTS [Toronto]: Ontario Power Generation Inc.