Post on 14-Nov-2014
description
EPSRC, Cambridge March 2008
Statistics and Experimental Design
Shirley Coleman
Industrial Statistics Research Unit
EPSRC, Cambridge March 2008
Outline of Talk
Purpose of Stats and Experimental Design
History and Applications Skill set needed Examples
Importance of planning Subtleties
Summary
EPSRC, Cambridge March 2008
Purpose of Statistics and Experimental Design
Investigate Make objective decisions Experiment efficiently Ensure reproducibility Build a model Predict Monitor ……….
EPSRC, Cambridge March 2008
History of experimental design
Agriculture http://www.rothamsted.ac.uk/ Oldest agricultural research station Rothamsted, Park Grass, 1856 Cockle Park, Palace Leas, 1897
Palace Leas, Cockle Park, Northumberland
1897
Park Grass, Rothamsted
Herts, 1856
Long term grass experiments
EPSRC, Cambridge March 2008
Palace Leas, Cockle Park
EPSRC, Cambridge March 2008
Palace Leas
Experiment on effect of fertilisers on hay yield Can also look at root structure, species,…
Meteorological station in next field from 1898
14 plots of land 8 for an experiment with N,P and K 6 additional (some abandoned in WWII)
Results were apparent in1898
Fabulous set of data
EPSRC, Cambridge March 2008
Palace Leas plots
N P2O5 K2O1 20 17 30 342 203 2 20/0 17 30 344 2 20/05 4 40/0/0/0 17 30 3467 358 609 67
10 35 6011 35 6712 60 6713 35 60 6714 100 66 100
Plot Year of Cycle
FYM
t ha-1 Fertilizer (kg ha-1)
EPSRC, Cambridge March 2008
Palace Leas plots
N P2O5 K2O1 20 17 30 342 203 2 20/0 17 30 344 2 20/05 4 40/0/0/0 17 30 3467 358 609 67
10 35 6011 35 6712 60 6713 35 60 6714 100 66 100
Plot Year of Cycle
FYM
t ha-1 Fertilizer (kg ha-1)
EPSRC, Cambridge March 2008
Rothampsted, Park Grass
Term
Effect
AC
AB
ABC
C
BC
A
B
1400120010008006004002000
316Factor NameA NB PC K
Pareto Chart of the Effects(response is HAY, Alpha = .05)
Lenth's PSE = 84
EPSRC, Cambridge March 2008
Importance of publishing!
COLEMAN, S.Y., SHIEL, R.S & EVANS, D.A. (1987) The effect of weather and nutrition on the yield of hay from Palace Leas meadow hay plots, at Cockle Park
Experimental Farm, over the period from 1897 to 1980. Grass and Forage Science 42, 353-358.
EPSRC, Cambridge March 2008
corr with smd
hay k
g/ha
-0.30-0.35-0.40-0.45-0.50-0.55
7000
6000
5000
4000
3000
2000
NPK
PK
NK
NP
K
P
N
control
FYM
FYM
FYM
FYM
FYM
Scatterplot of hay kg/ ha vs corr with smd
EPSRC, Cambridge March 2008
History and Applications
Agriculture Given industrial slant by G.Taguchi
(b 1924, published 1951, in NE 2000) Used in manufacturing Gradually used in
Business, service, health, Finance, marketing,
Bespoke nomenclature
EPSRC, Cambridge March 2008
Skill set needed
Logical thinking Attention to detail Presentation skills Analytical tools Knowledge of where to go next
Optimisation, RSA, simulation……
EPSRC, Cambridge March 2008
Examples
Pressure, temperature, pointer setting, haul off speed, welding current, granule size, nozzle width,….
N, P, K
Beer experiment
EPSRC, Cambridge March 2008
Beer experiment
(What affects frothing when pouring beer?)
EPSRC, Cambridge March 2008
EPSRC, Cambridge March 2008
Management Methodology
Six Sigma
Define Measure Analyse Improve Control
Lean
PDCA
EPSRC, Cambridge March 2008
Lean Six Sigma
Lean focuses on removing complexity
Six Sigma focuses on process improvement
Lean Six Sigma attempts to combine the best of both
Lean involves less statistics and is very popular in
some applications, such as healthcare.
EPSRC, Cambridge March 2008
Define problem
QI tools, brain-storming, team roles Identifying factors and levels
Measurement issues
Decide what, how, when, who and where to measure Analyse
Use current knowledge, Set experimental design and pilot
Improve
Run experiment and analyse Control
Recommend method for best or least froth Look for other opportunities to use what has been learnt
EPSRC, Cambridge March 2008
Team roles
Secretary
Waiter
Pourer
Measurer
Observers
EPSRC, Cambridge March 2008
Factors
Materials (beer type, temperature of bottles)
Machines (glass shape)
Man (steadiness)
Milieu (pressure, humidity, temperature)
Method (angle, speed, height, time opened)
Measures (volume, height)
EPSRC, Cambridge March 2008
Experimental design
More information out requires more data in
However, statistically designed experiments
help reduce the number of trials with least
reduction in information
Eg up to 7 factors can be tested in 8 trials
Taguchi uses Plackett-Burman designs
Eg up to 11 factors can be tested in 12 trials
EPSRC, Cambridge March 2008
Saturated L8 design
Trial A B C D E F G
1 -1 -1 1 -1 1 1 -1
2 1 -1 -1 -1 -1 1 1
3 -1 1 -1 -1 1 -1 1
4 1 1 1 -1 -1 -1 -1
5 -1 -1 1 1 -1 -1 1
6 1 -1 -1 1 1 -1 -1
7 -1 1 -1 1 -1 1 -1
8 1 1 1 1 1 1 1
EPSRC, Cambridge March 2008
Taguchi saturated L8 design
Trial A B C D E F G
1 1 1 1 1 1 1 1
2 1 1 1 2 2 2 2
3 1 2 2 1 1 2 2
4 1 2 2 2 2 1 1
5 2 1 2 1 2 1 2
6 2 1 2 2 1 2 1
7 2 2 1 1 2 2 1
8 2 2 1 2 1 1 2
EPSRC, Cambridge March 2008
Identifying factors and levels 3 factors each at 2 levels
Factors - +
Beer type Belgian French
Glass type Flat Rounded
Glass angle Upright Tilted
Response Froth height (mm)
EPSRC, Cambridge March 2008
Orthogonal array
Trial Beer Glass Angle
1 -1 -1 -1
2 1 -1 -1
3 -1 1 -1
4 1 1 -1
5 -1 -1 1
6 1 -1 1
7 -1 1 1
8 1 1 1
EPSRC, Cambridge March 2008
Experimental trials
Eg Belgian beer poured into a flat
bottomed glass without tilting
Eg French beer poured into a round
bottomed glass without tilting
EPSRC, Cambridge March 2008
Randomisation
Reduces false replication and bias
Eg drug trial
first group of mice have treatment and second have placebo,
if mice are selected by grabbing from cage,
fittest are caught last and placebo can
appear to be better than the treatment
EPSRC, Cambridge March 2008
Team roles
Secretary : Read trial and register result
Waiter : Pick the right glass and bottle
Pourer : Pour the beer into the glass
Measurer : Use the ruler to read off the froth
Observers: Note that procedures are followed
EPSRC, Cambridge March 2008
Mean o
f Fr
oth
1-1
30
25
20
15
10
1-1 1-1
Beer Glass Angle
Main effects plot
EPSRC, Cambridge March 2008
Factors and factor levels
Factors - +
Beer type Belgian French
Glass type Flat Rounded
Glass angle Upright Tilted
Response Froth height (mm)
EPSRC, Cambridge March 2008
Interaction plot: glass*angle
Angle
Mean
1-1
40
30
20
10
0
Glass-11
EPSRC, Cambridge March 2008
ANOVA table for beer
Source DF SS MS F P
Beer 1 138 138 4.78 0.06
Glass 1 189 189 6.54 0.03
Angle 1 2233 2233 77.26 0.00
Glass*Angle 1 638 638 22.06 0.00
Beer*Angle 1 0 0 0 0.96
Beer*Glass 1 11 11 0.37 0.56
Error 9 260 29
Total 15 3468
EPSRC, Cambridge March 2008
Graphical analysis of significance
Standardized Effect
Perc
ent
5.02.50.0-2.5-5.0-7.5-10.0
99
95
90
80
70
605040
30
20
10
5
1
Factor NameA BeerB GlassC Angle
Effect TypeNot SignificantSignificant
BC
C
B
Normal Probability Plot of the Standardized Effects(response is Froth, Alpha = .05)
EPSRC, Cambridge March 2008
Residual
Perc
ent
1050-5-10
99
90
50
10
1
Fitted Value
Resi
dual
403020100
10
5
0
-5
-10
Residual
Fre
quency
7.55.02.50.0-2.5-5.0-7.5-10.0
8
6
4
2
0
Observation Order
Resi
dual
16151413121110987654321
10
5
0
-5
-10
Normal Probability Plot of the Residuals Residuals Versus the Fitted Values
Histogram of the Residuals Residuals Versus the Order of the Data
Residual Plots for Froth
EPSRC, Cambridge March 2008
Context and implications
Know which glasses to buy
Know how to pour
Adapt to alternative requirements, eg no froth
Apply methodology in other contexts
EPSRC, Cambridge March 2008
Other examples
Human Resources Training requirements Explore effects of commands given
Accounts Explore effect of timing and nature
of reminders Preferences
Explore trade-offs Conjoint analysis
EPSRC, Cambridge March 2008
Conjoint Analysis
Customers CONsider JOINTly and give their opinions,
trading off factors to reach a desired end. For example, to design a conference, consider:
In University or hotel 2 days or 3 days Evening speaker or not 20 minute or longer talks ….
Present the different options, eg in a questionnaire and analyse results
Helps determine what people value in different product features
or service attributes.
EPSRC, Cambridge March 2008
Format of questions (ENBIS)
‘How successful do you think the following
conference would be?’
An informal, applied conference held in a
conference suite that has an evening session,
it mostly features presentations involving
industrialists’
1= not very successful 5= very successful
EPSRC, Cambridge March 2008
Results - ENBIS
ANOVA with categorical responses gave
Applied vs theoretical and
Industrialists vs academics
as important factors
EPSRC, Cambridge March 2008
Results - ENBIS
Taguchi style analysis for the variability of
responses showed
significantly greater variation in views about
applied talks than for theoretical talks
significantly greater variation for
workshops than for presentations
EPSRC, Cambridge March 2008
Online Conjoint Analysis
Used in automobile feature testing to find the features
consumers are willing to give up in order to get something
they value more.
Outcomes help guide new product design, old product
redesign or repositioning decisions.
Used in travel industry to determine how much consumers
are willing to pay for a ticket in order to get more leg room.
Building “Voice of Customer” into Product DevelopmentSource: Siegel (2004)
EPSRC, Cambridge March 2008
Other examples
Kansei Engineering Incorporates emotion into design
Orthogonal array for products Semantic scales for emotion Sample of customers
Highly developed in Japan KENSYS 2003-6 in Europe
Do you think that people would be happy wearing these products? Please circle the number which is closest to your feelings for each picture. Picture Number
extremely unhappy
unhappy
neither unhappy or happy
happy
extremely happy
1 1
2
3
4
5
2 1
2
3
4
5
EPSRC, Cambridge March 2008
Analysis relates emotions to design
Model Response is happiness Design factors are colour, style, heel, etc
Aim is to advise designers Which shoes will give the desired emotional
response How to develop a balanced portfolio
Similar results for logistic regression or ANOVA
EPSRC, Cambridge March 2008
Design a waiting room
Design factors
Sofa or chair Lighting soft or bright Service desk Windows
Other factors
Waiting time max 30 minutes ……..
EPSRC, Cambridge March 2008
Please rate where you feel
the image fits on each of the
following semantic scales.
Comfortable Uncomfortable
At Ease Uneasy
Efficient Inefficient
Trustworthy Untrustworthy
Calm Stimulating
Boring Interesting
1 2 3 4 5
EPSRC, Cambridge March 2008
Design Rules
ComfortableComfortable
Seating (sofa)Windows (yes)Lighting (soft)Service desk (yes)
Seating (chair)Service desk (yes)Max Waiting Time (30min)
EfficientEfficient
EPSRC, Cambridge March 2008
Key Drivers of Satisfaction
[Ease of Use][Navigation, clarity, fresh/relevant content, etc.]
[Graphic Style][Colour, layout, print size, type, no. of photographs, graphics and animation.]
[Perceived channel advantage ][Price, Speed, etc.]
[Privacy and Security ] [Brand, reputation, appearance of the site (more
imp than security logos appearing on website)] [Fulfilment and Reliability]
[Timeliness of service, availability, breadth and depth of
products/services, responsiveness/access (availability of service
personnel, multiple communication channels) and personalisation.]
Source: Baur, Schmidt & Hammersmith, 2006
EPSRC, Cambridge March 2008
Website design process
The BriefThe Brief Trial PagesTrial Pages The PrototypeThe Prototype The PrototypeThe Prototype
Client requirements and goals
LaunchApprovalResponse and
refinement
CLIENT
WEB DESIGN FIRM
Strategic planning, engineering
Style book, training, quality tests
Final design, testing and coding
Design content and marketing
User SurveyConjoint Analysis
Kansei Design for
Emotional Appeal
Content & NavigationSPC
Building “Voice of Customer” into Design by courtesy of A.Parulekar
Web Designfor
Sticky Relationship
EPSRC, Cambridge March 2008
Statistics
Quantitative management
Software Graphical Tabular
Comparative tests, ANOVA
Statistical models
Regression Logistic regression
Multivariate analysis
EPSRC, Cambridge March 2008
Tutorials, eg MINITAB
These easy step-by-step tutorials introduce you to the
Minitab environment and provide a quick overview of
some of Minitab's most important features. The tutorials
are designed to explain the fundamentals of using Minitab:
how to use the menus and dialog boxes, how to manage
and manipulate data and files, how to produce graphs, …. Session One: Graphing Data Session Two: Entering and Exploring Data Session Three: Analyzing Data Session Four: Assessing Quality Session Five: Designing an Experiment
EPSRC, Cambridge March 2008
Data exploration, eg SAS JMP
Key processes and inputs associated with excessive variation in 60-minute dissolution
Recursive Partitioning Decision Tree
EPSRC, Cambridge March 2008
Importance of planning
Collecting all the data that is needed
Getting the right people involved
Getting the right materials in place
Ensuring time for input from others
Poke yoka
EPSRC, Cambridge March 2008
Subtleties include
Measurement issues
Sample sizes
Use continuous variables where possible Robustness
Bias and confounding, especially time trends
Piloting
EPSRC, Cambridge March 2008
Summary
All of the above
Thanks