Quantifying Risk of End Result Specifications
-
Upload
california-asphalt-pavement-association -
Category
Engineering
-
view
243 -
download
5
Transcript of Quantifying Risk of End Result Specifications
![Page 1: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/1.jpg)
1
Quantifying Risk of End Result
Specifications
CalAPA Fall Conference October 25 – 26, 2017
Sacramento, CA
Tony Limas
Granite Construction Inc.
![Page 2: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/2.jpg)
Today’s Discussion
2
Specification Tolerances – Should We care? Evolution of Specifications Best Practice for Establishing Specification Limits Types of Risk Measuring Risk - Examples Risk vs. Number of Observations Questions…
![Page 3: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/3.jpg)
Managing Risk
3
![Page 4: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/4.jpg)
Statistics – Ugh…
4
![Page 5: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/5.jpg)
Specification Tolerances When specifications contain unreasonable or
unattainable material tolerances it is likely that
a contractor, providing a product using all the
care and skill normally exercised within the
industry, will fail to meet the specified
acceptance requirements. Such specifications
are said to be unbalanced assigning excessive
risk to the contractor and thus not suitable for
use.
FHWA - NHI Course No. 13442
5
![Page 6: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/6.jpg)
Evolution of Specifications
Pre 1958
Acceptance Primarily based on Inspections vs Test Results
Specification tolerances Primarily based on Anecdotal or
shoot from the hip criteria
Contractors Struggled to Meet Acceptance Limits
Post 1958
1958 AASHTO Road Test Collected “Real Time” Test Data
Variability of Material Properties Better Understood
Information was used to establish End-Result Spec Limits 6
![Page 7: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/7.jpg)
Evolution of Specifications (con’t)
Specifications Must:
Recognize Total Variability of Materials and Construction
Properties (Standard Deviation)
Must Apply Reasonable Risk to All Parties
7
![Page 8: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/8.jpg)
Types of Risks
Buyer’s Risk β = Risk of Accepting “Bad” Material
Seller’s Risk α = Risk of Rejecting “Good” Material
FHWA Recommended Seller’s Risk (α): 5.0% Max.
Typically 2s About the Mean
8
![Page 9: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/9.jpg)
9
Are Risks
Determine Buyer’s and
Seller’s Risk of
Proposed Spec Limits
Modify Spec limits,
Sample Size
and/or lot Size
Finalize The Specifications
No
Yes Acceptable
Yes
Specification
Development
Process
![Page 10: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/10.jpg)
Measuring Risk Standard Deviation Is the measure of dispersion of a set of data from its mean. It measures
the absolute variability of a distribution; the higher the dispersion or variability,
the greater is the standard deviation and greater will be the magnitude of the
deviation of the value from their mean.
10
![Page 11: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/11.jpg)
Measuring Risk
Transportation Related Material Properties Are:
Symmetrically/Normally Distributed About the Mean
Mean
Test Results Test Results
11
![Page 12: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/12.jpg)
Measuring Risk (con’t) N
o. o
f S
am
ple
s
12
Mean
![Page 13: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/13.jpg)
Material Properties Distribution PWL N
o. o
f S
am
ple
s
-3s -2s -1s +1s +2s +3s
68%
95%
99.7%
Air Voids (S) = 0.75
Target ± 0.75%
13
PWL = 68%
Target
![Page 14: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/14.jpg)
Sellers Risk (α) N
o. o
f S
am
ple
s
-3s -2s -1s +1s +2s +3s
68%
95%
99.7%
Air Voids (s) = 0.75
Target ± 0.75%
16%
14
Sellers Risk
α = 32%
![Page 15: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/15.jpg)
Sellers Risk (α) N
o. o
f S
am
ple
s
-3s -2s -1s +1s +2s +3s
68%
95%
99.7%
Air Voids (s) = 0.75
Target ± 1.5%
2.5%
15
Seller’s Risk
α = 5%
Target
![Page 16: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/16.jpg)
Sellers Risk (α) N
o. o
f S
am
ple
s
-3s -2s -1s +1s +2s +3s
68%
95%
99.7%
Air Voids (s) = 0.75
Target ± 1.5%
16
Target Mean
Seller’s
Risk >5.0%
![Page 17: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/17.jpg)
Risk (α) Evaluation Examples
Based on Specification Tolerances (vs SD)
USL = Upper Specification limit
LSL = Lower Specification Limit
Mean or Target Value
Standard Deviation: Population (S) or Sample (s)
Z Score Chart for Normal Distribution
17
![Page 18: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/18.jpg)
Risk Evaluation Examples
Z Score Chart
18
![Page 19: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/19.jpg)
Risk (α) Evaluation Examples
Z Score Chart
(USL - x̄)/S= Z score
(LSL - x̄)/S= Z score
Z of ≥1.96 = ≤ 5.0% Risk
19
![Page 20: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/20.jpg)
Risk Evaluation Examples
Caltrans Proposed Binder Content Tolerance
CT Proposal ± 0.3% (1 observation)
Evaluate the Risk Associated with the Proposed Limits
20
![Page 21: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/21.jpg)
Risk Evaluation Examples
What is the Variation for Binder Content?
Based on Statewide Pooled Data from QC/QA Projects
Population Standard Deviation (S) = 0.20
21
![Page 22: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/22.jpg)
22
Seller’s Risk (With 0.3% Tolerance)
S
(n=1)x̄ PWL
Risk
(α)
0.20 5.0 86 14%
Binder Content
1.7 3.7 4.7 5.0 5.3 6.3 7.3
Upper limitLower limit
Target
5.0%
16%7%7%
NTS
86%
![Page 23: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/23.jpg)
23
Sellers (other) Risk(With 0.3% Tolerance)
S
(n=2)*x̄ PWL
Risk
(α)
0.141 5.0 96 4%
Binder Content
1.7 3.7 4.7 5.0 5.3 6.3 7.3
Upper LimitLower Limit
Target
5.0%
16%2%2%
NTS
96%
*Avg. of Two Independent Samples
15
![Page 24: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/24.jpg)
Balancing Risk and Cost A
gen
cy a
nd
/or
Co
ntr
act
or
Ris
k
Dir
ect
Co
st (
$)
1 2 3 4 5 6 7
Number of Test Samples (n)
![Page 25: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/25.jpg)
Total Variability (SD)
Variability = variability + variability + variability
(sampling) (test method) (mat./const.)
S2QC/QA = S2
s + S2t + S2
m/c
![Page 26: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/26.jpg)
3
26
Sellers Risk(With 0.4% Tolerance)
S
(n=1)x̄ PWL
Risk
(α)
0.20 5.0 95 5%
Binder Content
3.8 4.2 4.6 5.0 5.4 5.8 6.2
Upper LimitLower Limit
Target
5.0%
16%2.5% 1.72.5%
NTS
95%
![Page 27: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/27.jpg)
27
Seller’s Risk(With -0.3 +0.5% Skewed Tolerance)
S
(n=1)PWL
Risk
(α)
0.20 5.0 92 8%
Binder Content
3.8 4.2 4.7 5.0 5.5 6.0 6.5
Upper LimitLower Limit
Target
5.0%
16%1% 1.77%
NTS
49%43%
x̄
![Page 28: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/28.jpg)
28
Seller’s Risk(With -0.3 +0.5% Skewed Tolerance)
S
(n=1)PWL
Risk
(α)
0.20 5.1 95 5%
Binder Content
4.7 5.1 5.5
Upper LimitLower Limit
Mean
5.1%
16%2.5%2.5%
NTS
47.5%47.5%
x̄Target Value + 0.10
![Page 29: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/29.jpg)
29
Skewed Binder Content Effect on Volumetrics
Air Voids
- 4.0 +
Design Binder
5.0%
16%
NTS
Target Binder + 0.105.1%
Percent Defective
![Page 30: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/30.jpg)
3
30
Seller’s Risk(With 0.4% Tolerance)
S
(n=1)x̄ PWL
Risk
(α)
0.20 5.0 95 5%
Binder Content
3.8 4.2 4.6 5.0 5.4 5.8 6.2
Upper LimitLower Limit
Target
5.0%
16%2.5% 1.72.5%
NTS
95%
![Page 31: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/31.jpg)
Evaluating Risk Examples (con’t)
Local Agency Relative Density Specification Local Agency 92 – 97 % (n=1)
Contractors’ Could Not Meet Specifications
Spec was Evaluated to Determine Contractors Risk
31
![Page 32: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/32.jpg)
Risk Evaluation Examples
What is the Variation for Relative Density?
Sample Standard Deviation (s) = 1.84
Based on <30 observations from projects
32
![Page 33: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/33.jpg)
9
33
Seller’s Risk(With 2.5% Tolerance)
S
(n=1)x̄ PWL
Risk
(α)
1.84 94.5 82 18%
Relative Density92.0 94.5 97.0
Upper LimitLower Limit
Target
16%9%9%
NTS
82%
![Page 34: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/34.jpg)
34
Seller’s Risk(With 4.5% Tolerance)
S
(n=1)x̄
PW
L
Risk
(α)
1.84 94.5 98 2.0%
Relative Density90.0 94.5 99.0
Upper LimitLower Limit
Target
16%1%1%
NTS
98%
Option 1= Open Spec Band ±2.0%
![Page 35: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/35.jpg)
Relative Density Specifications
35
Relative Density Pay Factor
97.1 0r Higher (Over-asphalted mix) 90% Pay Factor
92-97% (Ideal) 100% Pay Factor
89 – 91.9 (Marginal Air Voids) 85% Pay Factor
88.9 Or Less Reject (RQL)
Pay Factors
For all asphalt concrete pavement subject to acceptance testing, the
finished asphalt concrete pavements that do not conform to the
specified relative compaction requirements will be paid for using the
following pay factors:
![Page 36: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/36.jpg)
36
Buyers Risk (β)(With 4.5% Tolerance)
S
(n=1)x̄
Buyer’s
Risk (β)
1.84 88.9 28%
Relative Density
88.9 90.0
Lower Limit
16%28%
NTS
RQL
Reject
Rejectable Quality Limit = 89.9
![Page 37: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/37.jpg)
37
Buyers Risk (β)
88.9 LSL Target USL
16%Buyers
Risk (β)
NTS
AQL
RQL
α
![Page 38: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/38.jpg)
38
Seller’s Risk(With 2.5% Tolerance)
S
(n=2)*x̄ PWL
Risk
(α)
1.30 94.5 98 2%
Relative Density92.0 94.5 97.0
Upper LimitLower Limit
Target
16%1%1%
NTS
98%
Avg. of Two Independent Samples
![Page 39: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/39.jpg)
Risk vs Number of Observations (n)
The myth of the Single Representative Sample:
“The idea persists that a test on a single sample shows
the "true" quality of the material, and that if any test result
is not within some limit, there is something wrong with the
material, construction, sampling or testing. Thus, terms
such as investigational, check, and referee samples are
in common use to either confirm or document these
"failures.“ Nature dislikes identities; variation is the rule.
Therefore, any acceptance or process control sampling
must account for variability of materials or construction.
Multiple sampling accomplishes this objective”
FHWA - NHI Course No. 13442
39
![Page 40: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/40.jpg)
Risk Vs Number of Test A
gen
cy a
nd
/or
Co
ntr
act
or
Ris
k
1 2 3 4 5 6 7
Number of Test Samples (n)
Best Practice:
Never make a decision to reject material
based on a single observation!
40
![Page 41: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/41.jpg)
3
41
Seller’s Risk(QC/QA Multiple Observations)
S
(n=1)x̄ PWL
Risk
(α)
0.20 5.0 95 5%
Binder Content
3.8 4.2 4.6 5.0 5.4 5.8 6.2
Upper LimitLower Limit
Target
16%2.5% 1.72.5%
NTS
95%
![Page 42: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/42.jpg)
42
Buyer’s/Seller’s Risk (Standard Spec With Single Observation)
Asphalt Content
16%
NTS
RQL Population
.
AQL Population
![Page 43: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/43.jpg)
Single Test Specification
FHWA Peer Review Team Recommendation:
For other items without pay factors it is recommended that if one
test falls outside the specification limit then another test will
be taken. If the specification limit is met on the subsequent
test, production continues without any penalties.
If the second consecutive test falls outside the specification limit,
production will cease until the contractor demonstrates that the
specification limit can be met.
43
![Page 44: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/44.jpg)
44
Sellers Risk (Test Result with Check or Referee Test)
Asphalt Content
16%
NTS
RQL Population
.
.
.
AQL Population
![Page 45: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/45.jpg)
45
Sellers Risk (Population Defined with Additional Test)
Asphalt Content
16%
NTS
RQL Population
.
AQL Population
.
.
.. .. .....
...
![Page 46: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/46.jpg)
Solution To Single Test Dilemma
The 2007 FHWA Peer Review Team Recommendations:
Pay equations need to be established for binder content for standard
and method specifications
Penalties should be commensurate with the performance of the
pavement.
46
![Page 47: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/47.jpg)
And Remember
Never Reject Material Based on a Single Observation!
47
![Page 48: Quantifying Risk of End Result Specifications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6491e87f8b9a52568b4c2d/html5/thumbnails/48.jpg)
48