Uncertainty Analysis for Flow Measurements and Techniques using Standardized Methodology Marian...

Uncertainty Analysis for

Flow Measurements and Techniquesusing

Standardized Methodology

Marian MusteMarian Muste11

Juan Gonzalez-CastroJuan Gonzalez-Castro22

Dongsu KimDongsu Kim11

Kwonkyu YuKwonkyu Yu11

1 IIHR- Hydroscience & Engineering, The University of Iowa2 South-Florida Water Management District, West Palm Beach

Overview

BackgroundBackground Uncertainty Analysis (UA) FrameworksUncertainty Analysis (UA) Frameworks

AIAA (1995)AIAA (1995)

UA Implementation ExampleUA Implementation Example MethodologyMethodology Assessment of Elemental UncertaintiesAssessment of Elemental Uncertainties Customized GUI for UA ImplementationCustomized GUI for UA Implementation

Conclusions Conclusions OutlookOutlook

Background

SAMPLE REQUEST regarding uncertainty analysis originated from a Hydrologic Service

…. Has anybody out there had to defend the validity of an ADCP flow measurement against a legal challenge from a third party?

……When current meters were used to undertake these measurements we could claim that the flow measurement was undertaken in conformance with British

and International standards for current meter gauging and that the current meter had a valid calibration certificate…

In the case where flow measurements are now taken using ADCPs we feel more vulnerable to legal challenges. This is for two reasons:

1. There is no ISO document in place. The Agency has to rely on its own internal document on gauging procedures which is based on the draft ISO document.

2. ADCPs do not have "certificates of calibration" . The only checks on the performance that can be made are against other ADCPs or other types of flow

monitoring equipment.

(posted on the USGS’ Hydro-Acoustics Work Group webpage by R. Iredale, The Environment Agency of England and Wales, 2005)

Over the last 50 years, considerable efforts have been made by professional societies to develop and implement uncertainty

analysis (UA). One of the rigorous UA methodology (based on sound statistical and

engineering concepts):

Guide to the Expression of Uncertainty Measurement (ISO, 1993) - adopted widely by various scientific & research communities, e.g., NIST (1994), NF

ENV 13005 (1999)- the guide recognizes the need for further adaptation for specific areas

Specific adaptations for engineering:- Assessment of Wind Tunnel Data Uncertainty (AIAA, 1995)- Test Uncertainty (ASME, 1998)

Key assumptions/concepts for ISO (1993)-based standards - Gaussian pdf-s for the error sources - 2 sample standard deviations for 95% confidence level - for large samples (N ≥ 10), special procedures for handling small samples - RSS used for combining uncertainties - Taylor-series expansion for propagation of uncertainties

- total uncertainties expressed using confidence intervals

UA Frameworks

Terminology for ISO (1993) - based standards

UA Frameworks

The 3 standards provide the same total measurement uncertainty

ISO (1993) AIAA-S-071-1995 ASME PTC 19.1-1998 Uncertainty of a measurement

input quantity type A standard uncertainty type B standard uncertainty combined standard uncertainty

individual variable bias limit precision limit (differently estimated

for single and multiple tests) total uncertainty

independent parameter systematic uncertainty random uncertainty (differently estimated single and multiple tests)

measurement uncertainty

Uncertainty of a result functional relationship sensitivity coefficients combined standard uncertainty (accounts for correlated errors)

expanded uncertainty (accounts for the level of confidence)

coverage factor f( degrees of freedom and t Distribution)

data reduction equation

bias limit precision limit (differently estimated

for single test with single readings and averaged readings, multiple tests)

sensitivity coefficients combined standard uncertainty

(accounts for correlated errors) uncertainty at specified confidence

level coverage factor f( degrees of freedom

and t Distribution)

derived result systematic uncertainty random standard deviation

(differently estimated for single and multiple tests)

sensitivity coefficients uncertainty of the result (accounts

for correlated errors and the level of confidence)

coverage factor f(degrees of

freedom and t Distribution)

= to tal e rro r = b ias e rro r = p recis ion error

M A G N ITU D E O F X

FR

EQ

UE

NC

Y O

F O

CC

UR

RE

NC

E

X

X

X

tru e

X tru e

k

k+ 1 k

k+ 1

k+ 1XX k

(a ) tw o re a d in g s

(b ) in f in ite n u m b e r o f re a d in g s

Bias error (): fixed, systematic

Bias limit (B): estimate of

Precision error (): random

Precision limit (P): estimate of

Total error:

Engineering approach, simple, clear, widely applied

AIAA (1995)

Determine the data reduction equation

JXXXrr ,,, 21

Assess relative significance of uncertaintysources (order of magnitude estimates)

For the experimental result r, determine theprecision and bias limits and overall uncertainty

Considering the significant sources, estimate theprecision and bias limits for each iX

Identify sources of uncertainty for each iX

Implementation Sequence

Key feature: data-reduction equation

r = r(X1, X2, X3,…, Xj)

AIAA (1995)

r = r (X , X ,......, X ) 1 2 J

1 2 J

M EASUREM ENTOF INDIVIDUALVARIABLES

INDIVIDUALM EASUREM ENTSYSTEMS

ELEM ENTALERROR SOUR CES

DATA REDUCTIONEQUATIO N

EXPERIM EN TALRESULT

XB , P

1

1 1

XB , P

2

2 2

XB , P

J

J J

rB , P

r r

Implementation Aspects

Measurement systems for each individual variable Xi : instrument, data acquisition and reduction procedures, operational environment (laboratory, in situ), the flow and its interaction with the instrument and the environment

Estimates of errors are meaningful only when considered in the context of the process leading to the value of the quantity under consideration

Uncertainties estimated following the signal propagation from sensor to the final result

Uncertainties estimated with a pre-established confidence level (95% for most engineering areas)

UA differently conducted dependent on the type of experiment:

Single test (for complex or expensive experiments): one set of measurements (X1, X2, …, Xj) for r

Multiple tests (ideal situations): many sets of measurements (X1, X2, …, Xj) for r at a fixed test condition with the same measurement system

AIAA (1995)

MULTIPLE TESTS (recommended)

Given a data reduction equation for a measurement ),,,( 21 JXXXrr

ii X

r

;M

tSP r

r

2/1 P + B = U 2r

2rr

The result and its uncertainty is

and the precision limit of the result is

where the bias limit of the result is

The uncertainty in the final result

rUr

M

kkrM

r1

1with

;

1

2/1

1

2

M

k

kr M

rrS 102 Mfort

J

i

J

i

J

ikikkiiir BBB

1

1

1 1

222 2

AIAA (1995)

SINGLE TEST

Given a data reduction equation for a measurement

ii X

r

2/1 P + B = U 2r

2rr

The result and its uncertainty is

and the precision limit of the result is

where the bias limit of the result is

The uncertainty in the final result

2/1

1

2

1

M

k

kr M

rrS

),,,,( 321 jXXXXrr

rUr

J

i

J

i

J

ikikkiiir BBB

1

1

1 1

222 2

Based on prior informationrr tSP

AIAA (1995)

AIAA (1995)


sound engineering judgment to optimize the output with minimum costs, e.g.: use of end-to-end uncertainty estimation approach uncertainty sources < 1/4 or 1/5 of the largest sources are usually considered negligible

specific procedures for single and multiple measurements

specific procedures for dealing with small statistical samples

methodology for assessment of calibration uncertainties

methodology for data validation


Integration of UA in all phases of the measurement

AIAA (1995) D E F IN E P U R P O S E O F T E S T A N D

R E S U LT S U N C E R TA IN T Y R E Q U IR E M E N T S

U N C E R TA IN T YA C C E P TA B L E ?

IM P R O V E M E N TP O S S IB L E ?

D E T E R M IN E E R R O R S O U R C E SA F F E C T IN G R E S U LT S

Y E SN O

N O

Y E S Y E S

Y E S

N O

S E LE C T U N C E R TA IN T Y M E T H O D

E S T IM AT E E F F E C T O FT H E E R R O R S O N R E S U LT S

- M O D E L C O N F IG U R AT IO N S (S )- T E S T T E C H N IQ U E (S )- M E A S U R E M E N T S R E Q U IR E D- S P E C IF IC IN S T R U M E N TAT IO N- C O R R E C T IO N S T O B E A P P L IE D

- D E S IR E D PA R A M E T E R S (C , C , ... .)D R

D E S IG N T H E T E S T

- R E F E R E N C E C O N D IT IO N- P R E C IS IO N L IM IT- B IA S L IM IT- T O TA L U N C E R TA IN T Y

D O C U M E N T R E S U LT S

N O T E S T

C O N T IN U E T E S T

IM P L E M E N T T E S T

S O LV E P R O B LE M

R E S U LT SA C C E P TA B L E ?

M E A SU R E-M E N T

S YS T E MP RO BLE M ?

N O

P U R P O S EA C H IE V E D ?

Y E S

N O

S TA R T T E S T

E S T IM AT EA C T U A L D ATAU N C E R TA IN T Y

IMPLEMENTATION EXAMPLE

Extensively used in laboratory measurements and field conditions,

from simple (Pitot tube) to complex (LDV) instruments

Widely applied for teaching and research purposes

Successful implementation to discharge measurements: conventional instruments (Muste et al. 2007)

contemporary, nonintrusive techniques: Large-Scale Particle Image Velocimetry (Y-S. Kim et al, 2007)

Acoustic-Doppler Current Profilers (Gonzalez-Castro & Muste, 2007)

AIAA (1995)

Currently, ADCPs are the most efficient instrument for riverine environment characterization (monitoring and research needs)

If properly operated, the instrument can accurately document discharges, mean velocities, and selected turbulence characteristics

Despite their extensive use, there are aspects regarding their capabilities, operation, and uncertainty analysis not documented yet

ADCP UA: Implementation

ADCP Uncertainty Analysis (UA) status

Past efforts (non-standardized methodologies)

Discharge: Simpson & Oltman (1992), Gordon (1993), Lipscomb (1995), Morlock (1996), Simpson (2001), Gartner (2002), Muller (2002), Yorke & Oberg (2002), USGS-RDI (2005)

Turbulence measurements: Droz (1998), Stacey (1999), Nystrom (2002), Schemper & Admiraal (2002)

On-going efforts (standardized methodology) UA formulated within the framework of authoritative

engineering standards


Discharge Measurement with ADCP mounted on a boat

Measurable Area

Unmeasurable Near-bank Areas

Unmeasurable Top Area

Unmeasurable Bottom Area

etQ

mQ

elQerQ

ebQ

erelebetemmt QQQQQQQ

;lllel ZLKVQ rrrer ZLKVQ where


Error identification

ADCP UA: Elemental Uncertainty Assessment

Source Biases Estimation of Accounted in Reduction Equations through 1

Depends upon Can be estimated from

e1: Spatial resolution Water and boat velocities, depths † ADCP, mode, settings, boat speed End-to-end calibration 2

e2: Doppler noise Water and boat velocities Bva, Bvb ADCP frequency, mode, settings,

speed of sound, gating time UA of signal processing algorithms,

instrument intercomparison

e3 : Velocity ambiguity Water and boat velocities † Mode, settings End-to-end calibration

e4 : Side-lobe interference Discharge through unmeasured areas * Beam angle, settings, bathymetry End-to-end calibration

e5: Temporal resolution High frequency velocity components † Settings End-to-end calibration

e6: Sound speed Water and boat velocities, depths BC Water properties UA of C(Salinity, Temperature) with data

from reference meter

e7 : Beam angle Water and boat velocities, depths B ADCP Manufacturer’s specifications

e8 : Boat speed Water and boat velocities, depths † Site, flow, boat operation End-to-end calibration

e9: Sampling time Long-term means †

Frequency of large-scale flow structures 3

Instrument intercomparison based on long data records under steady conditions

e10 : Near-transducer Velocities near the ADCP Bnt ADCP, draft, settings, velocity, flow

depth Experimental Measurements and CFD

Modeling

e11: Reference boat velocity Water and boat velocities, depths Bvb Sediment concentration, flow 4 Manufacturer’s Specifications

e12: Depth Discharge through unmeasured areas BDa, BDp, BDp, BDo, BDavg 5 ADCP, settings, draft, bathymetry,

water properties, time gating

UA of depths as f(C and gating time) and BC, Bt and BDADCP and concurrent depth

range measurements

e13: Cell positioning Measured and unmeasured discharge Bt , BDa, BDp, BDo, BDavg ADCP, setting, water properties

e14: Rotation Water and boat velocities, depths and geographic orientation

Bp , Br , Bh ADCP, setup, site Manufacturer’s Specifications

e15: Timing Distances by gating and discharge Bt ADCP, speed of sound, gating time Manufacturer’s Specifications

e16: Edge Discharges through channel edges B ,BL ADCP settings, bathymetry, cross

section, edge distances Manufacturer’s Specifications

e17: Vertical profile model Discharge through unmeasured top and bottom areas

BQ1 6 Velocity distribution model, turbulence

intensity Field and Laboratory Experiments with

reliable CFD-LES Modeling

e18: Discharge model Discharge through measured area BQ2 6 Discharge model

Highly resolved data / End-to-end calibration

e19: Finite summation Discharge through measured area BQ3 6 ADCP settings, boat velocity

e20: Site conditions & operation Total discharge † Site, boat operation Concurrently measured data

fVADCP9

Data Reduction Equations (Teledyne/RDI’s ADCP)

fff vuV ,

bbb vuV ,

x

y

E

dtdzkVVQT tz

tz

bfm

U

L

0

)(

)(


Exact approach – discharge in the direct measured area

Using BT

1

1 1,134212134

11

sin2

sN

i

n

jjibbbb

iijj

m vvvvvvvvttzz

Q

jiQ ,

unit area for jiQ ,

i 1isNi 1i

n

j1j


Exact approach: in-bin discharge Using BT

zthprFFFFCFFFFFfQ abDbDbDbDSDDDDm ji ,,,,,,,,,,,,,,, 11114321,

If , the discharge is a functional relationship of the form:

WATER VELOCITY WITH RESPECT TO ADCP

34432121 cscsin2seccossin+csccos24

1vvPvvvvPRvvRua

432134 secsin-csccos24

1vvvvPvvPva

BOAT VELOCITY WITH RESPECT TO CHANNEL BED

34432121 cscsin2seccossin+csccos24

1bbbbbbbbb vvPvvvvPRvvRu

432134 secsin-csccos24

1bbbbbbb vvvvPvvPv

ACTUAL WATER VELOCITY

bababaf vvuuVVV ,

DEPTHS

BADCPtop DDD ;

20 ap

bB

DDDDD

for Mode 1;

2ap

bB

DDDD

for Mode 5

2

cos 0max

DDDDD p

ADCPavgLG

for Mode 1;

2cosmax

pADCPavgLG

DDDD for Modes 5, 8 & 11

ADCPavgtotal DDD cos ; 21

aLGtotal

DDDZ ;

22a

toptotal

DDDZ ; totalDZ 3

PARTIAL DISCHARGES

11

11

1

12

1

bb

m

jjabbaii

ba

t ZZ

vuvuttZD

Qi

11

11

11

12

23

bb

m

jjabbaii

bba

b ZZ

vuvuttZZD

Qi

lllel ZLCVQ ; rrrer ZLCVQ

TOTAL DISCHARGE

erelem

N

ibt

N

i

m

jmtotal QQQQQQQ

s

ii

s

ji

1

1

1

1 1,

iijjjiabbam ttzzvuvuQji

11,1,

SD F

CFv

2


Exact approach – top and bottom discharges (extrapolation)

BTM Q

MID Q

TOP Q 3 Z

1 Z

Z

DEPTH CELL Da

ADCP TRANSDUCER FACE

D total

POWER FIT

SCALAR TRIPLE (m2/s2) PRODUCT

ADCP

MEASURED DISCHARGE

TOP LAYER (ESTIMATED)

BOTTOM LAYER (ESTIMATED)

DISCHARGE (m3/s)

ACTUAL PROFILE

ADCP VELOCITIES

D

2 Z CONSTANT

POWER 3-POINT SLOPE

POWER

CONSTANT POWER IN LOW 0.2 D total

D avg

D ADCP

D top B D

D LG

11

11

11

12

23

bb

m

jjabbaii

bba

b ZZ

vuvuttZZD

Qi

11

11

1

12

1

bb

m

jjabbaii

ba

t ZZ

vuvuttZD

Qi


Uncertainty Propagation to Final Result: Bias Limit2222222

erelebetemmt QQQQQQQ BBBBBBB

i

N

i

n

jj

rvv

jar

m

rja

m

n

j

j

ti

m

ti

m

zj

m

zj

m

rv

br

m

v

ra

mm

r

m

p

m

Q

s

jrarja

jiji

i

ji

i

ji

i

ji

i

ji

rb

ji

ra

jijijiji

m

BBv

Q

v

Q

Bt

QB

t

QB

z

QB

z

Q

Bv

QB

v

QB

QB

r

QB

p

Q

B

1

1

1

1

4

1 )1(

12

2

2

2

1

2

2

2

2

1

4

1

2

2

2

2

2

2

2

2

2

2

2

)1(

,,

,

1

,,

1

,

,,,,,

15.0

i

N

i

n

jj

rvv

jar

et

rja

ett

i

ett

i

et

Db

etD

avg

etD

o

etD

p

etD

a

et

et

j

n

j tv

bt

etv

ta

etr

etp

et

Q

s

jrarja

ii

i

i

i

i

b

i

avg

i

o

i

p

i

a

i

i

tb

i

ta

iii

et

BBv

Q

v

QB

t

QB

t

Q

BD

QB

D

QB

D

QB

D

QB

D

Q

BQ

Bv

QB

v

QB

r

QB

p

Q

B

1

1

1

1

4

1 )1(

2

2

2

2

1

2

2

2

2

2

2

2

2

2

2

2

2

1

4

1

2

2

2

2

2

2

2

2

2

)1(115.0

2

2

2

2

2

2

2

2

2

2

2

BQ

BD

QB

D

QB

L

QB

V

QB el

D

ADCP

elD

avg

elL

e

elV

el

elQ ADCPavgeelet

2

2

2

2

2

2

2

2

2

2

2

BQ

BD

QB

D

QB

L

QB

V

QB er

D

ADCP

erD

avg

erL

er

erV

er

erQ ADCPravg

r

ererer


M

tSP t

t

Q

Q

22

ttt QQQ PBU

Uncertainty Propagation to Final Result: Precision Limit

Uncertainty Propagation to Final Result: Total Uncertainty


bV

= beam angle, = angle of the flow to instrumentβ = angle of the boat velocity = in beam water velocities = boat velocity

where

41 ~ VV

Practical approach(pitch and roll neglected in DRE; errors accounted through end-to-end calibrations)

velocity (instrument coordinates neglecting the pitch and roll angle)

erel

N

i

n

jiijjijiibjift QQttzzVVQ

s

1

1 1111,11,,1 )sin(

total discharge

2

1

221

2432

2

221

243 )cos(

sin2

)()(2

sin4

)()(,1

bbf VVVVV

VVVVV

Vji


• Software Configuration

ADCPOutput Binary Files

ADCPOutput ASCII Files

Statistical Analysis for UA

Processing Database - Vertical, Horizontal Velocity Profile - Discharge (WinRiver & WinADCP Homologuous) - Visualizations

Prior Information ( ) - Calibrations - Manufacturer specifications - Literature compilation - prior measurements - User UA archive

Error Propagation to Final Results (Embedded in Uncertainty Analysis GUI)

- Velocity Uncertainty Analysis - Discharge Uncertain Analysis

Uncertainty Analysis Graphical User Interfaces

Graphical DisplayNumerical Display

12

34

5

Uncertainty Analysis Output

ADCP Uncertainty Analysis & GUI Flow Chart

Configuration File - operational and environmental specifications

New Measurements191 ~ uu

ADCP UA Software - architecture• Developing tools - Borland C++ Builder (v.6) & Microsoft Access

Archive database

- Elemental uncertainties are archived in categories based on river characteristics. - Users with limited level of preparedness can estimate uncertainties using default values obtained in similar environment and operating conditions. - The stored information is updated as soon as new measurements are processed. - User can also create new archives using new classification categories

ADCP UA Software - GUIs

Information for archiving


Discharge

River Characteristics

Vessel Moving Path

Flow Direction

Channel Profile

Assessment of bias limit


Individual Error Source Input

River Characteristics

Default Values based on Archive Database

Relevant Literature for error sources

1

2

3

4

Assessment of precision limit


Repeated Measurements for a transect

Result

Variation of Discharge Measurements

Assessment of total uncertainty

ADCP UA Software & GUIs

Calculated Uncertainty in Discharge Measurement

Feasibility of UA engineering standards for implementation to ADCP measurements

The methodology is comprehensive, simple to implement Easily upgradeable as new info occur UA allows tracing of the measurement accuracy to primary

standards withstand legal and strict QA/QC requirements Finalization of UA – an extensive and expensive effort

Collaboration between manufacturers and users in a coordinated effort = key to complete UA for the variety of measurement situations and operating conditions encountered in monitoring practice

The framework was adopted by ASCE’s HME Task Committee and the UNESCO group on Data Requirements for Integrated Urban Water Management (Fletcher et al., 2007)

Currently evaluated by the ISO committee (Herschy)

Conclusions

Conclusions

The UA customized software for ADCP velocity and discharge measurements requires minimum user preparation

Autoarchiving uncertainties for specific environments and operating conditions can provide information about dominant sources of uncertainties at various sites.

By continuously increasing the sample size through archiving, the UA output is progressively enhanced.

Outlook

Work closely with manufacturers and users to assess elemental error sources (manufacturer, operator, environment, or combinations) and integrate them in the AIAA (1995) uncertainty assessment framework for rigorous documenting velocity and discharge measurement accuracy

Conduct sensitivity analysis and field tests for compiling uncertainty minimizations guidelines

Develop operational guidelines for conducting accurate measurements in various flow regimes

Outlook

Need for coordination and extensive collaboration among ADCP manufacturer, operators, data users, and third-party evaluators

Need for evaluation of the status of current developments and to strategize for integrative efforts to assess methodologies for operation and accuracy assessment of the ADCP as well as other flow measurement techniques over an extend the range of flow conditions (present WMO effort)

IIHR is willing to be actively involved in the WMO initiative

Thank you!

Questions?

Uncertainty Analysis for Flow Measurements and Techniques using Standardized Methodology Marian...

Documents

Transcript of Uncertainty Analysis for Flow Measurements and Techniques using Standardized Methodology Marian...