Pierluigi CauEnergy and Environment ProgramCenter for Advanced Studies, Research and Development in Sardiniaplcau@crs4.it

CRS4Sardegna Ricerche, 09010 Pula CA, Italy http://www.crs4.it

Sensitivity Analysis, Calibration, Validation e Uncertainty Analysis

SA, Calibration and Validation

Sensitivity, calibration and uncertainty analysis of a complex watershed model such as SWAT is beset with a few complex issues that need to be accounted for, such as:

1) Model parameterization: quality of data and resolution, parameterization and choices done to set up a watershed model

2) Definition of what is a “calibrated watershed model” and what are the limits of its use

3) Uncertainty

4) Non uniqueness of a solution (many calibrated model may exist)

Model parameterization

A soil /land cover unit appearing in different locations in a watershed, under different land management and/or climate zones, should have different parameters. This may also be due to small and large scale heterogeneity.

When calibrating a model, such spatial and temporal differentiation can be brought as far as one decides. Naturally there is a practical limit.

On the one hand we could have thousands of parameters to calibrate, and on other we may not have enough spatial resolution in the model to see the difference between different regions.

What is a calibrated model

Calibration depends mostly on the calibration target (values used to calibrate a model).

If a watershed model is calibrated using discharge data at the watershed outlet, when we add water quality to the data and recalibrate, the hydrologic parameters obtained based on discharge alone will change.

This process can go as far as one wants.

UncertaintiesAnother issue with watershed models is that uncertainty in the predictions can be due to:

1. conceptual model uncertainty, 2. input uncertainty (meteo-climatological forcing, etc.), 3. parameter uncertainty.

The conceptual model uncertainty (or structural uncertainty) could be the result of the following situations:

a) Model uncertainties due to simplifications in the conceptual model, b) Model uncertainties due to processes occurring in the watershed but not included in the model, c) Model uncertainties due to processes that are included in the model, but their occurrences in the watershed are unknown to the modeler, d) Model uncertainties due to processes unknown to the modeler and not included in the model

Data uncertainties within SWAT• Land Use/Land Cover:

- one land use /land cover is used throughout the simulation

- course resolution map and parameterization

• Soil: - course resolution map and parameterization

• Climate date (precipitation, temperature, wind, solar radiation, etc.)

- distribution of climate gages

- error in the measures and missing values

• Land and water resources management practices

……..

Data quality / Model Calibration

Model calibration consists of changing values of model parameters in an attempt to match field conditions within some acceptable criteria

This requires that field conditions at a site be properly characterized

Data uncertainty must be evaluated

Lack of proper site characterization may result in a model that is calibrated to a set of conditions which are not representative of actual field conditions

Model Calibration in HydrologyModel calibration is complex due to:

§ inadequate description of processes and interactions § scarcity of appropriate field data

o small and large scale heterogeneity o lack of detailed information about the geometry of the

systemo inadequate knowledge of water exploitation scenarioso others

§ variability of boundary conditions§ low quality of control data

Calibration must take into account all sources of uncertainty. Errors must be incorporated into the modeling process.

The degree of the uncertainty in model based decision needs to be quantified.

Sensitivity analysis (I)

Sensitivity is a measure of the effect of change in one factor on another factor.

Sensitivity analysis is potentially useful in all phases of the modeling process:

1. model formulation, 2. model calibration 3. and model validation.

The sensitivity of model parameters should be recognized as a special case of the above general definition.

Parametric sensitivity is a vital part of most optimization techniques. However, other facets of sensitivity need to be recognized.

Sensitivity analysis (II)

The time-dependent nature of sensitivity should be considered in the formulation of hydrologic models.

A variety of simplified hydrologic models are used to demonstrate the potential of sensitivity in all phases of the modeling process.

The failure to recognize and exploit the potential of sensitivity analysis may result in the inadequacy of model formulation.

The objective function

No objective function characterizes in an exhaustive manner a model performance.

The model performance depends:1. model structure, 2. objective function used in calibration, 3. data quality and calibration data length4. model complexity

Some models are calibrated at an annual / monthly time-scale; this may not guarantee good simulation performance on a monthly / daily time step.

An equation to be optimized given certain constraints and with variables that need to be minimized or maximized in the calibration process

The objective function

The Nash-Sutcliffe objective function:

1

Q1,i is the observed sample for time i, and Q2,i is the corresponding simulated value.

The Nash-Sutcliffe index ranges between -� and 1, with a perfect match (KNS = 1) when Q1,i = Q2,i for all i

����

�

�

����

�

�

�

���

�

�

�

�

21

1,1

2,2

1,1

)(

)(1

QQ

QQK

N

ii

i

N

ii

N S

Other objective functions The EQM and Correlation:

2)

3)

Some situations

In highly managed watersheds, natural processes may play a secondary role

Some situations

Calibration plot