Hydrological Sciences -Journal- des Sciences IIydrologiques,4Ui) June 1996 3 9 9

Artificial neural networks as rainfall-runoff models

A. W. MINNS & M. J. HALL International Institute for Infrastructural, Hydraulic and Environmental Engineering (IHE), PO Box 3015, 2601 DA Delft, The Netherlands Abstract A series of numerical experiments, in which flow data were generated from synthetic storm sequences routed through a conceptual hydrological model consisting of a single nonlinear reservoir, has demonstrated the closeness of fit that can be achieved to such data sets using Artificial Neural Networks (ANNs). The application of different standardization factors to both training and verification sequences has underlined the importance of such factors to network performance. Trials with both one and two hidden layers in the ANN have shown that, although improved performances are achieved with the extra hidden layer, the additional computational effort does not appear justified for data sets exhibiting the degree of nonlinear behaviour typical of rainfall and flow sequences from many catchment areas. Modlisation pluie-dbit par des rseaux neuroneaux artificiels Rsum Dans une srie d'expriences numriques, des dbits ont t gnrs partir de squences synthtiques d'vnements pluvieux grce l'utilisation d'un modle hydrologique conceptuel constitu d'un seul rservoir non linaire. Ces expriences ont montr la qualit de l'ajuste-ment que l'on peut obtenir pour ce type de donnes en mettant en oeuvre des Rseaux Neuronaux Artificiels (RNA). L'utilisation de diffrents facteurs de standardisation au cours des squences d'apprentissage et de vrification a permis de mettre en vidence la grande influence de ces facteurs sur la qualit des performances d'un rseau. Les essais effectus avec des RNA comprenant une ou deux couches caches on montr que, si une amlioration de la performance est obtenue avec une couche cache supplmentaire, l'effort de calcul correspondant ne semble pas tre justifi pour les ensembles de donns manifestant le degr de comportement non linaire typique pour des squences de pluies et de dbits rencontres dans la plupart des bassins versants.

INTRODUCTION

Some thirty years ago, Amorocho & Hart (1964) commented upon the growth of two distinct approaches to the problem of establishing the relationship between rainfall and streamflow which they referred to as physical hydrology and systems investigation. The former term was used to describe investigations into the behaviour of and interdependence between hydrological processes, the long term objective being a complete synthesis of the hydrological cycle. The progress achieved with this approach during the last three decades has

Open for discussion until 1 December 1996

400 A. W. Minns & M. J. Hall

materially assisted with the development of hydrological models that are both physically-based and spatially-distributed, such as the Systme Hydrologique Europen (Abbott et al., 1986). Nevertheless, there remains a high degree of empiricism in the representation of certain hydrological processes such that the ideal of determining parameter values by direct measurement rather than cali-bration remains some distance away.

In contrast, systems investigation, which Amorocho & Hart (1964) regarded as being concerned with the direct solution of technological problems subject only to the constraints imposed by the available data and so not subject to 'physical' considerations, has recently undergone something of a renaissance, largely through the adoption of artificial intelligence techniques such as Artificial Neural Networks (ANNs) and Genetic Algorithms (e.g. Babovic & Minns, 1994). The particular advantage of the ANN is that, even if the 'exact' relationship between sets of input and output data is unknown but is ack-nowledged to exist, the network can be 'trained' to iearn' that relationship, requiring no a priori knowledge of the catchment characteristics.

In the hydrological context, the input pattern consists of rainfall depths and the output the discharges at the catchment outlet. Since the contributions from different parts of the catchment arrive at the outlet at different times, the variations in the discharge output may be considered to be determined by the rainfall depths at both the concurrent and previous time intervals. Preliminary work (Hall & Minns, 1993) has indicated that the number of antecedent rainfall ordinates required is broadly related to the lag time of the drainage area. Since the ANN relates the pattern of inputs to the pattern of outputs, volume con-tinuity is not a constraint. However, care must be taken to avoid the presenta-tion to the ANN of contradictory information. More specifically, the input pattern may contain many zeros both at the start of the rising limb of the output hydrograph and during the recession when rainfall has ended and flows are decreasing. These two situations could be distinguished by providing an extra input consisting of a binary variable (say, zero for pre-storm and unity for post-storm conditions), but previous work (Hall & Minns, 1993) has indicated that antecedent flow ordinates both perform the same function and provide additional information about the input pattern, i.e. the longer the input rainfalls remain zero, the more the output decreases. The use of an output variable in the input is encountered in other applications of ANNs (Hertz et al., 1991) and is referred to as recurrent back-propagation. The inclusion of the flow at time t 1 as an input to determine the flow at time t may appear to introduce an element of flood routing into the model, but that is not the purpose of the ANN. Unlike the conventional rainfall-runoff model, the network seeks to learn patterns and not to replicate in detail the physical processes involved in transforming input into output. The learning process does not depend upon any assumptions relating to the form of the input-output transfer function, the number of (active) parameters or their possible physical interaction. In the terms of the discussion by Amorocho & Hart (1964), the ANN could perhaps be regarded as the ultimate black-box model.

ANNs as rainfall-runoff models 401

Given the encouraging results obtained by Hall & Minns (1993), an important further consideration is the applicability of ANNs to more complex 'real-world' catchments. However, although the standard solution algorithm for ANNs will achieve convergence for almost any problem (Rumelhart et al., 1994), it would appear that the most simple ANN architectures have more difficulty in learning more nonlinear relationships. This paper therefore describes a series of numerical experiments that were undertaken with the specific purpose of evaluating the performance of ANNs on rainfall and runoff data from theoretical catchments exhibiting a range of behaviour patterns varying from the linear to the highly (in hydrological terms) nonlinear. Owing to the virtual impossibility of collecting hydrometric data from catchments that could be classified a priori as either linear or nonlinear, but were otherwise identical in catchment characteristics and input rainfall patterns, a well-established conceptual hydrological modelling package, RORB (Mein et al., 1974), was employed to generate streamflow responses from a synthetic time series of storm events for representative (linear and nonlinear) catchments. In this manner, the ANN could be tested solely on its performance in learning the (linear or nonlinear) relationship between rainfall and runoff, all other factors being regarded as equal. These numerical experiments are, of course, only the first step towards testing the generality of ANNs for use on more complex, real-world catchments, since all the problems of spatial distribution of rainfall and seasonal changes in catchment response are avoided. The latter effects are currently the subject of on-going investigation.

ARTIFICIAL NEURAL NETWORKS

The ability of the brain to perform difficult operations and to recognize complex patterns, even if those patterns are distorted with a high degree of noise, has fascinated scientists for centuries. The particular ability of the brain to learn from experience without a predefined knowledge of the underlying physical relationships makes it an exceptionally flexible and powerful calcu-lating device that scientists would also like to mimic.

Yet other scientists are devoted to reproducing, or modelling, physical phenomena by making use of electronic computational machines to solve ever-increasingly complex partial differential equations and empirical relationships. These scientists are supported by a rapid increase in the computational capacity of modern computers and an emerging recognition of the advantages of massively parallel computation (parallel distributed processing) that performs the required calculations with ever-increasing speed. However, although the design and construction of the hardware for parallel computation is relatively straightforward, the software required for creating algorithms to utilize this parallel architecture most efficiently is still quite limited.

These two groups of scientists, pursuing what appear to be quite different goals, have found a common ground in the field of artificial neural networks.

402 A. W. Minns & M. J. Hall

One of the major applications of ANNs is in pattern recognition and classi-fication or, more generally, system identification. In brief, an ANN consists of layers of processing units (representing biological neurons - see Hopfield, 1994) where each processing unit in each layer is connected to all processing units in the adjacent layers (representing biological synapses and dendrites). Many publications describe in much greater detail the architecture of various types of ANNs (for example, Beale & Jackson, 1990; Aleksander & Morton, 1990; Hertz et al., 1991). The selection of an appropriate architecture for an ANN will depend upon the problem to be solved and the type of learning algorithm to be applied. In particular, the use of Kohonen networks for unsupervised classification of patterns and the use of Hopfield networks for recalling previously learned patterns are two approaches commonly used in pattern recognition. For the more general approach to systems identification, one wishes to train an ANN to provide a correct output response to a given input stimulus. In particular, for rainfall-runoff modelling, the input stimulus corresponds to the measured rainfall and the output response to the measured runoff from a catchment. A multi-layer, feed-forward, perceptron-type ANN is one of the most suitable types of ANN for learning the stimulus-response relationship for a given set of measured data. Figure 1 shows a general schema-tization of a 3-layer, feed-forward ANN of the type that was used in this study.

The working of an ANN can best be described by following the opera-tions involved during training and computation. An input signal, consisting of an array of numbers xi is introduced to the input layer of processing units or nodes, as shown in Fig. 1. The signals are carried along connections to each of the nodes in the adjacent layer and can be amplified or inhibited through

Output signal

'idden layer or nternal representation

units

Input Signal Fig. 1 Representation of a multi-layer, feed-forward artificial neural network (ANN).

ANNs as rainfall-runoff models 403 weights, wt, associated with each connection. The nodes in the adjacent layer act as summation devices for the incoming (weighted) signals (Fig. 2). The incoming signal is transformed into an output signal, Oj, within the processing units by passing it through a threshold function. A common threshold function for the ANN depicted in Fig. 1 is the sigmoid function defined as:

fix) = _ J _ (1) which provides an output in the range 0 < f(x) < 1. In most thresholding routines, the threshold function usually takes the form of a single-valued, hard-delimiter. The sigmoidal threshold function is chosen for mathematical con-venience because it resembles a hard-limiting step-function for extremely large positive and negative values of the incoming signal and also gives useful information about the response of the processing unit to inputs that are close to the threshold value. Furthermore, the sigmoid function has a very simple derivative that makes the subsequent implementation of the learning algorithm much easier.

Input pattern wi Summation and w*5\threshold unit

Output pattern > y

X !

Fig. 2 A typical ANN node.

The output from the processing unit is then:

O = L__ (2)

This output signal is subsequently carried along the weighted connections to the following layer of nodes and the process is repeated until the signal reaches the output layer. The one or more layers of processing units located between the input and output layers have no direct connections to the outside world and are referred to as hidden layers. The output signal can then be interpreted as the response of the ANN to the given input stimulus.

The ANN can be trained to produce known or desired output responses for given input stimuli. The ANN is first initialized by assigning random numbers to the interconnection weights. An input signal is then introduced to the input layer and the resulting output signal is compared to the desired output

404 A. W. Minns & M. J. Hall

signal. The interconnection weights are then adjusted to minimize the error between the ANN output and the desired output. This process is repeated many times with many different input/output tuples until a sufficient accuracy for all data sets has been obtained. The adjustment of the interconnection weights during training employs a method known as error back-propagation in which the weight associated with each connection is adjusted by an amount propor-tional to the strength of the signal in the connection and the total measure of the error {see Rumelhart et al., 1986). The total error at the output layer is then reduced by redistributing this error value backwards through the hidden layers until the input layer is reached. The next input/output tuple is then applied and the connection weights readjusted to minimize this new error. In this way, the back-propagation algorithm can be seen to be a form of gradient descent for finding the minimum value of the multi-dimensional error function. This procedure is repeated until all training data sets have been applied. The whole process is then repeated starting from the first data set once more and continued until the total error for all data sets is sufficiently small and subsequent adjustments to the weights are inconsequential. The ANN is now said to have learned a relationship between the input and output training data sets. The exact form of this relationship cannot be extracted from the ANN but rather is encapsulated in the stored series of weights and connections between nodes. The absolute values of the individual weights cannot be interpreted to have any deeper physical meaning (Minns, 1995).

Although the error back-propagation method does not guarantee con-vergence to an optimal solution since local minima may exist, it appears in practice that the back-propagation method leads to solutions in almost every case (Rumelhart et al., 1994). In fact, Hornik et al. (1989) concluded that standard multi-layer, feed-forward networks are capable of approximating any measurable function to any desired degree of accuracy. They further state that errors in representation appear to arise only from having insufficient hidden units or the relationships themselves being insufficiently deterministic. For this reason, a standard, multi-layer, feed-forward ANN using standard back-propagation learning techniques was used in this study.

GENERATION O F DATA FOR TRAINING AND VERIFYING THE ANN

Since the purpose of the numerical experiments reported below was only to evaluate the ability of an ANN to 'learn' the relationship between the pattern of inputs provided by a sequence of rainfalls and the outputs in the form of the pattern of flows generated by a hypothetical (but realistic) catchment from those rainfalls, the precise form of the model used to generate the runoffs from the rainfalls was of little importance. The ANN is not being applied to identify this model, and the principal requirement is only that it should produce responses typical of those encountered in hydrological work.

ANNs as rainfall-runoff models 405

Rainfall data

For the purposes of the numerical experiments, six sequences of storm events of varying duration, total depth and profile, occurring at irregular intervals, were required that could be routed through simple conceptual hydrological models with different degrees of nonlinearity in order to produce the corresponding streamflow outputs. For simplicity, these rainfalls were treated as areal averages. Since several storm sequences were required, they were produced using Monte Carlo methods based on the following assumptions:

1. storm durations were normally-distributed, with a mean of 20 h and a standard deviation of 6 h;

2. storm rainfall depths were lognormally-distributed, with a mean of 25 mm and a standard deviation of 2 mm (these statistics imply that the distribution of depths had a coefficient of variation of 0.785 and a skewness coefficient of 2.84);

3. the shapes of the six storm profiles could be described by simple polynomial functions, broadly based on those of the UK Flood Studies Report (Natural Environment Research Council, 1975), and including early-peaked and late-peaked as well as symmetrical events (a constant intensity profile was also included as an extreme case); and

4. the inter-event times were taken as double the previous storm duration minus one hour.

Initially, three sequences of 14 storm events were generated, the profile shapes being selected by sampling from a distribution uniform over the range zero to six. The first was a training sequence with a total duration of 764 h. Five of the six profiles were represented, with durations having an average of 19.2 h and a standard deviation of 6.95 h. The average depth was 31.6 mm, with a standard deviation of 1.9 mm. The other two sequences were employed for verification purposes. The first of these verification data sets was generated so that the maximum values all fell within the range defined by the training sequence. However, if an ANN were to be applied to a real catchment, even if the training data included all the available measurements, there is always a small but non-negligible probability that an extreme event beyond the range of recorded experience may occur in the future. In order to evaluate the per-formance of an ANN under these circumstances, the second verification sequence was generated that contained rainfall maxima outside the range of those upon which the training data was based.

The two verification sequences had a total duration of 794 h, and all profiles were represented. In the first data set, individual events had an average duration of 19.8 h and a standard deviation of 4.9 h, and a mean depth of 24.6 mm with a standard deviation of 2.1 mm. The second verification sequence was constructed by employing the same seed as that for the first, but assuming that storm depths were lognormally-distributed with a mean of 25 mm and a standard deviation of 3 mm, which implies a coefficient of variation of 1.53 and

406 A. W. Minns & M. J. Hall

a skewness coefficient of 8.2. The actual mean and standard deviation of storm depths produced was 24.3 mm and 3.3 mm respectively.

Runoff data generation

The conceptual hydrological model that was adopted to produce the flow series corresponding to the storm sequences was the RORB model (Mein et al., 191 A), the basic element of which is a single nonlinear reservoir for which the relationship between storage, S, and discharge, Q, is given by:

S = KcKrQm (3) where Kc is a storage constant applicable to all sub-areas within the catchment and Kr is a relative delay time applicable to individual channel reaches within the network estimated from the expression:

Kr=fh. (4) where Lt is the length of the reach represented by the storage element, Lav is the average flow distance of sub-catchment inflows within the channel network, and/is a factor depending upon the type of channel reach, i.e. natural, lined or unlined.

According to Laurenson & Mein (1988), the exponent in equation (3) is rarely less than 0.6 or greater than 1.0 when modelling catchment runoff response to rainfall, and a trial value of 0.8 is recommended on beginning a modelling exercise. A brief review of the available literature shows that the values adopted for exponents has ranged from 0.67 (Watt & Kidd, 1975) to 0.8 (Selvalingham et al., 1987), with a predominance of values between 0.7 and 0.8 (Laurenson, 1964; Askew, 1970; Mein et al, 191 A; Hong & Mohd Nor, 1988). Three models were therefore adopted to cover the range of possible catchment behaviour: (i) m = 0.8 to represent the typical nonlinear relationships encountered in

practice; (ii) m = 1.0 to represent the extreme linear case; and (iii) m = 0.5 to represent an extreme nonlinear type of behaviour. In order to run the RORB software, a hypothetical catchment area and main channel length had to be assumed in order to establish the value of Kr. The chosen values of these characteristics were consistent with those of a rural drainage area of about 30 km2 in southern England. Although these considerations are not particularly relevant to the learning of patterns, the size of the catchment broadly determines how many antecedent rainfall depths are required in developing the ANN and therefore influences the overall size of the network. For simplicity, no losses were separated and the catchment was con-sidered to have no impervious area. The Kc value was set to 20. The time

ANNs as rainfall-runoff models 407 series of flows so obtained reflected very well the range in response charac-teristics represented by the three models, with model (iii) showing rapid rises and recessions in contrast to the slow rises and sustained recessions of model (ii). For the purposes of illustration, the rainfall hyetographs and flow hydrographs generated by model (i) are presented in Fig. 3.

I ^ 60

i 12

_, 20

J 0 time

Fig. 3 Rainfall and runoff data for model (i).

Standardization of data

Prior to presenting the data to the ANN for training, a standardization must be applied in order to restrict the data range to the interval of zero-to-one, corresponding to the output limits of the nodes of the network as expressed in equation (1). The significance of this standardization should not be under-estimated. When different standardization factors are applied to the training and verification sequences, the actual numbers represented by unity in the two data sets are different. In practice, a trained ANN can only be used in the recall mode with data that it has 'seen' before; the ANN should not be used for extrapolation. For example, if the maximum flow that the ANN has learned to predict is 50 m3 s"1 (corresponding to, say, an output from the node of 1.0), it is impossible for the ANN ever to predict a flow value exceeding 50 m3 s"1.

The choice of the range for standardization may therefore influence sig-nificantly the performance of the ANN. For the experiments used here, the standardization factors adopted were the maximum generated rainfall depths and flow ordinates rounded up to the next highest multiple of 10 mm or 10 m3 s"1 respectively. For the training sequence and the first verification

408 A. W. Minns & M. J. Hall

sequence, the same rainfall factor of 10 mm was found to be applicable. However, the flow factors were found to vary with the m value. For the training data, values of 30, 40 and 50 m3 s"1 were employed for models (ii), (i) and (iii) respectively. The same values were found applicable to the first verification sequence (referred to below as normal verification) except that 60 m3 s"1 was adopted for model (iii).

For the second verification sequence (referred to here as extreme verification because of the wider range of extremes that it contained), the required standardization factors were 20 mm for the rainfall depths, and 60, 90 and 120 m3 s"1 for models (ii), (i) and (iii) respectively. By using these factors, a somewhat unrealistic situation was created in which the ANN was trained with standardized rainfall depths with unity representing 10 mm and stan-dardized flows with unity representing 30, 40 and 50 m3 s"1 for models (ii), (i) and (iii) respectively, but then applied to verification data in which unity corresponded to at least double these values. In practice, a trained ANN would have an associated set of standardization factors, and any new data introduced to the network in recall mode should be standardized using those factors. In order to illustrate the implications of this procedure, an extra verification sequence was developed by applying the factors from the training data to the second verification sequence. For convenience, the latter is referred to below as out-of-range verification.

ARTIFICIAL NEURAL NETWORK MODELLING

For all three conceptual models, initial trials were carried out with a 3-layer ANN. The program employed was a BP-simulator produced by IBP-Pietzsch GmbH of Ettlingen, Germany. In ANN terminology, the problem of rainfall-runoff modelling can be reduced to the problem of pattern recognition. The ordinates on the rainfall hyetograph quite clearly represent a pattern that is unique for each rainfall event. The object of ANN modelling is then to relate each of these patterns to its corresponding runoff hydrograph ordinate. The network output therefore consisted of a single flow value. The input to the ANN consisted of the concurrent and different numbers of antecedent rainfall depths, the number of the latter defining the input window length. The number of nodes in the intervening hidden layer was chosen to be roughly half the number of input nodes. These trials confirmed the experience gained in a previous study with ANNs applied to rainfall and runoff data (Hall & Minns, 1993) that the use of rainfall inputs alone is insufficient, and that antecedent flows should be employed as additional inputs. The network configurations finally chosen involved the use of the concurrent and 14 antecedent rainfall depths and three antecedent flow ordinates.

In effect, each set of input values and its corresponding output becomes an event, and the series of events within the storm sequence is presented to the network in turn. Once the sequence has been exhausted, the network returns

ANNs as rainfall-runoff models 409 to the first event, and the cycle is repeated. This procedure is continued until the global error of the network, which is based upon the sums of squares of the differences between observed and computed values, is driven down to an acceptable level. In the majority of runs, in order to ensure that the global error had truly reached its minimum, the training was continued until the number of events had exceeded 106. Since the global error as implemented in the software package employed was dependent upon the number of nodes in the network, a more general fitting criterion was sought. As the review by Diskin & Simon (1977) has shown, a variety of such indices have been applied in hydrological modelling, but perhaps the form that has been used most widely is the coefficient of efficiency defined as one minus the quotient of the mean square error and the variance of the observed flows, i.e.:

i m

F = 1 - m=l (5) 1 m I - \2

m-l/=iv ' where qt are the model estimates of the flow ordinates, qi,i = \,2,...,m and g is the mean of the qv Since the network inputs included the flows at previous time steps, the ANN could be considered to be modelling the change inflows rather than their absolute values. In these circumstances, the variance of the differences in flows, qv - qiA, could be preferred to the variance of the observed flows in equation (5). However, investigation showed that, for the data sets employed in this study, the variance of the differences was usually of the order of 10~2 times the variance of the observed flows, but that the mean square error could be as high as 10 times the variance of the differences. In these circumstances, use of the latter would then lead to F values well below minus one, whereas equation (5) remains between zero and one and was there-fore preferred.

Training an ANN can take several hours on a powerful, desk-top personal computer. However, once the weights have been determined the running time for the model with a new input data sequence is only a few seconds.

In order to demonstrate the degree of fit obtained, two consecutive events from the training sequence, including the largest of the 14 generated storms, have been selected for illustration. Figure 4 shows the performance of the 3-layer ANN for each of the three models for these events. In all cases, the hydrograph from the smaller event is well simulated, but the 3-layer ANN marginally underestimates the six or seven peak ordinates from the larger event. In addition, Table 1 summarizes the results from both training and verifying the 3-layer ANN on the data from each of the three models. In each case, the ANN was trained on the training sequence and verified on all three verification sequences as described above.

Table 1 shows that the goodness-of-fit obtained was such that the majority of the coefficients of efficiency varied only in the third place of decimals. In both training and verification, the performance of the ANN on the

410 A. W. Minns & M. J. Hall

model iii __ training data A 3-layerANN o 4-layerANN

time, h

Fig. 4 Training of 3- and 4-layer ANNs on input and output data from each of three conceptual catchment models: (a) model i; (b) model ii; and (c) model iii. Two events only have been selected for clarity of illustration.

ANNs as rainfall-runoff models 411

linear case was marginally the best, although there was little to choose between that and the two nonlinear cases. Comparison of the three verification cases underlines the importance of the standardization. Whereas the normal and extreme verification results are comparable for all three models, those for the out-of-range case are notably poorer, essentially because the ANN depresses the extremes in the data sets.

Table 1 Coefficients of efficiency for 3-layer ANNs fitted to rainfall and runoff series from three different conceptual hydrological models

Model

i

ii

iii

Training sequence

0.9955

0.9973

0.9942

Verification :

normal

0.9963

0.9980

0.9867

sequences

extreme

0.9943

0.9945

0.9807

out-oi -range

0.7800

0.8383

0.7718

In order to provide a visual impression of the performances of the trained ANNs in verification, two consecutive storms, including the largest of the 14 events, have been extracted. Figure 5 shows the results for normal verification where once again the ANN marginally underestimated the peak ordinates for the largest event for models (i) and (ii) using the 3-layer network; for model (iii), this peak was captured but the recession was somewhat delayed. Extreme verification (Fig. 6) gave very similar results, although the largest peak generated by model (iii) was overestimated by the 3-layer ANN. The out-of-range verification of Fig. 7 amply illustrates the inability of the ANN to produce an output greater than unity, even though the simulation of those events that are contained within range, e.g. the first storm with the linear model in Fig. 7(b), was reasonably reproduced.

With the performance of the 3-layer network being marginally poorer with the data from the two nonlinear models, the exercise was repeated using four layers, i.e. two hidden layers, in order to determine whether such a configuration could adapt better to nonlinear relationships. The results, which are summarized in Figs 4-7 and Table 2, demonstrate that the performances of the 4-layer ANNs in normal and extreme verification were better in all but the extreme nonlinear case see Figs 5(c) and 6(c) where overestimation of the largest peak occurred. However, in terms of F values (see Table 2), the performance of the ANN on extreme verification for the model with m = 0.5 was better than that on normal verification. The training of the 4-layer ANN (Fig. 4) removed completely the underestimation of the largest peak flow and differences in F values were only observed in the fourth decimal place. Once again the failure to simulate the extremes in the data sets for the out-of-range verification resulted in a notably poorer performance. Nevertheless, Table 2 is sufficient to indicate that, should the performance of the 3-layer network be

412 A. W. Minns & M. J, Hall

model ii normal verification fl 3-layerANN o 4-layerANN

BSeec'

ft

f L

model iii normal verification A 3-layerANN o 4-IayerANN

M

- ^ 3S8QS#_ SSSaj3a^Q3aQG3Q

Fig. 5 Normal verification of 3- and 4-layer ANNs trained on input and output data from each of three conceptual catchment models: (a) model i; (b) model ii; and (c) model iii. Two events only have been selected for clarity of illustration.

ANNs as rainfall-runoff models 413

(aP

0.2

(b)

model I _ extreme verification & 3-layerANN o 4-layerANN

k

^fiEeageeasaeaao

0.2 J -

_J

.

L

/ ' " L

'I i

- i

J

model ii extreme verification fl 3-layerANN o 4-layerANN

/

V i \ /

%

V X=,

x*^ ^ w ^

(0

0.6

0.2

# o

r n

i i - i t

i

model iii extreme verification

A 3-layerANN o 4-layerANN

on>

1

i \ ! U

I b

V

f t T ^ f e w m w l ^ ^ ^ ^

20 60 time, h

Fig. 6 Extreme verification of 3- and 4-layer ANNs trained on input and output data from each of three conceptual catchment models: (a) model i; (b) model ii; and (c) model iii. Two events only have been selected for clarity of illustration.

414 A. W. Minns & M. J. Hall

(a)

(b) o

2.5

_

model i out-of-range A 3-layerANN o 4-layerANN

- A it ^

/erification r\ / \ i \ i \

1 \

0.5

model ii out-of-range verification A 3-iayerANN o 4-layerANN

(0

2

1.5

1

0.5

n

model iti out-of-range A 3-layerANN o 4-layerANN

A f>AA A l

I \

verification

'f&smmffpK

1 1

I

I 1 \

\ \ \

/cgpasB^gp^

\ A

i 40 60

time, h 80

Fig. 7 Out-of-range verification of 3- and 4-layer ANNs trained on input and output data from each of three conceptual catchment models: (a) model i; (b) model ii; and (c) model iii. Two events only have been selected for clarity of illustration.

ANNs as rainfall-runoff models 415

deemed unsatisfactory, a 4-layer ANN may well bring about some improve-ment. This conclusion appears valid over the range of linear and nonlinear behaviour normally encountered in rainfall-runoff modelling, and confirms the potential of the approach.

Table 2 Coefficients of efficiency for 4-layer ANNs fitted to rainfall and runoff series from three different conceptual hydrological models

Model

i

ii

iii

Training sequence

0.9998

0.9993

0.9997

Verification

normal

0.9996

0.9993

0.9836

sequences

extreme

0.9992

0.9971

0.9866

out-of-range

0.8135

0.8502

0.7923

CONCLUDING REMARKS

The results of the numerical experiments summarized above, based upon rainfall and flow data generated by conceptual catchment models varying from linear to extremely (in hydrological terms) nonlinear cases, have reinforced the conclusion reached by Hall & Minns (1993) that ANNs are capable of identi-fying usable relationships between discharges and antecedent rainfalls. When the fitted ANNs were verified on storm sequences containing the same range of extremes as the training data (normal verification), the coefficients of efficiency were comparable to the second decimal place. The performance of the ANN deteriorated with increasing nonlinearity - but only in the third decimal place. In terms of individual storm hydrographs, the largest peaks were not always reproduced closely. This performance can be expected when the number of 'high' peaks is small compared with that of 'average' peaks; the ANN assigns relatively more importance to the latter rather than to matching the former. These findings are sufficient to suggest that extreme caution should be applied if ANNs were to be employed in studies of extreme floods.

When the ANNs were verified on sequences having larger extremes than the training data (extreme verification), F values were reduced but not as much as expected. The larger range of standardization introduced a squashing of the hydrographs such that the sequence provided a surplus of small (on the scale of zero-to-one) rises on which the ANNs had some difficulty. In contrast the larger peaks were simulated rather well and the largest events tended to be overestimated (Fig. 6).

The out-of-range verification sequences (Fig. 7) serve to emphasize the care required in choosing standardization factors. Nevertheless, even though the largest events were significantly in error, those scaled to below unity were modelled well.

416 A. W. Minns & M. J. Hall

In a search for improved performance, the possibility that the ability of the ANN to resolve complex patterns could be improved by introducing an extra hidden layer was investigated. The results showed the anticipated improvement, but the increased computational effort involved raises some doubt as to whether the increase in F values is justified by the additional learning time. An alternative approach might be to increase the number of nodes in the hidden layer rather than to add another layer to the network. Indeed, further tests on the extreme nonlinear data set showed that, by increasing the number of nodes in the hidden layer from 10 to 15, the training F value was increased from 0.9942 (Table 1) to 0.9966, about half the improvement obtained by adding the extra hidden layer (0.9997; Table 2). However, using the normal verification data, the F value rose from 0.9867 (Table 1) to 0.9917, which is higher than that of 0.9836 (Table 2) achieved with the 4-layer network.

These results tend to support the contention by Rumelhart et al. (1994) that minimal networks can offer better generalized performance than more complex networks. The extreme accuracy of the ANNs for the typical nonlinear case (m = 0.8), which would appear representative of many rainfall-runoff data sets, indicates that a 3-layer network should be sufficient for the majority of real-world applications. Nevertheless, several outstanding problems, such as those of choosing appropriate standardization factors and input window lengths, remain to be explored before the approach can be widely applied in practice.

An ANN has therefore demonstrated its ability to relate a runoff ordinate to the pattern of antecedent rainfall depths. In hydrological modelling terms, the ANN does not identify a form of model such as the nonlinear reservoir model of equation (3). However, a form of model is implicit in the ANN within the distribution of weights. Moreover, this distribution is obtained automatically with no user intervention. Since the ANN works with total rainfalls and total flows there is no necessity to apply loss functions and baseflow separation techniques as in conventional approaches. The ANN is indeed the ultimate hydrological black-box. However, in the words of an anonymous reviewer (gratefully acknowledged), the latency of the model appears to be a virtue, which is even more dangerous since the model is a prisoner of its training data.

REFERENCES

Abbott, M. B., Bathurst, J. C , Cunge, J. A., O'Connell, P. E. & Rasmussen, J. (1986) An introduction to the European Hydrological System - Systme Hydrologique Europen, "SHE", 1: History and philosophy of a physically-based, distributed modelling system. / . Hydrol. 87, 45-59.

Aleksander, I. & Morton, H. (1990) An Introduction to Neural Computing. Chapman and Hall, London. Amorocho, J. & Hart, W. E. (1964) A critique of current methods in hydrologie systems investigation.

Trans. AGU 45, 307-321. Askew, A. J. (1970) Derivation of formulae for variable lag time. / . Hydrol. 10, 225-242. Babovic, V. & Minns, A. W. (1994) Use of computational adaptive methodologies in hydroinformatics. In:

Hydroinformatics '94 (Proc. 1st Internat. Conf. on Hydroinformatics, Delft, The Netherlands) ed. A. Verwey, A. W. Minns, V. Babovic & C. Maksimovic, 201-210. Balkema, Rotterdam, The Netherlands.

ANNs as rainfall-runoff models 417 Beale, R, & Jackson, T. (1990) Neural Computing: An Introduction. Institute of Physics, Bristol, UK. Diskin, M. H. & Simon, E. (1977) A procedure for the selection of objective functions for hydrologie

simulation models. J. Hydrol. 34, 129-149. Hall, M. J. & Minns, A. W. (1993) Rainfall-runoff modelling as a problem in artificial intelligence:

experience with a neural network. In: Proc 4th Nat. Hydrol. Symp. (Cardiff, UK), 5.51-5.57. British Hydrological Society, London, UK.

Hornik, K., Stinchcombe, M. & White, H. (1989) Multilayer feedforward networks are universal approximators. Neural Networks 2, 359-366.

Hertz, J., Krogh, A. & Palmer, R. G. (1991) Introduction to the Theory of Neural Computation. Addison-Wesley, Redwood City, California, USA.

Hong Kee An & Mohd. Nor Bin Hj. Mohd. Desa (1988) Application of runoff routing model to the Krian river basin. Bull Instn. Engrs. Malaysia, 12-17.

Hopfield, J. J. (1994) Neurons, dynamics and computation. Physics Today 47(2), 40-46. Laurenson, E. M. (1964) A catchment storage model for runoff routing. / . Hydrol. 2, 141-163. Laurenson, E. M. & Mein, R. H. (1988) RORB Version 4 runoff routing program user manual.

Monash University, Clayton, Australia. Mein, R. H., Laurenson, E. M. & McMahon, T. A. (1974) Simple nonlinear model for flood estimation.

Proc. Am. Soc. Civ. Engrs, J. Hydraul. Div. 100(HY11), 1507-1518. Minns, A. W. (1995) Analysis of experimental data using artificial neural networks. In: Hydra 2000 (Proc.

XXVI Congress IAHR), vol. 1, 218-223. Thomas Telford Services Ltd, London, UK. Natural Environment Research Council (1975) Flood Studies Report, Vol II, Meteorological studies. Natural

Environment Research Council, London, UK. Rumelhart, D. E., Hilton, G. E. & Williams, R. J. (1986) Learning internal representations by error

propagation. In: Parallel Distributed Processing. Explorations in the Microstructure of Cognition, Vol 1, Foundations, 318-362. The MIT Press, Cambridge, Massachusetts, USA.

Rumelhart, D. E., Widrow, B. & Lehr, M. A. (1994) The basic ideas in neural networks. Communications of the ACM 37(3), 87-92.

Selvalingham, S., Liong, S. Y. & Manoharan, P. C. (1987) Application of RORB model to a catchment in Singapore. Wat. Resour. Bull. 23(1), 81-90.

Watt, W. E. & Kidd, C. H. R. (1975) QUURM - a realistic urban runoff model. J. Hydrol. 27, 225-235.

Received 16 August 1995; accepted 12 January 1996