Estimate Evapotranspiration from Remote Sensing Data-- An Artificial Neural Network Approach

CS539 Artificial Neural Network and Fuzzy Logic

Final Project

December 15, 2003

Feihua Yang

TABLE OF CONTENTS

1. INTRODUCTION....................................................................................................................................1

2. WORK PERFORMED..........................................................................................................................2

2.1 ..........................................................................................................DATA COLLECTION

.......................................................................................................................................2

2.2 DATA PREPROCESSING...................................................................................................3

2.3 ANN DESIGN.................................................................................................................4

2.3.1 ANN Structure.......................................................................................................4

2.3.2 ANN Testing..........................................................................................................6

2.3.3 Matlab Program Description...............................................................................6

3. RESULTS....................................................................................................................................................6

3.1 ANN RESULTS.................................................................................................................6

3.2 BASELINE STUDY.............................................................................................................8

4. DISCUSSION.............................................................................................................................................8

APPENDIX A BASELINE STUDY METHOD................................................................................11

APPENDIX B SELECTIVE MATLAB PROGRAMS..................................................................12

Estimate Evapotranspiration from Remote Sensing Data-- An Artificial Neural Network Approach

1. IntroductionEvapotranspiration (ET) is the combination of water that is evaporated and transpired by

plants. Its energy equivalence is latent heat flux (LE) which is the energy required to

transform water into vapor. ET is critical in understanding climate dynamic, watershed

management, agriculture and wild fire assessment.

Evapotranspiration is a complex process which is affected by environmental factors

including land surface temperature, vapor pressure deficit, surface radiation, humidity, wind

velocity, vegetation coverage. Several methods have been developed to estimate ET by

integrating ground measurements with remote sensed data (Nemani 1989 and 1993, Nishida

2003). However, the results are far from promising when compared to ground truth.

Artificial neural network (ANN) is a computing system motivated by the function of

human brain. The basic element of an ANN is called neuron. Each neuron is linked to other

neurons with varying coefficients of connectivity called weights. Learning is accomplished

by adjusting weights to make the overall network output desired results. Once a network has

gained its weights through learning, the weights can be used to produce output when an input

pattern is given. The learning process is called training in ANN.

ANN is powerful in investigating the mechanism of a complex system from its past

behaviors (Haykin 1999). In many ecosystem modeling and remote sensing, the mechanism

behind ecosystem processes is often too complicated to be expressed explicitly by physical or

mathematical equations. ANN provides an alternative way to explore the underlying

relationships in those ecosystem processes.

1

Four environmental variables including land surface temperature (LST), saturated vapor

pressure (SVP), surface radiation (RA), and vegetation index (EVI) are used in this study to

examine the relationships between ET and its formative environmental conditions. These

four variables are far from complete in terms of factors affecting ET. However, they are the

main variables controlling evapotranspiration process and are remote sensing available.

The objective of this study is to apply artificial neural network and remote sensing

techniques to explore the relationships between ET and its formative environmental factors.

This relationship can then be used to map the distribution of ET over continental to global

scales and provide information for other ecosystem modeling and management activities.

2. Work Performed

2.1 Data Collection

The data used to explore the relationships between ET and its formative environmental

conditions in this study were collected in year 2001. Ground truth of ET came from

AmeriFlux tower measurements. Table 1 is a list of 12 AmeriFlux tower sites used in this

study.

Table 1 AmeriFlux Towers Information

ABBR. AmeriFlux Tower Site Name Latitude Longitude Elevation(meter)CA_1 Blodgett Forest 38.8953 -120.6328 1300~1500IL_1 Bondville 40.0061 -88.2919 300MA_1 Harvard Forest 42.5378 -72.1715 180-490IN_1 Morgan Monre State Forest 39.3213 -86.4134 275MT_1 Fort Peck 48.3079 -105.1005 634CO_1 Niwot Ridge Forest 40.0329 -105.5464 3050CT_1 Great Mountain Forest, Norfolk 41.9667 -73.2333 380~480OR_1 Metolious Research Natural Area – young 44.4372 -121.5668 1188CA_2 Tonzi Ranch 38.4312 -120.9660 177CA_3 Vaira Rnach, Ione 38.4067 -120.9507 129WI_1 Lost Creek 46.0827 -89.9792 >480WI_2 Willow Creek 45.8059 -90.0799 >520

2

Hourly or daily latent heat flux (LE) which is the energy equivalence of ET is measured

at each of the twelve AmeriFlux towers and is public available at AmeriFlux website. The

hourly or daily LE is then aggregated into 8-day average LE using Perl programs. There are a

total of forty-five 8-day periods from Julian day 1 to 360.

Daily surface radiation (RA) comes from GEOS satellite (Geostationary Operational

Environmental Satellite) and is also public available. The daily surface radiation at each of

the twelve AmeriFlux tower sites is extracted and aggregated into 8-day average surface

radiation using IDL programs (Interactive Data Language from Research Systems Inc.)

Land surface temperature (LST) comes from MOD11 and vegetation index (EVI) comes

from MOD13 boarded on EOS-TERRA satellite and are public accessible. Both LST and

EVI datasets are in 8-day composite. LST and EVI at each of the twelve AmeriFlux tower

sites are extracted using C programs. Saturated vapor pressure (SVP) is derived from LST by

the following equation:

, where LST is in degree Celsius and SVP in Kpa

Missing data in latent heat flux (LE), land surface temperature (LST), saturated vapor

pressure (SVP), surface radiation (RA), and vegetation index (EVI) are represented as -999.

2.2 Data Preprocessing

The dataset collected from ground measurements and remote sensing including missing

data filled in as -999. To facilitate data processing in ANN, data records that have one or

more elements labeled as -999 are filtered out. There are a total of 348 records after the

missing data is taken out. The input features are then normalized to make differences in

magnitude between features small. After randomization, the whole dataset is partitioned into

three subsets with each set having approximate one-third of the total data records.

3

2.3 ANN Design

2.3.1 ANN Structure

Multi-layer perceptron (MLP) neural network using back-propagation with one hidden

layer is chosen to estimate ET from its formative environmental variables. Preliminary

examination of the dataset found that increasing the number of hidden layers does not have

significant improvement on overall performance of neural network. This is probably due to

the fact that only four environmental factors are included in this study. Therefore, the

emphasis is on finding the best configuration of neural network in terms of number of

neurons in the hidden layer, learning rate, momentum, epoch size, and maximum number of

epochs to run.

In order to find the best configuration of the five parameters listed, each parameter is

tested with six different values while fixing other five parameters to default values. Table 2

lists the six tested values for each of the five parameters and the default setting for each

parameter.

Table 2 Back-propagation parameters and testing values

Trials Parameters 1 2 3 4 5 6 Defaultnumber of neurons 3 5 12 20 30 50 5learning rate 0.01 0.1 0.3 0.6 0.8 1.0 0.1Momentum 0.1 0.3 0.4 0.5 0.8 1.0 0.5epoch size 10 20 30 64 80 120 30Max. number of epochs to run 200 300 500 1000 2000 5000 300

Each of the six values of a given parameter is tested ten times while setting other five

parameters to default values. The value with the smallest average testing error based on the

ten trials is chosen as the best setting for the parameter tested. After all five parameters being

examined, the result is a combination of the best configuration for the five parameters. This

4

process is repeated 10 times to obtain 10 combinations of the best configuration for the five

parameters. Each of the 10 combinations is then tested 10 times to compare their average

testing errors. The one with the smallest testing error is chosen as the best configuration for a

given training and testing sets.

As mentioned in data preprocessing section, the whole dataset is partitioned into three

subsets with each set having approximate one-third of the total data records. A 3-way cross

validation is performed with any two of the three subsets serving as training data, and the rest

set serving as testing data. However, the 3-way cross validation yields different results in

terms of the best configuration. Table 3 is the result of best configurations after 3-way cross

validation with 3 trials. By comparing these different configurations and their training and

testing errors, the final configuration for the five parameters is chosen as number of neurons

in the hidden layer 3, learning rate 0.3, momentum 0.8, epoch size 64, and maximum number

of epochs to run 1000.

In this study, a hyperbolic tangent activation function is used for the hidden layer and a

sigmoid function is used for the output layer.

Table 3 Best configuration for each of the 3-way cross validation with 3 trials

ParametersTrials

Number of neurons Learning rate Momentum

Epoch size

Max. number of epochs to run

CV1 3 0.3 0.8 80 1000CV2 50 0.3 0.3 64 5000CV3 30 0.1 0.5 64 5000CV1 3 0.3 0.5 80 5000CV2 120 0.3 0.3 64 2000CV3 3 0.3 0.8 80 1000CV1 20 0.6 0.8 30 2000CV2 50 0.3 0.5 64 2000CV3 30 0.1 0.1 80 2000

5

2.3.2 ANN Testing

Once the neural network structure and parameters are determined, a 3-way cross

validation based on 10 trials is applied on the dataset. In sigmoid activation function, the

output is in the range of 0.2 to 0.8. Therefore the outputs have to be scaled back to original

data range for comparison with ground truth measurements.

2.3.3 Matlab Program Description

The driving program is LE_main.m which calls programs of data preprocessing

(LE_read.m and LE_preprocessing.m), data loading (LE_loaddata.m, LE_para.m), finding

best configuration (LE_bestconf.m), 3-way cross validation and comparing to baseline study

(LE_cv.m) in sequence. The programs will try to find the best configuration automatically

based on the parameter values specified in LE_para.m. However, as mentioned in ANN

structure section, the 3-way cross validation yields different results in terms of best

configurations. Therefore users are given an option to choose whether to use system

generated configuration or want to specify each parameter interactively. After the parameters

are chosen, the program runs a 3-way cross validation based on 10 trials and calculates R-

squared values for the results from MLP and baseline study.

3. Results

3.1 ANN Results

The whole dataset is originally tested with being partitioned into four subsets, among

which three subsets are used for 3-way cross validation and the forth for final testing. It turns

out that the R-squared value for the forth set is around 0.8, which is much higher than 0.62,

the R-squared value from a baseline study. However, the R-squared value for baseline study

is based on the whole dataset with a total of 348 records, while the R-squared value for the

6

forth subset is only based on one-forth of the total records. To make the result comparable,

the whole dataset is re-partitioned into three subsets with any two of the three subsets used

for training and the third for testing. The three testing outputs from 3-way cross validation

are then combined together to form a new dataset. The R-squared value based on this new

dataset is around 0.7, which is improved compared to a baseline study. Furthermore, the

slope between ground truth LE and approximated LE is 0.84, which is closer to 1 than 0.62

from the baseline study (Figure 1).

Figure 1 Comparison between MLP and baseline study

7

3.2 Baseline Study

The baseline study is based on a method proposed by Dr. Ramakrishna Nemani

(unpublished). In the method, evapotranspiration (ET) is proposed to be proportional to

potential evaporation (PET), which is the maximum rate of ET in an area, by a scaling factor.

The scaling factor is determined by analyzing the relationships between ground measurement

of vapor pressure deficit (VPD) and remote sensing derived saturated vapor pressure (SVP).

The potential evaporation is computed based on Priestley-Taylor’s equation as following:

Where PET is the potential evaporation, Q is surface radiation, is the derivative of the

saturation-vapor pressure versus temperature, , is the

psychrometric constant, and is Priestley-Taylor parameter. The method is employed to

estimate ET at 12 AmeriFlux sites (Table 1) using the same dataset described in this study.

The R-squared value between estimation and ground truth measurement is 0.62. The details

of the method are given in Appendix A.

4. Discussion

Evapotranspiration is a complex process affected by a variety of environmental factors.

The four factors included in this study are far from exhaustive, yet they are currently remote

sensing available. The advantage of remote sensing availability is the potential to apply ET

models over continental to global scales.

Previous researches on evapotranspiration modeling attempt to explicitly describe the

relationships between ET and its formative environmental factors. However, those

approaches become hard to accomplish when the relationships are complicated, which is

8

often the case in many ecosystem modeling. Artificial neural network (ANN) provides an

alternative way to explore underlying mechanisms of many ecosystems.

This study employs a multi-layer perceptron neural network to investigate the possibility

of predicting ET from remote sensed environmental factors. The result is better than a

baseline study conducted on explicit formulation of the relationships between ET and its

environmental variables. However, ecosystems are often driving by fundamental underlying

physical theories. If those underlying theories and knowledge can be integrated in neural

network training process, the expected performance of neural network in this study would be

much better than the baseline study.

Because of the complexity of data preprocessing and remote sensing data ordering, the

dataset used in this study only covers ground true evapotranspiration in 12 AmeriFlux tower

sites during year 2001. Worldwide multiple year datasets would be helpful in training ANN

and improving ANN performance. Applying the neural network obtained in this study to

create evapotranspiration maps over time and space can be conducted in future study.

This study only uses multi-layer perceptron back-propagation in approximating ET from

its environmental variables. Other neural network structures such as support vector machine,

radial basis network and mixture of expert systems would be worthwhile being tested to find

the best neural network structures for ET prediction.

9

Reference

Haykin, Simon. 1999. Neural networks: a comprehensive foundation. Prentice Hall, New Jersey, 2nd edition.

Nemani, R.R., and Running, S.W. 1989. “Estimation of regional surface resistance to evapotranspiration from NDVI and thermal-IR AVHRR data.” Journal of Applied Meteorology 28: 276-284.

Nemani, R.R., Pierce, L., Running, S.W., and Goward, S. 1993. “Developing satellite-derived estimate of surface moisture status.” Journal of Applied Meteorology 32: 548-557.

Nishida, K., Nemani R.R., Glassy J.M., and Running, S.W. 2003. “Development of an evapotranspiration index from Aqua/MODIS for monitoring surface moisture status.” IEEE Transactions on Geoscience ad Remote Sensing 41: 493-501.

Surface Radiation: ftp://nimbus32.umd.edu/pub/srb/gcip/

AmeriFlux: http://public.ornl.gov/ameriflux/navigation_bar/html_nav/Data.cfm

MODIS: http://daac.gsfc.nasa.gov/MODIS/data_access.shtml

10

Appendix A Baseline Study Method

The following codes are based on IDL programs, Interactive Data Language from

Research Systems Inc.

minvpd=700.maxvpd=3500.vslope = (1.0/(minvpd-maxvpd))vintercept = 1.0 - (vslope * minvpd)

; SVP(kPa) -> VPD (Pa); 0.36 is the slope between VPD and SVPVPD = SVP * 0.36 * 1000.

; VPD -> f(VPD) (0-1)kvpd = 1.0if (vpd gt minvpd) then begin

kvpd = vpd * vslope + vinterceptif (kvpd lt 0.0) then kvpd = 0.0if (kvpd gt 1.0) then kvpd = 1.0

endif

;Ra_avg is the 8-day average radiation derived from GOESalbedo = 0.05netr = Ra_avg * [1.0 - albedo]ground = (1.0 - EVI) * Ra_avgnetr = netr - ground

; delta (hPa/K)dt=0.2t1 = LSTd - dtt2 = LSTd + dtpvs1 = 6.107 * exp(17.38 * t1 / (239.0 + t1))pvs2 = 6.107 * exp(17.38 * t2 / (239.0 + t2))s = (pvs1-pvs2) / (t1-t2)

; psychrometric constant (hPa/K)psy = 0.646 + 0.0006 * LSTd

; PET (MJ/day); energy term (MJ/day)etf1 = s/(s + psy)* netr

;AET, kvpd is a scaling factorpre_et = etf1 * kvpd

; MJ -> mmpre_et[j] = pre_et/2.5

11

Appendix B Selective Matlab Programs

Back-propagation driving programs modified from Professor Yu Hen Hu’s codes include

LE_bp.m, LE_bpconfig.m, LE_bptestap.m, LE_cvgtest.m, actfun.m, and actfunp.m. The

codes for these programs are not included in the following code lists.

1. LE_main.m

% File: LE_main% Description: % This project is to explore the relationship between evaportranspiration% (LE) and the factors affecting LE through back-propagation approach. To% examine the relationship, the data are partitioned into 3 parts. Various% kinds of configuration are tested to find the best configuration for % # of neuron,learning rate, momentum, epoch size and maximum # of epochs% to run. Then a 3-way cross validation is applied. The result is compared% to a baseline study.%% Feature sequence: EVI:1; LSTd:2; SVPd:3; Ra_avg:4; LE:5;clear all,

LE_read;LE_preprocessing; %data preprocessingLE_loaddata; %load the data

LE_para; %load the testing values for each of the 5 parameter %# of hidden neuron, alpha, mom, epoch size, %maximum # of epoch to run LE_bestconf; %driving program, call: %LE_bpconfig,LE_bp, %LE_cvgtest,LE_bptestap %return best configuration

disp(['the best configuration matrix for each of the partition:']);best,

%choose the parameters to use%either use the one determined by the program disp('Enter use: 1 first row of best configuration matrix');disp(' 2 second row of best configuration matrix');disp(' 3 third row of best configuration matrix');datatype=input(' 4 user input (default):');

12

if isempty(datatype)||datatype>4||datatype<1, datatype=4; end;if datatype==4 cvbest(1)=input('# of hidden neuron ='); cvbest(2)=input('learning rate (alpha) ='); cvbest(3)=input('momentum ='); cvbest(4)=input('epoch size ='); cvbest(5)=input('maximum # of epoch to run ='); else cvbest=best(datatype,:);end

%cvbest=[3 0.3 0.8 64 2000];LE_cv;

2. LE_read.m

% File: LE_read - read dataset% Description: % flux_modis_all.txt includes 8-day average% surface flux recorded by Ameriflux towers;% Land surface temperature from MOD11,NDVI and EVI % from MOD13,and remote sensed surface radiation.% All the data are in year 2001.%% In this project EVI, LSTd, SVPd, and Ra_avg% are used to estimate LE.% EVI: enhanced vegetation index% LSTd: daytime land surface temperature% SVPd: daytime saturated vapor pressure% Ra_avg: average radiation% LE: latent heat flux (ET)%% Corresponding column number in flux_modis_all.txt:% LE:2; EVI:12; LSTd:13; SVPd:14, Ra_avg:17% After reading the data file, the corresponding column% sequence: EVI:1; LSTd:2; SVPd:3; Ra_avg:4; LE:5;

clear all,

load flux_modis_all.txt;modis=[flux_modis_all(:,12:14) flux_modis_all(:,17) flux_modis_all(:,2)];

% Original data includes filled-in data with -999. % Only valid data, ie. non -999 data are used.

13

ind=find(modis(:,1)~=-999&modis(:,2)~=-999&modis(:,3)~=-999&modis(:,4)~=-999&modis(:,5)~=-999);mdata=modis(ind,:);

% save datasavefile1 = 'RMODIS.txt'; %Raw modis datasave(savefile1,'mdata','-ASCII');

3. LE_preprocessing.m

% File: LE_preprocessing-normalize the data% mfile used: randomize.m% Description: normalize feature inputs% partition the data into 3-parts for% 3-way cross validation

clear all,load 'RMODIS.txt';RMODIS=randomize(RMODIS);[mrow mcol]=size(RMODIS);

% normalize input features;nfeature=mcol-1;

MTEMP=RMODIS(:,1:nfeature);mmean=mean(MTEMP);mstd=std(MTEMP);

% set denominator to 1 if it's 0ind=find(mstd(:)==0);mstd(ind)=1;

NMODIS=(MTEMP - ones(mrow,1)*mmean)./(ones(mrow,1)*mstd);NMODIS=[NMODIS RMODIS(:,nfeature+1)]; savefile1 = 'NMODIS.txt';save(savefile1,'NMODIS','-ASCII');

% modis data partition: 3 way cross validationN1=round(mrow/3);N2=N1;N3=mrow-N1-N2;

NMODIS1=NMODIS(1:N1,:);NMODIS2=NMODIS(N1+1:N1+N2,:);NMODIS3=NMODIS(N1+N2+1:mrow,:);

14

savefile1 = 'NMODIS1.txt';savefile2 = 'NMODIS2.txt';savefile3 = 'NMODIS3.txt';save(savefile1,'NMODIS1','-ASCII');save(savefile2,'NMODIS2','-ASCII');save(savefile3,'NMODIS3','-ASCII');

4. LE_read.m

% File: LE_read - read datasetclear all,

load flux_modis_all.txt;modis=[flux_modis_all(:,12:14) flux_modis_all(:,17) flux_modis_all(:,2)];

% Original data includes filled-in data with -999. % Only valid data, ie. non -999 data are used.

ind=find(modis(:,1)~=-999&modis(:,2)~=-999&modis(:,3)~=-999&modis(:,4)~=-999&modis(:,5)~=-999);mdata=modis(ind,:);

% save datasavefile1 = 'RMODIS.txt'; %Raw modis datasave(savefile1,'mdata','-ASCII'); 5. LE_para.m

%File: LE_para - set the parameter values being tested%Description: %testing different configuration, each parameter is tested%with 6 different values to find the best configuration.%The following is the configuration vectors. The %last column is the default setting.

setting=[3,5,12,20,30,50,5; %# of neurons 0.01,0.1,0.3,0.6,0.8,1.0,0.3; %learning rate 0.1,0.3,0.4,0.5,0.8,1.0,0.5; %momentum 10,15,20,30,64,80,30; %epoch size 200,300,500,1000,2000,5000,300]; %maximum # of epochs to run

%the 7th parameter is the default settingdef=7;

15

6. LE_bestconf.m

% File: LE_bestconf - find the best configuration% Description: % using 3-way cross validation to find the best % configuration for: #of hidden neuron,alpha,% momentum,epoch size,maximum #of epoch to run.%% Each of the 5 parameters is tested with 6 different values % while fixing other 4 paramters, i.e., other 4 parameters% are set to default value. Each value is run for 10 times. % The one with the smallest average testing error based on % 10 trials is chosen as the best configuration for that% parameter. The result is a combination of the 5 parameters% based on smallest mean testing error. This process is tested % for 10 times to get 10 combination of the 5 parameters. Then% each of the 10 combination of the 5 parameters is run for% 10 times and the one with the smallest testing error is chosen% as the best configuration for one data partitioning.%% mfile used: LE_bpconfig,LE_bp,LE_bptestap %% feature sequence: % EVI:1; LSTd:2; SVPd:3; Ra_avg:4; LE:5; disp(['begin to find the best configuration']); % best configuration for each of the 3-way cross validationbest=[];for q=1:3, %3-cross way validation to find best %configuration disp(['now processing loop q=' int2str(q) 'th partition(3CV)']); train0=trainr{q}; test0=testt{q}; %the index of the best configuration for all 5 par tested in %the 10 runs bestind1=[]; bestconf1=[]; %run 10 times to decide which configuration is the best for %qth partition: q=1,2,3 for i=1:10, disp(['now processing loop i=' int2str(i) 'th iterations']); %the index of the best configuration for all 5 par tested bestind2=[]; %the value of the best configuration for all 5 par tested

16

bestconf2=[]; %5 parameters are tested, after j-loop is over, the result %is the best configuration for the 5 parameters tested. for j=1:5, disp(['now processing loop j=' int2str(j) 'th paramters(total 5)']); %the average error(10 runnning) for each tested value %of a given parameter when other parameters are fixed Errtr3=[]; Errtt3=[]; %each parameter is tested with 6 different values, the %result is the best configuration for the parameter %tested when other parameter is fixed (using default value) for k=1:6, switch j case 1 para=[setting(j,k) setting(2,def) setting(3,def) setting(4,def) setting(5,def)]; case 2 para=[setting(1,def) setting(j,k) setting(3,def) setting(4,def) setting(5,def)]; case 3 para=[setting(1,def) setting(2,def) setting(j,k) setting(4,def) setting(5,def)]; case 4 para=[setting(1,def) setting(2,def) setting(3,def) setting(j,k) setting(5,def)]; case 5 para=[setting(1,def) setting(2,def) setting(3,def) setting(4,def) setting(j,k)]; end %switch LE_bpconfig; Errtr4=[]; %hold training errors for the 10 running Errtt4=[]; %hold testing error for the 10 running

%each value for a given parameter is tested for 10 times for p=1:10, LE_bp; %computing the best weight %Final training results: SSr=LE_bptestap(wbest,train,atype,zminr,zmaxr); %Apply trained MLP network to the testing data [SSt,target,approx]=LE_bptestap(wbest,test0,atype,zmint,zmaxt); Errtr4=[Errtr4 SSr]; Errtt4=[Errtt4 SSt]; end %loop p

17

Errtr3=[Errtr3 mean(Errtr4)]; Errtt3=[Errtt3 mean(Errtt4)]; end %loop k:6 different values for each parameter are tested %the best configuration for the given parameter tested when other %parameters are fixed is the one having the smallest testing %error. [tmpt2,indt2] = min(Errtt3); bestind2=[bestind2 indt2]; %1 by 5 bestconf2=[bestconf2 setting(j,indt2)]; %1 by 5 end %loop j:5 parameters

%10 by 5: 10 running for 5 par bestind1=[bestind1;bestind2]; bestconf1=[bestconf1;bestconf2]; end %loop i:10 running for best configuration

%now find which combination of the parameters in the %10 combinatin is the best Errtrjj=[]; Errttjj=[]; for jj=1:10, para=bestconf1(jj,:); LE_bpconfig; %hold training errors for the 10 times par combination=[]; Errtrpp=[]; Errttpp=[]; %each combination run 10 times for pp=1:10, LE_bp; %computing the best weight %Final training results: SSr=LE_bptestap(wbest,train,atype,zminr,zmaxr); %Apply trained MLP network to the testing data [SSt,target,approx]=LE_bptestap(wbest,test0,atype,zmint,zmaxt); Errtrpp=[Errtrpp SSr]; Errttpp=[Errttpp SSt]; end %loop pp

18

Errtrjj=[Errtrjj mean(Errtrpp)]; Errttjj=[Errttjj mean(Errttpp)]; end %loop jj

%the best combination of the parameters is the one %having the smallest testing error. [tmptjj,indtjj] = min(Errttjj); best=[best;bestconf1(indtjj,:)]; %3 by 5 end %loop: q: 3-cross way validation

disp(['the best configuration for each of the partition:']);best,

7. LE_cv.m

% File: LE_cv - 3-way cross validation% Description: % After running LE_bestconf, users have to% decide what is the best configuration. Then% LE_cv applies those configuration in 3-way% cross validation. The result is based on an % average of 10 trials. LE_fit is called to plot% the result and compared to a baseline study%% mfile used: LE_bpconfig, LE_bp, LE_bptestap, LE_fit

approx=[];target=[];for i=1:10, approx0=[]; target0=[]; for q=1:3, %3-cross way validation to find best %configuration disp(['now processing loop q=' int2str(q) 'th partition']); train0=trainr{q}; test0=testt{q}; para=cvbest; LE_bpconfig; LE_bp; %computing the best weight

%Final training results: SSr=LE_bptestap(wbest,train,atype,zminr,zmaxr);

19

%Apply trained MLP network to the reserved testing data [SSt,targ,est]=LE_bptestap(wbest,test0,atype,zmint,zmaxt); approx0=[approx0 est]; target0=[target0 targ]; end %loop: q: 3-cross way validation approx=[approx;approx0]; target=[target;target0]; end

approx=mean(approx);target=mean(target);

% call LE_fit.m to fit a line and % calculate R-squared[Rsquared,fx,fwdx,wd]=LE_fit(target,approx);[Rsquaredb,fxb,fwdxb,wdb]=LE_fit(baseline(1,:),baseline(2,:));

% plot the resultfigure(1),set(1,'Position',[100 100 650 600]),clf

subplot(211);plot(target,approx,'g.',fx,fwdx,'b','linewidth',2);text(0.1,3.5,['Equation: y = ' num2str(wd(1),2) ' + ' num2str(wd(2),2) 'x']);text(0.1,3.0,['R-squared: ' num2str(Rsquared,2)]);xlim([0,4]);ylim([0,4]);xlabel('True LE (mm/d)');ylabel('Predicted LE (mm/d)');title('Back-propagation','FontSize',11,'Fontweight','bold');

subplot(212);plot(baseline(1,:),baseline(2,:),'g.',fxb,fwdxb,'b','linewidth',2);text(0.1,3.5,['Equation: y = ' num2str(wdb(1),2) ' + ' num2str(wdb(2),2) 'x']);text(0.1,3.0,['R-squared: ' num2str(Rsquaredb,2)]);xlim([0,4]);ylim([0,4]);xlabel('True LE (mm/d)');ylabel('Predicted LE (mm/d)');title('Baseline Study','FontSize',11,'Fontweight','bold');

20

8. LE_cv.m

function [Rsquared,fx,fwdx,wd]=LE_fit(target,approx)% File: LE_fit - fit a line% Description: % fit a line and calculate R-squared% called by LE_cv;

x=target';y=approx';

[nrowr,ncolr]=size(x); % nrowr: #of row for training data, ncolr:#of col

% created samples data for drawing best fit linefx=[0:0.05:4]'; fl=length(fx);

%get least square minimized weight maxtrixxmatr=ones(nrowr,1); % xmat for training dataxmatr=[xmatr x];wd=pinv(xmatr)*y; % get (w0 w1 .... wn) % created samples data for the best fit functionxmatf=ones(fl,1);xmatf=[xmatf fx];

fwdx=xmatf*wd;

% calculating errorestr=xmatr*wd;errr=y-estr;errsum=sum(errr.*errr);sstot=sum((y-mean(y)).^2);Rsquared=(sstot-errsum)/sstot,

21