Nonlinear Regression - MATH FOR...

3/19/2015 http://numericalmethods.eng.usf.edu 1

Nonlinear Regression

Major: All Engineering Majors

Authors: Autar Kaw, Luke Snyder

http://numericalmethods.eng.usf.eduTransforming Numerical Methods Education for STEM

Undergraduates

http://numericalmethods.eng.usf.edu/


http://numericalmethods.eng.usf.edu



)( bxaey =

)( baxy =

+=

xbaxy

Some popular nonlinear regression models:

1. Exponential model:

2. Power model:

3. Saturation growth model:

4. Polynomial model: )( 10m

mxa...xaay +++=

http://numericalmethods.eng.usf.edu3

Nonlinear RegressionGiven n data points ),( , ... ),,(),,( 2211 nn yxyx yx best fit )(xfy =

to the data, where )(xf is a nonlinear function of x .

Figure. Nonlinear regression model for discrete y vs. x data

)(xfy =

),(nn

yx

),(11

yx

),(22

yx

),(ii

yx)(

iixfy −


RegressionExponential Model


Exponential Model),( , ... ),,(),,( 2211 nn yxyx yxGiven best fit bxaey = to the data.

Figure. Exponential model of nonlinear regression for y vs. x data

bxaey =

),(nn

yx

),(11

yx

),(22

yx

),(ii

yxibx

i aey −


Finding Constants of Exponential Model

( )∑=

−=n

i

bx

ir iaeyS

1

2The sum of the square of the residuals is defined as

Differentiate with respect to a and b

( )( ) 021

=−−=∂∂ ∑

=

ii bxn

i

bxi

r eaeyaS

( )( ) 021

=−−=∂∂ ∑

=

ii bxi

n

i

bxi

r eaxaeybS


Finding Constants of Exponential Model

Rewriting the equations, we obtain

01

2

1=∑+∑−

==

n

i

bxn

i

bxi

ii eaey

01

2

1=∑−∑

==

n

i

bxi

n

i

bxii

ii exaexy


Finding constants of Exponential Model

Substituting a back into the previous equation

01

2

1

2

1

1=∑

∑

∑−∑

=

=

=

=

n

i

bxin

i

bx

bxn

ii

bxi

n

ii

i

i

i

i exe

eyexy

The constant b can be found through numerical methods such as bisection method.

∑

∑=

=

=n

i

bx

n

i

bxi

i

i

e

eya

1

2

1

Solving the first equation for a yields


Example 1-Exponential Model

t(hrs) 0 1 3 5 7 91.000 0.891 0.708 0.562 0.447 0.355

Many patients get concerned when a test involves injection of a radioactive material. For example for scanning a gallbladder, a few drops of Technetium-99m isotope is used. Half of the Technetium-99m would be gone in about 6 hours. It, however, takes about 24 hours for the radiation levels to reach what we are exposed to in day-to-day activities. Below is given the relative intensity of radiation as a function of time.

Table. Relative intensity of radiation as a function of time.

γ


Example 1-Exponential Model cont.

Find: a) The value of the regression constants A λand

b) The half-life of Technetium-99m

c) Radiation intensity after 24 hours

The relative intensity is related to time by the equationtAeλγ =


Plot of data


Constants of the Model

The value of λ is found by solving the nonlinear equation

( ) 01

2

1

2

1

1=∑

∑

∑−∑=

=

=

=

=

n

i

tin

i

t

n

i

ti

ti

n

ii

i

i

i

i ete

eetf λ

λ

λ

λγ

γλ

∑

∑

=

== n

i

t

n

i

ti

i

i

e

eA

1

2

1

λ

λγ

tAeλγ =


Setting up the Equation in MATLAB

( ) 01

2

1

2

1

1=∑

∑

∑−∑=

=

=

=

=

n

i

tin

i

t

n

i

ti

ti

n

ii

i

i

i

i ete

eetf λ

λ

λ

λγ

γλ

t (hrs) 0 1 3 5 7 9γ 1.000 0.891 0.708 0.562 0.447 0.355


Setting up the Equation in MATLAB

( ) 01

2

1

2

1

1=∑

∑

∑−∑=

=

=

=

=

n

i

tin

i

t

n

i

ti

ti

n

ii

i

i

i

i ete

eetf λ

λ

λ

λγ

γλ

t=[0 1 3 5 7 9]gamma=[1 0.891 0.708 0.562 0.447 0.355]syms lamdasum1=sum(gamma.*t.*exp(lamda*t));sum2=sum(gamma.*exp(lamda*t));sum3=sum(exp(2*lamda*t));sum4=sum(t.*exp(2*lamda*t));f=sum1-sum2/sum3*sum4;

1151.0−=λ


Calculating the Other Constant

The value of A can now be calculated

∑

∑

=

== 6

1

2

6

1

i

t

i

ti

i

i

e

eA

λ

λγ9998.0=

The exponential regression model then iste 1151.0 9998.0 −=γ


Plot of data and regression curve

te 1151.0 9998.0 −=γ


Relative Intensity After 24 hrs

The relative intensity of radiation after 24 hours

( )241151.09998.0 −×= eγ2103160.6 −×=

This result implies that only

%317.61009998.0

10316.6 2

=×× −

radioactive intensity is left after 24 hours.


Homework• What is the half-life of Technetium-99m

isotope?• Write a program in the language of your

choice to find the constants of the model.

• Compare the constants of this regression model with the one where the data is transformed.

• What if the model was ?


teλγ =

THE END

http://numericalmethods.eng.usf.eduhttp://numericalmethods.eng.usf.edu20

Polynomial Model),( , ... ),,(),,( 2211 nn yxyx yxGiven best fit

mm

xa...xaay +++=10

)2( −≤ nm to a given data set.

Figure. Polynomial model for nonlinear regression of y vs. x data

mm

xaxaay +++=

10

),(nn

yx

),(11

yx

),(22

yx

),(ii

yx)(

iixfy −


Polynomial Model cont.The residual at each data point is given by

mimiii xaxaayE −−−−= ...10

The sum of the square of the residuals then is

( )∑

∑

=

=

−−−−=

=

n

i

mimii

n

iir

xaxaay

ES

1

210

1

2

...


Polynomial Model cont.To find the constants of the polynomial model, we set the derivatives with respect to ia where

( )

( )

( ) 0)(....2

0)(....2

0)1(....2

110

110

1

110

0

=−−−−−=∂∂

=−−−−−=∂∂

=−−−−−=∂∂

∑

∑

∑

=

=

=

mi

n

i

mimii

m

r

i

n

i

mimii

r

n

i

mimii

r

xxaxaayaS

xxaxaayaS

xaxaayaS

,,1 mi = equal to zero.


Polynomial Model cont.These equations in matrix form are given by

=

∑

∑

∑

∑∑∑

∑∑∑

∑∑

=

=

=

==

+

=

=

+

==

==

n

ii

mi

n

iii

n

ii

mn

i

mi

n

i

mi

n

i

mi

n

i

mi

n

ii

n

ii

n

i

mi

n

ii

yx

yx

y

a

aa

xxx

xxx

xxn

1

1

1

1

0

1

2

1

1

1

1

1

1

2

1

11

......

...

...........

...

...

The above equations are then solved for maaa ,,, 10


Example 2-Polynomial Model

Temperature, T(oF)

Coefficient of thermal

expansion, α (in/in/oF)

80 6.47×10−6

40 6.24×10−6

−40 5.72×10−6

−120 5.09×10−6

−200 4.30×10−6

−280 3.33×10−6

−340 2.45×10−6

Regress the thermal expansion coefficient vs. temperature data to a second order polynomial.

1.00E-06

2.00E-06

3.00E-06

4.00E-06

5.00E-06

6.00E-06

7.00E-06

-400 -300 -200 -100 0 100 200

Temperature, oF

Ther

mal

exp

ansi

on c

oeffi

cien

t, α

(in/in

/o F)

Table. Data points for temperature vs

Figure. Data points for thermal expansion coefficient vs temperature.

α


Example 2-Polynomial Model cont.

2210 TaTaaα ++=

We are to fit the data to the polynomial regression model

=

∑

∑

∑

∑∑∑

∑∑∑

∑∑

=

=

=

===

===

==

n

iii

n

iii

n

ii

n

ii

n

ii

n

ii

n

ii

n

ii

n

ii

n

ii

n

ii

T

Taaa

TTT

TTT

TTn

1

2

1

1

2

1

0

1

4

1

3

1

2

1

3

1

2

1

1

2

1

α

α

α

The coefficients 210 , a,aa are found by differentiating the sum of thesquare of the residuals with respect to each variable and setting thevalues equal to zero to obtain


Example 2-Polynomial Model cont.The necessary summations are as follows

Temperature, T(oF)

Coefficient of thermal expansion,

α (in/in/oF)

80 6.47×10−6

40 6.24×10−6

−40 5.72×10−6

−120 5.09×10−6

−200 4.30×10−6

−280 3.33×10−6

−340 2.45×10−6

Table. Data points for temperature vs. α 57

1

2 105580.2 ×=∑=i

iT

77

1

3 100472.7 ×−=∑=i

iT

107

1

4 101363.2∑=

×=i

iT

57

1103600.3 −

=

×=∑i

iα

37

1106978.2 −

=

×−=∑i

iiTα

17

1

2 105013.8 −

=

×=∑i

iiT α


Example 2-Polynomial Model cont.

××−×

=

××−××−××−××−

−

−

−

1

3

5

2

1

0

1075

752

52

105013.8106978.2

103600.3

101363.2100472.7105800.2100472.7105800.210600.8

105800.2106000.80000.7

aaa

Using these summations, we can now calculate 210 , a,aa

Solving the above system of simultaneous linear equations we have

×−××

=

−

−

−

11

9

6

2

1

0

102218.1102782.6100217.6

aaa

The polynomial regression model is then

21196

2210

T101.2218T106.2782106.0217

α−−− ×−×+×=

++= TaTaa


Transformation of DataTo find the constants of many nonlinear models, it results in solving simultaneous nonlinear equations. For mathematical convenience, some of the data for such models can be transformed. For example, the data for an exponential model can be transformed.

As shown in the previous example, many chemical and physical processes are governed by the equation,

bxaey =Taking the natural log of both sides yields,

bxay += lnlnLet yz ln= and aa ln0 =

(implying) oaea = with ba =1

We now have a linear regression model where xaaz 10 +=


Transformation of data cont.Using linear model regression methods,

_

1

_

0

1

2

1

2

11 11

xaza

xxn

zxzxna

n

i

n

iii

n

ii

n

i

n

iiii

−=

−

−=

∑ ∑

∑∑ ∑

= =

== =

Once 1,aao are found, the original constants of the model are found as

0

1aea

ab

=

=


THE END

http://numericalmethods.eng.usf.eduhttp://numericalmethods.eng.usf.edu31

Example 3-Transformation of data

t(hrs) 0 1 3 5 7 9

1.000 0.891 0.708 0.562 0.447 0.355

Many patients get concerned when a test involves injection of a radioactive material. For example for scanning a gallbladder, a few drops of Technetium-99m isotope is used. Half of the Technetium-99m would be gone in about 6 hours. It, however, takes about 24 hours for the radiation levels to reach what we are exposed to in day-to-day activities. Below is given the relative intensity of radiation as a function of time.

Table. Relative intensity of radiation as a function

γ

of time

0

0.5

1

0 5 10

Rela

tive

inte

nsity

of r

adia

tion,

γ

Time t, (hours)

Figure. Data points of relative radiation intensityvs. time


Example 3-Transformation of data cont.

Find: a) The value of the regression constants A λand

b) The half-life of Technetium-99m

c) Radiation intensity after 24 hours

The relative intensity is related to time by the equationtAeλγ =



tAeλγ =Exponential model given as,

( ) ( ) tA λγ += lnlnAssuming γln=z , ( )Aao ln= and λ=1a we obtain

taaz10

+=This is a linear relationship between z and t



Using this linear relationship, we can calculate 10 ,aa

∑ ∑

∑∑ ∑

= =

== =

−

−=

n

i

n

ii

n

ii

n

i

n

iiii

ttn

ztztna

1

2

1

21

11 11

andtaza 10 −=

where

1a=λ

0aeA =


Example 3-Transformation of Data cont.

123456

013579

10.8910.7080.5620.4470.355

0.00000−0.11541−0.34531−0.57625−0.80520−1.0356

0.0000−0.11541−1.0359−2.8813−5.6364−9.3207

0.00001.00009.000025.00049.00081.000

25.000 −2.8778 −18.990 165.00

Summations for data transformation are as follows

Table. Summation data for Transformation of data model

i it iγ iiz γln= ii

zt 2it

∑

With 6=n

000.256

1∑=

=i

it

∑=

−=6

18778.2

ii

z

∑=

−=6

1990.18

iii

zt

00.1656

1

2 =∑=i

it



Calculating 10 ,aa( ) ( )( )

( ) ( )21 2500.16568778.225990.186

−−−−

=a 11505.0−=

( )62511505.0

68778.2

0 −−−

=a 4106150.2 −×−=

Since( )Aa ln0 =0aeA =

4106150.2 −×−= e 99974.0=

11505.01 −== aλalso



Resulting model is te 11505.099974.0 −×=γ

0

0.5

1

0 5 10

Time, t (hrs)

Relative Intensity

of Radiation,

te 11505.099974.0 −×=γ

Figure. Relative intensity of radiation as a function of temperature using transformation of data model.



The regression formula is thente 11505.099974.0 −×=γ

b) Half life of Technetium-99m is when02

1

=

=t

γγ

( ) ( )

( )hours.t

.t..e

e.e.

t.

.t.

0248650ln115050

50

99974021999740

115080

0115050115050

==−

=

=×

−

−−



c) The relative intensity of radiation after 24 hours is then( )2411505.099974.0 −= eγ

063200.0=This implies that only %3216.6100

99983.0103200.6 2

=×× −

of the radioactive

material is left after 24 hours.


Comparison Comparison of exponential model with and without data Transformation:

With data Transformation

(Example 3)

Without data Transformation

(Example 1)

A 0.99974 0.99983

λ −0.11505 −0.11508

Half-Life (hrs) 6.0248 6.0232

Relative intensity after 24 hrs. 6.3200×10−2 6.3160×10−2

Table. Comparison for exponential model with and without dataTransformation.


Additional ResourcesFor all resources on this topic such as digital audiovisual lectures, primers, textbook chapters, multiple-choice tests, worksheets in MATLAB, MATHEMATICA, MathCad and MAPLE, blogs, related physical problems, please visit

http://numericalmethods.eng.usf.edu/topics/nonlinear_regression.html

http://numericalmethods.eng.usf.edu/topics/nonlinear_regression.html

THE END

http://numericalmethods.eng.usf.edu


Nonlinear Regression - MATH FOR...

Documents

Transcript of Nonlinear Regression - MATH FOR...