1

Basic statistical concepts and least-squares. Sat05_61.ppt, 2005-11-28.

1. Statistical concepts2. Distributions

• Normal-distribution

2. Linearizing

3. Least-squares• The overdetermined problem

• The underdetermined problem

2

Histogram.

Describes distribution of repeated observationsAt different times or places !

Distribution of global 10 mean gravity anomalies

3

Statistic:

Distributions describes not only random events !

We use statistiscal description for ”deterministic” quantities as well as on random quantities.

Deterministic quantities may look as if they have a normal distribution !

4

”Event”

Basic concept:Measured distance, temperature, gravity, …

Mapping:

Stocastisc variabel .In mathematics: functional, H – function-space – maybe

Hilbertspace.Gravity acceleration in point P: mapping of the space of all

possible gravity-potentials to the real axis.

RHX :

5

Probability-density , f(x):

What is the probability P that the value is in a specific interval:

dx f(x) = b) x P(ab

a

6

Mean and variance, Estimation-operator E:

))E(( :momentth n'

))(()()(

Variance

)()(x

:value-Mean

-

222x

-

nxx

xxEdxxfxx

xEdxxfx

7

Variance-covariance in space of several dimensions:

Mean value and variances:

jijjiiij

ijxx

dxdxxxxx ))((

2

8

Correlation and covariance-propagation:

Correlation between two quantities: = 0: independent.

Due to linearity

1,1jjii

ijij

)()()( ybExaEYbXaE

9

Mean-value and variance of vector

If X and A0 are vectors of dimension n and A is an n x m matrix, then

The inverse P = generally denoted the weight-matrix

TX

TY AAYEYYEyE

XEAAYEXAAY

)))(())(((

)()( 00

10

Distribution of the sum of 2 numbers:

• Exampel Here n = 2 and m = 1. We regard the sum of 2 observations:

• What is the variance, if we regard the difference between two observations ?

2211YY21

22

11

X0

+ = ,X + X = Y

0

0 = 1}, ,{1 = A {0}, = A

11

Normal-distribution

1-dimensional quantity has a normal distribution if

• Vektor of simultaneously normal-distributed quantities if

• n-dimensional normal distribution.

e2

1 = f(x) 2)/()E(X)--(x

xx

xx2

e )()(2

1 = )x,...,xF( E(X))/2-(X P E(X))--(X

X1/2n/2n1

T

det

12

Covarians-propagation in several dimensions:

X: n-dimensional, normally distributed,

D nxm matrix , then

Z=DZ also normal distributed,

E(Z)=D E(X)

TXZ DDZE )( 2

13

Estimate of mean, variance etc

products ofnumber

/)ˆ)(ˆ(),cov( :Covariance

1

)ˆ( :deviation-Standard

/ˆ :Mean

1

1

2

1

n

nyyxxyx

n

xx

nxx

n

iii

n

i

ix

n

ni

14

Covarians function

If the covariance COV(x,y) is a function of x,y then we have a

Covarians-function

May be a function of

• Time-difference (stationary)

• Spherical Distance, ψ on the unit-sphere (isotrope)

15

Normally distributed data and resultat.

If data are normaly dsitributed, then the resultats are also normaly distributed

If they are linearily related !

We must linearize –

TAYLOR-Development with only 0 and 1. order terms.

Advantage: we may interprete error-distributions.

16

Distributions in infinite-dimensional spaces

V( P) element in separable Hilbert-space:

Normal distributed with sum of variances finite !

ijij

ij

ijiji

i

ij

GMCVX

V

PVCGMPV

)(

:(example) variableStochastic

functions-base orthogonal

),()(0

17

Stochastic process.

What is the probability P for the event is located in a specific interval

Exampel: What is the probability that gravity in Buddinge lies in between -20 and 20 mgal and that gravity in Rockefeller lies in the same interval

),( 21 dXcbXaP

18

Stokastisc process in Hilbertspace

What is the mean value and variance of ”the Evaluation-functional”,

2

0

2

2

0

0

22

)(:

)()()(

0)()()(

0)(,)(,0)(

),()(

iiP

iiiiqp

iiiP

jiiii

P

EvEVariance

QVPVEvEvE

PVXEEvE

XXEXEXEwith

PTTEv

19

Covariance function of stationary time-series.

Covariance-function depends only on |x-y|

Variances called ”Power-spectrum”.

y))-(i(x

= f(y)) E(f(x) = y)COV(x,

2i

N

=0i

2i

N

=0i

= y)) (ix) (i + y) (ix) (i(

cos

sinsincoscos

2

i2

i2

iii

N

=0i

= ))bE(( = ))aE(( x)), N

i( b + x)

N

i( a( = f(x)

2sin

2cos

20

Covariance function – gravity-potential.

• Suppose Xij normal-distributed with the same variance for constant ”i”.

1)+/(2i= = ))CR

GM(E( 2

i2ij

2ij

ts.coefficien normalizedfully

radius,mean sEarth' R longitude, latitude,

),,(),,()(2

ij

i

i

ijij

i

ij

C

Vr

RC

r

GMrTPT

21

Isotropic Covariance-function for Gravity potential

distance. spherical

spolynomial Legendre ),(cos'

)','(),('

)12/(

))(),((),(

2

122

2

122

iii

i

i

iijij

ii

iji

PPrr

R

VVrr

Ri

QTPTEQPCOV

22

Linearizering: why ?

We want to find best estimate (X) for m quantities from n observations (L).

Data normal-distributed, implies result normaly distributed, if there is a linear relationship.

If m > n there exist an optimal metode for estimating X:

Metode of Least-Squares

23

Linearizing – Taylor-development.

If non-linear:

Start-værdi (skøn) for X kaldes X1

Taylor-development with 0 og 1. order terms after changing the order

1010 ,),( XXxLLyXL

|}X

{ = A x, A = v +y X 1

.parameters ofFunction noisensobservatio

or )(

XL

24

Covariance-matrix for linearizered quantities

If measurements independently normal distributed with varians-covariance

Then the resultatet y normal-dsitributed with variance-covarians:

ij

Tijy AA

25

Linearizing the distance-equation.

Linearized based on coordinates

)X ,X(X-X = |

X

orden.-2 af led + )X-X(|X

+ )X ,X( = )X (X,

01

0i1i

X1i

1iiX1i

3

=1i010

1

1

2033

2022

20110 )()()(),( XXXXXXXX

),,( 131211 XXX

26

On Matrix form:

If

3 equations with 3 un-knowns !

)( 0iii XXdX

yxAorcomputedobserved

XXXXdXX

XXi

T

Xi

,

),(),(),(

0100

0

27

Numerical-example

If (X11, X12,X13) = ( 3496719 m, 743242 m, 5264456 m).

Satellite: (19882818.3, -4007732.6 , 17137390.1)

Computed distance: 20785633.8 m

Measured distance: 20785631.1 m

((3496719.0-19882818.3)dX1 + (743242.0-4007732.6) dX2+(5264456 .0-17137390.1) dX3)/20785633.8 =

( 20785631.1 - 20785633.8) or:

-0.7883 dX1 -0.1571 dX2 + 1.7083 dX3 = -2.7

28

Linearizing in Physical Geodesy based on T=W-U

In function-spaces the Normal-potential may be regarded as a 0-order term in a Taylor-development.We may differentiate in Metric space (Frechet-derivative).

anomaly)(gravity 2

anomaly-height /

Trdr

dTg

T

29

Method of Least-Square. Over-determined problem.

More observations than parameters or quantities which must be estimated:

Examples: GPS-observations, where we stay at the same

place (static)We want coordinates of one or more points.

Now we suppose that the unknowns are m linearily independent quantities !

30

Least-squares = Adjustment.

• Observation-equations:

• We want a solution so that

Differentiation:

•

xAy

minimum(x) = )x A-(y x) A -(y = v v T-1y

T-1y

ningerne)(Normalligy A )A A( = x

y A = x A A

0 = x)A()A(2 - )(A y2

0 = )Ax -(y Ax) -(y xd

d

1-y

T1-1-y

T

1-y

1-y

T

ii1-

yT

ii1-

yi

T1-y

i

31

Metod of Last-Squares. Variance-covariance.

)A A(

= )A)A A( A )A A(

=

1-1-y

T

T1-y

T1-1-y

Ty

1-y

T1-1-y

T

x

32

Metod of Least-Squares. Linear problem.

Gravity observations:

H, g=981600.15 +/-0.02 mgal

G

I10.52+/-0.03

12.11+/-0.03

-22.7+/-0.03

33

Observations-equations..

22.70-

10.52

12.11

981600.15

=

g

g

g

1- 0 1

1 1- 0

0 1 1-

0 0 1

I

H

G

1- 0 1

1 1- 0

0 1 1-

0 0 1

030. 0.00 0.00 0.00

0.00 030. 0.00 0.00

0.00 0.00 030. 0.00

0.00 0.00 0.00 020.

1- 1 0 0

0 1- 1 0

1 0 1- 1

=

2

2

2

2 -1

1-x

34

Method of Least-Squares. Over-determined problem.

Compute the varianc-covariance-matrix

35

Method of Least-Squares.

Optimal if observations are normaly distributed + Linear relationship !

Works anyway if they are not normally distributed !

And the linear relationship may be improved using iteration.

Last resultat used as a new Taylor-point.

Exampel: A GPS receiver at start.

36

Metode of Least-Squares. Under-determined problem.

We have fewer observations than parameters: gravity-field, magnetic field, global temperature or pressure distribution.

We chose a finite dimensional sub-space, dimension equal to or smaller than number of observations.

Two possibilities (may be combined):

• We want ”smoothest solution” = minimum norm

• We want solution, which agree as best as possible with data, considering the noise in the data

37

Method of Least-Squares. Under-determined problem.

Initially we look for finite-dimensional space so the solution in a variable point Pi becomes a linear-combination of the observations yj:

If stocastisk process, we want the ”interpolation-error” minimalized

y = x jij

n

j=1i ~

)minimum( = )yyE( y 2- )xE( =

)y - E(x = )x - E(x

ijiji

n

j=1

n

=1iii

n

=1i

2

2ii

n

=1i

2

+ )E(x

~

38

Method of Least-Squares. Under-determined problem.

Covariances:

Using differentiation:

Error-variance:

)xE( = C ),yE(x = C ),yyE( = C 20iPijiij

y)C()C( = x -1ijPi ~

C = E(xy) ,C = ))y()yE(( 0, = E(y) 0, = E(x) med

C C C -

= C C )yE( C C +

y)E(x CC 2 - )xE( = ))x - E((x

PjijT

ji

Pjij1-

PiT2

x

PjT

ij1-2

ij1-

Pi

ij-1

PiT22

~

39

Method of Least-Squares. Gravity-prediction.

Example:

Covarianses: COV(0 km)= 100 mgal2

COV(10 km)= 60 mgal2

COV(8 km)= 80 mgal2

COV(4 km)= 90 mgal2

P6 mgal

Q10 mgal

R

10 km

8 km 4 km

40

Method of Least-Squares. Gravity prediction.

Continued:

Compute the error-estimate for the anomaly in R.

mgal 9 = 6

10

100 60

60 100 80 90 = g

-1

R

~

41

Least-Squares Collocation.

Also called: optimal linear estimation

For gravity field: name has origin from solution of differential-equations, where initial values are maintained.

Functional-analytic version by Krarup (1969)

Kriging, where variogram is used closely connected to collocation.

42

Least-Squares collocation.

We need covariances – but we only have one Earth.

Rotate Earth around gravity centre and we get (conceptually) a new Earth.

Covariance-function supposed only to be dependent on spherical distance and distance from centre.

For each distance-interval one finds pair of points, of which the product of the associated observations is formed and accumulated. The covariance is the mean value of the product-sum.

43

Covarians-function for gravity anomalies, r=R.

distance. spherical fixed

,),','(),,()(

),(2

2

Earthazimuth

i

i

ijij

i

ij

ddRgRgC

Vr

RC

r

GMg

44

Covarians function for gravity-anomalies:

Different models for degree-variances (Power-spectrum):

Kaula, 1959, (but gravity get infinite variance)

Tscherning & Rapp, 1974 (variance finite).

ser)gradvarian mali(tyngdeano ,C )1-(iR

GM =

),(P = )C(

ij2

j

-ij=2

2

2i

i2i

2=i

cos