Design of Data Association Filter Using NeuralNetworks for Multi-Target Tracking

Yang Weon Lee and Chil Woo Lee

Department of Information and Communication Engineering, Honam University,Seobongdong, Gwangsangu, Gwangju, 506-714, South Korea

Department of Computer Engineering, Chonnam University, Yongbongdong,Gwangju, South Korea

ywlee@honam.ac.kr, cwlee@chonnam.ac.kr

Abstract. In this paper, we have developed the MHDA scheme for dataassociation. This scheme is important in providing a computationally fea-sible alternative to complete enumeration of JPDA which is intractable.We have proved that given an articial measurement and tracks con-guration, MHDA scheme converges to a proper plot in a nite numberof iterations. Also, a proper plot which is not the global solution canbe corrected by re-initializing one or more times. In this light, even ifthe performance is enhanced by using the MHDA, we also note thatthe diculty in tuning the parameters of the MHDA is critical aspectof this scheme. The diculty can, however, be overcome by developingsuitable automatic instruments that will iteratively verify convergenceas the network parameters vary.

1 Introduction

Generally, there are three approaches in data association for MTT : non-Bayesianapproach based on likelihood function[1], Bayesian approach[2,4,3], and neuralnetwork approach[5]. The major dierence of the rst two approaches is howtreat the false alarms. The non-Bayesian approach calculates all the likelihoodfunctions of all the possible tracks with given measurements and selects thetrack which gives the maximum value of the likelihood function. Meanwhile,the tracking lter using Bayesian approach predicts the location of interest us-ing a posteriori probability. These two approaches are inadequate for real timeapplications because the computational complexity is tremendous.

As an alternative approach, Sengupta and Iltis[5] suggested a Hopeld neuralnetwork probabilistic data association (HNPDA) to approximately compute aposteriori probability tj , for the joint probabilities data association lter(JPDAF)[7] as a constrained minimization problem. This technique based onthe use of neural networks was also started by comparison with the travelingsalesman problem(TSP). In fact tj is approximated by the output voltage X

tj

of a neuron in an (m + 1) n array of neurons, where m is the number of mea-surements and n is the number of targets. Sengupta and Iltis[5] claimed that theperformance of the HNPDA was close to that of the JPDAF in situations wherethe numbers of measurements and targets were in the ranges of 3 to 20 and 2

D.-S. Huang, K. Li, and G.W. Irwin (Eds.): ICIC 2006, LNCS 4113, pp. 981990, 2006.c Springer-Verlag Berlin Heidelberg 2006

982 Y.W. Lee and C.W. Lee

to 6, respectively. The success of the HNPDA in their examples was credited tothe accurate emulation of all the properties of the JPDAF by the HNPDA.

However, the neural network developed in [5] has been shown the two prob-lems. First, the neural network developed in [5] has been shown to have improperenergy functions. Second, heuristic choices of the constant parameters in the en-ergy function in [5] didnt guarantee the optimal data association.

The outline of this paper is as follows. In section 2 the Hopeld neural networkused in [5] is briey reviewed and some comments are made on the assumptionswhich used to set up the energy function in [5]. Then, the modied scheme ofHNPDA is proposed as an alternative data association method for MTT. Finally,we present our simulation results in Section 4, and conclusions in Section 5.

2 Review of the Energy Function in the HNPDA andComments

Suppose there are n targets and m measurements. The energy function used in[5] is reproduced below.

EDAP =A

2

m

j=0

n

t=1

n

=1 =tXtjX

j +

B

2

n

t=1

m

j=0

m

l=0l =jXtjX

tl +

C

2

n

t=1

(m

j=0

Xtj 1)2

+D

2

m

j=0

n

t=1

(Xtj tj)2 +E

2

m

j=0

n

t=1

n

=1 =t(Xtj

m

l=0l =jl )

2. (1)

In [5], Xtj is the output voltage of a neuron in an (m+1)n array of neuronsand is the approximation to the a posteriori probability tj in the JPDAF[7]. Thisa posteriori probability, in the special case of the PDAF[7] when the probabilityPG that the correct measurement falls inside the validation gate is unity, isdenoted by tj . Actually, PG is very close to unity when the validation gate sizeis adequate. In (1), A,B,C,D, and E are constants.

In the HNPDA, the connection strength matrix is a symmetric matrix of ordern(m+1). With the given energy function EDAP in (1), the connection strengthW tjl from the neuron at location (, l) to the neuron at location (t, j) is

W tjl =

[C + D + E(n 1)] if t = and j = l self feedback,A if t = and j = l row connection,(B + C) if t = and j = l column connection,0 if t = and j = l global connection.

(2)

The input current Itj to the neuron at location (t, j), for t = 1, 2, . . . , n, andj = 0, 1, . . . ,m, is

Itj = C + (D + E)tj + E

(n 1

n

=1

j

). (3)

AdministratorHighlight

Design of Data Association Filter Using Neural Networks 983

Clearly from (2)and (3), the input current Itj but not the connection strengthT tjl depends on the

tj s, which are computed form the measurements that com-

prise the input data. Ironically, in the neural network for the TSP[9], only theconnection strengths depend on the input data which, in this case, are the dis-tances between pairs of cities.

In order to justify the rst two terms of EDAP in (1), the authors of [5]claimed that the dual assumptions of no two returns form the same target andno single return from two targets are consistent with the presence of a domi-nating Xtj in each row and each column of the (m + 1) n array of neurons.However, these assumptions are not constraints on the values of the tj s inthe original JPDAF. Those assumptions should be used only in the generationof the feasible data association hypotheses, as pointed out in [7]. As a mat-ter of fact, there could be two tj s of comparable magnitude in the same rowand in the same column as shown in Chapter 4 of [8]. Therefore, the pres-ence of a dominating Xtj in each row and each column is not a property of theJPDAF.

The third term of EDAP is used to constrain the sum of the Xtj s in eachcolumn to unity i.e.

mj=0 X

tj = 1. This constraint is consistent with the re-

quirement thatm

j=0 tj = 1 in both the JPDAF and the PDAF[10]. Therefore,

this constraint, by itself, does not permit us to infer whether the tj s are fromthe JPDAF, or from the PDAF. The assumption used to set up the fourth termis that this term is small only if Xtj is close to

tj , in which case the neural net-

work simulates more closely the PDAF for each target rather than the intendedJPDAF in the multitarget scenario. Finally, the fth term is supposed to be min-imized if Xtj is not large unless for each = t there is a unique l = j such thatl is large. Unfortunately, this constrained minimization may not be possible asshown in [8]. This is consistent with the heuristic nature of the derivation of theenergy function in [5], which could lead to the problems in the implementationof the HNPDA as discussed next.

3 Modied Scheme of HNPDA

3.1 Modication of Energy Equation

In Hopeld network, when the operation is approaching the steady state, atmost one neuron gets into the ON state in each row and column and the otherneurons must be in the OFF state. To guarantee the this state, we add thefollowing constraints additionally for Hopeld network :

Es =12

m

j=1

n

t=0

n

=tjtj +

12

n

t=1

m

j=1

m

l =jjtlt. (4)

By summing (4) and (10) in [11], we get the nal energy equation for Hopeldnetwork:

984 Y.W. Lee and C.W. Lee

EHDA =A

2

m

j=1

n

t=0

n

=tjtj +

B

2

n

t=1

m

j=1

m

l =jjtlt +

C

2

n

t=1

m

j=1

(wjt wjt)2

+D

2

n

t=1

(m

j=1

wjt 1)2 + F2m

j=1

(n

t=0

wjt 1)2 + G2n

t=1

m

j=1

rjtwjt. (5)

The rst two terms of (5) correspond to row and column inhibition and thethird term suppressed the activation of uncorrelated part( i.e. if jt = 0, thenjt = 0). The fourth and fth terms biased the nal solution towards a nor-malized set of numbers. The last term favors associations which have a nearestneighbor in view of target velocity.

3.2 Transformation of Energy Function into Hopeld Network

A Hopeld network with m(n + 1) neurons was considered. The neurons weresubdivided into n + 1 targets column of m neurons each. Henceforward wewill identify each neuron with a double index, tl(where the index t = 0, 1, . . . , nrelates to the target, whereas the index l = 1, . . . ,m refers to the neurons in eachcolumn), its output with Xtl , the weight for neurons jt and l with W

tjl , and

the external bias current for neuron tl with Itl . According to this convention, wecan extend the notation of the Lyapunov energy function[12] to two dimensions.Using the Kronecker delta function

ij ={

1 if i = j,0 if i = j, (6)

The Lyapunov energy function[12] can be written as

EHDA =A

2

l

j

t

lj(1 t )Xtl Xj +B

2

l

j

t

t (1 lj)

(1 ot)Xtl Xj +C

2

l

j

t

tlj(1 ot)Xtl Xj C

l

t

jt(1 ot)Xtl +C

2

l

t

2jt(1 ot) +Dn

2 D

l

t

(1 ot)Xtl

+D

2

l

j

t

t (1 ot)Xtl Xj +Fm

2 F

l

t

Xtl

+F

2

l

j

t

ljXtl X

j +

G

2

l

t

rjt(1 ot)Xtl . (7)

We also can extend the notation of the Lyapunov energy function[12] to twodimensions:

E= 12

l

j

t

W tjl Xtl X

j

l

t

Itl Xtl . (8)

Design of Data Association Filter Using Neural Networks 985

-F

-F

-F

-(C+D+F)-(A+F)

-(A+F)

-(A+F) -(A+F)

-(A+F)-(A+F)

-(A+F) -(A+F)

-(A+F)

-(B+D)

-(B+D) -(B+D)

Fig. 1. Example of Hopeld network for two targets and three plots

By comparing (8) and (7), we get the connection strength matrix and inputparameters each :{

W tjl =[A(1 t )+Ct (1ot)+F

]lj

[B(1 lj) + D

]t (1 ot),

Itl =(Cjt + D G2 rjt)(1 ot) + F.(9)

Here we omit the constant terms such as Dn+Fm2 +C2

2jt(1ot) . These

terms do not aect the neurons output since they just act as a bias terms duringthe processing.

Using the (9), the connection strength W tjl from the neuron at location (, l)to the neuron at location (t, j) is

W tjl =

[(C + D)(1 ot) + F ] if t = and j = l self feedback,(A + F ) if t = and j = l row connection,(B + D)(1 ot) if t = and j = l column connection,0 if t = and j = l global connection.

(10)

Fig.1 sketches the resulting two-dimensional network architecture as a directedgraph using the (10). We note that only 39 connections of possible 81 connectionsare achieved in this 3 3 neurons example. This means that modied Hopeldnetwork can be represented as a sparse matrix. In Fig.1, we also note that thereare no connections between diagonal neurons.

With the specic values from (9), the equation of motion for the MTT becomes

dStldt

= Stl

so

j

[{A(1 t ) + Ct (1 ot) + F}lj

{B(1 lj) + D}t (1 ot)]Xj

+(Cjt + D rjt2 G)(1 ot) + F. (11)

986 Y.W. Lee and C.W. Lee

The nal equation of data association is

dStldt

= Stl

So A

n

=1, =tXl B(1 ot)

n

j=1,j =lXtj D(1 ot)(

m

j=1

Xtj 1)

F (m

=1

Xl 1) C(1 ot)(1 jt) rjt2

(1 ot)G. (12)

The parameters A,B,C,D, F and G can be adjusted to control the emphasison dierent constraints and properties. A larger emphasis on A,B, and F willproduce the only one neurons activation both column and row. A large value ofC will produce Xtl close to jt except the duplicated activation of neurons in thesame row and column. A larger emphasis on G will make the neuron activatedepending on the value of targets course weighted value. Finally, a balancedcombination of all six parameters will lead to the most desirable association. Inthis case, a large number of targets and measurements will only require a largerarray of interconnected neurons instead of an increased load on any sequentialsoftware to compute the association probabilities.

3.3 Computational Complexity

The computational complexity of the modied Hopeld data association(MHDA) scheme, when applied to the target association problem, depends onthe number of tracks and measurements and the iteration numbers to be reachedstable state. Suppose that there are n tracks and m measurements. Then accord-ing to the (12) for data association, the computational complexity per iteration ofthe MHDA method require O(nm) computations. When we assume the averageiteration number as k, the total data association calculations require O(knm)computations. Therefore, even if the tracks and measurements are increased ,the required computations are not increasing exponentially . However JPDAF asestimated in [5] requires the computational complexity O(2nm) , so its compu-tational complexity increases exponentially depending on the number of tracksand measurements.

4 Simulation Results

4.1 Data Association Results

To exactly test the data association capability of the MHDA method, predenedtargets and measurements value are used to exclude any eects due to missdetection that are moderately occurring in real environment. An example ofthree targets and seven measurements is depicted in Fig. 2. In Fig. 2, large circlesrepresent track gates and symbol * means plots of measurements and small circleson the some measurements plots represent the plots of measurements whichare associated with tracks by MHDA. During the iteration, Fig. 3 and 4 showhow the distance and matching energy change respectively. In this example, the

Design of Data Association Filter Using Neural Networks 987

0 2 4 6 8 10 12 14 16 18 200

2

4

6

8

10

12

14

16

18

20

x axis [m]

y ax

is [m

]

m1m2

m3

m4

m5

m6

m7

1

2

3

Fig. 2. Diagram of Hopeld network for three targets and seven plots

0 50 100 150 200 250 3000

5

10

15

20

25

Iteration Number

Dis

tanc

e En

ergy

Fig. 3. Distance energy convergence for three targets and seven plots

0 50 100 150 200 250 3000

0.5

1

1.5

2

2.5

Iteration Number

Mat

chin

g En

ergy

Fig. 4. Matching energy of Hopeld network for three targets and seven plots

988 Y.W. Lee and C.W. Lee

0 5 10 15 20 25 30 35 400

0.2

0.4

0.6

Time (k=40)

Posi

tion

Erro

r[km]

0 5 10 15 20 25 30 35 400

0.02

0.04

0.06

0.08

Time (k=40)

Velo

city

Erro

r[km/

sec]

Fig. 5. RMS errors in X axis for target 8 : MHDA HNPDA

0 5 10 15 20 25 30 35 400.02

0.04

0.06

0.08

0.1

0.12

Time (k=40)

Posi

tion

Erro

r[km]

0 5 10 15 20 25 30 35 400

0.02

0.04

0.06

Time (k=40)

Velo

city

Erro

r[km/

sec]

Fig. 6. RMS errors in X axis for target 9 : MHDA HNPDA

association pairs are track 1 and measurement 4, track 2 measurement 2 ,andtrack 3 and measurement 6. Note that the results of data association is correctwith respect to nearest neighbor. In the simulation, the constants A = 50, B =50, C = 100, D = 1000, F = 1000and G = 100 appeared to be suitable for thisscenario. So was selected to be 1 s.

4.2 Sequential Tracking Results

The crossing, parallel and maneuvering targets whose initial parameters aretaken from target 1,2,3,4,8 and 9 respectively in Table 1 in [11] are tested. InFig. 5 and 6, the rms estimation errors for the maneuvering targets are shown.HNPDA can not track the dog leg maneuvering targets but the constant accel-eration target. Table 1 summarizes the rms position and velocity errors for eachtarget. The rms errors of the HNPDA about maneuvering targets have not beenincluded since it loses track of one of targets. The performance of the MHDA issuperior to that of HNPDA in terms of tracking accuracy and track maintenance.

Design of Data Association Filter Using Neural Networks 989

Table 1. RMS Errors in case of ten targets

Position error Velocity error Track maintenanceTarget (km) (km/s) (%)

i HNPDA MHDA HNPDA MHDA HNPDA MHDA1 0.048 0.044 0.024 0.021 95 982 0.051 0.048 0.028 0.018 95 983 0.065 0.044 0.021 0.018 85 984 0.049 0.041 0.020 0.018 93 985 0.041 0.044 0.018 0.018 100 1006 0.042 0.043 0.021 0.018 100 1007 0.040 0.040 0.018 0.018 100 1008 - 0.295 - 0.118 0 539 0.058 0.047 0.027 0.022 100 10010 0.037 0.039 0.011 0.012 100 100

5 Conclusions

In this chapter, we have developed the MHDA scheme for data association. Thisscheme is important in providing a computationally feasible alternative to com-plete enumeration of JPDA which is intractable. We have proved that given anarticial measurement and tracks conguration, MHDA scheme converges to aproper plot in a nite number of iterations. Also, a proper plot which is not theglobal solution can be corrected by re-initializing one or more times. In this light,even if the performance is enhanced by using the MHDA, we also note that thediculty in tuning the parameters of the MHDA is critical aspect of this scheme.The diculty can, however, be overcome by developing suitable automatic in-struments that will iteratively verify convergence as the network parameters vary.

Acknowledgements

This research has been supported by research funding of Center for High-QualityElectric Components and Systems, Chonnam National University, Korea.

References

1. Alspach, D. L.: A Gaussian Sum Approach to the Multi-Target IdenticationTracking Problem. Automatica , 11(1975) 285-296

2. Bar-Shalom, Y.: Extension of the Probabilistic Data Association Filter in Multi-Target Tracking. Proceedings of the 5th Symposium on Nonlinear Estimation,Sep.(1974) 16-21

3. Reid, D. B.: An Algorithm for Tracking Multiple Targets. IEEE Trans. on Automat.Contr., 24(1979) 843-854

4. Reid, D. B.: A Multiple Hypothesis Filter for Tracking Multiple Targets in a Clut-tered Environment. Lockheed Palo Alto Research Laboratory Tech. Report LMSC-D560254, Sept. (1977). (J. Basic Eng.,) 82(1960) 34-45

990 Y.W. Lee and C.W. Lee

5. Sengupta,D., Iltis, R. A.: Neural Solution to the Multitarget Tracking Data Asso-ciation Problem. IEEE Trans. on AES, AES-25, Jan.(1999) 96-108

6. Kuczewski, R.: Neural Network Approaches to Multitarget Tracking. In proceed-ings of the IEEE ICNN conference, (1987)

7. Fortmann, T. E., Bar-Shalom, Y., Schee, M.: Sonar Tracking of Multiple TargetsUsing Joint Probabilistic Data Association. IEEE J. Oceanic Engineering, OE-8,Jul. (1983) 173-184

8. Zhou, B.: Multitarget Tracking in Clutter : Algorithms for Data Association andState Estimation. PhD thesis, Pennsylvania State University, Department of Elec-trical and Computer Engineering, University Park, PA 16802, May (1992)

9. Hopeld, J. J., Tank, D. W.: Neural Computation of Decisions in OptimizationProblems. Biological Cybernatics,(1985) 141-152

10. Fortmann, T. E., Bar-Shalom, Y.: Tracking and Data Association. Orland AcdemicPress, Inc., (1988) 224

11. Yang Weon Lee: Adaptive Data Association for Multi-target Tracking Using Re-laxation. LNCS 35644, (2005) 552-561

12. Yang Weon Lee, Seo, J. H., Lee, J. G.: A Study on the TWS Tracking Filter forMulti-Target Tracking. Journal of KIEE, 41(4), (2004) 411-421

IntroductionReview of the Energy Function in the HNPDA and CommentsModified Scheme of HNPDAModification of Energy EquationTransformation of Energy Function into Hopfield NetworkComputational Complexity

Simulation ResultsData Association ResultsSequential Tracking Results

Conclusions

/ColorImageDict > /JPEG2000ColorACSImageDict > /JPEG2000ColorImageDict > /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 150 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 600 /GrayImageDepth 8 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.01667 /EncodeGrayImages true /GrayImageFilter /FlateEncode /AutoFilterGrayImages false /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict > /GrayImageDict > /JPEG2000GrayACSImageDict > /JPEG2000GrayImageDict > /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 2.00000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict > /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile (None) /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName (http://www.color.org) /PDFXTrapped /False

/SyntheticBoldness 1.000000 /Description >>> setdistillerparams> setpagedevice