EURASIP Journal on Applied Signal Processing 2004:15, 22672277c 2004 Hindawi Publishing Corporation

Resampling Algorithms for Particle Filters:A Computational Complexity Perspective

Miodrag BolicDepartment of Electrical and Computer Engineering, State University of New York at Stony Brook,Stony Brook, NY 11794-2350, USAEmail: mbolic@ece.sunysb.edu

Petar M. DjuricDepartment of Electrical and Computer Engineering, State University of New York at Stony Brook,Stony Brook, NY 11794-2350, USAEmail: djuric@ece.sunysb.edu

Sangjin HongDepartment of Electrical and Computer Engineering, State University of New York at Stony Brook,Stony Brook, NY 11794-2350, USAEmail: snjhong@ece.sunysb.edu

Received 30 April 2003; Revised 28 January 2004

Newly developed resampling algorithms for particle filters suitable for real-time implementation are described and their analysisis presented. The new algorithms reduce the complexity of both hardware and DSP realization through addressing common issuessuch as decreasing the number of operations and memory access. Moreover, the algorithms allow for use of higher samplingfrequencies by overlapping in time the resampling step with the other particle filtering steps. Since resampling is not dependenton any particular application, the analysis is appropriate for all types of particle filters that use resampling. The performance ofthe algorithms is evaluated on particle filters applied to bearings-only tracking and joint detection and estimation in wirelesscommunications. We have demonstrated that the proposed algorithms reduce the complexity without performance degradation.

Keywords and phrases: particle filters, resampling, computational complexity, sequential implementation.

1. INTRODUCTION

Particle filters (PFs) are very suitable for nonlinear and/ornon-Gaussian applications. In their operation, the mainprinciple is recursive generation of random measures, whichapproximate the distributions of the unknowns. The ran-dom measures are composed of particles (samples) drawnfrom relevant distributions and of importance weights ofthe particles. These randommeasures allow for computationof all sorts of estimates of the unknowns, including mini-mummean square error (MMSE) andmaximum a posteriori(MAP) estimates. As new observations become available, theparticles and the weights are propagated by exploiting Bayestheorem and the concept of sequential importance sampling[1, 2].

The main goals of this paper are the development of re-samplingmethods that allow for increased speeds of PFs, thatrequire less memory, that achieve fixed timings regardless ofthe statistics of the particles, and that are computationallyless complex. Development of such algorithms is extremely

critical for practical implementations. The performance ofthe algorithms is analyzed when they are executed on a dig-ital signal processor (DSP) and specially designed hardware.Note that resampling is the only PF step that does not de-pend on the application or the state-space model. Therefore,the analysis and the algorithms for resampling are general.

From an algorithmic standpoint, the main challenges in-clude development of algorithms for resampling that aresuitable for applications requiring temporal concurrency.1

A possibility of overlapping PF operations is considered be-cause it directly aects hardware performance, that is, it in-creases speed and reduces memory access. We investigate se-quential resampling algorithms and analyze their computa-tional complexity metrics including the number of opera-tions as well as the class and type of operation by perform-ing behavioral profiling [3]. We do not consider fixed point

1Temporal concurrency quantifies the expected number of operationsthat are simultaneously executed, that is, are overlapped in time.

2268 EURASIP Journal on Applied Signal Processing

precision issues, where a hardware solution of resamplingsuitable for fixed precision implementation has already beenpresented [4].

The analysis in this paper is related to the sampleimportance resampling (SIR) type of PFs. However, the anal-ysis can be easily extended to any PF that performs resam-pling, for instance, the auxiliary SIR (ASIR) filter. First, inSection 2 we provide a brief review of the resampling opera-tion.We then consider random and deterministic resamplingalgorithms as well as their combinations. The main featureof the random resampling algorithm, referred to as residual-systematic resampling (RSR) and described in Section 3, isto perform resampling in fixed time that does not depend onthe number of particles at the output of the resampling pro-cedure. The deterministic algorithms, discussed in Section 4,are threshold-based algorithms, where particles with mod-erate weights are not resampled. Thereby significant savingscan be achieved in computations and in the number of timesthe memories are accessed. We show two characteristic typesof deterministic algorithms: a low-complexity algorithm andan algorithm that allows for overlapping of the resamplingoperation with the particle generation and weight computa-tion. The performance and complexity analysis are presentedin Section 5 and the summary of our contributions is out-lined in Section 6.

2. OVERVIEWOF RESAMPLING IN PFs

PFs are used for tracking states of dynamic state-space mod-els described by the set of equations

xn = f(xn1

)+ un,

yn = g(xn)+ vn,

(1)

where xn is an evolving state vector of interest, yn is a vectorof observations, un and vn are independent noise vectors withknown distributions, and f () and g() are known functions.The most common objective is to estimate xn as it evolves intime.

PFs accomplish tracking of xn by updating a randommeasure {x(m)1:n ,w(m)n }Mm=1,2 which is composed of M parti-cles x(m)n and their weights w

(m)n defined at time instant n,

recursively in time [5, 6, 7]. The random measure approxi-mates the posterior density of the unknown trajectory x1:n,p(x1:n|y1:n), where y1:n is the set of observations.

In the implementation of PFs, there are three importantoperations: particle generation, weight computation, and re-sampling. Resampling is a critical operation in particle filter-ing because with time, a small number of weights dominatethe remaining weights, thereby leading to poor approxima-tion of the posterior density and consequently to inferior es-timates. With resampling, the particles with large weights arereplicated and the ones with negligible weights are removed.

2The notation x1:n signifies x1:n = {x1, x2, . . . , xn}.

After resampling, the future particles are more concentratedin domains of higher posterior probability, which entails im-proved estimates.

The PF operations are performed according to

(1) generation of particles (samples) x(m)n (xn|x(imn1)

n1 ,y1:n), where (xn|x(i

mn1)

n1 , y1:n) is an importance densityand i(m)n is an array of indexes, which shows that the

particlem should be reallocated to the position i(m)n ;(2) computation of weights by

w(m)n =w(imn1)n1

a(imn1)n1

p(ynx(m)n )p(x(m)n x(imn1)n1 )

(x(m)n

x(imn1)n1 , y1:n) (2)

followed by normalization w(m)n = w(m)n /Mj=1w( j)n ;(3) resampling i(m)n a(m)n , where a(m)n is a suitable resam-

pling function whose support is defined by the particle

x(m)n [8].

The above representation of the PF algorithm provides acertain level of generality. For example, the SIR filter with

a stratified resampling is implemented by choosing a(m)n =w(m)n for m = 1, . . . ,M. When a(m)n = 1/M, there is no re-sampling and i(m)n = m. The ASIR filter can be implementedby setting a(m)n = w(m)n p(yn+1|(m)n+1) and (xn) = p(xn|x(m)n1),where (m)n is the mean, the mode, or some other likely value

associated with the density p(xn|x(m)n1).

3. RESIDUAL SYSTEMATIC RESAMPLINGALGORITHM

In this section, we consider stratified random resampling al-

gorithms, where a(m)n = w(m)n [9, 10, 11]. Standard algorithmsused for random resampling are dierent variants of strati-fied sampling such as residual resampling (RR) [12], branch-ing corrections, [13] and systematic resampling (SR) [6]. SRis the most commonly used since it is the fastest resamplingalgorithm for computer simulations.

We propose a new resampling algorithm which is basedon stratified resampling, and we refer to it as RSR [14]. Sim-ilar to RR, RSR calculates the number of times each particleis replicated except that it avoids the second iteration of RRwhen residual particles need to be resampled. Recall that inRR, the number of replications of a specific particle is deter-mined in the first loop by truncating the product of the num-ber of particles and the particle weight. In RSR instead, theupdated uniform random number is formed in a dierentfashion, which allows for only one iteration loop and pro-cessing time that is independent of the distribution of theweights at the input. The RSR algorithm for N input and Moutput (resampled) particles is summarized by Algorithm 1.

Figure 1 graphically illustrates the SR and RSR methodsfor the case of N = M = 5 particles with weights givenin Table 1. SR calculates the cumulative sum of the weights

C(m) = mi=1w(i)n and compares C(m) with the updated uni-form number U (m) for m = 1, . . . ,N . The uniform number

Computational Complexity of Resampling in Particle Filters 2269

Purpose: generation of an array of indexes {i}N1 at timeinstant n, n > 0.

Input: an array of weights {wn}N1 , input and outputnumber of particles, N andM, respectively.

Method:(i) = RSR(N ,M,w)Generate a random number U (0) U[0, 1/M]for m = 1N

i(m) = (w(m)n U (m1)) M + 1U (m) = U (m1) + i(m)/M w(m)n

end

Algorithm 1: Residual systematic resampling algorithm.

U (0) is generated by drawing from the uniform distributionU[0, 1/M] and updated by U (m) = U (m1) + 1/M. The num-ber of replications for particle m is determined as the num-ber of times the updated uniform number is in the range[C(m1),C(m)). For particle 1, U (0) and U (1) belong to therange [0,C(1)), so that this particle is replicated twice, whichis shown with two arrows that correspond to particle 1. Par-ticles 2 and 3 are replicated once. Particle 4 is discarded(i(4) = 0) because no U (m) for m = 1, . . . ,N appears in therange [C(3),C(4)).

The RSR algorithm draws the uniform random numberU (0) = U (0) in the same way but updates it by U (m) =U (m1) + i(m)/M w(m)n . In the figure, we display bothU (m) = U (m1)+i(m)/M andU (m) = U (m)w(m)n . Here, theuniform number is updated with reference to the origin of thecurrently considered weight, while in SR, it is propagated withreference to the origin of the coordinate system. The dierenceU (m) between the updated uniform number and the cur-rent weight is propagated. Figure 1 shows that i(1) = 2 andthat U (1) is calculated and then used as the initial uniformrandom number for particle 2. Particle 4 is discarded becauseU (3) = U (4) > w(4), so that (w(4)n U (3)) M = 1 andi(4) = 0. If we compare U (1) with the relative position of theU (2) and C(1) in SR, U (2) in RSR with the relative positionof U (3) and C(2) in SR, and so on, we see that they are equal.Therefore, SR and RSR produce identical resampling result.

3.1. Particle allocation andmemory usage

We call particle allocation the way in which particles areplaced in their new memory locations as a result of resam-pling. With proper allocation, we want to reduce the numberof memory accesses and the size of state memory. The allo-cation is performed through index addressing, and its execu-tion can be overlapped in time with the particle generationstep. In Figure 2, three dierent outputs of resampling forthe input weights from Figure 1 are considered. In Figure 2a,the indexes represent positions of the replicated particles. Forexample, i(2) = 1 means that particle 1 replaces particle 2.Particle allocation is easily overlapped with particle genera-tion using x(m) = x(i(m)) for m = 1, . . . ,M, where {x(m)}Mm=1is the set of resampled particles. The randomness of the re-sampling output makes it dicult to realize in-place storageso that additional temporary memory for storing resampled

particles x(m) is necessary. In Figure 2a, particle 1 is replicatedtwice and occupies the locations of particles 1 and 2. Particle2 is replicated once andmust be stored in the memory of x(m)

or it would be rewritten. We refer to this method as particleallocation with index addressing.

In Figure 2b, the indexes represent the number of timeseach particle is replicated. For example, i(1) = 2 means thatparticle 1 is replicated twice. We refer to this method as par-ticle allocation with replication factors. This method still re-quires additional memory for particles and memory for stor-ing indexes.

The additional memory for storing the particles x(m) isnot necessary if the particles are replicated to the positionsof the discarded particles. We call this method particle alloca-tion with arranged indexes of positions and replication factors(Figure 2c). Here, the addresses of both replicated particlesand discarded particles as well as the number of times theyare replicated (replication factor) are stored. The indexes arearranged in a way so that the replicated particles are placedin the upper and the discarded particles in the lower part ofthe index memory. In Figure 2c, the replicated particles takethe addresses 1 4 and the discarded particle is on the ad-dress 5. When one knows in advance the addresses of the dis-carded particles, there is no need for additional memory forstoring the resampled particles x(m) because the new particlesare placed on the addresses occupied by the particles that arediscarded. It is useful for PFs applied to multidimensionalmodels since it avoids need for excessive memory for storingtemporary particles.

For the RSR method, it is natural to use particle alloca-tion with replication factor and arranged indexes because theRSR produces replication factors. In the particle generationstep, the for loop with the number of iterations that corre-sponds to the replication factors is used for each replicatedparticle. The dierence between the SR and the RSR meth-ods is in the way the inner loop in the resampling step for SRand particle generation step for RSR is performed. Since thenumber of replicated particles is random, the while loop inSR has an unspecified number of operations. To allow for anunspecified number of iterations, complicated control struc-tures in hardware are needed [15]. The main advantage ofour approach is that the while loop of SR is replaced with afor loop with known number of iterations.

4. DETERMINISTIC RESAMPLING

4.1. Overview

In the literature, threshold-based resampling algorithms arebased on the combination of RR and rejection control andthey result in nondeterministic timing and increased com-plexity [8, 16]. Here, we develop threshold-based algorithmswhose purpose is to reduce complexity and processing time.We refer to these methods as partial resampling (PR) becauseonly a part of the particles is resampled.

In PR, the particles are grouped in two separate classes:one composed of particles with moderate weights and an-other with dominating and negligible weights. The particles

2270 EURASIP Journal on Applied Signal Processing

1 2 3 4 5

Particle

U(0)

U(1)

U(2)

U(3)

U(4)

1

C(1)

C(2)C(3)

C(4)C(5) = 1C

(a)

1 2 3 4 5

Particle

U(0)

U(1)

U(2)

U(3) U(4)

U(5)

U(1)

U(2)

U(3)U(4)

w(1)

w(2)

w(3)

w(4)

w(5)

C

(b)

Figure 1: (a) Systematic and (b) residual systematic resampling for an example withM = 5 particles.

Table 1: Weights of particles.

m w(m) i(m)

1 7/20 2

2 6/20 1

3 2/20 1

4 2/20 0

5 3/20 1

with moderate weights are not resampled, whereas the negli-gible and dominating particles are resampled. It is clear thaton average, resampling would be performed much faster be-cause the particles with moderate weights are not resampled.We propose several PR algorithms which dier in the resam-pling function.

4.2. Partial resampling: suboptimal algorithms

PR could be seen as a way of a partial correction of the vari-ance of the weights at each time instant. PR methods con-sist of two steps: one in which the particles are classified asmoderate, negligible, or dominating and the other in whichone determines the number of times each particle is repli-cated. In the first step of PR, the weight of each particle iscompared with a high and a low threshold, Th and Tl, re-spectively, where Th > 1/M and 0 < Tl < Th. Let the num-ber of particles with weights greater than Th and less than Tlbe denoted by Nh and Nl, respectively. A sum of the weightsof resampled particles is computed as a sum of dominat-ing Wh =

Nhm=1w

(m)n for w

(m)n > Th and negligible weights

Wl =Nl

m=1w(m)n for w

(m)n < Tl. We define three dierent

types of resampling with distinct resampling functions a(m)n .

1

1

2

3

5

1

2

3

4

5

(a)

2

1

1

0

1

1

2

3

4

5

(b)

1

2

3

5

4

1

2

3

4

5

2

1

1

1

(c)

Figure 2: Types of memory usages: (a) indexes are positions of thereplicated particles, (b) indexes are replication factors, and (c) in-dexes are arranged as positions and replication factors.

The resampling function of the first PR algorithm (PR1)is shown in Figure 3a and it corresponds to the stratified re-sampling case. The number of particles at the input and atthe output of the resampling procedure is the same and equalto Nh +Nl. The resampling function is given by

a(m)n =w(m)n for w

(m)n > Th or w

(m)n < Tl,

1Wh WlM Nh Nl otherwise.

(3)

The second step can be performed using any resampling al-gorithm. For example, the RSR algorithm can be called using

(i) = RSR(Nh+Nl,Nh+Nl,w(m)n /(Wh+Wl)), where the RSR isperformed on the Nh +Nl particles with negligible and dom-inating weights. The weights have to be normalized beforethey are processed by the RSR method.

The second PR algorithm (PR2) is shown in Figure 3b.The assumption that is made here is that most of thenegligible particles will be discarded after resampling, and

Computational Complexity of Resampling in Particle Filters 2271

0 Tl 1/M Th 1 w(i)

1Wh WlM Nh Nl

1

a(i)

(a)

0 Tl 1/M Th 1 w(i)

1Wh WlM Nh Nl

1

a(i)

(b)

0 Tl 1/M Th 1 w(i)

1/M

Nh +NlM

1

a(i)

(c)

Figure 3: Resampling functions for the PR algorithms (a) PR1, (b) PR2, and (c) PR3.

consequently, particles with negligible weights are not usedin the resampling procedure. Particles with dominatingweights replace those with negligible weights with certainty.The resampling function is given as

a(m)n =

w(m)n +WlNh

for w(m)n > Th,

1Wh WlM Nh Nl for Tl < w

(m)n < Th,

0 otherwise.

(4)

The number of times each particle is replicated can be foundusing (i) = RSR(Nh,Nh + Nl, (w(m)n + Wl/Nh)/(Wh + Wl)),where the weights satisfy the condition w(m)n > Th. There areonly Nh input particles and Nh +Nl particles are produced atthe output.

The third PR algorithm (PR3) is shown in Figure 3c. Theweights of all the particles above the threshold Th are scaledwith the same number. So, PR3 is a deterministic algorithm

whose resampling function is given as

a(m)n =

Nh +NlM

for w(m)n > Th,1M

for Tl < w(m)n < Th,

0 otherwise.

(5)

The number of replications of each dominating particle maybe less by one particle than necessary because of the round-ing operation. One way of resolving this problem is to assignthat the first Nt = Nl Nl/NhNh dominating particles arereplicated r = Nl/Nh + 2 times, while the rest of Nh Ntdominating particles are replicated r = Nl/Nh + 1 times.The weights are calculated as w(m) = w(m), where m rep-resents positions of particles with moderate weights, and asw(l) = w(m)/r +Wl/(Nh +Nl), wherem are positions of par-ticles with dominating weights and l of particles with bothdominating and negligible weights.

2272 EURASIP Journal on Applied Signal Processing

m

1

2

3

4

5

Sum

w(m)

7/10

6/10

2/10

2/10

3/10

2

Initial weights

T3 = 1

T2 = 1/2

T1 = 1/4

T0 = 0

P1,P2

P5

P3,P4

1/4 < W/M = 2/5 < 1/2 =

Classification

m

1

2

3

4

5

Sum

i(m)

2

2

0

0

1

5

w(m)

4.5/20, 4.5/20

4/20, 4/20

3/20

1

PR3 algorithm

Figure 4: OPR method combined with the PR3 method used for final computation of weights and replication factors.

Another way of performing PR is to use a set of thresh-olds. The idea is to perform initial classification of the parti-cles while the weights are computed and then to carry out theactual resampling together with the particle generation step.So, the resampling consists of two steps as in the PR2 algo-rithm, where classification of the particles is overlapped withthe weight computation. We refer to this method as over-lapped PR (OPR).

A problem with the classification of the particles is thenecessity of knowing the overall sum of nonnormalizedweights in advance. The problem can be resolved as follows.The particles are partitioned according to their weights. Thethresholds for group i are defined as Ti1, Ti for i = 1, . . . ,K ,where K is the number of groups, Ti1 < Ti and T0 = 0. Theselection of thresholds is problem dependent. The thresholdsthat define the moderate group of particles satisfy Tk1