F. Jelinek

Fast Sequential Decoding Algorithm Using a Stack*

Abstract: In this paper a new sequential decoding algorithm is introduced that uses stack storage at the receiver. It is much simplerto describe and analyze than the Fano algorithm, and is about six times faster than the latter at transmission rates equal to Reomp, therate below which the average number of decoding steps is bounded by a constant, Practical problems connected with implementingthe stack algorithm are discussed and a scheme is described that facilitates satisfactory performance even with limited stack storagecapacity. Preliminary simulation results estimating the decoding effort and the needed stack size are presented.

675

p

Figure 1 The graph of Roomp/e as a function of the cross-over probability p of a binary symmetric channel.

0.7

0.6

\..l 0.5

000

Figure 2 Example of a binary tree code of rate R 1;3.

Let us further assume that the information source gen-

n

II wo('1dt.).i=l

erates outputs 8 E 10, 1, ... , d I} that are to becommunicated to the user located at the other end of thechannel. An appropriate tree code of rate R 1/(llQ log, d)bits per channel used will have d branches leaving everynode, each branch being associated with a sequence of110 channel input digits. An example of a tree code ap-propriate for a binary source (d = 2) and a channelwith binary inputs (a = 2) is given in Fig. 2. Since no = 3,the code has rate R = 1/3. The correspondence betweensource outputs and channel inputs (i.e., the encodingrule) is as follows.

At the start of the encoding process the encoder is"located" at the root of the tree. If the first digit 81 gen-erated by the source is a 0, the encoder follows the upperbranch out of the root node to the next node and putsout (transmits through the channel) the sequence asso-ciated with that branch (in this case 000). If 81 = I, theencoder follows the lower branch to the next node andputs out the corresponding sequence (in this case 111).In general, just before the source generates the ith digitSi, the encoder is located at some node i 1 branchesdeep in the tree. If s, = 0 the encoder leaves this nodealong the top branch and, if s, = 1, it leaves the nodealong the bottom branch; in both cases it transmitsthe sequence corresponding to the branch along whichit traveled. In this way a message sequence Sh 82, ."traces a path through the tree and the tree sequencecorresponding to that path is then transmitted. (Thegeneralization to a d-nary tree is obvious: At time i theencoder leaves the current node along the (Si + l)thbranch of the fan-out and transmits the correspondingsequence, where s; E 10, 1, ... ,d - I}.) Thus in Fig. 2the message sequence 0011 determines the path indicatedby the thick line and causes the sequence 000101011010to be sent through the channel.

In principle, the encoding tree can be continued in-definitely and thus the total number r of levels it canhave (the tree displayed in Fig. 2 has four levels) isarbitrary. Since a d-nary tree with I' levels has dl' paths,there is a problem of how the encoder can store its treecode. We will not concern ourselves here with that issue.Let the reader be assured that no difficulty arises. Thetree is never actually stored; all that is stored is a very sim-ple algorithm that can generate the digits associated withthe tree branches whenever the former are required. Theusual tree codes are called conoolutianal and their descrip-tion can be found on pp. 377 to 383 of Ref. 4. It will beeasier for us to continue to act as if the encoder stored theentire tree.

3. The decoding algorithmFrom the preceding description of tree encoding itfollows that the natural transmission units we are dealingwith are not channel digits themselves, but sequences

I 1 o II 1 I 1

1 0 11 0 1

0 t 1 II 0 t 0

0 0 00 0 I

1 1 1 II 1 1 0

1 t 0t 0 {

0 1 o II 0 1 1

-

() 0 0I 0 1 I

I 1 1 10 1 0

1 0 10 0 0 I

I 0 1 ()1 { 1

0 0 tI 0 0 I

I I I 00 0 I

I 0 I0 0 I I

0 I 1

o

i!1

We next describe the outcome of simulations that enableus to compare the stack and Fane's algorithm (Fig. 6).In Section 6 we modify the algorithm to limit the growthof the stack, and show how effectively this can be done(Fig. 7). Section 7 contains a procedure that handlesstack overflows and thereby prevents catastrophic termi-nations. Finally, Section 8 describes how a stack is actu-ally implemented and in what manner it specifies therequired information.

2. Tree encodingLet us assume that we desire to communicate over adiscrete memoryless channel characterized by a transmissionprobability matrix [wo('rJ/OJ, where ~ E {O, 1, ... , a - I}are the channel inputs and '11 E {O, 1, . .. ,{3 - I} are thecorresponding channel outputs. Thus, for any integer n,the probability that an arbitrary output sequence 'rJ1>'rJ2, '" , 'l7n is received, given that an arbitrary inputsequence ~h ~2' , ~n was transmitted, is equal to

676

F. JELINEK IBM J. RES. DEVELOP.

If the sequence yf was received, we will be interestedin the likelihoods

of the various paths s' = (s", SH1> .,. , Sf), where thebranch likelihood function ",(Si) of the branch leadingfrom node 1 to node s i (note that a path uniquelydefines the tree node on which it terminates, and vice versa)is defined by

of no of these that correspond to the tree branches. Ac-cordingly, it will simplify further discussion if we hence-forth restrict our attention to the no-product channel[wry/x)] whose input symbols x and output symbols yare strings of no inputs and outputs of the underlyingchannel [wo('1J/~)]. From this point of view, the productchannel input alphabet corresponding to the tree codeof Fig. 2 is octal, and the message sequence 0011 causes0532 to be transmitted. Let x* represent some sequencen, ~~, , ~,,~ and let y* represent '1J~, '1)~, , '1),,~.Then

Thus the product channel has inputs x E {O, 1,' .. , a-IIand outputs y {a, 1, .,. , b - I}, where a = a.",b = {3", and a and {3 are the sizes of the input and outputalphabets of the underlying channel, respectively. Atree path of length i is specified by the vector Si 50' (S1>S2, .. , , s,) (we will use boldface for vectors and super-scripts will indicate their length) formed from the cor-responding message digits. We will denote by x/(s'), j S i,the transmitted symbol associated with the jth branchof the path a'. Thus in Fig. 2 xs(OlOs4) 7 for all S4,and x2(lOs3s4) 2 for all 83S4. Let us now assume thata sequence yf was received through the channel (I' isthe number of levels in the tree) and that we wish todecode this sequence, i.e., to determine the identity ofthe message sequence sr put out by the source. We willdenote by ST the receiver's estimate of s r and, of course,we aim at having sr equal to Sf. We recall that, whensr was inserted into the encoder, the latter producedthe channel input sequence Xr(ST) = Xl(ST), X2(Sf), .. ,Xf(Sf) in the way described in the preceding section.Our problem is to specify the operation of the decoder.

Let r(x), x E to, 1, ... ,a - 1\ be a suitable probabilitydistribution (its choice will be clarified below) over theinput symbols of the product channel, and define theoutput distribution

677

From the decoder's point of view, L(Sf) and A,(S')are functions of the paths only, since the received se-quence yT is fixed throughout the decoding processand the branch symbols x.(s') are determined by thetree code that is known in advance.

We are now ready to describe the decoding algorithmfor a binary tree (d = 2). This restriction will make theexplanation easier to follow, but will be subsequentlyremoved.

(1) Compute L(O) = MO) and L(l) = Ai(l), the likeli-hoods of the two branches' leaving the root node (seeFig. 2), and place them into the decoder's memory.(2) If L(O) ~ L(1), eliminate L(O) from the decoder'smemory and compute L(OO) = (0) + ).2(00) and L(Ol) =L(O) + A2(OI). Otherwise, eliminate L(l) and computeL(10) = L(I) ~(10) and L(11) = L(I) + Mll).Therefore we end with the likelihoods of three pathsin the decoder's memory, two paths of length 2 and oneoflength 1.(3) Arrange the likelihoods of the three paths in de-creasing order. Take the path corresponding to the top-most likelihood, compute the likelihoods of its twopossible one-branch extensions, and eliminate frommemory the likelihood of the just extended path [e.g.,if (0), L(IO), L(11) are in the memory and, say, L(10) ~L(O) ~ L(11), then L(10) is replaced in the memory bythe newly computed values of (100) L(IO) + ).3(100)and L(101) L(IO) + A3(101). In case, say, L(O) ~(11) ~ L(10), then L(O) is replaced by L(OO) and L(Ol)].At the end of this step the decoder's memory will containthe likelihoods of four paths, and either two of thesewill be of length 3 and one each of lengths 1 and 2, orall four paths will be of length 2.(4) The search pattern is now clear. After the kth step,the decoder's memory will contain exactly k + 1 likeli-hoods corresponding to paths of various lengths anddifferent end nodes. The (k + l)th step will consist offinding the largest likelihood in the memory, determiningthe path to which it corresponds, and replacing thatlikelihood with the likelihoods corresponding to thetwo one-branch extensions of that path.(5) The decoding process terminates when the path tobe extended next is of length I', i.e., leads from the rootnode to the last level of the tree.

Because of the ordered nature of the decoder's memory,we refer to it as a stack. We next illustrate the stackdecoding algorithm by an example. Consider the binarytree of Fig. 3 with nodes as numbered. Let paths beassociated with their terminal nodes. Let the numberswritten on top of the branches represent the values ofthe corresponding branch likelihood function for somereceived sequence y3. Thus, the likelihood of path 8is equal to -6 + 1 + 1 = -4. The state of the stack

(1)

(3)

(4)

(2)

IT wo('1)V~~).i-l

w(y*/x*)

L(si) == A.(s')' ... 1

Wr(y) 50' L w(y I x)r(x).

~ J i) = I w[y, I x.(s')] _ RI\,\S - 082 (y.) no .

Wr ,

NOVEMBER 1969 FAST SEQUENTIAL DECODING

-6

4. Probabilty of error and average numberof decoding steps

We now wish to compute upper bounds to the probabilityof decoding error and to the average number of decodingsteps. Much of the argument is identical to that whichapplies to the Fano algorithm':" and we will thereforelimit ourselves to the differences between the two analysesand to a statement of results. A complete treatmentcan be found in our earlier IBM research report. 6

It will be convenient to partition the nodes of thetree into sets ;[);, i = 0, 1, "', r, defined relative to thepath sr actually taken by the encoder.

Definition 1: The incorrect subset ;[). consists of theend node of the initial segment Si of the true path sr =(s\ 8'+1> , sr), of the d-l terminal nodes of the in-correct branches S!+1 ~ Si+1 stemming from the endnode of s', and of all nodes lying on paths leaving thesed-l nodes.

It remains to generalize our algorithm to d-nary trees.The notion of a stack will facilitate a concise statementof the procedure.

(1) At the beginning of the decoding process, the stackcontains only the root node of the tree (i.e., the emptypath) with its likelihood set arbitrarily to zero.(2) A decoding operation consists of ascertaining thepath Si that corresponds to the likelihood L, at the topof the stack (i.e., the largest of the likelihoods in thestack), eliminating L, from the stack, computing thelikelihoods h~+ll ... , A~+l of the branches that leavethe end node of the path s ', and inserting the newpath likelihoods L;+l = L; + A;+1> i 1, 2, . , d,into their proper positions according to size (clearly thestack contains path identifications and their likelihoodvalues).(3) The search ends when the decoder finds at the topof the stack a path whose length is r. That path is thenconsidered to have been taken by the encoder.

In our algorithm, the likelihoods are used as a distancemeasure between the received sequence and the codeword sequences on the various paths of the tree. Theheuristic reasons for this choice of the measure weremade clear in an IBM research report by the author" inwhich the stack algorithm was developed as an intuitivelynatural way of taking advantage of the tree structureof the code. This aspect makes the algorithm very at-tractive from a pedagogical point of view. The Fanomethod" can then be considered to be a particular imple-mentation of the stack search in which the decoder'smemory is eliminated for the price of an increase in thenumber of steps necessary for decoding (see Fig. 6). Wewill see in the next section that the stack algorithm lendsitself to a relatively simple analysis.

o

Figure 3 An example of likelihood-value assignment to treebranches induced by a code and a received sequence.

Figure 4 The 1>, classification of nodes of a binary tree rela-tive to the transmittedmessage (81) 82, $2,8..... ).

during decoding would then be as follows (topmostpath on the left).1st state: 02nd state: 2, 13rd state: 5, 6, 14th state: 6, 1, 11, 125th state: 1, 11, 12, 13, 146th state: 3, 11, 12, 13, 14, 47th state: 8, 11, 12, 13, 14, 7, 4.

When trying to perform the eighth step, the decoderwould find path 8 on top of the stack and, since the latter

678 has length 3 I'), decoding would terminate.

F. JELINEK IBM J. RES. DEVELOP.

(5)

A binary tree illustration of the sets ;:[), is given inPig. 4. Let N, be the number of nodes belonging to ;.ojthat the decoder "visits." (These are end nodes of pathsthat have at one time appeared on top of the stack andwere thus extended during the decoding process.) Thensince V:-~o ;:[)i is the set of all nodes in the tree, L:i=o N,is equal to the total number of decoding steps and wewill want to estimate it.

Obviously, a path ; ;:[);(i < j :::; I') will not beextended in the decoding process unless its likelihoodL(st) is such that the path ; appears on top of the stack.But then L(s;) must exceed the minimum of the likeli-hoods {L(Si+I), '" , L(Sf)} along the true path (thiscondition is necessary but not sufficient). Let

be the rate below which the average number of decodingsteps E[N.] is bounded by a constant. Then it can beshown that, for rates R ::::; Room-", there exist codes suchthat

Figure (i Average number of decoding steps necessary todecode a tree branch using the stack algorithm (solid line)and the Fano algorithm (interrupted line) as a function ofthe R/R eo m p ratio. Coding rate is held constant at R =in, but the crossover probabilities of the binary symmetricchannel vary as indicated. Stack algorithm results are basedon 1000 runs of binary information blocks of length 1000.

where K(R) is an increasing finite function of R. Sinceit can be shown that, for R > Room", E[N.] is an expo-nentially increasing function of I' (if we assume a goodcode), then for large I' [which is needed to keep therate loss factor tier + t) small] one would ordinarilynot attempt to use sequential decoding at rates thatexceed Roomp (The ratio of Roomp to channel capacity Cis plotted in Fig. 1 for binary symmetric channels.)

We close this section by remarking that the argumentthat leads to bounds (6) and (9) for the Fano algorithmis a very elaborate one. It is not an exaggeration to statethat the vast majority of students of graduate level in-formation theory courses always remain uncomfortablewith it.

5. Experimental evaluation and comparisonWe have not yet fully evaluated the performance ofthe stack algorithm, but we can present some resultsof computer simulation and compare these with thepublished performance of the Fano sequential decodingalgorithm.

We have run simulations of the performance of a rateR = 1/2 code when used over a binary symmetricchannelwhose crossover probability p is allowed to vary. Inour experiment the block length I' = 1000 and thelength of the last-level branches was t = 26 symbols.The results of Fig. 6 are based on 1000 such blocks runfor each of the different values of p. We have run thedecoding as described in Section 3 with two branchesleaving each node. The solid line in Fig. 6 indicates theaverage number of stack algorithm decoding steps nec-essary to decode one tree branch. The interrupted lineplots the same quantity for the Fano decoding algorithm(the data is taken from Fig. 6.50 of the text by Wozen-craft and Jacobs"). It is seen that at Reo mp the stack algo-rithm has a better than sixfoldadvantage.

It must be stressed that Fig. 6 does not presentthe entire comparison and is actually unfavorable to thestack decoding algorithm: When decoding in real timewith a fixed maximum delay imposed on the releaseof decoded information to the user (relative to the timeof reception), it is the peak demands on computationthat should be compared. Since the advantage of thestack algorithm grows substantially as the rate approachesRoomp, it is expected that this advantage will be even morepronounced during periods of high channel noise andconsequent high computational demand.

Because of the above considerations the decoder speedfactor (maximum number of decoding steps performablein the time it takes to receive one branch) must exceedsubstantially the averages plotted in Fig. 6. This is es-pecially true for the Fano decoder that will be idle duringthe low noise time intervals when it is "caught up" withthe received signal. (In such situations it can performonly one step per received branch interval, regardlessof what its speed factor isl) The stack decoder neednever be idle. If the length of the path on the top of thestack is equal to the length of the received sequence,the decoder may profitably extend the highest situated pathof those paths in the stack that are shorter than the topone. This apparently premature work costs nothingand will not have to be done later should the channelnoise increase.

The complete mutual independence of the stack order-ing and path extending portions of the algorithm is alsoworthy of note. These two functions can be performedin parallel by different (but communicating) machines.The stack need not even be in order-s-only the top pathmust be available for extension. To achieve further speed-

(12)

(11)tRoomp =0 - max Eo(l, r)no r

680

F. JELIt'lEK IBM J. RES. DEVELOP.

681

1.00.9

16 branchesper node ,../Y'"

5)"-"-..-"-

0..-'..-""....-

____---

682

F. JELINEK

branch (not bit!) had virtually the same value for bothversions of the stack algorithm at all values of the cross-over probability p.

7. Handling of stack overflowsObviously, any decoder, no matter how expensive, willhave only a limited stack available for storing the likeli-hoods. When all of its locations are filled, the one con-taining the smallest likelihood must be purged to makeroom for a newly inserted path. The worry is that thepurged entry may correspond to the true path, in whichcase a decoding error will necessarily result unless otherprecautions are taken. One possibility, described in somedetail below, is to switch temporarily into a Fano decoding"mode. In this way catastrophy can be avoided when stackoverflow takes place and even relatively short stackscapable of storing only several hundred paths can beused to speed up the decoding process. To keep thefollowing discussion brief, it will be assumed that thereader is thoroughly familiar with some version of theFano decoding algorithm (e.g., the one presented inSection lOA of Ref. 4).

Let T be the integral part of the largest path likelihoodthat was ever purged from the stack. Obviously, T is amonotonically nondecreasing function of time whichmight as well be defined as being equal to - CD until the firstpath is purged from the stack. Let us never insert a pathinto the stack whose likelihood is less than the currentvalue of T. Stack overflow will be said to take place whenthe decoder finds the stack empty and is therefore unableto continue to carry out the algorithm. Let s i be the lastpath extended before stack overflow took place. Thenobviously L(Si);::: Tand L(s\ Si+l) < Tfor all branchesSi+l leaving the node s'. First, the decoder backs up tothe nearest preceding node s;U::; i, s' = (s', Si+l, ... ,Si)]whose immediate predecessor node S;-1 has likelihoodL(S;-I) < T. Next, the decoder is switched into the Fanoforward mode and its cumulative threshold To is set equaltoT - T, where T is the Fano decoder threshold quantum.Decoding continues in the Fano mode with To varyingas needed until such time as the decoder arrives for thefirst time at some node Sk at which the current value ofTo is to be raised (until this time the variation in To wasdownward only). At this moment Sk is inserted into thestack, Tis set equal to To+ T [note that L(Sk);::: To+ T]and decoder operation resumes in the stack mode. Ob-viously, if Sk is not the true path, with high probabilityall of the paths leaving it will have likelihood less thanthe new value of T. In such a case the stack will againbe found empty and the decoder will switch back tothe .Fano mode as described above. It is importantto note that this switchback will occur after fewer decodingsteps have been completed than would have been neces-sary .for the Fano decoder to lower its threshold

from To + T. Thus our strategy wastes no steps by apossibly premature switch into the stack decoding mode.

The flow chart of the complete stack-Fano algorithmis shown in Fig. 8. The notation of Section lOA of Ref. 4is used.

8. Some remarks on implementationIn this section we will discuss the problem of maintainingthe stack in proper likelihood order and the problemof specifying the tree paths entered into the stack. Weshall describe the way our implementation has beencarried out.

Although we found the concept of a stack useful fordescribing the algorithm, it turned out to be preferable toarrange the decoder's memory as a random access storage.In fact, a physical stack would necessitate a sequentialcomparison of its entries with the likelihood value ofthe path to be inserted, followed by a large relocationof data to make space for the new entry at the appropriatestack position. The amount of work connected with aninsertion would vary as a function of stack content.

Instead, we found it advantageous to establish equiv-alence classes for likelihoods (e.g., all likelihoods withthe same integral part belong to the same class) and toprovide corresponding class buckets for insertion ofnew entries. The buckets are then ordered and a decodingstep consists of selecting an arbitrary path of the topbucket, computing the likelihoods of its d extensions,computing the class membership of these likelihoods,and inserting the corresponding paths into the appropriatebuckets. To be more specific, we will describe the systemused by the author during simulation.

There are two sets of reserved storage locations, theauxiliary stack and the stack itself, referred to by indicesIE {-K - k + 1 ... 0 1 ... J} and g E {I 2 ...

, " '" '"M}, where K and J are chosen so that the likelihoodvalues of paths visited during the decoding process liebetween -K and J with sufficiently high probability.M is the size of the stack, each entry of which consistsof three parameters: S(g}-the path specification, L(g}-the likelihood value, and P(g}-a pointer to be described.The stack is filled sequentially as follows: The first pathis put into location 1 (this path is then on top of thestack). Whenever a path is extended, its parameters arereplaced by the corresponding.parameters of the extendedpath. If an additional entry into the stack is made it isplaced into the first unfilled location, if there is one.(When the stack is full, the bottom-of-the-stack path isreplaced, as discussed below.) Let G(l) be the entry atlocation I of the auxiliary stack. Its value is equal to thestack address of the last entered path whose likelihoodhas an integral part equal to I (if no such path exists inthe stack, G(l) = 0). The pointer P(g) is equal to the

ffiM J. RES. DEVELOP.

Discard or insertextensions intostack and adjust

T if necessary

I!iI

_-I

r---------No

1-~- ~ - - - ---- - --------- --- ----------- ----------- ----- - ------------- -------

IIIIIIIII

IIIIIIIIIII

C' =C(s;-1,5;) I1 ;=;-1 iI IL ~ ~

Figure 8 Flow chart of the combined stack-Fano decoding algorithm; notation as in Ref. 4.

stack address of the previously entered path whose likeli-hood has an integral part equal to the integral part ofL(g) [if no such path exists, peg) = 0].

The appropriate parameter values are maintained inthe two storages as follows: Suppose the path at locationg of the stack is to be extended (and therefore eliminatedfrom the stack) and suppose I equals the integral partof the likelihood L(g). Then the entry G(l) of the auxiliarystack will be set equal to the pointer peg). Next, supposethe path S' of likelihood L' is to be inserted into locationg' of the stack. If I' is the integral part of L', then peg')is set equal to the current entry G(l') of the auxiliarystack, whereupon G(l') is reset equal to s'. Finally, L(g')

is set equal to L' and S(g') to S'. Table 1 illustrates thestack maintenance procedure for the unmodified algo-rithm of Section 3 in the case of the likelihood situationof Fig. 3.

The auxiliary stack entries G(l) are used to find thetop-of-the-stack path to be extended and the bottom-of-the-stack path to be eliminated when a new entryis to be made into a full stack. The address of the topof the stack is G(l*), where 1* is such that G(l) = 0 forI = 1* + 1, ... , J and G(l*) ~ O. When the stack isfull, the new entry is put into location g + = G(l*), whereG(l) = 0 for 1= -K, ... ,1+ - I and G(n ~ O. Beforethis is done, G(t) is set equal to peg "). 683

NOVEMBER 1969 FAST SEQUENTIAL DECODING

684

F. JELINEK

Table 1 The contents of the stack and the auxiliary stackduring the search of the tree of Fig. 3.

Step 2

b S L P gJ 0 -6 0

2 10 -3 0

11 -3 .2 -3

-6

Step 4 Step 6b S L P g b S L P g

0 -6 0 000 -11 0

2 100 -7 4 2 100 - 7 4

3 110 -7 0 3 110 - 7 0

4 111 -7 3 4 III - 7 3 -4 7

5 101 -7 2 -5 0 5 101 -7 2

-6 6 01 -12 0

-7 5 7 001 - 4 .0 -7 5

-8 0

-11

-12 6

Next, let us consider how a path is to be specified inthe stack. If the tree code is convolutional (see Section10.12 of Ref. 4), then at least the stack parameters S(b)must characterize the tree depth i of the path and thesequence of p last message digits 8;-.+1> , , 8;, wherep is greater than or equal to the constraint length of thecode. If p is less than I' + t, the block length of the code,then a way must be found to determine the decodedpath. It would be natural to release to the user the digit8.-.+1 whenever the tree depth i of the path at the topof the stack exceeds the depth of all the previously ex-tended paths. In order to keep the probability of a wronglyreleased digit small, p might have to be three times as largeas the constraint length. This would make the stackstorage unacceptably large and so a more subtle methodof path specification must be devised.

A possible solution is to maintain in the storage amap of the paths contained in the stack. The constructionof the map is based on the tree structure of the code

and on the observation that the decoder extends a nodeat most once. (This is not strictly true of the modifiedalgorithm of Section 6, but a simple adjustment of ourmethod will take care of that problem.) The idea is simplyto store, for each node of an investigated path, the digit8 E {O, I, . .. ,d - I} corresponding to the last branchand a backward pointer p to the preceding digit-pointercombination. When a path is extended from a node,the d newly created pointers will point to the digit-pointercombination of the former (again, we refer to the un-modified algorithm of Section 3).

The following more specific description will generalizesomewhat the preceding notion.

An integer k is chosen and map storage space is al-located in which entries V*(j) and Q*(j) at location j rep-resent a sequence of k message digits and a pointer,respectively. The stack parameter S(g) itself consistsof three parts: V(g), capable of representing k - 1 messagedigits; I(g), representing the path depth; and Q(g), a pointer.At the beginning V*(j) = Q*(j) = V(g) = Q(g) =I(g) = 0 for all j and g. As long as the path depth I(g) < k,then Q(g) = 0 and V(g) = 81> '" , 8I(g), the messagesequence. As soon as a path Sk-1 is to be extended to(Sk-\ Sk)for the first time, we set V*(1) = (Sk-\ Sk), V(g) =0, I(g) = k, and the pointer Q(g) is set equal to 1, thuspointing to the first map location. In general, wheneverpath extension occurs from a path S;-l to s', where i isnot a multiple of k, neither the pointer Q(g) nor Q*(j)is changed, no new entry in the map is made, I(g) is setto i, and S; is added to the content of V(g). Suppose thati is a multiple of k and that j is the first free location inthe map. Then the pointer Q*(j) is set equal to the oldpointer value Q(g), V*(j) is set equal to S;-k+l> .. ,8;,Q(g) is made to point to the map location i. and we makeI(g) = i and V(g) = 0. It is obvious that in this way theentire path corresponding to any stack entry g is keptavailable in the storage. In fact, the path digits 8 i, S ;-1>' . "81 are represented in the registers V(g), V*(j1), ... , V*(jt),where i, = Q(g), j2 = Q*(M, ... , i. = Q*(jt-1)' andQ*(M = o.

If it is desired to purge the map of entries made un-necessary by path purges of the stack, this may be ac-complished best by adding another register C*(j) at thevarious map locations. C*(j) will at all times be equalto the number v (~ dk) of pointers pointing to the maplocation j. Suppose the entry g is to be purged from thestack and i, = Q(g). Then the value of C*(j1) is loweredby 1. If the new value is not equal to 0 nothing furtheris done, if it is equal to 0 and j2 = Q*(j1) then C*(Mis lowered by 1 and the i, map location is made availableto a pool for new refilling. The new value of C*(j2) issimilarly compared with 0 and, if it is 0, then C*(ja) Ua =Q*(j2)] is in turn lowered by 1 and the j2 map locationbecomes available, etc.

IBM J. RES. DEVELOP.

Table 2 The path identification map for stack size greater Table 3 The path identification map for stack size equal tothan 6 and k = 1. 3 and k = 1. Steps 1 and 2 are the same as in Table 2.

Step I Step 3 Step 4

Node Q v* C* Node v* Q* C' Node Q v* v* Q* C*1 0 0 0 0 0 0 0 0

2 2 0 2 0 2 11 5 0 2 11 5 0 2

6 2 4 0 2 16 2 4 0 2 13 6 0 2

2 0 3 2

0 3Step 4 0 4

Node Q v* C* Node v* Q* c*0 0 II 0 0 Step 5 Step 6

11 0 4 11 0 4 Node Q v* C* Node v* Q* C*13 3 7 0 2 2 13 0 2 2 2 7 0 0 2 0 0

12 6 2 2 12 6 2 2 11 3 5 0 11 3 5 0 2

14 3 8 0 3 14 8 0 13 3 6 0 2 13 3 6 0 2

3 4 2 10 2 2

0 4 12 0 4 0 0 3

4 4 () 4 0 4

0 2 0 o

7

0 9

9

Table 2 illustrates the map structure for the case of thelikelihood situation of Fig. 3 if the stack size is greaterthan or equal to 7 and k ~ 1(thus the V entries are absent).Table 3 does the same for the case in which the stack sizeis limited to 3 entries.

AcknowledgmentThe author thanks John Cocke for his inspiration andmany valuable discussions and suggestions.

References1. J. Ziv, "Successive Decoding Scheme for Memoryless

Channels," IEEE Trans. Information Theory IT9, 97(1963).

2. J. M. Wozencraft, Sequential Decoding for ReliableCommunication, MIT-RLE Tech. Rep. TR325, 1957.

3. R. M. Fano, "A Heuristic Introduction to ProbabilisticDecoding," IEEE Trans. Information Theory IT9, 64(1963).

4. F. Jelinek, Probabilistic Information Theory, McGraw-Hill Book Co., Inc., New York 1968.

NOVEMBER 1969

5. F. Jelinek, "An Upper Bound on Moments of Sequen-tial Decoding Efforts," IEEE Trans. Information TheoryITIS, 140 (1969).

6. F. Jelinek, "A Stack Algorithm for Faster SequentialDecoding of Transmitted Information," IBM ResearchReport RC 2441, April 15, 1969.

7. I. M. Jacobs and E. Berlekamp, "A Lower Bound tothe Distribution of Computation for Sequential De-coding," IEEE Trans. Information Theory IT13, 167(1967).

8. 1. M. Wozencraft and I. M. Jacobs, Principles of Com-munication Engineering, John Wiley & Sons, Inc., NewYork 1965.

9. D. Falconer, "A Hybrid Sequential and Algebraic De-coding Scheme," Sc.D. thesis, Dept. of Elec, Eng.,M.LT., 1966.

10. F. L. Huband and F. Jelinek, "Practical Sequential De-coding and a Simple Hybrid Scheme," IntI. Conf. onSystem Sciences, Hawaii, January 1968.

11. F. Jelinek and J. Cocke, "Adaptive Hybrid SequentialDecoding," submitted to Information and Control.

Received April 10,1969

685

FAST SEQUENTIAL DECODING