7/28/2019 LinWSJThgmjgjghj

1/5

DSP for Dummies(or What I Learned About DSPWhen Writing LinWSJT)

Jonathan Naylor HB9DRD/G4KLX

Abtract: The JT44 data mode has become verypopular over the last year; its success has beenoutstanding, both in terms of usage and in makingEME available to modest stations. This paperattempts to explain the methods used to decodeit. It is hoped that this article makes powerful DSPtechniques such as Fast Fourier Transforms andCorrelation more understandable to the averageamateur. The mathematical level of this paper isbasic.

Introduction

The purpose of this paper is to describe what Ihave learned about DSP (Digital SignalProcessing) in the course of implementing theJT44 data mode for Linux. What I have learned isapplicable to any operating system andprogramming language and no specific referenceswill be made to either Linux/UNIX or C++.

With the power of modern PCs and with standardsound cards, it is possible to do DSP work onhome PCs without having to invest in specialisthardware. Many pieces of software haveappeared over the last few years that make use ofthis fact, the most ambitious being LinRad by LeifSM5BSZ, while the most revolutionary beingWSJT by Joe K1JT [1]. I use the termrevolutionary in that WSJT (in JT44 mode) makesmaximum use of a number of powerful DSPtechniques in order to push back the frontiers ofweak signal decoding in amateur radio.

Many of the DSP techniques are not intrinsicallycomplex, however most DSP books tend to betextbooks for university courses and include alarge amount of relatively advanced mathematics.However if you treat the techniques as blackboxes, then a lot of the mathematics can be

dispensed with. This article aims to demystifysome of the more interesting techniques that Ihave used when implementing JT44 in LinWSJT.

What is the JT44 Data Mode?A JT44 transmission lasts for slightly over 25seconds, in that time, a 22-character message isrepeated three times. Each character isrepresented by a unique tone. In addition,

between many of the characters is asynchronisation tone, and there are 69 of those.This leads to a complete transmission consistingof 135 characters transmitted at 5.38 baud. Thepattern of the synchronisation tone and thecharacters is fixed and uses a pseudo randompattern.

The main problem in decoding JT44 is identifyingthis synchronisation tone within a range offrequencies (600 Hz is specified) and also in arange of time (-2 to +4 seconds). Once that isdone, the recovery of the message becomes arelatively simple problem. But in order to do any ofthese things, you need to understand two basicDSP mechanisms, Fast Fourier Transforms andCorrelation.

Fast Fourier Transforms (FFT's)The FFT is a mechanism that may be used to

convert data between the time domain and thefrequency domain. What does that mean? Inpractical terms, it means that I can feed in audiodata and get out data that represents theindividual frequencies that make up the originaldata. An FFT can be used as the main part of aspectrum analyser, unlike an RF spectrumanalyser; an FFT provides all of the data for all ofthe frequencies simultaneously. The software thatamateurs use to "see" weak signals, for exampleFFTDSP by AF9Y, is based on a FFT. I do notneed to know how it works, just how to apply it,and how to interpret the resulting data.

The main attribute of a FFT is its length that mustbe specified before using it. The length isimportant as this information along with the speedat which the incoming audio is being sampled,allows you calculate how wide the bandwidth ofeach output of the FFT is, and what frequency itrepresents. These outputs from the FFT arecalled "bins". In the case of JT44, the audio isbeing sampled at 11025 times per second, theFFT has a length of 2048 (i.e. 2048 output bins),which means that each bin is 5.4 Hz wide and thatbin 236 is centred on 1270.5 Hz. The

mathematical relationships between these valuesare:

Bin Width (Hz) = Sample Rate (Hz) / FFT Length

Bin Freq. (Hz) = Bin Number * Bin Width (Hz)

If you were to feed white noise from your radiointo an FFT, the results would be similar to whatcan be seen in figure 1.

7/28/2019 LinWSJThgmjgjghj

2/5

Figure 1: White Noise Through an FFT

A Sine wave fed into an FFT would cause aneffect similar to figure 2.

Figure 2: Sine Wave Through an FFT

With strong signals and narrow FFT bins, thevalues in the bins adjacent to the bin containingthe incoming signal will have increased values.The FFT is good, but it is not perfect.

The FFT is a very powerful tool, and it has abrother called the Inverse Fast Fourier Transform

(IFFT) that does the opposite. It allows you tocreate an arbitrary spectrum from input data thatrepresent the frequencies to be generated; thesame mathematical relationships apply to an IFFTas to an FFT. An IFFT is used as the basis for thetransmitting part of LinWSJT.

There is a very important property of an FFT thatis extremely valuable, and is essential for thedecoding of JT44 (and many other data modes). Itis that a N-point FFT is perfectly matched todecoding data that has symbols that are N audiosamples long. In the case of JT44, N is 2048. Theproblem is knowing where to place the FFT in theincoming data stream.

Figure 3: Optimal Location for an FFT

Figure 3 shows incoming data being sampled in

three places by FFTs, however only one of themis optimal. The outputs of FFT1 and FFT2 willcontain elements from two different symbols,neither of which will be at full amplitude. Theoutput from FFT3 will contain data from only onesymbol that is perfectly matched to the FFT. Theproblem that faces the program using the FFT isto know where to place the FFT relative to theinput data stream. This problem will be returned tolater.

CorrelationCorrelation is the mathematical process of

comparing two waveforms, and producing a result(the correlation value) that represents how closelythe two waveforms match. The correlation valuein itself is not that important except as acomparative indication.While the mathematics behind the FFT arecomplex, the mathematics behind Correlation aresimple. All that is required is that a copy of theexpected incoming waveform is available for

Freq.

FFT1 FFT3FFT2

Time

Time

Ampl.

FFT

FFT Bins

Ampl.

Time

Ampl.

FFT

FFT Bins

Ampl.

7/28/2019 LinWSJThgmjgjghj

3/5

comparison purposes. The process of Correlationonly requires the use of multiplication andaddition, albeit many of them.

Figure 4: Correlation Examples

The best way to describe Correlation is to worksome simple examples. A number of differentwaveforms can be seen in figure 4, the top one isthe original that we will correlate the othersagainst. Wave 1 is the exact opposite waveform,wave 2 is a continuous high value, wave 3 is acontinuous low value, wave 4 is a single peak,and wave 5 is an exact duplicate of the original

wave. Although this example is using squarewaves (which also happens in LinWSJT), theexact same principle applies to any other waveshapes.

To Correlate the different waveforms we mustcompare the original waveform and each of theother waveforms at the same points in time. Forthis example we will sample them at regularintervals, using the middle of each level of theoriginal waveform as a guide. The values for eachof the waveforms can be seen in table 1.

Position-------

1 2 3 4 5 6

Original 1 -1 1 -1 1 -1

Wave 1 -1 1 -1 1 -1 1

Wave 2 1 1 1 1 1 1

Wave 3 -1 -1 -1 -1 -1 -1

Wave 4 -1 -1 1 -1 -1 -1

Wave 5 1 -1 1 -1 1 -1

Table 1: Waveform Values

To calculate the correlation value, the values ofthe waves at the same points in time must bemultiplied with the value of the original wave, andthen added together. The calculations and theirresults (correlation values) can be seen in figure2.

Calculation Result

Wave 1 1 * -1 + -1 * 1 + 1 * -1 +-1 * 1 + 1 * -1 + -1 * 1

-6

Wave 2 1 * 1 + -1 * 1 + 1 * 1 +-1 * 1 + 1 * 1 + -1 * 1

0

Wave 3 1 * -1 + -1 * -1 + 1 * -1+ -1 * -1 + 1 * -1 + -1 *-1

0

Wave 4 1 * -1 + -1 * -1 + 1 * 1 +-1 * -1 + 1 * -1 + -1 * -1

+2

Wave 5 1 * 1 + -1 * -1 + 1 * 1 +-1 * -1 + 1 * 1 + -1 * -1

+6

Table 2: Correlation Value Calculations

The results are as expected. The copy of theoriginal waveform, wave 5, is clearly the bestcorrelation. Wave 1, which is the inverse of theoriginal wave, has the worst. Two other values areworth noting, wave 2 can be thought of asrepresenting the output of an FFT that is receivinga continuous carrier, it has a low correlation value.Wave 3 can be thought of as no signal (whitenoise), again it has a low correlation value.

Therefore with Correlation, interference signals

(birdies) do not cause false triggering, anextremely useful property.

Decoding JT44The input for DSP programs such as LinWSJT isusually a sound card in a PC. Many modernsound cards boast 3D Sound, MIDI interfaces andthe like, however these are not needed in thisapplication. All that is required is for the soundcard to act as an audio Analog-to-Digital andDigital-to-Analog converter. The importantparameters for the sound card are the samplerate and the number of bits of precision available.

The basic rule for sampling signals is that thesampling rate must be at least twice the maximumfrequency that is to be processed. For a normalamateur SSB radio this is 2500 Hz, so the JT44sampling rate of 11025 Hz easily fulfils thisrequirement. The number of sampling bitsdetermines the dynamic range of the receivedaudio data. In the case of JT44 where (a) the

Time

1Original-1

1Wave 1-1

1Wave 2

Wave 3-1

1Wave 5-1

1Wave 4-1

7/28/2019 LinWSJThgmjgjghj

4/5

signals will be weak, and (b) the receivers AGC isenabled, even 8-bit sound cards are adequate.

As mentioned earlier, the key to decoding JT44 isto identify where the synchronisation tone is interms of time and frequency. Once that is done, itis a simple matter then to apply FFTs to the inputaudio data at the correct points in time, and tomatch the data tones to the letters and numbersof the message. In the LinWSJT program, the textdecoding is the simple part; the interestingchallenges are to be found in the synchronisationtone processing. The structure of the JT44decoding section within LinWSJT can be seen infigure 5.

The first step is to process the incoming audiodata with a 2048-point FFT. The values for bins124 to 347 (the limits of the synchronisation tonefrequencies) are then fed into a Correlator that

operates with a local copy of the synchronisationtone waveform, and slides it in time relative tothe incoming data from 2 to +4 seconds. At theend of the incoming audio data, the correlationvalues from these different frequencies and timesare scanned to find the highest value, this givesboth the frequency and time offset of thesynchronisation tone. These values are then fedinto the text decoder.

Within LinWSJT, a correlation against a squarewave that represents the on-off waveform of thesynchronisation tone is used, rather than a simple

copy of the expected audio waveform. Even withthis simplification there would still be manycalculations that need to be done and a lot of datastorage needed for the results. Instead of testingfor every time point between 2 and +4 seconds(which equate to 2 * 11025 and +4 * 11025samples), the program only does a check every1024 samples within that time range, which isapproximately every 0.1 seconds. This has theeffect of saving memory because there are lesscorrelation values to store, and saving timebecause the processing has been reduced to1/1024 of the previous value. The value of 1024

was determined experimentally, using samples ofweak JT44 signals. The results of thisapproximate correlation are the exact frequency(FFT bin number) and the approximate time of thesynchronisation tone.

Once an approximate correlation is found, acontinuous check is made for correlation in thetime range of 1024 samples relative to theapproximate correlation time. Such a continuous

scan is processor intensive but within such anarrow range it is completed within a fraction of asecond. This results in the exact time of thesynchronisation tone. The resulting data from thiscorrelation process is used to calculate the signalstrength, the synchronisation quality, and producesome of the graphs in the program display.

Figure 5: JT44 Decoding Structure withinLinWSJT

The next stage is the text decoding. A simple andeffective way to do this is to add the values for thedifferent letters and numbers for each of the 22characters (from three locations each) andchoose the strongest. This works well, however inthe presence of interference it may becomelocked to the interfering signal and cause the

wrong letter or number to be decoded.

The answer to this is to use another Correlatorthat checks that a letter/number tone is notpresent when there should be a synchronisationtone. This has the effect of rejecting continuoussignals that are typical of birdies, leaving the realJT44 data to be decoded. On real world signalsthat are weak and in the presence of interference,

Sample incomingAudio Data at 11025

Hz

Save Audio Data toDisk

[Optional]

Perform Correlationevery 1024 Samples

Perform exactCorrelation on single

FFT Bin

Extract MessageData

Add Message Data

to Average Message

WAV File format

Exact FFT Bin andApproximate Timing

Exact Timing

Data for SignalStrength andGraphic Display

Message Text

Average Message Text

7/28/2019 LinWSJThgmjgjghj

5/5

a noticeable improvement in recovered text canbe seen.

Further improvements in extracting text from veryweak signals can be obtained by averaging thereceived data both within a single JT44 messageand over a number of JT44 messages. Within asingle message, the odd characters and the evencharacters are averaged together which can makethe decoding of messages such as ROROROmuch better, also the last four characters in themessage are also decoded as a single characterfor messages that contain a fixed ending such as OOOOOO. These are standard in the originalWSJT program.

Each incoming message (subject to certainrestrictions) is added to an average message thatis displayed separately. When the same messageis received a number of times, even when the

individual messages have not been decodedcorrectly, the cumulative average of thesemessages may result in meaningful text beingdecoded. In order for this to work, it is important tobe able to have control over which incomingmessages make up this average message. It isimportant to be able to clear the averagemessage and start it again once the remotestation has changed the message content.

Further DevelopmentJT44 despite its success is not suitable formicrowave EME operating. The top frequency at

which JT44 is useful has not been ascertained, itis still usable on 1296 MHz, but at 10 GHz thespreading of the signal would cause each tone tooverlap many FFT bins and make decodingimpossible. For these frequencies a different datamode is needed which uses wider bins, thisreduces sensitivity which must then be made upin other ways, possibly by extending the transmitand receive periods from the current thirtyseconds. This is an area of potentialdevelopment.

My next challenge is to implement FSK441, the

Meteor Scatter mode. It is quite a differentproblem than the one that JT44 represents, forexample Correlation will probably not be needed.ConclusionDevelopment of the JT44 program has been mostinteresting for me; probably the most fun part hasbeen learning new techniques. My source ofinformation and inspiration has been TomiManninen OH2BNS who also learned DSP

development by writing software. He has spentlong periods on the WW Converse networkexplaining DSP concepts to me, and answeringnumerous e-mails. I would like to thank him forthat.

I have had two DSP books recommended to me,both provide program examples and I have listedthem in the references section [2][3], one of themis freely available on the Internet. Most DSPbooks are heavily mathematical and for someonetrying to solve a real life problem, they are not souseful.

LinWSJT is developed using C++ on the Linuxoperating system. It makes use of the wxWindowsmulti-platform GUI framework that should meanthat it could eventually be ported over to Windowsand the Mac with relatively little effort. LinWSJTand wxWindows (as well as Linux) are open

source software and are downloadable for freefrom the Internet. The complete source code forLinWSJT is available, it can be found at [4] underthe entry Software.

References[1]http://pulsar.princeton.edu/~joe/K1JT,http://www.vhfdx.de/wsjt

[2] E. Ifeachor & B. W. Jervis, Digital Signal

Processing, 2nd Edition, Prentice-Hall, ISBN0201596199[3]http://www.dspguide.com/[4]http://www.qsl.net/g4klx

http://pulsar.princeton.edu/~joe/K1JThttp://pulsar.princeton.edu/~joe/K1JThttp://pulsar.princeton.edu/~joe/K1JThttp://www.vhfdx.de/wsjthttp://www.dspguide.com/http://www.dspguide.com/http://www.qsl.net/g4klxhttp://www.qsl.net/g4klxhttp://pulsar.princeton.edu/~joe/K1JThttp://www.vhfdx.de/wsjthttp://www.dspguide.com/http://www.qsl.net/g4klx