Towards Real-Time Impulsive RFI Mitigation for Radio Telescopes

Kaushal D. Buch*, Shruti Bhatporia, Yashwant Gupta, Swapnil Nalawade, Aditya Chowdhury,Kishor Naik, Kshitij Aggarwal and B. Ajithkumar

Giant Metrewave Radio Telescope, National Centre for Radio AstrophysicsTIFR, Pune, Maharashtra 411007, India

*kdbuch@gmrt.ncra.tifr.res.in

Received 2016 August 15; Accepted 2016 November 29; Published 2017 January 13

Radio Frequency Interference (RFI) is a growing concern for contemporary radio telescopes. This paperdescribes techniques for real-time threshold-based detection and ¯ltering of broadband and narrowbandRFI for the correlator and beamformer chains of a telescope back-end, with speci¯c applications to theupgraded Giant Meterwave Radio Telescope (uGMRT). The Median Absolute Deviation (MAD) estimatoris used for robust estimation of dispersion of the received signal in temporal and spectral domains. Resultsfrom the tests carried out for the GMRT wide-band backend (GWB) using this technique show 10 dBimprovement in the signal-to-noise ratio. MAD-based estimation and ¯ltering was also found to be usefulfor ¯ltering beamformer data. The RFI ¯ltering technique demonstrated in this paper will ¯nd applicationsin other radio telescopes as well as receivers for digital communication and passive radiometry.

Keywords: RFI, median absolute deviation, radio telescope, GMRT.

1. Introduction

Radio telescopes are wide-band radio receivers usedto detect and analyze faint radio emissions fromcelestial sources. Typical radio telescope receivershave almost 50 dB higher sensitivity than theirterrestrial-communication counterparts. Thisincreases their susceptibility towards man-madeRadio Frequency Interference (RFI) which impairsthe detection of weak radio sources and transientevents (Edwards, 2014). Thus, mitigation of RFI isan important aspect in the design and operation ofradio telescopes (Dewdney et al., 2009; Ellingson,2004). In terms of its frequency-domain character-istics, RFI be categorised as broadband or narrow-band. In the time-domain, broadband RFImanifests as short duration, impulsive bursts of in-terfering signal, typically from spark-like events(e.g. lightning discharges, power line discharges,spark ignition discharges etc); whereas narrowbandRFI is due to long duration spectrally con¯ned

sources such as transmissions from communicationsystems — terrestrial or otherwise.

A radio telescope receiver gets a combination ofthree signals at its input — system noise, sky noiseand RFI. System noise is a combination of skybackground noise and receiver noise, both of whichare zero-mean uncorrelated Gaussian distributedrandom signals. In general, RFI has a non-Gaussiandistribution (Fridman & Baan, 2001) and getsadded to the overall receiver noise. Thus, a gener-alized time-domain model for a signal received by aradio telescope in the presence of RFI can be givenas (Fridman & Baan, 2001),

xðtÞ ¼ sðtÞ þ nðtÞ þ rðtÞ; ð1Þwhere, sðtÞ is the contribution from the astronomi-cal source, nðtÞ is the system noise and rðtÞ is theRFI. The distribution of signal with impulsive RFIis heavy-tailed Gaussian distribution (Fridman,2008). Sources of broadband RFI are sparking andcorona discharges on high-power transmission lineswhich occur at submultiples of the power-linefrequency (Swarup, 2008) with a time duration*Corresponding author.

Journal of Astronomical Instrumentation, Vol. 5, No. 4 (2016) 1641018 (14 pages)#.c World Scienti¯c Publishing CompanyDOI: 10.1142/S225117171641018X

1641018-1

ranging from 5ns to 200 ns (Langat, 2011). Themain sources of narrowband RFI are emissions fromTV broadcast transmitters, cellphone base stationtransmitters, mobile communication devices andsatellites.

A class of RFI mitigation techniques, known asRFI excision, is used for mitigating impulsive time-domain or frequency-domain RFI by determiningsignal dispersion and robust thresholding (ITU-R,2013). Moment-based estimation using StandardDeviation (STD) and Kurtosis has also been usedfor RFI excision in radio telescopes (ITU-R, 2013),(Niamsuwan et al., 2004) and passive microwaveradiometers (Guner et al., 2007). For RFI having aheavy-tailed Gaussian distribution, Median Abso-lute Deviation (MAD) has been proposed as arobust rank-based statistical estimator (Fridman,2008). MAD-based excision is also shown to be ef-fective in radio astronomy applications (Fridman,2008; Shankar & Pandey, 2006) as well as in othersignal processing areas like time-series analysis andimage processing (Crnojevic et al., 2004). However,the authors have not come across any account ofMAD-based real-time implementation in a wide-band radio telescope backend.

This paper describes techniques for real-timebroadband and narrowband RFI detection and ex-cision (¯ltering) using MAD estimator. The detec-tion of RFI is followed by a nonlinear ¯lteringoperation wherein the RFI is replaced by a constantvalue or sample from noise source. An optimizedimplementation is developed which enablesresource-e±cient real-time ¯ltering of RFI in a wide-band system. The e®ect of real-time RFI ¯ltering onthe sensitivity and signal correlation are analyzedfor di®erent con¯gurations of a radio telescopebackend. RFI ¯ltering using the proposed techniquehas been implemented in real-time and tested withthe wide-band receiver chain at the Giant Meter-wave Radio Telescope (GMRT) (Swarup et al.,1991). The broadband and narrowband RFI ¯lter-ing results show an improvement of 10 dB in signal-to-noise ratio and a similar improvement in thecross-correlation power spectrum, thus improvingthe receiver sensitivity.

2. Radio Telescope Signal ProcessingBackend

A radio telescope consists of either a single antennaor an array of antennas with a sensitive radio re-ceiver(s) for processing weak astronomical signal.

A typical digital backend of a radio telescope re-ceiver processes baseband signals downconvertedfrom radio frequency signals obtained from indi-vidual antenna elements. The primary objective ofan array radio telescope receiver is to computecorrelations between pairs of antennas also knownas visibilties. Towards that end, in a typical digitalbackend of a radio telescope (Fig. 1), the signals aretime-aligned through delay correction at sample andsubsample level. After Fourier transformation, theauto-and cross-correlation spectra are generated.For N antenna inputs, the MAC block producesNðN � 1Þ=2 time-integrated cross-correlation andN auto-correlation spectra. Alternatively, receviedsignals may be combined to produce single or mul-tiple beam outputs. The signals in the beamformercan either be combined after phase alignment toproduce Phased-Array (PA) beam or without phasealignment to produce Incoherent Array (IA) beam.The temporally-integrated signal from the corre-lator and beamformer signal is recorded for furthero®line processing. The beamformer mode is gener-ally used for pulsar observations. To get the ¯nalpulsar pro¯le, the beam output undergoes the pro-cess of de-dispersion and folding. De-dispersion iscarried out to remove the temporal smearing thatthe pulse undergoes while it passes through theinter-stellar medium. Folding is used to improve thesignal-to-noise ratio by adding successive pulses.

2.1. GMRT Wide-band Backend

The GMRT Wide-band Backend (GWB) is a wide-band digital signal processing system (Ajithkumaret al., 2013) being developed as part of the Upgra-ded GMRT (uGMRT). The real-time processingbandwidth of 400 MHz is achieved using a combi-nation of contemporary Field Programmable GateArray (FPGA) and Graphics Processing Unit(GPU) (Ajithkumar et al., 2013). GWB is builtusing ROACH-1 boards(a) which digitize the dataand provide it through 10GbE links to the GPUcluster. The data from high-speed 8-bit, 1 GspsADC (e2V AT84AD001B) is provided to ROACH-1board which consists of Xilinx Virtex-5(XC5VSX95T-1FF1136) FPGA. Nvidia K20 GPUsare used for real-time signal processing in the GWBand are hosted by server-class machines havingmulti-core CPUs. The number of FFT channelssupported by the GWB range from 2,048 to 16,384.

ahttps://casper.berkeley.edu/wiki/ROACH.

K. D. Buch et al.

1641018-2

GWB supports a correlator output and two simul-taneous beam outputs for a 32-antenna dual-polar-ization input. The data processed by the GPUcluster is sent to three recording nodes, one forcorrelator and one each for the two beam outputs.The shortest integration times for GWB system are671ms and 1.3ms for the correlator mode and thebeamformer mode, respectively.

Real-time broadband RFI ¯ltering is carried outon Nyquist-sampled time-series, on FPGA. Corre-lation and beamforming operations in the GWB arecarried out in the frequency-domain on spectralchannels having a 100 kHz resolution. NarrowbandRFI ¯ltering is carried out (post-Fourier transfor-mation) on the CPU–GPU cluster. A data acquisi-tion system receives the processed data from thiscluster and stores it for o®line analysis. In order toavoid potential clipping of useful astronomical data,RFI ¯ltering is carried out before the data under-goes the process of de-dispersion and folding.

3. Robust Signal Processing Techniquesfor Real-Time RFI Excision

A conventional measure of the signal dispersion isits STD. In case of a radio telescope signal a®ectedby strong RFI, the STD estimator is biased by theRFI. Median-based estimators provide a robust es-timate of the dispersion (scale parameter) in pres-ence of RFI (Fridman, 2008). One such estimator is

MAD which has a breakdown point of 50%, whichindicates that its estimate of the dispersion is un-a®ected till the number of outliers in a data set isless than 50%. Here, MAD is proposed for estimat-ing signal dispersion for symmetric Gaussian dis-tributed data with heavy tails. When the underlyingdistribution is Gaussian, the robust STD can becalculated from the MAD value using a scalingfactor of 1.4826 (Rousseeuw & Croux, 1993). Weextend the process to compute robust thresholdaround the median of the distribution which is fol-lowed by nonlinear ¯ltering of RFI. This type of¯ltering is also called decision-based nonlinear ¯l-tering (Astola & Kuosmanen, 1997) or Hampel ¯l-tering (Pearson et al., 2016).

3.1. MAD-based RFI Excision

Real-time MAD-based RFI detection and excisionproposed in this paper is explained through thefollowing steps:

(1) Consider a window of W input data samplesX ¼ ½x1x2 . . .xW �T . Let Mð:Þ represent themedian operator such that the median of theinput X is MðXÞ ¼ M. Now, let Y ¼½y1y2 . . . yW �T , where Yi ¼ xi �M 8i 2 1; 2;. . . ;W which is median subtracted from theinput data and the di®erence converted to ab-solute values. MAD is obtained by performingmedian operation on Y , i.e. MAD ¼ MðY Þ. The

Fig. 1. Block diagram of the digital backend for the uGMRT, illustrating the points where real-time RFI ¯ltering can be carriedout.

Towards Real-Time Impulsive RFI Mitigation for Radio Telescopes

1641018-3

resultant MAD value is scaled by a factor of1.4826 (for Gaussian distribution) to get robustSTD (�R).

(2) Assuming Gaussian distributed data at theinput, the detection thresholds are computed asmultiples of �R on either side of the medianvalue M. The thresholds are computed onconsecutive windows of W samples. The upperand lower thresholds are denoted by �U and �L,respectively, and are

�U ¼ MðXÞ þ nU � ð1:4826�MðY ÞÞ;�L ¼ MðXÞ � nL � ð1:4826�MðY ÞÞ; ð2Þ

where nU and nL are the multiplying factors forthe upper and lower thresholds, respectively. Ifthe distribution is symmetric, nU = nL.

(3) A comparison between the input data andthreshold values provides a one-bit detection°ag where `1' indicates the detection of RFI and`0' indicates otherwise.

(4) The samples detected as RFI can be eitherreplaced by the threshold value (clipping) or bya user-de¯ned value. The choice of replacementvalue can be modi¯ed in real-time.In case of clipping, the RFI detected samples areheld at their respective upper and lowerthreshold values (�U and �L). Alternatively, thevalues can be replaced by the median, a con-stant value, or noise sample (having statisticalproperties like mean and STD similar to theunderlying distribution as estimated using me-dian and MAD). The constant value could beany number within the range of numbers sup-ported by the bit-precision. For example, in caseof 8-bit signed number system, the constantvalue can be any integer between �128 andþ127. In Eq. (3), the element zi in the outputvector z ¼ ½z1z2 . . . zW �T is replaced by K if theinput is outside the robust threshold.

zi ¼ K forxi � �U or xi � �L

¼ xi for �L < xi < �U : ð3Þ

The window size for MAD estimation is determinedby the worst-case duration of RFI and the samplingrate of the system. In order to get an unbiased es-timate using MAD, less than 50% of the samples ina data window should be a®ected by RFI. Thewindow size thus depends on the ratio of the dura-tion of RFI (TR) to the sampling interval (TS), i.e.2ðTR=TSÞ samples. The factor of 2 is due to the 50%

breakdown point of the MAD estimator. For thetechnique described in this paper, the window sizeremains constant once the above parameters arecomputed and the design is implemented.

The values of nU and nL in Eq. (2) are chosen insuch a way that the astronomical data does not getdetected and ¯ltered as RFI. If these values arelower, the ¯ltering operation would remove much ofthe useful data and if they are too high, most of thedata would pass through without ¯ltering. We ¯ndthat for most of the receiver system parameters andRFI conditions, values of nU and nL around 3 areoptimal. However, this is a tunable parameter andin the implementation described in this paper, it canbe changed on-the-°y to account for the dynamicchanges.

3.2. Performance of MAD-based detection

The performance of MAD-based detection is eval-uated through simulation. The MAD estimator wassubjected to varying levels of impulsive RFI. Forthis purpose, impulsive RFI above 3� was generatedand added to samples of noise having zero mean and¯xed STD. RFI was added to the noise at randominstances in time in a block of 1,024 samples. As thepercentage of impulsive RFI increases above 50%,the MAD estimator gets biased. The bias is positivedue to larger dispersion and hence the obtainedthreshold is higher than the expected threshold.This causes the ¯lter to not detect (under-¯lter) theRFI samples. Hence, the di®erence of total numberof RFI samples and detected samples indicates thee±ciency of the ¯lter. These values are computedfor di®erent fractions of impulsive RFI in the noisesample and is an average of 100 blocks each having1,024 samples. The di®erence values range from 0indicating all RFI samples are successfully detectedto 1,024 which means that none of the RFI samplesare detected.

The behavior of MAD-based threshold and thedi®erence for di®erent values of nU and nL in com-parison with the conventional STD estimator-basedthreshold (3�) is shown in Fig. 2. STD estimatorgets biased even for a small percentage of impulsiveRFI. On the other hand, MAD estimator performsreasonably well up to 50% impulsive RFI. There is asteep rise in the di®erence value for more than 50%impulsive RFI because of the breakdown point ofMAD estimator. The best performance in this caseis observed for 3� detection threshold because, in

K. D. Buch et al.

1641018-4

this simulation, the samples of impulsive RFI areadded above 3� value of the underlying distributionof noise.

The number of di®erence values for nU and nL

equal to 4 increases gradually with the increase inpercentage of impulsive RFI. This shows that as thethreshold increases, there are more number ofsamples which are missed by the detector. Themissed samples are the ones between 3� and 4�. Forvalues of nU and nL equal to 2, the samples whichare not RFI are also detected leading to a negativevalue of the di®erence. This simulation shows thatMAD-based detection performs better in terms of¯ltering impulsive RFI as compared to the STD-based estimator. Also, the values of nU and nL inthe detection threshold needs to be chosen based onthe strength of RFI and the system parameters, toachieve optimal ¯ltering.

3.3. Quantitative metric for analyzing RFI¯ltering

The performance of RFI ¯ltering is quanti¯ed by aratio of mean-to-the Root Mean Square (RMS)value of the signal. This ratio is a normalizedquantity and is robust to system gain °uctuations inthe RF and analog receiver subsystems. The mean-to-RMS ratio (�=�) for a radio telescope receiver isgiven as

ffiffiffiffiffiffiffiffiffiffiffiffiffiB� T

p, where �, �, B, T are the sample

mean, sample RMS, bandwidth and integrationtime of the system, respectively.

The e®ect of RFI on mean-to-RMS ratio, com-parison with the theoretical limit and its use asquantitative metric is illustrated through a time-series plot of a single spectral channel in Fig. 3

corresponding to 651MHz radio frequency (RF)(200 MHz bandwidth, 2,048 spectral channels and1.31ms integration time) having instances ofbroadband RFI. The ¯ltering is carried out in time-domain on Nyquist-sampled digital time-series at 3�threshold and the detected samples are replaced byzero. The results can be compared with the theo-retical limits that can be achieved for a Gaussianrandom signal without RFI, which, for a single

spectral channel and the system speci¯cations

used in Fig. 3 can be given byffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiðB � T Þp

which

is ððð200 � 106Þ=2; 048Þ � ð1:31 � 10

�3ÞÞ0:5 ¼ 11:31.Mean-to-RMS ratio is computed for 2,000 samplesaround regions 1 and 2. It can be seen that themean-to-RMS ratio reduces in case of stronginstances of RFI. The mean-to-RMS ratio aroundregion 1, before and after ¯ltering, is 3.27 and 11.06,respectively, and that of noise (un¯ltered) aroundregion 2 is 10.99.

Percentage of Impulsive RFI0 10 20 30 40 50 60 70 80 90 100

Diff

eren

ce (

Tot

al R

FI S

ampl

es -

Det

ecte

d R

FI S

ampl

es)

-400

-200

0

200

400

600

800

1000

1200

STD - 3 sigmaMAD - 1 SigmaMAD - 2 SigmaMAD - 3 SigmaMAD - 4 Sigma

Fig. 2. Simulation showing performance of MAD-based impulsive RFI detection.

0 50 100 150 200 250 3000

100

200

300

400

500

Time(s)

Inte

nsity

(A

rb. U

nits

)

UnfilteredFiltered

1

2

Fig. 3. Mean-to-RMS ratio for broadband RFI ¯ltering.

Towards Real-Time Impulsive RFI Mitigation for Radio Telescopes

1641018-5

The improvement metric (I) in dB derived fromthe mean-to-RMS ratio is

I ¼ 10 log ðMRF=MRUÞ dB; ð4Þwhere MRU and MRF are the mean-to-RMS ratiofor the un¯ltered and ¯ltered signal, respectively. Ifmean-to-RMS for ¯ltered and un¯ltered is the same,the improvement is 0 dB. If the mean-to-RMS of the¯ltered signal is higher than that of the un¯lteredsignal, the improvement (in dB) is positive.

4. Broadband RFI Filtering in the GWB

Real-time broadband RFI ¯ltering is carried out onFPGA for 60 inputs (30 antennas, dual-polariza-tion) from the GMRT antennas, each having400MHz bandwidth. For processing Nyquist-sam-pled time-series at 8-bit precision, in real-time,hardware parallelism available on the FPGA isutilized. Figure 4 shows the block diagram of MAD-based RFI ¯ltering.

Here, the ¯rst step is to compute MAD, whichrequires two successive median computations. Eachmedian computation requires sorting the input dataset which is a resource intensive operation. Severaloptimized algorithms with varying computationalcomplexities are available for median computation.Their performance is governed by the input dataprecision and the window size (Juhola et al., 1991).A few examples of optimized sorting algorithms arequick sort, heap sort and k-selection method. As thisapplication requires a moderate window size andoperates on ¯xed point data, it was found that thehistogram-based approach is computationally opti-mal (Juhola et al., 1991). Also, the histogrammethod of median calculation is amenable to real-time implementation (Fahmy et al., 2009). We op-timize the histogram method to meet the real-timerequirements within the constraints of availablehardware resources on Xilinx Virtex-5 SX95TFPGA. The design(b) is developed and tested on

ROACH-1 board and is available as open-sourcemodule under the Collaboration for AstronomicalSignal Processing and Electronics Research (CAS-PER)(c) library.

4.1. Real-time architecture

An optimized implementation enables mediancomputation of W samples following an initial la-tency of W clock cycles. There are two mediancomputation blocks which operate simultaneouslyin order to compute MAD on contiguous datawindows. To begin with, the binning of data iscarried out to get a cumulative histogram. As partof the optimized implementation, this cumulativehistogram is not stored, instead, a median valuesearch is carried out. This is done by locating thedata value for which the cumulative frequencycrosses half the total number of samples in thewindow. The median so obtained is subtracted fromthe input data and the di®erence is converted toabsolute values which are then used by the secondmedian computation block to get the MAD value.Both the median computation blocks are architec-turally similar except that the ¯rst one operates on arange of N-bit signed data, i.e. �2

N2 to 2

N2 � 1,

whereas the second one operates on N-bit unsigneddata, i.e. 0 to 2N � 1.

The median and MAD values are then used tocompute the robust threshold as per Eq. (2). Sub-sequently, the input data is compared with thethreshold. The detection °ag is set to 1 if the sampleis outside the threshold. The ¯ltering operation usesa data multiplexer to replace the °agged samplewith one of the replacement options chosen by theuser. For replacement by noise, samples havingstandard normal distribution are generated onFPGA and shifted and scaled to con¯rm to themean and STD obtained from the median (M) androbust STD (�R), respectively. An FPGA-baseddigital noise source is used to generate Gaussian

Fig. 4. Block diagram of MAD-based RFI ¯ltering.

bhttps://casper.berkeley.edu/wiki/Impulsive RFI Excision:CASPER Library Block. chttps://casper.berkeley.edu/.

K. D. Buch et al.

1641018-6

distributed noise samples noise with programmablemean and STD. The detailed architecture of thedigital noise source and its properties are describedin Buch et al. (2014).

4.2. Results from antenna and emulatortests

The e®ect of RFI ¯ltering carried out in the time-domain is observed by recording data in thebeamformer and correlator mode of the GWB si-multaneously. Since broadband RFI a®ects all thespectral channels equally, the analysis is carriedout on a single spectral channel output for a pair ofantennas. Three copies of each signal are created inorder to get a simultaneous comparative analysis ofthe two ¯ltered and one un¯ltered output. Twodi®erent replacement options are applied for boththe antennas and one copy from each antennapasses without ¯ltering. Detection and ¯ltering arecarried out on consecutive windows in time-domainbefore computing the Fourier transform. Thesampling rate is 800MHz and each window con-tains 4,096 data samples with 8-bit ¯xed-pointprecision. The processed data has an integrationtime of 1.3ms and 671ms for the beamformer andcorrelator mode, respectively. This setup is com-mon to all the three examples mentioned in thissection.

4.2.1. Tests using antenna signals

The temporal behavior of a single spectral channelwith un¯ltered and ¯ltered data corresponding to636MHz radio frequency is shown in Fig. 5. In thisexperimental setup, two antennas, say, A1 and A2are connected to the input of the GWB system andobserve the astronomical source 3C147. The resultsshow a comparative analysis between un¯ltered and¯ltered data processed simultaneously through theGWB. The ¯rst subplot is the beamformer modeintensity data for ¯ltered (blue) and un¯ltered (red)options. The X-axis shows the time at which thedata was recorded. RFI is ¯ltered at 3� threshold intime-domain and the detected samples are replacedby zero. The negative values of intensity are causedby saturation of beam output indicating the pres-ence of strong RFI. The second subplot shows im-provement in the mean-to-RMS ratio which iscalculated as shown in Sec. 3.3 over 1,024 samples ofthe beamformer data. A maximum of 10 dB im-provement is observed in this case.

The third and fourth subplots are time alignedwith the ¯rst subplot and show amplitude andphase of the cross-correlation for antenna A2 withrespect to antenna A1. Generally, for short base-lines, the cross-correlation amplitude increases inthe event of correlated RFI (shown in red). RFI¯ltering reduces the amplitude of the cross-correla-tion towards its desired values. The phase of thecross-correlation °uctuates in the event of RFI asseen in the fourth subplot. Under normal conditions,the phase for a point source should not have large°uctuations especially when observed for a shortduration. The phase of the ¯ltered signal is main-tained at the same value as it was in the absence ofRFI. The e®ect of replacement with zero (shown inblue) and threshold (shown in green) value areoverlayed in the third and fourth subplots. In thiscase, replacement with zero provides better perfor-mance over replacement by threshold.

Another example of ¯ltering is shown in Fig. 6.In this case, moderate level of RFI is persistent for alonger duration. It is not as strong and distinct as itwas in the previous example. Filtering is carriedout at 3� threshold and replacement options arethreshold (shown in green) and noise (shown inmagenta). Subplot 2 shows uniform improvement inmean-to-RMS ratio by 5 dB. Improvement is alsoobserved in the magnitude and phase of the cross-correlation outputs (subplots 3 and 4).

4.2.2. Tests using RFI emulator

This is a controlled test carried out to probe thebehavior of the RFI ¯ltering technique for a knowninput signal. RFI emulator is an analog instrumentdeveloped at the GMRT to generate RFI with de-sired duration and power level in order to test the¯ltering system. It generates a broadband noise °oorand adds periodic broadband RFI signal or pulsednoise with a speci¯c duty cycle and having averagepower greater than the noise °oor. The duty cycle isprogrammable. The output from the emulator canbe fed as baseband signal to the GWB system. Forthe example shown in Fig. 7, the on period of theRFI is 16�s and total period is 2 s. The tests havebeen carried out with the GWB and RFI ¯lteringcon¯guration similar to those for the antenna sig-nals described earlier in this section. Figure 7 showsthe e®ect of ¯ltering on the signal generated usingemulator. As can be seen in the ¯rst subplot, theun¯ltered time-series shows periodic occurrences ofRFI (pulsed-noise) which are ¯ltered out at 3�

Towards Real-Time Impulsive RFI Mitigation for Radio Telescopes

1641018-7

threshold and replaced by zero. The correspondingimprovement (in dB) is shown in the second sub-plot. Since the output time resolution is 671 mswhich is much larger compared to the ON-time ofRFI, and since the emulator has a low duty cycle,the power gets averaged at the correlator output.Thus, the third and fourth subplots show an

averaged e®ect in the magnitude and phase ofthe cross-correlation. The average value of cross-correlation magnitude increases due to addition ofpulsed-noise (RFI). Filtering at 3� and replacementby zero brings this value to the actual value of cross-correlation magnitude which is equal to that ob-served in absence of RFI.

15:52:50.880 15:52:59.520 15:53:08.160 15:53:16.800 15:53:25.440 15:53:34.080 15:53:42.720 15:53:51.360 15:54:00.000 15:54:08.640 15:54:17.280

-2

0

2

4

Inte

nsi

ty(A

rb. U

nit

s)

×10 4 IA beam timeseries of antenna A1

unfiltered3s filtered (rpbdn)

15:52:50.880 15:52:59.520 15:53:08.160 15:53:16.800 15:53:25.440 15:53:34.080 15:53:42.720 15:53:51.360 15:54:00.000 15:54:08.640 15:54:17.2800

5

10

Imp

rove

men

t(d

B)

Improvement in mean by rms of IA beam data of antenna A1

15:52:50.880 15:52:59.520 15:53:08.160 15:53:16.800 15:53:25.440 15:53:34.080 15:53:42.720 15:53:51.360 15:54:00.000 15:54:08.640 15:54:17.2800

20

40

60

Po

wer

(Arb

. Un

its)

Interferometry Mode Cross Correlation Power of antenna A2 with respect to antenna A1

unfiltered3s filtered (rpbdn)3s filtered (rpbt)

15:52:50.880 15:52:59.520 15:53:08.160 15:53:16.800 15:53:25.440 15:53:34.080 15:53:42.720 15:53:51.360 15:54:00.000 15:54:08.640 15:54:17.280Time(HH:MM:SS.FFF)

-100

0

100

200

Ph

ase

(Deg

rees

)

Interferometry Mode Cross Correlation Phase of antenna A2 with respect to antenna A1

Fig. 6. RFI ¯ltering simultaneously in GWB correlator and beamformer mode outputs for antenna signals.

15:07:46.560 15:07:55.200 15:08:03.840 15:08:12.480 15:08:21.120 15:08:29.760 15:08:38.400 15:08:47.040-5

0

5

Inte

nsi

ty(A

rb. U

nit

s)

×10 4 IA beam timeseries of antenna A1

unfiltered3s filtered (rpl0)

15:07:46.560 15:07:55.200 15:08:03.840 15:08:12.480 15:08:21.120 15:08:29.760 15:08:38.400 15:08:47.0400

5

10

Imp

rove

men

t(d

B)

Improvement in mean by rms of IA beam data of antenna A1

15:07:46.560 15:07:55.200 15:08:03.840 15:08:12.480 15:08:21.120 15:08:29.760 15:08:38.400 15:08:47.0400

100

200

Po

wer

(Arb

. Un

its)

Interferometry Mode Cross Correlation Power of antenna A2 with respect to antenna A1

unfiltered3s filtered (rpl0)3s filtered (rpbt)

15:07:46.560 15:07:55.200 15:08:03.840 15:08:12.480 15:08:21.120 15:08:29.760 15:08:38.400 15:08:47.040Time(HH:MM:SS.FFF)

-200

-100

0

Ph

ase

(Deg

rees

)

Interferometry Mode Cross Correlation Phase of antenna A2 with respect to antenna A1

Fig. 5. RFI ¯ltering in GWB correlator and beamformer mode outputs simultaneously for antenna signals.

K. D. Buch et al.

1641018-8

5. Narrowband RFI Filtering in the GWB

Detection of narrowband RFI in the GWB is carriedout in the post-FFT domain (Fig. 1) using MAD-based estimation (of dispersion) across the spec-trum. Based on the relative strength of the nar-rowband RFI as compared to the total receivernoise, the detection can be carried out either on thereal and imaginary parts of the spectrum or ontemporally integrated spectrum. For example, nar-rowband RFI buried within the system noise can bedetected only after temporal integration of conse-cutive spectra. The ¯ltering threshold is calculatedby computing median and MAD across spectralchannels for each spectrum. As a proof-of-concept,this scheme is implemented on recorded data,however, a real-time implementation is being de-veloped.

5.1. Spectral normalization

If there are large variations in the power level acrossthe frequency band, the median and MAD estima-tors for computing the ¯ltering threshold get af-fected. The estimation is better when there is equalpower in all the spectral channels. Hence, a com-pensation scheme equalizes the power level acrossthe spectral channels before computing the robustthreshold. Of the various schemes for spectral

equalization, we propose a technique where the en-tire band is normalized to unity. This highlights thelocations in the spectrum that are corrupted by RFImaking them easier to identify. The normalizationprocess uses median-based estimator as the antennasignal may have RFI. The steps involved in nor-malizing spectrum are:

(i) For the mth spectral channel, m 21; 2; . . . ;Nch, the median spectrum (Mm) isobtained by taking median (M) of over a rangeof time-stamps T ¼ t0; t1; . . . ; tn. Thus,

Mm ¼ MðSm;T Þ: ð5Þ

(ii) This median spectrum would still containtemporally persistent RFI. Thus, a runningmedian ¯lter is applied across the medianspectrum. This step smoothens the medianspectrum and reduces any spectrally impulsiveRFI present in it. The value of each sample inthe median spectrum is replaced by the medianof the values in the window around that sam-ple. This spectrum can be called smoothedspectrum (MS) as it is impulse free. Thisspectrum can now be used for the normaliza-tion process. For a median ¯lter having awindow of 2wþ 1 spectral channels, the rthchannel of the smoothed median spectrum can

11:04:42.240 11:04:46.560 11:04:50.880 11:04:55.200 11:04:59.520 11:05:03.840 11:05:08.160 11:05:12.480 11:05:16.8000

50

100

150

200

Intensity

(Arb. Units)

IA Beam timeseries of RFI (Pulse Noise + Noise Floor)

11:04:42.240 11:04:46.560 11:04:50.880 11:04:55.200 11:04:59.520 11:05:03.840 11:05:08.160 11:05:12.480 11:05:16.80050

60

70

80

Power

(Arb. Units)

Interferometry Mode Cross Correlation with respect to Noise Floor

11:04:42.240 11:04:46.560 11:04:50.880 11:04:55.200 11:04:59.520 11:05:03.840 11:05:08.160 11:05:12.480 11:05:16.800150

151

152

153

Time(HH:MM:SS.FFF)

Phase

(Degrees)

Interferometry Mode Cross Correlation phase with respect to Noise Floor

11:04:42.240 11:04:46.560 11:04:50.880 11:04:55.200 11:04:59.520 11:05:03.840 11:05:08.160 11:05:12.480 11:05:16.8000

0.5

1

1.5

2

Improvement

(dB)

Improvement in mean by rms of IA beam data of RFI

unfiltered3s filtered (rpl0)

unfiltered3s filtered (rpl0)

unfiltered3s filtered (rpl0)

Fig. 7. RFI ¯ltering simultaneously in GWB correlator and beamformer mode outputs for RFI emulator signals.

Towards Real-Time Impulsive RFI Mitigation for Radio Telescopes

1641018-9

be calculated as

MSðrÞ ¼ MðMSðr� wÞ; . . . ;MSðrÞ; . . . ;MSðrþ wÞÞ: ð6Þ

(iii) Steps (i) and (ii) are used to get a spectrum forthe normalization process even in the presenceof RFI. During the ¯ltering process, medianand MAD estimates are computed for eachindividual spectrum. The un¯ltered spectrumis divided by the smooth spectrum to get nor-malised data. For the ith spectral channel, SðiÞis calculated from incoming spectrum SðiÞ foreach time-stamp t as

Sði; tÞ ¼ Sði; tÞMSðiÞ

: ð7Þ

Filtering is carried out on this spectrum asshown in Sec. 3.1. It has to be ensured in thedesign that the values of MS do not becomezero or vanishingly small.

5.2. Narrowband RFI ¯ltering on recordeddata

Figure 8 shows ¯ltering on the GMRT data a®ectedby narrowband RFI in the L-band (1,100–1,450MHz). MAD-based ¯ltering is carried outacross 2,048 spectral channels and values above 3:5�threshold are replaced by the robust threshold.Normalization of the spectrum has been carried out

before ¯ltering. The median spectrum is computedover 1,024 time-stamps and the smooth spectrumover a window size of 20 spectral channels. Figure 8shows 5,000 s of data in a time–frequency plot (withand without ¯ltering). The °agged data shownbelow the un¯ltered spectrum corresponding to thespectral channels that are detected as RFI andreplaced by the threshold value. In this case, the¯ltered spectrum shows signi¯cant post-¯lteringimprovement (15 dB maximum).

To illustrate the e®ect of normalization ofspectrum on the ¯ltering, the spectrum shown inFig. 8 is ¯ltered with normalization. Figure 9 showsthat the amount of data °agged is lesser as com-pared to that seen in Fig. 8. The amount of nar-rowband RFI detected without normalization islesser due to power variation across the spectrum.This shows that the ability to detect RFI fromspectrum having variations in the power level isa®ected in absence of the normalization process.The e®ect is more pronounced towards the edge ofthe spectrum where there is a roll-o® in the powerlevel.

6. RFI Filtering in the Beamformer Chain ofthe GWB

The beam mode of a telescope is used primarily forpulsar observations. Section 6.1 describes RFI mit-igation for the GWB beamformer data. Sections 6.2

Fig. 8. Narrowband RFI ¯ltering at 3:5� threshold (with smoothing and normalization).

K. D. Buch et al.

1641018-10

and 6.3 describe the real-time implementation andsome test results from GMRT pulsar observations.

6.1. Description

The entire length of beamformer output data is di-vided into blocks. Each block has a few seconds ofdata. The size of the block is chosen based on thetime-scales on which the RFI at GMRT persists. Ithas been observed that the spectral RFI switchesON and OFF on a few seconds time-scale. Thesoftware utility which has been developed performsthe following steps of pulsar signal processing and¯ltering RFI on each block:

(1) Filter narrowband RFI: Search and °ag nar-rowband RFI from a normalized spectrum (asdiscussed in Sec. 5).

(2) Filter broadband impulsive RFI: Collapse allspectral channels and use MAD-based detectionand ¯ltering (Sec. 3.1) to ¯nd (and °ag) outliersfrom this time-series. There is an advantage ofdetecting RFI from this time-series; because fora large number of pulsars, owing to the pulsegetting smeared (due to dispersion in the in-terstellar medium), the signal is buried underthe noise °oor and hence would not be picked upas an outlier.

(3) The time and frequency °ags generated in theabove steps are kept track of, and the data

samples corresponding to the °ags are maskedas the data undergoes the process of de-disper-sion.

(4) The de-dispersed time-series generated in theabove step undergoes a signal processing oper-ation called folding wherein successive blocksare added and averaged to improve the signal-to-noise ratio. The de-dispersed and foldedtime-series is recorded on to a hard disk.

6.2. Implementation

The utility is developed in C++ and can run on anygeneral-purpose, multi-core CPU. It uses OPENMPto manifest parallelism and takes an approach thatensures scalability, and optimum utilization, of theavailable resources. The division of tasks betweenthreads is illustrated in Fig. 10. Tasks are performedfor k blocks at a given point in time. All I/Ooperations for each of the k blocks happen seriallyon a single thread to avoid any bottleneck arisingout of simultaneous disk or memory access. Theother tasks are divided into three teams, each ofwhich perform a certain set of operations on the kblocks, in parallel. Thus the tool uses 2þ 3k numberof threads for the operation, where k can be opti-mally chosen at run-time by the user. With ¯vethreads (k ¼ 1), the utility can perform all of theabove operations, write out the ¯ltered as well asun¯ltered time-series (corrected for dispersion) to

Fig. 9. Narrowband RFI ¯ltering at 3:5� threshold (without smoothing and normalization).

Towards Real-Time Impulsive RFI Mitigation for Radio Telescopes

1641018-11

the disk and fold the time-series in real-time at asampling interval of 40�s with 2,048 spectralchannels.

6.3. Results

The utility has shown improvement in the qualityof pulsar data across various parameters. This isillustrated by two examples of ¯ltering on di®erentpulsars observed from the GMRT. The observa-tion parameters of these pulsars are listed inTable 1.

The parameters used to quantify the improve-ment achieved due to RFI ¯ltering: (1) the ratio ofmean-to-RMS of each block after ¯ltering as com-pared to the un¯ltered value (Eq. (4)) and (2) theratio of the on-pulse signal to the o®-pulse noise inthe folded pro¯le. The ¯ltered and un¯ltered pro-¯les, and the improvement in mean-to-RMS ratiofor B0138+59 in Fig. 11 and that for J1807-0847 areshown in Fig. 12. Filtering improves the pulse sig-nal-to-noise ratio by a factor of 2 for B0138+59pulsar and by a factor of 10 for J1807-0847 pulsar asseen in Table 2.

7. Summary and Discussion

7.1. Summary

Real-time techniques for impulsive RFI mitigationfor broadband and narrowband RFI were proposedalong with results from implementation on GWB.Broadband RFI ¯ltering showed signi¯cant im-provement (10–12 dB) in the signal-to-noise ratioand cross-correlation performance. MAD-basedspectral domain RFI ¯ltering showed up to 15 dBimprovement for spectral channels a®ected by RFI.MAD-based estimation for real-time RFI ¯lteringwas also found to be e®ective in improving the sig-nal-to-noise ratio by a factor of 10 during pulsarobservations carried out using GWB. These real-time RFI mitigation techniques will ¯nd applica-tions in the existing radio telescopes and theupcoming ones like the Square Kilometer Array(SKA).

7.2. Discussion of future work

The information regarding the samples that wereremoved or altered during the RFI ¯ltering opera-tion needs to be conveyed to the user along with the¯nal output of the backend system. In the case ofGWB, the ¯ltering is carried out at various loca-tions. The information of the ¯ltered samples isgenerated at di®erent rates depending on the loca-tion of the RFI ¯ltering block in the signal processingchain. Hence, it is a challenging problem to collatethe information of samples °agged at di®erent ratesfor the correlator and beamformer modes of theGWB. This information needs to pass through the

Table 1. Observation parameters of pulsars during beamfor-mer RFI ¯lter testing.

PulsarRF band(MHz)

Samplinginterval (�s) Channels

Observationduration(min)

J1807-0847 1260–1460 1310.72 2,048 10B0138+59 200–300 81.92 2,048 5

Fig. 10. Division of tasks among various threads in the beamformer RFI mitigation software.

K. D. Buch et al.

1641018-12

1.01

1.015

1.02

1.025

1.03

1.035

1.04

1.045

1.05

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Nor

mal

ized

Inte

nsity

Pulsar Phase

UnfilteredFiltered

0

0.5

1

1.5

2

0 20 40 60 80 100 120 140

Impr

ovem

ent (

dB)

Block Number

Fig. 11. E®ect of RFI ¯ltering on folded pro¯le (subplot 1) and blockwise improvement in mean-to-RMS ratio (subplot 2) for PSRB0138+59.

0.998

0.999

1

1.001

1.002

1.003

1.004

1.005

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Nor

mal

ized

Inte

nsity

Pulsar Phase

UnfilteredFiltered

0

0.5

1

1.5

2

2.5

3

0 50 100 150 200 250 300

Impr

ovem

ent (

dB)

Block Number

Fig. 12. E®ect of RFI ¯ltering on folded pro¯le (subplot 1) and blockwise improvement in mean-to-RMS ratio (subplot 2) for PSRJ1807-0847.

Towards Real-Time Impulsive RFI Mitigation for Radio Telescopes

1641018-13

signal processing chain and come out as a numberindicating the fraction of original data that wasdetected as RFI for a given block and spectralchannel. A technique is being developed for the same.

The characterization of the e®ect of real-timeRFI ¯ltering using the technique shown in the pre-vious sections is carried out through antenna tests.The parameters measured are the e®ect of di®erentthresholds and replacement options on the auto-correlation and cross-correlation spectrum (ampli-tude and phase) of the output. Also, tests are beingcarried out to determine the e®ect of RFI ¯lteringon the astronomical image. These tests will also helpin ¯ne-tuning the parameters of the technique likethe estimation window size, optimum ¯lteringthreshold, and the choice of replacement optionsunder varying RFI conditions.

The algorithm for narrowband RFI ¯ltering cur-rently works on recorded correlator data. This algo-rithm would be optimized and ported on the CPU–GPU cluster of the GWB for real-time narrowbandRFI ¯ltering. To achieve real-time performance, thedata would be read directly from the shared memoryof the compute nodes processing the correlator data.The challenge is to perform this operation in real-timefor 60 inputs (30 antennas, dual polarization) on atime-scale of few hundred milliseconds.

As a next level of RFI mitigation strategy forthe uGMRT, there is a plan to develop techniqueswhich result in minimal loss of astronomical dataand which work e®ectively even for RFI buriedwithin the system noise. The techniques underconsideration are adaptive cancellation using refer-ence antenna, spatial nulling and adaptive spatial¯ltering.

Acknowledgments

The authors would like to thank Sanjay Kudale forhis help in data analysis. The authors would alsolike to thank the members of the backend group andcontrol room at GMRT.

References

Ajithkumar, B., Choudhari, S., Buch, K., Muley, M., Shelton,G., Reddy, S., Kudale, S., Roy, J. & Gupta, Y. [2013] Nextgeneration digital backends for the GMRT, in IOP Conf.Series: Materials Science and Engineering (IOP Publishing),p. 012024.

Astola, J. & Kuosmanen, P. [1997] Fundamentals of NonlinearDigital Filtering, Vol. 8 (CRC Press, Boca Raton, NY).

Buch, K. D., Gupta, Y. & B. A. Kumar, [2014] Variable cor-relation digital noise source on FPGA—A versatile tool fordebugging radio telescope backends J. Astron. Instrum. 3,1450007.

Crnojevic, V., Senk, V. & Trpovski, Z. [2004] Advanced impulsedetection based on pixel-wise MAD, IEEE Signal Process.Lett. 11, 589.

Dewdney, P. E., Hall, P. J., Schilizzi, R. T. & Lazio, T. J. L.[2009] The square kilometre array, Proc. IEEE97, 1482.

Edwards, J. [2014] Signal processing: On the edge of Astron-omy's new frontier special reports, IEEE Signal Process.Mag. 31, 10.

Ellingson, S. W. [2004] RFI mitigation and the SKA, Exp.Astron. 17, 261.

Fahmy, S. A., Cheung, P. Y. & Luk, W. [2009] High-through-put one-dimensional median and weighted median ¯lters onFPGA, IET Comput. Digit. Tech. 3, 384.

Fridman, P. [2008] Statistically stable estimates of variance inradio-astronomy observations as tools for radio-frequencyinterference mitigation, A J 135, 1810.

Fridman, P. & Baan, W. [2001] RFI mitigation in radioastronomy, A & A 378, 327.

Guner, B., Johnson, J. T. & Niamsuwan, N. [2007] Time andfrequency blanking for radio-frequency interference mitiga-tion in microwave radiometry, IEEE Trans. Geosci. RemoteSens. 45, 3672.

ITU-R [2013] Techniques for mitigation of radio frequency in-terference in radio astronomy, RA.2126-1, RA Series, In-ternational Telecommunication Union.

Juhola, M., Katajainen, J. & Raita, T. [1991] Time and fre-quency blanking for radio-frequency interference mitigationin microwave radiometry, IEEE Trans. Signal Process.39, 204.

Langat, P. K. [2011] Power-line sparking noise characterisationin the SKA environment, PhD thesis, Stellenbosch Univer-sity, Stellenbosch, South Africa.

Niamsuwan, N., Johnson, J. T. & Ellingson, S. W. [2004] Ex-amination of a simple pulse blanking technique for RFImitigation, Radio Sci 40 RS5S03.

Pearson, R. K., Neuvo, Y., Astola, J. & Gabbouj, M. [2016]Generalized Hampel ¯lters, EURASIP J. Adv. Signal Pro-cess., 2016(1), 87.

Rousseeuw, P. J. & Croux, C. [1993] Alternatives to medianasbolute deviation, J. Am. Stat. Assoc. 88, 424.

Swarup, G. [2008] Power-line radio frequency interference atthe GMRT, NCRA-TIFR Technical Report.

Swarup, G., Ananthakrishnan, S., Kapahi, V., Rao, A.,Subrahmanya, C. & Kulkarni, V. [1991] The giant metre-wave radio telescope, Curr. Sci. 60, 95.

Shankar, N. U. & Pandey, V. [2006] Hierarchical RFI mitiga-tion system at the Mauritius Radio Telescope, in IAU Spe-cial Session 6, 22 August 2006, Prague, Czech Republic.

Table 2. Improvement in pulse signal-to-noiseratio (Arb. units) due to RFI ¯ltering.

Pulse signal to o®-pulse noise

Un¯ltered Filtered

J1807-0847 25 116B0138+59 376 647

K. D. Buch et al.

1641018-14