DARPA Sensor Information Technology (SenseIT) Program Steve BeckJoe Reynolds

Marconi Sensor Agent Processing Software (SAPS)

AARD - For Official Use OnlyOct 7, 1999 1

DARPASensor Information Technology

(SenseIT)Program

Steve Beck Joe Reynolds512-929-2034 512-929-2541

[email protected] [email protected]

Analysis and Applied Research Division6500 Tracor Lane, MS.1-8

Austin TX 78725



Presentation Outline

1. SenseIT Problem

2. Focus of Marconi’s Work

3. Detection and Ground Truth.

4. Experimental Data Collection

5. Data Analysis

6. WinCE Processor Status.



SenseIT Detection Problem

• Target detection in an unconstrained surveillance environment.

• System use/reuse during a changing tactical picture.

• Provide the eyes, ears, and communication capability of human sentries.

• Build a bigger picture than the traditional “I got one” detection system.

- Query -Have You Heard

Any Vehicles?- Sentry 2 -

No, But I HeardTroops at Time T2

- Sentry 1 -No, But I Heard

Troops at Time T1- Reasoning -

We have no troops in this area.We know the locs of Sentries 1 &

2.Detection Time T2 > Time T1.

- Therefore –=>Enemy troop movement

in Direction D and At Velocity V.



Military Critical Technologies: Passive Sensor Detection

Military Use of Passive Sensors• Intruder detection.• Target vehicle detection and localization.• Direct fire weapons detection and localization.

Performance Objectives for Military Passive Sensors• 98% correct detection of target from background.• Localize weapons within 10 meters out of 5000 meters range.• Operate in conditions close to weapons fire and explosions.

Requirements for Superior Passive Sensor Detection• Signal Processing.• Weighting and selection of clues.• Validated decision criteria.• Performance in cluttered background.



Marconi’s Main Objective• Provide target detection software that runs on a distributed network of nodes.

• Provide superior Pdet with low Pfa in an operational scenario.

• Demonstrate the effectiveness of collaborative signal processing.

• Collect data that is representative of the scenario, the targets, and the environment.

• Optimize the processing algorithms for targets of interest.

• Implement agents for multiple task requests and dynamic reconfiguration.

• Optimize the detection performance using networked collaboration.

• Optimize the software for the hardware platform.

Marconi’s Top Level Approach



Network Signal Processing

BeamformingSource LocalizationBackground Cancellation

Multi-Sensor Info Fusion.

Multi-Node Info Fusion

Reasoning on the spatialand temporal grid.

Build up a tactical picture.

Respond with appropriatedecisions based on the global picture.

Signal Processing

Detection

Reasoning

Imp

rove

d D

etec

tion

Detection Performance Improves with• Increased Node Collaboration and• Increased Information Content

Information Content



Detection and Ground Truth

• Robust detection requires generalized models for all events (targets and non-targets).

• The models must reflect the realistic recording environment, including

sensor characteristics, background noise, and digitizer characteristics.

• Detection models are constructed from actual data.

• The data must have accurate ground truth.

Data Collection for Detection Experiments

• Use multiple channels with time synchronization.

• Make recordings with high bandwidth and high fidelity.

• Video tape the experiment for ground truth and demonstration.



Data Collection Experiments

Marconi performed three data collection experiments for detection studies:1. Human occupancy detection in a high noise urban environment.2. Tracked and wheeled vehicle detection in a rural environment.3. Explosion (balloon pop) detection and localization in indoors and outdoors.

Preliminary Data Collection Equipment

Audio Recording - HP3964 4-channel analog instrumentation recorder usingdirect mode, speed=3.75 ips, bw=16 KHz.Ithaco amplifiers (4) with individual gain setting.

Audio Digitizer - PC-based II PC32 4-channel simultaneous, 16 KHz/channel, 16-bits.Video Recording - Sony 8-mm CamcorderVideo Digitizer - Mac AV with Fusion Recorder



Preliminary Data Collection - Experiment 1

Test DescriptionScenario: Urban, indoors and outdoors.Targets: Balloon pops.

Doors opening and closing.People walking by.

Clutter: Highway noise.

Sensor DescriptionSensor 1 Dipole mic, outdoors, towards walkway.Sensor 2 Dipole mic, outdoors, toward highway.Sensor 3 Omni mic, outdoors.Sensor 4 Omni mic, indoors.

RecordersAudio Data 4-Channel analog recorder for Sensors 1-4

providing simultaneous channel recording.Video Sony camcorder

4

Omni Mic

2 1

3

Dipole Mics

Omni Mic

4-Channel AnalogRecorder withVariable Gain

Camcorder

Door

Balloons

Hig

hway

Sou

nds

(Indoors)

(Outdoors)

(Hallway)

(Walkway)



4

Omni Mic

2 1

3

Dipole Mics

Omni Mic


Camcorder

Door

Balloons

Hig

hway

Sou

nds

(Indoors)

(Outdoors)

(Hallway)

(Walkway)

Video #1 with Mics 1&4 Sound

Video #2 with Mics 1&4 Sound

Experiment 1 - Audio Visual



Experimental Data Collection 1 – Male Voice Time Stereo

0 5 10 15 20 25 30-0.5

0

0.5File=Brian114 Channel=1

Am

plit

ude

0 5 10 15 20 25 30-1

-0.5

0

0.5

1File=Brian114 Channel=4

Time in Seconds

Am

plit

ude



Experimental Data Collection 1 – Male Voice Detection Analysis

0.1 0.2 0.5 1 2 5 10 20 40

0.1

0.2

0.5

1

2

5

10

20

40

False Alarm probability (in %)

Mis

s p

rob

ab

ility

(in

%)

DET for Data Collection #1, Test=Brian1d8k

Sensor=1, BW =8000Sensor=2, BW =8000Sensor=3, BW =8000Sensor=4, BW =8000

Energy Detection, Bandwidth=8000 Hz.

Chan 1, EER=31%Chan 2, EER=45%Chan 3, EER=39%Chan 4, EER=20%

0.1 0.2 0.5 1 2 5 10 20 40

0.1

0.2

0.5

1

2

5

10

20

40


Mis

s p

rob

ab

ility

(in

%)

DET for Data Collection #1, Test=Brian1dp5k

Sensor=1, BW = 500Sensor=2, BW = 500Sensor=3, BW = 500Sensor=4, BW = 500

Energy Detection, Bandwidth=500 Hz.

Chan 1, EER=52%Chan 2, EER=55%Chan 3, EER=58%Chan 4, EER=23%

Simple Energy DetectionNo discrimination processingEach det based on 1 sec framesNo sequential det or fusion.



-2000

0

2000File=Brian1d8k Chan=1 EER=29 Thresh=-212.8603

-5000

0


-1000

0


0 10 20 30 40 50 60-5000

0

5000File=Brian1d8k Chan=4 EER=20 Thresh=0

Experimental Data Collection 1 - Male, Energy PlotsThe BLUE line is the average energy level for defined for each second.The RED line is the EER threshold.The YELLOW area is the time during Target Present for that sensor.The RED area is False Alarm times.The GREEN area is Missed Detection times.

* Red and Green shaded areas that appear on the wrong side of the line are boundary plotting problems, not mistakes.



Preliminary Data Collection - Experiment 2

Test DescriptionScenario: Rural, outdoors.Targets: Balloon pops.

APC start-up, idle, drive-by.HUMV start-up, idle, drive-by.

Clutter: Generator, highway noise.

Sensor DescriptionSensor 1 Dipole mic, outdoors, pointed south.Sensor 2 Dipole mic, outdoors, pointed north.Sensor 3 Omni mic, co-located with dipole mics.Sensor 4 Omni mic, 20 meters north of dipole mics.

RecordersAudio Data 4-Channel analog recorder for Sensors 1-4

providing simultaneous channel recording.Video Sony camcorder

4

Omni Mic

2

13

Omni & Dipole Mics


Camcorder

APC&

HUMV

Generator

Balloons

Balloons

Dirt Track



4

Omni Mic

2

13

Omni & Dipole Mics


Camcorder

APC&

HUMV

Generator

Balloons

Balloons

Dirt Track

Video with Camcorder Sound

Video with Mics 1&4 Sound

Experiment 2 - Audio Visual



0.1 0.2 0.5 1 2 5 10 20 40

0.1

0.2

0.5

1

2

5

10

20

40


Mis

s pr

obab

ility

(in

%)

DET for Data Collection #1, Test=HumVn8k

Sensor=1, BW=8000Sensor=2, BW=8000Sensor=3, BW=8000Sensor=4, BW=8000

Chan 1, EER=2.9%Chan 2, EER=7.0%Chan 3, EER=6.7%Chan 4, EER=6.4%

Chan 1, EER=8.6%Chan 2, EER=7.0%Chan 3, EER=6.2%Chan 4, EER=6.5%

Simple Energy DetectionNo discrimination processingEach det based on 1 sec framesNo sequential det or fusion.

Experimental Data Collection 2 – Vehicle Detection Analysis

0.1 0.2 0.5 1 2 5 10 20 40

0.1

0.2

0.5

1

2

5

10

20

40


Mis

s pr

obab

ility

(in

%)

DET for Data Collection #1, Test=HumVn500

Sensor=1, BW= 500Sensor=2, BW= 500Sensor=3, BW= 500Sensor=4, BW= 500



Detection Signal Processing - Set 1

Detection Signal Processing - Set 2

FFTTime Series

AdaptiveNormalizer

EnergyDetect Decision

Timer

Threshold

Event

No Event

Multi-resNormalization

Time Series

SignalSpecific

Processing

Detector-Classifier Decision

Timer

Threshold

Event

No Event



PreProcessorwith AdaptNormalizer

Time Series

Real-ValuedFeature Set

ProbDetector/Classifier

Proposed Detection Signal Processing

Discrete-ValFeature Set

ProbDetector/Classifier

LRT/Fusion

&Reasoning

Event/No Event

Threshold

Preprocessor Processor

IPAgent

Alternate ProcessUpdate Models

DataBaseSP

Agent

Alternate ProcessUpdate Models

Decision

Scenario/TacticalInformation



Bayesian Decision Criteria

ThresholdHP

HP

H

H

HzP

HzPz

|

|

1

0

0

1

0

1

Binary Hypothesis Testing

True Hypothesis Decision Description

H0 D0 Correct Decision

H0 D1 Type I Error (False Alarm)

H1 D1 Correct Decision

H1 D0 Type II Error (Miss)

11011

00100

CCHP

CCHP

z

Modified Threshold

New Likelihood RatioDecision Rule

C00 is the cost of making a correct null hypothesis decision.C11 is the cost of making a correct target present decision.C10 is the cost of making a Type I error (FA).C01 is the cost of making a Type II error (miss).



Summary and Status

• Initial Data Collection

• Preliminary Algorithm Development

• WinCE Platform Running

• Preliminary Design of Software Infrastructure

DARPA Sensor Information Technology (SenseIT) Program Steve BeckJoe Reynolds

Documents

Transcript of DARPA Sensor Information Technology (SenseIT) Program Steve BeckJoe Reynolds