Electrosensory data acquisition and signal processing strategies

in electric fish

Mark E. Nelson

Beckman InstituteUniv. of Illinois, Urbana-Champaign

How Electric Fish Work

Distribution of Electric Fish

Fish tank upstairs

blackghost

knifefish

elephant-nosefish

Electric Organ Discharge (EOD) - Spatial

EOD - Temporal

Electric Organ Discharge (EOD)

Principle of active electrolocation

mech

an

o

MacIver, fromCarr et al., 1982

Electroreceptors

~15,000 tuberous electroreceptor organs1 nerve fiber per electroreceptor organ

up to 1000 spikes/s per nerve fiber

Individual Sensors (Electroreceptors)

VIN

nerve spikesOUT

Neural coding inelectrosensory afferent fibers

Probability coding(P-type) afferent spike trains

00010101100101010011001010000101001010

Phead = 0.333

Phead = 0.337 Phead =

0.333

Principle of active electrolocation

Electrosensory Image Formation

Electrosensory Image Formation

Electrosensory Image Formation

Prey-capture video analysis

Prey capture behavior

Fish Body Model

Motion capture softwareMotion capturesoftware

MOVIE: prey capture behavior

Electrosensory Image Reconstruction

Voltage perturbation at skin :

Estimating Daphnia signal strength

waterprey

waterpreyfish ar

rE

/21

/133

electrical contrastprey volume

fish E-field at prey

distance from prey to receptor

THIS FORMULA CAN BE USED TO COMPUTE THE SIGNAL AT EVERY POINT ON THE BODY

SURFACE

MOVIE: Electrosensory Images

System Capabilities

Electric fish can analyze electrosensory images to extract information on target

direction (bearing) distance size shape composition (impedance)

Distance Discrimination

Distance Discrimination

Shape Discrimination

Shape Discrimination

Shape Generalization

Shape “completion”

Impedance Discrimination

How Do They Do It? Electric fish analyze dynamic 2D electrosensory images on the body surface to determine target direction, distance, size, shape and

composition (impedance)

Fish might perform an inverse mapping from 2D sensor data to obtain a dense 3D neural representation of world conductivity sensor data 3D conductivity action

Alternatively, fish might use sensor data to directly estimate target parameters sensor data target parameters action

Parameter estimation

(bearing)

Parameter Estimation (cont.)

Dynamic Movement Strategies

Fish are constantly in motion not a single, static ‘snapshot’ dynamic, spatiotemporal data stream

With respect to target objects in the environment, fish body movements simultaneously influence the relative positioning of the sensor array the electric organ effector organs (e.g. mouth)

MOVIE: Electrosensory Images

Active motor strategies: Dorsal roll toward prey

Probing Motor Acts

chin probing back-and-forth (va et vient )

lateral probing

tangentialprobing

stationaryprobing

Fish exploring a 4 cm cube

CNS Signal Processing Strategies

Multi-scale filtering spatial and temporal

Adaptive background subtraction tail-bend suppression

Attentional ‘spotlight’ mechanisms local gain control

Multiple Maps

Multi-scale Filtering

INPUT

(from skin receptors)

Centromedial map High spatial acuity Low temporal acuity

Centrolateral map Inter spatial acuityInter temporal acuity

Lateral map Low spatial acuityHigh temporal acuity

tem

pora

l

inte

grat

ion

bothspatial

integration

HINDBRAIN PROCESSING

PERIPHERALSENSORS

Adaptive Background Subtraction

Adaptive Background Subtraction

Attentional ‘spotlight’ mechanism

Summary Fish can evaluate direction, distance, size, shape and composition of target objects

How? model-based parameter estimation based on 2D image

analysis, not full 3D reconstruction presumably some sort of (adaptive) (extended)

(unscented) Kalman-like algorithm extensive pre-filtering (virtual sensors?)

self-calibrating, adaptive noise suppression, multi-scale spatial and temporal signal averaging

dynamic control of source and array position

Acknowledgements

Colleagues Curtis Bell (OHSU) Len Maler (Univ. Ottawa) Gerhard von der Emde (Univ. Bonn)

Nelson Lab Members Ling Chen, Rüdiger Krahe, Malcolm MacIver

Funding Agencies NIMH, NSF