Extension of Superior Colliculus Based Models of Multisensory Integration to

the Cortex

Michael Hadley

Advisor: Dr. Elaine ReynoldsReaders: Dr. Chun Wai Liew, Dr. Jeffrey Pfaffmann, Dr. Luis Schettino

OutlineBackgroundSOMMMA ModelMy ExtensionsConclusion

The Multisensory WorldWindows to the worldInformation is informationImplication for learning, sensory loss,

sensory substitution

http://www.medical-look.com/systems_images/Sensory_organs.gif

http://media.photobucket.com/image/the%20senses/Kassiah/WotD/senses.jpg

Literature

Superior Colliculus (SC)

Cerebral Cortex

Ascending and Descending Connections

Multisensory Enhancement

Inverse Effectiveness

System Level Explanation

Developmental Explanation

Sigmoidal Activation

Multisensory Depression

Inhibition

Time Dynamics

Multiple Receptive Fields

Chemical Explanation

Different Thresholds

More than Three Senses

Behavioral Effects

ModelingUrsino, Cuppini, Magosso, Serino and Pellegrino

(2009)

Anastasio and Patton (2003)

Martin, Meredith and Ahmad (2003)

OutlineBackgroundSOMMMA ModelMy ExtensionsConclusion

Self-Organizing MapsTopographicalTrainedVector Form A = ( 1,0,0,0,0 ) B = ( 2,0,0,0,0 ) 3 = ( 3,3,6,2,3 )

SOMs: How they Work

A1 A2 A3 A4 A5

2 0 0 0 0B

OutlineBackgroundSOMMMA ModelMy ExtensionsConclusion

Martin, Meredith and Ahmad

Eyes Ears Skin

MMA Design

1

2

3

Inputs ( 0, 0, 0 )( 0, 0, 1 )( 0, 1, 0 )( 1, 0, 0 )( 0, 1, 1 )( 1, 0, 1 )( 1, 1, 0 )( 1, 1, 1 )

Eyes

EarsSkin

Experiments On MMAParameters:

SizeInputsActive and Noisy LevelsProbability Functions

Conclusions:Sensitive integrationSignal-to-noise ratio balance

OutlineBackgroundSOMMMA ModelMy ExtensionsConclusion

My Work

My Extensions

More Complex Senses

Inhibition

Overlapping RFs

Complex Senses

Original Model My Extensions

2 x 2

Within-Sense Enhancement

4 x 4

Conclusions1. Works for small cases

2. Larger cases are a problem

3. Both cases still have a problem

Inhibition

Conclusions1. Rudimentary

2. Signal to noise ratio

Overlapping Receptive Fields

Overlapping Receptive FieldsxPos yPos Bimodal MSE

No Overlapping RFs

Overlapping RFs

ResultsSOMs only handle 1 relation

12

3EyesEars

Skin1

2

EyesEars Skin

1

1 12

2 2

3 3

3 3

MMA is Not ExtendableWithin-modality integration

Requires inhibition

SOM can only handle 1 relation

OutlineBackgroundSOMMMA ModelMy ExtensionsConclusion

Theoretical Model

Unisensory Layer 2

Unisensory Layer 1

Multisensory Layer

Areas in the multisensory layer are SOMs

Why?Increases explanatory power

Foundation