Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations...

63
Article overview by Ilya Kuzovkin Umut Güclü and Marcel A. J. van Gerven Computational Neuroscience Seminar University of Tartu 2015 Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Transcript of Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations...

Page 1: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Article overview by Ilya Kuzovkin

Umut Güclü and Marcel A. J. van Gerven

Computational Neuroscience Seminar University of Tartu

2015

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 2: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 3: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

…

pixels

classes

Linear

“spider”

“cat”

Page 4: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

…

pixels

classes

… hidden layer

Non-linear

“cat”

“spider”

Page 5: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

…

pixels

classes

… hidden layer… hidden layer

Deep

“cat”

“spider”

Page 6: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

“spider”

“cat”important

feature

Page 7: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

“spider”

important feature

RUN!

“cat”

Page 8: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

“spider”

important feature

RUN! Convolutional filter

“cat”

Page 9: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Convolutional (and pooling) layer

Page 10: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

…

pixels

classes

… hidden layer… hidden layer

… convolutional layer

Deep Convolutional Neural Network

“cat”

“spider”

Page 11: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 12: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 13: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 14: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Matthew D. Zeiler, Rob Fergus Visualizing and Understanding Convolutional Networks 2013

Page 15: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Matthew D. Zeiler, Rob Fergus Visualizing and Understanding Convolutional Networks 2013

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 16: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Matthew D. Zeiler, Rob Fergus Visualizing and Understanding Convolutional Networks 2013

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 17: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Matthew D. Zeiler, Rob Fergus Visualizing and Understanding Convolutional Networks 2013

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 18: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Matthew D. Zeiler, Rob Fergus Visualizing and Understanding Convolutional Networks 2013

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 19: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 20: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Two-stream hypothesis

Page 21: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 22: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 23: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 24: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 25: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 26: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 27: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

?

Page 28: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 29: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 30: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream
Page 31: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream
Page 32: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

96 x 37x 37 = 131,424

Page 33: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

96 x 37x 37 = 131,424

Page 34: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

96 x 37x 37 = 131,424256 x 17x 17 = 73,984

Page 35: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

96 x 37x 37 = 131,424256 x 17x 17 = 73,984

Page 36: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

96 x 37x 37 = 131,424256 x 17x 17 = 73,984

Page 37: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

96 x 37x 37 = 131,424256 x 17x 17 = 73,984

Page 38: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

96 x 37x 37 = 131,424256 x 17x 17 = 73,984

Train linear regression

model

Page 39: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

96 x 37x 37 = 131,424256 x 17x 17 = 73,984

Train linear regression

model

Test it

Page 40: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

96 x 37x 37 = 131,424256 x 17x 17 = 73,984

Train linear regression

model

Test it r = 0.22

Page 41: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

96 x 37x 37 = 131,424256 x 17x 17 = 73,984

Train linear regression

model

Test it r = 0.22

Train linear regression

model

Test it

Page 42: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

96 x 37x 37 = 131,424256 x 17x 17 = 73,984

Train linear regression

model

Test it r = 0.22

Train linear regression

model

Test it r = 0.67

Page 43: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

96 x 37x 37 = 131,424256 x 17x 17 = 73,984

Train linear regression

model

Test it r = 0.22

Train linear regression

model

Test it r = 0.67

Page 44: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Page 45: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

NEXT COOL THING: CATEGORIES OF FEATURES

…ImageNet validation set

Page 46: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

NEXT COOL THING: CATEGORIES OF FEATURES

…ImageNet validation set

... . 1888

Page 47: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

NEXT COOL THING: CATEGORIES OF FEATURES

…ImageNet validation set

... . 1888

Page 48: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

NEXT COOL THING: CATEGORIES OF FEATURES

…ImageNet validation set

... ..

1888

Page 49: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

NEXT COOL THING: CATEGORIES OF FEATURES

…ImageNet validation set

... ..

1888

deconvolution.

Page 50: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

NEXT COOL THING: CATEGORIES OF FEATURES

…ImageNet validation set

... ..

1888

deconvolution.Low Mid High• blob • contrast • edge

• contour • shape • texture

• pattern • object • object part

human-assigned to 9 categories

Page 51: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

NEXT COOL THING: CATEGORIES OF FEATURES

…ImageNet validation set

... ..

1888

deconvolution.Low Mid High• blob • contrast • edge

• contour • shape • texture

• pattern • object • object part

human-assigned to 9 categories

1. Divide 1888 neurons into 9 categories

Page 52: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

NEXT COOL THING: CATEGORIES OF FEATURES

…ImageNet validation set

... ..

1888

deconvolution.Low Mid High• blob • contrast • edge

• contour • shape • texture

• pattern • object • object part

human-assigned to 9 categories

1. Divide 1888 neurons into 9 categories

!2. Predict activity of each voxel

from group-by-group

Page 53: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

NEXT COOL THING: CATEGORIES OF FEATURES

…ImageNet validation set

... ..

1888

deconvolution.Low Mid High• blob • contrast • edge

• contour • shape • texture

• pattern • object • object part

human-assigned to 9 categories

1. Divide 1888 neurons into 9 categories

!2. Predict activity of each voxel

from group-by-group !3. For each voxel find the

group, which best predicts voxel’s activity

Page 54: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

NEXT COOL THING: CATEGORIES OF FEATURES

…ImageNet validation set

... ..

1888

deconvolution.Low Mid High• blob • contrast • edge

• contour • shape • texture

• pattern • object • object part

human-assigned to 9 categories

1. Divide 1888 neurons into 9 categories

!2. Predict activity of each voxel

from group-by-group !3. For each voxel find the

group, which best predicts voxel’s activity

!4. Assign each of 1888 DNN

neurons to a visual layer: V1, V2, V4, LO

Page 55: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

NEXT COOL THING: CATEGORIES OF FEATURES

…ImageNet validation set

... ..

1888

deconvolution.Low Mid High• blob • contrast • edge

• contour • shape • texture

• pattern • object • object part

human-assigned to 9 categories

1. Divide 1888 neurons into 9 categories

!2. Predict activity of each voxel

from group-by-group !3. For each voxel find the

group, which best predicts voxel’s activity

!4. Assign each of 1888 DNN

neurons to a visual layer: V1, V2, V4, LO

!5. Map visual layers to

categories

Page 56: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

NEXT COOL THING: CATEGORIES OF FEATURES

Page 57: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

OTHER RESULTS

Correlation between predicted responses between pairs of voxel groups

Page 58: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

OTHER RESULTS

Selectivity of visual areas to feature maps of varying complexity

Page 59: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

OTHER RESULTS

Distribution of the receptive field centers

Page 60: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

OTHER RESULTS

Biclustering of voxels and feature maps

Page 61: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

SUMMARY

Page 62: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

An intracranial dataset we have. How to repeat the result?

Page 63: Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

An intracranial dataset we have. How to repeat the result?

vs.


Category-selective patterns of neural response in the ventral visual ...ta505/CogganLiuBakerAndrewsNeuroimage201… · Category-selective patterns of neural response in the ventral

Category-selective patterns of neural response in the ventral visual ...ta505/CogganLiuBakerAndrewsNeuroimage201… · Category-selective patterns of neural response in the ventral

Ventral hernia

Ventral hernia

Anatomía del canal vertebral y la médula espinalespinal cordón lateral cordón ventral fisura mediana ventral cuerno ventral cuerno intermedio raíz ventral tronco simpático ramos

Anatomía del canal vertebral y la médula espinalespinal cordón lateral cordón ventral fisura mediana ventral cuerno ventral cuerno intermedio raíz ventral tronco simpático ramos

Human Pluripotent Stem Cell-Derived Neural Cells and Brain ... · 7/28/2020  · 1 Human Pluripotent Stem Cell-Derived Neural Cells and Brain Organoids Reveal SARS-CoV-2 Neurotropism

Human Pluripotent Stem Cell-Derived Neural Cells and Brain ... · 7/28/2020  · 1 Human Pluripotent Stem Cell-Derived Neural Cells and Brain Organoids Reveal SARS-CoV-2 Neurotropism

Metamers of neural networks reveal divergence from human ......Previous neural network visualizations have used gradient descent on the input signals to visualize the representations

Metamers of neural networks reveal divergence from human ......Previous neural network visualizations have used gradient descent on the input signals to visualize the representations

NEURAL CONTROL OF BREATHING MEDULLA PONS. Neural Control Of Breathing Voluntary Cerebral cortex Autonomic Medullary Centers Dorsal Respiratory Group Ventral.

NEURAL CONTROL OF BREATHING MEDULLA PONS. Neural Control Of Breathing Voluntary Cerebral cortex Autonomic Medullary Centers Dorsal Respiratory Group Ventral.

The neural tube origin of ventral root sheath cells in …...Origin of ventral root sheath cells 249. •» Fig. 1. A composite of transverse semithin sections through the caudal (A)

The neural tube origin of ventral root sheath cells in …...Origin of ventral root sheath cells 249. •» Fig. 1. A composite of transverse semithin sections through the caudal (A)

Ventral stream

Ventral stream

DEVELOPMENTAL BIOLOGY Copyright © 2019 Dorsal-ventral ... · DEVELOPMENTAL BIOLOGY Dorsal-ventral patterned neural cyst from human pluripotent stem cells in a neurogenic niche Y.

DEVELOPMENTAL BIOLOGY Copyright © 2019 Dorsal-ventral ... · DEVELOPMENTAL BIOLOGY Dorsal-ventral patterned neural cyst from human pluripotent stem cells in a neurogenic niche Y.

Making sense out of spinal cord somatosensory development · in the dorsal neural tube, and the remaining five (V0-V3 and MN) are found in the ventral neural tube (Fig. 2). In addition,

Making sense out of spinal cord somatosensory development · in the dorsal neural tube, and the remaining five (V0-V3 and MN) are found in the ventral neural tube (Fig. 2). In addition,

Individual Differences in Regulatory Focus Predict Neural ...jamiehanson.org/files/2016_SocialNeuroscience_Scult.pdf · oriented an individual was, the lower their ventral striatum

Individual Differences in Regulatory Focus Predict Neural ...jamiehanson.org/files/2016_SocialNeuroscience_Scult.pdf · oriented an individual was, the lower their ventral striatum

Decreased demands on cognitive control reveal the neural ......Decreased demands on cognitive control reveal the neural processing benefits of forgetting Brice A Kuhl1, Nicole M Dudukovic1,

Decreased demands on cognitive control reveal the neural ......Decreased demands on cognitive control reveal the neural processing benefits of forgetting Brice A Kuhl1, Nicole M Dudukovic1,

The neural tube origin of ventral root sheath cells in the ...

The neural tube origin of ventral root sheath cells in the ...

Ventral abdominal hernia1

Ventral abdominal hernia1

A neural signature of the unique huessro.sussex.ac.uk/id/eprint/66363/3/A neural signature of...The current study aimed to reveal a neural representation of the unique hues by measuring

A neural signature of the unique huessro.sussex.ac.uk/id/eprint/66363/3/A neural signature of...The current study aimed to reveal a neural representation of the unique hues by measuring

Within- and cross-participant classifiers reveal different neural … · 2013. 2. 12. · Within- and cross-participant classifiers reveal different neural coding of information

Within- and cross-participant classifiers reveal different neural … · 2013. 2. 12. · Within- and cross-participant classifiers reveal different neural coding of information

24. Hernia Ventral

24. Hernia Ventral

2005class 7b 8 - Massachusetts Institute of Technology · neural crest, dorsal and ventral ... G. E. Schneider 2005 ... • Parasympathetic innervation (cranio-sacral system); dual

2005class 7b 8 - Massachusetts Institute of Technology · neural crest, dorsal and ventral ... G. E. Schneider 2005 ... • Parasympathetic innervation (cranio-sacral system); dual

Retinogeniculate visual pathway - Center for Neural Sciencedavid/courses/perceptionAdvanced/lectureSlides/… · Retinogeniculate visual pathway Ventral view optic nerve optic chiasm

Retinogeniculate visual pathway - Center for Neural Sciencedavid/courses/perceptionAdvanced/lectureSlides/… · Retinogeniculate visual pathway Ventral view optic nerve optic chiasm

Robust Hi-C Maps of Enhancer-Promoter Interactions Reveal ... · Article Robust Hi-C Maps of Enhancer-Promoter Interactions Reveal the Function of Non-coding Genome in Neural Development

Robust Hi-C Maps of Enhancer-Promoter Interactions Reveal ... · Article Robust Hi-C Maps of Enhancer-Promoter Interactions Reveal the Function of Non-coding Genome in Neural Development