Perspectives on Applications of a Stochastic Spiking Neuron Model to Neural Network Modeling
Transcript of Perspectives on Applications of a Stochastic Spiking Neuron Model to Neural Network Modeling
Perspec'vesonApplica'onsofaStochas'cSpikingNeuronModelto
NeuralNetworkModelingAntonioC.Roque
USP,RibeirãoPreto,SP,BrazilJointworkwithLudmilaBrochini1,AriadneCosta3,
ViníciusCordeiro2,RenanShimoura2,MiguelAbadi1,OsameKinouchi2andJorgeStolfi3
1USP,SãoPaulo;2USP,RibeirãoPreto;3Unicamp,Campinas
PNLD2016,Berlin
Whystochas'cneuronmodels?
• Invivoandinvitrorecordingsofsingleneuronspiketrainsarecharacterizedbyahighdegreeofvariability
• ThefollowingexamplesaretakenfromthebookbyGerstner,Kistler,NaudandPaninski,NeuronalDynamics,CUP,2014
Awakemouse,cortex,freelywhisking
Crochetetal.,2011
Spontaneousac'vityinvivo
Trialtotrialvariabilityinvivo15repe''onsofthesamerandomdotmo'onpa\ern
AdaptedfromBairandKoch,1996;DatafromNewsome,1989
Trialtotrialvariabilityinvitro
4repe''onsofthesame'me-dependents'mulus
ModifiedfromNaudandGerstner,2012
Sourcesofnoise:extrinsicandintrinsictoneurons
Lindner,2016
Twotypesofnoisemodelforaneuron
• Spikegenera'onisdirectlymodeledasastochas'cprocess
• Spikegenera'onismodeleddeterminis'callyandnoiseentersthedynamicsviaaddi'onalstochas'cterms
Stochas'cmodelforsystemsofinterac'ngneurons
Thestochas'cmodel
• Vi(t): time dependent membrane potential of neuron i at time t for i = 1, …, N; • t: discrete time given by integer multiples of constant step Δ small enough to
exclude possibility of a neuron firing more than once during each step; • Xi(t): number of times neuron i fired between t and t+1, namely 0 or 1; • If neuron fires between t and t+1, its potential drops to VR by time t+1; • wij: weight of synapse from neuron j to neuron i; • µ: decay factor (in the interval [0, 1]) due to leakage during time step Δ; • Xi(t) = 1 with probability Φ(Vi(t)); • Φ(V) is assumed to be monotonically increasing and saturating at some
saturation potential VS.
Comment• IfΦ(V) = Θ(V−Vth),i.e.0 for V<Vth and1for
V>Vth,themodelbecomesthedeterminis'cdiscrete-'meleakyintegrate-and-firemodel(LIF).
• AnyotherchoiceofΦ(V) givesastochas'cneuron
Vs
Inthefollowing,Iwillshowsomeanaly'calandnumericalresultsof
networkmodelsusingthisstochas'cneuronmodel
Networkwithall-to-allcouplingMeanfieldanalysis
Analy'calandnumericalresults
Macroscopicquan''es• Poten'aldistribu'on: frac'onofneuronswithpoten'alin therange(V,V+dV)at'met
• Networkac'vity: frac'onofneuronsthatfiredbetweentand t+1
Shapeofthepoten'aldistribu'onP(V,t)hasacomponentthatisaDiracpulseatV=VRwithamplitude,accoun'ngfortheneuronsthatfiredbetweentandt+1
Mean-fieldanalysis
• Fullyconnectednetwork:eachneuronreceivesinputsfromallotherN−1neurons;
• VR=0;• Uniformconstantexternalinput:Ii(t)=I;• Allweightsareequal:
Themean-fieldpoten'aldistribu'on• Onceallneuronshavefiredatleastonce,thedensityP(V,t)becomesacombina'onofdiscreteimpulseswithamplitudesη0(t),η1(t),η2(t),…,atpoten'alsU0(t),U1(t),U2(t),...,suchthat.
• Thevaluesofηk(t)andUk(t)evolvebytheequa'ons:
• Theamplitudeisthefrac'onofneuronswith“age”k:neuronsthatfiredbetweent – k – 1andt – k anddidnotfirebetweent – k andt
• Forthistypeofdistribu'ondenetworkac'vityρ(t)is:
• Givenvaluesforµ,Wandthefunc'onΦ(V):– Therecurrenceequa'onscanbesolvednumerically
– Insomecasestheycanbesolvedanaly'cally
ExamplesofΦ(V)Satura'ngmonomialfunc'onofdegreer
Ra'onalfunc'on
Γ = 1; VT = 0 NB.:Thedeterminis'cLIFmodelcorrespondstothemonomialfunc'onwith
S
Resultsforthemonomialsatura'ngfunc'onwithμ=0
• Inthecasewithµ = 0,neurons“forget”allpreviousinputsignals,exceptthosereceivedat t – 1.
• P(V,t)containsonlytwopeaksatpoten'als:V0(t)=0andV1(t)=I+Wρ(t−1)
• Takingintoaccountthenormaliza'oncondi'on,thefrac'onsη0(t)andη1(t)evolveas:
• Assumingthatneuronscannotfireatrest,Φ(0) = 0:
• In a stationary regime, the recurrence equations
reduce to:
• Since Φ(V) ≤ 1, any stationary regime must have ρ ≤
1/2
Φ(V) = (ΓV)r, I = 0; r = 1Con'nuousphasetransi'ons
Absorbing State ρ = 0
Fixed point ρ > 0
2-cycle ρ1 = ½ − a ρ2 = ½ + a
a ≤ ½ − Vs/W
WC=1/Γ
Γ=1
WB=2/Γ
Brochinietal.,2016
Φ(V) = (ΓV)r, I = 0; r > 1Discon'nuousphasetransi'ons
r=1.2 r=2
ρ+
ρ−
Nontrivialsolu'onρ+onlyfor1≤r≤2Forr=2thissolu'onisapointatWC=2/ΓThediscon'nuitygoestozeroforr=1
W=WC(r)
Γ = 1 Γ = 1
Brochinietal.,2016
Φ(V) = (ΓV)r, I = 0; r < 1Ceaselessac'vity
Noabsorbingρ=0solu'on
Brochinietal.,2016
Γ = 1
ρ>0foranyW>0
Numerical solutions for µ > 0 Φ(V) = (ΓV)r, I = 0; r = 1
Brochinietal.,2016
Discon'nuousphasetransi'onsforVT > 0
Φ(V) = (Γ(V-VT))r, I = 0, µ = 0 ; r = 1, Γ = 1
VT=0 VT=0.05 VT=0.1
Thediscon'nuityρCgoestozeroforVTà0
Brochinietal.,2016
Neuronalavalanches(simula'onstudiesatthecri'calpointof
thecon'nuousphasetransi'on)
• Anavalanchethatstartsat'met=aandendsat'met=bhas:– Dura'ond=b−a;– Size
Neuronalavalanchesatthecri'calpointΦ(V) = (ΓV)r, I = 0, µ = 0; r = 1, ΓC = WC = 1
Brochinietal.,2016
Avalanchesizesta's'cs
Avalanchedura'onsta's'cs
Brochinietal.,2016
Self-organiza'onwithdynamicneuronalgainsIdea:fixtheweightsatW=1andallowthegainsΓtovary
u=1,A=1.1,τ=1000ms Brochinietal.,2016
Networkwithrealis'cconnec'vityExcitatoryandinhibitoryneurons
Simula'onresults
ThePotjans-DiesmannModel
105neurons(80%excit.20%inhibit.)109synapses
Availableatwww.opensourcebrain.org
• Modelforlocalcor'calmicrocircuit
• Integratesexperimentaldataofmorethan50experimentalpapers
• Excitatoryandinhibitoryneuronsmodeledbythesamedeterminis'cLIFmodel
• Asynchronous-irregularspikingofneurons
• Higherspikerateofinhibitoryneurons
• Replicateswellthedistribu'onofspikeratesacrosslayers
Potjans&Diesmann,2014
Fitofaveragebehaviorofstochas'cmodel(monomialΦ(V))toIzhikevichmodelneurons
Regularspikingneuron(excitatory) Fastspiking
neuron(inhibitory)
−40 −35 −30 −25
μ=0.9ΓΓ
win<<wex win<wex
win>wex win>>wex
Computa'onalcost
TIzhikevich
Tstochas'c_______
No.ofsynapses
----------------------------------------------------
Timetosimulate5secofnetworkac'vity(reducednetworkwith4000neurons)
Whichmodeltouseforcor'calspikingneurons?Izhikevich,2004
Conclusions
• Thestochas'cneuronmodelintroducedbyGalvesandLöcherbachisaninteres'ngelementforstudiesofnetworksofspikingneurons
• Enablesexactanaly'cresults:– Phasetransi'ons– Avalanches,SOC
• Simpleandefficientnumericalimplementa'on
ResearchTeam
USP,SaoPaulo USP,RibeirãoPreto
Thanks!
Unicamp,Campinas
NeuroMat
L.Brochini M.Abadi
A.Galves
A.CostaO.Kinouchi J.StolfiR.Shimoura V.Cordeiro
E.LöcherbachUniv.Cergy-PontoiseUSP,SaoPaulo