Sensorimotor interplay in vocal communication

SENSORIMOTOR INTERPLAY IN VOCAL COMMUNICATION

Ms. Satadru DeM.Phil II year

Dept. of NeurophysiologyNIMHANS

Date: 05.08.2016

Significance of communication in societyEvolution of communication (taxonomical aspects)Human speech - History of the development of linguistic studies - Peripheral phonation system - Anatomical areas in speech perception - Functional pathways - Models - Anatomical areas in speech production - Models Speech DisordersAphasias

Significance of communication in societyEvolution of communication (taxonomical aspects)Human speech - History of the development of linguistic studies - Peripheral phonation system - Anatomical areas in speech perception - Functional pathways - Models - Anatomical areas in speech production - ModelsSpeech DisordersAphasias

PLANTS

o The fungal body is made up of a mass of thread-like structure called the myceliumo They act as a kind of an underground internet linking the roots of different plants

o This fungus-plant relationship unit is called mycorrhizao Boosts the plant’s immune-systemo Suzanne Simard of British Columbia Univ in 1997 showed the evidence of mycorrhiza between Doughlas Fir and Paper Birch trees.

“These plants are not really individuals in the sense Darwin thought they were, competing for survival of the fittest.” –Simard,2011

Jim Westwood (Professor of plant pathology, physiology and weed science) discovered the fact that plats communicate with each other on a MOLECULAR LEVEL -Science, August 15,2014

EXCHANGE OF THOUSANDS OF mRNA MOLECULES CREATES AN “OPEN DIALOGUE” BETWEEN TWO SPECIES.

Westwood examined the parasitic interation between Dodder (Cuscuta pentagona) and tomato host plant.

Transport of RNA bwetween the two species.

mRNA from dodder “commands” the host plant to perform actions, like lowering the defense system, via several proteins encoded by the mRNA.

http://vimeo.com/virginiatech

ANIMALS(honey bees, fishes, songbirds, primates)

ReproductionFood foragingSelf Defense

-Vocal sounds (Crickets, frogs, monkeys)- Gestures (honey bees, fireflies)- Release of pheromones (moths)The competition for the attention of the opposite sex has developed more conspicuous signals and evolution of displays, signalling and morphology (Gould and Gould, 1996).

Sexual competition – an important selective factor

Cues

Simultaneous Sequential

Fund

amen

tal N

euro

scie

nce,

Squ

ire, 3

rd e

dn.

Animal communication is mostly innate

Feature detectors in their nervous system that automatically isolate visual and acoustic stimuli

Fundamental Neuroscience, Squire, 3rd edn.

The amorous hums of the Midshipman Fishes (Porichthys notatus)

Science, July 18 2008

Andrew H. Bass, Cornell Prof. of Neurobiology and Behavior, New York University: - mapped the brain cells of newly hatched larvae using fluorescent dyes, and compared them to other species - clusters of cells in the larvaes’ hindbrain that developed into networks controlling vocalisation - mainly in rhombomere 8 (origin of CPG) that gives rise to the vocal motor nucleus - the neural network for voice production in vertebrates originated with aquatic animals, and the fundamental attributes are conserved over the entire phylum of vertebrates.

Please click to see videoPlease click on speaker icon to hear if video isnt runing

Birds have innate songs

Combination instinct and direct innate learning produce different results

Critical period of learning has impact on the perfection of the adult bird’s songA chick deafened during critical period is unnable to produce even the innate sound


Learning process involves:- Listening to each member of the species-Checking to see of they match any element of memorised song- Discarding unnecessary tones- Rearranging, scoring and modifying

Component of auditory feedback system

Evolutionary remnant of the forward prediction system???

Neural substrates for birdsongSome of the brain centres involved (White crowned sparrow, Zebra Finch, Canary etc):

HVC (High vocal centre)RA (Robust nucleus of arcopallium)DMnXIIts (nucleus of twelfth cranial nerve)Uva (Nuc. Uvaformis in thalamus)CMM (Caudalateral mesopallidium)NCM (Caudomedial neostriatum)AFP (Anterior forebrain pathway, part of the basal ganglia circuit)Area X (in the AFP)RAmPAmLMAN (Lateral magnocellular nucleus of anterior neostriatum)MMAN (Medial magnocellular nucleus of anterior neostriatum)

Peripheral parts include:LungsTracheaSyrinx

General neural circuitry for bird song production

HVCUva CMM

Area XRA Intereurons

DM

nXIIts

NXIIts (tracheosyringeal portion

of CNXII)

Uva Nlf HVC (Nuc. Interface of neostriatum)

Nottebohm , F. (2005) in “The neural basis of birdsong”, PLoS Biology, 3 (5).

General neural circuitry for bird song production

RAm PAm

Motoneurons in nXIIts Respiratory neurons in brainstem

DM RA

CN SO LL

Mld (avian homologue of inferior colliculus)

Nuc. Ov (avian homologue of MGN)

Field L NCM (activated during conspecific song perception)

CMM

Reiner et al.,(2004) Songbirds and the revised avian brain nomenclature, in Behavioral neurobiology of birdsong, Sci.,1016, pp 77-108.

General neural circuitry for bird song productionAFP HVC RA

Area X DLM LMAN

DMP MMAN

HVC

Song Output

Reiner et al.,(2004) Songbirds and the revised avian brain nomenclature, in Behavioral neurobiology of birdsong, Sci.,1016, pp 77-108.

NON-HUMAN PRIMATESo Most linguistic studies done using Vervetso Large repertoire of innate calls for social communication (for e.g., 36 signals for 4

types of alarm calls)o No voluntary control over respiration, vocal tract or vocalisationo Researchers have tried to teach chimpanzees comm. system comparable to human

speech but with no fruitful results.

o Primate vocal centres lie medially in the Cingulate area, unlike humans who have lateral cortical areas in speech controlo Motor planning takes place in brainstemo Auditory cortex perceive conspecific calls

o Specialised nuclei for vocal sound productiono Assymmetry of hemisphereso Sound perception and production centres overlap


EVOLUTION OF HUMAN COMMUNICATIONThe evolution of human communication can be summarized as follows:Oral Language (30,000 to 100,000 years ago)Cave Drawings (about 15,000 B.C.)The Written Word (5,000 B.C.)The Printing Press (around 1450)The Telegraph (1844)The Radio (1920s)Television (became more wide-spread in the 1950s)Digital Technology

•The FOXP2 gene (forkhead box P2) has a lineage in Eukaryota; Metazoa; Chordata to Hominidae; Homo sapiens.•It is expressed in fetal and adult brain as well as in several other organs such as the lung and gut. •It is an evolutionarily conserved transcription factor; autosomal dominant•This gene is required for proper development of speech and language regions of the brain during embryogenesis•Involved in a variety of biological pathways and cascades that may ultimately influence language development. Mutations in this gene cause speech-language disorder 1 (SPCH1), also known as autosomal dominant speech and language disorder with orofacial dyspraxia.

Significance of communication in societyEvolution of communication (taxonomical aspects)Human speech - History of the development of linguistic studies - Peripheral phonation system - Anatomical areas in speech perception - Functional pathways - Models - Anatomical areas in speech production - Models Speech DisordersAphasias

o Saussure founded the study of linguistics in 20th century. Regarded language as a rule-governed system, much like chess.

o B.F. Skinner (pre and post WW2) founded the paradigm of behaviorism. The concept of Tabula rasa. Verbal behavior is malleable – initially lacking structure,richness induced by environment.

o Noam Chomsky (1950’s) – The Chomskyan Nativism, complete contradiction against Skinner’s behavioral theory.

o John Austin (philosopher) - Speech Act Theory – speech is for action in social world.

o Howard Patte’s Biosemiotics – “Natural constraints” in evolution reduce the possibility of evolutionary outcomes. Evolution of spoken language overcame the natural constraint.

o Paul Broca (1800’s) – discovery of Broca’s area (BA44,BA45) in a patient with speaking disorder (Broca’s Aphasia).

o Karl Wernicke, Norman Geschwind(late 19th century) – Wernicke’s area (BA22) and Wernicke’s Aphasia from postmortem study of patient with a different speech disorder.

o Penfield and Japser (1950) – mapping of brain areas using local anesthesia.o Juhn Wada (1960) – Wada test using sodium amytal.

Current seminal works on neurolinguistics and psycholinguistics with fMRI, PET, MEG, SPECT etc can

be credited to Gregory Hickok, Poeppel et al., Nina Dronkers, Damasio et al.

The peripheral phonation system

LungsLarynxPharynxOral and nasal cavitiesTongue,teeth,lips

•Vowels are “voiced sounds” – have periodicity;•Consonants are “unvoiced” – vocal folds are either fully open /f/, /s/, or in intermediate position /h/;•Consonants can be:Labial - /p/, /b/Dental - /f/, /v/Palatal - /sh/Glottal - /h/

•Natural resonances of the vocal tract - FORMANTS

Neuroscience, Purves 5th edn

Functionally these areas are connected by a Ventral pathway and a Dorsal pathway

Ventral pathway – Auditory to comprehension – Local syntactic structure building Dorsal pathway – Auditory to motor mapping – Syntactic processing

First dorsal pathway:

TEMPORAL CORTEX PREMOTOR CORTEX

IFC

SLFSecond dorsal pathway:

TEMPORAL CORTEX BA44 Arcuate fasciculus

ANTERIOR INFERIOR VENTRAL FRONTAL CORTEX TEMPORAL POLE

Inferior frontal cortex Temporal cortex Occipital cortex

Uncinate fasciculus (UF)

Extreme capsule fibre system (ECFS)

Inferior fronto-occipital fasciculus (IFOF)

Anterior temporal cortex Posterior temporal cortex

ILF

MLF

o The Dual-route model of speech processing - acoustic speech must be linked to conceptual semantic representations,i.e., speech much be understood; the brain must link acoustic speech information to the motor speech system, i.e., speech sounds must be reproduced with the vocal tract.

o This paper focuses on two things curently: The Dual stream model as it is understood till date; The concept of predictive coding

o Predictive coding is a notion that is focal in both motor control and in perceptual processing, i.e., in the two processing streams.

THE VENTRAL STREAM:-1. Dominant processing network for speech comprehension –

lexical,semantic,syntactical;2. More left lateralised forming a gradient from superior temporal lobe towards ATL;3. Hierarchically organised – auditory perception starts in the posterior STL and

semantic processing progresses towards ATL.

Posterior Superior Temporal Areas:-1. Consists of primary and secondary auditory cortices;2. Belongs neither to ventral or dorsal stream per se;3. The dorsal and ventral stream diverges from the middle and posterior STS.

Middle and posterior STS:-1. Speech comprehension network;2. Post. STS – complex sounds; Middle STS – phonological and sub-lexical

information;3. Division of labor between left and right STS:- fMRI studies – left STS in phonetic and phonological processing (Turkeltaub & Coslett,2010); right STS responds to all vocal sounds, processing of emotional prosody (Belin et al., 2004,2011).

Posterior middle and inferior temporal gyrus:-1. Posterior MTG (BA21) Posterior ITG (BA20) Lexico-semantic interface;

2. ITG and IFC – lexical system (Hickok & Poeppel, 2007);3. MTG, angular gyrus, prefrontal areas, posterior cingulate gyrus, fusiform and

parahippocampal gyri – semantic system (Binder et al.,2009);4. Border of posterior ITG with occipital lobe forms the VWFA – visual stimuli

processing during reading;5. Anterior area to post. ITG – modality independent (Cohen et al.,2004).

Anterior Temporal lobe:-1. The posterior-anterior extension of the STG – hierarchy of the ventral stream;2. More linguistically complex information – greater spread of info. to ATL

IFG, ITG;3. Sentence processing (Price, 2012) in concert with angular gyrus (Jang et al., 2013);4. Strongly left lateralised semantic processing.

Inferior Frontal gyrus:-1. The ventral and dorsal streams terminate in IFG;2. Serves as a component of speech production;3. Modality independent lexical processing;4. BA45 maybe the end point of ventral stream.

Additional Areas:- 1. Supramarginal gyrus – categorical speech perception (Turkeltaub & Coslett, 2010)2. Angular gyrus – phonological decision making Semantic and conceptual retrieval Finger counting, writing, reading aloud

(contd in next slide)

3. Wernicke-Geschwind Model of language (1965)

4. Basal ganglia – processing of emotional prosody5. Cerebellum – internal model of speech motor control parietal and cerebellar circuits in sensory-motor coordination:- Cortico-cortical circuits – motor control at syllable level Cerebellar-cortical circuits – motor control at phonetic level

THE DORSAL STREAM:-1. Initially thought to be involved in spatial hearing (Rauschecker,1998);2.Supports auditory-motor integration (Hickok & Poeppel, 2000,2004.2007);3.Links speech sounds with the motor system to reproduce sounds with the vocal tract;4.Dominating network for repitition learning(sub-lexical);5.“Goal of speech production is not a motor configuration but to achieve a speech sound” – Guenther et al.,1998;

6.Integrates the functioning of the following areas: Auditory areas in STS, motor areas in lateral IFG (BA44), dorsal premotor site and area Spt (Sylvian fissure at parieto-temporal boundary in left planum temporale) (Hickok et al., 2010);

7. STS codes sensory based representation of speech Motor regions code motor based representation of speech (Hickok 2003, 2009, 2011, 2012)

8. Auditory-motor integration in dorsal stream is based on internal forward models.

Spt as a sensorimotor integration system transforming sensory speech representations to motor speech acts.

The State Feedback Control

The State Feedback Control Model

The Hierarchical state feedback control model

aSMG anterior supramarginal gyrus; M1 primary motor cortex; S1 primary somatosensory cortex; vBA6 ventral BA6

In the left hemisphere:1. Middle STG/STS2. Posterior STG/STS (Wernicke’s area)3. Middle MTG4. Posterior MTG5. Posterior fusiform gyrus6. Posterior IFG (Broca’s area)7. Ventral precentral gyrus (primary and premotor cortex)8. Supplementary motor area SMA9. Thalamus10. Anterior insula11. Medial cerebellum

In the right hemisphere:1. Middle STG/STS2. SMA3. Lateral cerebellum4. Medial cerebellum

The Lemma Model of Speech Production

Cognitive Neuroscience of Language, Kemmerer, 1st edn 2015 Levelt et al., 2001

o Two main subsystems – one for “lexical selection”, i.e., identifying the most appropriate word in the mental lexicon; and another for “form encoding”, i.e., preparing the word’s articulatory shape.

o Conceptual focussing and Perspective taking – to map the expression into a lexical concept (“thinking for speaking”)

Choosing words that express certain subjective conceptualisations of the situation;

o Lemma selection – the activated target lexical concept is propagated to the lemma level so that the lemma matches the concept;

o Retrieving morphemic phonological codes – accessing the morphemic representation and “spelling out” the segmental phonemic content.

- Slips of the tongue may not occur at this level. - Retrieved incrementally


o Syllabification – ordered phonemic segments of the words are bundled into syllables yielding a phonological word;

o Phonetic encoding – draws heavily upon the mental syllabary; phonological representation of a word is matched in each unit with the corresponding syllabary node;

o Articulation – activated syllable node is sent to the motor system that controls the vocal apparatus;

o Self monitoring – to detect and correct our own speech errors. - External feedback loop error detection – monitors auditory signals of self produced speech; - Internal feedback loop error detection – monitors the process

of generating phonological words.


Type of disorder Speech Comprehension Other diagnoses

Causes/Areas affected

APRAXIA

Impaired articulation; errors with consonant clusters and fricatives (eg. ‘ch’ of church or

‘strict’)

Intact

Pauses between syllables and

words; slowed rate of speech

Broca’s area,left frontal and

temporoprietal cortex,regions of

insula, basal ganglia

DYSARTHRIASlurred, mumbled speech difficult to understand; slow

rate; abnormal pitch and rhythm

Intact

Limited tongue,jaw movement;

hoarse or breathy voice

Cerebellum, motoneurons and skeletal muscles (seen in PD, HD, ALS), muscular

dystrophy

DEVELOPMENTAL STUTTERING

Words or syllables are repeated or

prolongedIntact

Talking or singing in chorus improves speech

Area Spt; Sensory-motor integration is dysfunctional; the error correction

signal is inaccurate

APROSODIAWithout tone or

expression/emotions

IntactDifficulty in perceiving

emotional tones

Right hemisphere speech areas/ right

STS

APHASIAS

Do lower animals and non human primates suffer from communication disorders???

Then…..

….and now!

THANK YOU!

Sensorimotor interplay in vocal communication

Documents

Transcript of Sensorimotor interplay in vocal communication