Introduction.doc

Arbib: Mirror Neurons and Action Recognition August 2001

Mirror Neurons and the Neural Basis for Action Recognition:A Computational Model

Principal Investigator: Michael A. Arbib

Project SummaryThe monkey brain's mirror system for grasping matches the neural command for an action with the neural code for the

recognition of the same action executed by another. Analysis of this system has major implications for the understanding of imitation and language in humans, but the focus of the present proposal is a new theory of the brain mechanisms underly -ing action recognition in monkeys. We will explore this by the development of a detailed neural model of the mirror sys -tem and related brain regions, with special emphasis on development and learning. The model will explain a large body of recent neurophysiological and neuroanatomical data from monkeys and lead to new experiments.

The modeling environment will include a primatoid hand-arm avatar for generating actions (to provide output in studies of learning to grasp, and input stimuli for studies of action recognition); preprocessing routines for visual input; and tools for modeling adaptive networks of biologically plausible neurons responsive to the constraints of neurophysiological and neuroanatomical data. Within this environment, we will provide and validate models for a range of specific aims: those re -lated to the development of grasping and the mirror system, paying particular attention to the hypothesis that visual feed -back on hand configuration during grasping underlies the mirror mechanism; those analyzing the function and plasticity of the mirror system in guiding and recognizing single actions, with new hypotheses on population coding in the mirror sys -tem; and finally those which extend recognition to compound actions in context, showing how this extension may bridge the gap from action recognition to understanding.

Our modeling will be enriched by collaboration with experimentalists working on monkey neurophysiology and neu-roanatomy at the University of Parma. To facilitate the exchange and comparison of experimental data and modeling re -sults, we will further develop two of our neuroinformatics tools, NeuroBench for analysis of neurophysiological data, and the NeuroHomology Database for analysis of neuroanatomical data.

1. Introduction

1.1 The Interest of the Mirror SystemAs Vilayanur Ramachandran (2000) has commented:

[I'm] fascinated by the rostral part of the ventral premotor area. Giacomo Rizzolatti at the University of Parma has elegantly explored the properties of neurons in this part of the brain the so-called "mirror" neurons (which) will fire when a test monkey performs a single, highly specific action with its hand … [and] in response to [similar actions performed by others]. … With knowledge of these neurons, you have the basis for understanding a host of very enigmatic aspects of the human mind: imitation learning, intentionality, "mind reading," … Rizzolatti and Michael Arbib … suggest that mir -ror neurons may also be involved in miming lip and tongue movements, … [and] present the crucial missing link between vision and language.

More specifically, mirror neurons in area F5 of the monkey, one of the motor areas forming the frontal agranular cortex, become active both when the monkey makes a particular action (like grasping an object or holding it) and when it observes another individual (monkey or human) making a similar action. The vast majority of F5 mirror neurons shows a marked similarity between the action effective when observed and the action effective when executed. This congruence is some -times extremely strict, but broadly congruent neurons are of particular interest because they generalize the goal of the ob -served action across many instances of it. Typically, mirror neurons do not respond to the sight of a hand mimicking an ac -tion in the absence of a target object nor do they respond to the observation of an object alone, even when it is of interest to the monkey (Gallese et al. 1996; Rizzolatti et al.1996).

Ramachandran's account typifies the high level of current interest in the mirror system. Indeed, PET experiments in hu -mans showed that both observation and execution of hand actions activate left Broca's area (Rizzolatti et al., 1996), while fMRI (Iacoboni et al. 1999) and MEG (Nishitani & Hari 2000) recordings showed the involvement of right Broca’s area in imitation. Broca's area is homologous to area F5 of monkey premotor cortex (Rizzolatti and Arbib 1998). Analysis of the human mirror system and especially its role in imitation and language is a vital topic for our future research. However, the research proposed here will focus on neural mechanisms for the mirror system and action recognition in the monkey, developing a detailed neural model based on monkey neurophysiology and related data. We will use the model to investi -


gate a complete system from sensing to acting by coupling it to a primatoid “avatar”, an extensible biomechanical simula -tion of the arm and hand of a primate. The avatar will have components adjustable to match the dimensions of monkey and human since we must analyze, among other things, how a monkey can recognize actions whether those of a monkey or a human.

We next outline the specific aims of the proposed research, the collaborations which will provide data to ground and test our models, and the responsibilities of the personnel for whom funding is requested. We then review key data from neu -roanatomy and neurophysiology relevant to the mirror system in monkey and provide a sample of our own prior modeling of the monkey nervous system. The proposal concludes with a detailed Research Plan for modeling and validation for each of the specific aims in turn: those related to the development of grasping and the mirror system; those analyzing the func -tion and plasticity of the mirror system in guiding and recognizing single actions; and finally those which extend recogni -tion to compound actions in context, showing how this extension may bridge the gap from action recognition to under -standing.

1.2 Specific AimsThe present proposal focuses on modeling the neural mechanisms of action recognition in monkeys, leaving modeling

related to brain imaging studies of action recognition and imitation in humans to other proposals. This work builds upon our long-standing and ongoing collaboration with Rizzolatti (Jeannerod et al., 1995; Grafton et al., 1996, 1997; Arbib and Rizzolatti, 1997; Rizzolatti and Arbib, 1998) to study the mirror neuron system in monkeys and humans. Our collaboration has been funded in part by the Human Frontier Science Program (HFSP), which also funded recent collaboration with the UCLA group of Woods and Iacoboni (Modeling: Arbib et al., 2000; fMRI: Iacoboni et al., 1999). Since HFSP funding will expire in 2001, it is timely to seek NSF funding to extend this research effort. We will develop the MNS2 model for the monkey mirror system (Figure 1; see the section "Grasping and the Mirror System in Monkey" for a review of relevant data, including a description of each of the brain regions referred to here) to elucidate how visual recognition of actions rests on neural structures common to all primates, including neurons in STS coding for the tracking of gaze direction and body motions, neurons in the parietal lobe which code hand shapes for different types of grasps; areas involved in move -ment composition (we focus on SMA and basal ganglia); and the mirror neuron cells which respond to both observation and execution of actions. Our five Specific Aims are grouped under 2 broad headings, "Development of the Mirror Sys -tem", and "Recognition of Novel and Compound Actions and their Context".

Figure 1. Overview of the proposed MNS2 mirror neu-ron model with brain regions assigned. Current modeling has assessed overall func-tionality to predict physio-logical responses of the F5 mirror neurons. The pro-posed modeling will incor-porate and explain a wide range of detailed anatomical and physiological data, and will offer new insights into the development and func-tionality of the mirror sys-tem.

Development of the Mirror System:

In this proposal, we seek to explain the circuitry that makes mirror neuron activity possible, and then explore a number of hypotheses on the role of mirror neurons in the brain mechanisms for a range of adaptive behaviors related to action recognition. We are concerned with three basic types of F5 neurons: motor neurons, canonical neurons, and mirror neu -rons. Non-visual motor neurons comprise about 80% of F5 neurons. The other two classes comprise visuomotor neurons. They have motor properties indistinguishable form those of motor neurons, but, in addition, they respond to specific type of visual stimuli – canonical neurons to object observation; mirror neurons to action observation. All F5 neurons are motor neurons. Some of them receive connection from parietal region AIP, other from parietal region PF, others have no connec -tions with the parietal lobe. A major goal of our modeling will be to show how, during development, motor neurons with parietal connections from AIP and PF will acquire visual properties and become respectively canonical or mirror neurons:

Development of Grasp Specificity in F5 Motor and Canonical Neurons: This project explores the development of the neurons which form the action generation circuitry which is an inseparable partner of the action recognition circuitry of the mirror system. We will demonstrate how somatosensory feedback can play a crucial role in defining the population of F5


motor neurons, and how input from the parietal region AIP shapes up the F5 canonical subpopulation and is shaped up in turn, as the developing. F5 canonical neurons select via re-afferent connections visual neurons describing a variety of sur -faces. Only those selected become AIP neurons that code affordances.

Visual Feedback for Grasping: A Possible Precursor of the Mirror Property: We offer a new hypothesis for the general-ization from the visual description of action made by the acting individual to that made by other individuals. We propose to demonstrate how neurons which develop to provide feedback for self-generated goal-directed grasping movements, us -ing the association between F5 motor activity and the visual stimuli resulting from this activity, will extract "hand configu -ration" data concerning the relation of the moving hand to an object that will readily generalize to the movements of oth -ers' hands. The model will involve a self-organization process which exploits bidirectional connectivity across F5, PF and STS areas to illuminate the developing role of STS and PF in the functioning of the mirror neuron system

Recognition of Novel and Compound Actions and their Context:

The modeling of development defined above emphasizes how the infant monkey builds a basic motor repertoire of reach-and-grasp actions and how these come to be integrated with a set of visual processes that, we hypothesize, develops first to provide feedback for the monkey's own actions and then serves to recognize hand-object relations in other monkeys which signal similar actions. Here, each grasp is seen as related to the affordance of a target object. Our task in the present section is to propose models which place the mirror system in a broader perspective. As we have noted, our analysis of the monkey mirror system is to be seen as grounding efforts to understand the brain mechanisms underlying imitation in hu -mans. For imitation, the key issue is the recognition of the structure of novel actions, extending the prior repertoire of ac -tions. For the monkey studies proposed here, we focus on three related issues: (i) How does a variant of a known action come to be recognized? (ii) How can a novel action be recognized a compound of (variants of) known actions. (iii) How are actions "understood"? We argue that understanding will in general involve more than recognition of the action (move-ment + goal) in isolation, but will also involve recognition of the context in which the action occurs and expectations as to its consequences.

The Pliers Experiment: Extending the Visual Vocabulary: The mirror neurons of the monkey will not fire when a mon-key first sees the experimenter grasp a raisin with a pair of pliers, but certain mirror neurons will come to fire after the monkey has been repeatedly exposed to this stimulus. We will model how the system may recognize such novel stimuli, thus extending its "input vocabulary" beyond a prior set of hand configurations. This will lead us to adopt a more subtle ap -proach to the visual input than used in our earlier work, and will involve modeling processes of learning which "work back in time" so that recognition that a raisin is being picked up can draw attention to the way in which earlier movement of the novel gripper can be predictive of the grasp. We will also explore the hypothesis that it is a set of mirror neurons that pro -vides a nuanced representation of an action, rather than the broadly tuned response of a single neuron. This modeling of population coding will be strongly linked with the increasing push of the Parma group to use multi-electrode recording.

The present proposal contains three kinds of model: those strongly driven by available data; those which will be devel -oped in tandem with empirical efforts now planned by our collaborators in Parma; and those which offer conceptual analy -ses which will set the stage for design of experiments several years in the future. The last two Specific Aims belong more to the third class, but will nonetheless be grounded in empirical data.

Recognition of Compounds of Known Movements: We will extend our analysis of population coding of single actions to model the learning and recognition of compounds. We will first extend our earlier work on the interaction of basal ganglia (BG) and supplementary motor area (SMA) on the generation of sequences of movements to analyze their linkage with the mirror system for recognition of sequences of movements; we will then generalize this approach to handle the recognition of novel actions that are formed as a temporally coordinated superposition, rather than a sequence, of known actions.

From Action Recognition to Understanding: Context and Expectation: Finally, we will analyze "understanding" within the framework created by all the earlier work. Our key idea here is that understanding is not simply the recognition of an action in isolation, but must involve some notion of "meaning", e.g., the context in which the action is appropriate or the expectations that such a behavior evokes. This relates to preliminary findings from Parma which will serve as the basis for further collaboration.


Figure 2. Left: Data from the Rizzolatti group take the form of records of a cell's firing across a number of trials; a histogram summing such records provides the view of the "expected" behavior that is used to challenge our modeling. In this example, the experimenter grasps a piece of food with his hand, then moves it toward the monkey who, at the end of the trial, grasps it. The neuron dis -charges during observation of the experimenter's grasp, ceases to fire when the food is given to the monkey and discharges again when the monkey grasps it. The rasters are aligned with the moment when the food is first grasped (vertical line). Each small vertical line in the rasters corresponds to a spike. Right: An example of the output from 2 mirror neurons simulated in the MNS1 model de-scribed below. The curves are to approximate or predict the histogram data for physiologically iden-tifiable neurons. This simulation demonstrates resolution by the mirror neurons between a power and precision grasp when the observed avatar makes a precision grasp, grasping a long thin object by its ends. In the initial portion of the trajectory, the hand preshape is mistaken for a power grasp, but the simulated mirror system corrects this misclassification as more of the trajectory is taken into account. This is an example of a novel prediction from our modeling which is readily amenable to testing in the Rizzolatti laboratory.

1.3 Validation: Models, Collaborations, and DatasetsThe models proposed here will yield predictions and explanations addressing data from neurophysiological correlates of

monkey action recognition and will be linked to the work of the Rizzolatti group. Throughout the research, empirical pre -dictions and tests will accompany our modeling. In many cases, new experiments will support our modeling; in other cases, the data will lead us to revise the models, leading on to further predictions as we extend the scope of the models. The present section summarizes the datasets to be gathered by our collaborators (in addition we will, of course, continue to ad -dress many data culled from the literature) and also outlines a collaborative effort on biomechanical simulation. Support letters are appended to this proposal.Neurophysiology and Behavior

Giacomo Rizzolatti of Parma has focused on the neurophysiology of the premotor cortex of monkeys during visually guided reaching (Figure 2 Left); this has been broadened to include parietal inputs to the mirror system. In addition to col -laboration on a variety of papers (with the proposed modeling of the ontogeny of the grasping and mirror system a current target for future joint papers), our modeling will build upon extant data to make predictions for new experiments (Figure 2 Right). An important issue (discussed further below) is that the data published in papers do not always yield sufficient de -tail for our modeling. We have thus developed new software, NeuroBench, for viewing and analysis of neurophysiological records. The Parma group recently released recordings from 37 mirror neurons to us and these have been entered into Neu-roBench. We find these preliminary efforts promising, and thus propose to develop NeuroBench into an advanced knowl -edge discovery tool and incorporate many more data into the database. We will supply interface routines to enable the modelers (at USC) to transfer the results of a simulation run directly to NeuroBench with annotations, date, and author in -formation, with similar tools to enable neurophysiologists to transfer their recordings with annotations to NeuroBench. NeuroBench will not only serve as a database and visualization tool, but will also perform comparative and data mining computations. Since the Parma group is now planning multi-electrode experiments, we propose to develop routines for temporal analysis including tools to compare simulation-generated data with multi-electrode data. The result will provide the Parma neurophysiologists with new tools for data analysis, as well as stimulating more detailed modeling by our group. The results of data analysis and modeling will feed back into design of new experiments, thus advancing the theory-experi -ment cycle. Neuroanatomy

Also in Parma, Massimo Matelli and Giuseppe Luppino have focused on neuroanatomy of the monkey, with new find -ings on the relation between premotor cortex, parietal cortex and other regions. These data have constrained our models as we begin to make functional sense of the connections they have revealed. They will provide further data on connections of the monkey brain to constrain our modeling in the coming years, as well as engaging in further analysis of homologies be -


tween the human and monkey brains relevant to the broader research program of which the present effort is part. The anal -ysis of anatomical data will benefit from our work on the NeuroHomology Database (NHDB; Bota, 2001; Bota & Arbib, 2001), an online knowledge management system that handles information about brain structures, neuroanatomical connec -tions and similarities between brain structures. Currently, NHDB comes in two online versions, NHDB-I and NHDB-II (Bota, 2001a,b). It contains modules for Brain Structures, Connections (users can search connection information and also evaluate the confidence level for a given neuroanatomical tract between two brain structures) and Homologies (our homol -ogy inference engine uses eight criteria to evaluate the degree of homology between two brain structures). To date, the Ho -mology module of NHDB-I contains more than 50 similarities between different brain structures from rodents, macaques and humans. Clearly, the focus on NHDB for the present proposal will be on systematizing macaque data from Matelli and Luppino with data culled from the literature. The analysis of macaque-human homologies will be a target for other re -search. Simulation of Biomechanics

Figure 3: The reach/grasp simulator we have already developed takes the representation of the shape and position of a (virtual) 3D object and the initial position of the (virtual) arm and hand and computes a trajectory that ends with simulated grasping of the object. The system, programmed in Java, simulates a 19 degree of freedom (DOF) arm and hand that can be con-trolled through a user interface or automatically (e.g. for learning and testing). The 19 DOFs are 3 at the shoulder, 1 for elbow flexion/extension, 3 for wrist rotation, 2 for each finger joint with 2 more for the thumb. The system provides routines to perform realistic grasps. In "inverse model" mode, it solves the inverse kinematics problem by simulated gradient de -scent with noise added to the gradient

Finally, we mention a collaboration focused on simulation methodology rather than empirical data. Stefan Schaal (Com -puter Science, USC, and Computational Learning, Kawato Dynamic Brain Project) will share his expertise on developing learning algorithms for both robot control and human motor control (e.g., Schaal 1997). He will make available his simula -tion software package for use in extending our present reach-and-grasp simulator (Figure 3) to include estimates of contact forces including slip. We expect this collaboration to also yield fruitful interactions with his own NSF-funded research on imitation and rehabilitation for human patients with motor disorders. Schaal's Simulation-Laboratory (SL) software pack -age (http://www-slab.usc.edu/publications/schaal-TRSL) allows rapid development of arbitrary rigid body dynamics sys-tems with complete control and data visualization tools. For studies of learning to grasp and development of the mirror sys -tem, we will interface the arm-hand model with our neural models of brain regions. The simulator will also simulate con -tact force with external objects that will be essential for our work on learning to grasp, where we want to use the slippage of the object in the hand as a negative reinforcement learning signal to the learning system. Future development of the sim -ulator will also allow us to include muscle-based motor control, which we envision to be interesting for our project at an advanced stage. The simulation software is available to us at no expense. The dimensions of the arm and hand will be ad-justable to match the dimensions of monkey and human to explore the monkey's recognition of human as well as monkey actions. Integration of Databases and Modeling Tools

In addition to the above, we will link NeuroBench and NHDB with our Brain Models on the Web database (BMW; Ar -bib and Bischoff-Grethe, 2001). BMW currently contains a set of models developed using our NSL Neural Simulation Lan -guage, and will be extended to include biomechanical simulations of the kind just described. An important feature of NSL 3.0 is the development of a Schematic Capture System (SCS) to allow hierarchical viewing of model structures (Arbib et al., 2001). We will extend SCS to provide a hierarchical view of biological data as well, enabling the user to compare the structures of the modules and connections in our models with the neurobiological information from NHDB and Neu -roBench. We can thus ensure strong integration of empirical data into the design and testing of our models.

1.4 PersonnelThe USC investigators will work closely with our collaborators on the analysis of relevant data to both constrain and val -

idate our modeling. Funding is requested only for the modeling effort, plus travel funds to allow members of the USC team to visit the Parma group and vice versa. Michael Arbib has long experience in computational neuroscience, especially in modeling neural mechanisms underlying visually guided behavior in close collaboration with experimentalists. He will continue his work on detailed neural modeling of the monkey nervous system while managing the collaboration with the Parma group. To a first approximation, the responsibilities that the 2 RAs will share with Arbib are to be divided as fol -lows:

RA 1 (currently Erhan Oztop) will be responsible for developing NeuroBench as a repository of neurophysiological data from Parma and elsewhere; for developing the primatoid hand-arm avatar; for the Development of the Mirror System com -ponent of the modeling; and for analyzing data and developing predictions related to population coding in the mirror sys -tem. RA2 (Salvador Marmol) will be responsible for developing NHDB as a repository of neuroanatomical data from


Parma and elsewhere; for linking BMW, NeuroBench and NHDB to facilitate the use of empirical data in the grounding and testing of models; and for the Compound Actions and their Context component of the modeling.

2 Background

Figure 4. Monkey brain areas relevant to the proposed modeling: The brain regions involved in the mirror neuron system constitute a large portion of the brain parietal cortex, premotor and motor cortex, the superior temporal sulcus and prefrontal cortex. In the macaque brain, pos-terior parietal cortex and the cortex of caudal su-perior temporal sulcus (STS) have been subdi-vided into numerous areas mainly involved in spatial analysis of the visual environment and in the control of spatially oriented behavior (Maioli et al. 1998). F1: motor cortex; F3: SMA-proper; F4: proximal ventral premotor area; F5: distal ventral premotor area; F6: pre-SMA; LIP: lateral intraparietal sulcus; MIP & VIP: medial and ventral intraparietal sulcus; AIP: anterior intra-parietal sulcus; 7b: parietal area PF; 7a: parietal area PG.

2.1 Grasping and the Mirror System in MonkeyThe rostral (F5) and caudal (F4) inferior premotor cortex of monkey (Figure 4) are involved in reaching and grasping

movements, while F1 is primary motor cortex. Neurons in F4 appear to be primarily involved in the control of proximal movements (Gentilucci et al., 1988), whereas the neurons of F5 are involved in distal control (Rizzolatti, et al., 1988). Neurons located in area F5 of the monkey discharge during specific goal-directed hand and/or mouth movements. Our in -terest centers on a subclass of F5 neurons that become active not only during the monkey's own movements, but also when the monkey observes the experimenter or another monkey make an action similar to the one that, when actively performed, triggers the neuron. Neurons endowed with these properties are referred to as mirror neurons (di Pellegrino et al. 1992; Gallese et al., 1996). In monkeys trained to grasp objects requiring different types of grip, about half the neurons in the an -terior intraparietal sulcus (AIP) related to hand movements fired almost exclusively during one type of grip, with precision grip being the most represented grip type (Murata et al., 1997; Sakata & Kusunoki, 1992; Taira et al., 1990). In addition, this region has very significant recurrent cortico-cortical projections with area F5 (Matelli 1994, Sakata, 1997a). Some AIP cells demonstrate specificity toward the size of the object to be grasped, while showing a certain degree of independence from the type of object; other cells demonstrate independence from the size of the object. The fact that there are cells that are both size-selective and size-independent indicates that within a population of cells that code for a particular object, a sub-population of these cells are responsible for capturing size of the object. Finally, a small number of cells shows modu -lation based upon the object's position and/or orientation in space.

Based on cytoarchitectonic and connectional criteria, Brodmann’s area 7 (inferior parietal lobule) includes areas 7a, 7b/PF and 7ip. Brodmann's area 7 reaches its highest development in primates (Cavada and Goldman-Rakic, 1989). Damage to this area can cause impairments in spatial perception, neglect of sensory stimuli contralateral to damage, and defects in visually guided reaching and oculomotor control (Stein 1991). Complementing AIP, PF receives an important input from the cortex of the superior temporal sulcus (STS) and sends its output to F5. Many neurons in PF respond to the observation of biological actions and some of them show mirror properties. This area appears to represent the link between STS, where biological actions are heavily represented, and the F5 mirror system, where an observed biological action is matched to the internal motor representation of the same action. As noted earlier, all F5 neurons are motor neurons. Some of them receive connections from AIP, others from parietal region PF, while some have no parietal input. Motor neurons with parietal con-nections from AIP and PF will acquire visual properties and become canonical or mirror neurons, respectively.

Perrett et al. (1990a,b) report neurons in the superior temporal sulcus (STS) responsive to goal directed hand motion; while Barnes and Pandya (1992) argue that neurons of multimodal areas of the STS could be concerned with analyzing the position of the organism relative to the environment. We posit that STS codes, in addition to biological motion, some phys -ical aspect of it such as velocity and direction. While there is evidence for direction coding, there are no studies that we are aware of where the effect of stimulus speed was analyzed in STS. Note however that MT and MST project to STS, thus suggesting that some of the STS neurons may code some parametric aspects of hand movements. We assume also that STS can code the distance between hand and object. There is evidence that a specific set of neurons discharge selectively during the observation of the hand approaching and coming in contact with the object. This suggests that some distance aspects may be encoded in STS neurons. Again no firm evidence on this point is available at the moment.


We briefly introduce key data on other regions of the brain relevant to the design and validation of the proposed model:

Sakata et al. (1997a,b) discovered binocular visual neurons in the lateral bank of the caudal intraparietal sulcus (cIPS) and the neighboring area V3a. These neurons respond preferentially to an object in a particular orientation in space. They also found neurons selective for a particular axis and orientation of surfaces . Intraparietal regions MIP/LIP/VIP encode the space around the animal (Colby and Goldberg 1999). Although it is hard to assign a single type of encoding for these re-gions, a rough separation is that VIP is concerned with ultra-near space (less than 5cm from the face), MIP with stimuli within reaching distance while LIP represents the space that we explore best with our eyes. Area PG of parietal cortex plays an important role in representing space (e.g., Stein 1991; Siegel and Read 1997; Andersen et al. 1999). The recent findings from Parma on mirror-like neurons in area PF of parietal cortex (Fogassi et al., 1998) and the connection of this area with the mirror neuron region of area F5 (Matelli 1994) indicates an intimate relation between PF and F5 mirror neu -rons.

The supplementary motor area (SMA) is formed by two cytoarchitectonically distinct areas, SMA-proper (F3) and pre-SMA (F6) (Luppino et al. 1991). F3 contains a complete somatotopic representation of body movements (Luppino et al. 1991). Most F3 neurons discharge in association with active movements whereas area F6 requires more complex move-ments to be activated (Matelli et al., 1993) in the human. F3 is connected strongly with F1 and F4. The F5 connection to F6 is rather modest (Luppino et.al.,1993). F6 on the other hand has strong connections to F5 and a strong input originates from prefrontal cortex (area 46). F6, unlike F3, is not connected to F1. The data suggest that F6 is responsible for higher-level motor functions, whereas F3 is more closely related to execution (Luppino et al., 1993).

In non-human primates, the ventrolateral prefrontal cortex (PFC) has a role in response inhibition, whereas the dorsolat -eral PFC is implicated in spatial processes, working memory and sequencing of behavior (Pandya and Yeterian 1998; Barone and Joseph, 1989; Jenkins et al. 1994). In monkey the dorsolateral prefrontal convexity above the principal sulcus is strongly connected with area PG, whereas the ventrolateral convexity is mainly connected with PF and 7ip (Cavada and Goldman-Rakic, 1989). In addition, both the dorsal and ventral prefrontal areas receive input from STS and SMA. PFC projects to pre-SMA. SMA-proper contains neurons representing an overall task sequence and the subsequences within the task (Tanji and Shima 1994; Tanji et al., 1996). Since pre-SMA is responsible for preparing a movement sequence, particu -larly if visually guided, PFC may be a source for the sequences that pre-SMA “knows.” We see pre-SMA and the basal ganglia as together crucial for action-sequence parsing.

2.2 Prior Modeling of the Monkey Nervous SystemOur group has a long history of models for visually directed grasping and grasp recognition (e.g., Arbib 1981; Iberall and

Arbib, 1990; Hoff and Arbib, 1992, 1993; and Jeannerod et al., 1995). Here, after a review of our modeling methodology, we present a brief summary of recent models of direct relevance to this proposal: the FARS model of monkey grasping, our first Mirror Neuron System model, and our models of the role of basal ganglia and SMA in control of compound behaviors.Modeling Methodology

We seek where possible to provide a detailed structural description of the biological neural works of brain regions rele -vant to our research. However, in some cases we "schematize" a brain region, such as the way cIPS can recognize surface orientation from visual input. This frees resources for detailed neural modeling of those regions most central to action recognition. This does not imply that we think that neural details of regions such as cIPS are irrelevant. Rather, their analy -sis is a topic whose scope requires separate funding. In any case, most of the regions here are modeled as neural networks, whether "connectionist" or "neurophysiological". Connectionist modeling replaces the schema by an adaptive artificial neural network, using "training" rather than explicit programming either to achieve context for other more detailed analy-ses or to ensure that the overall schema network meets behavioral criteria. To the extent that "training" yields output neu -rons whose response is similar to that of neurophysiologically observed neurons, the network may yield novel predictions concerning the behavior of real neurons in new conditions exercised in the modeling (see Figure 2).

Indeed, our main concern is with models which are grounded in, and can be tested by, neurophysiological data. Here, a survey of the neurophysiological and neuroanatomical data on a brain region calls attention to its basic cell types. As mod -elers we then create one array of cells for each such type. The data tell us what the activity of the cells should be in a vari -ety of situations, but in many cases experimenters do not know the action of, e.g., their synapses in any quantitative detail. Thus, the modeler has to make a number of hypotheses about weights, time constants and so on, to get the model to run. The modeler may even have to postulate cell types that experimenters have not yet looked for, and show by simulation that the resulting network will indeed perform in the observed way when known experiments are simulated, in which case it must match the external behavior and, for those populations that were based on cell populations with measured physiologi -cal responses, it must match those responses at some level of detail. What raises the ante is that the modeler's hypotheses suggest new experiments on neural dynamics and connectivity, and the model can be used to simulate experiments that have never been conducted with real nervous systems.

While much work on artificial neural networks focuses on networks of simple discrete-time neurons whose connections obey various learning rules, most work in brain theory now uses continuous-time models that represent either the variation


in average firing rate of each neuron or the time course of membrane potentials. The study of a variety of connected "com-partments" of membrane in dendrite, soma, and axon can help us understand the detailed properties of individual neurons (see, e.g., Bower and Beeman, 1998; Hines & Carnevale, 1997; Hodgkin and Huxley, 1952), but much can still be learned about the large-scale properties of interconnected regions of the brain by modeling each region as a set of subpopulations ("arrays") using the leaky integrator model for each neuron. In this case the internal state of the neuron is described by a single variable, the membrane potential m(t) at the spike initiation zone. The time evolution of m(t) is given by the differ -

ential equation = - m(t) + i wi Xi(t) + h, with resting level h, time constant , Xi(t) the firing rate at the ith input, and

wi the corresponding synaptic weight. This model does not compute spikes on an individual basis – firing when the mem -

brane potential reaches threshold – but rather defines the firing rate as a continuously varying measure of the cell's activity, approximated by a simple, sigmoid function of the membrane potential, M(t) = (m(t)). Such models can then be tested by careful comparison of histograms of firing of a cell across repeated trials against the average firing rate computed by the model for cells of this type (Figure 2). This is the level for much of the modeling proposed here, and we have developed a range of tools for simulating networks of such neurons in our Neural Simulation Language NSL (Arbib et al., 2001). The FARS Model

In our approach to reaching and grasping, we first modeled the AIP-F5canonical circuitry (the FARS model described in this section) and then proceeded therefrom to develop MNS1, our first mirror neuron system model, described in the next section. The FARS model (Fagg and Arbib 1998) analyzed the data of Sakata and Rizzolatti to show how F5 and AIP may act as part of a visuo-motor transformation circuit. The FARS model is based on the physiological properties of the circuits for transforming object intrinsic properties into grasping movements. The main components of this circuit are the AIP neu -rons and the F5 canonical and motor neurons. Gibson (1966) observed that the pattern of optic flow, or the movement of features across the retina from moment to moment, contained valuable information that could be used to guide navigation through the environment without prior recognition of objects. We thus adopted Gibson's term affordances to mean parame-ters for motor interaction that are signaled by sensory cues without invocation of high-level object recognition processes. The FARS model developed two hypotheses: (1) The anterior intra-parietal area AIP serves a dual role of computing a set of affordances appropriate for the object being attended and then maintaining an active memory of the single affordance as the corresponding grasp is executed. (2) Premotor area F5 integrates many constraints to decide on the single grasp that is to be executed. The model gave the first causal account of the firing patterns observed in AIP and F5, made novel predic -tions for response to situations involving multiple affordances, and led to Synthetic PET predictions (Arbib et al., 1995; Arbib et al., 2002) and related PET studies of humans (Grafton et al., 1998).

An important feature of the work leading to the development of the FARS model was that it involved analysis of an ex -tensive set of unpublished records of F5 and AIP cell activity made available to us by Rizzolatti and Sakata. Whereas pub -lished work showed very few records and emphasized only the correlation of cell activity with a specific type of grasp, our data analysis revealed that neurons also showed a variety of temporal relationships with different phases of the monkey's behavior. This led us both to a population model of F5 that provided both the first exploration of this rich data set, and to a convincing explanation of the data. This collaboration with experimentalists to integrate extensive data analysis with the development of new models is paradigmatic of the effort proposed here. Having offered this paradigm, we trust to the reader's powers of generalization to infer related details for each Specific Aim below.MNS1: The First Mirror Neuron System Model

The first Mirror Neuron System model (MNS1; Oztop and Arbib to appear) analyzes the role of F5 and related brain re -gions in the recognition of single actions. MNS1 analyzes pathways directed toward F5 mirror neurons which match arm-hand trajectories to the affordances and location of a potential target object. The crucial notion here was the extraction of the hand state which relates the hand to an object. In our MNS1 simulations the feature vector that serves as the hand state input is given by trajectories F(t) with 7 components d(t): distance to target at time t; v(t): tangential velocity of the wrist ; a(t): aperture of the virtual fingers involved in grasping at time t; o1(t): angle between the object axis and the (index

finger tip – thumb tip) vector; o2(t): angle between the object axis and the (index finger knuckle – thumb tip) vector; and

o3(t), o4(t): angles defining how close the thumb is to the hand as measured relative to the side of the hand and the inner

surface of the palm. The choice of the ok's was based on our earlier work on opposition spaces (Iberall et al., 1986).

MNS1 combines three "grand schemas" Visual Analysis of Hand State, Reach and Grasp, and the Core Mirror Circuit for each of which we presented a detailed implementation. With this implementation we showed how the mirror system may learn to recognize actions already in the repertoire of the F5 canonical neurons, and then studied the performance of the model under conditions of spatial perturbation, altered kinematics, and grasp object mismatch. Despite the use of artifi -cial neural networks, our MNS1 model yielded solid empirical predictions for the temporal firing patterns of mirror neu-rons, showing that neurophysiologists should analyze them in a more quantitative way with respect to observed actions. For just one example from many (Figure 2 Right), we created a scene where the observed action consisted of grasping an un -usually wide object with a precision pinch. MNS1 first recognized the action (in its early stages) as a power grasp, but as


the action progressed the modeled mirror neurons representing precision pinch became active while the power grasp activ -ity started to decline. Thus we specifically predict that in such a grasp observation experiment performed with monkeys, power-grasp related cells will fire actively but only initially, while precision-pinch cells will start firing as the power grasp activity declines. This particular prediction, like the others already developed for MNS1, is clearly testable using experi -mental setups such as those in the Rizzolatti lab. Our new modeling will yield further predictions.Modeling Sequential Behavior

Patients with diseases of the BG, particularly Huntington’s disease and Parkinson’s disease, do not have significant mo -tor control difficulties when visual input is available (Bronstein and Kennard, 1985), but do have problems with specific forms of internally driven sequences of movements (Curra et al. 1997). This implies that the basal ganglia (BG) may be in-volved in assisting cortical planning centers in some fashion as well as providing sequencing information for cross-modal movements, e.g., simultaneous arm reach, hand grasp, head movement and eye movement. Dominey and Arbib (1992) ex -plicated the interaction between cerebral cortex and basal ganglia in saccade control. They showed how thalamocortical loops could implement target memory, while remapping of retinotopic targets during saccades was conducted by circuitry intrinsic to the region LIP of posterior parietal cortex. We (Bischoff, 1998; Bischoff-Grethe and Arbib, in preparation) built on this effort to model the interactions between BG, prefrontal cortex (PFC) and supplementary motor area (pre-SMA and SMA-proper) in the control of sequences of skeletomotor actions. We modeled PFC as responsible for working memory and as a source for the sequences that pre-SMA “knows.” Pre-SMA projects sequential information both to SMA-proper and to the basal ganglia’s indirect pathway. SMA-proper is involved in the internal generation of sequences and repetitive movements. SMA-proper neurons contain information on the overall sequence to be performed; they keep track of which movement is next and which movement is currently being performed; they project the current movement to be performed to MC and to the direct pathway of the basal ganglia; and they project the preparations of the next movement of the se -quence to another population of premovement neurons within motor cortex (F1) and to the indirect (inhibitory) pathway of basal ganglia. The motor cortex, then, carries out the motor command and handles the fine tuning of the movement (e.g., target position, amplitude, force) partly based upon information provided by the basal ganglia. The motor cortex projects the motor parameters to both the brainstem and the basal ganglia’s direct pathway. We postulate that the basal ganglia’s two pathways perform two different roles: the indirect pathway inhibits upcoming motor commands from being performed while the current movement is in progress, while the direct pathway projects the next sensory state back to cortex. This in -forms SMA-proper and F1 of the expected next state and allows these regions to switch to the next movement of the se -quence.

3 Research Plan

The five modeling efforts which together define MNS2, our new mirror neuron system model, will be described under two headings, "Development of the Mirror System" and "Recognition of Novel and Compound Actions and their Context".

3.1 Development of the Mirror SystemIn Section 4.2, we will model the "mature" mirror system, making contact with a growing body of data on the role of ar -

eas like STS and PF in providing necessary input concerning hand movements and their relation to objects. However, there is immense importance in analyzing how a system develops in addition to exploring and understanding its adult structure. This topic was the focus of discussion when Arbib and Oztop visited Parma in May 2001, and the present section offers a plan of research based on this phase of our collaborative effort with Parma. To illuminate the development of so apparently complicated a system will be no small achievement. Indeed, the idea that complex cognitive functions, like action under -standing and imitation learning, can be based on simple motor schemas and a system that recognizes the actions they gen -erate offers exciting new ideas for the research on human imitation that will complement the research proposed here.

We will model three interwoven stages for mirror neuron development: a) the formation of F5 motor neurons on the ba -sis of random grasping and the haptic feedback generated by successful grips; b) the formation of F5 canonical neurons on the basis of random grasping and visual input via AIP concerning object properties; and c) the formation of F5 mirror neu -rons on the basis of self-generated goal-directed grasping movements using the association between F5 motor activity and the visual stimuli from STS and PF concerning hand movements in relation to the grasped object. This work will involve collaboration with the Schaal group on the biomechanical model of the arm-hand system and its interactions with objects.


Figure 5. The diagonal hatching marks the regions involved in the grasp learning task. F5canonical F1 Hand SII + AIP F5 is the loop that we model in this study. The horizontal-lined rectangle marks the main regions involved in the full visuo-motor transformation for grasping. These were previously "hard-wired" in the FARS model; the challenge now is to show how the circuitry could "self-organize" via the grasp learning circuit, with which it overlaps. F4 supplies the egocentric position of the object. The vertical-lined rectangle marks the main regions involved in mirror neuron functioning.

Where research to date has assumed that the mirror system arose to facilitate social understanding, it has not enquired into the possible biological roots of this function. MNS1 showed how the observation of self-action may serve as the learn -ing stimulus for shaping the mirror system but did not address the issue of why the brain might contain such learning hard -ware in the first place. We here offer a dramatically new hypothesis: that it was the need for precise visual feedback for delicate hand actions that led to the appearance of mechanisms for extracting "hand configuration", and that it is this that was readily exapted to form the bias for recognition of hand movements made by others.Development of Grasp Specificity in F5 Motor and Canonical Neurons

In modeling to date (and in the experiments), the tendency was to pair mirror neurons with an externally defined set of grasps. In the proposed modeling, we would hope to show how a basic set of grasps emerges from a repertoire of basic movements (e.g. reaching and enclosing) through learning. To model the development of the F5 motor neurons and canoni-cal neurons, we will demonstrate how somatosensory feedback can play a crucial role in defining the population of F5 mo -tor neurons, and how AIP input shapes up the F5 canonical subpopulation and is shaped up in turn, as the developing. F5 canonical neurons select visual neurons describing a variety of surfaces via re-afferent connections. Only those selected be -come AIP neurons that code affordances. The model will take visual input from a non-neural schema for cIPS which will encode surface orientation in a fashion which captures the data of Sakata et al. (1997a,b), as well as somatosensory infor -mation computed using the proposed extension of the hand-arm avatar to determine contact forces and slip when the hand encounters an object. The output will initially associate a random pattern of grasping with F5 activity but will, through learning, create a repertoire of grasp actions (precision grip, power grip etc.) that are appropriate for the objects to which the model is exposed. In particular, we expect to see the emergence of a population code in F5 for grasping actions. Our predictions on grasp population coding may lead to experiments that will complement the reach-related findings (e.g., Georgopoulos et al., 1988 for data; Lukashin et al., 1996 for modeling). The modeling will be constrained by available data on the development of reaching and grasping; the performance of the "adult" neurons of the model will be tested against data from Parma.

Based on the assumption that development of reaching and grasping involve similar processes in human and monkey, we will use studies of human infants to set behavioral goals for our grasp-learning model. Human infants are able to reach an object by around 12 weeks of age, which precedes by 3 to 4 weeks the time when the infant starts to grasp objects (Clifton et al. 1993). The fractionated control of finger movements is not possible at this stage of reflex grasping and early volun -tary grasping since this requires the cortico-motoneuronal system, which has not been developed by this stage (Lantz et al, 1996). Therefore, it is unlikely that the premotor specialisation for the different types of grasp (e.g., precision grasp, side grasp) has been formed at this age. During these 3 to 4 weeks, the motor primitives for grasping are developed, but are not properly triggered by visual stimuli. However, when the infant's hand touches the object, grasping will often triggered by somatosensory stimuli. This is due either to the innate reflex grasp or to the joy of grasping the object. The reflex grasp stays with the infant until six months of age and it takes 4 more weeks to stabilize the grasp (Clifton et al., 1993).

The first components of our MNS2 model are aimed at capturing the discovery of grasp configurations (i.e. the premotor specializations) starting from a reach capable (model) stage. The learning procedure which yields the basic population of F5 mirror neurons will adjust the connectivity of the circuits within and between F5 and F1 based on somatosensory inputs, so to encode different grasp actions, assuming that the mechanism for producing a reach to a given target in peripersonal space already exists. Through learning, the reaches directed to objects will be shaped into grasp actions via the enclosure (palmar reflex) triggered by the touch of the object to the hand. Then the haptic feedback from the fingers will be used to


determine a successful grasp. The model postulates that this information will be available in the somatosensory cortex after a grasp attempt. This area encodes information about slip and force feedback during prehension of objects (Salimi et al., 1999a,b) and is involved in sensory monitoring of hand-object interactions (Debowy et al., 2001). In the model, somatosen-sory cortex will supply the training or reinforcing signal generated by our expanded hand simulator's estimate of contact force and slippage to adjust the grasp planning circuit (F5-F1) connection strengths.

A perfect, adult-like grasp requires considerable visual analysis of the object (affordance extraction). We postulate that the visual information to the grasp learning circuit is very limited at the early phases of grasping (4-9 months) as suggested by Rosenbaum (1991): “what is most likely is that infants younger than nine months have not yet learned to pre-program grip size on the basis of visual information, just as infants younger than nine months have not yet learned to pre-program hand orientation based on vision.” Thus, augmenting the development of F5 motor neurons in general will be the develop -ment of F5 canonical neurons on the basis of AIP's recoding of the surface orientation data provided by cIPS. We will at -tempt to understand how the reciprocal connections between AIP and F5 canonical neurons enable each to shape the other so that, as the F5 canonical neurons develop, so too do AIP neurons become better adapted to convert cIPS input into affor -dance-encoding output. At the early stages of learning, the affordance representation covers only egocentric object location (this information must be present because the reach task can be performed). Based on currently available information about affordances, F5 specifies to F1, and the motor plant executes, a specific grasp action. If it turns out that the plan was suc -cessful, that is if somatosensory cortex signals a success, then the connections (cIPS AIP, AIP F5, F5 AIP) con-tributing to that decision are enhanced. If it was a failure then the connections contributing to the decision are decreased. Thus the learning we use can be called supervised anti-Hebbian or simple reinforcement learning. A minor problem is that the grasp plan for a given input need not be unique. For example, given an apple to grasp we can approach the target from any. In some sense then the circuit learns the conditional probabilities of grasp parameters given the affordance input (and other contextual biasing) using reinforcement learning.Visual Feedback for Grasping: A Possible Precursor of the Mirror Property

We will explore here the novel hypothesis that the F5 mirror neurons develop by selecting, via re-afferent connections, patterns of visual input describing those relations of hand shapes and motions to objects that prove effective in visual guid -ance of a successful grasp. Such a hypothesis will let us tap data on error correction to refine our developing model of ac -tion recognition. The validation here is computational: if the hypothesis is correct, we will be able to show that such a hand control system indeed exhibits most of the properties needed for a mirror system for grasping. For a reaching task, the sim -plest visual feedback is some form of signal of the distance between object and hand. This may suffice for grabbing ba -nanas, but for peeling a banana, feedback on the shape of the hand relative to the banana, as well as force feedback become crucial. We predict that the parameters needed for such visual feedback for grasp will look very much like those we speci -fied explicitly for our MNS1 hand state.

We will model superior temporal sulcus (STS) and PF as providing crucial inputs for the premotor mirror system. The result will be a neural representation of the observed scene to be processed by other regions. The modeling will address available data, but also feed into the design of new experiments. For example, Jellema et al. (2000) found cells in anterior STS (STSa) which respond selectively to the sight of reaching but only when the agent performing the action is seen to be attending to the target position of the reaching. One cell population responds selectively to faces, eye gaze, and body pos -ture, and they argue that subsets of these cells code for the direction of attention of others. A second cell population is se -lectively responsive to limb movement in certain directions. The responses of a subset of cells sensitive to limb movement are modulated by the direction of attention (indicated by head and body posture of the agent performing the action). The Parma group plans to record from monkey STS with multiple electrodes to understand better its organization (somatotopy, motor properties, movement vs. action).1 The recent findings from Parma on mirror-like neurons in area PF of parietal cor-tex (Fogassi et al., 1998) and the connection of this area with the mirror neuron region of area F5 (Matelli 1994) indicates an intimate relation between PF and F5 mirror neurons. We propose that PF mirror neurons provide crucial input for F5 mirror neurons. The similarity of STS and PF responses to active hands, combined with the connectivity pattern of superior temporal sulcus and area 7 makes the STS-PF circuit a plausible approximation to the primate hand shape-motion recogni -tion circuit. Fogassi is studying the mouth field and will continue his PF studies.

1 The Parma group will also study imitation learning in humans, with the idea that the efferent copy of a movement activates STS (as shown by preliminary data with our HFSP colleague Iacoboni at UCLA), and that STS (and/or PF) compares the observed action with the efferent copy of the action, thus allowing the match-ing necessary to learn new actions. We will explore similar ideas in our analysis of the monkey data as a basis for work on human imitation under other funding.


Figure 6. The paired forward-in-verse model view of the F5 grasp control circuit of Arbib and Rizzolatti (1997). The vertical path at right is the execution system from "view of object" via AIP and F5 to "grasp of object" it provides mechanisms for grasping a seen object. The loop on the left of the figure provides mech-anisms for imitating observed ges-tures in such a way as to create ex-pectations which enable the visual feedback loop to serve both for "so-cial learning" (i.e., learning an action through imitation of the actions of others) and also for (delayed) error correction during, e.g., reaching to-wards a target. It combines the ob-servation matching system (inverse model) from "view of gesture" via gesture description (STS) and ges-ture recognition (PF) to a represen-tation of the "command" for such a gesture, and the expectation system (direct model) from an F5 command via the expectation neural network ENN to MP, the motor program for generating a given gesture. The lat-ter path may mediate a comparison between "expected gesture" and "observed gesture" for the monkey’s self-generated movements.

Arbib and Rizzolatti (1997) gave an informal view of how the mirror system might be viewed as pairing a forward and inverse model (Figure 6). This approach was inspired by the work of Jordan and Rumelhart (1992) who proposed a "distal learning" scheme wherein the forward model (a neural circuit predicting the outcome of a given motor command) is trained first, and then the inverse model (a neural circuit computing the command required to achieve a desired outcome) is trained using the error that is propagated backward through the forward model. Leaving aside Arbib and Rizzolatti's in -terest in imitation for the present proposal, we now propose formal neural modeling to focus on the role of error correction in the development of the mirror system. This effort will take into account recent findings (Wolpert and Kawato, 1998; Haruno et al. 2001) suggesting the utility of multiple paired forward and inverse models in motor control it may be easier to combine controllers specialized for specific subtasks to meet the needs of a wide variety of tasks rather than to have a single super-complex controller which can handle every task directly.

Our earlier model of the mirror neuron system, MNS1 actually modeled the forward model of Figure 6 where the output of visual processing trained the mirror neurons. For MNS1 we did not model the low level motor control and learning of the inverse model but used the Grasp Simulator to perform the desired grasp (the lower row of the figure). Now we propose to implement full learning in the forward model and inverse model with the sole goal of accomplishing grasp control. Then what we predict is that output units of the forward model will be armed with mirror property while the output units of the inverse model will attain the F5 canonical property. The muscimol study of Fogassi et al. (2001) strongly supports our pro -posal. The inactivation of mirror neurons does not abolish grasping but only slows down the actions. However, the inacti -vation of canonical neurons heavily degrades the grasping performance in terms of preshaping and orienting the hand. As can be seen in Figure 6, in the proposed model the inverse model generates motor programs, thus the destruction of this structure will abolish the motor output of the model while the destruction of the forward model will not have an effect in the short term. In the long term, assuming learning still continues, it will disrupt the inverse model by channeling wrong er -ror signals. Just as we propose to analyze plasticity of the connections cIPS AIP, AIP F5, and F5 AIP in under-standing the tuning of AIP to more accurately encode affordances for F5 canonical neurons, so too will we use the present study to better understand the tuning of STS and PF in providing input to the F5 mirror neurons. Implementation of the model will enable us to make more detailed predictions on the relation between F5 canonical and mirror neurons.

3.2 Recognition of Novel and Compound Actions and their ContextThe modeling of development defined above emphasizes how the infant monkey builds a basic motor repertoire of

reach-and-grasp actions and how the infant comes to recognize hand-object relations in other monkeys which signal similar


actions. In the present section we propose models for the recognition of novel actions, presenting hypotheses for: (i) How a variant of a known action comes to be recognized. (ii) How a novel action may be recognized as a compound of (variants of) known actions. (iii) How actions are "understood". We argue that this will in general involve more than recognition of the action (movement + goal) in isolation, but will also involve recognition of the context in which the action occurs and expectations as to the consequences of that action.The Pliers Experiment: Extending the Visual Vocabulary

When a monkey first watched the experimenter grasping a raisin with a pair of pliers, no mirror neurons discharged, but after several demonstrations, some of the previously silent neurons started to fire when the pliers were approaching the raisin (G. Rizzolatti, personal communication). We will analyze this as part of a more general study of how an action comes to be recognized when it is a variant of a known action. Experimental studies to date have emphasized the actions (both observed and executed) that are encoded by a single mirror neuron. However, many neurons are active for each type of grasp, and we thus argue that it is more appropriate to see a set of neurons providing a nuanced representation of a grasp rather than focusing on the very broadly tuned response of a single neuron. A key aspect of this modeling will thus be a fo -cus on population coding which will be strongly linked with the increasing push of the Parma group to use multi-electrode recording. We expect to show that population coding is an emergent property from our modeling of development and learning in the mirror system that once one has a variety of grasps, cells will be more or less correlated with distinctive subsets of mirror neurons. We further predict that learning a variation on a movement can be done more efficiently by building on the population code for the original movement and others that have already been learned than by training dedi -cated "grandmother" cells, and we will model this phenomenon to provide explicit predictions. Building upon the pliers study, we will explore the hypothesis that the mirror system can (a) recognize an action as similar to a known action while (b) delivering a crude analysis of the difference between the observed motion and this approximation as the means for rec -ognizing a new class of actions. Such capabilities should be facilitated by population coding in the mirror neuron system. The activity of the population will either indicate a confident recognition of an action or a representation of how different the action is from the ones in the action repertoire (i.e., departure from the projection onto the basis vectors formed by rec-ognizable actions). The representation of this difference could lead to extension of the basis to accommodate novel features of the new action. In either case, we have to study what is a good representation, computationally and biologically , to give the system the flexibility that we observe in monkey mirror neurons.

We have already noted in our discussion of the FARS model that our analysis of unpublished data "revealed that neurons … showed a variety of temporal relationships with different phases of the monkey's behavior." Our modeling will thus make predictions to be tested by timing studies in answer to questions such as: How soon after a movement has begun will a mirror neuron become active? Such temporal relations will be tested using proposed new features of NeuroBench to ana -lyze Parma multi-electrode data for temporal patterns and for population coding across neurons.

Grammont (Parma) has trained monkeys to grasp with tools. One of his tools (“escargot device”) grasps an object when the monkey opens the hand. The issue is whether the action is coded by F5 neurons in abstract terms (grasp) or in terms of movements when action and movements are in opposite directions. He will also examine the mirror properties of the tool-using monkey after tool learning. To model this, we must show how the brain develops an extended hand configuration representation which includes such extensions as a hand holding a tool. In particular, we must update visual input process -ing in our models. Although we had completed encouraging studies of the recognition of hand state from photographs of human hands (with colored patches to aid segmentation), our MNS1 work used a quasi-neural encoding of the positions of arms and hands relative to objects based on the hand state from the reach-grasp simulator as input. But this will not work for modeling the pliers experiment. Thus a subgoal of this work will be to develop a more generic vision system to recog -nize a hand holding a tool, and then combine this with affordance data to recognize extensions of hand configuration. It would be very useful to have a vision system that can work on real video data and extract the information required for the circuits we model, but modeling the actual biological circuitry (e.g., that of inferotemporal cortex) is in itself a major re -search project beyond the scope of the present proposal.. Nevertheless there are successful vision studies from which we can benefit and which can be adapted for useful integration with our models to obtain structurally complete systems. We plan to accomplish such a task using Arbib's collaboration with Dr. E.J. Holden and Professor Robyn Owens of the Univer-sity of Western Australia who have worked on vision algorithms for the recognition of hand, head and lip movements (Holden et al., 1996, 2000; Holden and Owens, 2000, 2001). Recognition of Compounds of Known Movements

Our prior modeling of "compound actions" (see "Modeling Sequential Behavior") has focused on sequences of known actions, whether saccades or arm movements. Indeed, Byrne (in press) suggests that a novel action can be imitated (and so, a fortiori, recognized) by dissecting it into a string of simpler sequential parts that are already in the observer’s repertoire. Rizzolatti et al. (to appear) comment that Byrne's analysis "opens new empirical possibilities and may clarify why only hu -mans and some species of primates appear to be able to imitate in the proper sense." However, Arbib (2000) has already observed that imitation is far more subtle than Byrne suggests. Figure 7 offers a stripped-down example of an alternative view of the recognition of novel actions. This open up even more "new empirical possibilities."


Figure 7. This diagram makes the point that the recogni-tion of an action need not reduce to recognition of a se-quence of familiar actions but may instead involve in-creasing success in approximation as details are attended to. In this case, the movement at top is first recognized as a linear movement from start to finish, then as a move-ment with a "bump" in the middle, and it is only after this second approximation has been learned that the loop can be properly attended to and recognized. The action is then recognized as a temporally coordinated superposi-tion of movements, rather than a sequence of known ac-tions.

We will extend our analysis of population coding of single actions to model learning and recognition of compounds. We will first extend our earlier work on the interaction of basal ganglia (BG) and supplementary motor area (SMA) working memory and sequences of movement (e.g., Dominey and Arbib, 1992; Bischoff-Grethe and Arbib, in preparation) to de -velop hypotheses on how BG and SMA interact with the mirror system so that temporal sequences ("the Byrne case") can be extracted, stored and learned. We will then generalize this approach to handle the recognition of novel actions that are formed as temporally coordinated superpositions, rather than sequences, of known actions. This modeling effort will pro -vide a conceptual analysis that sets the stage for design of experiments several years in the future. "To see which is the best (or possible) solution in terms of modeling can help a lot in terms of future experiments" (Rizzolatti, personal communica -tion). However, the model will contain components that will be richly constrained by available neurophysiological data (e.g., Tanji and Shima 1994; Tanji et al., 1996). We also note that our USC colleague Aude Billard has developed a learn -ing mechanism for the recognition of variants and compounds of known movements in robots and humans (Billard, 2000a, b; Billard and Hayes, 1999; Billard and Mataric, 2000) so that comparison of our work on the monkey mirror system with her work on mechanisms of human imitation will enrich the research proposed here.From Action Recognition to Understanding: Context and Expectation

In one Parma study of PF, 61 cells were responsive when the monkey observed biological actions, and 2/3 of these were also active during the monkey's own actions. However, about a quarter of these “PF mirror neurons” do not match ob-served actions to congruent executed actions. For example, a cell active for observation of downward motion of the hand when grasping an object may also be active during execution of grasping by mouth. At first this may seem counter to the notion of a mirror neuron but for us it sets the stage for a deeper analysis. It has often been said that mirror neurons are in -volved in "understanding" of actions, but for the present effort we stress that understanding will in general involve more than the recognition of an action in isolation, and may also involve some notion of "meaning", e.g., the context in which the action is appropriate and the expectations that such a behavior evokes. This relates to our earlier work on Schema-Based Learning (Corbacho, 1997). Thus where the previous Aim focused on the recognition of compound actions, the present Aim emphasizes the recognition of context and expectations, where, for example, recognition of one action may be seen as a preliminary for either doing something or predicting what the observed primate will do next (e.g., bringing food to the mouth to eat). The context and expectations set the stage for action recognition, action recognition modifies the con-text and expectations, and so on. This will let us explore the notion that mirror neurons can act as the basis for "understand-ing" if a given action can be placed in the context of its observed (in self and/or others) consequences.

The modeling here will build on recent work (Suri, Bargas and Arbib, 2001; Suri and Schultz 1999, 2001) to extend the Temporal Difference (TD) Model (Sutton and Barto 1998; Doya 1996, 1999) of reinforcement learning. It will also build on our earlier efforts relating the basal ganglia to various types of learning (Dominey, Arbib and Joseph, 1995). The TD Model reproduces reward-predictive aspects of dopaminergic activity (Apicella et al., 1992, Montague et al. 1996, Schultz et al. 1993, 1997) but, in contrast to the proposed modeling, cannot reproduce predictive neural activity discriminating be -tween events. Such neural activity was reported in several studies (Hikosaka et al. 1989, Duhamel 1992, Kermadi and Joseph 1995, Watanabe 1996) relevant to the work proposed here. We also note the finding of Mauritz and Wise (1986) of neuronal activity in rhesus premotor cortex anticipating predictable environmental events, and of Romo and Schultz (1992) of the relation between basal ganglia and supplementary motor area in the internal generation of movements. Suri et al. (2001) showed that the capability for planning is improved by influences of dopamine on the durations of membrane poten-tial fluctuations and by manipulations that prolong the reaction time of the model, suggesting that responses of dopamine neurons to conditioned stimuli contribute to sensorimotor reward learning, that novelty responses of dopamine neurons stimulate exploration, and that transient dopamine membrane effects are important for planning – all factors of relevance in explicating the role of basal ganglia in recognition of novel compound actions. In Model Predictive Control (Garcia et al., 1989), predictions for future sequences of control actions are computed and then the actions are selected which opti -mize the outcome. Showing how neural activity might reflect variables of such control algorithms will help to bridge the gap between these brain processes and well-known control algorithms.


References

Andersen R.A., Shenoy K.V., Snyder L.H., Bradley D.C., Crowell J.A., 1999, The contributions of vestibular signals to the representations of space in the posterior parietal cortex. Ann NY Academy of Science, 871: 282-292

Apicella, P., Scarnati, E., Ljungberg, T. and Schultz, W. , 1992, Neuronal activity in monkey striatum related to the expec -tation of predictable environmental events. J. Neurophysiol. 68:945-960.

Arbib MA, Bischoff A, Fagg A, Grafton S (1995). Synthetic PET: Analyzing Large-Scale Properties of Neural Networks. Human Brain Mapping 2: 225-33.

Arbib, M. A., 1981, Perceptual Structures and Distributed Motor Control, in Handbook of Physiology, Section 2: The Ner-vous System, Vol. II, Motor Control, Part 1 (V. B. Brooks, Ed. ), American Physiological Society , pp. 1449-1480.

Arbib, M., and Rizzolatti, G., 1997, Neural expectations: a possible evolutionary path from manual skills to language, Communication and Cognition, 29, 393-424.

Arbib, M.A., 2000, The Mirror System, Imitation, and the Evolution of Language, in Imitation in Animals and Artifacts, (Chrystopher Nehaniv and Kerstin Dautenhahn, Editors), The MIT Press (in press).

Arbib, M.A., Alexander, A., and Weitzenfeld, W., 2001, NSL Neural Simulation Language, in Computing the Brain: A Guide to Neuroinformatics, (M. A. Arbib, and J.S. Grethe, Eds.), San Diego: Academic Press, pp.71-90.

Arbib, M.A., and Bischoff-Grethe, A., 2001, Summary Databases and Model Repositories, in Computing the Brain: A Guide to Neuroinformatics, (M. A. Arbib, and J.S. Grethe, Eds.), San Diego: Academic Press, pp.287-296.

Arbib, M.A., Billard, A., Iacoboni, M., and Oztop, E., 2000, Synthetic Brain Imaging: Grasping, Mirror Neurons and Imita-tion, Neural Networks, 13: 975-997.

Arbib, M.A., Fagg, A.H., and Grafton, S.T., 2002, Synthetic PET Imaging for Grasping: From Primate Neurophysiology to Human Behavior, in Explorative analysis and data modelling in functional neuroimaging, (F. Sommer and A. Wichert, Eds.), Cambridge MA: The MIT Press (to appear).

Barnes C.L., Pandya D.N., 1992, Efferent Cortical Connections Of Multimodal Cortex of the Superior Temporal Sulcus In The Rhesus-Monkey. Journal of Comparative Neurology, 318: (2) 222-244

Barone, P. and Joseph, J.-P., 1989, Prefrontal cortex and spatial sequencing in macaque monkey, Exp Br Res 78:447-464.

Barto, A. G., S. Richard and C. W. Anderson (1983). Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Trans. on Systems, Man, and Cybernetics SMC-13: 834-846.

Billard, A, 2000a, Imitation: a means to enhance learning of a synthetic proto-language in an autonomous robot. In C. Ne -haniv and K. Dautenhahn, editors, Imitation in Animals and Artifacts. MIT Press, to appear.

Billard, A., 2000b, Learning motor skills by imitation: a biologically inspired robotic model . Cybernetics & Systems Jour-nal, special issue on Imitation in animals and artifacts, to appear.

Billard, A., and Hayes, G., 1999, Drama, a connectionist architecture for control and learning in autonomous robots. Adap-tive Behavior, 7:35-64.

Billard, A., and Mataric, M., 2000, Learning human arm movements by imitation: Evaluation of a biologically-inspired connectionist architecture. In Proceedings, First IEEE-RAS International Conference on Humanoid Robotics (Hu-manoids-2000), MIT, Cambridge, MA, Sep 7-8, 2000.

Bischoff, A., 1998, Modeling the Basal Ganglia in the Control of Arm Movements, Ph.D. Thesis (May 1998), Department of Computer Science, University of Southern California.

Bischoff-Grethe, A., and M.A. Arbib, Sequential movements: A computational model of the roles of the basal ganglia and the supplementary motor area, (in preparation).

Bota, M., 2001, Neural homologies: principles, databases and models, University of Southern California, Ph.D. Thesis in Neuroscience (to appear).

Bota, M., 2001a, The NeuroHomology Database Version I, NHDB-I,http://bsl9.usc.edu/scripts/webmerger.exe?/database/homologies-main.html.

Bota, M., 2001a, The NeuroHomology Database Version I, NHDB-II, http://java.usc.edu/neurohomologies/apb/webdriver?MIval=homologies-main.

Bota, M., and Arbib, M.A., 2001, The Neurohomology Database, in Computing the Brain: A Guide to Neuroinformatics, (M.A. Arbib, and J.S. Grethe, Eds.), San Diego: Academic Press, pp.337-351.

Bower, J.M., and Beeman, D., 1998, The Book of GENESIS: Exploring Realistic Neural Models with the GEneral NEural SImulation Systems, 2nd edition, TELOS/Springer-Verlag.

Bronstein, A.M., and C. Kennard, Predictive ocular motor control in Parkinson's disease , Brain. 108:925-940 (1985).

Byrne, R.W. (in press). Imitation in action. Advances in the study of behaviour.

http://java.usc.edu/neurohomologies/apb/webdriver?MIval=homologies-main

http://bsl9.usc.edu/scripts/webmerger.exe?/database/homologies-main.html


Cavada C., Goldman-Rakic P.S., 1989, Posterior parietal cortex in Rhesus monkey: II. Evidence for segregated corticocor -tical networks linking sensory and limbic areas with the frontal lobe. Journal of Comparative Neurology 287:422-445

Clifton, R.K., W.M. Darwin, D.H. Ashmead, M.G. Clarkson, 1993, Is Visually Guided Reaching in Early Infancy a Myth? Child Development, 64: 1099-1110.

Colby, C. L., Goldberg, M. E., 1999. Space and Attention in Parietal Cortex. Annu. Rev. Neurosci. 22:319-49.

Corbacho, F., 1997, Schema Based Learning: Towards a Theory of Organization for Adaptive Autonomous Agents, Ph.D. thesis, Computer Science, University of Southern California, August 1997.

Curra, A., A. Berardelli, R. Agostino, N. Modugno, C.C. Puorger, N. Accornero, and M. Manfredi, Performance of sequen -tial arm movements with and without advance knowledge of motor pathways in Parkinson's disease , Mov Disord. 12:646-654 (1997).

Debowy DJ, Ghosh S, Ro JY, Gardner EP, 2001, Comparison of neuronal firing rates in somatosensory and posterior pari -etal cortex during prehension. Exp Brain Res 137: 269-291

di Pellegrino, G., L. Fadiga, L. Fogassi, V. Gallese, and G. Rizzolatti, 1992. Understanding motor events: a neurophysio -logical study. Experimental Brain Research, 91:176-180.

Dominey, P. F., and Arbib, M. A., 1992, A Cortico-Subcortical Model for Generation of Spatially Accurate Sequential Saccades, Cerebral Cortex, 2:153-175.

Dominey, P.F., Arbib, M.A., and Joseph, J.-P., 1995, A Model of Corticostriatal Plasticity for Learning Associations and Sequences, J. Cog. Neurosci., 7:311-336.

Doya, K., 1996, Temporal difference learning in continuous time and space, in Advances in Neural Information Processing Systems (Touretzky, D. S., Mozer, M. C., and Hasselmo, M. E., editors), The MIT Press, 8:1073-1079.

Doya, K., 1999, Reinforcement learning in continuous time and space. Neural Computation, 12:243-269.

Duhamel JR, Colby CL, Goldberg ME, 1992, The updating of the representation of visual space in parietal cortex by in -tended eye movements. Science 255:90-92.

Fagg, A. H., and Arbib, M. A., 1998, Modeling Parietal-Premotor Interactions in Primate Control of Grasping, Neural Net-works, 11:1277-1303.

Fogassi L., Gallese V, Buccino G, Craighero L, Fadiga L, Rizzolatti G., 2001, Cortical mechanism for the visual guidance of hand grasping movements in the monkey. Vol. 124, No. 3: 571-586

Fogassi L., Gallese V., Fadiga L., Rizzolatti G., 1998, Neurons responding to the sight of goal-directed hand/arm actions in the parietal area PF of the macaque monkey. 28th Annual Meeting of Society for Neuroscience.

Gallese V., Fadiga L., Fogassi L., Rizzolatti G., 1996, Action recognition in the premotor cortex. Brain, 119: 592-609

Garcia CE, Prett DM and Morari M., 1989, Model Predictive Control: Theory and Practice-a survey. Automatica, 25:335-348.

Gentilucci M., Fogassi L., Luppino G., Matelli M., Camarda R., Rizzolatti G., 1988, Functional Organization of Inferior Area 6 in the Macaque Monkey I. Somatotopy and the Control of Proximal Movements. Experimental Brain Research, 71: 475-490.

Georgopoulos, A. P., Kettner, R. E., and Schwartz, A. B., 1988, Primate motor cortex and free arm movements to visual targets in three-dimensional space, II: Coding of the direction of movement by a neuronal population, J. Neurosci., 8:2928-2937.

Gibson, J. J., 1966, The Senses Considered as Perceptual Systems, Allen and Unwin.

Grafton, S. T., Fagg, A. H., & Arbib, M. A., 1998, Dorsal Premotor Cortex and Conditional Movement Selection: A PET Functional Mapping Study. Journal of Neurophysiology 79:1092-1097

Grafton, S.T., Arbib, M.A., Fadiga, L., and Rizzolatti, G., 1996, Localization of grasp representations in humans by PET: 2. Observation compared with imagination. Experiment Brain Res. 112:103-111.

Grafton, S.T., Fadiga, L., Arbib, M.A., and Rizzolatti, G., 1997, Premotor Cortex Activation during Observation and Nam -ing of Familiar Tools, NeuroImage, 6:231-236.

Haruno M, Wolpert DM, Kawato M, 2001, Multiple paired forward-inverse models for sensorimotor learning and control. Neural Computation, in press

Hikosaka O, Sakamoto M, Usui S, 1989, Functional properties of monkey caudate neurons. III. Activities related to expec -tation of target and reward. J Neurophysiol 4(4)814-832.

Hines, M.L. & Carnevale N.T., 1997, The NEURON simulation environment, Neural Computation, 9:1179-1209.

Hodgkin, A.L., and A.F. Huxley, 1952, A quantitative description of membrane current and its application to conduction and excitation in nerve, J. Physiol. Lond., 117:500 – 544.


Hoff, B., and Arbib, M. A., 1992, A model of the effects of speed, accuracy, and perturbation on visually guided reaching, in Control of Arm Movement in Space: Neurophysiological and Computational Approaches , (R. Caminiti, P. B. John-son, and Y. Burnod, Eds. ), Experimental Brain Research Series 22, pp. 285-306.

Hoff, B., and Arbib, M. A., 1993, Simulation of Interaction of Hand Transport and Preshape During Visually Guided Reaching to Perturbed Targets, J . Motor Behav. 25: 175-192.

Holden, E. J., Roy, G.G., Owens, R., 1996, Hand movement classification using an adaptive fuzzy expert system Interna -tional Journal of Expert Systems V9(4): 465-480.

Holden, E.J. and Owens, R., 2000, Visual Speech Recognition using Cepstral Images Proceedings of the IASTED Interna-tional Conference on Signal and Image Processing, pp.331-336.

Holden, E.J. and Owens, R., 2001, Visual Sign Language Recognition Multi-Image Search and Analysis (Klette, R., Huang T., and Gimel'farb, G., Eds), Lecture Notes in Computer Science, Springer (to appear).

Holden, E.J., Loy, G., and Owens, R., 2000, Accommodating for 3D head movement in visual lipreading. Proceedings of the IASTED International Conference on Signal and Image Processing, pp.166-171.

Iacoboni M, Woods RP, Brass M, Bekkering H, Mazziotta JC, Rizzolatti G., 1999, Cortical mechanisms of human imita -tion. Science 286:2526-8.

Iberall T., Arbib M.A., 1990, Schemas for the Control of Hand Movements: An Essay on Cortical Localization. In Goodale M.A., editor. Vision and action: the control of grasping. Norwood, NJ: Ablex, 163-180.

Iberall, T., Bingham, G., and Arbib, M.A., 1986, Opposition Space as a Structuring Concept for the Analysis of Skilled Hand Movements, Experimental Brain Res. Series, 15:158-173.

Jeannerod, M. , Arbib, M. A. , Rizzolatti, G. , and Sakata, H. , 1995, Grasping objects: the cortical mechanisms of visuo -motor transformation, Trends in Neurosciences, 18:314-320.

Jellema T, Baker CI, Wicker B, Perrett DI, 2000, Neural representation for the perception of the intentionality of actions. Brain Cogn 44:280-302

Jenkins, I.H., D.J. Brooks, P.D. Nixon, R.S.J. Frackowiak, and R.E. Passingham, Motor sequence learning: A study with positron emission tomography, J Neurosci. 14:3775-3790 (1994).

Jordan MI, Rumelhart DE, 1992, Forward models: supervised learning with distal teacher. Cognitive Science 16: 307-354

Kermadi I, Joseph JP, 1995, Activity in the caudate nucleus of monkey during spatial sequencing. J Neurophysiol 74: 911-933.

Lantz, C., K. Melen, H. Forssberg, 1996, Early infant grasping involves radial fingers, Developmental Medicine and Child Neurology, 38: 668-674.

Lukashin, A. V., Amirikian, B. R., and Georgopoulos, A. P., 1996, Neural computations underlying the exertion of force: a model, Biol. Cybern., 74:469-478.

Luppino G., Matelli M., Camarda R.M., Gallese V., Rizzolatti G., 1991, Multiple representations of body movements in mesial area 6 and the adjacent cingulate cortex: an intracortical microstimulation study in the macaque monkey. Journal of Comparative Neurology. 311:463-482

Luppino G., Matelli M., Camarda R.M., Rizzolatti G., 1993, Corticocortical connections of area F3 (SMA-proper) and area F6 (pre-SMA) in the macaque monkey. Journal of Comparative Neurology. 338(1):114-140.

Maioli M.G., Squatrito S., Samolsky-Dekel B.G. and Sanseverino E.R.(1998) Corticocortical connections between frontal periarcuate regions and visual areas of the superior temporal sulcus and the adjoining inferior parietal lobule in the macaque monkey. Brain research, 798: 118-125.

Matelli M., Camarda R., Glickstein M., Rizzolatti G., 1985, Afferent and efferent projections of the inferior area 6 in the macaque monkey. Journal of Comparative Neurology, 251: 281-298

Matelli M., Luppino G., Murata A., Sakata H., 1994, Independent anatomical circuits for reaching and grasping linking the inferior parietal sulcus and inferior area 6 in macaque monkey. Soc. Neurosci. Abstr. 20:404.4

Mauritz KH, Wise SP, 1986, Premotor cortex of the rhesus monkey: neuronal activity in anticipation of predictable envi -ronmental events. Exp Brain Res 61 (2): 229-244.

Montague, P.R., Dayan, P. and Sejnowski, T.J., 1996, A Framework for mesencephalic dopamine systems based on predic -tive hebbian learning. J Neurosci 16(5):1936-1947.

Murata A., Fadiga L., Fogassi L., Gallese V., Raos V., Rizzolatti G., 1997, Object representation in the ventral premotor cortex (Area F5) of the monkey. Journal of Neurophysiology, 78: 2226-2230

Nishitani, N., and Hari R, 2000, Temporal dynamics of cortical representation for action. Proceedings of the National Academy of Sciences of the United States of America. 97(2: 913-918.

http://gateway2.ovid.com/ovidweb.cgi?S=IDNJHKEKBCHKJM00D&Search+Link=%22Hari+R%22.au.

http://gateway2.ovid.com/ovidweb.cgi?S=IDNJHKEKBCHKJM00D&Search+Link=%22Nishitani+N+%22.au.


Oztop, E., and Arbib, M.A., to appear, Schema Design and Implementation of the Grasp-Related Mirror Neuron System, Biological Cybernetics.

Pandya, D.N., Yeterian, E.H., 1998, Comparison of prefrontal architecture and connections. In The Prefrontal Cortex, Ex-ecutive and Cognitive Functions, Eds. Roberts, A.C., Robbins, T.W., Weiskrantz, L. Oxford University Press.

Perrett D.I., Harries M.H., Benson P.J., Chitty A.J. and Mistlin, A.J., 1990a, Retrieval of structure from rigid and biologi -cal motion: an analysis of the visual responses of neurons in the macaque temporal cortex. In Blake A. and Troscianko T.(eds.) AI and the eye. John Wiley and Sons Ltd., 181-199,

Perrett D.I., Mistlin A.J., Harries M.H., Chitty A.J., 1990b, Understanding the visual appearance and consequence of hand actions. In Goodale M.A., editor. Vision and action: the control of grasping. Norwood, NJ: Ablex, 163-180.

Petrides M. and Pandya D. N. (1984) Projections of the frontal cortex from the posterior parietal region in the rhesus mon -key. Journal of Comparative Neurology, 228: 105-116.

Ramachandran, V., http://www.feedmag.com/brain/parts/ramachandran.html, in the internet magazine Feed.

Rizzolatti G., Camarda R., Fogassi L., Gentilucci M., Luppino G., Matelli M., 1988, Functional Organization of Inferior Area 6 in the Macaque Monkey II. Area F5 and the Control of Distal Movements. Experimental Brain Research, 71: 491-507.

Rizzolatti G., Luppino G., Matelli M., 1998, The organization of the cortical motor system: new concepts. Electroen -cephalography and clinical Neurophysiology 106: 283-296

Rizzolatti, G., and Arbib, M.A., 1998, Language Within Our Grasp, Trends in Neuroscience, 21(5):188-194.

Rizzolatti, G., Craighero, L., and Fadiga, L., to appear, The mirror system in humans.

Rizzolatti, G., L. Fadiga, V. Gallese, and L. Fogassi. Premotor cortex and the recognition of motor actions. Cognitive Brain Research, 3:131-141, 1996.

Romo, R. and Schultz, W., 1992, Role of primate basal ganglia and frontal cortex in the internal generation of movements. iii. neuronal activity in the supplementary motor area. Exp.Brain Res. 91:396-407.

Rosenbaum, D. A., 1991, Human motor control. Academic Press.

Sakata H, Taira M, Murata A, Mine S., 1995, Neural mechanisms of visual guidance of hand action in the parietal cortex of the monkey. Cereb Cortex 5(5): 429-38.

Sakata H., Taira M., Kusunoki M., Murata A., Tanaka Y., 1997a, The parietal association cortex in depth perception and visual control of action. Trends in Neuroscience, 20: 350-357

Sakata H., Taira M., Murata A., Gallese V., Tanaka Y., Shikata E., Kusunoki M., 1997b, Parietal Visual Neurons Coding Three-Dimensional Characteristics of Objects and Their relation to Hand Action, In Parietal Lobe Contributions to Ori-entation in 3D Space. Their P. and Karnath H.O. (Eds.). Springer-Verlag, Heidelberg.

Salimi I, Brochier T, Smith AM, 1999a, Neuronal activity in somatosensory cortex of monkeys using a precision grip. I. Receptive Fields and Discharge Patterns. J Neurophysiol 81:825-834

Salimi I, Brochier T, Smith AM, 1999b, Neuronal activity in somatosensory cortex of monkeys using a precision grip. III Responses to altered friction perturbations. J Neurophysiol 81:845-857

Schaal, S., Learning from demonstration, 1997, M.C. Mozer and M. Jordan and T. Petsche (Eds), Advances in Neural In-formation Processing Systems, 9, 1040-1046, Cambridge, MA: MIT Press.

Schultz W, Dayan P, Montague PR, 1997, A neural substrate of prediction and reward. Science 275:1593-1599.

Schultz, W., Apicella, P. and Ljungberg, T., 1993, Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J.Neurosci. 13: 900-913.

Schultz, W., R. Romo, T. Ljungberg, J. Mirenowicz, J. R. Hollerman and A. Dickinson (1995). Reward-related signals car -ried by dopamine neurons. In J. R. Houk et al. (Eds.), Models of information processing in the basal ganglia. Cam-bridge, MA, The MIT Press. 233-248.

Siegel R.M., Read H.L., 1997, Analysis of optic flow in the monkey parietal area PG. Cereb Cortex, 7: (4) 327-346

Stein J.F., 1991, Space and the parietal association areas. In Paillard J. Editor. Brain and Space, Oxford University Press, chapter 11.

Suri, R.E., Bargas, J., and Arbib, M.A., 2001, Modeling Functions of Striatal Dopamine Modulation in Learning and Plan -ning, Neuroscience, 103:65-85.

Suri RE, Schultz W. 1999, A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience, 91:871-90.

Suri RE, Schultz W. 2001, Temporal difference model reproduces anticipatory neural activity, Neural Comput. 13:841-62.

Sutton RS and Barto AG, 1998, Reinforcement learning, an introduction. MIT Press/Bradford Books, Cambridge, MA,

http://www.feedmag.com/brain/parts/ramachandran.html


Sutton, R. (1988). Learning to predict by the methods of temporal differences. Machine Learning 3: 9-44.

Sutton, R.S., Barto, A.G. 1990, Time derivative models of Pavlovian reinforcement. In: Learning and computational neu-roscience: Foundations of adaptive networks (eds Gabriel M. and Moore. J.) MIT Press, Cambridge: pp.539-602.

Taira M., Mine S., Georgopoulos A. P., Murata A., Sakata H., 1990, Parietal Cortex Neurons of the Monkey Related to the Visual Guidance of Hand Movement. Experimental Brain Research, 83: 29-36.

Tanji, J., and Shima, K., 1994, Role for supplementary motor area cells in planning several movements ahead , Nature. 371:413-416.

Tanji, J., Shima, K., Mushiake, H., 1996, Multiple cortical motor areas and temporal sequencing of movements. Cognitive Brain Research, 5: 117-122

Watanabe M, 1996, Reward expectancy in primate prefrontal neurons. Nature 382(6592):629-632.

Wolpert DM, Kawato M, 1998, Multiple paired forward and inverse models for motor control. Neural Networks 11:1317-1329.

Introduction.doc

Documents

Transcript of Introduction.doc