Darpa Baa Synapse
-
Upload
vasanthi-challa -
Category
Documents
-
view
98 -
download
7
Transcript of Darpa Baa Synapse
Approved for Public Release, Distribution Unlimited 1
Systems of Neuromorphic Adaptive Plastic Scalable Electronics
Bidder’s Workshop and Teaming Meeting
March 4, 2008
Dr. Todd Hylton, Program ManagerDARPA DSO
Approved for Public Release, Distribution Unlimited 2
Introduction and Motivation
Approved for Public Release, Distribution Unlimited 3
Motivation and Objective
The SyNAPSE program seeks to break the programmable machine paradigm and define a new path for creating useful, intelligent machines
von Neumann Machines
Neuromorphic Machines
Machine Complexity
e.g. Gates;Memory;Neurons;
SynapsesPower;
Size
[log]
•Human level performance•Dawn of a new age
Dawn of a new paradigm
“simple” “complex”
Environmental Complexitye.g. Input Combinatorics
[log]
Program Objective
A trade between universality and efficiency
Problem
• As compared to biological systems, today’s intelligent machines are less efficient by a factor of a million to a billion in complex environments.
• For intelligent machines to be useful, they must compete with biological systems.
Objective
• Develop electronic, neuromorphic machine technology that scales to biological level.
Human Cortex Simulated Human Cortex
15 Watts 1010 Watts
I Liter 4x 1010 Liters
Lansner et al
Approved for Public Release, Distribution Unlimited 4
Vision & Impact
IC TransistorµProcessor& memory
Programmablemachines
• End of scaling• Defect intolerant• Architectural bottleneck• Software limited• No path to biologically
competitive intelligence 60 years
• Increased component density• Increased component function• Defect tolerant• Neuromorphic information,
learning, cognition, understanding architecture
• Path to biologically competitive intelligence
<<60 years
The SyNAPSE program seeks to extend the development of modern electronics into a new revolutionary new era using a similar paradigm.
“Cortical”Microcircuit
ElectronicSynapse
“Cortex”Fabric
Intelligentmachines
DARPA SyNAPSE
Historical Evolution of Modern Electronics
Vision for the Future
Approved for Public Release, Distribution Unlimited 5
Human NeoCortex Neuromorphic Electronics
~1010 synapses/cm2 1010 intersection/cm2 in crossbar arrays w/ 100 nm pitch
~106 Neurons/cm2 ~5x108 transistors/cm2 in state of the art CMOS
~5 x 108 long range axons
@ ~1 Hz~30 Gbit/sec multiplexed digital addressing
Biological-Scale Neuromorphic Electronic Devices
Inspiration
Conclusion: Gross statistics of biological neural systems might be realized in modern electronics.
Approved for Public Release, Distribution Unlimited 6
Key Challenges and Goals
Approved for Public Release, Distribution Unlimited 7
Key Goal: Electronic Synapse
The electronic synapse performs computation, memory, and adaptation in a neuromorphic system. Computation occurs in the electron current (i=v*g) injected through the synapse conductance g between neurons in response to (spike) voltage v. Memory occurs as a slowly changing electrophysical property that modifies g. Neuromorphic adaptation (aka plasticity) occurs as g changes in response to the same voltages used for computation.
Crossbar synapse
Axonic electrode
Dendritic electrode
Soma
Approved for Public Release, Distribution Unlimited 8
Key Goal: Spike Time Dependent Plasticity
Pulse interferenceat the synapse
Pre-synapticNeuron
Post-synapticNeuron
Neurons encode information as “spikes” and communicate to other neurons in both both forward (axonic) and backward (dendritic) directions. The time-relation between forward and backward spikes arriving at a synapse determines if the synaptic connection should be increased or decreased. Connection strength increases (decreases) whenever forward spikes are causally (acausually) correlated to backward spikes.
tpre tpost
Δt
syn
ap
tic p
ote
ntia
l
time
% c
ha
ng
e in
syn
ap
tic c
on
du
cta
nce
Δt = (tpre – tpost)
0
0
+
-
Approved for Public Release, Distribution Unlimited 9
Key Goal: Neuromorphic Architecture
• Possible approaches
– “Bottom-up” based on neuro-psycho-physical models of biological systems
– “Top-down” based on large scale neuro-informatics / connectomics
– Artificial Neural Networks
– First principles design
– “Evolutionary” optimization of model structures
Approved for Public Release, Distribution Unlimited 10
Key Goal: Electronic Implementation
• Chip fabrication– Novel materials and structures on CMOS
• Spike processing– Spike time encoding
– Spike time dependent plasticity
• Connectivity– Hardwired
– Addressed / programmable
– On-chip / off chip
• Power
• Size
• Supports Neuromorphic Architecture
Approved for Public Release, Distribution Unlimited 11
Key Goal: Large Scale Simulation
• Using programmable machines to design and test intelligent machines– Architectural design, validation, development
– Chip design / validation
– Mammalian scale simulations of systems and components
– Functional performance testing in environments
• Large scale digital hardware– “Supercomputer” scale
– Specialized hardware development may be appropriate
– Rebuilding the current computer architecture “from scratch” is outside the scope of this solicitation
Approved for Public Release, Distribution Unlimited 12
Key Goal: Training & Evaluation Environments
• Train and evaluate machine intelligence across capabilities found in mammalian species (106 range of brain size)
• Virtual environment for the evolution of intelligent machines• Fill long-standing need for authoritative machine intelligence evaluation
(Image removed)
Approved for Public Release, Distribution Unlimited 13
Approach: Training & Evaluation Environments
Sensory Perception
Decision & Planning
Navigation & Survival
Task Area Features Cognitive Area
• Quantitative measures of complexity• Objective measures of performance• Easily scaled• Human interaction• “Abstract” cognition
• Identification/classification of spatio-temporal objects in animation or video
• Multi-dimensional complexity variability
• Core task of all cognitive systems
• Interaction in complex, dynamic environments.
• Comparison to small animal studies • Exercises all levels of cognition• Most difficult to score and scale
(Image removed)
(Image removed)
(Image removed)
Approved for Public Release, Distribution Unlimited 14
Disciplinary Integration Challenge
Neuroscience• Neuroinformatics
• Neurophysiology• Neuroanatomy
• Neural models• Neural simulation• Animal models
Computer Science & Electrical Engineering• Large Scale Computation• CAD Tools• Design Validation• Electronic Architecture
VLSI CMOS• Device Design
• Analog-Digital• Asynchronous• Sub-threshold neuromorphic
• Fabrication• Test• Packaging
Materials & Physics• Crossbars• Electronic Synapses• CMOS Integration
Theory• Information• Computation• Communication• Cognition• Learning
Disciplinary Gap
SyNAPSE must bridge the disciplinary gap
(Image removed)
Approved for Public Release, Distribution Unlimited 15
Program Plan and Milestones
Approved for Public Release, Distribution Unlimited 16
Program Approach
MeasureMake
Model
Employ theoretical and empirical approaches constrained by practicality.
Hardware
Architecture
Simulation
Environment
Sponsor a suite of complementary capabilities to build, train, and evaluate devices.
Attack the problem “bottom-up” and “top-down” and force disciplinary integration with a common set of objectives.
Top-down(simulation)
Bottom-up(devices)
Biological ScaleMachine Intelligence
Materials(e.g. memristors)
Components(e.g. synapse / neuron)
Circuits(e.g. center-surround)
Networks(e.g. cortical column)
Modules(e.g. visual cortex)
System (SyNAPSE)
Approved for Public Release, Distribution Unlimited 17
Program Components
• Hardware will likely include CMOS devices, novel synaptic components, and combinations of hard-wired and programmable/virtual connectivity and will support critical information processing techniques like spike time encoding and spike time dependent plasticity.
• Architectures will support critical structures and functions observed in biological systems such as connectivity, hierarchical organization, core component circuitry, competitive self-organization, and modulatory/reinforcement systems.
• Large scale digital simulations of circuits and systems will be used to prove component and whole system functionality and to inform overall system development in advance of neuromorphic hardware implementation.
• Environments will be evolving virtual platforms for the training, evaluation and benchmarking of intelligent machines
Approved for Public Release, Distribution Unlimited 18
Program OutlinePhase 1 Phase 2 Phase 3 Phase 4
Em
ula
tio
n &
Sim
ula
tio
n
Simulate large neural subsystem dynamics
“Mouse” level benchmark(~ 106 neuron)
Ha
rdw
are Component
synapse (and neuron) development
CMOS process and core circuit development
~106 neuron single chip implementation “Mouse” level
Arc
hit
ec
ture
& T
oo
ls Microcircuit architecture development
~106 neuron design for simulation and hardware layout
~108 neuron multi-chip robot at “Cat” level
~108 neuron design for simulation and hardware layout
“Cat” level benchmark (~ 108 neuron)
Build Sensory, Planning and Navigation environments
“Small mammal” complexityE
nv
iro
nm
en
t
Comprehensive design capability
Phase 0
CMOS process integration
System level architecture development
Add Audition, Proprioception and Survival
“All mammal” complexity
Add Touch and Symbolic environments
Sustain
Preparatory studies only
Preparatory studies only
Program Phases 1-4 may be combined per the BAA instructions
Approved for Public Release, Distribution Unlimited 19
Phase 0 Go No-Go MetricsHardware• Synaptic density scalable to > 1010/cm2
• Operating speed >10 Hz• Consumes < 10-12 Joules per synaptic operation (at scale)• Dynamic range of synaptic conductance > 10 with >3 bit resolution• Synaptic conductance increase >1%/pulse for presynaptic spike applied somewhere
within 80-1 msec before a postsynaptic spike• Synaptic conductance decrease >1%/pulse for presynaptic spike applied somewhere
within 80-1 msec after postsynaptic spike.• 0%-0.02% conductance decrease if presynaptic spike applied > 100 msec before or
after postsynaptic spike• Maintains performance over 3 x 108 synaptic operations
Architecture• Specify and validate by simulation the function of core microcircuit assemblies using
measured synaptic properties. • The microcircuits must support the larger system architecture and support spike time
encoding, spike time dependent plasticity, and competitive neural dynamics.
~ 9 months
Approved for Public Release, Distribution Unlimited 20
Go/No-Go Milestones Set 1Hardware• Demonstrate all core micro-circuit functions in hardware• Specify a chip fabrication process supporting the architecture with >1010 synapse/cm2 and
>106 neurons/cm2
Architecture• Demonstrate a complete neuromorphic design methodology that can specify all the
components, subsystems, and connectivity of a complete system. • Specify a corresponding electronic implementation of the neuromorphic design
methodology supporting > 1014 synapses, > 1010 neurons, mammalian connectivity, < 1 kW, < 2L
Simulation• Demonstrate dynamic neural activity, network stability, synaptic plasticity and self-
organization in response to sensory stimulation and system-level modulation/reinforcement in a system of ~ 106 neurons modeled on mammalian cortex
Environment• Demonstrate virtual Visual Perception, Decision and Planning, and Navigation
Environments with a selectable range of complexity corresponding roughly to the capabilities demonstrated across a ~104 range in brain size in small-to-medium mammalian species
Approved for Public Release, Distribution Unlimited 21
Go/No-Go Milestones Set 2
Hardware• Demonstrate chip fabrication of >1010 synapse/cm2, >106 neurons/cm2
Architecture• Design a neural system of ~106 neurons and ~1010 synapses for simulation testing• Design a corresponding single chip neural system of ~106 neurons and ~1010
synapses
Simulation• Demonstrate a simulated neural system of ~106 neurons performing at “mouse”
level in the virtual environment
Environment• Expand the Sensory Environment to include training and evaluation of Auditory
Perception and Proprioception• Expand the Navigation Environment to include features stressing Competition for
Resources and Survival• Demonstrate a selectable range of complexity corresponding roughly to the
capabilities demonstrated across a ~106 range in brain size mammalian species
Approved for Public Release, Distribution Unlimited 22
Go/No-Go Milestones Set 3
Hardware
• Fabricate a single chip neural system of ~106 neurons and package into a fully functioning assembly. Show “mouse” level performance in the virtual environment.
Architecture
• Design a neural system of ~108 neurons and ~1012 synapses for simulation testing
• Design a corresponding single chip neural system of ~108 neurons and ~1012 synapses
Simulation
• Demonstrate a simulated neural system of ~108 neurons performing at “cat” level
Environment
• Add Touch to the Sensory Environment
• Add Symbolic Environment
Approved for Public Release, Distribution Unlimited 23
Final Metric – Milestone Set 4
Hardware
• Fabricate a multi-chip neural system of ~108 neurons and instantiate into a robotic platform performing at “cat” level
Approved for Public Release, Distribution Unlimited 24
Proposal Technical Requirements
Approved for Public Release, Distribution Unlimited 25
Proposal Requirements (1)• Describe an approach to developing an integrated neuromorphic
architecture serving as a foundation for the development of intelligent machines.
– Describe the base components of your architecture and their function. These base components may be the analogs of biological neurons, synapses and/or small assemblies of such elements. Describe the computational, communication and learning functions of these base components.
– Describe one or more core micro-assemblies of the base components and their corresponding function.
– Describe your approach for developing functional assemblies from the core assemblies. These assemblies should provide core cognitive functions such as sensory perception, motor control, executive control and others.
– Describe your approach to integrate functional assemblies into complete cognitive systems including sensory perception, declarative learning and memory, procedural learning and memory, executive control, and motor function.
– Describe any plan to incorporate neuro-anatomical/physiological data into the architecture.
Approved for Public Release, Distribution Unlimited 26
Proposal Requirements (2)
• Describe a high-level, conceptual electronics implementation capable of supporting the neuromorphic architecture of (1) having
– 1010 neurons– 1014 synapses– operating with temporal dynamics comparable to biological
systems– total power <1kW– total volume <2L– interfaces for sensory inputs and motor outputs
Approved for Public Release, Distribution Unlimited 27
Proposal Requirements (3)
• Describe an approach to developing nanometer-scale, plastic synaptic components consistent with (1) and (2). Multiple approaches are encouraged for this task.
• Describe an approach to developing electronic neuronal processing units (neurons) consistent with (1), (2) and (3).
• Describe an electronic coding, communication and synaptic update scheme consistent with (1), (2), and (3).
• Describe a plan of computer simulation/emulation to enable the near real-time simulation of neuromorphic systems up to 108 neurons and 1012 synapses.
• Describe a plan to obtain and import descriptions of neural systems from neuro-biological databases (as appropriate).
• Describe key technical challenges and approaches to achieving these goals and any other items in the critical path.
Approved for Public Release, Distribution Unlimited 28
Proposal Requirements (4)• Describe an approach for developing a virtual training and evaluation
environment comprised of the following tasks.
– A Planning and Decision (Game) Task that provides quantitative measures of complexity and objective and comparative measures of performance;
– A Sensory Perception Task that provides quantitative measures of performance of identification/classification of spatio-temporal objects in animation or video;
– A Navigation Task that captures the challenges confronted in navigating in complex, dynamic environments. The purpose of this task is to evaluate a collection of cognitive capabilities and to provide a point of comparison to animal studies.
• Describe a means to scale the complexity of these tasks over the entire range of mammalian intelligence (~106 range in brain size).
• Describe a capability for hosting the environment including hardware, software and system support.
• Describe an interface for interacting with the environment.
Approved for Public Release, Distribution Unlimited 29
Proposal Requirements (5)Environmental tasks will require
• Adaptation in dynamic, uncertain, probabilistic environments that include partial, erroneous and sometimes contradictory information
• Response times that force speed-accuracy tradeoffs
• Knowledge Integration over
– Different sources and times of knowledge acquisition; and
– Multiple levels of perception, planning and reasoning.
• Interaction with other (human or machine) agents.
• Feedback based on
– Reinforcement of generic, high-level goals
– Supervision using a tutor (learning mode)
• Scalability to match system complexity and support incremental learning
• Scoring to provide quantitative measures of performance
• Benchmarking to provide comparative measures of performance.
Approved for Public Release, Distribution Unlimited 30
Proposal Evaluation
Approved for Public Release, Distribution Unlimited 31
Evaluation Criteria
1) Ability to Meet Go/No-Go Metrics
2) Scientific and Technical Merit
3) Value to Defense
4) Management Approach and Proposer’s Capabilities and Related Experience
5) Cost and Schedule Realism.
Approved for Public Release, Distribution Unlimited 32
Ability to Meet Go/No-Go Metrics
• The proposal establishes clear and well defined research go/no-go metrics to be used as exit and entry criteria for Government approval to progress through phases of the proposed effort.
• The feasibility and likelihood of the proposed approach for satisfying the program go/no-go metrics are explicitly described and clearly substantiated.
• The proposal reflects a mature and quantitative understanding of the proposed go/no-go metrics, the statistical confidence with which they may be measured, and their relationship to the concept of operations that will result from successful performance
Approved for Public Release, Distribution Unlimited 33
Scientific and Technical Merit
• Proposers must demonstrate that their proposal is innovative and unique, that the technical approach is sound, that they have an understanding of critical technical issues and risk, and that they have a plan for mitigation of those risks.
• A significant improvement in capability or understanding above the state of the art must be demonstrated.
• All milestones must be clearly and quantitatively described.
Approved for Public Release, Distribution Unlimited 34
• Proposers must demonstrate the long-term potential of successful research to radically change military capability or improve national security with a clear statement of the goals of their program, and a quantitative comparison with existing technology as appropriate.
Value to Defense
Approved for Public Release, Distribution Unlimited 35
Management Approach and Proposer’s Capabilities and Related Experience
• The appropriateness, effectiveness, and reliability of the management structure are appropriate to the diversity of tasks, technologies and partnering strategy.
• The qualifications of Principal Investigator and key Task Leaders are appropriate and support the overall management plan.
• The qualifications of the proposer’s key personnel are of adequate range, depth, and mix of expertise to address all technical and programmatic aspects of the proposal.
• The proposer's prior experience in similar efforts must clearly demonstrate an ability to deliver products that meet the proposed technical performance within the proposed budget and schedule.
• The proposed team has the expertise to manage the cost and schedule.
• Similar efforts completed/ongoing by the proposer in this area are fully described including identification of other Government sponsors.
Approved for Public Release, Distribution Unlimited 36
Cost and Schedule Realism
• The objective of this criterion is to establish that the proposed costs are realistic for the technical and management approach offered, as well as to determine the proposer’s practical understanding of the effort. This will be principally measured by cost per labor-hour and number of labor-hours proposed.
• The evaluation criterion recognizes that undue emphasis on cost may motivate proposers to offer low-risk ideas with minimum uncertainty and to staff the effort with junior personnel in order to be in a more competitive posture. DARPA discourages such cost strategies.
• Cost reduction approaches that will be received favorably include innovative management concepts that maximize direct funding for technology and limit diversion of funds into overhead.
• The proposer’s abilities to aggressively pursue performance metrics in the shortest timeframe and to accurately account for that timeframe will be evaluated, as well as proposer’s ability to understand, identify, and mitigate any potential risk in schedule
Approved for Public Release, Distribution Unlimited 37
Administrative Items
Approved for Public Release, Distribution Unlimited 38
BAA Solicitation Schedule
• BAA 08-28– Estimated posting date – March 17, 2008
• Proposal Due Date– May 2, 2008, no later than 4:00PM EST– BAA will remain open for 1 year
• Anticipated Contract Award– August 2008
Approved for public release, distribution unlimited
Approved for Public Release, Distribution Unlimited 39
Proposal Format
• Proposals must consist of two volumes-technical and cost.
• Technical- Maximum of 55 pages including references, tables, and charts. Please do not include separate articles or CDs as these will not be used in the review process.
• Cost-contains a cover sheet, detailed cost break down, and supporting cost and pricing information.
• For detailed description of proposal format see the BAA at http://www.darpa.mil/baa/BAA08-28.html
Approved for public release, distribution unlimited
Approved for Public Release, Distribution Unlimited 40
Other Comments on the Proposal
• DARPA requests proposals for the full scope of development – All proposals must address all of the technical areas listed in
the BAA
– Proposals addressing only individual components of the overall program will be considered non-responsive
• Coherent integration and management of multidisciplinary research organizations is required.
• Structure proposals to reduce risk early and to give the government flexibility in task/phase funding
Approved for public release, distribution unlimited
Approved for Public Release, Distribution Unlimited 41
Teaming Website
• http://www.sainc.com/SyNAPSETeaming/index.asp
A teaming website has been created to facilitate the organization of teams to address all program component areas.
Approved for public release, distribution unlimited
Approved for Public Release, Distribution Unlimited 42
Discussion
Discussions are
strongly encouraged during
teaming and proposal formulation.
Please submit questions by noon so that they may be answered during the FAQ segment of the
workshop.
Approved for public release, distribution unlimited