Post on 17-Dec-2015
Kevin H. Knuth, Ph.D.Departments of Physics and Informatics
University at Albany
The Age of Robotics
“If every tool, when ordered, or even of its own accord, could do the work that befits it... If the weavers' shuttles were to weave of themselves, then there would be no need either of apprentices for the master workers or of slaves for the lords.”
Aristotle, “De Republica Atheniensium”, ~322 BC
2 May 2008 Kevin H Knuth NTIR 2008
Leonardo da Vinci1452-1519
Leonardo da VinciThe Father of Robotics
2 May 2008 Kevin H Knuth NTIR 2008
Leonardo’s RobotBased on da Vinci’s sketches from 1495
The robot is a knight in German-Italian Medieval Armor
2 May 2008 Kevin H Knuth NTIR 2008
Leonardo’s AutomataA small wooden car, powered by springs
could be programmed by inserting various cogs and gears.
(Image: Institute and Museum of the History of Science, Florence)
2 May 2008 Kevin H Knuth NTIR 2008
The Dream of Robotics has long been with us…
Maria from Fritz Lange’s Metropolis (1927)
2 May 2008 Kevin H Knuth NTIR 2008
The Dream of Robotics has long been with us…
The Tin ManFrank L. Baum’sThe Wizard of Oz
(1900 Book, 1939 Film)
Robby the RobotForbidden Planet
1956
2 May 2008 Kevin H Knuth NTIR 2008
And has changed with us…
R2D2 and C3POStar Wars
(1977)
BenderFuturama
(1999-Present)
2 May 2008 Kevin H Knuth NTIR 2008
Reality is always Stranger than Fiction…
Cassini in orbit around SaturnOpportunity currently on Mars
2 May 2008 Kevin H Knuth NTIR 2008
And creeps up on us Unsuspectingly…
2 May 2008 Kevin H Knuth NTIR 2008
And creeps up on us Unsuspectingly…
2 May 2008 Kevin H Knuth NTIR 2008
How do we make them INTELLIGENT?
2 May 2008 Kevin H Knuth NTIR 2008
How Do We Function?
2 May 2008 Kevin H Knuth NTIR 2008
The Brain: The Living State of Matter
1011-1012 Neurons
104-105 Connections per Neuron
Maximum Firing Rate: 1 ms
1kHz massively parallel computer
Information Processed on order of 100s ms
MUST use Prior Information
The Virtual Hospital, Ch 5, Williams, Gluhbegovic, and Jew
2 May 2008 Kevin H Knuth NTIR 2008
A Powerful Computer
Hree is an ecxlelnet eaxmlpe of how yuor wnodreful mnid can raed
tihs txet eevn touhgh its all jmbuled.
2 May 2008 Kevin H Knuth NTIR 2008
Sounds and Prior Information
Listen to these sounds…
Sounds from Haskins Laboratories, Rubin, Remez, Pardo
2 May 2008 Kevin H Knuth NTIR 2008
Sounds and Prior Information
Now listen to this one…
Sounds from Haskins Laboratories, Rubin, Remez, Pardo
2 May 2008 Kevin H Knuth NTIR 2008
Sounds and Prior Information
And now go back to this one…
Sounds from Haskins Laboratories, Rubin, Remez, Pardo
2 May 2008 Kevin H Knuth NTIR 2008
Sounds and Prior Information
What about the rest?
Sounds from Haskins Laboratories, Rubin, Remez, Pardo
2 May 2008 Kevin H Knuth NTIR 2008
Prior Information is Key
Only 10% of the inputs into primary visual cortex come from the retina via the lateral geniculate nucleus. The rest come from higher visual and frontal areas.
Perception can also be modified by attention.
Thus the brain can actively focus on relevant information.
Human Brain: basal view (front at top)The Virtual Hospital, Ch 5, Williams, Gluhbegovic, and Jew
2 May 2008 Kevin H Knuth NTIR 2008
The Brain Models its Environment
The frontal regions of the braincreate models of the world basedon prior experience. Thesemodels affect perception and attention.
In addition, the brain modelsitself.
Experiments in multi-sensoryprocessing has shown thatthe information processing is consistent with Bayes Theorem
The Virtual Hospital, Ch 5, Williams, Gluhbegovic, and Jew
2 May 2008 Kevin H Knuth NTIR 2008
Thinking Machines
Your frontal lobes carry a model of yourself that is continually updated from data received from a dense sensor network. This implements both ‘Instrument Health Monitoring’ and ‘Calibration’
You learn from new data by updating your model of the world.
You actively seek new data by asking relevant questions.
2 May 2008 Kevin H Knuth NTIR 2008
Body and Brain form a Symbiotic Unit
2 May 2008 Kevin H Knuth NTIR 2008
Relevant Information
2 May 2008 Kevin H Knuth NTIR 2008
Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, 1967 (Originally published in Russian 1962)
2 May 2008 Kevin H Knuth NTIR 2008
Free Examination
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, 1967 (Originally published in Russian 1962)
Three minute recording
2 May 2008 Kevin H Knuth NTIR 2008
Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, 1967 (Originally published in Russian 1962)
2 May 2008 Kevin H Knuth NTIR 2008
Estimate Ages of the People
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, 1967 (Originally published in Russian 1962)
Three minute recording
2 May 2008 Kevin H Knuth NTIR 2008
Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, 1967 (Originally published in Russian 1962)
2 May 2008 Kevin H Knuth NTIR 2008
Remember their Clothes
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, 1967 (Originally published in Russian 1962)
Three minute recording
2 May 2008 Kevin H Knuth NTIR 2008
Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, 1967 (Originally published in Russian 1962)
2 May 2008 Kevin H Knuth NTIR 2008
Estimate Material Circumstances
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, 1967 (Originally published in Russian 1962)
Three minute recording
2 May 2008 Kevin H Knuth NTIR 2008
Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, 1967 (Originally published in Russian 1962)
2 May 2008 Kevin H Knuth NTIR 2008
How Long has the Visitor been away?
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, 1967 (Originally published in Russian 1962)
Three minute recording
2 May 2008 Kevin H Knuth NTIR 2008
Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, 1967 (Originally published in Russian 1962)
2 May 2008 Kevin H Knuth NTIR 2008
Do We Analyze Everything?
2 May 2008 Kevin H Knuth NTIR 2008
http://viscog.beckman.uiuc.edu/grafs/demos/15.html
2 May 2008 Kevin H Knuth NTIR 2008
Inattentional Blindness
http://viscog.beckman.uiuc.edu/grafs/demos/15.html
http://viscog.beckman.uiuc.edu/grafs/demos/10.html
http://viscog.beckman.uiuc.edu/grafs/demos/12.html
2 May 2008 Kevin H Knuth NTIR 2008
AUTOMATED INQUIRY
2 May 2008 Kevin H Knuth NTIR 2008
More and more are our instruments required to perform science operations further from the intervention of humans.
Spirit and Opportunity: Remote Science
Dust devils whip across Gusev Crater on Mars
2 May 2008 Kevin H Knuth NTIR 2008
The Expansive Floor of Gusev Crater
2 May 2008 Kevin H Knuth NTIR 2008
Rock Outcrop (Methuselah)
2 May 2008 Kevin H Knuth NTIR 2008
Underwater Robotic Explorers
At the Monterey Bay Aquarium Research Institute (MBARI), researchers are employing robotic submarines to explore the Deep Pacific Ocean.
2 May 2008 Kevin H Knuth NTIR 2008
Intelligent Autonomous Instruments
Require:
Stability Control
Instrument Health Monitoring
Automated Calibration
Accurate Onboard Data Analysis
Adequate Data Coverage
Ability to Actively Seek Data
2 May 2008 Kevin H Knuth NTIR 2008
Novel Instrument Design
To accomplish these goals, these novel instruments must
Monitor their own state (health and calibration) Infer their state from self-sensing Be equipped with dense sensor networks Infer calibration parameters
Learn from data Make inferences from data Perform hypothesis testing
Ask new questions Actively seek new data Select optimal experiments
2 May 2008 Kevin H Knuth NTIR 2008
The Basic Components
2 May 2008 Kevin H Knuth NTIR 2008
The LEGO Mindstorms NXT System
1 The NXT Brick is the brain of the system.
2 Touch Sensor
3
4
5
6
Microphone
Light Sensor
UltrasonicRangefinder
Servo Motors$250!
2 May 2008 Kevin H Knuth NTIR 2008
Lego teams with HiTecnic
AccelerometerNEW!
Color Sensor
Digital Compass
PrototypeBoard
Sensor and Motor Multiplexers
2 May 2008 Kevin H Knuth NTIR 2008
Robotic Scientists
This robot is equipped with a light sensor.
It is to locate and characterize a white circle on a black playing field with as few measurements as possible.
LANDMINE DETECTION!
2 May 2008 Kevin H Knuth NTIR 2008
The Robot’s “Thoughts”
Set of Hypothesized CirclesMean CircleArea within Robot’s Reach
colored according to ENTROPY
Past Measurement DARK Past Measurement LIGHT Next Measurement
Robot Center
2 May 2008 Kevin H Knuth NTIR 2008
'Am I already in the shadow of the Coming Race? and will the creatures who are to transcend and finally supersede us be steely organisms, giving out the effluvia of the laboratory, and performing with infallible exactness more than everything that we have performed with a slovenly approximativeness and self-defeating inaccuracy?'
George Eliot (Mary Anne Evans),
The Impressions of Theophrastus Such, 1879.
Special Thanks to:
John SkillingAriel CatichaJanos AczélKeith EarlePhilip ErnerDeniz GencagaPhilip GoyalSteve GullJeffrey JewellCarlos Rodriguez
And also to:
Emily KnuthRockne KnuthAnn KnuthJoshua KnuthRoland DerouenBernadette Derouen
For their patience with the robotic invasion that has been thrust upon them.