Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics...
Transcript of Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics...
![Page 1: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/1.jpg)
Sensors
Cognitive Robotics
Hans-Dieter Burkhard/Monika Domanska
![Page 2: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/2.jpg)
Outline Introduction Sensors: General Considerations Signals Sensors: Special Types Vision (introductory, later more) Camera Model Image Processing (introductory, later more) Scene Interpretation (introductory, later more)
Burkhard/Domanska 2 Cognitive Robotics Sensors
![Page 3: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/3.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 3
Sensors Sensus (lat.): the sense • Recording information related to state or change of state
(physical, chemical ...). • Transformation between state/change of state by differentiation/integration e.g. distance – speed – acceleration drift problems over time (e.g. odometry) • Conversion to internally processable information Technically: mostly electronic signals Nature: electrochemical processes • Direct influence on actuators in case of sensor actor coupling
![Page 4: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/4.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 4
Human senses (more than 5) see listen smell taste tactil heat pain balance hunger thirst muscle tension, joints, ...
Further senses in nature e.g. magnetism, electricity
![Page 5: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/5.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 5
Processing of Sensations in Nature • Stimulus excites a receptor • Release of nerve impulses • Forwarding to the spinal cord / brain:
About 1 million receptor signals per second in the Central Nervous System
• Unconscious reflexes activated by spinal cord or brain “Sensor-Actor Coupling”
• Filtering in Thalamus: Only selected signals are consciously perceived in the
cerebral cortex.
![Page 6: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/6.jpg)
Problems in Perception
Humans can deal with incomplete and unreliable data Humans use redundancies Humans use world knowledge and experience Humans can deal with high complexity Recent machines are far from human performance Useful results only in special cases Missing robustness and reliability
6 Burkhard/Domanska Cognitive Robotics Sensors
![Page 7: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/7.jpg)
Outline Introduction Sensors: General Considerations Signals Sensors: Special Types Vision (introductory, later more) Camera Model Image Processing (introductory, later more) Scene Interpretation (introductory, later more)
Burkhard/Domanska 7 Cognitive Robotics Sensors
![Page 8: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/8.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 8
Sensors of Nao (Academic Version)
4 Microphones (head) 2 CMOS digital cameras (head) 32 Hall effect sensors (joints) 1 Gyrometer 2 axis (torso) 1 Accelerometer 3 axis (torso) 2 Bumpers (feets) 2 Channel sonar (torso) 2 Infrared sensors (torso) 9 Touch sensors (head, hands) 8 Force Resistance Sensor (feets)
![Page 9: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/9.jpg)
http://www.aldebaran-robotics.com/en/Discover-NAO/Key-Features/hardware-platform.html
Burkhard/Domanska Cognitive Robotics Sensors 9
![Page 10: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/10.jpg)
Sensors
• Passive sensors
− record signals created in the environment • Active sensors
− send signals (sonar, laser, radar, infrared, ...) and measure the reflections − disadvantage: recognizable through their signals
• Proprioceptive sensors − bodily sensation
10 Burkhard/Domanska Cognitive Robotics Sensors
![Page 11: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/11.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 11
Sensors Internal sensors: „Proprioceptive sensors“
("self") • Position (body, joints) • Motion • Internal forces • Temperature (inside) • Resources • Energy • ...
External sensors: ("Environment") • Light, Vision • Sound • Smell • Distance • External forces • Temperature (outside) • …
![Page 12: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/12.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 12
Sensors in Robotics Exploit physical/chemical … features, e.g.
• Current - power - resistance - inductance - conductivity ...
• Wavelength - frequency - phase shift - echo - runtime ...
• Mass - force - speed - acceleration - inertia ... Transformations by related physical laws. e.g. State to velocity by differentiation Conversion into internal information (mostly electronic signals)
![Page 13: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/13.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 13
Sensor Model and Observation Model s = state/feature of the world o = observation: sensory data according to s Sensor Model: „Forward model“ o = fsensor(s) Observation Model: „Backward model“ s = f -1sensor (o)
![Page 14: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/14.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 14
Sensor Model: Sonar Acoustic propagation (ca. 330 m/sec)
Image from “Where am I?" -- Systems and Methods for Mobile Robot Positioning by J. Borenstein, H. R. Everett, and L. Feng
![Page 15: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/15.jpg)
Y
X
Burkhard/Domanska Cognitive Robotics Sensors 15
Sensor Model: Camera Projection
Xw,Yw,Zw : world coordinates X,Y, Z: camera coordinates x,y : image coordinates
Image from “Where am I?" -- Systems and Methods for Mobile Robot Positioning by J. Borenstein, H. R. Everett, and L. Feng
R T
(Xw,Yw,Zw)
![Page 16: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/16.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 16
Sensor Model: Camera Projection Y
X
For given camera parameters (R,T, f): Image coordinates (x,y) are uniquely determined by object coordinates (Xw,Yw,Zw)
R T
(Xw,Yw,Zw)
(Xw,Yw,Zw)
Detailed descriptions will be given later
![Page 17: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/17.jpg)
Problems with Observation Model • fsensor often not bijective (f -1sensor not unique) • noisy data: o = fsensor(s) + fnoise(s)
Burkhard/Domanska Cognitive Robotics Sensors 17
For given camera parameters (R,T, f): Object coordinates (Xw,Yw,Zw) are not uniquely determined by image coordinates (x,y) . „Badly posted problem“
![Page 18: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/18.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 18
Criteria/Quality/Improvement of Sensors Sensitivity Measuring range Noise filtering Calibration (adapt measured values to scale)
![Page 19: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/19.jpg)
Problems with Measurements • Systematic errors (e.g. wrong position of sensors).
• Noise (caused by many inside and outside reasons):
− Modeling by noise models (often statistically). − Regarded in perception methods.
Burkhard/Domanska Cognitive Robotics Sensors 19
PhD thesis J.N.E. Barrantos
![Page 20: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/20.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 20
Further Effects of Noise High frequency noise: fnoise(s) = k sin(ωs) with large ω Differentiation (for example, speed from position difference): results in high errors:
o´ = f´sensor(s) + f´noise (s) = f´sensor(s) + k ω cos(ωs)
![Page 21: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/21.jpg)
Outline Introduction Sensors: General Considerations Signals Sensors: Special Types Vision (introductory, later more) Camera Model Image Processing (introductory, later more) Scene Interpretation (introductory, later more)
Burkhard/Domanska 21 Cognitive Robotics Sensors
![Page 22: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/22.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 22
Signals Information by frequency, amplitude, pulse duration, ... (see lecture signal processing) Analog vs. discrete: Depends on recording and processing Conversion in both directions possible:
− Quantization − Sampling − Interpolation
![Page 23: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/23.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 23
Quantization Discrete instead of continuous values (by rounding).
Small differences of continuous values can lead to larger differences of rounded values. Oscillations by noisy signals.
0 0 0 0 0 1 0 0 0 1 1 0 0 1 1 0
![Page 24: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/24.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 24
Interpolation
• Needs further knowledge about the class of curve.
• Depends on number of discrete points.
![Page 25: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/25.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 25
Sampling Theorem Problem: The red curve is measured only at few points: Only the black points are registered.
The black dots are interpreted as a lower frequency curve: "Alias"
![Page 26: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/26.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 26
Sampling Theorem Example: Smaller intervals for measurements: -- More points
How many measurements are needed?
![Page 27: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/27.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 27
Sampling Theorem
Sampling Theorem
For correct reproduction we must have:
More than 2 sampling points per wavelength T,
i.e. sampling rate ∆x < T/2 (Nyquist criterion)
or:
Sampling frequency must be more than twice as
large as the highest occurring frequency.
Holds also for more dimensional signals (e.g. images).
![Page 28: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/28.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 28
Example: sin(x·y) With Different Sampling
-2 -1 0 1 2-2
-1
0
1
2
x,y ∈[-2 , 2]
-20 -10 0 10 20-20
-10
0
10
20
x,y ∈[-20 , 20]
-1000 -500 0 500 1000-1000
-500
0
500
1000
x,y ∈[-1000, 1000]
![Page 29: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/29.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 29
Darstellung von sin(x·y) in ungenügender Rasterung
-20 -10 0 10 20-20
-10
0
10 x,y ∈[-20 , 20]
x,y ∈[-2 , 2]
![Page 30: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/30.jpg)
Outline Introduction Sensors: General Considerations Signals Sensors: Special Types Vision (introductory, later more) Camera Model Image Processing (introductory, later more) Scene Interpretation (introductory, later more)
Burkhard/Domanska 30 Cognitive Robotics Sensors
![Page 31: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/31.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 31
Resistance Sensors Potentiometer: Voltage dependents on position on a resistor. Sensor: Transformation of mechanical values (e.g. position) into electrical signals. Straingages: Resistance depends on length (e.g. of meandering material) Sensor: Measurement of deformations.
![Page 32: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/32.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 32
Hall Effekt Sensor
As a sensor: • Rotation of the plate • Displacement of the plate • Acting forces are measurable by Hall voltage
N
S
Hall Voltage depends on the position of the current-carrying plate in the magnetic field
![Page 33: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/33.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 33
Inductive Sensors Inductivity depends on • the cross-section of a coil • the position of the magnetic core
Tachogenerator: • generated electricity depends on rotation speed
![Page 34: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/34.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 34
Piezoelectric Sensors For special materials (e.g. quartz, SiO2): Charge differences on the surface are proportional to pressure. Usage as sensor: • pressure shifts ions • voltage measured Usage as actor: • applying electrical voltage leads to deformation of the crystal
![Page 35: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/35.jpg)
Light Sensor / Infrared Sensor Device with varying electronic properties (charge, resistance, …)
depending on light intensity.
• Single sensor for measurement of brightness (cf. Braitenberg vehicle)
• Sensor fields (1D, 2D) with optics for cameras (visual sensor)
• Infrared sensor: measures temperature (alarm systems)
• Active infrared sensor for close distance measurements: − sends coded signals, measures reflected echo − similar to Sonar: no accurate measurement, cheap − arrangement as a ring: "non-contact bumpers“
Burkhard/Domanska Cognitive Robotics Sensors 35
![Page 36: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/36.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 36
Measurement of Distances Many possibilities, e.g. - Measure the performed path of a vehicle (wheel encoder) - Send Signal, receive echo:
• Time difference proportional to distance • Phase shift proportional to distance
- Image interpretation: • Size of objects reciprocally proportional to distance • Vertical view angle proportional to distance • Stereo vision: Shift proportional to distance
Sonar Laser Radar
![Page 37: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/37.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 37
Incremental Wheel Encoder Measurement of rotation by identical markers • speed (distance by integration) • no wheel position, no direction • error drifting Multichannel Encoder: • speed • direction • zero position (C)
Images from “Where am I?" -- Systems and Methods for Mobile Robot Positioning by J. Borenstein, H. R. Everett, and L. Feng
![Page 38: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/38.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 38
Absolute Wheel Encoder • Each position has individual word pattern
Gray code (a), BinaryCode (b)
• Disturbances without affecting
• 12 bits: 0.1 degree accuracy
![Page 39: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/39.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 39
Odometry Known start position Actual position by measuremaent of pathes • Wheel encoder • Motion of legs • Control • Inertial sensors
![Page 40: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/40.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 40
Systematic errors By sensors (e.g. wheel encoder) By controls (e.g. unsymmetric wheels)
Non-systematic errors Ground External forces (e.g. other robots)
Odometry: Measurement Errors
Main problem: Errors of direction
![Page 41: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/41.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 41
Sonar Sensors
Sonar = sound navigation and ranging
Active ultrasonic sensor (> 20 kHz)
Cheap, but noisy and inaccurate
Arrangement as a ring for obstacle detection.
![Page 42: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/42.jpg)
Sonar Range Sensor Nao
Burkhard/Domanska Cognitive Robotics Sensors 42
Distances to obstacles in the range 15–70 cm. Presence of an object (no distance value) for 0-15 cm. http://www.aldebaran-robotics.com/en/Discover-NAO/Key-Features/tactile-sensors.html
![Page 43: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/43.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 43
Sonar Sensors Send pulse - receive echo: • Time difference is proportional to the distance alternatively: • Phase shift proportional to the distance
V proportional to
phase shift
![Page 44: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/44.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 44
Sonar Sensors
Sensor model:
Amplitude strength depends on the direction relative to the center of the signal
Image from “Where am I?" -- Systems and Methods for Mobile Robot Positioning by J. Borenstein, H. R. Everett, and L. Feng
![Page 45: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/45.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 45
Sonar Sensors
Distance (in m) d = 0,5 ⋅ c ⋅ t by echo runtime t (in s):
c = c0+0,6T m/s
with c0=331 m/s , T=Temperature e(Celsius)
Device transmits a short sound, then switch to work as microphon (receive echo). No measurements in close distance ( < 6cm) („blanking Intervall“: internal echos)
![Page 46: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/46.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 46
Problems with Sonar Sensors • „Crosstalk“ Interference of reflexions:
- Direct (a) - Indirect (b)
• Missing reflection • Multiple reflection
To avoid:
Use different frequencies and signals by the sensors.
Image from “Where am I?" -- Systems and Methods for Mobile Robot Positioning by J. Borenstein, H. R. Everett, and L. Feng
![Page 47: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/47.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 47
Problems with Sonar Sensors Artefacts: RCD („Regions of constant distance“)
Echo of one sensor received by neighbouring sensors:
Interpretation as a regions of constant distance
To avoid:
Use different frequencies and signals by the sensors.
![Page 48: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/48.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 48
Problems with Sonar Sensors Sonar measurements of a robot driving on the depicted path
![Page 49: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/49.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 49
Sonar Sensors
Ultrasound organs in nature: Dolphins, Bats. Bats use different frequencies and can identify flying insects. • very complex skills • not yet fully investigated
![Page 50: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/50.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 50
Radar Sensors Radar = radio detecting and ranging (active sensor) Echos of radio waves (similar to sonar)
![Page 51: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/51.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 51
Laser Sensor Active sensor using echo of laser impulses Laser = light amplification by stimulated emission of radiation High intensity with short pulse Different forms of production • Very accurate distance measurement • Very high range • Short sampling time: even at high speeds • Expensive devices
![Page 52: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/52.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 52
Laser Sensors Detection of a RoboCup field by a Midsize League Robot
![Page 53: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/53.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 53
Laser Sensor Different methods: • Time of fly • Phase shift • Triangulation • Blur
Problems:
• Multiple reflection
• No echo at transparent objects (glass)
• Eye sensitivity
![Page 54: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/54.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 54
Laser Sensor: Time of Flight
counter
time of fly
(distance)
![Page 55: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/55.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 55
Laser Sensor Time of flight: Duration of the laser-pulse-echo is measured Speed of light c = 300,000 km / s Precise timing (nanosec) required. Time of flight ∆t in sec:
d = 0,5 ⋅ c ⋅ ∆t = 150 000 ⋅ ∆t km
Example for 0,1 ns:
d = 0,5 ⋅ c ⋅ ∆t = 150 mm/ns ⋅ 0,1 ns = 15 mm
![Page 56: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/56.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 56
Laser Sensor: Phase Shift Phase shift λ between transmitting and receiving signal
∆t / T = φ / 2π
∆t = time difference
T = 10-12 … 10-15 sec
φ = Phase shift
distance = 150 000 ⋅ ∆t km
= T• φ ⋅ 150 000 / 2π km
Ambiguity for larger distances: Example:
10 MHz Signal, i.e. period length = 100 ns
For φ = 180° : ∆t = 50 ns, d = 7,5 m
Combine multiple frequencies.
![Page 57: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/57.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 57
Lens
x2 can be calculated from • known α and x0 • measured x1
x = x0 + x2
Laser Sensors: Triangulation
![Page 58: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/58.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 58
Stereoscopy 2 camera images from different positions (by 2 cameras or a moving camera) Calculation of distances by different view angles of objects Correspondence problem: Which objects/pixels belong together?
Comparison of image features Correlation methods
![Page 59: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/59.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 59
Distance measurement with structured light Projected pattern is distorted by the geometry of the object. Kinect uses infra red.
![Page 60: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/60.jpg)
Force Sensors Transformation of force into electronical signals: Change of electrical properties (e.g. resistance, capacity, inductivity) by mechanical deformation caused by forces • Touch sensors (hand, feet, artificial skin …) • Collision detection (bumper) • Coupling with actuators
Burkhard/Domanska Cognitive Robotics Sensors 60
pet
![Page 61: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/61.jpg)
Inertialsensor: Accelerometer Spring / mass combination (separately for each direction) Measures acceleration b by deflection s • Inertial sensor (needs no contact with outside world). • Must regard gravity.
Measurement of speed and path by integration. Measurement of position by gravity. Burkhard/Domanska Cognitive Robotics Sensors 61
K = m • b = k • s2
b = k • s2 / m
K = force, m = mass, b = acceleration, s = deflection, k =spring constant
![Page 62: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/62.jpg)
Inertialsensor: Rotation
Burkhard/Domanska Cognitive Robotics Sensors 62
Gyroscop Rapidly rotating gyro in cardan suspension is stable due to conservation of angular momentum ("artificial horizon" in a airplane) Internal sensor: • Orientation in space • Measurement of rotation (force caused by changing direction) Problems: Drift over time, Earth's rotation
![Page 63: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/63.jpg)
Optical Gyroscop Light (laser) is passed in both directions through glass fiber. Different run times of light when fiber rotates (Sagnac effect). Difference measured by inference of light beams.
Burkhard/Domanska Cognitive Robotics Sensors 63
![Page 64: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/64.jpg)
Inertial System Combination of accelerometer and gyroscop: Measurement of linear and rotational motions. Localization by determination of the path from starting point using only internal sensors
Burkhard/Domanska Cognitive Robotics Sensors 64
![Page 65: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/65.jpg)
Energy Consumption Conclusion to external forces by measuring the needed energy consumption (current) or the generated heat. Possibilities for feedback control.
Burkhard/Domanska Cognitive Robotics Sensors 65
Weight of objects by current needed at the shoulder joints
![Page 66: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/66.jpg)
Acoustic Sensors: Microphone Transformation of sound waves (forces) into electrical signals (e.g. membrane in magnetic field) Time-dependent signals (limited polling frequency) Noisy (internal, external noise) Applications: • noise detection • speech recognition • bearing (echo)
Burkhard/Domanska Cognitive Robotics Sensors 66
![Page 67: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/67.jpg)
Language processing Complex process with many levels: • Preprocessing • Identification of sounds, syllables, words • Identification of relationships (e.g. dereferencing pronouns) • Interpretation: Identification of meanings/intentions Requires knowledge about • Sentence structure, grammar, syntax, ... • Relationships, contexts, ... Requires knowledge about the world
Burkhard/Domanska Cognitive Robotics Sensors 67
“AI-hard”: Turing Test. Similar to image processing.
![Page 68: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/68.jpg)
Outline Introduction Sensors: General Considerations Signals Sensors: Special Types Vision (introductory, later more) Camera Model Image Processing (introductory, later more) Scene Interpretation (introductory, later more)
Burkhard/Domanska 68 Cognitive Robotics Sensors
![Page 69: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/69.jpg)
Vision
Burkhard/Domanska Cognitive Robotics Sensors 69
Magritte
![Page 70: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/70.jpg)
Vision Humans use about 50% of brain for image processing and interpretation Preprocessing already performed in the eyes
Burkhard/Domanska Cognitive Robotics Sensors 70
![Page 71: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/71.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 71
Optical Sensors Light sensitive elements (e.g. CCD = Charge Coupled Device)
Arranged in form of a matrix with filters for different colors.
Result stored in a pixel matrix („frame“).
Short intervals: e.g. 30 frames per second (fps).
Bayer Filter for CCD (from Wikipedia)
for colors red, green and blue (RGB-system)
![Page 72: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/72.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 72
Optical Sensors „Pixels“ = picture elements
Usually three color values
for intensity of red, green, blue
(higher value = more intensity)
RGB-system
![Page 73: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/73.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 73
Color Channels in RGB-System
![Page 74: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/74.jpg)
Spectral Colors vs. RGB
Burkhard/Domanska Cognitive Robotics Sensors 74
Light consists of a continuum of frequencies, but RGB measures only 3 colors?
![Page 75: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/75.jpg)
Spectral colors vs. RGB
Burkhard/Domanska Cognitive Robotics Sensors 75
Intensities of day light
Intensities of a light that looks red
![Page 76: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/76.jpg)
Human eye … … has only three types of color sensors („cones“) • red (64%) middle area • green (32%) central area • blue (4%) peripheral area
and additional light intensity sensors („rods“)
Burkhard/Domanska Cognitive Robotics Sensors 76
Blue text is exhausting to read
![Page 77: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/77.jpg)
Sensivity of human eye sensors …
Burkhard/Domanska Cognitive Robotics Sensors 77
red cones
green cones blue cones
rods
Note that different sensitivies are found in literature. Usually, green has highest sensivity.
![Page 78: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/78.jpg)
Response by Sensors Sensor response e depends on light intensity I and sensor sensitivity φ for all frequencies λ :
Burkhard/Domanska Cognitive Robotics Sensors 78
∫= λλλφ dIe )()(
Response e for red sensitive sensor
Response e for green sensitive sensor
Response e for blue sensitive sensor
Example:
![Page 79: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/79.jpg)
RGB-System Different light may have identical RGB values: Metamerism of light. RGB tries to mimic human eyes and therewith to produce acceptable rendering. But: Those colors don‘t „exist“ in nature, they are only physiolocically grounded (by individuals!). Burkhard/Domanska Cognitive Robotics Sensors 79
Other color systems for other applications (YUV, CMYK etc. later in the course)
![Page 80: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/80.jpg)
Problems with Colors Distortion of colors by lighting and preprocessing in the camera
Burkhard/Domanska Cognitive Robotics Sensors 80
(a, b, c): images taken under different lighting conditions (d, e, f): resulting color classifications by unique parameters (from Diploma thesis Matthias Jüngel)
![Page 81: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/81.jpg)
Adaptation/Calibration Human color perception adapts to changing conditions. This may result in illusions.
Burkhard/Domanska Cognitive Robotics Sensors 81
![Page 82: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/82.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 82
Color Calibration Tools for manual calibration Different approaches for automatic calibration (later more)
![Page 83: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/83.jpg)
Outline Introduction Sensors: General Considerations Signals Sensors: Special Types Vision (introductory, later more) Camera Model Image Processing (introductory, later more) Scene Interpretation (introductory, later more)
Burkhard/Domanska 83 Cognitive Robotics Sensors
![Page 84: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/84.jpg)
Camera Model (Colors) A complete color model had to regard: - The sources of illumination:
- Their spectral characteristics (frequencies, intensities) - The illuminated objects:
- Their characteristics w.r.t. absorbance/reflection (directions, frequencies, intensities)
- The spatial relations between all sources/objects.
Very complex calculations: Only simplified models. Color spaces like RGB are not exact models. Difficulties in calibration.
Burkhard/Domanska Cognitive Robotics Sensors 84
![Page 85: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/85.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 85
Camera Model (Geometry)
Diploma thesis Matthias Jüngel
![Page 86: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/86.jpg)
Conventions
Burkhard/Domanska Cognitive Robotics Sensors 86
Conventions: • right hand coordinate systems • angles are measured counter clockwise • orthogonal matrices, hence R -1 = R T
x
y
z
![Page 87: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/87.jpg)
Y
X
Burkhard/Domanska Cognitive Robotics Sensors 87
Camera Model
Xw,Yw,Zw : world coordinates X,Y, Z: camera coordinates x,y : image coordinates
Image from “Where am I?" -- Systems and Methods for Mobile Robot Positioning by J. Borenstein, H. R. Everett, and L. Feng
R T
(Xw,Yw,Zw)
![Page 88: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/88.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 88
Camera Parameters („Camera Matrix“) Extrinsic parameters:
Pose w.r.t. world coordinates Xw,Yw,Zw : • Location of focal point (3 DOF) • Orientation (3 DOF):
Camera Coordinates X,Y,Z with origin in focal point direction of Z is optical axis
Intrinsic parameters: Position of image plane (w.r.t. camera coordinates)
• Focal length f (1 DOF) • Intersection point of optical axis (2 DOF): Image coordinates x, y with origin at Z-axis and
orientation parallel to XY plane
![Page 89: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/89.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 89
Perspective Projection (Central Perspective)
Intercept Theorem Z : f = X : x = Y : y x = f/Z • X y = f/Z • Y
The image coordinates (x, y) are uniquely determined by Camera coordinates (X, Y, Z). Exactly valid under
ideal conditions (Pinhole camera)
Y
X
![Page 90: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/90.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 90
From World to Camera Coordinates
(X,Y,Z)
(XW,YW,ZW)
Camera
Y
X
Z
World
![Page 91: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/91.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 91
From World to Camera Coordinates: Translation
(X‘,Y‘,Z‘) = (XW,YW,ZW) - T = (XW,YW,ZW) - (XT,YT,ZT)
χ Translation T = (XT,YT,ZT)
Z‘
Y‘
X‘
(XW,YW,ZW)
Camera
Y
X
Z
![Page 92: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/92.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 92
From World to Camera Coordinates: Translation Usually, the translation vector T is not directly known. It must be computed along the „Kinematic Chain“, i.e. with calculations by translations along limbs and rotations in joints.
![Page 93: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/93.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 93
From Translated World to Camera Coordinates: Rotation
Camera Rotation R =
(X,Y,Z) = R ⋅ (X‘,Y‘,Z‘) = R ⋅ ( (XW,YW,ZW) - T ) = R ⋅ ( (XW,YW,ZW) - (XT,YT,ZT) )
Camera
Y
X
Z
Z‘
Y‘
X‘
r11 r12 r13
r21 r22 r23
r31 r32 r33
![Page 94: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/94.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 94
Ordering is Important
(X,Y,Z) = R ⋅ (X‘,Y‘,Z‘) = R ⋅ ( (XW,YW,ZW) - T ) = R ⋅ ( (XW,YW,ZW) - (XT,YT,ZT) )
First translation then rotation:
(X,Y,Z) = R ⋅ (X‘,Y‘,Z‘) = R ⋅ ( (XW,YW,ZW)) - T = R ⋅ ( (XW,YW,ZW)) - (XT,YT,ZT)
First rotation then translation:
![Page 95: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/95.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 95
Rotation in 2D Euclidean Space
changes to cos α sin α
1 0
(A,B)
x
y (a,b)
α
Rotation of a point by angle α from (a,b) to (A,B)
-sin α cos α
cos α sin α
= A B
a b
changes to -sin α cos α
0 1
changes to a cos α – b sin α a sin α + b cos α
a b Rotation matrix
-sin α cos α
cos α sin α
R =
a b
1 0
= a + b 0 1
![Page 96: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/96.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 96
Rotation in 2D Euclidean Space
x
y
X Y
(a,b) (A,B)
α
Rotation of a the old coordinate system (lower letters x,y, blue) by angle α into new rotated coordinate system (capital letters X,Y, red). It changes coordinates of a point from (a,b) to (A,B)
sin α cos α
cos α -sin α
= A B
a b
Rotation matrix sin α cos α
cos α - sin α
R =
Corresponds to rotation of the point (a,b) by inverse rotation R-1
Corresponds to rotation of the point (a,b) by angle - α
![Page 97: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/97.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 97
Rotation in 2D Euclidean Space
Angles between old and new axis: α,β,γ,δ Direction cosine: cosine of those angles. β = π/2−α γ = π/2+α δ = α cos β = - sin α cos γ = sin α
x
y
X Y
(a,b) (A,B)
α β
γ
δ
Rotation matrix with direction cosine
sin α cos α
cos α - sin α R =
cos γ cos α
cos δ cos β
=
![Page 98: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/98.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 98
Rotation in 3D Euclidean Space
r11 r12 r13
r21 r22 r23
r31 r32 r33
R =
rij = direction cosine, i.e. cosine of angles between old and new axis (dot product of the unit vectors)
Can also be described by the direction cosine matrix
x
y
z
X
Y Z
![Page 99: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/99.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 99
Rotation in 3D Euclidean Space
Can also be described by the direction cosine matrix
r11 r12 r13
r21 r22 r23
r31 r32 r33
R =
rij = direction cosine, i.e. cosine of angles between old and new axis (dot product of the unit vectors)
But: Only 3 of 9 parameters are independend, 3D-Rotation has only 3 degrees of freedom (DOF) Usually, other angles are given, e.g. rotation around an axis x
y
z
X
Y Z
![Page 100: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/100.jpg)
Rotation Around a Single Axis in 3D
Burkhard/Domanska Cognitive Robotics Sensors 100
x
y
z
Y Z
Rotation around old x-axis rotates coordinates in y-z-plane from y-z to Y-Z
1 0 0
0 cos α sin α 0 -sin α cos α
α
![Page 101: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/101.jpg)
Rotation Around a Single Axis in 3D
Burkhard/Domanska Cognitive Robotics Sensors 101
x
y
z
Y Z
Convention: Rotation around an axis are oriented counter clockwise when looking „from above“, i.e. against orientation of the rotation axis.
cos α 0 −sin α 0 1 0 sin α 0 cos α
1 0 0 0 cos α sin α 0 -sin α cos α
cos α sin α 0 0 0 1
X
Rotation by α around x-axis
Rotation by α around y-axis
Rotation by α around z-axis
-sin α cos α 0
![Page 102: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/102.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 102
Yaw, Pitch, Roll in Aviation and Nautics
x, roll
y, pitch
z, yaw
Angles are measured w.r.t. world coordinates: Extrinsic parameters.
![Page 103: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/103.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 103
Yaw, Pitch, Roll in Robotics
Angles are measured (by perceptors) with respect to local coordinates: Intrinsic parameters.
z
y x
![Page 104: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/104.jpg)
Euler Angles Rotations in 3D have 3 degrees of freedom (DOF). They can be described by 3 successive planar rotations, e.g.:
Burkhard/Domanska Cognitive Robotics Sensors 104
From Wikipedia
![Page 105: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/105.jpg)
Euler Angles
Burkhard/Domanska Cognitive Robotics Sensors 105
From Wikipedia
Any target coordinate system can be reached using a specific sequence of rotations with “Euler Angles” (α,β,γ). The example uses the Z-X'-Z" system : Rotation around z-axis (Z) Rotation about new X-axis (X ') Rotation about new Z-axis (Z'')
N is the intersection line of x-y-plane and X-Y-plane (perpendicular to both z and Z axis )
![Page 106: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/106.jpg)
Euler Angles
Burkhard/Domanska Cognitive Robotics Sensors 106
There are 12 different possibilities to define “Euler Angles”. Rotations are performed around the new axis: “Intrinsic rotations”. The resulting rotation can be expressed by multiplication of planar rotations.
Another example: ZY‘X''-system: Rotation around z-axis (Z) Rotation about new Y-axis (Y ') Rotation about new X-axis (X'')
![Page 107: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/107.jpg)
Euler Angles
Burkhard/Domanska Cognitive Robotics Sensors 107
From Wikipedia
Rotation around z-axis (Z) Rotation about new X-axis (X ') Rotation about new Z-axis (Z'')
1 0 0 0 cos β sin β 0 -sin β cos β
cos α sin α 0 0 0 1
Rotation by β around new X-axis
Rotation by α around new z-axis
-sin α cos α 0
cos γ sin γ 0 0 0 1
Rotation by γ around new Z-axis
-sin γ cos γ 0 R =
![Page 108: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/108.jpg)
Camera Model Nao
Burkhard/Domanska Cognitive Robotics Sensors 108
(XW,YW,ZW) World coordinates
Translation TNP =(XNP,YNP,ZNP) to Neck Pitch joint
Translation TF =(XF,YF,ZF) to focal point Rotation Neck Yaw α Rotation Neck Pitch β
Neck Pitch is located at axis of Neck Yaw: No translation needed between these joints using translation from world to NeckPitch.
Camera Coordinates (X,Y,Z)
XW
YW
ZW
View direction
Y
Z
X
Note: View direction is X-axis (while usually it is Z-axis)
![Page 109: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/109.jpg)
Camera Model Nao
Burkhard/Domanska Cognitive Robotics Sensors 109
World coordinates are transformed: Translation to neck pitch joint (XNP,YNP,ZNP) Rotation Neck Yaw α Rotation Neck Pitch β Translation to camera focal point (XF,YF,ZF)
XW
YW
ZW
x
y
z
x y
z
x
y
z
(XW,YW,ZW) (X,Y,Z)
View direction
Y
Z
X
= 1 0 0 0 cos α sin α 0 -sin α cos α
cos β 0 −sin β 0 1 0 sin β 0 cos β
XW - XNP
ZW - ZNP YW - YNP −
XF
ZF
YF
X
Z
Y
![Page 110: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/110.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 110
Perspective Projection (Central Perspective)
Intercept Theorem Z : f = X : x = Y : y x = f/Z • X y = f/Z • Y
The image coordinates (x, y) are uniquely determined by Camera coordinates (X, Y, Z).
Y
X
![Page 111: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/111.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 111
Perspective Projection (Central Perspective)
Intercept Theorem X : f = Y : y = Z : z y = f/X • Y z = f/X • Z
The image coordinates (x, y) are uniquely determined by Camera coordinates (X, Y, Z).
Y
X
Usually Z-axis points in view direction, while for Simulated Nao, view direction is X-direction. X
y
z
Y
Z
(x,y)
(X,Y,Z)
![Page 112: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/112.jpg)
Camera Model Simulated Nao
Burkhard/Domanska Cognitive Robotics Sensors 112
X = R ( X W - TNP ) - TF
y = f/X • Y z = f/X • Z
where f is the focal length, and X,Y, Z are calulated by
Camera Model
Rotation matrix by multiplication of matrices
= 1 0 0 0 cos α sin α 0 -sin α cos α
cos β 0 −sin β 0 1 0 sin β 0 cos β
XW - XNP
ZW - ZNP YW - YNP −
XF
ZF
YF
Y
X
Z
![Page 113: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/113.jpg)
Vision Perceptor of Simulated Nao
Burkhard/Domanska Cognitive Robotics Sensors 113
Calculation of the values for the vision perceptor:
• The distance is given by X, the angles can be calulated from y, z, f . • Alternatively, the angles can be calculated as portions from the view range (120 degrees in total), e.g. y : Y = delta-y : 60 , i.e. delta-y = 60 y / Y Y is maximum value of the image-area
![Page 114: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/114.jpg)
Inverse Camera Model Nao
Burkhard/Domanska Cognitive Robotics Sensors 114
X W = R -1(X +TF ) + T
X = R ( X W - TNP ) - TF Camera Model
Inverse Camera Model
X= (X,Y,Z) can not be completely reconstructed from x, y only Additional information is needed, e.g.
• Distance Z • Size of an object • Location on ground
x = f/Z • X y = f/Z • Y
X = x • Z / f Y = y • Z / f
Change for Simulated Nao as before: View direction is X-direction.
![Page 115: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/115.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 115
Omnidirectional Camera 360 degrees of view Can be realized by special (conic) mirror: Different surface curvature for better resolution at close range Needs appropriate camera model and methods
![Page 116: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/116.jpg)
Rotation Matrix for Intrinsic Rotations Intrinsic Rotations: Rotations are given w.r.t. recent object coordinates (e.g. Euler angles in ZX‘Z‘‘ system).
Burkhard/Domanska Cognitive Robotics Sensors 116
If A, B, C are successive intrinsic rotations, then the resulting rotation is described by R = C B A
![Page 117: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/117.jpg)
Rotation Matrix for Extrinsic Rotations Extrinsic Rotations: Rotations are given w.r.t. fixed world coordinates (e.g. yaw-pitch-roll in Aviation/Nautics).
result of the first rotation is given by A result of the first two rotations is given by intrinsic rotations (ABA-1) and A resulting in (ABA-1)A = AB result of all three rotations is then given by intrinsic rotations ((AB)C(AB)-1) and (AB) resulting in ((AB)C(AB)-1) (AB) = ABC
Burkhard/Domanska Cognitive Robotics Sensors 117
If A, B, C are extrinsic rotations, then the resulting rotation is described by R = A B C
![Page 118: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/118.jpg)
Problems with Camera Model
Burkhard/Domanska Cognitive Robotics Sensors 118
Position of camera (extrinsic parameters): Errors in the kinematic chain: • joint angles (backslash, sensor noise) • distortion during motion Can be determined/corrected by known landmarks (cf. localization methods: later)
![Page 119: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/119.jpg)
Problems with Camera Model Geometrical distortion by optics (intrinsic parameters) by refraction of the light at the inlet and outlet from the media
Burkhard/Domanska Cognitive Robotics Sensors 119
![Page 120: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/120.jpg)
Burkhard/Domanska Cognitive Robotics Sensors 120
Problems with Camera Model Distortions determined/corrected/calibrated by experiments: Imaging parameters determined by corresonding points in reality and in image.
![Page 121: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/121.jpg)
Problems with Camera Model
Burkhard/Domanska Cognitive Robotics Sensors 121
Example for calibration: Ceiling camera in Small Size League (FU-Fighters Berlin)
![Page 122: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/122.jpg)
Problems with Camera Model Motion Blur (delays while reading pixels during motion)
Burkhard/Domanska Cognitive Robotics Sensors 122
You can also see color distortion (blue in the corners)
![Page 123: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/123.jpg)
Outline Introduction Sensors: General Considerations Signals Sensors: Special Types Vision (introductory, later more) Camera Model Image Processing (introductory, later more) Scene Interpretation (introductory, later more)
Burkhard/Domanska 123 Cognitive Robotics Sensors
![Page 124: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/124.jpg)
Image Processing Given a pixel matrix: what is the content of the image? Can include many processes: • Signal processing (noise reduction, …) • Low level identification (line detection, color detection,…) • Object identification • Relation between objects • Scene reconstruction (3D-model) • Scene interpretation
Burkhard/Domanska Cognitive Robotics Sensors 124
![Page 125: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/125.jpg)
125
Visual Information for Real Nao: Images
Interpretation needs complex image processing.
It is possible to provide synthetic images for simulation, but standard in 3D league are already preprocessed data.
Burkhard/Domanska Cognitive Robotics Sensors
![Page 126: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/126.jpg)
Sequential Image Processing by Machines
126 Burkhard/Domanska Cognitive Robotics Sensors
![Page 127: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/127.jpg)
127 Burkhard/Domanska Cognitive Robotics Sensors
![Page 128: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/128.jpg)
Preprocessing of Images
128 Burkhard/Domanska Cognitive Robotics Sensors
Color classification
Boundary of objects
Identification of objects
![Page 129: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/129.jpg)
Identification Identification of an individual object: Based on known features Features computed e.g. from • Colors • Shapes • Size • Statistics in useful regions (SIFT, SURF) • Relations between points
129 Burkhard/Domanska Cognitive Robotics Sensors
![Page 130: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/130.jpg)
Identification Each object has a (high dimensional) feature vector (“signature”)
+
+ + + Object 2
+
Object 1
+
+
+
130 Burkhard/Domanska Cognitive Robotics Sensors
In simple cases, the objects can be identified using explicit world knowledge (e.g. “the ball is orenge”). In general, the world is more complex.
![Page 131: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/131.jpg)
Identification Nearest Neighbor Method Compare observed object by similarity to known objects Choose most similar object (or reject)
Example: Face Recognition 1. Identify related regions 2. Identify (biometric) features 3. Compare with database
Available by commercial products Works well with frontal faces Depends on available resources
+
+ + + Object 2
+
Object 1
+
+
+ ?
131 Burkhard/Domanska Cognitive Robotics Sensors
![Page 132: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/132.jpg)
Classification Classify objects Based on known features, properties, relations Problems with diversity of objects in the same class
132 Burkhard/Domanska Cognitive Robotics Sensors
![Page 133: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/133.jpg)
Classes = partitions of the feature room
-
+ + + + +
+
+ - - -
- -
- -
-
-
- -
+
+ + + +
+ +
+
+ + + +
+ +
- - - - - -
-
- - - - - -
- - - -
- -
- - - - -
?
Classification methods e.g. • Nearest Neighbor • Decision tree • Neural Network • Support Vector Machine (SVM)
Problems: • High Dimensionality
133
Classification
Burkhard/Domanska Cognitive Robotics Sensors
![Page 134: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/134.jpg)
Support Vector Machines (SVM): Classification by orientation relative to partition line
?
Construction of a partition line from examples (Machine Learning)
134
Classification
Burkhard/Domanska Cognitive Robotics Sensors
![Page 135: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/135.jpg)
Construction of a partition line by examples
-
+ + + + +
+
+ - - -
- -
- -
-
-
- -
+
+ + + +
+ +
+
+ + + +
+ +
- - - - - -
-
- - - - - -
- - - -
- -
- - - - -
Problem: Which line is the best?
135
Classification
Burkhard/Domanska Cognitive Robotics Sensors
![Page 136: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/136.jpg)
Construction of a partition line by examples
Problem: Which line is the best?
136
Classification
-
+ + + + +
+
+ - - -
- -
- -
-
-
- -
+
+ + + +
+ +
+
+ + + +
+ +
- - - - - -
-
- - - - - -
- - - -
- -
- - - - -
Burkhard/Domanska Cognitive Robotics Sensors
![Page 137: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/137.jpg)
Construction of a partition line by examples
Problem: Which line is the best? Data may have been noisy! (overfitting problem)
137
Classification
-
+ + + + +
+
+ - - -
- -
- -
-
-
- -
+
+ + + +
+ +
+
+ + + +
+ +
- - - - - -
-
- - - - - -
- - - -
- -
- - - - -
Burkhard/Domanska Cognitive Robotics Sensors
![Page 138: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/138.jpg)
Generalization Problem: The classification of new objects depends on the choice of the learning method (inductive bias)
138
Classification
-
+ + + + +
+
+ - - - - -
- -
-
-
- -
+
+ + + + +
+ +
+ + + + + +
- - - - - - -
- - - - - - - - - -
- - - - - - -
-
+ + + + +
+
+ - - - - -
- -
-
-
- -
+
+ + + + +
+ +
+ + + + + +
- - - - - - -
- - - - - - - - - -
- - - - - - - -
+ + + + +
+
+ - - - - -
- -
-
-
- -
+
+ + + + +
+ +
+ + + + + +
- - - - - - -
- - - - - -
- - - -
- - - - - - -
Burkhard/Domanska Cognitive Robotics Sensors
![Page 139: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/139.jpg)
Outline Introduction Sensors: General Considerations Signals Sensors: Special Types Vision (introductory, later more) Camera Model Image Processing (introductory, later more) Scene Interpretation (introductory, later more)
Burkhard/Domanska 139 Cognitive Robotics Sensors
![Page 140: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/140.jpg)
What the Robot Sees
Burkhard/Domanska Cognitive Robotics Sensors 140
![Page 141: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/141.jpg)
Scene Interpretion Badly posed problem: Reconstruction of a 3D scene from 2D image
M.C.Escher
141 Burkhard/Domanska Cognitive Robotics Sensors
![Page 142: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/142.jpg)
Scene Interpretion There are many available informations • i.g. enough to reconstruct a scene even from 2D images by
using world knowledge. • i.g. redundant for dealing with noise. But: It is hard to compute.
Burkhard/Domanska Cognitive Robotics Sensors 142
![Page 143: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/143.jpg)
Burkhard/Domanska Cognitive Robotics Sensors
Exploiting Redundancy
Where am I ? Where is the ball ?
143
![Page 144: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/144.jpg)
Burkhard/Domanska Cognitive Robotics Sensors
Exploiting Redundancy
The size of the goal defines a circle of possible positions of the observer
144
![Page 145: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/145.jpg)
Burkhard/Domanska Cognitive Robotics Sensors
Exploiting Redundancy
The size of the ball defines a circle of possible positions of the ball relative to the observer
145
![Page 146: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/146.jpg)
Burkhard/Domanska Cognitive Robotics Sensors
Exploiting Redundancy
The ball lies on a line before the penalty border line
146
![Page 147: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/147.jpg)
Burkhard/Domanska Cognitive Robotics Sensors
Exploiting Redundancy
The ball lies on a certain line between goal post and observer
147
![Page 148: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/148.jpg)
Burkhard/Domanska Cognitive Robotics Sensors
Exploiting Redundancy
Combination yields 2 possible positions
148
![Page 149: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/149.jpg)
Burkhard/Domanska Cognitive Robotics Sensors
Exploiting Redundancy
Combination yields 2 possible positions
149
![Page 150: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/150.jpg)
Integration of Information
Processing on different levels
Preprocessing of sensory data
Feature detection
Object classification
Scene Interpretation
...
150 Burkhard/Domanska Cognitive Robotics Sensors
![Page 151: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/151.jpg)
Integration of Information Integration of different sensors
... ... ... ... ...
In most systems only partially implemented
151 Burkhard/Domanska Cognitive Robotics Sensors
![Page 152: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/152.jpg)
Integration of Information Integration over time Attendance/Focussing
... ... ... ... ...
... ... ... ... ...
... ... ... ... ...
152 Burkhard/Domanska Cognitive Robotics Sensors
In most systems only partially implemented
![Page 153: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/153.jpg)
World Model
Objectives behind: Keep perceived information because • Environment only partially observable • Observations are unreliable and noisy Using • Knowledge (e.g. maps) about the world • Tracking of objects over time
New belief= old belief + new sensory data
153 Burkhard/Domanska Cognitive Robotics Sensors
World Model is called “belief”. Because it needs not to be correct!
![Page 154: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/154.jpg)
Burkhard/Domanska Cognitive Robotics Sensors
World model
Belief_new := update (Perception, Belief_old);
new perception from recent image
+ =
Update of belief
154
![Page 155: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/155.jpg)
Scene Interpretation Calculcate spatial model from geometrical/topological data using • maps • perceived objects • relations between objects
Usually by statistical methods, e.g. Bayesian methods Probability to be at location s given an observation z: P(s|z) = P(z|s)·P(s) / P(z)
155 Burkhard/Domanska
Where am I?
Cognitive Robotics Sensors
![Page 156: Sensors - hu-berlin.de · Burkhard/Domanska Cognitive Robotics Sensors 12 Sensors in Robotics Exploit physical/chemical … features, e.g. • Current - power - resistance - inductance](https://reader033.fdocuments.net/reader033/viewer/2022050522/5fa5a5db5aa0bb13122ce4cc/html5/thumbnails/156.jpg)
Scene Interpretation Calculate mental attitudes of other actors using • communication • observation • behavior patterns
156 Burkhard/Domanska
What will he do?
Cognitive Robotics Sensors