Robótica Inteligente L. Enrique Sucar Leonardo Romero Marco López ITESM Cuernavaca.
Desenvolvimento de Aplicações de Robótica Inteligente em ...
Transcript of Desenvolvimento de Aplicações de Robótica Inteligente em ...
28 Sept. 2007
1
Prof. Ph.D. Prof. Ph.D. Fernando Fernando OsórioOsório -- Applied Computing / Applied Computing / UnisinosUnisinosProfaProfa . Ph.D. . Ph.D. SoraiaSoraia MusseMusse -- Computing Science / PUCComputing Science / PUC --RSRSProf. Prof. M.ScM.Sc. . FarleiFarlei HeinenHeinen -- Computing Eng. / Computing Eng. / UnisinosUnisinosM.ScM.Sc. Milton Roberto . Milton Roberto HeinenHeinen -- Ph.D. Student at UFRGSPh.D. Student at UFRGSProf. Prof. Ph.DPh.D Christian Christian KelberKelber -- Electrical Eng. / Electrical Eng. / UnisinosUnisinosGustavo Gustavo PessinPessin -- M.ScM.Sc. at . at UnisinosUnisinos
GRAPHIT - Computer Graphics and Vision Group (Unisinos/PUC-R S)GPVA - Autonomous Vehicles Research Group (Unisinos)GIA - Artificial Intelligence Research Group (Unisinos)RBV - Rede Brasileira de Visualização [FINEP/Brazil]
IEEE / DVP - Distinguished Visitors Program Latin America
Applied Computing Applied Computing PostPost--grad. Programgrad. Program-- PIPCAPIPCA
UNISINOS University UNISINOS University -- BrazilBrazil
Desenvolvimento de Aplicações de Desenvolvimento de Aplicações de Robótica Inteligente em Robótica Inteligente em
Ambientes de Realidade VirtualAmbientes de Realidade Virtual
Research group:
Fernando Osório, Ph.D. / IEEE CS DVP
I JAC - Workshop de I.A. / UNISC 2007 - Palestra Convidada
28 Sept. 2007
2
IEEE / DVP - Distinguished Visitors Program Latin America
Prof. Fernando Santos Osório - IEEE MemberApplied Computing Research Post-Graduation Program - PIPCAUNISINOS University - Brazil (Porto Alegre - Southern Region)IEEE Computer Society DVP Program
Jesuit UniversityWith: - 30.000 Students- 900 Professors- 16 PPGs (post-grad programs)
PPG CAPES PPG CAPES NotaNota 44
28 Sept. 2007
3
Presentation Topics
Agenda:
1. Introduction: VR - Hierarchy of Models
2. VR and Simulation
Geometry, Physics, Behaviour, Knowledge and Cognition
3. Physics Simulation Tools
Opensteer, ODE, PhysX, Deformable/Dynamic
4. Intelligent Behaviour
Agents: Perception, Action, Behaviour
Autonomous Robots and Agents - Control
Multi-Agents Systems - Knowledge
5. Applications: Autonomous Robots VR Simulation Tools
6. Conclusions and New TrendsVídeo Demo Web/Java
28 Sept. 2007
4
Introduction VR - Virtual Reality
[Paul Milgram et al. 95]
From REAL to VIRTUAL3D + Immersion + Interaction
Virtual Reality
28 Sept. 2007
5
VISUALIZING 3D & VIRTUAL ENVIRONMENTS
Virtual RealityVirtual Reality...
* VRML - 3D Worlds (Geometry)
* QTVR - Panorama 3D (Images)
Introduction VR - Virtual Reality
\\VRMLVRML
Virtual Reality
28 Sept. 2007
62424
IRISA / INRIA IRISA / INRIA -- FranceFrancehttp://www.irisa.fr/lagadic/demo/demohttp://www.irisa.fr/lagadic/demo/demo--ar3/demoar3/demo--ar3ar3--eng.htmleng.html
Real World Integrated with Virtual Objects
Augmented Reality
28 Sept. 2007
7
Virtual Reality
Virtual RealityVirtual Reality...
* 3D Virtual Environment
* Interaction => Virtual Reality Devices
* Immersion => Virtual Reality Devices
* Realism => Graphical Realism (photo-realism)
Movements
Interaction Real x Virtual
"Physics Realism"
VISUALIZING 3D & VIRTUAL ENVIRONMENTS
How to do it?
28 Sept. 2007
8
1. Introduction
Sources of Inspiration:
3D Virtual Worlds - Hierarchy of Models
[Funge 1999]
28 Sept. 2007
9
1. Introduction
Sources of Inspiration:
3D Virtual Worlds - Hierarchy of Models
[Funge 1999]
28 Sept. 2007
10
1. Introduction
Sources of Inspiration:
3D Virtual Worlds - Hierarchy of Models
[Funge 1999]
28 Sept. 2007
11
1. Introduction
Sources of Inspiration:
3D Virtual Worlds - Hierarchy of Models
[Funge 1999]
28 Sept. 2007
12
1. Introduction
Sources of Inspiration:
3D Virtual Worlds - Hierarchy of Models
[Funge 1999]
28 Sept. 2007
13
1. Introduction
Sources of Inspiration:
3D Virtual Worlds - Hierarchy of Models
[Funge 1999]
PhantomPhantom
HaptionHaption
CyberForceCyberForce
OmegaOmega
28 Sept. 2007
14
1. Introduction
Sources of Inspiration:
3D Virtual Worlds - Hierarchy of Models
[Funge 1999]
[Ari Chapiro - Dance]
[CromosLab]
Agents
28 Sept. 2007
15
1. Introduction
Sources of Inspiration:
3D Virtual Worlds - Hierarchy of Models
[Funge 1999]
28 Sept. 2007
16
1. Introduction
Sources of Inspiration:
3D Virtual Worlds - Hierarchy of Models
[Funge 1999]
28 Sept. 2007
17
1. Introduction
Sources of Inspiration:
3D Virtual Worlds - Hierarchy of Models
[Funge 1999]
Knowledge
AutonomousBehaviour
28 Sept. 2007
18
2. VR and Simulation
Sources of Inspiration:
3D Virtual Worlds - Hierarchy of Models
[Funge 1999]
Virtual World
xx
Real World
28 Sept. 2007
19
Presentation Topics
Agenda:
1. Introduction: VR - Hierarchy of Models
2. VR and Simulation
Geometry, Physics, Behaviour, Knowledge and Cognition
3. Physics Simulation Tools
Opensteer, ODE, PhysX, Deformable/Dynamic
4. Intelligent Behaviour
Agents: Perception, Action, Behaviour
Autonomous Robots and Agents - Control
Multi-Agents Systems - Knowledge
5. Autonomous Robots VR Simulation Tools
6. Conclusions and New TrendsVídeo Demo Web/Java
28 Sept. 2007
20
Sources of Inspiration:
3D Virtual Worlds - Hierarchy of Models
[Funge 1999]
Virtual World
xx
Real World
Increasing Reality in VR Applications: Physical and Behavioral Simulation
2. VR and Simulation
28 Sept. 2007
21
1. Introduction
RealisticVR
Increasing Reality in VR Applications: Physical and Behavioral Simulation
Virtual World
xx
Real World
Real WorldReal WorldSimulationSimulation
28 Sept. 2007
22
Simulation Tools:
* ODE - Open Dynamics Engine
* OpenSteer
* PhysX AGEIA
* Deformable Objects and Fluids:- Finite Elements Methods- Spring-Mass Systems- CFD (Computational Fluid Dynamics)- Level Set Methods
VR Simulation: Some important questions...
3. Physics Simulation Tools
28 Sept. 2007
23
Simulation Tools:
* ODE - Open Dynamics Engine
* OpenSteer
* PhysX AGEIA
* Deformable Objects and Fluids:- Finite Elements Methods- Spring-Mass Systems- CFD (Computational Fluid Dynamics)- Level Set Methods
VR Simulation: Some important questions...
3. Physics Simulation Tools
Physics:
Physical structure: resistance, mass, density, elasticity;Position and orientation in the 3D space; Kinematics and Dynamics;Linear and angular velocities; Motion (w/ forces and torques), trajectories; Acceleration, deceleration;Attraction and repulsion;Gravity, friction, inertia; Kinetic and potential energy; Laws of energy conservation, linear and angular momentum;Collisions and reaction to collisions;Steering models (wheeled cars, aircrafts, projectiles, boats and ships); Articulated Rigid Bodies Simulation (skeleton, robotic arm);Dynamic Simulation of Deformable Objects: elastic objects; Fluid simulation and Particle Systems (fire, smoke, clouds and liquids).
28 Sept. 2007
24
* Deformable Objects and Fluids
- Finite Elements Methods
- Spring-Mass Systems
- CFD (Computational Fluid Dynamics)
- Level Set Methods
3. Physics Simulation Tools
28 Sept. 2007
25
* ODE - Open Dynamics Engine
Simulation of Articulated Rigid Body Dynamics
Open Source Library (C/C++ API)
Used with OSG, Ogre3D, CrystalSpace, ...
3. Physics Simulation Tools
Slider jointSlider jointHinge jointHinge jointBall and socket jointBall and socket joint
Universal jointUniversal joint
28 Sept. 2007
26
* ODE - Open Dynamics Engine
Simulation of Articulated Rigid Body Dynamics
3. Physics Simulation Tools
Physics Simulation:
- Gravity, friction, acceleration, deceleration; - Generation of motion: applying forces and torques (motors);- Collision avoidance and treatment (reaction, object bounce);- Kinematics models and rigid body dynamics simulation; - Different types of joints with actuators (motors)
Webbots uses ODE [Cyberbotics]
28 Sept. 2007
27
* ODE - Open Dynamics Engine
3. Physics Simulation Tools
28 Sept. 2007
28
* ODE - Open Dynamics Engine
3. Physics Simulation Tools
28 Sept. 2007
29
* ODE - Open Dynamics Engine
3. Physics Simulation Tools
WebotsCyberbotics
28 Sept. 2007
30
* ODE - Open Dynamics Engine
3. Physics Simulation Tools
Juice[Nate W.]
28 Sept. 2007
31
3. Physics Simulation Tools
Simulation Tools:
* ODE - Open Dynamics Engine
* OpenSteer
* PhysX AGEIA
* Deformable Objects and Fluids:- Finite Elements Methods- Spring-Mass Systems- CFD (Computational Fluid Dynamics)- Level Set Methods
VR Simulation: Some important questions...REAL TIME SIMULATIONREAL TIME SIMULATION
28 Sept. 2007
32
Intelligent Agents:
Agents: Perception, ActionAgent Behaviours
Control ArchitecturesAutonomous Agents Multi-Agents SystemsKnowledge / Reasoning
4. Intelligent Behaviour
Comportamento
28 Sept. 2007
33
Intelligent Agents:
Agents: Perception, ActionAgent Behaviours
Control ArchitecturesAutonomous Agents Multi-Agents SystemsKnowledge Integration
4. Intelligent Behaviour
Reactive-Deliberative Architecture
Control Architectures: Reactive, Deliberative, Hierarchical, Hybrid
Hybrid Architecture
Architecture BDI (Beliefs-Desires-Intentions)
F. Osório et al.[Virtual Concept 2005]
28 Sept. 2007
34
4. Intelligent Behaviour
RobotSensors Actuators
PositioningEstimator
(Monte Carlo)
Environment Representation Maps
Polygonal
Grid
Topologicaland
Semantic
SharedMemory
Control Layers
Path Planning
Behaviors
Sequencer
Control Modules
Robust Hybrid Control COHBRA / HyCAR
SimRob3DUnisinos
COHBRACOHBRA: : Controle Híbrido Controle Híbrido de Robôs Autônomosde Robôs Autônomos
HyCARHyCAR -- Hybrid Control Hybrid Control for Autonomous Robotsfor Autonomous Robots
Published at HIS2002Hybrid Intelligent Systems 2002, Chile
28 Sept. 2007
35
Intelligent Agents:
Agents: Perception, ActionAgent Behaviours
Control ArchitecturesAutonomous Agents Multi-Agents SystemsKnowledge / Reasoning
4. Intelligent Behaviour
Robotic
Teams, Squads, Swarms
Artificial Intelligence Tools
28 Sept. 2007
36
Presentation Topics
Agenda:
1. Introduction: VR - Hierarchy of Models
2. VR and Simulation
Geometry, Physics, Behaviour, Knowledge and Cognition
3. Physics Simulation Tools
Opensteer, ODE, PhysX, Deformable/Dynamic
4. Intelligent Behaviour
Agents: Perception, Action, Behaviour
Autonomous Robots and Agents - Control
Multi-Agents Systems - Knowledge
5. Applications: Autonomous Robots VR Simulation Tools
6. Conclusions and New Trends
28 Sept. 2007
37
5. Applications: VR Simulation Tools
Applications @ Unisinos
1. Autonomous Robots in VR Environments
SimRob3D - Mobile Robots Simulator
SEVA 3D - Autonomous Vehicle Parking
LEGGEN - Legged (articulated) Robots Simulator
Robombeiros - Multi-Robots Fire Fighting
28 Sept. 2007
38
5. Applications: VR Simulation Tools
SimRob3D
- Our Simulation Tools: SimRob2D (Khepera)SimRob3D Seva2D
Autonomous Robots in VR Environments
28 Sept. 2007
39
5. Applications: VR Simulation Tools
SimRob3D Simulator
> Sensors: Infrared, Sonar, Bumper
> Actuators / Kinematics: Differential, Ackerman
> Realistic Simulation Model: 3D World + noise / error (imprecise sensors and actuators)
Autonomous Robots in VR Environments
28 Sept. 2007
40
5. Applications: VR Simulation Tools
SimRob3D
Simulator
Autonomous Robots in VR Environments
28 Sept. 2007
41
5. Applications: VR Simulation Tools
SEVA 3D - "Sistema de Estacionamento de Veículos Autônomos"
Sources of Inspiration:
- Baja Buggy remotely controlled by a cell phoneC. Kelber - UNISINOS, Brazil
Published at:IEEE WCCIIJCNN 2006
Autonomous Robots in VR Environments
28 Sept. 2007
42
5. Applications: VR Simulation Tools
SEVA 3D
28 Sept. 2007
43
SEVA 3D Simulator
> Vehicle Simulation xx Vehicle Control
SimRob3D SEVA3D
Commands
Sensors
Control:SEVA3D-A (FSA)SEVA3D-N (Neural)
ControlSimulation
MotorActions
SensorialInformation
Kinematics:Estimation ofPosition and Orientation
Perception:Sensor Simulation
3DWorld
RobotModel
Visualization
SimRob3D SEVA3D
Commands
Sensors
Control:SEVA3D-A (FSA)SEVA3D-N (Neural)
ControlSimulation
MotorActions
SensorialInformation
Kinematics:Estimation ofPosition and Orientation
Perception:Sensor Simulation
3DWorld
RobotModel
Visualization
Autonomous Robots in VR Environments
5. Applications: SEVA 3D
28 Sept. 2007
44
SEVA: FSA - Finite State Automaton
Inputs:
- Sonar Sensors: Stochastic ray-casting / 3D cone)
Outputs:
- Steering Wheel Angle- Gas pedal (car speed + direction: fwd, back)
States:
Autonomous Robots in VR Environments
5. Applications: SEVA 3D
28 Sept. 2007
45
SEVA: NEURAL FSA - Learning the FSA...
Si(t) + Si(t-1) + ...Temporal Window
Adapted Jordan-Net using RProp Learning
5. Applications: SEVA 3D
28 Sept. 2007
46
SEVA3D - Autonomous Vehicle Parking Simulator
3D
SensorsActuatorsKinematicsFSA CtrlANN Ctrl
5. Applications: SEVA 3D
28 Sept. 2007
47
5. Applications: VR Simulation Tools
Applications @ Unisinos
1. Autonomous Robots in VR Environments
SimRob3D - Mobile Robots Simulator
SEVA 3D - Autonomous Vehicle Parking
LEGGEN - Legged (articulated) Robots Simulator
Robombeiros - Multi-Robots Fire Fighting
28 Sept. 2007
48
5. Applications: VR Simulation Tools
Legged Robots Evolution and Walking Control
Sources of Inspiration:
Autonomous Robots in VR Environments
[EPFL]
28 Sept. 2007
49
5. Applications: VR Simulation Tools
Legged Robots Evolution and Walking Control
Sources of Inspiration: LEGGEN - Published at: IEEE WCCI CEC 2006SBIA 2006
33
Autonomous Robots in VR Environments
28 Sept. 2007
50
28 Sept. 2007
51
5. Applications: VR Simulation Tools
LEGGEN - Legged Robots Evolution and Walking ControlAutonomous Robots in VR Environments
Simulation of Robots: 3D Realistic Virtual Environments- Sensors: infrared, sonar, bumpers, gyro (accelerometers),
GPS, compass, light and vision sensors, etc. - Actuators: legs and arms with angular motors (joints) - Physics: collision, kinematics, rigid body dynamics
Simulation of Legged Autonomous Robots:- Robot ControlArchitectures Implementation
Genetic Evolved Control of Articulated Robots (w/legs)
28 Sept. 2007
52
5. Applications: VR Simulation Tools
LEGGEN - Legged Robots Evolution and Walking ControlAutonomous Robots in VR Environments
Simulation of 3D Realistic Virtual Legged Robots LEGGEN Simulator - Tools:
1. OSG- Open Scene Graph (OpenGL + Extensions)[ http://www.openscenegraph.org/ ]
2. ODE - Open Dynamics EngineRigid Body Physics Simulation(gravity, inertia, friction, collision, joints, etc)[ http://www.ode.org/ ]
3. GALib - Genetic Algorithms Simulation[ http://www.lancet.mit.edu/ga/ ]
4. Robot Control FSM: Finite State Machine = Sense + Act
28 Sept. 2007
53
5. Applications: VR Simulation Tools
LEGGEN - Legged Robots Evolution and Walking ControlAutonomous Robots in VR Environments
Simulation main goals:
- Evaluate different Robot Models(hardware configurations)IEEE WCCI / CEC 2006 - Vancouver, Canadá
- Evaluate different Fitness FunctionsIBERAMIA / SBIA - Ribeirão Preto, SP Robot Models
Boston Dynamics
Evaluate different robot models in order to select a better hardware configuration
28 Sept. 2007
54
LEGGEN SIMULATOR
Simulation Results:
28 Sept. 2007
55
LEGGEN SIMULATOR
Simulation RESULTS:
Tetrapod Video - Distance, Gyro
28 Sept. 2007
56
LEGGEN SIMULATOR
Simulation RESULTS: Tetrapod Video - "bloopers"
28 Sept. 2007
57
5. Applications: VR Simulation Tools
Applications @ Unisinos
1. Autonomous Robots in VR Environments
SimRob3D - Mobile Robots Simulator
SEVA 3D - Autonomous Vehicle Parking
LEGGEN - Legged (articulated) Robots Simulator
Robombeiros - Fire Fighting
28 Sept. 2007
58
Robombeiros - Fire Fighting VR Simulation
Virtual Simulation Environment:* 2D and 3D Simulation* Simulation of fire propagation* Autonomous fire-fighting team* Define: Strategy, Mission, Execution
Fire PropagationSimulation:
- Direction and Speedof wind
- Vegetation type andcoverture density
(speed of propagation)- Terrain
Figure: 2D Simulation using SDL library => http://pessin.googlepages.com/robombeiros
Published at SVR 2007 (Symposium on Virtual and Augmented Reality) [G. Pessin, F. Osório, S. Musse, V. Nonnenmacher, S. Ferreira]
28 Sept. 2007
59
Robombeiros - Fire Fighting VR Simulation
Virtual Simulation Environment:
http://pessin.googlepages.com/robombeirosPublished at SVR 2007 (Symposium on Virtual and Augmented Reality) [G. Pessin, F. Osório, S. Musse, V. Nonnenmacher, S. Ferreira]
3D Visualization:- Vegetation, Fire- Autonomous mobile Robots- Stereo 3D - Tools: OSG, ODE, Demeter
28 Sept. 2007
60
Robombeiros - Fire Fighting VR Simulation
Virtual Simulation Environment:
3D Simulation:- Fire propagation- Physics- Robot Control
http://pessin.googlepages.com/robombeiros
28 Sept. 2007
61
Presentation Topics
Agenda:
1. Introduction: VR - Hierarchy of Models
2. VR and Simulation
Geometry, Physics, Behaviour, Knowledge and Cognition
3. Physics Simulation Tools
Opensteer, ODE, PhysX, Deformable/Dynamic
4. Intelligent Behaviour
Agents: Perception, Action, Behaviour
Autonomous Robots and Agents - Control
Multi-Agents Systems - Knowledge
5. Applications: Autonomous Robots VR Simulation Tools
6. Conclusions and New Trends
28 Sept. 2007
62
New Trends
Claytronics - Nanotechhttp://www.cs.cmu.edu/~claytronics/[Published at IROS2006]
IEEE Intelligent Robot and Systems Conference
28 Sept. 2007
63
Conclusions and New Trends
Virtual Reality Environments:
Geometric + Kinematic + Physical + Behavioural + Cognitive=
Realistic VR Environments
New Trends: VR + PhysicsArtificial IntelligenceAR - Augmented RealityHaptic Interfaces
Autonomy IVREIntelligent Virtual Reality Environments
Simulation VR PBSimVR Physical and Behavioral Simulation
More Real VR++
28 Sept. 2007
64
CONTACT INFORMATION
UNISINOS University UNISINOS University -- BrazilBrazil
Applied Computing Research PostApplied Computing Research Post --grad Programgrad Program -- PIPCAPIPCAAutonomous Vehicles Research GroupAutonomous Vehicles Research Group -- GPVAGPVA
WebWeb: : GoogleGoogle -- veiculosveiculos autonomosautonomos
GPVA GPVA WebWeb PagePage: : http://www.eletrica.unisinos.br/~autonomhttp://www.eletrica.unisinos.br/~autonom
ContactContact -- WebWeb PagePage: : http://inf.unisinos.br/~osorio/http://inf.unisinos.br/~osorio/
This conference This conference -- Web Page: Web Page: http://inf.unisinos.br/~osorio/palestras/cerma07.ht mlhttp://inf.unisinos.br/~osorio/palestras/cerma07.ht ml
Contact: Prof. Dr. Fernando OsórioE-Mail: [email protected]