System Architectures
Basics
Duckietown
System decomposition
• All starts with a decomposition of a system in components.
• What is a system and what are components?
2
“A system is composed of components;a component is something you understand.”
— Howard Aiken (1900-1973)
Duckietown
Physical and logical architectures
• There are two aspects for “architectures”:
• Logical architecture - what the system is doing
• e.g., system decomposition, data flow
• Physical architecture - how it is doing it
• e.g., which computer runs which component
• Containerization: Which computer runs which virtual computer that runs which component?
3
Duckietown
Logical architecture
• The logical architecture describes:
• System decomposition in components
• Data flow (Who tells whom what)
• Representations
• Priors (Who knows what)
• The logical architecture is independent of language, middleware, and other implementation details.
4
Something Useful
Duckietown
control
5
given
to design
inputsoutputs
process and environment
agent / controller
world / plant
belief / estimate
perceptionplan
planning
A familiar example (autonomy) logical architecture
commands
actuators
observations
sensor(s)
Duckietown 6
Modeling
𝑐
𝑥𝐴
𝑦𝐼
𝑥𝐼
𝑦𝑊
𝑥𝑊
𝐴
𝐶
𝑣𝑙
𝑣𝑟
𝑂
I𝑑
𝑦𝐴
2𝑅
𝜔
2𝐿
𝜃
𝜑𝑙
𝜑𝑟
𝑦𝑅𝑥𝑅
Duckietown 7
Sensing
Z
YX
CameraCenterC
PPrincipalPoint
ImageP
lane
PrincipalAxis
u Aa
v
v
y:;<
x:;<
Duckietown 8
Planning
Duckietown 9
Control
Duckietown
Part of logical architecture for Duckietown• Part of the main perception pipeline.
• Purple refers to “static” information.
• The diagram does not show how things are implemented.
• It does show who-knows-what, and who-tells-what-to-whom.
11
sensor
learned transform
image anti-instagram
anti-instagram learner
transformed image
segment detector
ground projection
image-spacesegments
appearance prior
calibration
calibration procedure
3D segments
Duckietown
Computation graph
• Computation graph
• nodes = components
• edges = signals
• node weight = flops required
• edge weight = size in bytes
12
Duckietown
Physical Architecture
• “Physical” architecture: how things are implemented.
• The physical architecture includes:
• Which processor runs what process?
• How is the data communicated (TCP, UDP, etc.)
• Where is the data stored
• Protocols, formats, etc.
13
Duckietown
Physical architecture: Resource graph
• Resource graph:
• nodes = processors
• edges = network links
• node weight = processor power
• edge weight = bandwidth
14
Duckietown
Part of physical architecture of Duckietown• This is the Duckietown ROS graph
• It lists the nodes and the signals.
15
legend Sensors
Camera
Actuators
Visualization for operator
Line Detection and Projection
Parallel AutonomyFinate State Machine
Intersection Navigation
Coordination
F1: Illumination Robustness
F2-3: LED Detection
F4: Odometry Calibration from Sensor Measurements
F5: Lane Filter and Control
F6: Vehicle Detection
F8: Apriltags
F9: MDOAP
missing
designed
development
compiled
documented
tested
pi_cameracamera_node (SY,AC)
~image/compressedsensor_msgs/CompressedImage.msg
joy_node
joysensor_msgs/Joy.msg
joy_mapperjoy_mapper_node (SY)
led_joy_mapperled_joy_mapper_node
pi_cameradecoder_node (SY)
pi_cameracam_info_reader_node (SY)
line_detectorline_detector_node (LP,HZ)
anti_instagramanti_instagram_node (F1)
led_detectionLED_detector_node
camera_node/image/rawsensor_msgs/Image.msg
camera_node/camera_infosensor_msgs/CameraInfo.msg
ground_projectionground_projection_node (CC)
veh_detectorvehicle_filter_node (F6)
apriltags apriltags_global_node (F8)
veh_detectorvehicle_detector_node (F6)
apriltags april_pre_pros_node (F8)
mdoapstatic_object_detector_node (F9)
calibration/camera_intrinsic
led_interpreterLED_interpreter_node
led_emitterled_emitter_node
dagu_carwheels_driver_node (DY,SY)
wheels_driver_node/wheels_cmdduckietown_msgs/WheelsCmdStamped.msg
kinematicsforward_kinematics_node (F4)
~change_color_patternString
~image_with_linessensor_msgs/Image.msg
~object_detection_markersvisualization_msgs/MarkerArray.msg
~segment_listduckietown_msgs/SegmentList.msg
~segment_list_projduckietown_msgs/SegmentList.msg
visual_odometryodometry_training_pairs_node (F4)
lane_filterlane_filter_node (LP)
stop_line_filterstop_line_filter_node (LP)
calibration/camera_extrinsic
~joy_controlduckietown_msgs/Twist2DStamped.msg
lane_controllane_supervisor_node (LP)
~lane_control_supervisedduckietown_msgs/Twist2DStamped.msg
dagu_carcar_cmd_switch (SY)
fsmfsm_node (MN)
~modeduckietown_msgs/FSMState.msg
~car_command_switchduckietown_msgs/Twist2DStamped.msg
intersection_controlopen_loop_intersection_control_node (MN)
veh_coordinator_node (DH)
~intersection_controlduckietown_msgs/Twist2DStamped.msg
~intersection_doneduckietown_msgs/BoolStamped.msg
navigationrandom_april_tag_turns_node (MN)
~turn_typestd_msgs/Int16.msg
navigationgraph_planner_node (MN)
navigationaction_dispatcher_node (MN)
navigation/GraphSearch.srv
~turn_typestd_msgs/Int16.msg
~intersection_goduckietown_msgs/BoolStamped.msg
~corrected_image/rawsensor_msgs/Image.msg
~image_healthduckietown_msgs/AntiInstagramHealth.msg
~raw_led_detectionled_detection/LEDDetectionArray.msg
~signals_detectionled_detection/SignalsDetection.msg
~desired_signaling_statestd_msgs/String.msg odometry_training_pairs
kinematicskinematics_learning_node (F4)
calibration/inverse_kinematicscalibration/forward_kinematics
kinematicsinverse_kinematics_node (F4)
~poseduckietown_msgs/Pose2DStamped.msg
~velocityduckietown_msgs/Twist2DStamped.msg
~lane_poseduckietown_msgs/LanePose.msg
~in_laneduckietown_msgs/BoolStamped.msg
~at_stop_lineduckietown_msgs/BoolStamped.msg
~stop_line_diststd_msgs/Float32.msg
lane_controllane_controller_node (SY,SC)
~lane_controlduckietown_msgs/Twist2DStamped.msg
~cornersduckietown_msgs/ObstacleCorners.msg
~vehicle_poseduckietown_msgs/VehiclePose.msg
veh_detectorvehicle_avoidance_control_node (F6)
~controlduckietown_msgs/Twist2DStamped.msg
~flagduckietown_msgs/BoolStamped.msg
~fast/image_raw sensor_msgs/Image.msg
~global/image_raw sensor_msgs/Image.msg
apriltags apriltags_fast_node (F8)
~tags_image sensor_msgs/Image.msg
~apriltags duckietown_msgs/apriltags.msg
~tags_image sensor_msgs/Image.msg
~apriltags duckietown_msgs/apriltags.msg
apriltags apriltags_postprocessing_fast_node (F8)
~apriltags duckietown_msgs/apriltags.msg
apriltags apriltags_postprocessing_global_node (F8)
~apriltags duckietown_msgs/apriltags.msg
apriltagsDB.yaml
~detection_listduckietown_msgs/ObjectImageDetectionList.msg
mdoapobstacle_safety_node (F9)
~object_too_closeduckietown_msgs/BoolStamped.msg
mdoapsimple_stop_controller_node (F9)
~detection_list_projduckietown_msgs/ObjectProjectedDetectionList.msg
~controlduckietown_msgs/Twist2DStamped.msg
legend Sensors
Camera
Actuators
Visualization for operator
Line Detection and Projection
Parallel AutonomyFinate State Machine
Intersection Navigation
Coordination
F1: Illumination Robustness
F2-3: LED Detection
F4: Odometry Calibration from Sensor Measurements
F5: Lane Filter and Control
F6: Vehicle Detection
F8: Apriltags
F9: MDOAP
missing
designed
development
compiled
documented
tested
pi_cameracamera_node (SY,AC)
~image/compressedsensor_msgs/CompressedImage.msg
joy_node
joysensor_msgs/Joy.msg
joy_mapperjoy_mapper_node (SY)
led_joy_mapperled_joy_mapper_node
pi_cameradecoder_node (SY)
pi_cameracam_info_reader_node (SY)
line_detectorline_detector_node (LP,HZ)
anti_instagramanti_instagram_node (F1)
led_detectionLED_detector_node
camera_node/image/rawsensor_msgs/Image.msg
camera_node/camera_infosensor_msgs/CameraInfo.msg
ground_projectionground_projection_node (CC)
veh_detectorvehicle_filter_node (F6)
apriltags apriltags_global_node (F8)
veh_detectorvehicle_detector_node (F6)
apriltags april_pre_pros_node (F8)
mdoapstatic_object_detector_node (F9)
calibration/camera_intrinsic
led_interpreterLED_interpreter_node
led_emitterled_emitter_node
dagu_carwheels_driver_node (DY,SY)
wheels_driver_node/wheels_cmdduckietown_msgs/WheelsCmdStamped.msg
kinematicsforward_kinematics_node (F4)
~change_color_patternString
~image_with_linessensor_msgs/Image.msg
~object_detection_markersvisualization_msgs/MarkerArray.msg
~segment_listduckietown_msgs/SegmentList.msg
~segment_list_projduckietown_msgs/SegmentList.msg
visual_odometryodometry_training_pairs_node (F4)
lane_filterlane_filter_node (LP)
stop_line_filterstop_line_filter_node (LP)
calibration/camera_extrinsic
~joy_controlduckietown_msgs/Twist2DStamped.msg
lane_controllane_supervisor_node (LP)
~lane_control_supervisedduckietown_msgs/Twist2DStamped.msg
dagu_carcar_cmd_switch (SY)
fsmfsm_node (MN)
~modeduckietown_msgs/FSMState.msg
~car_command_switchduckietown_msgs/Twist2DStamped.msg
intersection_controlopen_loop_intersection_control_node (MN)
veh_coordinator_node (DH)
~intersection_controlduckietown_msgs/Twist2DStamped.msg
~intersection_doneduckietown_msgs/BoolStamped.msg
navigationrandom_april_tag_turns_node (MN)
~turn_typestd_msgs/Int16.msg
navigationgraph_planner_node (MN)
navigationaction_dispatcher_node (MN)
navigation/GraphSearch.srv
~turn_typestd_msgs/Int16.msg
~intersection_goduckietown_msgs/BoolStamped.msg
~corrected_image/rawsensor_msgs/Image.msg
~image_healthduckietown_msgs/AntiInstagramHealth.msg
~raw_led_detectionled_detection/LEDDetectionArray.msg
~signals_detectionled_detection/SignalsDetection.msg
~desired_signaling_statestd_msgs/String.msg odometry_training_pairs
kinematicskinematics_learning_node (F4)
calibration/inverse_kinematicscalibration/forward_kinematics
kinematicsinverse_kinematics_node (F4)
~poseduckietown_msgs/Pose2DStamped.msg
~velocityduckietown_msgs/Twist2DStamped.msg
~lane_poseduckietown_msgs/LanePose.msg
~in_laneduckietown_msgs/BoolStamped.msg
~at_stop_lineduckietown_msgs/BoolStamped.msg
~stop_line_diststd_msgs/Float32.msg
lane_controllane_controller_node (SY,SC)
~lane_controlduckietown_msgs/Twist2DStamped.msg
~cornersduckietown_msgs/ObstacleCorners.msg
~vehicle_poseduckietown_msgs/VehiclePose.msg
veh_detectorvehicle_avoidance_control_node (F6)
~controlduckietown_msgs/Twist2DStamped.msg
~flagduckietown_msgs/BoolStamped.msg
~fast/image_raw sensor_msgs/Image.msg
~global/image_raw sensor_msgs/Image.msg
apriltags apriltags_fast_node (F8)
~tags_image sensor_msgs/Image.msg
~apriltags duckietown_msgs/apriltags.msg
~tags_image sensor_msgs/Image.msg
~apriltags duckietown_msgs/apriltags.msg
apriltags apriltags_postprocessing_fast_node (F8)
~apriltags duckietown_msgs/apriltags.msg
apriltags apriltags_postprocessing_global_node (F8)
~apriltags duckietown_msgs/apriltags.msg
apriltagsDB.yaml
~detection_listduckietown_msgs/ObjectImageDetectionList.msg
mdoapobstacle_safety_node (F9)
~object_too_closeduckietown_msgs/BoolStamped.msg
mdoapsimple_stop_controller_node (F9)
~detection_list_projduckietown_msgs/ObjectProjectedDetectionList.msg
~controlduckietown_msgs/Twist2DStamped.msg
legend Sensors
Camera
Actuators
Visualization for operator
Line Detection and Projection
Parallel AutonomyFinate State Machine
Intersection Navigation
Coordination
F1: Illumination Robustness
F2-3: LED Detection
F4: Odometry Calibration from Sensor Measurements
F5: Lane Filter and Control
F6: Vehicle Detection
F8: Apriltags
F9: MDOAP
missing
designed
development
compiled
documented
tested
pi_cameracamera_node (SY,AC)
~image/compressedsensor_msgs/CompressedImage.msg
joy_node
joysensor_msgs/Joy.msg
joy_mapperjoy_mapper_node (SY)
led_joy_mapperled_joy_mapper_node
pi_cameradecoder_node (SY)
pi_cameracam_info_reader_node (SY)
line_detectorline_detector_node (LP,HZ)
anti_instagramanti_instagram_node (F1)
led_detectionLED_detector_node
camera_node/image/rawsensor_msgs/Image.msg
camera_node/camera_infosensor_msgs/CameraInfo.msg
ground_projectionground_projection_node (CC)
veh_detectorvehicle_filter_node (F6)
apriltags apriltags_global_node (F8)
veh_detectorvehicle_detector_node (F6)
apriltags april_pre_pros_node (F8)
mdoapstatic_object_detector_node (F9)
calibration/camera_intrinsic
led_interpreterLED_interpreter_node
led_emitterled_emitter_node
dagu_carwheels_driver_node (DY,SY)
wheels_driver_node/wheels_cmdduckietown_msgs/WheelsCmdStamped.msg
kinematicsforward_kinematics_node (F4)
~change_color_patternString
~image_with_linessensor_msgs/Image.msg
~object_detection_markersvisualization_msgs/MarkerArray.msg
~segment_listduckietown_msgs/SegmentList.msg
~segment_list_projduckietown_msgs/SegmentList.msg
visual_odometryodometry_training_pairs_node (F4)
lane_filterlane_filter_node (LP)
stop_line_filterstop_line_filter_node (LP)
calibration/camera_extrinsic
~joy_controlduckietown_msgs/Twist2DStamped.msg
lane_controllane_supervisor_node (LP)
~lane_control_supervisedduckietown_msgs/Twist2DStamped.msg
dagu_carcar_cmd_switch (SY)
fsmfsm_node (MN)
~modeduckietown_msgs/FSMState.msg
~car_command_switchduckietown_msgs/Twist2DStamped.msg
intersection_controlopen_loop_intersection_control_node (MN)
veh_coordinator_node (DH)
~intersection_controlduckietown_msgs/Twist2DStamped.msg
~intersection_doneduckietown_msgs/BoolStamped.msg
navigationrandom_april_tag_turns_node (MN)
~turn_typestd_msgs/Int16.msg
navigationgraph_planner_node (MN)
navigationaction_dispatcher_node (MN)
navigation/GraphSearch.srv
~turn_typestd_msgs/Int16.msg
~intersection_goduckietown_msgs/BoolStamped.msg
~corrected_image/rawsensor_msgs/Image.msg
~image_healthduckietown_msgs/AntiInstagramHealth.msg
~raw_led_detectionled_detection/LEDDetectionArray.msg
~signals_detectionled_detection/SignalsDetection.msg
~desired_signaling_statestd_msgs/String.msg odometry_training_pairs
kinematicskinematics_learning_node (F4)
calibration/inverse_kinematicscalibration/forward_kinematics
kinematicsinverse_kinematics_node (F4)
~poseduckietown_msgs/Pose2DStamped.msg
~velocityduckietown_msgs/Twist2DStamped.msg
~lane_poseduckietown_msgs/LanePose.msg
~in_laneduckietown_msgs/BoolStamped.msg
~at_stop_lineduckietown_msgs/BoolStamped.msg
~stop_line_diststd_msgs/Float32.msg
lane_controllane_controller_node (SY,SC)
~lane_controlduckietown_msgs/Twist2DStamped.msg
~cornersduckietown_msgs/ObstacleCorners.msg
~vehicle_poseduckietown_msgs/VehiclePose.msg
veh_detectorvehicle_avoidance_control_node (F6)
~controlduckietown_msgs/Twist2DStamped.msg
~flagduckietown_msgs/BoolStamped.msg
~fast/image_raw sensor_msgs/Image.msg
~global/image_raw sensor_msgs/Image.msg
apriltags apriltags_fast_node (F8)
~tags_image sensor_msgs/Image.msg
~apriltags duckietown_msgs/apriltags.msg
~tags_image sensor_msgs/Image.msg
~apriltags duckietown_msgs/apriltags.msg
apriltags apriltags_postprocessing_fast_node (F8)
~apriltags duckietown_msgs/apriltags.msg
apriltags apriltags_postprocessing_global_node (F8)
~apriltags duckietown_msgs/apriltags.msg
apriltagsDB.yaml
~detection_listduckietown_msgs/ObjectImageDetectionList.msg
mdoapobstacle_safety_node (F9)
~object_too_closeduckietown_msgs/BoolStamped.msg
mdoapsimple_stop_controller_node (F9)
~detection_list_projduckietown_msgs/ObjectProjectedDetectionList.msg
~controlduckietown_msgs/Twist2DStamped.msg
Duckietown
Deployment: Mapping logical architecture onto physical
• We need to map the computation graph onto the resource graph.
• Different choices will have different properties for latency, frequency, etc.
16
computation graph resource graphoptimization
deployment
Duckietown
• We need to map the computation graph onto the resource graph.
• Different choices will have different properties for latency, frequency, etc.
17
computation graph resource graph
deployment
optimization
Deployment: Mapping logical architecture onto physical
Duckietown
Example in Duckietown: Computation graph
• Computation graph: we collapse all computation in one node
18
camera driver
motors driver
(abstracting all computation as one node)
Duckietown
Example in Duckietown: Resource graph
19
camera actuator
4 core ARM
GPU
Raspberry PIYour laptop
GPU
4 core x86
wireless link
latency: 2 ms - 20 msbandwidth: 1-100 MB/s
Duckietown
Example in Duckietown: Deployment 1
• Option 1: Run everything on the Raspberry PI
20
camera actuator
Raspberry PIYour laptop
camera driver
motors driver
orchestrator
Duckietown
Example in Duckietown: Deployment 2
• Option 2: Run from your laptop
21
camera actuator
Raspberry PIYour laptop
camera driver
motors driver
orchestrator
Duckietown
Example in Duckietown: Deployment 3
• Option 3: Run heavy processing on the laptop
22
camera actuator
Raspberry PIYour laptop
camera driver
motors driver
orchestrator
Duckietown
Example in Duckietown: Deployment 3
23Duckietown
How is all that implemented in Duckietown?
• Dozens of nodes in Duckietown, hundreds of connections
• Multiple different deployment scenarios:
• On a Duckiebot
• In simulation
• As an AI Driving Olympics (AIDO) submission
• We keep it clean and manageable by splitting the functionality in multiple containers
Duckietown
Example in Duckietown: Deployment 3
24Duckietown
The Docker images family tree
Duckietown 25Duckietown
Example deployments
Driving with a joystick Driving through the Dashboard
Running a demo
Evaluating AIDO submission in
simulation
Running a demo in simulation
Evaluating AIDO submission on a
Duckiebot
An interesting read
• E. A. Lee and S. A. Seshia, Embedded Systems -- A Cyber-Physical Systems Approach
Top Related