Boston Elixir

Fun with Robots and Elixir

Presented by Jean-François Cloutier

At the Boston Elixir Meetup

January 21, 2016

Boston Elixir

Why program robots with Elixir?

Because robots are lots of fun and Elixir is the shiznit

And a robot's mind should be a “society of agents” (or, at least, a cacophony of agents)

Which is best implemented with processes, lots of processes, running concurrently

And Elixir (with OTP and the Erlang runtime system) totally shines here

Boston Elixir

Lego Mindstorms

The Greatest Toy Ever!

And it keeps getting better

1998 – RCX 2006 – NXT 2013 - EV3

EV3 brick● Faster CPU, more memory● Linux!● Bootable microSD card● USB port

● Daisy chain up to 4 bricks● Plug WiFi dongle

Boston Elixir

The goodies

The programmable brick

• 4 sensor input ports

• 4 motor output ports

• LEDs

• Sound

Sensors and actuators

• 2 large motors

• 1 medium motor

• 1 color sensor

• 1 IR sensor (with beacon/remote control)

• 1 touch sensor

• 1 ultrasonic sensor (extra)

• 1 gyro sensor (extra)

Boston Elixir

The goodies

And a whole bunch of

• Connectors

• Wheels, cogs, threads

• Etc.

Boston Elixir

Programming the EV3

The EV3 set comes with a customized version of LabView

Visual programming? Snazzy!

• If you don't mind coding everything as one big “sense and respond” loop

Boston Elixir

Blerch!

OK it's not that bad but we can do better!

Boston Elixir

ev3dev.org to the rescue

An updated LEGO Linux kernel (Debian)

• Bootable from a microSD card

• Can run most Debian packages● apt-get install …

EV3 software “rebuilt from the ground up”

• Motor, sensor and LED drivers use sysfs● Exposed as character files

● Sensing = reading files

● Controlling = writing to files

EV3 is now exposed to any and all programming languages running under Linux

Ergo, we can code the EV3 with Elixir!

Boston Elixir

Installing Elixir on the EV3

You will need to

• Connect the EV3 to the Internet (via WiFi dongle)

• Build the latest release of Erlang from sources

• Install Elixir from a precompiled zip

• Setup a user account with the right privileges

See details in my blog

• http://jfcloutier.github.io/robotex/

Boston Elixir

Meet the robot

Infrared sensor (to sense beacon

distance and direction)

Touch sensor(to detect collisions)

Large motor(to move and turn)

Medium motor (turned on when

“eating”)

LEDs(green when moving around, orange when finding food, red

when panicking)

Color sensor (to see the “food”)

Beacon(to simulate the smell of food)

Designed and built by my 12 year old son William

Ultrasonic sensor (to sense

proximity of obstacles)

Speaker(to emote audibly)

Powered by elixir

Boston Elixir

Boston Elixir

Perceptors

Perceptors generate high(er)-level percepts

Each Perceptor reacts to a new percept of interest

• and analyzes more or less recent memorized percepts

• to, possibly, create a new percept

• which it then notifies the CNS of

A “collision” Perceptor, for example, would analyze proximity and touch percepts

• Decreasing proximity = collision soon

• Close proximity in immediate past = collision imminent

• Recent collision imminent and touched = collision now

A “fear” Perceptor would analyze collisions and ambient light percepts

• It would create fear percepts when the robot collides in very low ambient light

Boston Elixir

Motivators

Motivators generate motives in response to new percepts

Motives are like emotions

• They trigger behaviors

A motive can inhibit other or all motives

• Hunger inhibits curiosity

• Fear inhibits all other motives

Motives are kept in memory for a while

Boston Elixir

Behaviors

Behaviors are finite-state machines triggered by motives

State transitions

• are caused by new percepts

• And typically generate intents

A behavior is frozen if all its triggering motives are inhibited

Curiosity

Roaming

CollisionNOW

CollisionIMMINENT

Avoid collision

Time elapsed

Started

Time elapsed

Behave this!Roam

Stuck

Back off

Boston Elixir

Actuators

Actuators translate intents into commands

Each actuator realizes one type of intent

• by executing a script that sends a sequence of commands to motors and LEDs

Different robots may require completely different scripts to realize the same intent

Boston Elixir

Memory

The Memory stores percepts, motives and intents

• For later recall by by Perceptors, Motivators and Behaviors

Expired, invalidated percepts are continuously removed from memory

• Percepts from different senses are retained for different amounts of time

Boston Elixir

The Central Nervous System

The CNS is an event dispatcher

Dispatched events include

• A percept, motive or intent was created

• An intent was realized

• An actuator is overwhelmed

The CNS connects the robot's mind via events

• Detectors, Memory, Perceptors, Motivators, Behaviors, Actuators... do not communicate directly

• They each listen to all events but only handle those of interest to them

Boston Elixir

CNS mediates all interactions

From percepts to actions to percepts

Detectors produce percepts when sensors detect changes

Perceptors generate higher-level percepts

Percepts cause motives via motivators

Motives trigger behaviors that are driven by percepts and generate intents

Intents are translated into commands by actuators

Commands make the robot move, generating new perceptions

• The passing of time is itself a periodic perception

Everything happens concurrently

• No single, big sense-respond loop

All interactions are via events dispatched by the CNS

Detector, Internal clock

Detector, Internal clock

PerceptorPerceptor

MotivatorMotivator

BehaviorBehavior

ActuatorActuatorActions cause measures to change

Actions cause measures to change

Boston Elixir

The urgency of now

A robot must deal with the “here and now”

• It can't afford to react to its situation as it was 10 seconds ago

When the robot is falling behind, it sees increasingly stale percepts, motives or intents

• At some point, they become dangerously out-of-sync with reality

One solution

• Ignore stale percepts, motives and intents● Strong intents take longer to become stale

• And make the robot faint (briefly)● To temporarily stop the production of Percepts by Detectors

and by the Internal Clock

● Giving the robot a chance to catch up

Boston Elixir

InspirationMarvin Minsky's society of mind

A core tenet of Minsky's philosophy is that "minds are what brains do".

The society of mind theory views the human mind and any other naturally evolved cognitive systems as a vast society of individually simple processes known as agents. These processes are the fundamental thinking entities from which minds are built, and together produce the many abilities we attribute to minds. The great power in viewing a mind as a society of agents, as opposed to the consequence of some basic principle or some simple formal system, is that different agents can be based on different types of processes with different purposes, ways of representing knowledge, and methods for producing results.

This idea is perhaps best summarized by the following quote:

What magical trick makes us intelligent? The trick is that there is no trick. The power of intelligence stems from our vast diversity, not from any single, perfect principle. —Marvin Minsky, The Society of Mind, p. 308

https://en.wikipedia.org/wiki/Society_of_Mind

Boston Elixir

InspirationRodney Brooks' subsumption architecture

Subsumption architecture is a reactive robotic architecture heavily associated with behavior-based robotics. The term was introduced by Rodney Brooks and colleagues in 1986.

Subsumption architecture is a control architecture that [decomposes] the complete behavior into sub-behaviors.

These sub-behaviors are organized into a hierarchy of layers. Each layer implements a particular level of behavioral competence, and higher levels are able to subsume lower levels in order to create viable behavior. For example, a robot's lowest layer could be "avoid an object". The second layer would be "wander around", which runs beneath the third layer "explore the world".

The layers, which all receive sensor-information, work in parallel and generate outputs. These outputs can be commands to actuators, or signals that suppress or inhibit other layers.

Each layer is made up by a set of processors that are augmented finite-state machines (AFSM), the augmentation being added instance variables to hold programmable data-structures. A layer is a module and is responsible for a single behavioral goal, such as "wander around." There is no central control within or between these behavioral modules. All AFSMs continuously and asynchronously receive input from the relevant sensors and send output to actuators (or other AFSMs). Input signals that are not read by the time a new one is delivered end up getting discarded. These discarded signals are common, and is useful for performance because it allows the system to work in real time by dealing with the most immediate information.

https://en.wikipedia.org/wiki/Subsumption_architecture

Boston Elixir

The robot's mind is constructed by defining its

• Perceptors

• Motivators

• Behaviors

• Actuators

Boston Elixir

Release the robot!

The robot starts hungry, so it forages first by...

• Trying to orient itself and move toward the beacon

• Looking for “blue” food, hopefully in well-lit areas (more plentiful)

• While avoiding obstacles or getting stuck

When not hungry, it expresses its innate curiosity by roaming

• Randomly going here and there

• And avoiding obstacles or getting stuck

If it collides into something, it panics

• And behaves like a headless chicken

• Until it calms down and resumes foraging or roaming

Boston Elixir

Test run

Beacon (as the smell of food)

“Food”

The robot starts up hungry and searches for food. It first heads in the wrong direction, gets stuck (“I'm stuck!”) but manages to free itself.

Then it smells the bacon beacon and makes a bee line toward the “food”. It slows down as it gets closer, adjusting its aim until it finds the food (blue paper). It then proceeds to eat it (“nom de nom de nom!”), turning on orange lights and running its small motor (the “mouth”).

When it becomes sated, curiosity takes over and the robot roams around, trying to avoid collisions. It sometimes panics (“I'm scared!”) when it runs into walls. The lights then turn red and the poor robot behaves like a headless chicken for a while until it calms down.

At some point, it gets hungry again and, this time, easily finds the food.

https://www.youtube.com/watch?v=NbquttnKuak

The beast

Boston Elixir

Coding the mind of a robot

I use Elixir file functions to write to and read from /sys/class/...

And take advantage of Elixir's functional programming

• Devices, percepts etc. as immutable data

• Detection, perception, behavior etc. as functional transformations

I made OTP my BFF

• The CNS and its event handlers are implemented with GenServer and GenEvent

• I use Agents for Detectors, Perceptors, Motivators, Behaviors and Actuators

• Memory is a singleton GenServer (creates a transaction boundary)

• Linked processes● Periodically remove stale memories

● Poll sensors

• Everything is supervised for fail-over

Boston Elixir

Accessing the EV3 via sysfs

Boston Elixir

Interacting with tacho-motors

ssh jf@192.168.1.125...cd /sys/class/tacho­motorcd motor0cat driver_nameecho 50 > duty_cycle_specho run-forever > commandecho stop > command

Large motor Medium motor

Boston Elixir

Interacting with sensors

Touch sensor

cd /sys/class/lego­sensorcd sensor1cat driver_namecat value0cat modesecho IR­SEEK > modecat value0cat value1

Infrared sensor and remote control Color sensor

Boston Elixir

Modeling EV3 devices

We can model the changing state of motors, sensors and LEDs

• As mutable global variables (the non-FP way)

● as embodied by /sys/class files

• Or as functions consuming and producing immutable data (the FP way)

● Done by wrapping /sys/class with pure functions that input and output device states

Device

Get connected

devices

Device

Device

Device

Alter controls

DeviceExecute

command

Get state

Device

Boston Elixir

Accessing EV3 devices the FP way

Devices are immutable data

Access to sensors and other devices

• Via pure functions

• Inputing and outputting devices as structs

Boston Elixir

A functional brain

The robot's state is modeled as a bunch of (immutable) data structures

• Percept, Motive, Behavior FSM, Intent

• Configurations for perception, motivation, behaviors and actuation

Memory queries are encoded as higher order functions and transformation pipelines

Functions-as-data encode

• Perception capabilities

• Motivation rules

• Elements of behavior

• Actuation scripts

Boston Elixir

OTP on the brain

In my implementation, I aimed for

• Maximum concurrency via processes ● A “society of mind” is inherently parallel

● Sequential processing happens only where absolutely necessary

• Maximum decoupling via events● Components don't know of each other

OTP is a spectacular fit

• Supervisor, GenServer, GenEvent, Agent etc. = high-level concurrency framework

• OTP makes it easy to implement a process-based design with predictable behavior● You don't need to be a genius to correctly interleave concurrent and sequential code

• The use of OTP favors economical and largely declarative code ● It says what it does and it does what it says

Boston Elixir

Processes in Elixir, a refresher

Processes are cheap (2.5K per process)

Preemptive scheduling (no CPU hogs) over multiple cores

Communication with processes:

• Via self-contained (deep-copied) messages

• Messages are sent to a process' mailbox (a queue)

• The receiving process handles one message a time

When a process creates another, it can...

• Link to it (it dies if the linked process dies abnormally)

• Monitor it (might receive a “wrongful death” message)

OTP is (among other things) a framework for easily and safely creating, interacting with and supervising processes

• Hides complexities and handles the corner cases

• Standardizes multi-process coding

ProcessProcess

ProcessProcess

linked

{:exit, pid, reason}

Boston Elixir

The Ev3 application'sprocess tree*

* Here running with mock motors and sensors

Boston Elixir

I OTP

The 'simple-one-for-one' supervisor strategy

• Supervised “worker” child processes are created on request and as needed

• Applied to (re)start Detectors, Perceptors, Motivators, Behaviors, Actuators

A singleton GenServer creates a transactional boundary

• N processes sending messages to one Genserver (transaction boundary!)

• Applied for Memory, CNS

GenServer

Process

Process

Process

Messages handled one at a time

Calls

Boston Elixir

I OTP

I use Agents (simplified GenServers) to implement

• Detectors, Perceptors, Motivators, Behaviors and Actuators

Agents are supervisable processes that

• Hold a non-shared state (while the process is alive)

Behaviors are finite state machines

• A behavior has states and state transitions

• I did not use OTP's gen_fsm but rolled my own FSM to fit my special needs

Boston Elixir

I OTP

The CNS is an event manager implemented as a GenServer that monitors its GenEvent handlers

• It notices and recovers from event handler failures

• See my Gist

I spawn processes to parallelize event dispatching

• Used by event handlers for Perceptors, and Actuators

Boston Elixir

Configuring a robot's mind

The particulars of a given robot's perception and execution are defined in these modules:

• Ev3.Perception● The ways by which the robot creates Percepts and interprets them into high-level

Percepts

• Ev3.Motivation● The rules by which the robot responds to Percepts by turning Motives on and off

• Ev3.Behaviors● The robot's finite state machines triggered by Motives and driven by Percepts to

generate Intents

• Ev3.Actuation● The rules by which the robot realizes Intents by sending scripted commands to its

motors and LEDs

Boston Elixir

Debugging a robot is painful

Debugging the real-world behavior of a physical being is not like debugging code

• The effects of commands are unpredictable (the robot hits a wall, a wheel slips)

• Sensors can lie (they sometimes see ghosts)

• Overall behavior is very sensitive to tuning (how much to turn, how far to move)

• The Heisenberg Principle is in full force (producing a debug trace drastically alters time-sensitive behavior causing new problems)

And debugging code is harder

• Some bugs only come out when dealing with real motors and sensors (vs mocks)

• The run-edit-deploy cycle on the EV3 is very long (it's a slooow 300MHz ARM9 single core computer)

Boston Elixir

Time to wrap it up

Functional programming FTW!

• I implemented a non-trivial robotics framework using FP

• I never once missed having classes, inheritance etc. (sorry OO, my old friend)

• Data structures + functional transformations make the code remarkably declarative and compact

OTP, how do I love thee?

• OTP is a treasure trove of battle-tested power tools for writing fault-tolerant, concurrent, event-driven code

• Elixir polishes it to a shine (minimal boilerplate, simple abstractions like Agent and Task)

Bottom line

• Coding the robot in Elixir/OTP is so much fun it at times verges on pure giddiness

Boston Elixir

Watch this space

http://jfcloutier.github.io/robotex/

Boston Elixir

Thank you all!

I organize the Portland (Maine) Erlang & Elixir meetup

• jean.f.cloutier@gmail.com

• Twitter: jfcloutier

• Github: https://github.com/jfcloutier/ev3