Neural Architectures for Lifelong Learning on Humanoid Robots

Vadym Gryshchuk

19.11.2018

Outline

• Motivation

• Background

• Approaches

• Results

• Discussion

• Conclusion

2Neural Architectures for Lifelong Learning on Humanoid Robots

• Motivation

• Background

• Approaches

• Results

• Discussion

• Conclusion

3Neural Architectures for Lifelong Learning on Humanoid Robots

What is Lifelong Learning?

• Continual acquisition of knowledge

• Fine-tuning of knowledge

• Learning from experiences

• Retaining of previously learnt experiences

Neural Architectures for Lifelong Learning on Humanoid Robots 4

Figure 1.1: NICO – Neuro-Inspired COmpanion (Source: Kerzel et al. [2]).

Catastrophic Forgetting

• Interference of learnt representations with new information

Neural Architectures for Lifelong Learning on Humanoid Robots 5

Representation 1 Representation 2

Inspiration from Biological Systems

• Neurosynaptic plasticity

• Hippocampus and cerebral cortex

• Transfer learning

• Intrinsic motivation

• Crossmodal learning

• Incremental learning

Neural Architectures for Lifelong Learning on Humanoid Robots 6

• Motivation

• Background

• Approaches

• Results

• Discussion

• Conclusion

7Neural Architectures for Lifelong Learning on Humanoid Robots

Neural Networks

Neural Architectures for Lifelong Learning on Humanoid Robots 8

Figure 2.1: Neural network representation (Source: McDonald [3]).

Convolutional Neural Networks (CNNs)

Neural Architectures for Lifelong Learning on Humanoid Robots 9

Figure 2.2: Convolutional neural network (Source: Cavaioni [1])

Self-Organizing Networks

• Self-Organizing Map (SOM)

• Grow When Required Network (GWR Network)

• Recurrent GWR

Neural Architectures for Lifelong Learning on Humanoid Robots 10

• Motivation

• Background

• Approaches

• Results

• Discussion

• Conclusion

11Neural Architectures for Lifelong Learning on Humanoid Robots

Object Recognition: CNN + Classifier

• Learning from video sequences

• Visual transformations of objects

• Changing environment

Neural Architectures for Lifelong Learning on Humanoid Robots 12

Figure 3.1: iCub (Source: Pasquale et al. [6]).

Object Recognition: CNN + Classifier

Neural Architectures for Lifelong Learning on Humanoid Robots 13

CNN

Classifier

app

le

cup

bal

l

tom

ato

iCub: Object Learning

Neural Architectures for Lifelong Learning on Humanoid Robots 14

Source: https://www.youtube.com/watch?v=ghUFweqm7W8

iCub: Object Learning

Neural Architectures for Lifelong Learning on Humanoid Robots 15

Source: https://www.youtube.com/watch?v=ghUFweqm7W8

iCub: Object Learning

Neural Architectures for Lifelong Learning on Humanoid Robots 16

Source: https://www.youtube.com/watch?v=ghUFweqm7W8

iCub: Object Learning

Neural Architectures for Lifelong Learning on Humanoid Robots 17

Source: https://www.youtube.com/watch?v=ghUFweqm7W8

iCub: Object Learning

Neural Architectures for Lifelong Learning on Humanoid Robots 18

Source: https://www.youtube.com/watch?v=ghUFweqm7W8

iCub: Object Learning

Neural Architectures for Lifelong Learning on Humanoid Robots 19

Source: https://www.youtube.com/watch?v=ghUFweqm7W8

Sensorimotor Learning: Self-Organization

• Latency in sensorimotor systems

• Predictive mechanisms for future motor states

• Online learning

Neural Architectures for Lifelong Learning on Humanoid Robots 20

Source: https://upload.wikimedia.org/wikipedia/commons/4/47/Nao_Robot_%28Robocup_2016%29.jpg

Sensorimotor Learning: Self-Organization

Neural Architectures for Lifelong Learning on Humanoid Robots 21

Figure 3.2: The imitation scenario (Source: Mici et al. [4]).

Sensorimotor Learning: Self-Organization

Neural Architectures for Lifelong Learning on Humanoid Robots 22

Figure 3.3: Visuomotor learning (Source: Mici et al. [4]).

Object Recognition: CNN + Self-Organization

Neural Architectures for Lifelong Learning on Humanoid Robots 23

RGB sequence

Pre-trained CNN

Pre-trained CNN

Depth sequence

Feat

ure

s

Self-organizing network

Label

Figure 3.4: Recognition pipeline (Adapted from Part et al. [5]).

• Motivation

• Background

• Approaches

• Results

• Discussion

• Conclusion

24Neural Architectures for Lifelong Learning on Humanoid Robots

Object Recognition: CNN + Classifier

Neural Architectures for Lifelong Learning on Humanoid Robots 25

Figure 4.1: Classification accuracy of the model, which was trained on an incremental number of objects (Source: Pasquale et al. [6]).

Object Recognition: CNN + Classifier

Neural Architectures for Lifelong Learning on Humanoid Robots 26

Figure 4.2: Classification accuracy of the model trained incrementally on different days (Source: Pasquale et al. [6]).

Sensorimotor Learning: Self-Organizing Architecture

Neural Architectures for Lifelong Learning on Humanoid Robots 27

Figure 4.3: Behaviour of the architecture (Source: Mici et al. [4]).

Object Recognition: CNN + Self-Organization

Neural Architectures for Lifelong Learning on Humanoid Robots 28

Figure 4.4: Recognition pipeline (Source: Part et al. [5]).

Discussion

• CNN + Classifier architecture for object recognition:• Features extracted from a CNN are dependent on a dataset the model was

trained on

• Old representations are overwritten by the new information

• Self-organizing architecture for sensorimotor learning:• Incremental online learning and prediction

• Unreliability of visual body tracking framework in complex body positions

• CNN + self-organization for object recognition:• Self-organizing network grows when required

• Temporal context is not considered

Neural Architectures for Lifelong Learning on Humanoid Robots 29

Conclusion

• Lifelong learning is crucial for intelligent robots

• Biological systems provide a basis for the incremental learning

• Self-organizing networks preserve the topology

• CNNs learn efficient feature descriptors

• Catastrophic forgetting increases during incremental tasks

Neural Architectures for Lifelong Learning on Humanoid Robots 30

Thank You!Questions?

Neural Architectures for Lifelong Learning on Humanoid Robots 31

References• [1] Cavaioni, M. Deep Learning series: Convolutional Neural Networks. https://medium.com/machine- learning-

bites/deeplearning-series-convolutional-neural-networks-a9c2f2ee1524 . [Online; accessed 13-November-2018].

• [2] Kerzel, M., Strahl, E., Magg, S., Navarro-Guerrero, N., Heinrich, S., Wermter, S.NICO - neuro-inspired companion: A developmental humanoid robot platform for multimodal interaction. In26th IEEE International Symposium on Robot and Human Interactive Communication, ROMAN 2017, Lisbon, Portugal, August 28 - Sept. 1, 2017, pages 113–120, 2017.

• [3] McDonald, C. Machine learning fundamentals (II): Neural networks. https://towardsdatascience.com/machine-learning-fundamentals-ii-neural-networks-f1e7b2cb3eef. [Online; accessed 13-November-2018].

• [4] Mici, L., Parisi, I. G., Wermter, S. An Incremental Self-Organizing Architecture for Sensorimotor Learning and Prediction. CoRR, abs/1712.08521, 2017.

• [5] Part l. J., Lemon, O. Incremental online learning of objects for robots operating in real environments. In Joint IEEE International Conference on Development and Learning and Epigenetic Robotics, ICDL-EpiRob 2017, Lisbon, Portugal, September 18-21, 2017, pages 304–310, 2017.

• [6] Pasquale, G., Ciliberto, C., Odone, F., Rosasco, L., Natale, L. Real-world object recognition with off-the-shelf deep conv nets: How many objects can iCub learn? CoRR, abs/1504.03154, 2015.

Neural Architectures for Lifelong Learning on Humanoid Robots 32