A String of Ponies - A String of Ponies Transparent Distributed Programming with Actors Sebastian...

download A String of Ponies - A String of Ponies Transparent Distributed Programming with Actors Sebastian Blessing

of 101

  • date post

    14-Jun-2020
  • Category

    Documents

  • view

    1
  • download

    0

Embed Size (px)

Transcript of A String of Ponies - A String of Ponies Transparent Distributed Programming with Actors Sebastian...

  • Imperial College of Science, Technology and Medicine Department of Computing

    A String of Ponies

    Transparent Distributed Programming with Actors

    Sebastian Blessing

    Supervised by Prof. Sophia Drossopoulou and Sylvan Clebsch

    Submitted in partial fulfilment of the requirements for the MSc Degree in Computing Science (Software Engineering) of Imperial College London,

    September 2013

  • Abstract

    We develop an extension to a concurrent, actor-based language runtime called Pony with the

    ability of transparent distributed programming, whereby programmers do not need to be aware of

    the underlying distributed system. Thus, any Pony application can be executed in a concurrent

    setting as well as in a distributed setting without any changes to the code being necessary.

    A distributed cluster of Ponies, managed based on a tree network topology, is easy to maintain

    and can be extended with slave nodes at runtime without any reconfiguration being necessary. We

    propose a joining algorithm which guarantees that the underlying tree topology is almost balanced

    at any point in time.

    Distribution is reflected through actor mobility. We develop a hierarchical work stealing al-

    gorithm with constant space complexity, specifically tailored for tree network topologies. Work

    stealing is provably optimal for task and data parallelism. Furthermore, the implementation of

    the proposed distributed work stealing algorithm is itself based on the actor programming model,

    which allowed us to extend Pony without sacrificing the performance of single-node configurations.

    Causal message delivery is a valuable property for message-passing systems, contributing to

    efficiency and improved reasonability. Pony guarantees causality for both the concurrent and

    distributed setting without any additional software overhead.

    A property unique to Pony is fully concurrent garbage collection of actors. We present an

    extended algorithm for concurrent garbage collection (including cycle detection) in distributed

    actor-based applications. Concurrent Pony programs are terminated based on quiescence. In order

    to retain that property in a distributed context, we propose a protocol for detecting quiescence in

    distributed actor systems. Both schemes strongly depend on causal message delivery.

    We evaluate our implementation based on a micro benchmark that allows us to simulate different

    application characteristics. Pony achieves considerable speed-up for computation-bound scenarios.

    i

  • ii

  • Acknowledgements

    I would like to acknowledge the support and input from a number of people who helped me to

    bring this thesis into being.

    Firstly, my two supervisors, Prof. Sophia Drossopoulou and Sylvan Clebsch, deserve my thanks

    for guiding me through this project, for investing more time than I could have ever expected and

    for offering me the opportunity to work on a programming language runtime. I would also like to

    thank them for changing my perspective on the importance of type systems for reasonability and

    efficiency, as well as for influencing my view on what future programming languages could be like.

    I would also like to thank Prof. Alexander Wolf for having increased my interest in distributed

    systems and algorithms as well as for preparing me with the necessary knowledge for a project like

    this, which helped me to identify and overcome the challenges that came up during the development

    of Distributed Pony.

    I owe a huge debt to my parents, Dieter and Monika, for having supported me in pursuing my

    dreams. Thanks for introducing me to the world of computers at a very young age and for allowing

    me the explore my interests. Without them, I would have never reached this point. I must also

    thank my siblings, Alexander and Ann-Kathrin, for sharing their experiences, which made so many

    things easier for me.

    I thank Reuben Rowe for having mentored me throughout this Masters program. I would like

    to extend my appreciation to the SLURP group of Imperial College London for the interesting

    discussions on various research topics of computer science.

    I thank Florian Oswald for his friendship and for joining me in founding a company. I am

    looking forward to developing innovative software systems, solving interesting problems and seeing

    our company evolve.

    Finally, I would like to thank Prof. Dr. Werner Zorn for having provided me with advice

    throughout my entire university career as well as Prof. Dr. h.c. Hasso Plattner for supervising my

    undergraduate studies at HPI Potsdam and for supporting me in evolving as a software engineer.

    iii

  • iv

  • Contents

    Abstract i

    Acknowledgements iii

    1 Introduction 1

    2 Background 3

    2.1 Programming Distributed Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

    2.2 The Actor Programming Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

    2.3 Causality in Distributed Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

    2.4 Failure Detection in Asynchronous Networks . . . . . . . . . . . . . . . . . . . . . . 7

    2.5 Scheduling of Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

    2.5.1 Preemptive and Cooperative Scheduling in Operating Systems . . . . . . . . 9

    2.5.2 Priority-based Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

    2.5.3 Operating System Scheduling vs. Actor Runtime Scheduling . . . . . . . . . 11

    2.5.4 Work Stealing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

    2.6 Pony . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

    2.6.1 The Pony Actor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

    2.6.2 Message Queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

    2.6.3 Work Stealing Queue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

    2.6.4 The Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

    2.6.5 Pool Allocator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

    2.6.6 Garbage Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

    2.6.7 Termination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

    2.7 Available Actor Languages and Runtimes . . . . . . . . . . . . . . . . . . . . . . . . 25

    2.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

    3 Distributed Pony 27

    3.1 Overview of Runtime Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

    3.2 Asynchronous I/O Multiplexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

    3.2.1 Epoll, KQueue and Kevent . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

    3.2.2 Select and Poll . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

    3.2.3 Scheduling I/O Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

    3.2.4 Framed Network Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

    v

  • 3.3 The Distribution Actor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

    3.3.1 Non-blocking Sockets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

    3.3.2 Connecting Slave Nodes and Routing . . . . . . . . . . . . . . . . . . . . . . . 39

    3.4 Serialization and Deserialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

    3.4.1 Stream Buffer and I/O Vector . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

    3.4.2 Pony Trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

    3.5 Distributed Work Stealing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

    3.5.1 Migration and Actor Proxies . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

    3.5.2 Causality-aware Migration Protocol . . . . . . . . . . . . . . . . . . . . . . . 48

    3.5.3 Handling Actor References and Actor Identity Comparison . . . . . . . . . . 50

    3.5.4 Hierarchical Work Stealing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

    3.5.5 Collapsing Actor Proxies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

    3.6 Distributed Object Identity Comparison . . . . . . . . . . . . . . . . . . . . . . . . . 55

    3.7 Distributed Garbage Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

    3.7.1 Deferred Reference Counting and Passive Object Collection . . . . . . . . . . 57

    3.7.2 Active Object or Actor Collection . . . . . . . . . . . . . . . . . . . . . . . . 58

    3.7.3 Centralized Cycle Detector . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

    3.7.4 Hierarchical Cycle Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

    3.8 Termination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

    4 Causality in Tree Network Topologies 66

    4.1 Informal View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

    4.2 Formal Argument . . . . . . . . . . . . . . . . . . . . . . . . . . . . .