Akka 2.4 plus new commercial features in Typesafe Reactive Platform
Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5...
Transcript of Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5...
![Page 1: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/1.jpg)
Viktor Klang Chief Architect
@viktorklang
or: How I Learned to Stop Worrying and Love the Flow
Dr Streamlove
![Page 2: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/2.jpg)
Typesafe
• Founded ~3.5 years ago • fusion of Scalable Solutions and Scala
Solutions • Offices in Uppsala, Lausanne and San
Francisco • 60+ employees all over the world • Main projects • Play, Akka, Scala, Slick
2
![Page 3: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/3.jpg)
My office door
3
![Page 4: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/4.jpg)
View from my office
4
![Page 5: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/5.jpg)
Agenda
• What is a Stream? • Live demo • What is Reactive? • Reactive Streams • Akka Streams • Live demo • What’s next / Opportunities • Outro
5
![Page 6: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/6.jpg)
What do we mean by “Stream”?
![Page 7: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/7.jpg)
7
“You cannot step twice into the same stream. For as you are stepping in, other waters are ever flowing on to you.”
- Heraclitus
![Page 8: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/8.jpg)
What is a Stream?
• Ephemeral flow of data • Possibly unbounded in length • Focused on describing transformation • Can be formed into processing networks
8
![Page 9: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/9.jpg)
What is a Collection?
• Oxford Dictionary: • “a group of things or people”
• wikipedia: • “a grouping of some variable number of data items”
• backbone.js: • “collections are simply an ordered set of models”
• java.util.Collection: • definite size, provides an iterator, query membership
9
![Page 10: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/10.jpg)
User Expectations
• an Iterator is expected to visit all elements(especially with immutable collections) • x.head ++ x.tail == x • the contents does not depend on who is
processing the collection • the contents does not depend on when the
processing happens(especially with immutable collections)
10
![Page 11: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/11.jpg)
Unexpected: observed sequence depends on
• when the subscriber subscribed to the stream • whether the subscriber can process fast enough • whether the streams flows fast enough
11
![Page 12: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/12.jpg)
java.util.stream
• Stream is not derived from Collection“Streams differ from Collections in several ways” • no storage • functional in nature • laziness seeking • possibly unbounded • consumable
12
![Page 13: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/13.jpg)
Streams vs. Collections
• a collection can be streamed • a stream processor can create a collection • … but saying that a Stream is just a lazy
Collection evokes the wrong associations
13
Streams are not Collections!
![Page 14: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/14.jpg)
Live Demo
![Page 15: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/15.jpg)
Reactive Applications
The Four Horsemen of Reactive
15
http://reactivemanifesto.org/
![Page 16: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/16.jpg)
The Problem: !
Getting Data across an Async Boundary
![Page 17: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/17.jpg)
Possible Solutions
• the Traditional way: blocking calls
17
![Page 18: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/18.jpg)
Possible Solutions
!
• the Push way: buffering and/or dropping
18
![Page 19: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/19.jpg)
Possible Solutions
!
!
• the Reactive way:non-blocking & non-dropping & bounded
19
![Page 20: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/20.jpg)
Reactive Streams Initiative
![Page 21: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/21.jpg)
21
“Reactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure on the JVM.”
- reactive-streams.org
![Page 22: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/22.jpg)
Collaboration between Engineers
• Björn Antonsson – Typesafe Inc.
• Gavin Bierman – Oracle Inc.
• Jon Brisbin – Pivotal Software Inc.
• George Campbell – Netflix, Inc
• Ben Christensen – Netflix, Inc
• Mathias Doenitz – spray.io
• Marius Eriksen – Twitter Inc.
• Tim Fox – Red Hat Inc.
• Viktor Klang – Typesafe Inc.
22
• Dr. Roland Kuhn – Typesafe Inc.
• Doug Lea – SUNY Oswego
• Stephane Maldini – Pivotal Software Inc.
• Norman Maurer – Red Hat Inc.
• Erik Meijer – Applied Duality Inc. • Todd Montgomery – Kaazing Corp.
• Patrik Nordwall – Typesafe Inc.
• Johannes Rudolph – spray.io
• Endre Varga – Typesafe Inc.
![Page 23: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/23.jpg)
Motivation
• all participants face the same basic problem • all are building tools for their community • a common solution benefits everybody • interoperability to make best use of efforts • e.g. use Reactor data store driver with Akka
transformation pipeline and Rx monitoring to drive a vert.x REST API (purely made up, at this point)
• propose to include in future JDK
23
See also: Jon Brisbin’s post on “Tribalism as a Force for Good”
![Page 24: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/24.jpg)
Recipe for Success
• minimal interfaces—essentials only • rigorous specification of semantics • TCK for verification of implementation • complete freedom for many idiomatic APIs
24
![Page 25: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/25.jpg)
Reactive Streams
• asynchronous & non-blocking • flow of data • flow of demand
• minimal coordination and contention • message passing allows for distribution across • applications, nodes, CPUs, threads, actors
25
![Page 26: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/26.jpg)
A Data Market using Supply & Demand
• data elements flow downstream • demand flows upstream • data elements flow only when there is demand • data in flight is bounded by signaled demand • recipient is in control of maximal incoming data rate
26
Publisher Subscriber
data
demand
![Page 27: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/27.jpg)
Dynamic Push–Pull
• “push”—when consumer is faster • “pull”—when producer is faster • switches automatically between these • batching demand allows batching data
27
Publisher Subscriber
data
demand
![Page 28: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/28.jpg)
Explicit Demand: One-to-many
28
demand
data
Splitting the data means merging the demand
![Page 29: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/29.jpg)
Explicit Demand: Many-to-one
29
Merging the data means splitting the demand
![Page 30: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/30.jpg)
The Meat: Scala
30
trait Publisher[T] { def subscribe(sub: Subscriber[T]): Unit } trait Subscription { def request(n: Int): Unit def cancel(): Unit } trait Subscriber[T] { def onSubscribe(s: Subscription): Unit def onNext(e: T): Unit def onError(t: Throwable): Unit def onComplete(): Unit }
![Page 31: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/31.jpg)
The dessert: Scala
31
!!!!!trait Processor[T, R] extends Subscriber[T] with Publisher[R]
![Page 32: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/32.jpg)
The Meat: Java
32
public interface Publisher<T> { public void subscribe(Subscriber<T> s); } public void Subscription { public void request(Int n); public void cancel(); } public interface Subscriber<T> { public void onSubscribe(Subscription s); public void onNext(T t); public void onError(Throwable t); public void onComplete(); }
![Page 33: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/33.jpg)
The dessert: Java
33
!!!!!public interface Processor<T, R> extends Subscriber<T>, Publisher<R> { }
![Page 34: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/34.jpg)
How does it Connect?
34
SubscriberPublisher
subscribe
onSubscribeSubscription
![Page 35: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/35.jpg)
How does it Flow?
35
SubscriberPublisher
request
onNextElements
request
onNextElements
request
![Page 36: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/36.jpg)
How does it Complete?
36
SubscriberPublisher
request
onNextElements
onComplete
![Page 37: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/37.jpg)
How does it Fail?
37
SubscriberPublisher
request
onNextElements
request
onError☠
![Page 38: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/38.jpg)
Akka Streams
![Page 39: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/39.jpg)
WAIT! What is ?
![Page 40: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/40.jpg)
40
![Page 41: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/41.jpg)
41
![Page 42: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/42.jpg)
42
![Page 43: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/43.jpg)
Akka
• Akka's unit of computation is called an Actor • Akka Actors are purely reactive components: • an address • a mailbox • a current behavior • local storage
• Scheduled to run when sent a message • Each actor has a parent, handling its failures • Each actor can have 0..N “child” actors
43
![Page 44: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/44.jpg)
Akka Actors
• An actor processes a message at a time • Multiple-producers & Single-consumer
• The overhead per actor is about ~450bytes • Run millions of actors on commodity hardware
• Akka Cluster currently handles ~2500 nodes • 2500 nodes × millions of actors
= “ought to be enough for anybody”
44
![Page 45: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/45.jpg)
Actor model fundamentals
• CREATE(behavior) • Creates a new actor
• BECOME(behavior) • Changes the actors behavior for the next message
• SEND(message) • Sends a message asynchronously and non-blocking to an
actor
45
![Page 46: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/46.jpg)
Actor model augmentations
• SUPERVISE(actor) • Lets an actor handle the failure(s) of another actor
• WATCH(actor) • Lets an actor observe the termination of another actor
46
![Page 47: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/47.jpg)
Actor Messages vs Failures
47
Child
Parent
Messages
Replies
Failures / Recovery
![Page 48: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/48.jpg)
Actor Hierarchies
48
system.actorOf(fooProps, “Foo”)
A
B
BarFoo
C
B E
A
DC
Guardian System Actor
context.actorOf(aProps, “A”)
![Page 49: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/49.jpg)
Actor Paths
49
A
B
BarFoo
C
B E
A
DC
/Foo
/Foo/A
/Bar/A/B
/Foo/A/D
Guardian System Actor
![Page 50: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/50.jpg)
Canonical papers
• Carl Hewitt; Peter Bishop; Richard Steiger (1973). A Universal Modular Actor Formalism for Artificial Intelligence. IJCAI. • Gul Agha (1986). Actors: A Model of Concurrent
Computation in Distributed Systems. Doctoral Dissertation. MIT Press.
50
![Page 51: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/51.jpg)
Akka Streams
• powered by Akka Actors because • execution • distribution • resilience
• Typesafe (pun intended) streaming through Actors with bounded buffering • Flow & Duct DSL is a lifted representation • Uses pluggable materialisation
51
![Page 52: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/52.jpg)
Akka HTTP Server Overview
52
TCP Connection
parsing
rend
ering
bypass
user handler
![Page 53: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/53.jpg)
Akka HTTP Server Part 1
53
val requestProducer = Flow(tcpConn.inputStream) .transform(rootParser) .splitWhen(_.isInstanceOf[MessageStart]) .headAndTail // Flow[(Start, Producer[…])] .tee(bypassConsumer) .collect { case (x: RequestStart, entityParts) => HttpServerPipeline.constructRequest(x, entityParts) } .toProducer(materializer)
![Page 54: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/54.jpg)
Akka HTTP Server Part 2
54
val (bypassConsumer, bypassProducer) = Duct[(RequestOutput, Producer[RequestOutput])] .collect[MessageStart with RequestOutput] { case (x: MessageStart, _) => x } .build(materializer)
![Page 55: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/55.jpg)
Akka HTTP Server Part 3
55
val responseConsumer = Duct[HttpResponse] .merge(bypassProducer) .transform(applyApplicationBypass) .transform(rendererFactory.newRenderer) .flatten(concat) .transform(logErrors) .toProducer(materializer) .produceTo(tcpConn.outputStream)
![Page 56: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/56.jpg)
Akka HTTP server Part 4
56
val logErrors = new Transformer[ByteString, ByteString] { def onNext(element: ByteString) = element :: Nil override def onError(cause: Throwable) = log.error(cause, "Response stream error") }
![Page 57: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/57.jpg)
Advanced Live Demo
![Page 58: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/58.jpg)
What’s next for Akka Streams?
![Page 59: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/59.jpg)
Opportunity: API
• Current API is minimal • Establish core functionality and take it from there
• Naming: Use established terminology or simplified? • Both Scala and Java APIs • Allows for use by other JVM-hosted languages
59
![Page 60: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/60.jpg)
Opportunity: Self-tuning back pressure
• Each processing stage can know • Latency between requesting more and getting more • Latency for internal processing • Behavior of downstream demand • Latency between satisfying and receiving more • Trends in requested demand (patterns)
• Lock-step • N-buffered • N + X-buffered • “chaotic”
60
![Page 61: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/61.jpg)
Opportunity: Operation Fusion
• Compile-time, using Scala Macros • filter ++ map == collect • map ++ filter == collect?
• Run-time, using intra-stage simplification • Rule: <any> ++ identity == <any>
Rule: identity ++ <any> == <any> • filter ++ dropUntil(cond) ++ map • filter ++ identity ++ map == collect
61
![Page 62: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/62.jpg)
Opportunity: Operation Elision
• Compile-time, using Scala Macros • fold ++ take(n where n > 0) == fold • drop(0) == identity • <any> concat identity == <any>
• Run-time, using intra-stage simplification • map ++ dropUntil(cond) ++ take(N) • map ++ identity ++ take(N) • map ++ take(N)
62
![Page 63: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/63.jpg)
Opportunity: Execution optimizations
• synchronous intra-stage execution N steps then trampoline and/or give control to other Thread / Flow • We already do inter-stage execution reduction
63
![Page 64: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/64.jpg)
Opportunity: Distributed Streams
• Encode Reactive Streams as a transport protocol • Possibility to run over • TCP • UDP • … essentially any bidirectional channel
• MUX-ing streams • Materialize a Flow on a cluster of Akka nodes
64
![Page 65: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/65.jpg)
Outro: How do I get my hands on this?
• http://reactive-streams.org/ • https://github.com/reactive-streams • Early Preview is available:"org.reactivestreams" % "reactive-streams-spi" % "0.3""com.typesafe.akka" %% "akka-stream-experimental" % "0.3" • check out the Activator template
"Akka Streams with Scala!"(https://github.com/typesafehub/activator-akka-stream-scala)
65
![Page 66: Dr Streamloveecoop14.it.uu.se/programme/DrStreamlove.pdfDr Streamlove Typesafe • Founded ~3.5 years ago • fusion of Scalable Solutions and Scala Solutions • Offices in Uppsala,](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9747021939034e86c7631/html5/thumbnails/66.jpg)
©Typesafe 2014 – All Rights Reserved