Streaming Api Design with Akka, Scala and Spray
40
Streaming Api Design with
-
Upload
natalino-busa -
Category
Software
-
view
511 -
download
2
description
An introduction on how to design APIs, followed by a recipe using scala, and a REST server implementation in Akka and Spray. This presentation looks at API design from two different angles. Firstly, from a API developer/user point of view, we define the API developer journey, we introduce and craft meaningful REST url paths and RESTful resources. Secondly, from an API implementation point of view, we dive into the mechanics of akka, and spray. During the first part we highlight the process of value creation from idea to API design, which translate into the definition of fluent API paths where resources are concatenated, rendering in facts the elements of a streaming analytical pipeline into a set of concatenated resources accessible via the api. We highlight the importance of a meaningful semantic for the API design process and a good match between the value proposition and the resources exposed by the api itself. To render the concept clearer we take the example of a streaming api which collects and gathers readings from a set of jogging wrist bands connected to internet. During the second part of the presentation, we focus on scalability, resiliency and availability of the solution. We introduce the concept of actor as basic processing element. We introduce the concept of location transparencies and hierarchical supervision when mapping actors to a cluster of computing resources. Finally we describe how to translate a url path into a set of routes, and how to dispatch and process the incoming json readings to the right analytical actors. This short overview gives an idea on the mechanics behind a real time, streaming api. And on how to implement api resources as asynchronous communicating actors, in order to provide a rudimental but effective streaming analytical engine.
Transcript of Streaming Api Design with Akka, Scala and Spray
Streaming Api Design with
By Gruban / Patrick Gruban from Munich, Germany (originally posted to Flickr as IMG_9038) [CC-BY-SA-2.0 (http://creativecommons.org/licenses/by-sa/2.0)], via Wikimedia Commons
Define an API that ...given a collection of wristband data from
a bunch of runners
... ...
{ "heart": 108, "name" : "Judy"}
...
Define an API that ...given a collection of wristband data from
a bunch of runners
{ "Fred": 125, "Judy": 110}
... ...
Aggregates the data,
calculate max heart rate each second
grouped by name ...
{ "heart": 108, "name" : "Judy"}
...
Let’s design this!Let’s create a stream ...
POST /api/streams
Let’s design this!Let’s create a stream ...
GET /api/streams/1
{
"id": 1,
"links": {
"uri": "/api/streams/1",
"input": "/api/streams/1/in",
"filters": "/api/streams/1/in/filters"
}
}
Let’s design this!Given a stream id, lets create a filter,
that gives the highest heart rate of the group,
in a window of 1 seconds ...
POST /api/streams/1/in/filters
{ "resolution": 1, "field" : "heart", "transform" : "max" "group_by" : "name"}'
Let’s design this!Given a stream id, lets create a filter,
that gives the highest heart rate of the group,
in a window of 1 seconds ...
GET /api/streams/1/in/filters/1
Let’s design this!Now I can feed running data in ...
POST /api/streams/1/in
{ "heart": 115, "name" : "Judy"}
Let’s design this!And get the filtered data out ...
GET /api/streams/1/in/filtered_by/1/out
{ "Fred": 121, "Judy": 110}
This API is fluent … nice.
/api/streams/1/in
/api/streams/1/in/filtered_by/1/out
Akka: why?●
●
●
●
●
Akka: asynchronous
…
Akka: asynchronous●
●
●
Akka: actors
●
●
●
●
Akka: actorsclass HeartRateActor extends Actor {
var heart_rate = 0.0
def receive = {
case value: Double =>
heart_rate = max(heart_rate, value)
case "get" =>
sender ! heart_rate
}
}
Actors as Resources
/api/streams
Streams Actor
POST
def receive = { case CreateStream => val stream = system.actorOf(StreamActor(id), s"stream-$id")
Actors as Resources
/api/streams/1
Streams Actor Stream-1 Actor
Actors as Resources
/api/streams/1/in/filters/1
Streams Actor Stream-1 Actor Filter-1 Actor
Actors as Resources
/api/streams/1/in/filters/2
Actors as Resources
/api/streams/2/in/filters/1
"New York marathon Verrazano bridge" by Martineric from Lille, France - Marathon de New York : Verrazano Bridge. Licensed under Creative Commons Attribution-Share Alike 2.0 via Wikimedia Commons -
Actor as Scalable Resources
jvm: This is where my actors run
Actor as Scalable Resources
jvm: This is where my actors run
Actor as Scalable Resources
Many jvm’s, cores, nodes, racks
jvm: This is where my actors run
Actor as Scalable Resources
Actors as Scalable Resources
●
●
●
Where is my http port?
Spray: HTTP toolkit based on Akka
Spray: Reactive and Scalable
●
●
●
Spray: Reactive and Scalable
●
●
●
●
●
Spray: Routing val route = { pathPrefix("api/streams" / IntNumber) { id => { get { ctx => (coreActor ? Get(id) ).mapTo[Int] .onSuccess { resource => complete(resource) } }
} ~ someOtherRoute } }
Spray: Web API get the output
●
●
●
●
APIs: back to back connections
…
"Pedalboard (995939579)-2" by Pedalboard_(995939579).jpg: Michael Morel from Barcelona, Spainderivative work: Atlantictire (talk) - Pedalboard_(995939579).jpg. Licensed under Creative Commons Attribution 2.0 via Wikimedia Commons -
![[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)](https://static.fdocuments.net/doc/165x107/558699b1d8b42aa2558b459b/tokyo-scala-user-group-akka-streams-reactive-streams-07.jpg)
