By:Harmeet Singh (Taara)Sr. Software Consultant

Agenda

1. Flows

2. Operator Fusion.

3. Graph DSL.

4. Leftover.

Akka Stream: Flows

The way to perform transformations to the data as it flows between Source and Sink.

Akka Stream: Flows

Akka Stream: Example

object StreamingCopyApp extends App {

  val spath = Paths.get("/home/harmeet/akka­stream/knolx.log")  val source: Source[ByteString, Future[IOResult]] = FileIO.fromPath(spath)

  val dpath = Paths.get("/home/harmeet/akka­stream­copy")  val sink: Sink[ByteString, Future[IOResult]] = FileIO.toPath(dpath)

  val runnableGraph: RunnableGraph[Future[IOResult]] = source.to(sink)

  implicit val system = ActorSystem("akka­stream")  implicit val ec = system.dispatcher  implicit  val materializer = ActorMaterializer()

  runnableGraph.run().foreach { result =>    println(s"${result.status}, ${result.count} bytes read")    system.terminate()  }}

Akka Stream: Flow Example

val frame: Flow[ByteString, String, NotUsed] = Framing.delimiter(ByteString("\n"), 10240).map(_.decodeString("UTF8"))

val parse: Flow[String, Event, NotUsed] = Flow[String].map(Event.parsing)

val filter: Flow[Event, Event, NotUsed] = Flow[Event].filter(_.log == "DEBUG")

val serialize: Flow[Event, ByteString, NotUsed] = Flow[Event].map{ event =>    ByteString(event.toJson.compactPrint)}

val composedFlow: Flow[ByteString, ByteString, NotUsed] = frame.via(parse)    .via(filter)    .via(serialize)

val runnableGraph: RunnableGraph[Future[IOResult]] = source.via(composedFlow).toMat(sink) (Keep.right)

Akka Stream: Flow Example

val composedFlow: Flow[ByteString, ByteString, NotUsed] = Framing.delimiter(ByteString("\n"), 10240)

    .map(_.decodeString("UTF8"))    .map(Event.parsing)    .filter(_.log == "DEBUG")    .map { event =>      ByteString(event.toJson.compactPrint)    }

val runnableGraph: RunnableGraph[Future[IOResult]] = source.via(composedFlow)    .toMat(sink) (Keep.right)

Akka Stream: Operator Fusion

Operator Fusion is used to performance optimization to mitigate the cost of passing elements between the

different stages.

Two main consequences are:

1. Passing elements fused stages faster.2. Fused stages no longer run in parallel.

Akka Stream: Operator Fusion

Q. How can we execute fused stages in parallel???

Ans:1. By using async method2. Turn off auto-fusing (akka.stream.materializer.auto­

fusing=off)

Example:

Source(List(1, 2, 3)).map(_ + 1).async.map(_ * 2).to(Sink.ignore)

Akka Stream: Operator Fusion

Akka Stream: Graph DSL

What if your data flow cannot be modeled as a simple linear process?

What if process is better modeled as computation graph instead?

Akka Stream: Graph DSL

The Graph DSL is kind of diagram ASCII art – in many cases you could translate a white board diagram of a graph into

the DSL.

A graph problem is one that typically involves the concept of fanning out and fanning in.

● Fanning out: Single input to multiple output.

● Fanning in: Multiple inputs single output.

Akka Stream: Graph DSL Example

val in = Source(1 to 5)

  

      val f1 = Flow[Int].map(_ * 2)

   val f2 = Flow[Int].map(_ * 1)

Akka Stream: Graph DSL Example

val f3 = Flow[Int].map(_ * 2)

  

     val f4 = Flow[Int].map(_ + 1)

val out = Sink.foreach[Int](println)

Akka Stream: Graph DSL Example

val bcast = builder.add(Broadcast[Int](2))

val merge = builder.add(Merge[Int](2))

Akka Stream: Graph DSL Example

 in ~> f1 ~> bcast ~> f2 ~> merge ~> f4 ~> out

Akka Stream: Graph DSL Example

       bcast ~> f3 ~> merge

Akka Stream: Graph DSL Example

val graph = RunnableGraph.fromGraph(GraphDSL.create() {    implicit builder: GraphDSL.Builder[NotUsed] =>      import GraphDSL.Implicits._

      val in = Source(1 to 5)      val out = Sink.foreach[Int](println)      val f1 = Flow[Int].map(_ * 2)      val f2 = Flow[Int].map(_ * 1)      val f3 = Flow[Int].map(_ * 2)      val f4 = Flow[Int].map(_ + 1)

      val bcast = builder.add(Broadcast[Int](2))      val merge = builder.add(Merge[Int](2))

      in ~> f1 ~> bcast ~> f2 ~> merge ~> f4 ~> out      bcast ~> f3 ~> merge      ClosedShape  })

Akka Stream: Graph DSL ClosedShape

The graph itself was fully self-contained and complete.

OR

It does not have any open input port and output port like a closed circuit.

Akka Stream: Partial Graphs

Whenever we need to create a reusable component for graph, which we can use according to requirements with

existing or new graph.

https://doc.akka.io/docs/akka/2.5/scala/stream/stream-graphs.html#constructing-and-combining-partial-graphs

Leftovers

➔ API’s

➔ Error Handling.

➔ Streaming HTTP

➔ Handling IO Stream

➔ Unit Testing

References:

✔ Akka in Action by Raymond Roestenburg, Rob Bakker, Rob Williams.

✔ Mastering Akka by Christian Baxter

✔ https://doc.akka.io/api/akka/current/akka/stream/index.html?_ga=2.250065414.1494764629.1507381545-176130805.1501847385