[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

94
Konrad 'ktoso' Malawski GeeCON 2014 @ Kraków, PL Konrad `@ktosopl` Malawski @ Reactive Systems Meetup streams How reactive streams change the JVM Ecosystem & reative streams akka streams JVM エコシステムをどのように変えるか

Transcript of [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Page 1: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Konrad 'ktoso' MalawskiGeeCON 2014 @ Kraków, PL

Konrad `@ktosopl` Malawski @ Reactive Systems Meetup, Tokyo

streamsHow reactive streams

change the JVM Ecosystem

&

reative streams と akka streams が JVMエコシステムをどのように変えるか

Page 2: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Konrad `ktoso` Malawski

(we’re renaming soon!)

Akka Team, Reactive Streams TCK, Maintaining Akka Http

TypesafeのKonrad `ktoso` Malawskiです。(Typesafeもうすぐ会社名を変更します)

Page 3: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Konrad `@ktosopl` Malawski

akka.iotypesafe.comgeecon.org

Java.pl / KrakowScala.plsckrk.com / meetup.com/Paper-Cup @ London

GDGKrakow.pl lambdakrk.pl

(we’re renaming soon!)

Page 4: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku
Page 5: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Nice to meet you! Who are you guys?

はじめまして!

Page 6: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Agenda for today:

• Story & landscape

• The Reactive Streams Protocol

• Akka Streams / Demo

• Akka Http / Demo

• Q/A?

本日のアジェンダです。経緯と展望、Reacive Streams プロトコル、Akka Streamsとデモ、Akka Http とデモ、Q/A

Page 7: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Reactive Streams - story: early FRP

http://blogs.msdn.com/b/rxteam/archive/2009/11/17/announcing-reactive-extensions-rx-for-net-silverlight.aspxhttp://infoscience.epfl.ch/record/176887/files/DeprecatingObservers2012.pdf - Ingo Maier, Martin Odersky

https://github.com/ReactiveX/RxJava/graphs/contributorshttps://github.com/reactor/reactor/graphs/contributors

https://medium.com/@viktorklang/reactive-streams-1-0-0-interview-faaca2c00bec#.69st3rndy

- .NETs’ Reactive Extensions

.NET 3.5

Reactive Streams のストーリー: 初期のFRP

.NET の Reactive Extensions がリリースされました。

Page 8: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Reactive Streams - story: 2013’s impls

~2013:

Reactive Programming becoming widely adopted on JVM.

- Play introduced “Iteratees”- Akka (2009) had Akka-IO (TCP etc.)- Ben starts work on RxJava

http://blogs.msdn.com/b/rxteam/archive/2009/11/17/announcing-reactive-extensions-rx-for-net-silverlight.aspxhttp://infoscience.epfl.ch/record/176887/files/DeprecatingObservers2012.pdf - Ingo Maier, Martin Odersky

https://github.com/ReactiveX/RxJava/graphs/contributorshttps://github.com/reactor/reactor/graphs/contributors

https://medium.com/@viktorklang/reactive-streams-1-0-0-interview-faaca2c00bec#.69st3rndy

Teams discuss need for back-pressure in simple user API.Play’s Iteratee / Akka’s NACK in IO.

}

2013年には Reactive Programming が JVM 上で広く採用されるようになりました。Play, Akkaのチームは簡潔なユーザAPIによる背圧制御の必要性を議論していました。

Page 9: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Reactive Streams - story: 2013’s impls

Play Iteratees – pull back-pressure, difficult API

http://blogs.msdn.com/b/rxteam/archive/2009/11/17/announcing-reactive-extensions-rx-for-net-silverlight.aspxhttp://infoscience.epfl.ch/record/176887/files/DeprecatingObservers2012.pdf - Ingo Maier, Martin Odersky

https://github.com/ReactiveX/RxJava/graphs/contributorshttps://github.com/reactor/reactor/graphs/contributors

https://medium.com/@viktorklang/reactive-streams-1-0-0-interview-faaca2c00bec#.69st3rndy

Akka-IO – NACK back-pressure; low-level IO (Bytes); messaging API

RxJava – no back-pressure, nice API

Play の Iteratee は pull 型の背圧制御でAPIが難しく、Akka-IO は NACK 背圧制御で低レベルI/OとメッセージングAPI, RxJava は背圧制御はないが優れたAPIでした。

Page 10: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Reactive Streams - Play’s Iteratees

def fold[B]( done: (A, Input[E]) => Promise[B], cont: (Input[E] => Iteratee[E, A]) => Promise[B], error: (String, Input[E]) => Promise[B]): Promise[B]

// an iteratee that consumes chunkes of String and produces an IntIteratee[String,Int]

https://www.playframework.com/documentation/2.0/Iteratees

Feb 2013 Iteratees solved the back-pressure problem,but were hard to use.

Iteratee & Enumeratee – Haskell inspired.

Play / Akka teams looking for common concept.

2013年2月, Play の Iteratee は背圧制御の問題を解決しましたが使いづらく、Play/Akkaのチームは共通のコンセプトを探していました。

Page 11: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Reactive Streams - expert group founded

October 2013

Roland Kuhn (Akka) and Erik Meijer (Rx .NET) meet in Lausanne,while recording “Principles of Reactive Programming” Coursera Course.

Viktor Klang (Akka), Erik Meijer, Ben Christensen (RxJava) and Marius Eriksen (Twitter) meet at Twitter HQ.

The term “reactive non-blocking asynchronous back-pressure” gets coined.

2013年10月, Roland Kuhn(Akka)、Erik Meijer (Rx .NET) らにより"リアクティブノンブロッキング非同期背圧制御"という言葉が造り出されました。

Page 12: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Reactive Streams - expert group founded

October 2013

Roland Kuhn (Akka) and Erik Meijer (Rx .NET) meet in Lausanne,while recording “Principles of Reactive Programming” Coursera Course.

Viktor Klang (Akka), Erik Meijer, Ben Christensen (RxJava) and Marius Eriksen (Twitter) meet at Twitter HQ.

The term “reactive non-blocking asynchronous back-pressure” gets coined.

Goals:- asynchronous- never block (waste)- safe (back-threads pressured)- purely local abstraction- allow synchronous impls.

Also, for our examples today:- compatible with TCP

非同期、ブロックしない、安全、純粋にローカルのような抽象化、同期の実装も許すことをゴールとしました。また、今日の前例にならいTCPとの互換性も目指しました。

Page 13: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Reactive Streams - expert group founded

October 2013

Roland Kuhn (Akka) and Erik Meijer (Rx .NET) meet in Lausanne,while recording “Principles of Reactive Programming” Coursera Course.

Viktor Klang (Akka), Erik Meijer, Ben Christensen (RxJava) and Marius Eriksen (Twitter) meet at Twitter HQ.

The term “reactive non-blocking asynchronous back-pressure” gets coined.

December 2013Stephane Maldini & Jon Brisbin (Pivotal Reactor) contacted by Viktor.

Sephane Maldini と Jon Brisbin (Pival Reactor) が Viktor にコンタクトしました。

Page 14: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Reactive Streams - expert group founded

October 2013

Roland Kuhn (Akka) and Erik Meijer (Rx .NET) meet in Lausanne,while recording “Principles of Reactive Programming” Coursera Course.

Viktor Klang (Akka), Erik Meijer, Ben Christensen (RxJava) and Marius Eriksen (Twitter) meet at Twitter HQ.

The term “reactive non-blocking asynchronous back-pressure” gets coined.

December 2013Stephane Maldini & Jon Brisbin (Pivotal Reactor) contacted by Viktor.

Soon after, the “Reactive Streams” expert group is formed.

Also joining the efforts: Doug Lea (Oracle), Endre Varga (Akka), Johannes Rudolph & Mathias Doenitz (Spray), and many others, including myself join the effort soon after.

まもなく "Reactive Streams" エキスパートグループが創設されました。Doug Lea(Oracle)や、私自身も含めた多くの人々が参画しました。

Page 15: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

October 2013

Roland Kuhn (Akka) and Erik Meijer (Rx .NET) meet in Lausanne,while recording “Principles of Reactive Programming” Coursera Course.

Viktor Klang (Akka), Erik Meijer, Ben Christensen (RxJava) and Marius Eriksen (Twitter) meet at Twitter HQ.

The term “reactive non-blocking asynchronous back-pressure” gets coined.

December 2013Stephane Maldini & Jon Brisbin (Pivotal Reactor) contacted by Viktor.

Soon after, the “Reactive Streams” expert group is formed.

Also joining the efforts: Doug Lea (Oracle), Endre Varga (Akka), Johannes Rudolph & Mathias Doenitz (Spray), and many others, including myself join the effort soon after.

Reactive Streams - expert group founded

I ended up implementing much of the TCK.Please use it, let me know if it needs improvements :-)

私はTCK(Technology Compatibility Kit) の大部分の実装を終えました。ぜひ使ってみて、もし改良が必要なら教えてください :-)

Page 16: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Reactive Streams - story: 2013’s impls

2014–2015:

Reactive Streams Spec & TCK development, and implementations.

1.0 released on April 28th 2015, with 5+ accompanying implementations.

2015Proposed to be included with JDK9 by Doug Leavia JEP-266 “More Concurrency Updates”

http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/6e50b992bef4/src/java.base/share/classes/java/util/concurrent/Flow.java

2014-2015に Reactive Streams の仕様とTCKが開発・実装され、2015年4月28日に 1.0 が5つ以上の実装と共にリリースされました。

Page 17: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

2014–2015:

Reactive Streams Spec & TCK development, and implementations.

1.0 released on April 28th 2015, with 5+ accompanying implementations.

2015Proposed to be included with JDK9 by Doug Leavia JEP-266 “More Concurrency Updates”

http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/6e50b992bef4/src/java.base/share/classes/java/util/concurrent/Flow.java

Reactive Streams - story: 2013’s impls

Doug Lea により、これを JDK9 に含めることがJEP-266 "More Concurrency Updates" を通じて提案されました。

Page 18: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

in a few words:

• Toolkit for building scalable distributed / concurrent apps.

• High Performance Actor Model implementation

• “share nothing” – messaging instead of sharing state

• millions of msgs, per actor, per second

• Supervision trees – built-in and mandatory

• Clustering and Http built-in

A

B

BarFoo

C

BE

A

D

C

/Foo

/Foo/A

/Foo/A/B

/Foo/A/D

Guardian System Actor

Name resolution—like a file-system

akka はスケーラブルな分散/並行アプリを構築するためのツールキットです。高性能なActorモデルの実装で、Supervisionツリー、クラスタリング/Httpを組込んでいます。

Page 19: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Why back-pressure?

?

なぜ背圧制御なのでしょうか?

Page 20: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Why back-pressure?

So you’ve built your app and it’s awesome.

皆さんはアプリを構築してきました、それは素晴らしいことです。

Page 21: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Why back-pressure?

Let’s not smash it horribly under load.

負荷をかけすぎて、壊さないようにしましょう。

Page 22: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

What is back-pressure?

?

背圧制御(back pressure) とは何でしょうか?

Page 23: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

What is back-pressure?

背中(back)が痛くて死にそうだ!

Page 24: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

No no no…! Not THAT Back-pressure!

No no no…! Not THAT Back-pressure!

What is back-pressure?

いやいやいや...! その back pressure ではありません!

Page 25: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Publisher[T] Subscriber[T]

Back-pressure explained

背圧制御の説明

Page 26: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Fast Publisher Slow Subscriber

What if…?

もしも、1秒間に100の命令を処理する高速なPublisherと、1秒間に1つの命令しか処理しない低速なSuscriberだったら?

Page 27: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Push + NACK model

Push と NACK モデル

Page 28: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Push + NACK model

Subscriber usually has some kind of buffer.

Subscriber は普通ある種のバッファを持っています。

Page 29: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Push + NACK model

Page 30: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Push + NACK model

Page 31: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

What if the buffer overflows?

Push + NACK model

もしバッファがオーバーフローしたら何が起こるでしょう?

Page 32: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Use bounded buffer, drop messages + require re-sending

Push + NACK model

大きさが限られたバッファを使い、メッセージを廃棄し、再送を要求します。

Page 33: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Kernel does this!Routers do this!

(TCP)

Use bounded buffer, drop messages + require re-sending

Push + NACK model

カーネルがこれを行います! ルーターがこれを行います! (TCP)

Page 34: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Increase buffer size… Well, while you have memory available!

Push + NACK model

バッファサイズを増やしてみましょう...

そう、使用できるメモリがある限りは!

Page 35: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Push + NACK model

メモリ不足

Page 36: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Negative ACKnowledgement

Page 37: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

NACKingBuffer overflow is imminent!

バッファオーバーフローが今にも起こりそうです!

Page 38: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Telling the Publisher to slow down / stop sending…

NACKing

Publisher にスローダウンするか、送信を止めるよう知らせます...

Page 39: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

NACK did not make it in time, because M was in-flight!

NACKing

NACK は間に合いませんでした、メッセージが飛んできている最中だったからです!

Page 40: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

What if…We don’t need to back-pressure, because:

speed(publisher) < speed(subscriber)

We need low-overhead for “happy case”

もしも、publisher のスピード < subscrber のスピードであれば、背圧制御は必要ありません。

Page 41: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

No problem!

Fast Subscriber => no problem

高速な Subscriber であれば、問題ありません!

Page 42: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Back-pressure?Reactive-Streams

= “Dynamic Push/Pull”

Fast Subscriber => no problem

背圧制御? Reactive Streams = "動的な Push/Pull"

Page 43: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Just push – not safe when Slow Subscriber

Just pull – too slow when Fast Subscriber

Reactive Streams: “dynamic push/pull”

push だけだと、Subscriberが低速な時には、安全ではありません。pull だけだと、Subscriberが高速な時には、遅すぎます。

Page 44: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Solution:Dynamic adjustment

Just push – not safe when Slow Subscriber

Just pull – too slow when Fast Subscriber

Reactive Streams: “dynamic push/pull”

解決策は、動的な調整です。

Page 45: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Slow Subscriber sees it’s buffer can take 3 elements. Publisher will never blow up its buffer.

Reactive Streams: “dynamic push/pull”

低速なSubscriberはバッファに3要素入れられることがわかります。Publsherはバッファをあふれさせることは決してありません。

Page 46: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Fast Publisher will send at-most 3 elements. This is pull-based-backpressure.

Reactive Streams: “dynamic push/pull”

高速なPublisherは、高々3つの要素を送ります。これがpullベースの背圧制御です。

Page 47: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Fast Subscriber can issue more Request(n), before more data arrives!

Reactive Streams: “dynamic push/pull”

高速なSubscriberは、データが到着する前に、より多くのリクエストを発行することができます!

Page 48: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Fast Subscriber can issue more Request(n), before more data arrives.

Publisher can accumulate demand.

Reactive Streams: “dynamic push/pull”

高速なSubscriberは、需要を蓄積することができます。

Page 49: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Publisher accumulates total demand per subscriber.

Reactive Streams: accumulate demand

Publisherはsubscriberごとに需要の合計を蓄積します。

Page 50: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Total demand of elements is safe to publish. Subscriber’s buffer will not overflow.

Reactive Streams: accumulate demand

需要の合計分の要素をpublishしても安全です。Subscriberのバッファはオーバーフローしないでしょう。

Page 51: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

http://reactive-streams.org

We want to make different implementations co-operate with each other.

Reactive Streams: Inter Op

私たちは、異なる実装がお互いに協力できるようにしたいのです。

Page 52: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

http://reactive-streams.org

We want to make different implementations co-operate with each other.

Reactive Streams: Inter Op

私たちは、異なる実装がお互いに協力できるようにしたいのです。

Page 53: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

RS is NOT a “daily use”, “end-user” API. It’s an SPI - Service Provider Interface.

Reactive Streams: Inter-Op

https://en.wikipedia.org/wiki/Service_provider_interface

Service Provider Interface (SPI) is an API intended to beimplemented or extended by a third party.

RS は "毎日使用する", "エンドユーザ向けの" API ではなく、サードパーティにより実装される、または拡張される SPI (Service Provider Interface) です。

Page 54: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

EmbeddedApp.fromHandler(new Handler { override def handle(ctx: Context): Unit = { // RxJava Observable val intObs = Observable.from((1 to 10).asJava)

// Reactive Streams Publisher val intPub = RxReactiveStreams.toPublisher(intObs)

// Akka Streams Source val stringSource = Source(intPub).map(_.toString)

// Reactive Streams Publisher val stringPub = stringSource.runWith(Sink.fanoutPublisher(1, 1))

// Reactor Stream val linesStream = Streams.create(stringPub).map[String]( new reactor.function.Function[String, String] { override def apply(in: String) = in + "\n" })

// and now render the HTTP response (RatPack) ctx.render(ResponseChunks.stringChunks(linesStream)) }

}).test(new Consumer[TestHttpClient] { override def accept(client: TestHttpClient): Unit = {

Reactive Streams: Inter-Op

https://en.wikipedia.org/wiki/Service_provider_interface

Page 55: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

EmbeddedApp.fromHandler(new Handler { override def handle(ctx: Context): Unit = { // RxJava Observable val intObs = Observable.from((1 to 10).asJava)

// Reactive Streams Publisher val intPub = RxReactiveStreams.toPublisher(intObs)

// Akka Streams Source val stringSource = Source(intPub).map(_.toString)

// Reactive Streams Publisher val stringPub = stringSource.runWith(Sink.fanoutPublisher(1, 1))

// Reactor Stream val linesStream = Streams.create(stringPub).map[String]( new reactor.function.Function[String, String] { override def apply(in: String) = in + "\n" })

// and now render the HTTP response (RatPack) ctx.render(ResponseChunks.stringChunks(linesStream)) }

}).test(new Consumer[TestHttpClient] { override def accept(client: TestHttpClient): Unit = {

Reactive Streams: Inter-Op

https://en.wikipedia.org/wiki/Service_provider_interface

Page 56: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Akka Streams

streams

Page 57: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Akka Streams & HTTP

streams& HTTP

Page 58: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Akka Streams in 20 seconds: // types: Source[Out, Mat] Flow[In, Out, Mat] Sink[In, Mat]

// generally speaking, it's always: val ready = Source(???).via(flow).map(_ * 2).to(sink)

val mat: Mat = ready.run() // the usual example: val f: Future[String] = Source.single(1).map(_.toString).runWith(Sink.head)

Proper static typing!

Akka Streams を 20秒で説明します。Source, Flow, Sink に適切な静的型付けをします。

Page 59: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Akka Streams in 20 seconds:

// types: _Source[Int, akka.NotUsed] Flow[Int, String, akka.NotUsed] Sink[String, Future[String]]

Source.single(1).map(_.toString).runWith(Sink.head)

Page 60: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Akka Streams in 20 seconds:

// types: _Source[Int, Unit] Flow[Int, String, Unit] Sink[String, Future[String]]

Source.single(1).map(_.toString).runWith(Sink.head)

Page 61: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Akka HTTP

Joint effort of Spray and Akka teams. Complete HTTP Server/Client implementation.

Soon prod ready, developed ~1.5 years. Learns from Spray’s 3-4 years history.

Since the beginning with streaming as first class citizen.

Akka HTTP は、SprayとAkkaチームが協力して開発した、完全なHTTPサーバ/クライアント実装です。1年半かけて開発されており、間もなく業務に耐えうるレベルになるでしょう。

Page 62: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

It’s turtles buffers all the way down!

親亀の上に子亀、子亀の上に孫亀がのるかのように、バッファの次にバッファ、その次にまたバッファ... と無数につらなります!

Page 63: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Streaming from Akka HTTP

Akka HTTP からのストリーミング

Page 64: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Streaming from Akka HTTP

Akka HTTP からのストリーミング

Page 65: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Streaming from Akka HTTP

Page 66: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Streaming from Akka HTTP

Page 67: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Streaming from Akka HTTPNo demand from TCP

= No demand upstream

= Source won’t generate tweets

TCPからの要求がない = 上流の要求がない = Source はツイートを生成しません

Page 68: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Streaming from Akka HTTPNo demand from TCP

= No demand upstream

= Source won’t generate tweets

=>

TCPからの要求がない = 上流の要求がない = Source はツイートを生成しません

Page 69: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Streaming from Akka HTTPNo demand from TCP

= No demand upstream

= Source won’t generate tweets

=>Bounded memory stream processing!

=>大きさが制限されたメモリーでのストリーム処理!

Page 70: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Client / Server “JSON Streaming” demo

Demo time

クライアント/サーバの "JSONストリーミング" デモ

Page 71: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Akka Streams

Hidden powers:

Parallelism &&

Pipelining

Akka Streams の隠されたパワー:

並列処理(Parallelism)とパイプライン

Page 72: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Pipelining Pancakes

http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0/scala/stream-parallelism.html

Page 73: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Pipelining

Page 74: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Pipelining

Page 75: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Pipelining

// Takes a scoop of batter and creates a pancake with one side cookedval fryingPan1: Flow[ScoopOfBatter, HalfCookedPancake, Unit] = Flow[ScoopOfBatter].map { batter => HalfCookedPancake() } // Finishes a half-cooked pancakeval fryingPan2: Flow[HalfCookedPancake, Pancake, Unit] = Flow[HalfCookedPancake].map { halfCooked => Pancake() } // With the two frying pans we can fully cook pancakesval pancakeChef: Flow[ScoopOfBatter, Pancake, Unit] = Flow[ScoopOfBatter].via(fryingPan1).via(fryingPan2)

Page 76: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Parallelism

???

Page 77: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Parallelism

Page 78: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Parallelism

val fryingPan: Flow[ScoopOfBatter, Pancake, Unit] = Flow[ScoopOfBatter].map { batter => Pancake() } val pancakeChef: Flow[ScoopOfBatter, Pancake, Unit] = Flow() { implicit builder =>

val dispatchBatter = builder.add(Balance[ScoopOfBatter](2)) val mergePancakes = builder.add(Merge[Pancake](2)) dispatchBatter.out(0) ~> fryingPan ~> mergePancakes.in(0) dispatchBatter.out(1) ~> fryingPan ~> mergePancakes.in(1) (dispatchBatter.in, mergePancakes.out)}

Page 79: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Parallelism

val fryingPan: Flow[ScoopOfBatter, Pancake, Unit] = Flow[ScoopOfBatter].map { batter => Pancake() } val pancakeChef: Flow[ScoopOfBatter, Pancake, Unit] = Flow() { implicit builder =>

val dispatchBatter = builder.add(Balance[ScoopOfBatter](2)) val mergePancakes = builder.add(Merge[Pancake](2)) dispatchBatter.out(0) ~> fryingPan ~> mergePancakes.in(0) dispatchBatter.out(1) ~> fryingPan ~> mergePancakes.in(1) (dispatchBatter.in, mergePancakes.out)}

Page 80: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Parallelism

val fryingPanFun: ScoopOfBatter ⇒ Future[Pancake] = batter ⇒ Future.successful(Pancake())

val pancakeChef: Flow[ScoopOfBatter, Pancake, Unit] = Flow[ScoopOfBatter].mapAsync(parallelism = 2)(fryingPanFun)

Or simply “mapAsync”:

Page 81: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Pipelining && Parallelism

Parallelism &&

Pipelining

do the heavy-work for you.

並列処理とパイプラインは、あなたのためにきつい作業をやってくれます。

Page 82: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

10/26/2015 spray-can: add websockets support (client & server) · Issue #134 · spray/spray

https://github.com/spray/spray/issues/134 1/13

� Pull requests Issues GistThis repository Search � P

2,092 496197D Watch ! Star � Forkspray / spray�

R

m

:

and others

Labels

Milestone

 akka-http

Assignee

No one assigned

111 participants

spray-can: add websockets support (client & server) #134 Closed sirthias opened this issue on Sep 4, 2012 · 129 comments

New issue

Feature

Notifications

You’re not receivingnotifications from thisthread.

� Subscribe

Ownersirthias commented on Sep 4, 2012

No description provided.

analytically commented on Oct 23, 2012

+1

tommcp commented on Nov 1, 2012

+1

t3hnar commented on Nov 10, 2012

+1

alexbool commented on Nov 10, 2012

+1

olger commented on Nov 16, 2012

+1

pjean commented on Nov 29, 2012

+1

edgurgel commented on Nov 29, 2012

+1

zerni commented on Dec 10, 2012

+1

Bathtor commented on Dec 10, 2012

+1

csenol commented on Dec 16, 2012

+1

WebSocketsA.K.A.

“Spray’s single most upvoted feature request ever”

98 * “+1”WebSockets は、"Spray のこれまでの機能リクエストのなかで、単独で最も票を集めた機能" としても知られています。

Page 83: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Spray’s most requested feature ever: WebSockets

path("ws") { val handler: Flow[Message, Message] = ???

handleWebsocketMessages(handler) }

Page 84: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Spray’s most requested feature ever: WebSockets

path("ws") { val handler: Flow[Message, Message] = ???

handleWebsocketMessages(handler) }

Page 85: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Spray’s most requested feature ever: WebSockets path("ws") { val handler = Flow.fromSinkAndSource( Sink.ignore, Source.single(TextMessage("Hello World!”)))

handleWebsocketMessages(handler) }

Page 86: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Summing up…

Summing up…

要約...

Page 87: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

buffers, buffers everywhere!

https://dev.twitter.com/streaming/overview/request-parameters#stallwarnings

あらゆるところにバッファが!

Page 88: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

JEP-266 – soon…!

public final class Flow { private Flow() {} // uninstantiable

@FunctionalInterface public static interface Publisher<T> { public void subscribe(Subscriber<? super T> subscriber); }

public static interface Subscriber<T> { public void onSubscribe(Subscription subscription); public void onNext(T item); public void onError(Throwable throwable); public void onComplete(); }

public static interface Subscription { public void request(long n); public void cancel(); }

public static interface Processor<T,R> extends Subscriber<T>, Publisher<R> { }}

JEP-266 がまもなく...!

Page 89: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Back-pressure as a feature

特徴(目玉機能)としての背圧制御

Page 90: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Roadmap Update: Streams & HTTP

Already pretty mature and complete implementation. WebSockets!

Play 2.5 (2.5+) uses Akka Streams. (Scala || Java) DSL == same power.

Last phases of polishing up APIs and features. 2.4.2 release in coming weeks.

Akka 2.4 requires JDK8. (that’s about time to do so!)

StreamsとHTTPの最新ロードマップ既に十分成熟しており、実装が完了しました。

Page 91: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

• Reactive Platform • Remoting / Cluster: Docker networking support • Cluster: Split Brain Resolver (beta)• Akka Persistence: Cross-Scala-version snapshot deserializer • Java 6: Extended LTS

• Akka 2.4.2 (released this month, binary compatible with 2.3)

• Akka Streams becomes non experimental in 2.4.2!

• Cluster Tools promoted to stable!

• Persistence promoted to stable!

• Persistence Queries (experimental)• Akka Typed (experimental)• Distributed Data (experimental)

Roadmap Update: Akka

Akka の最新ロードマップ

Page 92: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Links• The projects:

• akka.io • typesafe.com/products/typesafe-reactive-platform • reactive-streams.org

• Viktor Klang’s interview with all RS founding members• Akka HTTP in depth with Mathias and Johannes @ Scala.World

• Akka User - mailing list:• https://groups.google.com/group/akka-user

• Community chat: • http://gitter.im/akka/akka

Page 93: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

Thanks!

onNext(Q/A) (Now’s the time to ask things!)

ktoso @ typesafe.com twitter: ktosopl

github: ktosoteam blog: letitcrash.com

home: akka.io

Page 94: [Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

©Typesafe 2015 – All Rights Reserved