Scala {Culture}

Who Am I• Now: Dynalyst • Lang: Korean(30+ years) Japanese(8+ years) JAVA(8.2+ years) Scala(a year)

• AdTech: AMoAd DSP CAA Reward

What Dynalyst• Dynamic Retargeting + Analyst

•ユーザーの趣味嗜好に適した広告配信を行う国内初のスマートフォンに特化したダイナミックリターゲティング広告

Scala USAGE

Today’s Talk

•How to build Spray

•How to build Akka

•How to Monitor them

mark ad bid imp click cv

redshift

storedimportingloggingrequest tracking

Dcontinuous importing

redshift

reporting & bidding optimizationsummaringstored

date imp click cv

2014/09 10,000 500 10

2014/08 15,000 700 25

continuous reporting D

Benchmark

What is Spray?

http://spray.io

What is Spray?

spray is an open-source toolkit for building REST/HTTP-based integration layers on top of Scala and Akka. Being asynchronous, actor-based, fast, lightweight, modular and testable it's a great way to connect your Scala applications to the world.

Concurrency

Future[T]

implicit val ec: ExecutionContext

val f = Future { “hello world” }

// Future[String]

Future[T]

implicit val ec: ExecutionContext

for { greeting ← Future(“hello world”) name ← Future(“Han”)} yield s”$greeting, $name”// Future[String]

FutureDirectives

• spray-routing directives

• non-blocking actor thread

FutureDirectives

def onComplete[T] (future: ⇒ Future[T]) (implicit ec: ExecutionContext) :Directive1[Try[T]]

implicit val ec: ExecutionContext

def respondWithBidding = { onComplete(bidding) { case Success(bid: Bid) ⇒ … case Success(noBid: NoBid) ⇒ … case Failure(e) ⇒ … }}

FutureDirectives

Type of bidding

def bidding:Future[A<:BidResponse]

Implementation

def bidding:Future[A<:BidResponse]

!

• simple but bad implementation

Future { // All bidding computations :(}

Application Layers

Spray Router Handle raw level http request & response

Bidding Business Logic

External Resources"Fetch bidding candidates from data stores

Global Dispatcher

Application Layers

Spray Router Handle raw level http request & response

Bidding Business Logic

External Resources"Fetch bidding candidates from data stores

Application Layers

Spray Router Handle raw level http request & response

Bidding Business Logic

External Resources"Fetch bidding candidates from data stores

Router Dispatcher

Service Dispatcher

Repository Dispatcher

Implementationtrait Router { implicit val ec = RouterDispatcher def handle: Future[BidResponse]}

trait Service { implicit val ec = ServiceDispatcher def bidding: Future[Bidder]}

trait Repository { implicit val ec = RepositoryDispatcher def candidates: Future[List[Candidate]]}

How to test those Future[T] things

• Specs2 - Matcher[Future[T]]

bidding must beEqual(expected).await

bidding must beEqual(expected).await( retries = 2, timeout = 100.millis)

Abstract Future

http://logji.blogspot.jp/2014/02/the-abstract-future.html

Abstract Futuretrait Router[M[_]] { implicit val M: Monad[M] def handle: M[BidResponse]} trait Service[M[_]] { implicit val M: Monad[M] def bidding: M[Bidder] = M.point(bidder) def bidder = …} trait Repository[M[_]] { implicit val M: Monad[M] def candidates: M[List[Candidate]]}

Type class & Implicit

object Service { implicit def fs = new Service[Future] { implicit val ec = ServiceDispatcher val M = Monad[Future] } implicit def ids = new Service[Id] { val M = Monad[Id] }} // type Id[+A] = A

Test Again

val s = Service.ids

s.bidding must beEqual(expected)

M[_] for Monad

• Monad is a structure that represents computations defined as sequences of steps

• Don’t dive into “Monad” here!

What about Akka?

http://akka.io

redshift

storedimportingloggingrequest tracking

Dcontinuous importing

mark ad bid imp click cv

redshift

reporting & bidding optimizationsummaringstored

date imp click cv

2014/09 10,000 500 10

2014/08 15,000 700 25

continuous reporting D

Master/Worker

http://letitcrash.com/post/29044669086/balancing-workload-across-nodes-with-akka-2

Overview

Overview

How to Monitor Akka

• Typesafe Console

http://resources.typesafe.com/docs/console/manual/overview.html

• Kamon

http://kamon.io/

Typesafe console• Great user interface

• Support by Typesafe

• Expensive

Kamon• Open Source - Free

• Metrics: Akka, Spray, Play(over 50 categories)

• StatsD, New Relic, Datadog

• Migration could be hard

Kamon meets Zabbix

Running Thread Count

Summary

• Future Directive on Spray

• Abstract Future(Type class & Implicit)

• Akka master/worker pattern

• Monitoring Akka with Kamon

Best Practices?

• https://github.com/alexandru/scala-best-practices

• So many “MUST NOT, SHOULD NOT”

• In my opinion, Trial and Error to find right patterns

We are Hiring!• 一緒に働く仲間を募集しています！Scalaに興味のある方はぜひ！

adtech_scala@cyberagent.co.jp

Thanks

han_sangwon@cyberagent.co.jp