Scalability in Scala and Java Nadav Wiener

• Nadav Wiener (@nadavwr)

• Senior Consultant & Architect @ AlphaCSP

• Following Scala since 2007

About me

Akka

The problem with locks

Actors & message passing

High availability & remoting

STM & Transactors

Agenda

• A actor-based concurrency framework

• Provides solutions for non blocking concurrency

• Written in Scala, also works in Java

• Open source (APL)

• Now releasing 1.0 after 2 years in development

• Lead developer and founder: Jonas Boner

• JRockit, AspectWerkz, AspectJ, Terracotta

What is Akka?

“Queen of Lapland”

5

You may also remember…

6

The Problem

8

Ignorance Is a Bliss

If (account1.getBalance() > amount) {

account1.withdraw(amount)

account2.deposit(amount)

}

Sewing on a button © Dvortygirl, 17 February 2008.

Not an option if you plan on having business

10

So You Use Threads

Coloured Thread © Philippa Willitts, 24 April 2008.

11

But Without Synchronization You Get Unpredictable Behavior

12

Lock based concurrency requires us to constantly second guess our code

13

“The definition of insanity is doing the same thing over and over again and expecting different results. “ – Albert Einstein

People Are Bad At Thinking Parallel

14

15

So you synchronize

With locks?

16

locking

17

locking B

18

Lock too much Lock too little

Lock recklessly?

19

Using locks recklessly

20

22

Must Think Globally:

Lock ordering

Contention

Everybody’s code

Knowing shared-state concurrency != confidence

23

Keep those cores busy! Cores aren’t getting faster Default thread stack size on AMD64 = 1MB! Context switching hurts throughput

25

Actors

Shakespeare Programming Language

Romeo, a young man with a remarkable patience.

Juliet, a likewise young woman of remarkable grace.

Scene II: The praising of Juliet.

[Enter Juliet]

Romeo:

Thou art as sweet as the sum of the sum

of Hamlet and his horse and his black

cat! Speak thy mind!

[Exit Juliet]

Excerpt from "Hello World" in SPL

Actors have nothing to do with the Shakespeare Programming Language

• Each actor has a message queue

• Actors accept and choose what to do with messages

• Lightweight & asynchronous

Actors

• Each actor has a message queue

• Actors accept and choose what to do with messages

• Lightweight & asynchronous

Actors

• Each actor has a message queue

• Actors accept and choose what to do with messages

• Lightweight & asynchronous

Actors

• Each actor has a message queue

• Actors accept and choose what to do with messages

• Lightweight & asynchronous

Actors

• Each actor has a message queue

• Actors accept and choose what to do with messages

• Lightweight & asynchronous

Actors

• Each actor has a message queue

• Actors accept and choose what to do with messages

• Lightweight & asynchronous

Actors

• Actors tend to remain bound to a single thread

• Actors rarely block, thread can remain active for a long duration

• Minimizes context switches – throughput

• Akka actors occupy 650 bytes

Actors

4 cores ~4 threads

X millions

Actors

• Several actor implementations for Scala – Akka is considered the fastest

Benchmark

// Java

public class GreetingActor extends UntypedActor {

private int counter = 0;

public void onReceive(Object message) throws Exception {

counter++;

// 1) Hello, Juliet log.info( counter + ") Hello, " + message);

}

}

Akka Actors

// Scala

class GreetingActor extends Actor {

private var counter = 0

def receive = {

case message => {

counter += 1

// 1) Hello, Juliet log.info( counter + ") Hello, " + message)

} } }

// Java

ActorRef Romeo =

actorOf(GreetingActor.class).start();

greetingActor.sendOneWay("Juliet");

// 1) Hello, Juliet

Akka Actors

// Scala

val greetingActor =

actorOf[GreetingActor].start

greetingActor ! "Juliet“

// 1) Hello, Juliet

Akka Actors

• Once instantiated, actors can be retrieved by id or uuid

• uuid - generated by runtime, immutable, unique

• id - user assigned, by default that's the class name

class Romeo extend GreetingActor {

self.id = "Romeo"

}

actorOf[Romeo].start

val romeo = actorsFor("Romeo").head

romeo ! "Juliet"

Message Passing

• Let's build a bank with one actor per account, We’ll be able to :

• Check our balance

• Deposit money

• Withdraw money, but only if we have it (balance remains >= 0)

• We'll start by defining immutable message types:

case class CheckBalance()

case class Deposit(amount: Int)

case class Withdraw(amount: Int)

Message Passing

class BankAccount(private var balance: Int = 0) extends Actor {

def receive = {

// …

case CheckBalance =>

self.reply_?(balance)

// …

} }

Message Passing

class BankAccount(private var balance: Int = 0) extends Actor {

def receive = {

// …

case Deposit(amount) =>

balance += amount

// …

} }

Message Passing

class BankAccount(private var balance: Int = 0) extends Actor {

def receive = {

// …

case Withdraw(amount) =>

balance = (balance - amount) ensuring (_ >= 0)

// …

} }

Message Passing

• Now let’s make a deposit:

val account = actorOf[BankAccount].start

account ! Deposit(100)

• But how do we check the account balance?

Message Passing

• actorRef ! message - fire and forget

• actorRef !! message - send and block (optional timeout) for response

• actorRef !!! message - send, reply using future

...Jones Boner promised to stop at "!!!"

Java equivalents:

• ! = sendOneWay

• !! = sendRequestReply

• !!! = sendRequestReplyFuture

Bang! Bang Bang!

val balance = (account !! CheckBalance) match {

// do stuff with result

}

// or:

val balanceFuture = account !!! CheckBalance // does not block

// ... go do some other stuff ... later:

val balance = balanceFuture.await.result match {

// do stuff with result

}

Getting Account Balance

Fault Tolerance

Too Big to Fail

51

Jenga Architecture

52

The harder they fall

53

54

Self Healing,

Graceful Recovery

55

Supervisors: Kind of actors

Fault Handling Strategy: One for One or All for One

Lifecycle: Permanent or Temporary

Supervision Hierarchies

Fault Handling Strategy:

One For One

One for One

All for One, Permanent

One for One, Temporary

Romeo,

Permanent

Juliet,

Permanent

Chat Room, Permanent

59

One for One

All for One, Permanent

One for One, Temporary

Romeo,

Permanent

Juliet,

Permanent

Chat Room, Permanent

60

One for One

All for One, Permanent

One for One, Temporary

Romeo,

Permanent

Juliet,

Permanent

Chat Room, Permanent

61

One for One

All for One, Permanent

One for One, Temporary

Romeo,

Permanent

Juliet,

Permanent

Chat Room, Permanent

62

One for One

All for One, Permanent

One for One, Temporary

Romeo,

Permanent

Juliet,

Permanent

Chat Room, Permanent

63

Fault Handling Strategy:

All For One

All for One

All For One, Permanent

One for One, Permanent

Romeo,

Temporary

Juliet,

Temporary

Chat Room, Permanent

67

All for One

All For One, Permanent

One for One, Permanent

Romeo,

Temporary

Juliet,

Temporary

Chat Room, Permanent

68

All for One

All For One, Permanent

One for One, Permanent

Romeo,

Temporary

Juliet,

Temporary

Chat Room, Permanent

69

All for One

All For One, Permanent

One for One, Permanent

Romeo,

Temporary

Juliet,

Temporary

Chat Room, Permanent

70

All for One

All For One, Permanent

One for One, Permanent

Romeo,

Temporary

Juliet,

Temporary

Chat Room, Permanent

71

All for One

All For One, Permanent

One for One, Permanent

Romeo,

Temporary

Juliet,

Temporary

Chat Room, Permanent

72

val supervisor = Supervisor(SupervisorConfig(

AllForOneStrategy(

List(classOf[Exception]), 3, 1000),

List(Supervise(actorOf[ChatServer], Permanent),

Supervise(actorOf[ChatServer], Permanent,

RemoteAddess("host1", 9000))

)

))

Supervision, Remoting & HA

• You can’t have a highly available system on a single computer

• Luckily Akka supports near-seamless remote actors

High Availability

• Server managed remote actors:

// on host1

RemoteNode.start("host1", 9000)

// register an actor

RemoteNode.register(“romeo", actorOf[GreetingActor])

// on host2

val romeo = RemoteClient.actorFor(“romeo", "host1", 9000)

romero ! “juliet"

• RemoteClient handles the connection lifecycle for us

• Clients can also manage server actors, but enabling this might pose a security risk

High Availability

Actor model – a life choice?

High scalability Assumes state travels

Hgh availability along the message

Fast flow

Etc… Hostile towards shared state.

Minds not easily rewired for this!

Brain Transplant

Software Transactional Memory

Persistent Data Structures

Rich Hickey (Clojure)

• Share common immutable structure and data

• Copy-on-write semantics:

• When “modified”, minimal changes to structure are made to accommodate new data

Persistent Data Structures

© Rich Hickey 2009

val prices = TransactionalMap[String, Double]

atomic { prices += ("hamburger" -> 20.0) }

How many people seated in the audience?

If I started counting, by the time I finished…

1, 2, 3, 4, 5, …

…the room would be empty

Jonas Boner

Transactors

class BankAccount extends Transactor {

private val balanceRef = Ref(0)

def atomically = { // ...

case CheckBalance => self reply_? balance.get

// ...

}

}

Transactors

class BankAccount extends Transactor {

private val balanceRef = Ref(0)

def atomically = { // ...

case Deposit(amount) =>

balance alter (_ + amount) // ...

}

}

Transactors

class BankAccount extends Transactor {

private val balanceRef = Ref(0)

def atomically = { // ...

case Withdraw(amount) =>

balance alter (_ - amount) ensuring (_.get >= 0) // ...

}

}

Performing a money transfer transactionally:

val tx = Coordinated()

val fromBalance = (from !! tx(CheckBalance())) match {

balance: Int => balance

}

if (fromBalance >= 50) {

from ! tx(Withdraw(50))

to ! tx(Deposit(50)) }

Transactors

Coordinated Transactions

class Bank extends Actor { private val accounts =

TransactionalVector[BankAccount]

def receive = { // ...

case tx @ Coordinated(Join) => {

tx atomic {

accounts += self.sender.get

}

} // ...

}

Coordinated Transactions

class Bank extends Actor {

private val accounts = TransactionalVector[BankAccount]

def receive = { // ...

case tx @ Coordinated(Sum) => {

val futures = for (account <- accounts) yield

account !!! tx(CheckBalance)

val allTheMoney = futures map (_.await.result) sum

self reply_? allTheMoney

} // ...

} …and then: println (myBank !! Coordinated(Sum))

Takeaways

94

Knowing shared-state concurrency != confidence

References

http://akkasource.org

96

http://scalablesolutions.se

97

http://alphacsp.com

Questions?

• In Java we will usually avoid “untyped” actors, and use POJOs instead:

interface BankAccount { Future<Integer> balance(); void deposit(int amount); void withdraw(int amount); } class BankAccountImpl extends TypedActor implements BankAccount {

// Almost a regular POJO public Future<Integer> balance() { return future(balance); }

// ... }

Typed Actors in Java