Chicago, October 19 - 22, 2010

Concurrent Programming and Distributed Applications

Mark Fisher, Dave Syer - SpringSource

Goal

● Demystify concurrent and distributed programming.● Identify and help avoid the pitfalls● Show how Spring makes it easier to write multi-

threaded and multi-process applications

Agenda

• Concurrency• Asynchronous execution• Tasks, schedules and triggers• Events, Messaging and intra-process communication• Distributed systems

Concurrency

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

Why? - Performance - Responsiveness - Scalability

Where? - Event-driven Architecture - Scheduling Tasks

When threads were more esoteric,concurrency was an "advanced" topic;now, mainstream developers must beaware of thread-safety issues.

--Brian Goetz

Thread Safe?

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

public class Counter {

private boolean active = false;

public boolean isActive() { return active; }

public void setActive(boolean active) { this.active = active; }}

Thread Safe?

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

public class Counter {

private volatile int count = 0;

public int increment() { return count++; }}

Thread Safe?

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

public class Service {

private volatile Resource resource;

public void process(String data) { if (this.resource == null) { this.resource = new Resource(); } this.resource.process(data); }}

Thread Safety

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

What?Properly managing concurrent access to mutable state.

How?a) avoid mutabilityb) don't sharec) synchronize

Immutable?public class StringHolder {

private final int id; private final String value;

public StringHolder(int id, String value) { this.id = id; this.value = value; }

public void display() { Sysem.out.println(id + ": " + this.value); }}

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

Immutable?public class StringList {

private final int id; private final List<String> strings;

public StringList(int id, List<String> strings) { this.id = id; this.strings = strings; }

public void display() { Sysem.out.println(id + ": " + this.strings); }}

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

Immutable?public class MapHolder {

private final Map<Integer, StringValue> map;

public MapHolder(Map<Integer, StringValue> map) { this.map = new HashMap<Integer, StringValue>(map); }

public void display() { System.out.println(this.map); }}

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

Immutable?

private final Map<Integer, StringValue> map;

public MapHolder(Map<Integer, StringValue> map) { this.map = new HashMap<Integer, StringValue>(map); }

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

public static class StringValue { private String string; public StringValue(String string) { this.string = string; }

public void setString(String string) { this.string = string; }

}

public static class StringValue { private String string; public StringValue(String string) { this.string = string; }

public void setString(String string) { this.string = string; }

}

Immutable?

private final Map<Integer, StringValue> map;

public MapHolder(Map<Integer, StringValue> map) { this.map = new HashMap<Integer, StringValue>(map); }

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

public static class StringValue { private final String string; public StringValue(String string) { this.string = string; }

}

public static class StringValue { private final String string; public StringValue(String string) { this.string = string; }

}

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

Demo

Thread Safety: Confinement

When immutability is not an option, consider not sharing mutable state

• Stack Confinement– Method Parameters– Local Variables

• Thread Confinement– ThreadLocal– Custom Scopes in Spring

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

Demo

Synchronization

• The 'synchronized' keyword• Locks• wait/notify• Atomic variables

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

Demo

Lazy Initialization (recap)

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

public class Service {

private volatile Resource resource;

public void process(String data) { if (this.resource == null) { this.resource = new Resource(); } this.resource.process(data); }}

Dependency Injection and Singletons

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

public class Service {

private volatile Resource resource;

public void setResource(Resources resource) { this.resource = resource; }

public void process(String data) { this.resource.process(data); }}

Concurrency

• Background processing and performance optimization• Java Memory Model: final, volatile, synchronized• Locks: wait, notify, utility wrappers• Concurrent collections, immutable wrappers• Queue, BlockingQueue, DelayQueue• Deadlock, livelock, starvation• Immutability, stateless components, "hidden" state, thread

safety• Typical Spring application concerns• Stateful components in Spring

Executor and ExecutorService

JDK Functionality• Configurable thread pools• Execute Runnables asynchronously• Submit Callable<V> that returns Future<V>

Async Execution Considerations

• Blocking and timeouts• Cancellation• Interruption• Thread Pool Rejection Policies• Excessive thread creation• Context-switching overhead

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

Demo

Asynchronous Execution

• Futures, executors, completion service and thread pools• @Async• Lifecycle and SmartLifecycle, cf. InitializingBean• Gateways with Futures• Timeouts

Spring Support for Task Management

• TaskExecutor• TaskScheduler and Trigger• @Async• @Scheduled• File-Polling adapter in Spring Integration• Message listener containers

– JMS– AMQP– Spring Integration– GemFire

• Spring Batch

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

Demo

Events and Intraprocess Communication

• The observer pattern: application and framework design• ApplicationEvent and ApplicationListener• Messaging• SEDA• Ordering• Stateful Messaging patterns

Observer Pattern

Publisher

invoke(“input”)

confirmation

Multicaster

fire(“input”)

observe(“input”)

Observer

Event

ApplicationListener

// This is a listener

public class TransferListener implements

ApplicationListener<TransferEvent> {

public onApplicationEvent(TransferEvent event) {

this.auditLogger.log(event.getTransfer());

}

…

}

ApplicationEvent

java.util.Event

ApplicationEventPublisher

public class TransferService implements

ApplicationEventPublisherAware {

public setApplicationEventPublisher(

ApplicationEventPublisher publisher) {

this.publisher = publisher;

}

public void transfer(Transfer transfer) {

...

this.publisher.publish(new TransferEvent(transfer));

}

The Observer Pattern

Publisher

Multicaster

ObserverObserverObserverObserverApplicationListener

ApplicationContext

ApplicationEventPublisher

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

Demo

Messaging

• Decouple Producers and Consumers

• Observer Pattern but often with extra semantics– header and payload– publish-subscribe or point-to-point– acks and nacks and other protocol details– persistence and quality of service– once-only or at-least-once delivery

widgets

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

Demo

Staged Event Driven Architecture (SEDA)

Staged Event Driven Architecture (SEDA)

Staged Event Driven Architecture (SEDA)

Staged Event Driven Architecture (SEDA)

Dispatcher

Queue

Ordering

• Multi-threaded applications in general do not preserve order

• E.g. Messaging– Producer sends two widgets 1, 2– Consumer receives 2, 1

• To preserve order expect some overhead– Storage (stateful patterns)– Processing time (wait for out of order messages)

Stateful Messaging Patterns

• Scatter-Gather• Composed Message Processor• Claim Check

• Publish-Subscribe Channel• Recipient List Router• Splitter• Multi-valued Router

• Aggregator• Resequencer

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

Demo

Quality of Service, Transactions

Asynchronous hand off = potential lost messages

Quality of Service, Transactions

Transactional (receive, send) = no lost messages

Distributed Applications

• RPC, Spring Remoting• Messaging• Middleware: HTTP, JDBC, JMS, AMQP• Latency• QoS, transactions, durability and guaranteed delivery• Distributed data: partitioning and eventual consistency• Gemfire• Spring Integration adapters

Distributed Application or System

• Application:– Multiple JVM process nodes– Same binaries– Same release schedule

• System:– Multiple JVM process nodes– Different binaries– Different release schedule per node

• Blurred boundary if Application has to have a rolling upgrade with no downtime

Patterns of Distributed Transactions

• Full XA with 2PC• XA with the 1PC Optimisation• XA and the Last Resource Gambit• Shared Transaction Resource• Best Efforts 1PC• Non-transactional Access• Wing and a Prayer (anti-pattern)

http://www.javaworld.com/javaworld/jw-01-2009/jw-01-spring-transactions.html

JTA + Spring

Spring Data + Oracle

Spring JMS

Remote Procedure Call (RPC)

• RPC is synchronous inter-process communication• Only evil if the system is not an Application• Transports:

– RMI– HTTP– JMS– JDBC– ...

• Spring Remoting helps strategise and configure: defers important architectural choices

• Spring Integration provides additional options

Messaging

• Messaging is asynchronous inter-process communication• Can use Messaging to implement RPC• Transports (some point-to-point only):

– RMI– HTTP– JMS– JDBC– AMQP– ...

• Spring Integration: helps strategise and configure: defers important architectural choices

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

Demo

Latency

• All distributed systems and applications suffer from latency– EU-US = 6000km, min network latency 6e6/3e8 = 20ms– Add marshal/unmarshal overhead (variable)– Transaction (min 50ms)

• Often much worse, e.g. >100ms even on local network• Request-reply patterns double the problem• Over-modularization: each inter-process hop is expensive• Abstractions (e.g. Spring) are dangerous• Measure and analyse

Distributed Application and Data

Nosql & Spring Data

• Key value stores– redis, gemfire– coherence– riak, voldemort, memcached

• Document stores– couchdb, mongodb

• Sparse table or column stores– cassandra, bigtable

• Graph or object stores– neo4j

• Distributed filesystem– hdf

SpringSource Gemfire

• Distributed data cache, datastore and compute grid• Java (so embeddable)• Low-level API is java.util.Map• Many high-level abstractions

– Transactions– Functions and node affinity– Events, continuous queries– Replication

• Spring Gemfire:http://git.springsource.org/spring-gemfire

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

Demo

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

Q&A

Source code for demos:

http://git.springsource.org/s2gx-2010/concurrent-programming-distributed-applications

Distributed Transaction: Sunny Day

1. Start messaging transaction2. Receive message3. Start database transaction4. Update database5. Commit database transaction6. Commit messaging transaction

Distributed Transaction: Rollback

1. Start messaging transaction2. Receive message3. Start database transaction4. Update database, fail!5. Rollback database transaction6. Rollback messaging transaction

Distributed Transaction: Partial Failure

1. Start messaging transaction2. Receive message3. Start database transaction4. Update database5. Commit database transaction6. Commit messaging transaction, fail!

Distributed Data: Shared Database

Application

Cache

Distributed Data: Cache Overflow

Cache

Distributed Cache: Database Meltdown

Distributed Data

Distributed Data