Concurrency in Java (Shooting yourself in the foot) n.

Concurrency in JavaConcurrency in Java

(Shooting yourself in the foot)n(Shooting yourself in the foot)n

AcknowledgmentAcknowledgment

This lecture is derived almost exclusively from

Java Concurrency in Practice by Goetz, et. al.http://

www.javaconcurrencyinpractice.com

Additional notes from Dr. Dan Wallach

This lecture is derived almost exclusively from

Java Concurrency in Practice by Goetz, et. al.http://

www.javaconcurrencyinpractice.com

Additional notes from Dr. Dan Wallach

OutlineOutline

BackgroundBasic Thread SafetyConcurrent Object

ManagementConcurrent Library

ComponentsTask Execution and

Shutdown



ComponentsTask Execution and

Shutdown

Next TimeNext Time

Thread PoolsGUI ApplicationsSafety and PerformanceDocumentation and TestingAdv. Topics

Explicit LocksCustom SynchronizersNonblocking Synchronization

Thread PoolsGUI ApplicationsSafety and PerformanceDocumentation and TestingAdv. Topics

Explicit LocksCustom SynchronizersNonblocking Synchronization

BackgroundBackground

Why concurrency?Resource utilization - why

wait?Fairness - why wait?Convenience - why wait?

First concurrency: processes

Later: threads

Why concurrency?Resource utilization - why

wait?Fairness - why wait?Convenience - why wait?

First concurrency: processes

Later: threads

Benefits of ThreadsBenefits of Threads

Responsiveness (esp. GUIs)

Exploiting multi-processorsSimplicity of modelingSimplified handling of

asynchronous events

Responsiveness (esp. GUIs)

Exploiting multi-processorsSimplicity of modelingSimplified handling of

asynchronous events

Risks of ThreadsRisks of Threads

Java’s “built-in” threads means that concurrency is NOT an advanced topic

Safety hazards (correctness)Liveness hazards (progress)Performance hazards

(happiness)

Java’s “built-in” threads means that concurrency is NOT an advanced topic

Safety hazards (correctness)Liveness hazards (progress)Performance hazards

(happiness)

Threads are EverywhereThreads are Everywhere

Frameworks use threadsTimerServlets and JSPsRMISwing and AWT

You will use this one!

Frameworks use threadsTimerServlets and JSPsRMISwing and AWT

You will use this one!

Starting a Java Thread

Starting a Java Thread

public class HelloRunnable implements Runnable { public void run() { System.out.println("Hello from a thread!"); } public static void main(String args[]) { (new Thread(new HelloRunnable())).start(); }}public class HelloThread extends Thread { public void run() { System.out.println("Hello from a thread!"); } public static void main(String args[]) { (new HelloThread()).start(); }}

OutlineOutline



ComponentsTask Execution and Shutdown




SynchronizationSynchronization

Given a mutable variable vIf multiple threads access vAND if one can modify vALL must synchronize accessThis includes read access

Never attempt to ignore thisIf sync is broken, so is the

code (even if it passes tests)

Given a mutable variable vIf multiple threads access vAND if one can modify vALL must synchronize accessThis includes read access

Never attempt to ignore thisIf sync is broken, so is the

code (even if it passes tests)

Concurrent CorrectnessConcurrent Correctness

Fixing a broken “shared” varDon’t Share! OrMake the var immutable! OrSynchronize access

Better yet, design it rightEncapsulationImmutabilityClear specifications of

invariants

Fixing a broken “shared” varDon’t Share! OrMake the var immutable! OrSynchronize access

Better yet, design it rightEncapsulationImmutabilityClear specifications of

invariants

What is “Thread-Safe”?

What is “Thread-Safe”?

Definitions are vague and varyProblem: what is correctness?Thread Safe:

“Correct” in multi-threaded envOR, no more broken in a mult-

threaded environment then in a single-threaded one

Thread-safe classes encapsulate all synchronization

Definitions are vague and varyProblem: what is correctness?Thread Safe:

“Correct” in multi-threaded envOR, no more broken in a mult-

threaded environment then in a single-threaded one

Thread-safe classes encapsulate all synchronization

Example: Unsafe Counter

Example: Unsafe Counter

@NoThreadSafepublic class UnsafeSequence{ private int value; public int getNext() { return value++; }}

@NoThreadSafepublic class UnsafeSequence{ private int value; public int getNext() { return value++; }}

What’s Wrong with That?

What’s Wrong with That?

Invariant:getNext must return a

sequenceUnlucky execution timing:

Invariant:getNext must return a

sequenceUnlucky execution timing:

Value->9 9+1->10 Value=10

Value->9 9+1->10 Value=10

Race ConditionsRace Conditions

The problem is a race conditionDef: r.c. occurs when the

correctness of a computation depends on the relative timing of multiple threads by the runtime.

Often confused for data race

The problem is a race conditionDef: r.c. occurs when the

correctness of a computation depends on the relative timing of multiple threads by the runtime.

Often confused for data race

Achieving SafetyAchieving Safety

Go stateless!Use atomic operationsUse locking

Go stateless!Use atomic operationsUse locking

Using Atomics!(It’s not a treaty violation)

Using Atomics!(It’s not a treaty violation)

@ThreadSafepublic class UnsafeSequence{ private final AtomicLong value = new

AtomicLong(0); public long getNext() { return value.incrementAndGet(); }}

@ThreadSafepublic class UnsafeSequence{ private final AtomicLong value = new

AtomicLong(0); public long getNext() { return value.incrementAndGet(); }}

LockingLocking

Solved one mutable variable by making the var atomic

What if we have 2? Can we just make them both atomic?

NOT if they are dependentWe have to lock any

combined operations

Solved one mutable variable by making the var atomic

What if we have 2? Can we just make them both atomic?

NOT if they are dependentWe have to lock any

combined operations

ExampleExample

Suppose we have a dictionary that stores the last (key,value) pair

Setting the (k,v) pair must be locked. It is not enough to use an atomic var for k, and another one for v

Suppose we have a dictionary that stores the last (key,value) pair

Setting the (k,v) pair must be locked. It is not enough to use an atomic var for k, and another one for v

Intrinsic LocksIntrinsic Locks

Java provides built-in locksEach object is a lock, so is

each classMarked by synchronizedEnforces

AtomicityMemory visibility (more later)

Java provides built-in locksEach object is a lock, so is

each classMarked by synchronizedEnforces

AtomicityMemory visibility (more later)

ReentrancyReentrancy

Requesting a lock held by another causes a block

Intrinsic locks are reentrant

A thread that owns a lock, that requests the same lock, will succeed (good for inheritance!)

Requesting a lock held by another causes a block

Intrinsic locks are reentrant

A thread that owns a lock, that requests the same lock, will succeed (good for inheritance!)

Example LockingExample Locking

@ThreadSafePublic class LockSafe<K,V> { @GuardedBy(“this”) private K key; @GuardedBy(“this”) private V value;

public synchronized setKvPair(K k, V v) { this.key = k; this.value = v; }}

@ThreadSafePublic class LockSafe<K,V> { @GuardedBy(“this”) private K key; @GuardedBy(“this”) private V value;

public synchronized setKvPair(K k, V v) { this.key = k; this.value = v; }}

Locks: “Guarded By”Locks: “Guarded By”

“For each mutable state variable that may be accessed by more than one thread, all accesses to that variable must be performed with the same lock held. In this case, we say that the variable is guarded by that lock” (JCP 28)

“For each mutable state variable that may be accessed by more than one thread, all accesses to that variable must be performed with the same lock held. In this case, we say that the variable is guarded by that lock” (JCP 28)

Locks: Documentation

Locks: Documentation

“Every shared, mutable variable should be guarded by exactly one lock. Make it clear to maintainers which lock that is” - (JCP p28)

“Every shared, mutable variable should be guarded by exactly one lock. Make it clear to maintainers which lock that is” - (JCP p28)

Locks: Across Invariants

Locks: Across Invariants

“For every invariant that involves more that one variable, all the variables involved in that invariant must be guarded by the same lock” - (JCP p29)

“For every invariant that involves more that one variable, all the variables involved in that invariant must be guarded by the same lock” - (JCP p29)

Critical ConceptCritical Concept

Any mutable state that can be concurrently accessed, must be synchronized EVERYWHERE else in the program!

EXAMPLE: TimerTask (JCP p29)

Any mutable state that can be concurrently accessed, must be synchronized EVERYWHERE else in the program!

EXAMPLE: TimerTask (JCP p29)

Why Not Lock Everything?

Why Not Lock Everything?

Even if every method were synchronized, it doesn’t solve actions that combine those methods! (example: vector)

Also, possible liveness and/or performance problems

Poor concurrency example (JCP p30)

Even if every method were synchronized, it doesn’t solve actions that combine those methods! (example: vector)

Also, possible liveness and/or performance problems

Poor concurrency example (JCP p30)

Lock AdviceLock Advice

“Avoid holding locks during lengthy computations or operations at risk of not completing quickly such as network or console I/O” - (JCP p32)

“Avoid holding locks during lengthy computations or operations at risk of not completing quickly such as network or console I/O” - (JCP p32)

More SyntaxMore Syntax

Other use of synchronized

public somemethod() { synchronized(this) { // do stuff }}

Other use of synchronized

public somemethod() { synchronized(this) { // do stuff }}

OutlineOutline

BackgroundBasic Thread Safety

Concurrent Object Management

Concurrent Library ComponentsTask Execution and Shutdown

BackgroundBasic Thread Safety

Concurrent Object Management

Concurrent Library ComponentsTask Execution and Shutdown

Sharing ObjectsSharing Objects

Memory VisibilityShared object modification

should be visible to other threads

Without synchronization, this may not happen

Memory VisibilityShared object modification

should be visible to other threads

Without synchronization, this may not happen

More about VisibilityMore about Visibility

“In general, there is no guarantee that the reading thread will see a value written by another thread on a timely basis, or even at all” - (JCP p33)

“In general, there is no guarantee that the reading thread will see a value written by another thread on a timely basis, or even at all” - (JCP p33)

Public class NoVisibility { private static boolean ready; private static int number;

private static class ReaderThread extends Thread { public void run() { while (!ready) Thread.yield() System.out.println(number) } }

public static void main(String[] args) { new ReaderThread().start(); number = 42; ready = true; }}

NoVisibility :(NoVisibility :(

NoVisibility could loop forever!

NoVisibility could write 0 through reordering

Other badness and sadness

Don’t do this!

NoVisibility could loop forever!

NoVisibility could write 0 through reordering

Other badness and sadness

Don’t do this!

Synchronization Helps

Synchronization Helps

NoVisibility demonstrates stale data

Synchronization must be used on all shared variables, even just for reads

Enforces memory visibility

NoVisibility demonstrates stale data

Synchronization must be used on all shared variables, even just for reads

Enforces memory visibility

A Word about 64bitA Word about 64bit

When a thread reads a var w/o synchronization, at worst it is stale, but not random

Out-of-thin-air safetyOne exception: 64 bit

numbers (double and long)Read/write can be 2 32 bit

operations!

When a thread reads a var w/o synchronization, at worst it is stale, but not random

Out-of-thin-air safetyOne exception: 64 bit

numbers (double and long)Read/write can be 2 32 bit

operations!

Volatile VariablesVolatile Variables

Can declare a java var volatileVolatile ensures visibility, but not

locking.Volatile use can be fragile. Many

times, you shouldn’t use itGood uses of volatile

Ensure visibility of their own stateThat of the object they refer toOr indicate that an important event

has occurred (e.g. shutdown)

Can declare a java var volatileVolatile ensures visibility, but not

locking.Volatile use can be fragile. Many

times, you shouldn’t use itGood uses of volatile

Ensure visibility of their own stateThat of the object they refer toOr indicate that an important event

has occurred (e.g. shutdown)

Use Volatile ONLY when

Use Volatile ONLY when

Writes to the var do not depend on its current value or are only written by one thread

The var does not participate with invariants with other state vars

Locking is not required for any other reason when the var is being accessed

Writes to the var do not depend on its current value or are only written by one thread

The var does not participate with invariants with other state vars

Locking is not required for any other reason when the var is being accessed

Publication and Escape

Publication and Escape

Publishing an object - makes it available outside current scope

Many times, objects should not be published at allBreaks encapsulationNot fully constructed (!)

Object published when it should not have been is said to have escaped!

Publishing an object - makes it available outside current scope

Many times, objects should not be published at allBreaks encapsulationNot fully constructed (!)

Object published when it should not have been is said to have escaped!

Publication MethodsPublication Methods

Public static fieldsChained publication

Special Case: published inner class

Passing to an alien method

Public static fieldsChained publication

Special Case: published inner class

Passing to an alien method

How Escape Happens

How Escape Happens

All linked to poor designSemantic escapes:

Return ref instead of copyInadvertent chained publication

Syntactic escapes:Escaped this during construction

“Object not properly constructed”

All linked to poor designSemantic escapes:

Return ref instead of copyInadvertent chained publication

Syntactic escapes:Escaped this during construction

“Object not properly constructed”

This Escape ExampleThis Escape Example

Public class ThisEscape { // JCP p41 public ThisEscape(Eventsource source) { source.registerListener( new EventListener() { public void onEvent(Event e) { doSomething(e); } }); }}

Preventing EscapePreventing Escape

Thread ConfinementImmutabilitySafe Publication

Thread ConfinementImmutabilitySafe Publication

Thread ConfinementThread Confinement

Keep mutable vars confined to a single thread

Ad-hoc = enforced by impl.Stack Confinement

Local varsViolated by publication

ThreadLocalPer thread value holding

object

Keep mutable vars confined to a single thread

Ad-hoc = enforced by impl.Stack Confinement

Local varsViolated by publication

ThreadLocalPer thread value holding

object

Immutable ObjectsImmutable Objects

Always thread safeCan’t escape after

constructionDefinition

It’s state cannot be modified after construction

All fields are final*Properly constructed

Can have mutable variables

Always thread safeCan’t escape after

constructionDefinition

It’s state cannot be modified after construction

All fields are final*Properly constructed

Can have mutable variables

Final FieldsFinal Fields

Can’t be modifiedAND, have special semantics

in the Java Memory model (initialization safety)

Make all fields final by default“Mostly Immutable” is better

than mutable

Can’t be modifiedAND, have special semantics

in the Java Memory model (initialization safety)

Make all fields final by default“Mostly Immutable” is better

than mutable

Using Volatile Again!Using Volatile Again!

Volatile can be used to publish immutable objects

Still not locked, but can be safe depending on semantics

Example - JCP p49-50

Volatile can be used to publish immutable objects

Still not locked, but can be safe depending on semantics

Example - JCP p49-50

Improper PublicationImproper Publication

If synchronization is not used to publish a mutable object, the object is not properly published

Without proper publication, there are serious problems with stale data

If synchronization is not used to publish a mutable object, the object is not properly published

Without proper publication, there are serious problems with stale data

Safe PublicationSafe Publication

Object reference and object state must become visible at the same time!

Idioms:Initializing from static initializerStoring ref in volatile or

AtomicReferenceStoring ref in final field of

properly constructed objectStoring ref in a lock-guarded field

Object reference and object state must become visible at the same time!

Idioms:Initializing from static initializerStoring ref in volatile or

AtomicReferenceStoring ref in final field of

properly constructed objectStoring ref in a lock-guarded field

Good News!Good News!

Java Collections safe publication examples:Key or value in Hashtable,

synchronizedMap, or ConcurrentMapInsert in Vector,

CopyOnWriteArrayList, CopyOnWriteArraySet, synchronizedList, or synchronizedSet

Insert in BlockingQueue or ConcurrentLinkedQueue

Java Collections safe publication examples:Key or value in Hashtable,

synchronizedMap, or ConcurrentMapInsert in Vector,

CopyOnWriteArrayList, CopyOnWriteArraySet, synchronizedList, or synchronizedSet

Insert in BlockingQueue or ConcurrentLinkedQueue

More Good NewsMore Good News

Effectively Immutable Objects

Objects that are used as if they were immutable

Example: Date

Effectively Immutable Objects

Objects that are used as if they were immutable

Example: Date

Modification Vs Visibility

Modification Vs Visibility

Safe publication ensures visibility

Synchronization is required if the object can be modified post-publication

Safe publication ensures visibility

Synchronization is required if the object can be modified post-publication

Rules of Engagement

Rules of Engagement

Thread-Confined: no sharingShared read-only:

(effectively) immutable objectsShared thread-safe: internal

synchronizationGuarded: only access with the

associated guard

Thread-Confined: no sharingShared read-only:

(effectively) immutable objectsShared thread-safe: internal

synchronizationGuarded: only access with the

associated guard

Designing Thread-Safe Classes

Designing Thread-Safe Classes

Identify the variables that form the object’s state

Identify the invariants that contain the state variables

Establish a policy for managing concurrent access to the object’s state

Identify the variables that form the object’s state

Identify the invariants that contain the state variables

Establish a policy for managing concurrent access to the object’s state

Java Monitor PatternJava Monitor Pattern

Principle of instance confinement

Encapsulate all mutable state and guard it with intrinsic lock

Good coarse-grained locking

Principle of instance confinement

Encapsulate all mutable state and guard it with intrinsic lock

Good coarse-grained locking

Delegating Thread Safety

Delegating Thread Safety

Safety can often be delgated to an internally-used thread-safe object

Collections are especially useful

Only works for 1 state var, or multiple independent vars

Safety can often be delgated to an internally-used thread-safe object

Collections are especially useful

Only works for 1 state var, or multiple independent vars

Extending Thread-safe Classes

Extending Thread-safe Classes

BE CAREFUL! Extending means that the synchronization policy is distributed over multiple separately maintained classes!

Inheritance requires the base class to publish enough state

Wrappers require that the same lock is used

Composition is often less fragile

BE CAREFUL! Extending means that the synchronization policy is distributed over multiple separately maintained classes!

Inheritance requires the base class to publish enough state

Wrappers require that the same lock is used

Composition is often less fragile

Extension Examples:Extension Examples:

Inheritance (JCP p72)Wrapping (JCP p72-73)Composition (JCP p74)

Inheritance (JCP p72)Wrapping (JCP p72-73)Composition (JCP p74)

OutlineOutline

BackgroundBasic Thread SafetyConcurrent Object ManagementConcurrent Library


BackgroundBasic Thread SafetyConcurrent Object ManagementConcurrent Library


Synchronized Collections

Synchronized Collections

Are thread safe, butComposite actions require

additional locking for semantic correctness

Careless use of locking leads to poor performance (think iteration)

Are thread safe, butComposite actions require

additional locking for semantic correctness

Careless use of locking leads to poor performance (think iteration)

IteratorsIterators

The synchronized collections provide iterators

These iterators are fail-fastThrows

ConcurrentModificationException

The synchronized collections provide iterators

These iterators are fail-fastThrows

ConcurrentModificationException

Concurrent CollectionsConcurrent Collections

Designed for concurrent access

Improved performance with a few minor trade-offs:

Your assignment: go look up the docs on ConcurrentHashMap and CopyOnWriteArrayList and try them out!

Designed for concurrent access

Improved performance with a few minor trade-offs:

Your assignment: go look up the docs on ConcurrentHashMap and CopyOnWriteArrayList and try them out!

Blocking QueuesBlocking Queues

Support the producer-consumer pattern

Can be bounded or unbounded

Serial thread confinementTry it out!

Support the producer-consumer pattern

Can be bounded or unbounded

Serial thread confinementTry it out!

BlockingBlocking

Threads may blockWait for I/OWait for LockWait for Thread.sleepWait for computation

put and get from BlockingQueue block

Threads may blockWait for I/OWait for LockWait for Thread.sleepWait for computation

put and get from BlockingQueue block

InterruptionInterruption

interrupt() methodRequest a blocking thread

stopA blocking method should

throw an InterruptedExceptionIf you catch this

Propagate the exception ORRestore the interrupt

interrupt() methodRequest a blocking thread

stopA blocking method should

throw an InterruptedExceptionIf you catch this

Propagate the exception ORRestore the interrupt

Example:Example:

Try{someBlockingMethod();

} catch (InterruptedException) {

Thread.currentThread().interrupt();

}

Try{someBlockingMethod();

} catch (InterruptedException) {

Thread.currentThread().interrupt();

}

SynchronizersSynchronizers

Latches: one-time triggere.g. CountDownLatch

FutureTask: long-running resultHas a get() methodReturns value if doneBlocks until it is done if not

Latches: one-time triggere.g. CountDownLatch

FutureTask: long-running resultHas a get() methodReturns value if doneBlocks until it is done if not

More SynchronizersMore Synchronizers

Semaphore: virtual permitsNow a library class!

Barriers: wait for enough threadsUseful for threaded steppingExchanger 2-party barrier

Semaphore: virtual permitsNow a library class!

Barriers: wait for enough threadsUseful for threaded steppingExchanger 2-party barrier

OutlineOutline

BackgroundBasic Thread SafetyConcurrent Object ManagementConcurrent Library ComponentsTask Execution and

Shutdown

BackgroundBasic Thread SafetyConcurrent Object ManagementConcurrent Library ComponentsTask Execution and

Shutdown

Advanced ExecutionAdvanced Execution

The Executor FrameworkDecouples task submission

and task executionSimple interface:

void execute(Runnable command)

Easiest way to implement a producer-consumer design in an application

The Executor FrameworkDecouples task submission

and task executionSimple interface:

void execute(Runnable command)

Easiest way to implement a producer-consumer design in an application

Execution PoliciesExecution Policies

AnswersIn what thread will tasks be

runWhat order should they be runHow many concurrently?How many queued?“Victim” selectionSetup and Teardown

AnswersIn what thread will tasks be

runWhat order should they be runHow many concurrently?How many queued?“Victim” selectionSetup and Teardown

Policy ExamplesPolicy Examples

Single-thread (JCP p119)1-thread-per-client (JCP

p118)Thread Pools: some built in

newFixedThreadPoolnewCachedThreadPoolnewSingleThreadExecutornewScheduleThreadPool

Single-thread (JCP p119)1-thread-per-client (JCP

p118)Thread Pools: some built in

newFixedThreadPoolnewCachedThreadPoolnewSingleThreadExecutornewScheduleThreadPool

Cancellation & Shutdown

Cancellation & Shutdown

Reasons for cancellationUser-requestedTime-limitedEventsErrorsShutdown

Reasons for cancellationUser-requestedTime-limitedEventsErrorsShutdown

Cancellation PolicyCancellation Policy

“how” “when” and “what” of cancellation

Not enforced by Java, but generally, use interruption for cancellation

E.g. have a cancel() method call the interrupt

“how” “when” and “what” of cancellation

Not enforced by Java, but generally, use interruption for cancellation

E.g. have a cancel() method call the interrupt

Interruption PolicyInterruption Policy

Determines how a thread interprets an interruption request

Tasks are “guests” in a threadPreserve interruption status

Only call interrupt, when you know the interruption policy

Determines how a thread interprets an interruption request

Tasks are “guests” in a threadPreserve interruption status

Only call interrupt, when you know the interruption policy

Detecting InterruptsDetecting Interrupts

Polling the thread’s interrupted status

Detecting an InterruptedException

Polling the thread’s interrupted status

Detecting an InterruptedException

Non-interruptible Blocks

Non-interruptible Blocks

Synchronous socket I/O:Close the socket

Asynchronous I/O with Selector:Call wakeup

Lock Acquisition:Use the explicit Lock classHas a lockInterruptibly() method

Synchronous socket I/O:Close the socket

Asynchronous I/O with Selector:Call wakeup

Lock Acquisition:Use the explicit Lock classHas a lockInterruptibly() method

Stopping a Thread-Based Service

Stopping a Thread-Based Service

ExceutorService extends Executor

Provides “Lifecycle” optionsshutdown()awaitTermination(timeout, unit)

Any thread based service should provide similar methods

ExceutorService extends Executor

Provides “Lifecycle” optionsshutdown()awaitTermination(timeout, unit)

Any thread based service should provide similar methods

Waiting for ThreadsWaiting for Threads

In the “raw” case, use join()A number of Java library

classes have advanced “waits” like ExecutionService’s awaitTermination()

Obviously, if you implement your own ExecutionService, you’ll have to use the raw stuff

In the “raw” case, use join()A number of Java library

classes have advanced “waits” like ExecutionService’s awaitTermination()

Obviously, if you implement your own ExecutionService, you’ll have to use the raw stuff

Concurrency in Java (Shooting yourself in the foot) n.

Documents

Transcript of Concurrency in Java (Shooting yourself in the foot) n.