Threads

Written by Amir Kirsh, Dr. Yaron Kanza. Edited by Liron Blecher

Age

nda

• Threads Overview

• Creating threads in Java

• Synchronization

• wait() and notify()

• Notes

• Thread Pools

3

Threads Overview

– Threads allow the program to run tasks in parallel

– If two threads execute the same method, each will have its own copy of the local variables the methods uses

– In many cases threads need to be synchronized,

that is, be kept not to handle the same data in memory concurrently

– There are cases in which a thread needs to wait for another thread before proceeding

– Threads have access to the same memory, meaning, they can change the same objects

Age

nda

• Threads Overview

• Creating threads in Java

• Synchronization

• wait() and notify()

• Notes

• Thread Pools

5

Threads in Java

The operation we want to be threaded:

public class PrintNumbers { static public void printNumbers() {

for(int i=0; i<1000; i++) {System.out.println(

Thread.currentThread().getId() + ":" + i);}

}}

6

Threads in Java

Option 1 – extending class Thread:

public class Thread1 extends Thread {

@Overridepublic void run() {

System.out.println("Thread1 ThreadId: " + Thread.currentThread().getId());

// do our thingPrintNumbers.printNumbers();

}}

7

Threads in Java

Option 1 – extending class Thread (cont’):

static public void main(String[] args) {System.out.println("Main ThreadId: " +

Thread.currentThread().getId());for(int i=0; i<3; i++) {

new Thread1().start(); // don't call run! // (if you want a separate thread)

}printNumbers();

}

8

Threads in Java

Option 2 – implementing Runnable:

public class MyRunnable implements Runnable {

@Overridepublic void run() {

System.out.println("Thread2 ThreadId: " + Thread.currentThread().getId());

// do our thingPrintNumbers.printNumbers();

}}

9

Threads in Java

Option 2 – implementing Runnable (cont’):

static public void main(String[] args) {System.out.println("Main ThreadId: " +

Thread.currentThread().getId());for(int i=0; i<3; i++) {

new Thread(new MyRunnable()).start();// again, don't call run!// (if you want a separate thread)

}printNumbers();

}

10

Threads in Java

Option 3 – implementing Runnable as Anonymous:

static public void main(String[] args) {System.out.println("Main ThreadId: " +

Thread.currentThread().getId());new Thread(new Runnable() {

@Overridepublic void run() {

System.out.println("Thread3 ThreadId: " + Thread.currentThread().getId());

// do our thingprintNumbers();

}}).start(); // don't call run! ...printNumbers();

}

DEMO

examples.threads.intro

11

Age

nda

• Threads Overview

• Creating threads in Java

• Thread Scheduling

• Synchronization

• wait() and notify()

• Notes

• Thread Pools

13

Thread Scheduling

– Usually, threads run one at a time

– Unless several processors are being used

– Thread execution is merged in order to provide concurrency

I/O operation completes

start()

Currently executedthread

Ready queue

• Waiting for I/O operation to be completed

• Waiting to be notified• Sleeping• Waiting to enter a synchronized section

Newly createdthreads

Blocked queue

Thread Scheduling

14

Alive

New Dead

Running

Runnable

new PlusPrinter();

run() method returns

for (…) { … }

Blocked

Object.wait() Thread.sleep()

blocking IO call

waiting on a monitor

thread.start();

Don’t get confused with Runnable interface

Thread.getState()

Thread Scheduling

15

Thread scheduling is the mechanism used to determine how Runnable threads (in the ready queue) are allocated CPU time

Java scheduling is preemptive, priority based

Each thread has a priority - an integer number

Use thread.getPriority()/setPriority() to control priorities

In principle, the runtime system chooses the thread that has the highest priority

Priority has range 1-10

What is the priority of the main thread?

Thread Scheduling

16

If several threads have the same priority, an arbitrary one is chosen

Scheduling may violate the priority-based policy to avoid starvation of low-priority threads

Java's scheduling also uses time slicing, when it is supported by the operating system

Mainly used for sharing CPU among equal highest-priority threads

Thread Scheduling

17

A thread runs until one of the following occurs:

The thread dies (e.g., run() completes)

The thread becomes not Runnable (e.g., sleeps or waits for an IO operation to complete)

A higher-priority thread becomes Runnable

On systems that support time-slicing, its time allotment has expired

The thread yields (discussed later)

Thread Scheduling

18

Scheduling is OS Dependent!

Java maps Java threads to OS threads

In particular, Java relies on the operating system for

• Time slicing

• Priorities

Thus, scheduling differs from one system to another

Do not count on scheduling and priorities for algorithm correctness!

19

Relinquishing the CPU

A running thread can explicitly relinquish the CPU

for other threads to use

The static method Thread.yield() temporarily pauses

the currently executing thread and allows other

threads to execute

A thread can block for a while using the method

Thread.sleep(milliseconds)

20

Daemon Threads

There are two types of threads:

• daemon threads (like the garbage-collection thread)

• non-daemon threads (like the thread running main())

JVM will let all non-daemon threads complete (i.e., complete the execution of run())

When only daemon threads stay alive, they are killed and JVM exits

Controlled by thread.isDaemon() and thread.setDaemon()

21

Age

nda

• Threads Overview

• Creating threads in Java

• Thread Scheduling

• Synchronization

• wait() and notify()

• Notes

• Thread Pools

23

Synchronization

Synchronization of threads is needed for in order to control threads coordination, mainly in order to prevent simultaneous operations on data

For simple synchronization Java provides the synchronized keyword

For more sophisticated locking mechanisms, starting from Java 5, the package java.concurrent.locks provides additional locking options, see: http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/locks/package-summary.html

Monitors are key elements in Java's thread synchronization

Every object has a monitor

An object's monitor is used as a guardian that watches a block of code (called a critical section) and enables only one thread to enter that code

To enter a critical section, a thread must first acquire an ownership over the corresponding monitor

Synchronization

24

Only one thread can own a specific monitor

If a thread A tries to enter a block under a monitor and a different thread B has already entered that block, A will wait until B releases the monitor and (hopefully) that monitor will be passed to A

• Hence, monitors are related to as locks

When a thread leaves the critical section, the monitor is automatically released

Threads awaiting a monitor are blocked and queued

Synchronization

25

26

public class SynchronizedCounter {private int c = 0;public synchronized void increment() { c++; } public synchronized void decrement() { c--; } public synchronized int value() { return c; }

}

Synchronization

Example 1 – synchronizing methods:

The synchronized keyword on a method means that if this is already locked anywhere(on this method or elsewhere) by another thread,we need to wait till this is unlocked before entering the method

27

public void addName(String name) { synchronized(this) {

lastName = name;nameCount++;

}nameList.add(name);

}

Synchronization

Example 2 – synchronizing blocks:

When synchronizing a block, key for the locking should be supplied (usually would be this)The advantage of not synchronizing the entire method is efficiency

28

public class TwoCounters {private long c1 = 0, c2 = 0;private Object lock1 = new Object();private Object lock2 = new Object();public void inc1() {

synchronized(lock1) {c1++;

}}public void inc2() {

synchronized(lock2) {c2++;

}}

}

Synchronization

Example 3 – synchronizing using different locks:

You must be absolutely sure

that there is no tie between c1 and c2

DEMO

examples.threads.deadlock

29

30

public class Screen {private static Screen theScreen;private Screen(){…} // private c’torpublic static synchronized Screen getScreen() {

if(theScreen == null) {theScreen = new Screen();

}return theScreen;

}}

Synchronization

Example 4 – synchronizing static methods:

This is a Singleton example

It is not the mostefficient way to implement

Singleton in Java

31

Synchronization

Example 4 – synchronizing static methods …

Having a static method be synchronized means that ALL objects of this type are locked on the method and can get in one thread at a time.

The lock is the Class object representing this class.

The performance penalty might be sometimes too high – needs careful attention!

32

public class Screen {private static Screen theScreen = new Screen();private Screen(){…} // private c’torpublic static getScreen() {

return theScreen;}

}

Synchronization

Example 4’ – a better singleton:

No synchronization

Age

nda

• Threads Overview

• Creating threads in Java

• Thread Scheduling

• Synchronization

• wait() and notify()

• Notes

• Thread Pools

34

wait(), notify(), notifyAll()

wait() and notify() allows a thread towait for an event

A call to notifyAll() allows all threads that are on wait() with the same lock to be released

A call to notify() allows one arbitrary thread that is on a wait() with the same lock to be released

Read:(a) http://java.sun.com/docs/books/tutorial/essential/concurrency/guardmeth.html (b) http://java.sun.com/javase/6/docs/api/java/lang/Object.html#wait()

Instead of “busy wait” or

sleep loop!

wait(), notify(), notifyAll()

Suppose that an object has some monitor, but conditions disable it from completing the critical section

The wait/notify mechanism enables that object to release the monitor and wait until conditions are changed

35

wait()

The method Object.wait() requires the current thread to own the monitor of the object

When called, the current thread

• releases ownership on the object's monitor

• stops and waits until some other thread will wake it up and the monitor will be re-obtained

36

notify()

Like wait, requires the object to own the monitor

The method Object.notify() wakes up an arbitrary

thread that waits on the monitor of the object

Object.notifyAll() wakes all such threads

When a thread is waken up, it regularly waits for the monitor to be available (since it called Object.wait())

The thread calling notify should release the monitor for the waiting thread to continue (e.g

exit the synchronized scope)

37

38

public class Drop {

// Message sent from producer to consumerprivate String message;

// A flag, True if consumer should wait for// producer to send message, False if producer// should wait for consumer to retrieve messageprivate boolean empty = true;

...

wait(), notify(), notifyAll()

Example(from http://java.sun.com/docs/books/tutorial/essential/concurrency/example/Drop.java):

Flag must be used, never count only on the notify

39

public class Drop {...public synchronized String take() {

// Wait until message is availablewhile (empty) {

// we do nothing on InterruptedException// since the while condition is checked anyhowtry { wait(); } catch (InterruptedException e) {}

}// Toggle status and notify on the status changeempty = true;notifyAll();return message;

}...

}

wait(), notify(), notifyAll()

Example (cont’) Must be in synchronized context

40

public class Drop {...public synchronized void put(String message) {

// Wait until message has been retrievedwhile (!empty) {

// we do nothing on InterruptedException// since the while condition is checked anyhowtry { wait(); } catch (InterruptedException e) {}

}// Toggle status, store message and notify consumerempty = false;this.message = message;notifyAll();

}...

}

wait(), notify(), notifyAll()

Example (cont’) Must be in synchronized context

DEMO

examples.threads.producerconsumer

41

Age

nda

• Threads Overview

• Creating threads in Java

• Thread Scheduling

• Synchronization

• wait() and notify()

• Notes

• Thread Pools

Notes

Threads inherit their priority and daemon properties from their creating threads

The method thread.join() blocks and waits until the thread completes running

A thread can have a name for identification

Stopping a running thread was possible in old versions of Java, but it is now deprecated

Instead, interruption mechanisms should be used (thread.interrupt())

43

Notes

isInterupted() – can be used by a thread to check if another thread has been interrupted

isAlive() – can be used by a thread to check if another thread is alive

join() – allows one thread to wait for the completion of another

44

DEMO

examples.threads.poetthread

45

46

Notes

Some more reading material:

http://java.sun.com/docs/books/tutorial/essential/threads/index.html

http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Thread.html

Age

nda

• Threads Overview

• Creating threads in Java

• Thread Scheduling

• Synchronization

• wait() and notify()

• Notes

• Thread Pools

What is a Thread Pool?

A collection of threads that are created once (e.g. when a server starts)

That is, no need to create a new thread for every client request

Instead, the server uses an already prepared thread if there is a free one, or waits until there is a free thread

48

Why Using Thread Pools?

Thread pools improve resource utilization

The overhead of creating a new thread is significant

Thread pools enable applications to control and

bound their thread usage

Creating too many threads in one JVM can cause

the system to run out of memory and even crash

There is a need to limit the usage of system

resources such as connections to a database

49

Thread Pools in Servers

Thread pools are especially important in client-server applications

• The processing of each individual task is short-lived and the number of requests is large

• Servers should not spend more time and consume more system resources creating and destroying threads than processing actual user requests

When too many requests arrive, thread pools enable the server to force clients to wait until threads are available

50

The “Obvious” Implementation

There is a pool of threads

Each task asks for a thread when starting and returns the thread to the pool after finishing

When there are no available threads in the pool the thread that initiates the task waits till the pool is not empty

What is the problem here?

“Synchronized” model - the client waits until the server takes care of its request…

51

The “Obvious” Implementation is Problematic

When the pool is empty, the submitting thread has to wait for a thread to be available

• We usually want to avoid blocking that thread

• A server may want to perform some actions when too many requests arrive

Technically, Java threads that finished running cannot run again

52

Task Queue

Worker Threads

wait()

If Q is Empty

All the worker threads wait for tasks

A Possible SolutionEvery thread looks for tasks in the queue

53

Task Queue

Worker Threads

Task

The number of worker threads is fixed. When a task is inserted to the queue, notify is called

A Possible Solution

“A-synchronized” model: “Launch and forget”

54

Task Queue

Worker Threads

Task

The number of worker threads is fixed. When a task is inserted to the queue, notify is called

A Possible Solution

notify()

55

Task Queue

Worker Threads

A Possible Solution

The task is executed by the thread

56

Task Queue

Worker Threads

A Possible Solution

The task is executed by the thread

The remaining tasks are executed by the other threads

57

Task Queue

Worker Threads

A Possible Solution

When a task ends, the thread is released

While the Q is not empty, take the task from the Q and run it (if the Q was empty, wait() would have been called)

58

Task Queue

Worker Threads

A Possible Solution

A new task is executed by the released thread

59

Threads can leakA thread can endlessly wait for an I/O operation to complete

For example, the client may stop the interaction with the socket without closing it properly

What if task.run() throws a runtime exception (as opposed to other exceptions that a programmer of a client application has to catch in order to succeed compiling)?

Solutions: Bound I/O operations by timeouts using wait(time)

Catch possible runtime exceptions

60

Risks in Using Thread Pools

Pool Size

What is better: to have a large pool or a small pool?

Each thread consumes resourcesmemory, management overhead, etc.

A large pool can cause starvation

Incoming tasks wait for a free threadA small pool can cause starvation

Therefore, you have to tune the thread pool size according to the number and characterizations of expected tasks

There should also be a limit on the size of the task queue (why?)

61

Executor

The class Executors has 2 static methods to create thread pools

• ExecutorService newFixedThreadPool(int nThreads)• Pool of a fixed size

• ExecutorService newCachedThreadPool()• Creates new threads as needed

• New threads are added to the pool, and recycled

ExecutorService has an execute method• void execute(Runnable command)

62

63

Thread Pools

Prevent the thread-per-session pitfall!

Class ThreadPoolExecutor:

http://java.sun.com/javase/6/docs/api/java/util/concurrent/ThreadPoolExecutor.html

DEMO

examples.threads.threadexecutor

64