1

Atom-Aid: Detecting and

Surviving Atomicity Violations

Brandon Lucia†, Joseph Devietti†, Karin Strauss†‡, Luis Ceze†

ISCA 2008, Beijing, China

†University of Washington

‡Advanced Micro Devices, Inc

2

Motivation• Shared memory parallel programming is hard!

Deadlocks

Livelocks

Data RacesAtomicity

Violations

Plus Sequential Programming Bugs!

Ordering BugsMisplaced

Synchronization

• Testing currently a major challenge

• Need tools for bug detection

• Need to survive hard to detect bugs

Go Huskies!

4

Observations and Our Contribution•Observations:

1.Prior work shows that atomicity violations are common and hard to debug

2.With unrelated goals, systems supporting implicit atomicity were developed

3.Implicit Atomicity has an interesting effect on atomicity violations

1.Contributions:

1.We analyze the interaction of atomicity violations and implicit atomicity

2.We propose Atom-Aid which carefully controls memory interleaving to detect and survive atomicity violation bugs

3.We evaluate Atom-Aid with benchmarks including several real applications

4.Atom-Aid can report bugs to developers so they can be fixed

1. Atomicity Violations

6 An Atomicity Violation is a Hard Concurrency Bug

•BecauseBecause read_balanceread_balance and and write_balancewrite_balance can be interleavedcan be interleaved, but , but should should be atomicbe atomic, , Deposit Deposit has an has an atomicity violationatomicity violation

In a ‘08 study by Lu, et al, In a ‘08 study by Lu, et al, more than 2/3more than 2/3 of non-deadlocking bugs were of non-deadlocking bugs were atomicity violations atomicity violations

int balint bal = read_balance();= read_balance();Account.Deposit(amount){Account.Deposit(amount){

}}write_balance(bal);write_balance(bal);bal += amount;bal += amount;

What if this What if this happens?!happens?!

•Certain dynamic interleavings of Certain dynamic interleavings of DepositDeposit result in a result in a semantic semantic inconsistencyinconsistency

•These interleavings are called These interleavings are called unserializable interleavingsunserializable interleavings•If atomicity violations are unserializably interleaved, If atomicity violations are unserializably interleaved, bug bug behavior occursbehavior occurs!!

int bal int bal = read_balance();= read_balance();Account.Deposit(amount){Account.Deposit(amount){

}}write_balance(bal);write_balance(bal);bal += amount;bal += amount;

Y100 Y100Y150Y150

Both threads think the balance is Y150, but it Both threads think the balance is Y150, but it should be Y200!!should be Y200!!

T1: Deposit(50); T2: Deposit(50);

Y50Y50Y50Y50

balbalY100Y100

balbalY150Y150 balbal

Y100Y100balbal

Y150Y150

2.Implicit Atomicity

8

Implicit Atomicity

Read Read cc

Write Write bb

Read Read aa

Write Write cc

Read Read dd

Write Write bb

Read Read aa

Write Write bb

Fewer opportunities for Fewer opportunities for unserializable interleavings unserializable interleavings

to occurto occur

T1 T2

•Interleaving can Interleaving can onlyonly occur at chunk boundariesoccur at chunk boundaries

•Chunk size and boundaries Chunk size and boundaries can be can be adjusted arbitrarilyadjusted arbitrarily•Interleaving can be changed, Interleaving can be changed, and and memory semantics memory semantics preservedpreserved

•Implicit atomicity Implicit atomicity arbitrarily groups arbitrarily groups instructions into instructions into “chunks”“chunks”

{ChunksChunks {•Chunks execute Chunks execute

atomicallyatomically and in and in isolationisolation•(Think transactions)(Think transactions)

Many recent Implicit Atomicity Many recent Implicit Atomicity proposals: BulkSC, Implicit proposals: BulkSC, Implicit

Transactions, ASO, ...Transactions, ASO, ...

3.What Happens When They Get

Together

10 Atomicity Violations

and Implicit Atomicity

ExposedExposed violations can be violations can be

interleavedinterleaved

HiddenHidden violations violations execute execute

atomicallyatomically

read_balanceread_balance

write_balancewrite_balancewrite_balancewrite_balance

read_balanceread_balance

•Atomicity violations can be Atomicity violations can be exposedexposed •Atomicity violations can be Atomicity violations can be hiddenhidden

•Exposed violations Exposed violations may may manifest manifest themselvesthemselves if if unserializably unserializably interleavedinterleaved

•If a violation is If a violation is hidden hidden survival is survival is guaranteedguaranteed

Implicit Atomicity Implicit Atomicity Naturally HidesNaturally Hides Atomicity Atomicity ViolationsViolations

11

Natural Hiding

Implicit Atomicity alone survives a Implicit Atomicity alone survives a majoritymajority violations for these violations for these applicationsapplications

Chunk Size

Perc

en

t of

Vio

lati

ons

Hid

den

Can we do better?

13

Smart Chunking

write_balancwrite_balancee

read_balancread_balancee

ActivelyActively fit violation fit violation in only one chunkin only one chunk

Begin chunk closely Begin chunk closely to beginning of to beginning of

violationviolation•Atom-Aid survives Atom-Aid survives even moreeven more violations by dynamically violations by dynamically adjusting chunksadjusting chunks

•Atom-Aid infers where atomic Atom-Aid infers where atomic regions in an execution regions in an execution should beshould be

Exposed!Exposed!Hidden!Hidden!

14

Detecting Potential Atomicity Violations•Atom-Aid monitors Atom-Aid monitors dangerous dangerous addressesaddresses potentially involved potentially involved

in violationsin violationsT1 T2

Atom-Aid Monitors an address, AAtom-Aid Monitors an address, A,, if:if:

1.A thread makes 2 “nearby” 1.A thread makes 2 “nearby” accesses to Aaccesses to A

2.Another thread has “recently” 2.Another thread has “recently” accessed Aaccessed A

3.The accesses are potentially 3.The accesses are potentially unserializableunserializable

read_balanceread_balance

write_balancewrite_balance

read_balanceread_balance

write_balancewrite_balance

RdRd

WrWr

WrWr ?

Begin Monitoring Dangerous Address!Begin Monitoring Dangerous Address!

15 Atom-Aid Actively

Hides Atomicity Violations

•Smart chunking inserts a Smart chunking inserts a chunk boundary chunk boundary beforebefore dangerousdangerous accessesaccesses•Conservatively Conservatively prevents prevents potentially unserializable potentially unserializable interleavingsinterleavings

•Atom-Aid monitors Atom-Aid monitors potential potential unserializable interleavingsunserializable interleavings•Dangerous addresses can be Dangerous addresses can be observed observed without a violation without a violation necessarily occurringnecessarily occurring

write_balancewrite_balance

read_balanceread_balance

write_balancewrite_balance

Atom-Aid can detect atomicity violations Atom-Aid can detect atomicity violations before before they manifest themselvesthey manifest themselves and and prevent them prevent them

from occuringfrom occuring

16

Implementing Atom-Aid

Details in the paperDetails in the paper

•EfficientEfficient communication and set operations performed communication and set operations performed with signatureswith signatures

•This work uses a This work uses a BulkSCBulkSC-like system for Implicit Atomicity-like system for Implicit Atomicity•BulkSC provides sequential consistency at a coarse grain using BulkSC provides sequential consistency at a coarse grain using signaturessignatures•Each chunk maintains a read and write signatureEach chunk maintains a read and write signature

•Atom-Aid leverages Atom-Aid leverages bookkeepingbookkeeping and and communication communication for detectionfor detection•Additional signatures efficiently maintain memory Additional signatures efficiently maintain memory

access historyaccess history

Evaluation

18

Evaluating Atom-Aid

MySQLMySQL ApacheApache XMMSXMMS

java.lang.Stringjava.lang.StringBufferBuffer

Shared work Shared work queuequeue

See Section 5.2 in the paper for more infoSee Section 5.2 in the paper for more info

•Simulated Simulated real, real, unmodifiedunmodified applicationsapplications

•Simulated Simulated representative bug representative bug kernels extracted from kernels extracted from real softwarereal software and and prior prior workwork

•Simulations using PIN Simulations using PIN binary instrumentation binary instrumentation frameworkframework

19

Active Hiding

Atom-Aid hides virtually Atom-Aid hides virtually 100%100% of instances of the violations in of instances of the violations in these applicationsthese applications

Perc

en

t of

Vio

lati

ons

Hid

den

Chunk Size (insns.)

20

Hiding Bugs in Full Applications

•Atom-Aid hides most instancesAtom-Aid hides most instances of the of the violations in the violations in the unmodifiedunmodified applications we evaluatedapplications we evaluated

•Atom-Aid’s Atom-Aid’s performance impact is performance impact is negligiblenegligible, on top of performance , on top of performance impact of implicit atomicityimpact of implicit atomicity

•Atom-Aid requires Atom-Aid requires no modificationsno modifications to to software and software and no code annotationsno code annotations

Perc

en

t of

Vio

lati

ons

Hid

den

21

InspirationsAVIO - Lu, et al - Univ. of Illinois, Urbana-

Champagne Invariant based detection of Atomicity

Violations

SVD - Xu, et al - Univ. of Wisconsin, MadisonFully automated serializability checking of

atomic regions

ASPLOS ‘06

PLDI ‘05

22

Atom-Aid• Atomicity Violations are a challenging, subtle bug

• Implicity Atomicity can Naturally Hide some atomicity violations

• Atom-Aid uses Smart Chunking to Actively Hide nearly all instances of atomicity violations in applications evaluated

• Atom-Aid reports bugs back to programmers to aid in development, testing and debugging

23

Atom-Aid: Detecting and

Surviving Atomicity Violations

Brandon Lucia†, Joseph Devietti†, Karin Strauss†‡, Luis Ceze†

ISCA 2008, Beijing, China

†University of Washington

‡Advanced Micro Devices, Inc

24

SerializabilityAn interleaving is An interleaving is unserializableunserializable if there is no equivalent if there is no equivalent

sequential executionsequential execution

ReadRead ctrctr

WriteWrite ctrctr

WriteWrite ctrctr

RR

RR

WW

WW

RR

WW

WW

WW

RR

•If Atomicity Violation is If Atomicity Violation is unserializably interleaved, unserializably interleaved, atomicity violations atomicity violations manifests itselfmanifests itself..

•There are several cases There are several cases of unserializable of unserializable interleavinginterleaving

say “uninterleaved” execution

this list is exhaustive (in the one-variable case)

say “uninterleaved” execution

this list is exhaustive (in the one-variable case)

25

Atomicity Violation != Data Race

lock(L);tmp = ctr;unlock(L);

tmp++;

lock(L);ctr = tmp;unlock(L);

T1

lock(L);tmp = ctr;unlock(L);

tmp++;

lock(L);ctr = tmp;unlock(L);

T2

Violations manifest themselves when Violations manifest themselves when unserializableunserializable interleavings occurinterleavings occur

•Data RacesData Races are accesses to are accesses to shared data without shared data without synchronizationsynchronization

•Atomicity Violations may Atomicity Violations may exist in exist in race-freerace-free programs!programs!

•Atomicity ViolationsAtomicity Violations result result from false assumptions from false assumptions about atomicityabout atomicity

mention what the correct version of this program is

anywhere between the two critical sections

mention what the correct version of this program is

anywhere between the two critical sections

27

Probabilistic Survival

PPmanifestationmanifestation = P = Pexposedexposed x x PPbad bad

interleavinginterleaving

•If a violation is If a violation is exposedexposed the bug the bug may occurmay occur if a certain if a certain interleaving occursinterleaving occurs

•If PIf Pexposedexposed could be reduced to 0 the violation would could be reduced to 0 the violation would never manifest itselfnever manifest itself

•Implicit Atomicity Implicit Atomicity decreasesdecreases P Pexposedexposed so some violations are so some violations are naturally hiddennaturally hidden

Pinterleaved -> Pbadinterleaving

natural hiding - there is a chance operations will naturally be atomic hence natural hiding

ditch script

Pinterleaved -> Pbadinterleaving

natural hiding - there is a chance operations will naturally be atomic hence natural hiding

ditch script

28

Implicit Atomicity

Read Read ctrctr

Read Read ctrctr

Write Write ctrctr

Write Write ctrctr

Read Read ctrctr

Read Read ctrctr

Write Write ctrctr

Write Write ctrctr

Read Read ctrctr

Read Read ctrctr

Write Write ctrctr

Write Write ctrctr

Read Read ctrctr

Write Write ctrctr

Read Read ctrctr

Write Write ctrctr

Read Read ctrctr

Write Write ctrctr

Read Read ctrctr

Write Write ctrctr

Read Read ctrctr

Write Write ctrctr

Read Read ctrctr

Write Write ctrctr

Many recent Implicit Atomicity Many recent Implicit Atomicity proposals: BulkSC, Implicit proposals: BulkSC, Implicit

Transactions, ASO, ...Transactions, ASO, ...

•Traditional architectures Traditional architectures permit many permit many fine-grainedfine-grained interleavingsinterleavings

•Implicit atomicity Implicit atomicity arbitrarily groups arbitrarily groups instructions into instructions into “chunks”“chunks”

•Chunks are Chunks are notnot a a programming constructprogramming construct

•Chunks execute Chunks execute atomicallyatomically and in and in isolationisolation

{{

ChunksChunks

lock(L);tmp = ctr;unlock(L);

tmp++;

lock(L);ctr = tmp;unlock(L);

lock(L);tmp = ctr;unlock(L);

tmp++;

lock(L);ctr = tmp;unlock(L);

T1 T2

Possible Interleavings