CONCURRENCY CONTROL IN MOBILE DISTRIBUTED REAL-TIME DATABASE
Shared Database Concurrency
-
Upload
aivars-kalvans -
Category
Software
-
view
83 -
download
0
Transcript of Shared Database Concurrency
![Page 1: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/1.jpg)
Shared Database ConcurrencyAivars.Kalvans@(tieto|gmail).com
![Page 2: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/2.jpg)
Who am I
● Tieto Latvia– Payment Card Business Unit
● Lead software architect– Started as a junior developer
● Card Suite - payment card system– 400 institutions, 33 countries
– 100s and 1000s financial transactions per second
![Page 3: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/3.jpg)
Shared memory concurrency
● Concurrent components communicate by altering the contents of shared memory locations– Threads (C++, Java, C#, …)– Processes (POSIX shared memory)
● Buzz– Lock-free– Actors– Functional programming
![Page 4: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/4.jpg)
![Page 5: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/5.jpg)
That's nice, but...
● Most applications use a single database● Often database is the bottleneck● You might not know you have a problem
![Page 6: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/6.jpg)
Shared database concurrency
● Concurrent components communicate by altering shared data in a database
![Page 7: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/7.jpg)
DBA
![Page 8: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/8.jpg)
Oracle Real-World Performance
● Most problems come from incorrect use of the database
● DBA and a new hardware: maybe 2x improvement
● Code, design, algorithm: 10x or 100x improvement
![Page 9: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/9.jpg)
Database concurrency
● Pessimistic locking– Concurrent updates might happen
● Optimistic locking– Concurrent updates will not happen
● NoSQL
![Page 10: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/10.jpg)
NoSQL
![Page 11: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/11.jpg)
“Banks don't use transactions”
● Visible tip of the iceberg● Below the water
– Transactions still used within application
– Auditing, Reconciliation, Matching
– Code, Computing power, Manual work
● Money “reserved” for 2 or more weeks.
![Page 12: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/12.jpg)
Database concurrency
● Pessimistic locking– Concurrent updates might happen
● Optimistic locking– Concurrent updates will not happen
● NoSQL● Anything else?
![Page 13: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/13.jpg)
Oracle 101
● Lock table …● Row-level (TX) locks● Locks are held until Commit or Rollback
– Not until Rollback to savepoint
● Writes don't block reads● Reads don't block writes● Writes may block writes
![Page 14: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/14.jpg)
Placing a row-level lock
● Select … for update [nowait]– There is a non-blocking mode!
● Update, Delete● Insert
– Primary key or unique constraint violation
Let's call it “implicit locking”
![Page 15: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/15.jpg)
Other databases
● PostgreSQL: row-level locks● MySQL: it's complicated
– Table-level for MyISAM, MEMORY, MERGE
– Block-level for BDB
– Row-level for InnoDB
● DB2: row-level or page-level● Isolation levels
![Page 16: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/16.jpg)
Which one to use?
● Pessimistic locking● Optimistic locking● Implicit locking
![Page 17: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/17.jpg)
Difference?
Time Pessimistic Optimistic Implicit
t1 Select .. for update Select version, ... Select
t2 Modify data Modify data Modify data
t3 Update ... Update …set version=:next_versionwhere version=:known_version
Update ...
t4 Retry if 0 rows updated
t5 Commit Commit Commit
![Page 18: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/18.jpg)
Pessimistic vs. Optimistic
http://www.orafaq.com/papers/locking.pdfMore challenging than the technology is overcoming resistance from seasoned development professionals who have been using the trusted SELECT… FOR UPDATE for all of their Oracle careers.
These individuals may need to be convinced of the benefits of using optimistic and on large development projects their support will be crucial.
![Page 19: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/19.jpg)
Problems
● Lockout problem– Go to lunch between locking and commit
● Deadlock problem– No difference placing locks by select or update
● Performance– Select+Update vs. Select+Update vs.
Select+Update
![Page 20: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/20.jpg)
Optimistic?
● Shorter locking time– We don't expect concurrent updates, right?
● Wait for lock before update is executed– Locks are queued
![Page 21: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/21.jpg)
Lost updates
● Absolute updates– Setting a new salary
● Relative updates– Adding a positive or a negative number
– Min/Max
![Page 22: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/22.jpg)
How to choose locking
● Implicit locking by default– Relative updates– Last update wins
● Pessimistic locking– Prevent concurrency
● Optimistic locking– ?– Transaction isolation: Serializable
![Page 23: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/23.jpg)
Increasing concurrency
● Reduce the time locks are held● Reduce the need for locks● Fine-grained locks
![Page 24: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/24.jpg)
Warm-up
● Prepared statements● Reduce both Hard and Soft parses
– Create a statement just once
● Batching / bulk operations– Python: .executemany
– JDBC: addBatch
● Faster CPUs and disks– Not more
![Page 25: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/25.jpg)
Warm-up
● Design– Avoid absolute updates
● Inserts instead of updates– Create a change log
– Insert
– Aggregate and update once in a while
![Page 26: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/26.jpg)
Skip indexes
● Select a physical row identifier for subsequent use within transaction– Oracle: ROWID
– PostgreSQL: ctid
Select rowid from table1;
Update table1 set col1=:val1 where rowid=:rowid;
![Page 27: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/27.jpg)
Combining SQL
Select col1, col2 from table1;
Update table1 set col3=:val3;
Update table1 set col3=:val3
returning col1, col2 into :out1, :out2;
![Page 28: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/28.jpg)
Combining SQL
Insert into table1 (col1, col2) values (:val1, :val2);
Insert into table2 (col1, col2) values (:val1, :val2);
Insert all
into table1 (col1, col2) values (:val1, :val2)
into table2 (col1, col2) values (:val1, :val2)
Select * from dual;
![Page 29: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/29.jpg)
Reordering
● Do (b)locking statements last
Update table1 set col1=:val1;
Insert table2 (col1, col2) values (:val1, :val2);
Insert table2 (col1, col2) values (:val1, :val2);
Update table1 set col1=:val1;
![Page 30: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/30.jpg)
Combine SQL with a commit
● Do a commit with the last SQL statement– JDBC: setAutoCommit
Insert; Update Commit;
Insert; Update+Commit;
![Page 31: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/31.jpg)
No work between SQL
● Prepare data for all statements before executing the first one– JDBC: setInt, setString before any execute,
executeUpdate
● Not an option for some APIs– Python: .execute(SQL, parameters)
![Page 32: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/32.jpg)
“Partition” rows
● Multiple rows instead of one– Distribute evenly among processes
– Aggregate rows when reading
Id Balance
13 42
Id Balance Partition
13 15 0
13 7 1
13 11 2
13 9 3
![Page 33: Shared Database Concurrency](https://reader031.fdocuments.net/reader031/viewer/2022030222/58b8a5501a28abc06d8b5d5f/html5/thumbnails/33.jpg)
Thank you