Drupal camp london 2014 John Ennew Concurrency Session Deeson Online
-
Upload
lizziehodgson -
Category
Technology
-
view
835 -
download
0
Transcript of Drupal camp london 2014 John Ennew Concurrency Session Deeson Online
C O N C U R R E N C YA N I N T R O D U C T I O N T O
J O H N E N N E W, S O L U T I O N S A R C H I T E C T, D E E S O N O N L I N E C E N G O N D . O . @ D E E S O N _ L A B S H T T P : / / W W W. D E E S O N - O N L I N E . C O . U K / L A B S
… O R H O W I M A D E M I G R AT E FA S T E R
W H AT I S C O N C U R R E N C Y ?
DatabaseFilestore
Web service
Application
DatabaseFilestore
Web service
Application
B R E A K I N G T H E P R O B L E M D O W N …
– E D W A R D A . L E E
“… a folk definition of insanity is to do the same thing over and over again and to expect the results to be different. By this definition, we in fact require
that programmers of multithreaded systems be insane. Were they sane, they could not understand
their programs …”
T H E P R O B L E M W I T H T H R E A D S , 2 0 0 6
T H E H T T P : / / W W W. E E C S . B E R K E L E Y. E D U / P U B S / T E C H R P T S / 2 0 0 6 /E E C S - 2 0 0 6 - 1 . H T M L W I T H T H R E A D S , 2 0 0 6
T H E D I N I N G P H I L O S O P H E R S
H T T P : / / D E E S O N - O N L I N E . C O . U K / L A B S / M U LT I - P R O C E S S I N G - PA R T- 1 -H O W - M A K E - D R U S H - R U S H
• lock_acquire()
• drush_invoke_process()
• DB transactions (innodb)
D R U PA L 7 T O O L S
M A K I N G M I G R AT E M O V E …
W H Y S P E E D U P M I G R A T I O N S ?
• Reduce the time a migration takes
• Discover and fix problems quicker
• Remigrate faster / regular migrations
T H E P R O C E S S …
• https://github.com/johnennewdeeson/drush-multi-processing
• Download mt.drush.inc and place in sites/all/drush
• Update to migrate 7.x-2.6-rc1
• Configure your migration class for batched operation
• Create a custom drush import command (or modify mtm.drush.inc also at github URL above)
• Full instructions online at: http://deeson-online.co.uk/labs/multi-processing-part-2-how-make-migrate-move
Configuring your migration class
Configuring your migration class
Creating a Drush command
Starting the process thread manager
Process setup …
Process teardown …
L I V E D E M O …
F I N E T U N I N G …
• Ramp up slowly - monitor server load and db load
• /etc/my.cnfinnodb_locks_unsafe_for_binlog = 1 innodb_flush_log_at_trx_commit=2
• Use alternative cache mechanism to ease your database load (e.g. memcache)
• $conf['lock_inc'] = 'sites/all/modules/memcache/memcache-lock.inc';
• $conf['memcache_stampede_protection'] = TRUE;
T H A N K S ! … A N Y Q U E S T I O N S ?
J O H N E N N E W, S O L U T I O N S A R C H I T E C T, D E E S O N O N L I N E C E N G O N D . O . @ D E E S O N _ L A B S H T T P : / / W W W. D E E S O N - O N L I N E . C O . U K / L A B S