1. Drupal + Mongo: Biggeris Better? Alightningtalk by Forest Mars

2. Drupal (FTW!) Drupal: Aspect-oriented (modular) social-publishing framework, written in php (pdo) that allows easy creation and integration of multiple data-rich social networking sites and robust web applications & services.Used on large high-performance websites. Roadmap anticipates future web (rdf, ggg) & emerging technologies (MongoDB!) 3. The Problem with SQL* Webapps > Once you have related data, you need joins > Indexing on joins does not work that well ...So you:>Introduce denormalisation >Build extra tables *(RDBMS) 4. The Problem with Schema > Changes broadly lock data, requiring downtime or putting excessive, short-term load on the system. > The data integrity constraints dont quite support application integrity constraints. For example, theres no standard way in SQL to require a column to contain only valid URLs. > Coordinating schema changes with application code changes is difficult, largely because schema changes lack convenient coupling with code. 5. In theory, theory in practice are exactly the same. In practice, they're completely different. Practice WiP (works in practice) Scales well Future-proof Theory Clean abstractions Strong semantics Smart-proof 6. A C I D/B A S E Base Basically Available Scales well Eventually consistant Acid Atomic Consistent Isolated Durable 7. ( Why Mongo ?)Highly Available Easily Scalable Partition Tolerant 8. ( Why Mongo ?)Tableless Queriless SchemalessBlazingly fast Faster Development timesNicer learning curvesCode is trimmerFuture-proof 9. Performance / Scaling Whitehouse.gov / direct engagement 15K/day contact requests 2M records in db4GB db: replication risks MongoDB:180M+ documents in 1 collection 10. Writing Performant Queries Sort column must be the last column used in the index.Range query must also be the last column in an index,Only use a range query or sort on one column.Conserve indexes by re-ordering columns used in straight = queriesNever use Mongo's $ne or $nin operator'sNever use Mongo's $exists operatorhttp://jira.mongodb.org/browse/WEBSITE-12 11. Install Mongo public array authenticate(string $username,string $password ) public array command ( array $data ) __construct ( Mongo $conn , string $name ) public MongoCollection createCollection ( string $name [, bool $capped = FALSE [, int $size = 0 [, int $max = 0 ]]] ) public array createDBRef ( string $collection , mixed $a ) public array drop ( void ) public array dropCollection ( mixed $coll ) public array execute ( mixed $code [, array $args = array() ] ) public bool forceError ( void ) public MongoCollection __get ( string $name ) public array getDBRef ( array $ref ) public MongoGridFS getGridFS ([ string $prefix = "fs" ] ) public int getProfilingLevel ( void ) public array lastError ( void ) public array listCollections ( void ) public array prevError ( void ) public array repair ([ bool $preserve_cloned_files = FALSE [, bool $backup_original_files = FALSE ]] ) public array resetError ( void ) public MongoCollection selectCollection ( string $name ) public int setProfilingLevel ( int $level ) public string __toString ( void ) 12. Real World Example list the nodes of a user ordered by comment count uid is stored in the node table and the comment count is in node_comment_statistics > thus query cannot be indexed (Comparison of dissimilar columns may prevent use of indexes if values cannot be compared directly without conversion.) 13. What's already in Drupal *mongodb: support library for the other modules (D7/D6) *mongodb_block: Store block information in mongodb.Very close to the core block API. *mongodb_cache: Store cache items in mongodb. *mongodb_session: Store sessions in mongodb. *mongodb_watchdog: Store watchdog messages in mongodb *mongodb_queue: DrupalQueueInterface implementation using mongodb. *mongodb_field_storage: Store the fields in mongodb. 14. Mongo Watchdog 15. mongodb_watchdog mongo> db.watchdog.drop(); mongo> db.createCollection("watchdog", {capped:true, size:1000000, max:10000} ); 16. "It's really incredible how much you don't have to do."-chx 17. mongodb_cache $conf['page_cache_without_database'] = TRUE; 18. mongodb_sessions $conf['session_inc'] = 'sites/all/modules/mongodb/ mongodb_session/mongodb_session.inc'; 19. mongodb_sessions function mongodb_session_user_update($edit, $account) { if (!module_exists('mongodb_field_storage')) { $roles = _mongodb_session_get_roles($account); $save = (array) $account + array( '_id' => (int) $account->uid, '@bundle' => 'user', '@fields' => array(), 'roles' => $roles, ); foreach (array('uid', 'created', 'access', 'login', 'status', 'picture') as $key) { $save[$key] = (int) $save[$key]; } mongodb_collection('fields_current', 'user')->save($save); } return $roles; } 20. mongodb_sessions * The user-level session storage handlers: * - _drupal_session_open() * - _drupal_session_close() * - _drupal_session_read() * - _drupal_session_write() * - _drupal_session_destroy() * - _drupal_session_garbage_collection() are assigned by session_set_save_handler() in bootstrap.inc 21. mongodb_block function hook_block_view_alter(&$data, $block) { // Remove the contextual links on all blocks that provide them. if (is_array($data['content']) && isset($data['content']['#contextual_links'])) { unset($data['content']['#contextual_links']); } // Add a theme wrapper function defined by the current module to all blocks // provided by the "somemodule" module. if (is_array($data['content']) && $block->module == 'somemodule') { $data['content']['#theme_wrappers'][] = 'mymodule_special_block'; } } 22. Block rebuild Notice :Undefined variable: block_html_idininclude()(line4of/var/www/Drupal/drupal-7.0-alpha4/themes/garland/block.tpl.php ). Notice :Undefined variable: block_html_idininclude()(line4of/var/www/Drupal/drupal-7.0-alpha4/themes/garland/block.tpl.php ). Notice :Undefined variable: block_html_idininclude()(line4of/var/www/Drupal/drupal-7.0-alpha4/themes/garland/block.tpl.php ). Notice :Undefined variable: block_html_idininclude()(line4of/var/www/Drupal/drupal-7.0-alpha4/themes/garland/block.tpl.php ). Notice :Undefined variable: block_html_idininclude()(line4of/var/www/Drupal/drupal-7.0-alpha4/themes/garland/block.tpl.php ). 23. Render main content block function mongodb_block_theme() { 'block' => array( 'render element' => 'elements', 'template' => 'block', 'path' => drupal_get_path('module', 'block'),),} function mongodb_block_mongodb_block_info_alter(&$blocks) { // Enable the main content block. $blocks['system_main']['region'] = 'content'; $blocks['system_main']['weight'] = 0; $blocks['system_main']['status'] = 1;} function mongodb_block_rehash($redirect = FALSE) { $collection = mongodb_collection('block'); $theme = variable_get('theme_default', 'garland'); 24. mongodb_field_storage don't : variable_set('field_storage_default', 'mongodb_field_storage'); instead : $conf['field_storage_default'] = 'mongodb_field_storage'; in settings.php ESP. for session/caching backends 25. Drupal 7 Everything In MongoDB 26. Import all Nodes > MongoDB* (* in 14 l.o.c.) // Connect $mongo = new Mongo(); // Get the database (it is created automatically) $db = $mongo->testDatabase; // Get the collection for nodes (it is created automatically) $collection = $db->nodes; // Get a listing of all of the node IDs $r = db_query('SELECT nid FROM {node}'); // Loop through all of the nodes... while($row = db_fetch_object($r)) { print "Writing node $row->nid "; // Load each node and convert it to an array. $node = (array)node_load($row->nid); // Store the node in MongoDB $collection->save($node); } 27. Import all Nodes > MongoDB* (* in 14 l.o.c.) # drush script mongoimport.php # use testDatabase; # db.nodes.find( {title: /about/i} , {title: true}).limit(4); 28. Import all Nodes > MongoDB* (* in 14 l.o.c.) 29. What's Next? Multiple DB servers Data Persistance Query logging - Devel support Query builder Views integration DBTNG Full DB AbstractionMongoDB API 30. Query Logging Extend Mongo collection class Pass instance back from mongodb_collection Implement all collection methods 31. Drupal Mongo API$collection = mongodb_collection('myname'); $collection->find(array('key' => $value)); $collection->insert($object); $collection->remove(array('_id' => $item->id)); 32. Full DBTNG ImpementationDO NOT USE !!! 33. awesomesauce page callback => 'drupal_json' $items['node/%node/json']= array('page callback'=> 'drupal_json', 'page arguments'=> array(1), 'type'=> MENU_CALLBACK); 34. Where Mongo Won't Work Ex. list nodes belonging to users whose usernames starts with 'Ab' 35. Thanks! Comments & questions to:ForestMars @gmail.com ForestMars @googlewave.com Facebook, LinkedIn, etc.twitter: @elvetica (identica@forest)