EMu 24/7 @ Museum Victoria
23
EMu 24/7 @ Museum Victoria ‘The bird that never sleeps’ Donna Fothergill David Zhang View from Nicholson St, blades featuring Melbourne Museum venue branding. Photographer: Jennifer McNair Date: June 2007 Source: Museum Victoria MV Images
description
EMu 24/7 @ Museum Victoria. ‘The bird that never sleeps’ Donna Fothergill David Zhang. Overview. Why? The decision to make EMu available 24/7 at Museum Victoria How? The Infrastructure making this possible (hardware/scripts/resources) What’s Next? Other Tools. The MV EMu Guarantee. - PowerPoint PPT Presentation
Transcript of EMu 24/7 @ Museum Victoria
EMu 24/7@ Museum Victoria
‘The bird that never sleeps’
Donna FothergillDavid Zhang
View from Nicholson St, blades featuring Melbourne Museum venue branding.Photographer: Jennifer McNairDate: June 2007Source: Museum VictoriaMV Images
Overview
Why?The decision to make EMu available 24/7 at Museum Victoria
How?The Infrastructure making this possible (hardware/scripts/resources)
What’s Next? Other Tools
Eight-hour day banner, Melbourne, 1856Date: 1856Source: Museum VictoriaMV Images
The MV EMu Guarantee
In the event of:
Primary Server Crash
System/Database Maintenance
EMu upgrades
EMu available 24/7EMu available 24/7
Quote:
“we can rebuild him, we have the technology”
-Oscar Goldman
Importance
System Integration- MV Images- MVWISE
Live Feeds- e.g. The Learning Federation
Online collections- e.g. Natural Sciences
Core business systemCore business system
EMu FigurinePhotographer: David PaulDate: 11/07/2011Source: Museum VictoriaMV Images
+ =
EMu Mobile
Technology
MVWISE
MVWISE
‘EMu in the palm of your hand’‘EMu in the palm of your hand’
MV Images
Butterflies on Display: Bugs Alive! Exhibition : Melbourne MuseumPhotographer: Andrew CurtisDate: 03/09/2010Source: Museum VictoriaMV Images
EMu
Core database for all Collection items
MVWISE
Application for location and inventory control
MV Images
Digital asset management system for all digital images
and audio in the State Collection and other
resources
Full synchronisation –
desktop and handheld devices
Data sharing using iMu web services
Systems Integration
Live Data Feeds‘The Learning Federation’‘The Learning Federation’
Online Collections‘Search Natural Sciences Collections’‘Search Natural Sciences Collections’
Remote Access‘regardless of connection, location or device’‘regardless of connection, location or device’
Extending Access
Fieldwork Projects/Special Projects
Out of Hours Availability
Testing environments
‘Where ever Internet Access is available, so too is EMu! ’
‘Where ever Internet Access is available, so too is EMu! ’
The Moon, from a Photograph taken by the Great Melbourne Telescope, Date: 1874: Source: Museum VictoriaMV Images
Primary Server Failure
Synchronisation Synchronisation
Maintenance/Upgrades
Snapshot Snapshot
Backend processes
EMu is synchronised from live server to backup server every hour
Full EMu access is available using the EMu client (except during maintenance)
Snapshot taken of backup service prior to maintenance
During maintenance, the access becomes read-only.
Quote:
“(EMu) ... looks good at the front, but has a big backend”
Anonymous!
Infrastructure
Live EMu Server
‘EMu Synchronisation process’‘EMu Synchronisation process’
Changes to live EMu are transported to EMu on the
backup server
Changes to live EMu are transported to EMu on the
backup serverBackup EMu Server
Email notification sent to support staffEmail notification sent to support staff
Data processing is carried out on the backup server for ‘fast’
synchronisation
Data processing is carried out on the backup server for ‘fast’
synchronisation
Primary Server
Dedicated 10 GB link and RAM for synchronisation temp space
Dedicated 10 GB link and RAM for synchronisation temp space
Synchronisation Performance
Tue Aug 30 12:02:05 EST 2011...start emu sync to backup serverTue Aug 30 12:02:05 EST 2011...stop emu loadTue Aug 30 12:02:11 EST 2011...get all the deletions search for new/changed multimedia objectsTue Aug 30 12:02:18 EST 2011... start searching new imgTue Aug 30 12:02:19 EST 2011...start exporting modified/new recordsTue Aug 30 12:02:22 EST 2011...move data to backup serverTue Aug 30 12:02:25 EST 2011...moving loads directory to backup serverTue Aug 30 12:02:25 EST 2011...restart backgroud loadTue Aug 30 12:02:26 EST 2011...finish sync on bronte
Live EMu service = 75GBLive EMu service = 75GB
Synchronisation Summary
A dedicated private link and RAM for temp space
Data processing is deferred to the backup server
It takes less than 30 seconds for synchronisation to complete
‘Checks and whistles’ in place to avoid data corruption
Infrastructure
Live EMu Service
‘EMu Snapshot prior to maintenance ’‘EMu Snapshot prior to maintenance ’
Backup of live EMu Service
Live EMu is synched to the Backup serverBackup Server
PrimaryServer
Snapshot EMuis
read only
Snapshot EMuis
read only
Snapshot created from backup of live EMu Service
Service changes from emu to snapshot EMu
And back again when maintenance completes
Maintenance iscarried out on
Live EMu
Maintenance iscarried out on
Live EMu
Snapshot Summary
Live EMu is synched to backup server
The snapshot is taken from the backup live service and placed on the primary server.
It is a read only copy of the live service.
New connections and/or requests after 5pm are directed to the snapshot
When maintenance is complete, the service switches back to the live service.
Synchronisation process starts again
Camera - Parisio Cantini Empoli, Bellows Glass Plate, circa 1880Photographer: Taryn EllisDate: 03/11/2010Source: Museum VictoriaMV Images
What’s Next?
Live EMu 24/7 ?Live EMu 24/7 ?
Other ToolsManaging our large data files Managing our large data files
fupdateCompactSortUpdate load/data file compaction tool
fselectRecover Deleted Records
getDbRefDetails Delete batches of records in the backend
fupdateCompactSort
Example: update 80 fields in 284,297 catalogue records + compaction
Example: update 80 fields in 284,297 catalogue records + compaction
Data fileData file texloadtexload texcompacttexcompact Index Rebuild Index Rebuild
Data fileData fileUpdate +Compaction
Update +Compaction
1 hour 37 minutes
Index Rebuild Index Rebuild < 1 minute
getDbRefDetails
delete batches of recordsdelete batches of records
/usr/local/emu/bin/getDbRefDetails etaxonomy
Primary DB = "etaxonomy"; Linked Modules ecatalogue => TaxTaxonomyRef_tab 10 BirCuckooRef 1 IdeFiledAsRef 1 enarratives => TaxTaxaRef_tab 45 SpeLookalikesRef_tab 20 etaxonomy => HisAccNameRef_tab 20 TaxAvailabilityCorrectNameRef_tab 20 HomNameRef_tab 20 TaxAvailabilityCompetingHomRef_tab 20 ClaCurrentNameRef 1 PriTypeAboveSpeciesNameRef 1 ClaHybridParent2NameRef 1 SynNameRef_tab 100
fselect
Recover deleted recordsRecover deleted records
usage: fselect [options] dbname Options are: -d alternative data file -D print out ins forms -Sstatus record status -c set the status of selected
records to current -a analyse -kirn get the records with this irn -un get nth record with this irn -fFile save processed records into
File
Computer Walking and TalkingPhotographer: Frey MicklethwaitDate: June 2011Source: Museum VictoriaMV Images
