STK5800 and EPrints
description
Transcript of STK5800 and EPrints
![Page 1: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/1.jpg)
STK5800 and EPrintsServices for Object Storage and Preservation
March 2008
All content in these slides is considered work in progress. In no way does it represent an absolute view of any final end product and at this stage should purely be considered a set of realistic ideas.
![Page 2: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/2.jpg)
OutlineStorageTek 5800 (The Honeycomb) provides
high resilience data storage with a built in metadata layer.
EPrints is a piece of repository software for managing large collections of digital objects and their related metadata.
![Page 3: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/3.jpg)
EPrintsOpen Source repository software to provide open
access to institutional output.
Provides a powerful plugin based package which can easily be extended at any layer to suit a users requirements.
2 types of archive
Those used to manage publications and small objects.Those used to deposit large objects. These tend to
contain heavier customisation.
![Page 4: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/4.jpg)
Preserv2Preserv2 is the 2nd iteration of a project looking
at preservation services for repositories.
Beyond simple backupFormat Renderers, Format Translation, Risk Assessment, Interoperability and long term storage.
![Page 5: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/5.jpg)
Why use a Honeycomb?A Honeycomb is not just a “Big Disk”
A Service Based Architecture:Big object, big storage, more powerful
plugins/services.Smaller Repositories can jointly use a single
Honeycomb as a “Preservation Service”.
Preservation Service ProvidersCan combine several servers into a “Honeycomb
Cloud”
![Page 6: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/6.jpg)
EPrints Architecture
EPrints (Repository) Layer
Object Storage Metadata Storage
![Page 7: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/7.jpg)
EPrints and Honeycomb
EPrints (Repository) Layer
STK5800HoneyComb
![Page 8: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/8.jpg)
Services for Repositories
EPrints (Repository) Layer
Metadata Services
Storage Beans
Automated Wide Area
Backup
![Page 9: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/9.jpg)
Metadata ServicesSame resilience as data.
Averts the need to store a file id/url somewhere in order to find an object.
Enables collections to be constructed by independent parties.
Objects can be exported into many formats accurately.
![Page 10: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/10.jpg)
Storage BeansCan perform operations upon the objects in the system
without reliance upon the repository to manage these processes. (e.g. Object Translation)
Preservation services can provide feedback to repository administrators on potential risks to their objects. (e.g. Object Classification, age)
Can be used to extend the metadata layer to provide more powerful access to objects and their parts/pages. (e.g. Retrieve me page 10 of volume 6 of X)
![Page 11: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/11.jpg)
Wide Area Replication (Backup)
The possibility to link two or more Honeycombs together over a wide area to provide mirrored backup.
This can be implemented by the archive which can store its objects in a “Honeycomb Cloud”
![Page 12: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/12.jpg)
Possible Architectures (2)
Repository Repository Repository
![Page 13: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/13.jpg)
Possible Architectures (3)
Repository Repository Repository
![Page 14: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/14.jpg)
Possible Architectures (4)
Repository Repository Repository
![Page 15: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/15.jpg)
Preservation ServicesA “Honeycomb Cloud” provides the basis for a
preservation service which can be provided to many small scale (<200Gb) repositories.
Options for object storage: Locally with Honeycomb acting purely as a preservation
service. Hand all object storage and retrieval to Honeycomb Cloud. A half and half solution:
Small Objects served locally, Large Objects from Honeycomb.
Recent and Popular Objects served locally, Older Objects considered preserved.
![Page 16: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/16.jpg)
EPrints with the STK 4500 The out of the box repository solution for Large Repositories.
![Page 17: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/17.jpg)
Thumpers “Big Disk”The Thumper system (STK 4500) is essentially a
“Big Disk” server.
“Out of the Box” solution.
Expansions:Services to enable replication between 2 thumpers.Preservation services using a Honeycomb.
Aimed at Repositories where tape backup is not ideal.
![Page 18: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/18.jpg)
Ecrystals (Possible Use Case)Large Chemistry repository which currently stores
only processes result objects (small).
These result files are generated from >1Gb raw datasets.
8+ Datasets generated a day.
After 6 months results sets are of less worth.This represents 1TB of raw data in a 6 month period.
![Page 19: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/19.jpg)
ECrystals – Single Honeycomb ArchitectureCurrent Repository RemainsAll Results Sets Stored on
HoneyComb
ProsSimplistic ArchitectureSole use of HoneycombYear of “on-site” storage.
ConsCostBackup Procedure?
EPrints (Repository) Layer
![Page 20: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/20.jpg)
“
Thumper System
ECrystals – Thumper with “Honeycomb Cloud”
ProsSingle local machine6 months+ locally AccessibleAutomated PreservationPreservation Services managed by Honeycomb Cloud.Storage Beans on Honeycomb Cloud compress older/less popular objects
Cons?
EPrints (Repository) Layer
![Page 21: STK5800 and EPrints](https://reader035.fdocuments.net/reader035/viewer/2022062218/56816525550346895dd7ad01/html5/thumbnails/21.jpg)
SummaryHoneycomb provides:
Better separation of repository layer from storage layer.
Repository interoperability.A new approach to storing and preserving data
from institutional repositories based on EPrints and other software.