Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University...

54
Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:[email protected]

Transcript of Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University...

Page 1: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Preservation of Digital Content

Preservation of Digital Content

Jacob Nadal

Head, Craig Preservation LaboratoryIndiana University Libraries

(812) 855-6281 | mailto:[email protected]

Jacob Nadal

Head, Craig Preservation LaboratoryIndiana University Libraries

(812) 855-6281 | mailto:[email protected]

Page 2: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Abstract: Preservation of Digital Content

Abstract: Preservation of Digital Content

Libraries are being challenged to develop technologies for maintaining access to digital information. Although many of the same ethical and organizational issues govern preservation of any type of resource, the media-independent nature of digital information requires the development of new techniques and technologies to insure the survival of digital content.

This talk will review the major initiatives that are underway in the preservation of digital information and discuss technological methods available for reliably storing digital objects, keeping those objects usable over time, and recovering digital objects in the event of system failure or obsolescence. Examples from current projects will be used to illustrate activities in these areas.

Libraries are being challenged to develop technologies for maintaining access to digital information. Although many of the same ethical and organizational issues govern preservation of any type of resource, the media-independent nature of digital information requires the development of new techniques and technologies to insure the survival of digital content.

This talk will review the major initiatives that are underway in the preservation of digital information and discuss technological methods available for reliably storing digital objects, keeping those objects usable over time, and recovering digital objects in the event of system failure or obsolescence. Examples from current projects will be used to illustrate activities in these areas.

Page 3: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

OutlineOutline• Storage

• Centralized repositories• Distributed Storage (LOCKSS)

• Repair and Maintenance• Migration: Keeping content available• Emulation: Preserving the functionality of

software• Digital archaeology: Recovering damaged data

• Theory and Planning• The OAIS Reference Model Trusted Digital

Repositories• The DSEP Process Model• Cornell University Digital Preservation

Management Program

• Storage• Centralized repositories• Distributed Storage (LOCKSS)

• Repair and Maintenance• Migration: Keeping content available• Emulation: Preserving the functionality of

software• Digital archaeology: Recovering damaged data

• Theory and Planning• The OAIS Reference Model Trusted Digital

Repositories• The DSEP Process Model• Cornell University Digital Preservation

Management Program

Page 4: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Data StorageData Storage

•Storage facilities are necessary to ensure that data are available for future use

•Three functions:

•Ingest data into the archive

•Refresh data by making a duplicate copy of data to new media

•Retrieve data on request

•Common in most IT environments to insure operational support through routine backup

•Storage facilities are necessary to ensure that data are available for future use

•Three functions:

•Ingest data into the archive

•Refresh data by making a duplicate copy of data to new media

•Retrieve data on request

•Common in most IT environments to insure operational support through routine backup

Page 5: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Centralized StorageCentralized Storage

•Provides central management of a complete archive of digital content.

•Requires large amounts of infrastructure: hardware and staffing

•Provides high level of accountability

•Provides central management of a complete archive of digital content.

•Requires large amounts of infrastructure: hardware and staffing

•Provides high level of accountability

Page 6: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Centralized Storage Examples

Centralized Storage Examples

• IU MDSS

• Long term refreshing of bit-stream, with tape media replaced on a regular schedule, before EOL.

• Data is deposited on hierarchy of disks and tapes

• Available through a single file structure that is transparent to the end user, regardless of which media stores user’s data

• OCLC Digital Archive

• Long term refreshing of bit-stream.

• Ingest tools for batch uploading and web harvesting.

• Plans for some file migration support.

• IU MDSS

• Long term refreshing of bit-stream, with tape media replaced on a regular schedule, before EOL.

• Data is deposited on hierarchy of disks and tapes

• Available through a single file structure that is transparent to the end user, regardless of which media stores user’s data

• OCLC Digital Archive

• Long term refreshing of bit-stream.

• Ingest tools for batch uploading and web harvesting.

• Plans for some file migration support.

Page 7: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Hard Drive Replacement Cycle(5 year LE)

Hard Drive Replacement Cycle(5 year LE)

Data

Inte

grity

(%

)D

ata

Inte

grity

(%

)

Years in OperationYears in Operation

Page 8: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Magnetic Tape Replacement Cycle

(10 year LE)

Magnetic Tape Replacement Cycle

(10 year LE)

Data

Inte

grity

(%

)D

ata

Inte

grity

(%

)

Years in OperationYears in Operation

Page 9: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Magnetic Tape Replacement Cycle

(30 year LE)

Magnetic Tape Replacement Cycle

(30 year LE)

Data

Inte

grity

(%

)D

ata

Inte

grity

(%

)

Years in OperationYears in Operation

Page 10: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Costs of Media Replacement

(1 TB)

Costs of Media Replacement

(1 TB)Hard Disk

LE 10 Tape

LE 30 Tape

30 Years

$5284 $1,008 $360

60 Years

$10,568 $2,006 $720

180 Years

$31,703 $6,480 $2,160

Page 11: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Distributed StorageDistributed Storage

•"...let us save what remains: not by vaults and locks which fence them from the public eye and use in consigning them to the waste of time, but by such a multiplication of copies, as shall place them beyond the reach of accident." --Thomas Jefferson to Ebenezer Hazard, Philadelphia, February 18, 1791.

•"...let us save what remains: not by vaults and locks which fence them from the public eye and use in consigning them to the waste of time, but by such a multiplication of copies, as shall place them beyond the reach of accident." --Thomas Jefferson to Ebenezer Hazard, Philadelphia, February 18, 1791.

Page 12: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

LOCKSS: Lots of copies keeps stuff safe

LOCKSS: Lots of copies keeps stuff safe

•LOCKSS software turns a low-cost PC into a “digital preservation appliance.”

•A network of these appliances:

•Collects content from target e-journals using a web crawler.

•Continually compares the content collected by individual appliances and repair any differences

•LOCKSS software turns a low-cost PC into a “digital preservation appliance.”

•A network of these appliances:

•Collects content from target e-journals using a web crawler.

•Continually compares the content collected by individual appliances and repair any differences

Page 13: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

LOCKSS: Lots of copies keeps stuff safe

LOCKSS: Lots of copies keeps stuff safe

•Individual appliances also:

•Act as a web proxy that provides the library's community with publisher's content or preserved content as appropriate.

•Provide administrative tools to target new journals for preservation, monitor the state of the journals being preserved, and control access.

•Individual appliances also:

•Act as a web proxy that provides the library's community with publisher's content or preserved content as appropriate.

•Provide administrative tools to target new journals for preservation, monitor the state of the journals being preserved, and control access.

Page 14: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

LOCKSSLOCKSS

•Focused on collecting output of specific entities.

•Demonstrated viability format migration within their system.

•Successful verification and repair of damaged content (system survived a fire at LANL, network disruptions at Stanford, relocation of the machine at Berkeley, and flaky hardware at Columbia.)

•Focused on collecting output of specific entities.

•Demonstrated viability format migration within their system.

•Successful verification and repair of damaged content (system survived a fire at LANL, network disruptions at Stanford, relocation of the machine at Berkeley, and flaky hardware at Columbia.)

Page 15: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Repair & MaintenanceRepair & Maintenance

•Alteration of data to allow native use in new environments (Migration)

•Creation of tools to read unaltered data in new environments (Emulation)

•Recovery of data that is corrupt (Digital Archaeology)

•Alteration of data to allow native use in new environments (Migration)

•Creation of tools to read unaltered data in new environments (Emulation)

•Recovery of data that is corrupt (Digital Archaeology)

Page 16: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

MigrationMigration

•Translation of files from one format to another

•Hopes:

•Migration has already been used to move data from version to version and from obsolete software.

•Once the migration tools are properly designed, the process will require little staff activity.

•Translation of files from one format to another

•Hopes:

•Migration has already been used to move data from version to version and from obsolete software.

•Once the migration tools are properly designed, the process will require little staff activity.

Page 17: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Migration hopes & fearsMigration hopes & fears• Fears:

• Conversion of a document from its original logical format into subsequent formats as each format becomes obsolete.

• May be time consuming, since every document must be converted every time its current format becomes obsolete.

• May corrupt a document, losing its original appearance, structure, interactive behavior, and possibly its content.

• Corruption may be cumulative and the original document may not be useful to correct corruption, since the original may be unusable some time after it is first converted.

• Fears:

• Conversion of a document from its original logical format into subsequent formats as each format becomes obsolete.

• May be time consuming, since every document must be converted every time its current format becomes obsolete.

• May corrupt a document, losing its original appearance, structure, interactive behavior, and possibly its content.

• Corruption may be cumulative and the original document may not be useful to correct corruption, since the original may be unusable some time after it is first converted.

Page 18: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Migration in PracticeMigration in Practice

•Somali Market Posters

•Created as Kodak PhotoCD files

•Initial translation was desired to update libraries new website

•Decided that we also wanted to create TIFF masters

•Made a series of trial migrations...

•Somali Market Posters

•Created as Kodak PhotoCD files

•Initial translation was desired to update libraries new website

•Decided that we also wanted to create TIFF masters

•Made a series of trial migrations...

Page 19: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Resulting filesResulting filesGraphicConverterGraphicConverter PhotoShopPhotoShop

TIFF

TIFF

JPEG

JPEG

Page 20: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Comparison across applications

Comparison across applications

•PCD to PPM in Graphic Converter

•PCD to TIFF in Graphic Converter and Photo Shop

•TIFFs compared

•Both TIFF to PPM in Graphic Converter

•All 3 PPMs compared

•PCD to PPM in Graphic Converter

•PCD to TIFF in Graphic Converter and Photo Shop

•TIFFs compared

•Both TIFF to PPM in Graphic Converter

•All 3 PPMs compared

Page 21: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Comparison across applications

Comparison across applications

•Different management of header information across formats and applications

•Different color content across applications

•Different management of header information across formats and applications

•Different color content across applications

Page 22: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

L: 44.579 a: -11.680 b: 26.219 L: 51.113 a: -16.879 b: 31.961

L: 84.694 a: 9.730 b: 57.734L: 86.423 a: 6.469 b: 58.695

L: 61.083 a: 35.926 b: 45.340L:61.482 a: 39.602 b: 41.082

L: 63.708 a: 38.949 b: 9.207 L: 66.421 a: 34.215 b: 1.844

L: 36.178 a: 10.516 b: -20.324 L: 43.517 a: 18.680 b: -28.395

L: 59.899 a: -17.188 b: -32.664

L: 50.867 a: -6.105 b: -37.188

L: 92.576 a: -0.863 b: 3.945 L: 92.869 a: -1.516 b: 6.383

L: 33.185 a: 1.141 b: 16.164 L: 40.055 a: 2.773 b: 18.523

L: 25.002 a: -3.293 b: 11.734 L: 26.825 a: -1.285 b: 6.969

CIE L*a*b* ComparisonCIE L*a*b* Comparison

Page 23: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Multiple migrations within an application

Multiple migrations within an application

•Within GraphicConverter:

•PCD to PPM to TIFF to PICT to BPM to PPM

•Within PhotoShop:

•PCD to TIFF to PICT to BPM to TIFF

•Initial and final formats compared

•Intermediate formats to PPM for comparison

•Within GraphicConverter:

•PCD to PPM to TIFF to PICT to BPM to PPM

•Within PhotoShop:

•PCD to TIFF to PICT to BPM to TIFF

•Initial and final formats compared

•Intermediate formats to PPM for comparison

Page 24: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Multiple migrations within an application

Multiple migrations within an application

PhotoShopGraphic

Converter

TIFF

Header info lost in migration, Image data identical:#SOFTWARE: Adobe Photoshop 7.0#DATE: 2004:12:02 22:55:41##Ignored Tags: $02BC, $A002, $A003

Header info lost in migration,Image data identical:#SCANTIME: 5/1/95#MODTIME: 2/6/40#MEDIATYPE: 052/55 SPD 0000 #00#SCANNERVENDOR: KODAK#SCANNERPRODID: FilmScanner 2000#SCANNERFIRMREV: 3.55#SCANNERSERIAL: 0634#PIW: Eastman Kodak#PHOTOFINISHER: C.I.S. Photo

PICT Identical Identical

BMP Identical Identical

PPM n/a Identical

Page 25: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Migration ExampleMigration Example

TIFFTIFF

BMPBMP

PICTPICT

GraphicConverterGraphicConverter PhotoShopPhotoShop

Page 26: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

ObservationsObservations•There are some formats and situations

where successive migration is possible without any loss of content.

•Standardized, well-documented tools are necessary for consistency.

•Technical metadata is essential to maintaining accuracy of content.

•Different levels of metadata support within files makes it easy to lose this information.

•There are some formats and situations where successive migration is possible without any loss of content.

•Standardized, well-documented tools are necessary for consistency.

•Technical metadata is essential to maintaining accuracy of content.

•Different levels of metadata support within files makes it easy to lose this information.

Page 27: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

EmulationEmulation

• Emulation focuses on either recreating the internal design of the system (hardware, software or both) or creating an environment in which the original software can be run

• Simulation recreates involves creating a new application which runs original program and simulates the original look and feel in a new environment.

• Emulation focuses on either recreating the internal design of the system (hardware, software or both) or creating an environment in which the original software can be run

• Simulation recreates involves creating a new application which runs original program and simulates the original look and feel in a new environment.

Page 28: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Emulation hopes and fears

Emulation hopes and fears

•Hopes:

•Provides verification by maintaining original software image.

•Minimizes effort by focusing only on creation of emulator, not modification of software

•Fears

•Emulators will be too complex or costly to build.

•Hopes:

•Provides verification by maintaining original software image.

•Minimizes effort by focusing only on creation of emulator, not modification of software

•Fears

•Emulators will be too complex or costly to build.

Page 29: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Emulation levelsEmulation levels• Emulation can occur at several levels

• Operating system: The host machine will emulate the look, feel and functionality of another operating system. This makes it possible to run applications in new environments.

• Processor: Allows the host machine to carry out instructions exactly as the emulated processor would.

• Machine: Emulations have been developed to make it possible to emulate complete computing machines on other machines.

• Emulation can occur at several levels

• Operating system: The host machine will emulate the look, feel and functionality of another operating system. This makes it possible to run applications in new environments.

• Processor: Allows the host machine to carry out instructions exactly as the emulated processor would.

• Machine: Emulations have been developed to make it possible to emulate complete computing machines on other machines.

Page 30: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Emulation in PracticeEmulation in Practice•Successful trials in preservation contexts:

•BBC Domesday Book (Camileon Project)

•Sinclair Spectrum on Apricot (JISC eLib)

•Windows 95 dependent CD-ROM (Nedlib)

•Other applications:

•Vintage video game consoles

•System and software development

•Successful trials in preservation contexts:

•BBC Domesday Book (Camileon Project)

•Sinclair Spectrum on Apricot (JISC eLib)

•Windows 95 dependent CD-ROM (Nedlib)

•Other applications:

•Vintage video game consoles

•System and software development

Page 31: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Digital ArchaeologyDigital Archaeology

•Recovery of damaged, incomplete or corrupted data.

•Two varieties:

•Media recovery (repairing physical objects)

•Data recovery (repairing logical structures)

•Recovery of damaged, incomplete or corrupted data.

•Two varieties:

•Media recovery (repairing physical objects)

•Data recovery (repairing logical structures)

Page 32: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

8mm Video Tape8mm Video Tape

• F16 Flight Recorder: Left Multifunction Display 8mm Video Tape

• Sent to Eastman Kodak Recording Systems Analysis Laboratory

• Some media recovery techniques

• Analog signal digitized after demodulation but prior top error correction and subjected to analysis

• Brightness and contrast

• Filtering

• Noise reduction

• F16 Flight Recorder: Left Multifunction Display 8mm Video Tape

• Sent to Eastman Kodak Recording Systems Analysis Laboratory

• Some media recovery techniques

• Analog signal digitized after demodulation but prior top error correction and subjected to analysis

• Brightness and contrast

• Filtering

• Noise reduction

Page 33: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Electronic records of the GDR

Electronic records of the GDR

• Electronic records on hard disk and 9-track tape

• Descriptive metadata incomplete or missing

• Began by identifying headers and data blocks

• Reconstructed documentary chain

• Recovered non-binary data first

• Repaired file structures and normalized date formats

• Developed software to decipher binary sequences and decompress data

• Electronic records on hard disk and 9-track tape

• Descriptive metadata incomplete or missing

• Began by identifying headers and data blocks

• Reconstructed documentary chain

• Recovered non-binary data first

• Repaired file structures and normalized date formats

• Developed software to decipher binary sequences and decompress data

Page 34: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Organizing for Preservation

Organizing for Preservation

•Long-term institutional commitment.

•Implementation of DL preservation systems requires equipment and programmers

•Digital objects may require regular re-investment in computing equipment

•Cooperation between organizational units

•Long-term institutional commitment.

•Implementation of DL preservation systems requires equipment and programmers

•Digital objects may require regular re-investment in computing equipment

•Cooperation between organizational units

Page 35: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

The OAIS Reference Model

The OAIS Reference Model

Page 36: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

OAIS Entities: Data Management & Archival

Storage

OAIS Entities: Data Management & Archival

Storage

Page 37: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

OAIS Entities: PreservationOAIS Entities: Preservation

Page 38: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

OAIS PointsOAIS Points

•Preservation functions primarily in storage and maintenance of metadata

•Preservation entity is policy-making, indirectly connected to storage & data management

•Migration is primary strategy for long-term access

•No explicit role or obvious place for emulation

•Preservation functions primarily in storage and maintenance of metadata

•Preservation entity is policy-making, indirectly connected to storage & data management

•Migration is primary strategy for long-term access

•No explicit role or obvious place for emulation

Page 39: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

RLG: Trusted Digital Repositories

RLG: Trusted Digital Repositories

•OAIS compliance

•Administrative responsibility

•Organizational viability

•Financial sustainability

•Technological and procedural suitability

•System security

7.Procedural accountability

•OAIS compliance

•Administrative responsibility

•Organizational viability

•Financial sustainability

•Technological and procedural suitability

•System security

7.Procedural accountability

Page 40: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

DSEP Process ModelDSEP Process Model

Page 41: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

DSEP 6: Emulation Model

DSEP 6: Emulation Model

Page 42: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

DSEP NotesDSEP Notes

•Preservation function has a direct interaction with archival storage.

•Preservation function can provide for emulation & migration.

•Richer description of supply and delivery model for libraries.

•Preservation function has a direct interaction with archival storage.

•Preservation function can provide for emulation & migration.

•Richer description of supply and delivery model for libraries.

Page 43: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Cornell Digital Preservation Management

Cornell Digital Preservation Management

Page 44: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Organizational InfrastructureOrganizational Infrastructure

•Organizational Infrastructure: Mandate and commitment to preserve digital assets.

•Policy Framework: A set of explicit statements that defines the organization’s level and nature of commitment and responsibility:•OAIS compliance•Administrative responsibility•Organizational viability•Financial sustainability•Technological suitability•System securityA.Procedural accountability

•Organizational Infrastructure: Mandate and commitment to preserve digital assets.

•Policy Framework: A set of explicit statements that defines the organization’s level and nature of commitment and responsibility:•OAIS compliance•Administrative responsibility•Organizational viability•Financial sustainability•Technological suitability•System securityA.Procedural accountability

Page 45: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

OAIS complianceOAIS compliance

•An explicit statement that confirms the organization’s commitment to complying with the Open Archival Information System (OAIS) standard.

•An explicit statement that confirms the organization’s commitment to complying with the Open Archival Information System (OAIS) standard.

Page 46: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Administrative responsibility

Administrative responsibility

•A high-level statement that demonstrates a commitment to track and comply with current and emerging standards embraced by the preservation community.

•A high-level statement that demonstrates a commitment to track and comply with current and emerging standards embraced by the preservation community.

Page 47: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Organizational viabilityOrganizational viability

•A mission statement and comprehensive policies that document and authorize the steps an organization undertakes to receive, store, preserve, and provide access to digital materials under its care, encompassing legal, fiscal, and ethical considerations and requirements.

•A mission statement and comprehensive policies that document and authorize the steps an organization undertakes to receive, store, preserve, and provide access to digital materials under its care, encompassing legal, fiscal, and ethical considerations and requirements.

Page 48: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Financial sustainabilityFinancial sustainability

•Accounting and budget policies and procedures that are part of a business plan to define and protect requisite resources for the digital preservation program.

•Accounting and budget policies and procedures that are part of a business plan to define and protect requisite resources for the digital preservation program.

Page 49: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Technological suitabilityTechnological suitability

•A set of principles, policies, and procedures that define the plan for developing and maintaining requisite hardware, software, expertise, and techniques to support and enable the digital preservation program, including adherence to relevant standards and industry best practice.

•A set of principles, policies, and procedures that define the plan for developing and maintaining requisite hardware, software, expertise, and techniques to support and enable the digital preservation program, including adherence to relevant standards and industry best practice.

Page 50: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

System securitySystem security

•A set of policy statements and procedures that confirm the organization’s commitment to maintaining a constant and appropriate level of environmental and online protection; surveillance; and risk detection, response, and mitigation to safeguard the integrity of digital assets.

•A set of policy statements and procedures that confirm the organization’s commitment to maintaining a constant and appropriate level of environmental and online protection; surveillance; and risk detection, response, and mitigation to safeguard the integrity of digital assets.

Page 51: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Procedural accountability

Procedural accountability

•A coherent and systematic means for documenting, sharing, and applying the set of policy statements and associated procedures and prevailing practice. These are often external to the organization itself.

•A coherent and systematic means for documenting, sharing, and applying the set of policy statements and associated procedures and prevailing practice. These are often external to the organization itself.

Page 52: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Technological InfrastructureTechnological Infrastructure

• basic data encoding

• file formatting and compression schemes

• storage media specifications

• media storage techniques

• storage media formatting

• file system specifications

• operating system environment

• storage hardware

• “playback" hardware and software

• basic data encoding

• file formatting and compression schemes

• storage media specifications

• media storage techniques

• storage media formatting

• file system specifications

• operating system environment

• storage hardware

• “playback" hardware and software

Page 53: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

Resources FrameworkResources Framework

•Budget line items commitments

•Designation of dedicated staff

•Strategic planning documents:

•Projected cost models

•Funding scenarios to sustain the program.

•Budget line items commitments

•Designation of dedicated staff

•Strategic planning documents:

•Projected cost models

•Funding scenarios to sustain the program.

Page 54: Preservation of Digital Content Jacob Nadal Head, Craig Preservation Laboratory Indiana University Libraries (812) 855-6281 | mailto:jnadal@indiana.edu.

ReferencesReferences• Chapman, S. (2003). "Counting the Costs of Digital Preservation: Is Repository Storage Affordable?"

Journal of Digital Information 4(2).

• Council on Library and Information Resources. (2002) Building a National Strategy for Digital Preservation: Issues in Digital Media Archiving Washington, DC: Council on Library and Information Resources.

• Kodak Photo CD: http://www.kodak.com/US/en/digital/products/photoCD.shtml

• Lupovici, Catherine and Julien Masanès, Metadata for Long Term Preservation. Bibliothèque Nationale de France. - Den Haag : Koninklijke Bibliotheek, 2000. - (NEDLIB Report series ; 2). http://www.kb.nl/coop/nedlib/results/NEDLIBmetadata.pdf

• LOCKSS Project: http://www.lockss.stanford.edu

• Murray, James D. Encyclopedia of graphics file formats. Sebastopol, CA: O'Reilly & Associates, Inc., 1994.

• OIAS Model and Related Information: http://ssdoo.gsfc.nasa.gov/nost/isoas/

• Ross, Seamus and Ann Gow. Digital Archaeology: Rescuing Neglected and Damaged Data Resources. London: Library and Information Technology Centre, 1999. Available at (Accessed Nov 17, 2004): http://www.ukoln.ac.uk/services/elib/papers/supporting/pdf/p2.pdf

• Rosenthal, David S. H.; Thomas Lipkis; Thomas Robertson; Seth Morabito "Transparent Format Migration of Preserved Web Content.” http://arxiv.org/abs/cs.DL/0411077

• Rothenberg, Jeff. (1999). Avoiding Technological Quicksand: Finding a Viable Technological Foundation for Digital Preservation. Washington, DC: Council on Library and Information Resources.

• Rothenberg, Jeff. An Experiment in Using Emulation to preserve Digital Publications. RAND-Europe. - Den Haag : Koninklijke Bibliotheek, 2000. - (NEDLIB Report series; 1). http://www.kb.nl/coop/nedlib/results/NEDLIBemulation.pdf

• Chapman, S. (2003). "Counting the Costs of Digital Preservation: Is Repository Storage Affordable?" Journal of Digital Information 4(2).

• Council on Library and Information Resources. (2002) Building a National Strategy for Digital Preservation: Issues in Digital Media Archiving Washington, DC: Council on Library and Information Resources.

• Kodak Photo CD: http://www.kodak.com/US/en/digital/products/photoCD.shtml

• Lupovici, Catherine and Julien Masanès, Metadata for Long Term Preservation. Bibliothèque Nationale de France. - Den Haag : Koninklijke Bibliotheek, 2000. - (NEDLIB Report series ; 2). http://www.kb.nl/coop/nedlib/results/NEDLIBmetadata.pdf

• LOCKSS Project: http://www.lockss.stanford.edu

• Murray, James D. Encyclopedia of graphics file formats. Sebastopol, CA: O'Reilly & Associates, Inc., 1994.

• OIAS Model and Related Information: http://ssdoo.gsfc.nasa.gov/nost/isoas/

• Ross, Seamus and Ann Gow. Digital Archaeology: Rescuing Neglected and Damaged Data Resources. London: Library and Information Technology Centre, 1999. Available at (Accessed Nov 17, 2004): http://www.ukoln.ac.uk/services/elib/papers/supporting/pdf/p2.pdf

• Rosenthal, David S. H.; Thomas Lipkis; Thomas Robertson; Seth Morabito "Transparent Format Migration of Preserved Web Content.” http://arxiv.org/abs/cs.DL/0411077

• Rothenberg, Jeff. (1999). Avoiding Technological Quicksand: Finding a Viable Technological Foundation for Digital Preservation. Washington, DC: Council on Library and Information Resources.

• Rothenberg, Jeff. An Experiment in Using Emulation to preserve Digital Publications. RAND-Europe. - Den Haag : Koninklijke Bibliotheek, 2000. - (NEDLIB Report series; 1). http://www.kb.nl/coop/nedlib/results/NEDLIBemulation.pdf