SEDAC Long-Term Archive Development

Robert R. Downs

Socioeconomic Data and Applications Center

Center for International Earth Science Information Network (CIESIN)The Earth Institute, Columbia University

Prepared for presentation to the 2010 ESIP Federation Winter Meeting

Washington, DC January 5-7, 2010

SEDAC Long-Term ArchiveMission Statement

The SEDAC Long-Term Archive acquires, preserves, and maintains the content of selected high-quality data, data products, documentation, and services relevant to human dimensions of global change in a digital form to support the discovery, access, and use of archived resources by scientific, educational, and decision-making communities for at least the next 50 years.

2

Developing the LTA to Meet OAIS Requirements

• OAIS 3.2.1 Negotiate for and accept appropriate information from Producers.– LTA SIP preparation includes review of AIP for completeness.

• OAIS 3.2.2 Obtain sufficient control for Long-Term Preservation.– LTA appraisal criteria includes review for legality and confidentiality.

• OAIS 3.2.3 Determine the Designated Community.– LTA appraisal criteria includes identification of potential usability and use of data

• OAIS 3.2.4 Ensure information is understandable to Designated Community. – LTA appraisal criteria includes ensuring that the data is documented for accessibility.

• OAIS 3.2.5 Follow documented preservation policies and procedures.– Draft LTA policies and procedures are being reviewed by the LTA Board.

• OAIS 3.2.6 Make the preserved information available to Designated Community.– LTA Board designates the dissemination level to be assigned to data when approved for

accession.• Conducting a self-assessment of the LTA as a trustworthy digital repository.

– Using the Trustworthy Repositories Audit and Certification: Criteria and Checklist (OCLC & RLG, 2007).

LTA Governance and Management

• LTA Board– Approves the Selection Criteria for Submission of SEDAC Data to the LTA– Approves the submission of recommended data or information resources that meet

the LTA selection criteria

• LTA Manager– Reports to the LTA Board– Responsible for development and operations of LTA systems, staff and

procedures, to ensure the stewardship of LTA holdings

• LTA Staff– Report to the LTA Manager– Accession and maintain LTA holdings in accordance with LTA procedures– SEDAC Engineering staff support infrastructure development and maintenance

Adapted from SEDAC Long-Term Archive Implementation Plan (Draft revised 2009)

Contingency Plan: SEDAC Long-Term Archive Governance and Management

• Current LTA Board membership:– represented by the SEDAC Project Scientist, the SEDAC Systems

Engineer, and the SEDAC LTA Manager, – two representatives designated by the Earth Institute, and – two representatives designated by the Columbia University Library.

• In the event of a lapse in SEDAC funding:– Libraries will replace chair and one of the SEDAC members– CIESIN will name the other SEDAC member=> Columbia University Libraries will have majority of members– Columbia University will appoint the Long-Term Archive Manager and

other staff as needed

Adapted from SEDAC Long-Term Archive Implementation Plan (Draft revised 2009)

Selection Criteria for LTA Data Appraisal

Scientific or Historical Value– citation, research, and educational use as published in refereed scientific

publications/reports from recognized committee of scientists Potential Usability and Use

– evidence of usability, usefulness, and sufficient usage by the community interested in human dimensions of the environment. Adequate evidence indicate potential for future use justifies costs of long-term archiving

Uniqueness of Data (non-redundant stewardship)– not being preserved in any form in another archive and is at risk of loss if not

accessioned into the Long-Term ArchiveRelevance to LTA Mission

– currently endorsed or approved by community interested in human interactions in the environment. For the short-term, relevance includes content germane to SEDAC mission and SEDAC strategic plan

Documented for Accessibility– completeness and correctness of documentation to facilitate future discovery,

access, and useTechnological Accessibility (feasibility)

– received in format meeting technical criteria for the Service Level designated for the resource

Legality and Confidentiality– unrestricted permissions for preservation and future dissemination. No information

that is confidential or prohibited from dissemination Non-Replicability

– data replication not feasible, excessively costly or prohibitive

LTA Levels of Service for Preservation of SEDAC Data

• Preserve Content in Original Formats: – Content is maintained in Original Formats on accessible system for the

specified retention period.

• Preserve and Maintain Content in Supported Formats: – Content received in Supported Formats is maintained on accessible system and

is migrated to current Supported Formats.

• Supported Formats:– ASCII Text (txt, xml, html)– Comma Separated Values (csv)– Image Files (png, jpg, tif, gif)– Portable Document Format (pdf)

LTA Levels of Service for Dissemination of SEDAC Data

• Restricted Dissemination: – The resource and its Dissemination Information Package (DIP) are not accessible by the public. – The discovery metadata for the resource is included in the LTA restricted access catalog. – Access to the restricted resource is granted in compliance with the restrictions specified for the

resource.– Limited user support is provided for a restricted resource that is authored by SEDAC in compliance

with the restrictions specified for the resource.– The use of restricted resources and services is evaluated and reported.

• Public Dissemination: – The Dissemination Information Package (DIP) for the resource is freely accessible by the public in

digital form. – The discovery metadata for the resource is included in the LTA public access catalog. – Contact information for user support is provided on the LTA public access catalog.– Responses are provided for legitimate requests to correct publicly disseminated documentation or

access capabilities.– Answers or referrals are provided for scientific and technical questions about publicly disseminated

resources.– Changes are described for publicly disseminated resources or services.– The use of publicly disseminated resources and services is evaluated and reported.

Preservation and Dissemination Services for SEDAC Data Approved for Accession to the LTA

Data Set Title Pub Date Preservation Service Dissemination Service

Environmental Treaties and Resource Indicators (ENTRI) - The Update of the Treaty Status Data

1998 Preserve Content in Original Formats

Public Dissemination

HALOPH: A Data Base of Salt Tolerant Plants of the World

1989 Preserve Content in Original Formats

Public Dissemination

Environmental Subset of Collection of Multilateral Conventions at the Fletcher School of Law and Diplomacy

1992 Preserve and Maintain Content in Supported Formats

Public Dissemination

Freedom in the World (1995-1996) 1995, 1996

Preserve Content in Original Formats

Public Dissemination

World Resources 1996-97 1996 Preserve Content in Original Formats

Public Dissemination

World Resources 1998-99: A Guide to the Global Environment: Environmental Change and Human Health (Data Tables)

1998 Preserve Content in Original Formats

Public Dissemination

Gridded Population of the World (GPW) Version 1 1995 Preserve and Maintain Content in Supported Formats

Public Dissemination

Gridded Population of the World (GPW) Version 2 2000 Preserve and Maintain Content in Supported Formats

Public Dissemination

Gridded Population of the World, Version 2: Ancillary Data

2001 Preserve and Maintain Content in Supported Formats

Public Dissemination

Integrated Model for the Assessment of the Greenhouse Effect (IMAGE) 2.0 and Associated Scenarios

1994 Preserve Content in Original Formats

Public Dissemination

SEDAC Active Archive and Long-Term Archive: Data Organization

SEDAC Collections in Digital Data Repository

SEDAC Long-Term Archive Data and Information Products

Public Access to Data and Information

Restricted Access to Data and Information

Public Access to Data and Information

Restricted Access to Data and Information

SEDAC Active Archive Data and Information Products

Active Archive is for near-term dissemination with high levels of service. Primary users are discipline-specific scientists.Long-Term Archive is for the 50 – 100 year preservation time-frame with different expectations for levels of service.

Continuing Self-Assessment of LTA

• Conducting an initial self-assessment of the LTA using the Trustworthy Repositories Audit and Certification: Criteria and Checklist (OCLC & RLG, 2007).

• Reviewing the Draft Recommended Standard for Audit and Certification of Trustworthy Digital Repositories (July 2009 Red Book, CCSDS) for use as a self-assessment instrument.

11