1

Maine GeoArchives Project and Maine GeoArchives Project and the Digital Archives Conceptthe Digital Archives Concept

Presented at NAGARA, SAA, CoSA joint conference, Washington, DC, August 3, 2006

Jim Henderson, Maine State Archivist

2

Digital Archives GoalsDigital Archives Goals

Preserve permanently valuable State of Maine official records that are in digital form, either "born digital" or digitized for access, and

Provide continuing digital access to these records.

3

Digital Archives ContextDigital Archives Context

State has spent a billion dollars in the past fifteen years creating systems to produce digital records.

Often to replace "legacy" systems whose records are not accessible: media deterioration or obsolescence, software or O/S obsolescence, inadequate documentation.

Little has been done to protect the long-term and permanent value of these assets.

4

The Problem - ExamplesThe Problem - Examples State Planning Office data on 8” floppy disks: no longer readable

GIS data: deleted by agencies that have no current need

Human Services “scanned” vital records: inaccessible, indexing database corrupted.

E-mail containing policy level material: either deleted or exists on obsolete e-mail systems no longer supported.

Hundreds of thousands of indexes to marriage records 1967-1976: lost when Human Services' data became unreadable.

Agencies embarrassed, A.G. concerned, Governor very concerned when legal “discovery” and FOI requests bungled in 2006!

5

Archives Digital ExperienceArchives Digital Experience

1999 Digital Records Management Plan from “Camp Pitt” process and NHPRC

E-mail guidance for State employees since 1997

Scanned/cataloged hundreds of documents for Maine on-line “museums” and learning environments

Mass digitization: 19,000 images from archival microfilm - 2004

Education of Maine IT professionals and other agencies on need to retain selected digital records

GeoArchives GIS preservation project 2005, now institutionalized

Maine Digital Archives Planning - 2005-2006

6

Overall StrategyOverall Strategy

Born Digital - targets of opportunity

– GIS

– E-mail

Digitized Analog

– Media: paper, film, analog tape

– For access: researchers, agencies, Archives staff

7

Digital Archives ProcessDigital Archives Process

8

GeoArchives Project Goal, ObjectivesGeoArchives Project Goal, Objectives

Goal Digital system, based on current research and standards,

to preserve and provide access to, State GIS records having permanent value.

Objectives– Immediate reduction in the loss of archival GIS records – A prototype for continuing development and application

for all State and local GIS records.– A model for others to critique, improve, apply

9

Project Scope - LimitedProject Scope - Limited

Preservation– Data (layers, images) in GeoLibrary only

• A large subset of all GIS records in agencies• Vetted for FGDC metadata compliance

– Select Test Data from GeoLibrary• Relatively stable - municipal boundaries• Relatively dynamic - E911 Roads• Orthophoto Imagery from “Fly Overs”

Access– Use existing GeoLibrary– Compare layer status (boundaries, roads) at any historical time– Overlay different layers (boundaries 2005; roads 2010)

10

GeoArchives as OAISGeoArchives as OAIS

11

Project ProcessProject Process

Research Project Team - true collaboration User needs - survey Functional Requirements

– for the project– for the outcome

Consultants - multiple Metadata Requirements

– Dublin Core, MARC 21, FGDC– Crosswalks FGDC v. archival standards

12

Project Functional RequirementsProject Functional Requirements

Identify GIS administrative records Set appraisal level - attributes, layers Appraise data layers Determine additional metadata needs Provide for data integrity and security Provide access to archival GIS data Define Archive’s administrative procedures

Functional requirements document:http://www.maine.gov/geoarch/Project%20Proposal%20and%20Documents/Frgeo08.doc

FOR MORE INFO...

13

FGDC Metadata for Appraisal and FGDC Metadata for Appraisal and Archival DescriptionArchival Description Originator [Administrative

records]

Title [Unique Identifier]

Series_Name [Descriptive name]

Abstract [Legal, informational, evidential value; related digital data]

Purpose [Evidential value]

Time_Period_InformationBeginning_Date [Existing archival layer]

Ending_Date [Existing archival layer]

Status Progress [Periodicity of accession]

Maintenance_and_Update_Frequency

Point_of_Contact Contact_Organization_Primary

[Administrative records]

Contact_Address [Administrative records]

Overview_Description

[Legal, informational, evidential value]

Entity_And-Attribute_Overview [Legal, informational, evidential value]

Use_Constraints [Access restrictions]

Source_Information Originator [Scope and content accuracy]

Contribution [Scope and content accuracy]

14

Additional Archival MetadataAdditional Archival Metadata

Creator’s content in FGDC Metadata– Assess completeness for archival description– Enhance as needed

Archive’s content in FGDC Metadata– Citation - Maine State Archives as data owner

– Keywords - Library of Congress Thesaurus

– Contact - MSA phone, address ,etc.

– Process Description - description of accession process

– Metadata Access Constraints - notice “no changed

permitted to archival version”

15

Appraisal ProcessAppraisal Process

16

Outcomes: Retention FormatsOutcomes: Retention Formats

Preservation– All formats likely to change

– Embed archival records in systems holding critical operational records

– Migrate archival records along with others

– Export to XML initially, new solutions as needed

Access– ESRI - the 900 lb. gorilla, with tools

– Exportable to non-ESRI formats

17

Manual for Managing the GeoArchivesManual for Managing the GeoArchives

Appraisal Process– Key metadata fields– Decision-making process

Accession Process– Review current metadata– Add archival metadata to change ownership– Administrative Records– Post Accession (error correction)

18

Tools Required by ArchivesTools Required by Archives

Storage (ArcSDE versions 9.2+?)– Data storage for “snapshots”– Use SDE Archive functions to track changes

Management (ArcCatalog)– Metadata style sheets to support appraisal– View complete metadata for accession– Editing to add archival metadata fields

GIS Viewer (many free options)– View samples of data during appraisal and accession– Provide researchers access to GeoArchives

19

Maine GeoArchives: StatusMaine GeoArchives: Status

A model for Digital Archives: design, experience

“Snapshot” layers on Data Catalog

http://megis.maine.gov/catalog/

On-going assessment for “continuous layer history” solution

Manual for Managing the GeoArchives

Receiving update alerts for data layers

Appraisal and accessioning continues

20

The The DevilDevil - Lessons on Details - Lessons on Details

communications with IT partner: clear and continuous

documentation of applications supporting the GeoArchives: complete and understandable

documentation of servers hosting archival layers - location and physical controls: complete, understandable

metadata for layers accessioned must be complete and consistent with the GeoArchives Manual

Archives staff specifically assigned to GeoArchives maintenance

21

Maine GeoArchives Web SiteMaine GeoArchives Web Site Agendas, Minutes Project Documents - raw and refined

– Functional Requirements– Manual for Managing the GeoArchives– Administrative Records Schedule– Consultant reports– Project Final Reports

GeoArchives Web:

http://www.maine.gov/sos/arc/GeoArchives/geoarch.html

FOR MORE INFO...