6/21/2013

1

Archives of Michigan

Electronic Records Preservation Solutions

Archives of Michigan Mission: identify and

permanently preserve public and private records that document significant government activities and historic eras/events.

Historically significant public records: Document how

government operates Protect the rights of

citizens Document society or

important events

Michigan’s first constitution, 1835

100 Years of Preserving History Archives of Michigan was established in 1913, with

collections dating back to 1792

The first 100 years focused on the preservation of traditional records: paper, photographs, maps, etc.

The next 100 years will add electronic records to collections

6/21/2013

2

Michigan’s ER Program History 1977: initial evaluation study

1986 – 1994: consultants’ reports

1996: ER archivist hired

1999 – 2012: grant projects to develop ER preservation tools

2012: Michigan becomes the first state to acquire Preservica

Archives’ Responsibility Preserve historical records, regardless of format

Traditional Records Acid free containers

Environmentally controlled storage

Secure physical storage and reference space

Electronic Records Secure electronic storage with backup

Technology monitoring

On-going preservation activity

Access

Understand Your Operating Environment

Cohasset Survey Private records management consulting firm

Partnership with ARMA and others

Bi-annual survey of private and public sector

Identify the status of electronic records management programs

2011/2012 survey results (7th version) now available

www.cohasset.com

6/21/2013

3

2009 Cohasset Survey

www.cohasset.com

2009 Cohasset Survey

Survey results illustrate that most IT professionals do not

understand the need to migrate electronic records over time to

keep them accessible IT tends to focus on storage capacity, instead of format

How will IT help their customers retain records in accordance with

various requirements?

www.cohasset.com

Threats to Preservation Records in all formats need to be actively protected

Traditional Records Environment, elements, pests, etc.

Electronic Records Obsolescence, viruses, breaches, corruption, etc.

6/21/2013

4

Preservation Challenge Michigan needed:

System for storing and preserving electronic records

Employees with the skills required to use and manage the system

Funding

Michigan’s challenge is not unique Michigan did not want to work independently to solve this

problem

Michigan collaborated with others and monitored international research

Michigan’s ER Preservation Projects Records Management Application Pilot Project (RMA)

1999: NHPRC ($190,000) Installed and evaluated RMA software on user desktops

Preserving Electronic Records Stored in a RMA (PERM) 2001: NHPRC ($160,000) Developed and tested functional requirements for preservation

Persistent Archives Testbed (PAT) 2003: NHPRC ($242,500) Conducted a case study to preserve 30 years worth of elections

databases Distributed Custodial Archival Preservation Environments

(DCAPE) 2008: NHPRC ($257,800) Initiated design of a trusted digital repository service for preserving

electronic records

Michigan’s SERI Self-Assessment 2012

6/21/2013

5

Archival Preservation Activities Accessioning

Taking custody of archival records and responsibility for preservation

Storage Ensuring records are safe and secure

Preservation Keeping records accessible over time

Access Helping researchers/public use the records

ER Preservation Models Open Archival Information Systems (OAIS) Reference Model

Published January 2002

Developed by the Consultative Committee for Space and Data Systems

Adopted as ISO 14721 (2003)

Maps workflow of ER from creator to archival researcher

http://public.ccsds.org/publications/archive/650x0m2.pdf

OAIS Processes Ingest

SIP: Submission Information Package

Archival Storage AIP: Archival Information Package

Data Management

Administration

Preservation Planning

Access DIP: Dissemination Information Package

6/21/2013

6

Michigan’s Functional Requirements Compliant with ISO standards

Affordable

Can be implemented and maintained without internal IT support, and can be used by staff who do not have technical expertise No hardware purchase, installation or maintenance: cloud

storage preferred

No software installation and maintenance

No programming or coding: must function out of the box with an intuitive user interface

Secure storage with backup/disaster recovery

Michigan’s Functional Requirements Automated ingestion processes

File format identification

Virus check

Fixity check

Automated preservation processes File format monitoring

On-going fixity checks

Metadata maintenance

Automated dissemination processes Search

Retrieval of a duplicate

Exit strategy for records and metadata

Tessella U.K. company

“World leader in digital preservation solutions, technology, consulting and research.”

Safety Deposit Box (SDB) software first developed in 2003

Customers in 9 countries across 3 continents

Launched Preservica in June 2012

www.digital-preservation.com

6/21/2013

7

Timing is Everything Tessella launched Preservica in June 2012

Michigan contacts KY and MN about their pilots

Michigan demonstration and Q&A: August 2, 2012 Tessella offers Michigan ½ price deal, if contract executed by the

end of August

Archives of Michigan decides it wants to get Preservica

Michigan’s State Archivist meets new director that same afternoon

Acquisition is put on a fast track, and is completed October 10, 2012

Safety Deposit Box Key Features Ingest

Data Management

Storage

Access

Preservation Planning and Action

Administration

Safety Deposit Box Technology SDB is designed for archivists, to address electronic records

preservation

Tessella partners with international research projects addressing various aspects of ER preservation Company has the resources to monitor the state of the art

Many open source tools were developed via these projects

SDB provides a unified interface and workflow for using these tools

SDB will evolve as new tools are developed and user needs are defined

6/21/2013

8

Preservica SDB software as a service

Software is accessed via internet, and is not installed locally

Client does not install upgrades

Runs on Amazon Web Services (AWS) Cloud computing

Multiple redundant copies

No hardware to install and maintain

6 replications accurate to 99.99999999999%

Fees for service Licensing

Storage

Michigan’s ER Collections Stored on a network drive with a nightly backup

6.65 GB

563 Folders, 9,711 Files

Accessioned from 7 SOM departments, plus Supreme Court

13 collections

Formats: databases, digital images, word processed documents, etc.

6/21/2013

9

Preservica Governance Governance Team:

Employee responsibilities/permissions

Procedures for use

Metadata requirements

Technology monitoring and evaluation

Collections Review Team: Make preservation decisions specific to each collection

Develop preservation standards

Future Objectives Acquire more collections for preservation

Standardize use of Preservica

Offer a fee-based preservation service for government agencies with long-term and permanent records that do not belong to the Archives of Michigan Potential service models:

Preserve and return

Preserve and store

Preserve, store, provide access

Michigan ER Preservation Workflow

6/21/2013

10

Preservica Ingest Create SIP

Upload SIP

Ingest SIP workflow

AIP loaded into repository

Note: currently ER collections are still maintained on a local network drive (with back-up) post-ingest.

Michigan Lottery, Press Releases

6/21/2013

11

6/21/2013

12

AIP in Preservica

Files in the Collection

Preservica Collection Metadata Basic metadata is ingested with the SIP

Michigan is using Dublin Core metadata

Metadata is edited after AIP is created

MARC record is created Collections in formats that are not currently accessible

Do not get a MARC record

Identified on the “Trouble Collections List”

Researchers are allowed to access them, but we do not claim that they are usable

6/21/2013

13

Dublin Core Metadata--MichiganField Name Purpose NotesTitle Name given the resource Record series titleCreator Person or organization responsible for

the contentDepartment and agency name

Publisher Those responsible for making the resource available

Do not use

Contributor Those who added to the content Do not useDate When the resource was made

availableAccession Date, format as mm/dd/yyyy

Coverage Inclusive dates of resource First instance--dates, second instance--geographic location (if applicable)

Description Synopsis of content First instance--series description for retention schedule with agency code and item # in parentheses at the end, second instance--processor’s description of the contents of the accession

Subjects Topics Need a controlled vocabularyLanguage English Rare exceptionsSource Where the content is originally derived

fromArchives Accession Number

Resource Identifier Numerical identifier for the content First instance--Preservica, second instance--if DIP is published online, URL

Type A category for the content Need a controlled vocabulary: image, text, database, audio, etc.

Format File format of the majority of the files Use the PUID identified by PreservicaRelation How the content relates to other

resourcesOptional: could be the law that authorized the creation of the records, similar records series, etc.

Rights Security Level Public, Internal Use, Confidential, Restricted

Metadata After Ingest

Adding Dublin Core Schema

6/21/2013

14

Metadata After Cataloging

MARC Record

MARC Record

6/21/2013

15

Catalog Record

Preservica Migration Identify file formats in the repository

Determine if a migration pathway is available

Initiate workflow to migrate the ER and ingest the new manifestation

File preservation report

6/21/2013

16

Create Preservation Workflow

Create Preservation Plan

6/21/2013

17

Next Steps Continue to explore features of the software

Web crawling workflow

Transfer agreements

Access options

Security issues for confidential records

Recommend improvements to Tessella

Renew contract for another year

Secure long-term funding

Acquire and preserve more collections

Archives of Michigan702 W. Kalamazoo StreetLansing, MI 48913(517) 373-3559archives@michigan.gov

www.michigan.gov/archivesofmi

www.seekingmichigan.org