PAWN Progress

July 06, 2006

Overview of changes

New flexible environment for setting up and managing interactions between producers and the archive

Domains to organize accounts, record organization, and packages

Definable roles that can be flexibly combined and assigned to accounts

Interfaces for designing package builders and archival resource gateways

Components

Bulk Transfer

Scheduler

Producer Managed Archive Managed

Management Server

Producer data suppliers

Receiving Server

DistributedArchive

Schedule Request

AuthenticationPackage Information

Ingestion Status

Validation Services

Overall Organization

Producers organized into domains, each domain containing a record schedule negotiated with the archive.

Each domain contains a hierarchy of the types of data and record sets (convenient groupings from the record schedule).

An end-user operates within a domain with record sets associated with the account.

Package Workflow

1. Client selects a record set to use as a package template.

2. A package is built locally and then transferred to a PAWN receiving server.

3. Optionally lock package to signal complete submission.

4. Review and possible reject items.

5. Transfer items from PAWN into final archive.

6. Remove package from PAWN.

Record Organization

Previous version had one hierarchy with attachment points for items as leaf nodes.• Did not allow for linking of related leaf nodes

• Hierarchy performed multiple roles, record organization and administrative organization.

Current version based on Record Sets. • Separate administrative structure and record

structure.

• Record Sets are template packages.

Record Organization

Each domain contains a record schedule• Record schedule is a hierarchy containing authorities as

endpoints Domains also contain an organizational hierarchy.

Offices, projects, etc. Record Sets

• group of authorities from the record schedule• attached to a point in the record hierarchy.• Have access permissions• Presented to producers as package templates

College of Sciences Domain

Office of the DeanChemistryMathematicsPhysicsComputer Science

oBusiness OfficeoResearch GroupsoLabs

……

Record Sets

Record Schedule

•AdministrativeoStrategic and

Performance PlansoAppointment and

PromotionoPolicies and CommitteesoAlumni Affairs

•FinancialoContracts and GrantsoPayrolloDonations

•Publication ReportsoTechnical Reports

- Archiving RulesoPresentationsoPosters

Record SetName: Research ResultsNote: Reports, presentations,

and other published research results

Allowed Accounts

Record Schedule MappingPresentations

oPresentations Technical Reports

oTechnical Reports

Domains

Offices of the President and Vice-Presidents

College of SciencesCollege of EngineeringCollege of MedicineCollege of Arts and

HumanitiesCollege of Behavioral

and Social Sciences…..

Record Set Sample

Flexible Account Roles

Previous version had fixed accounts, producer, manager and administrator.

Current version allows actions in PAWN to be grouped into roles.

Each account is assigned a role. Sample actions in PAWN

• Record Set/Schedule management

• Package creation/deletion/modification

• Account management

SAML Usage

SAML Assertions are issued by managers• Contain manager namespace, domain, username

• Contain list of allowed actions by the client

• Contain client’s public key (holder-of-key)

• Signed by manager

SAML Assertions authenticate and authorize a client for archive-side services.

PackageManagement

Calls

Archive Management

Calls

Administrative Metadata

Calls

ArchiveProducer

Call Overlap

Sample SAML Assertion

<Assertion AssertionID="b5ad81157714985340250bc43d704c44" IssueInstant="2006-07-05T15:07:33.898Z" Issuer="http://umiacs.umd.edu" MajorVersion="1" MinorVersion="1">

<Conditions NotBefore="2006-07-05T09:07:33.898Z" NotOnOrAfter="2006-07-05T15:07:33.898Z"></Conditions> <AttributeStatement>

<Subject> <NameIdentifier NameQualifier="umiacs">umiacs:toaster</NameIdentifier> <SubjectConfirmation> <ConfirmationMethod>urn:oasis:names:tc:SAML:1.0:cm:holder-of-key</ConfirmationMethod> <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#"> <ds:X509Data> <ds:X509Certificate>MIIDxjCCAy+gAwIBAgIDEAACMA0GCSqGSIb3DQEB....</ds:X509Certificate> </ds:X509Data> </ds:KeyInfo> </SubjectConfirmation> </Subject>

<Attribute AttributeName="package_item" AttributeNamespace="http://umiacs.umd.edu/adapt/saml"> <AttributeValue>view</AttributeValue> <AttributeValue>create</AttributeValue> <AttributeValue>modify</AttributeValue> </Attribute> ... ... </AttributeStatement>

SAML Assertion (cont)

<ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#">

<ds:SignedInfo> <ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"></ds:CanonicalizationMethod> <ds:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"></ds:SignatureMethod> <ds:Reference URI="#b5ad81157714985340250bc43d704c44"> <ds:Transforms> <ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"></ds:Transform> <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"> <ec:InclusiveNamespaces xmlns:ec="http://www.w3.org/2001/10/xml-exc-c14n#" PrefixList="code ds kind rw saml samlp typens

#default"></ec:InclusiveNamespaces> </ds:Transform> </ds:Transforms>

<ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"></ds:DigestMethod> <ds:DigestValue>r7C4oNmlf4h8cXi1dGU+MIGmGbM=</ds:DigestValue> </ds:Reference> </ds:SignedInfo>

<ds:SignatureValue>Rstfd1HKTe68WLQrgAvmS5hDm7SVbXnEgMlotW3aiu....</ds:SignatureValue> <ds:KeyInfo> <ds:X509Data> <ds:X509Certificate>MIIDyjCCAzOgAwIBAgIDEAABMA0GCSqGSIb3DQ....</ds:X509Certificate> </ds:X509Data> </ds:KeyInfo>

</ds:Signature></Assertion>

Package Creation

Packages are built using a Record Set as a template.

Each category in a Record Set has a hierarchy of manifests attached.

Manifests are an abstraction of underlying METS documents

Custom package builders use manifest interface.

ManifestNamespaceTypeDescriptive Name

DataTypeDescriptive NameBits

Metadata…

Manifest…

MetadataTypeBitsName

Package Builders

Default Builder• Create files and folders• Attach descriptive

metadata to files or folders

ICDL Builder• Create ‘books’ with

dublin core metadata• Uses ICDL database as

source for book list and metadata

Package Scheduling and Submission

Scheduler decides which receiving server to store a package

Condor classad system used• Receiving server

periodically publishes available resources

• Client request space.

Client

Receiver

Scheduler

2. Evaluate classad

1. Space Requirements

3. Create Reservation

4. Allocated Server

5. Package Transfer

Receiver Classads

Publishing into Archival Resources

PAWN provides an interface for registering gateways into archival resources

Gateways provide:• Configuration gui• Client gui• Mover to transfer data from

PAWN to archive PAWN provides:

• Configuration storage• Access to all items in a

package• Access to contextual

information about a package• Infrastructure for storing and

loading gateway drivers.

SRB Publishing

PAWN Package

SRB Gateway

SRB

5. GUID or Path3. Package ItemsArchival Context

4. Package ItemsPAWN Client 2. SRB Path & item list

PAWN Scheduler1. SRB Configuration

Screenshots

Client Interface

Configuration Interface

Resulting Log Entry

XFDU publishing

Create XFDU compatible Information Packet. XFDU is similar to METS.

• Separate data definitions from structural information

• Similar file attributes (size, checksum, etc..) PAWN mapping

• InformationPackageMap contains ContentUnits to recreate the hierarchy of data in a PAWN package.

• DataObjects register individual files.

• XFDU manifest and data files combined to form an Information Package.

Demo