ResourceSync Quick Overview

Herbert Van de Sompel - ResourceSyncNISO Annual Update, ALA, Las Vegas, NV, 29 June 2014

http://www.openarchives.org/rs #ResourceSync

Herbert Van de SompelLos Alamos National Laboratory@hvdsomp - [email protected]

ResourceSyncA Quick Overview

The ResourceSync editors are: Martin Klein (LANL), Robert Sanderson (LANL), Herbert Van de Sompel (LANL), Simeon Warner (Cornell U.), Graham Klyne (Oxford U.), Bernhard Haslhofer (U. of Vienna), Michael L. Nelson (Old Dominion U.), Carl Lagoze (U. of Michigan)

ResourceSync was funded by the Sloan Foundation & JISC


ResourceSync

• Collaboration between NISO and the Open Archives Initiative, 2012-2013

• Funded by the Sloan Foundation and JISC

• Goal: Devise a specification for web-based resource synchronization• Subjects of synchronization: web resources identified by HTTP

URIs• HTTP as the communication protocol for synchronization• Servers involved in synchronization are on the web


Problem Statement

• Consideration:• Source (server) has resources that change over time: they

get created, modified, deleted• Destinations (servers) leverage (some) resources of the

Source

• Problem:• Destinations want to keep in step with the resource changes

at the Source

A Source’s Resources

A Source’s Resources Evolve over Time


Problem Statement

• Consideration:• Source (server) A has resources that change over time: they

get created, modified, deleted• Destination (servers) X, Y, and Z leverage (some)

resources of Source A

• Problem:• Destinations want to keep in step with the resource changes

at Source A

• Goal:• Design an approach for resource synchronization aligned

with the Web Architecture that has a fair chance of adoption by different communities

One to Many – Master Copy

Many to One - Aggregator

Metadata Harvesting


Solution Perspective

• Source communicates about the state of its resources:

• Publish inventory: snapshot of the state of resources at a moment in time

• Publish changes: enumeration of resource changes that occurred during a temporal interval

• Notify about changes: send notifications as changes occur


Solution Perspective – Communication Payload

• Payload of the communications:

• Always: HTTP URI of the resource

• For change communications: change type (create | update | delete) & datetime of change

• As needed: metadata and links pertaining to the resource


ResourceSync is Based on Sitemaps

• Sitemap is the document format used throughout the frameworko Used widely by web servers to advertise their resources to

search engines

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9”>

<url> <loc>http://example.com/res1</loc> <lastmod>2013-01-02T13:00:00Z</lastmod> </url>

<url> <loc>http://example.com/res2</loc> <lastmod>2013-01-02T14:00:00Z</lastmod> </url> …</urlset>


ResourceSync Extends Sitemaps

• Extensions to Sitemaps to support synchronizationo <rs:ln> for linkso <rs:md> for metadata

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9”> xmlns:rs="http://www.openarchives.org/rs/terms/”> <rs:ln …/> <rs:md …/>

<url> <loc>http://example.com/res1</loc> <lastmod>2013-01-02T13:00:00Z</lastmod> <rs:ln …/> <rs:md …/> </url> …</urlset>


Publish Resource List: Inventory at Tx

• In order to meet a Destination’s need for baseline synchronization, the Source may publish a Resource List

• A Resource List is an inventory, a snapshot of existing resources

• Per resource, it minimally provides the resource’s URI

• Process:- Destination obtains the Resource List- Destination obtains listed resources by their URI

Publish Resource List: Inventory at Tx

Resource List @Tx = { A ; B ; C }


<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/"> <rs:md capability="resourcelist" at="2013-01-03T09:00:00Z” /> <url> <loc>http://example.com/res1</loc> <lastmod>2013-01-02T13:00:00Z</lastmod> <rs:md hash="md5:1584abdf8ebdc9802ac0c6a7402c03b6" length="8876” type="text/html"/> <rs:ln href="http://example.com/meta/dc/res1" rel=”describedby” type=”application/xml"/> </url> …</urlset>

Resource List


Send Change Notification: Resource Changes as they Occur

• In order to meet a Destination’s need for incremental synchronization and low latency, the Source may send Change Notifications

• A Change Notification conveys resource change events as they occur

• For each event, it minimally lists the URI of the resource, the nature of the change, the datetime of the event

- Process:- Destination receives Change Notification- Destination obtains created/updated resources, removes

deleted resources


Send Change Notification: Resource Changes at Ta

Change Notification @Ta = { A updated @Ta }


Send Change Notification: Resource Changes at Tb

Change Notification @Tb = { D updated @Tb }


Send Change Notification: Resource Changes at Tc

Change Notification @Tc = { A updated @Tc ; B updated @Tc }


ResourceSync Notifications are Based on the PuSH Protocol

• ResourceSync builds on the PubHubsHubbub protocol (PuSH) used for syndication of Atom/RSS feeds

• Destination subscribes to a Source’s notifications via an intermediate Hub


Modular Framework

• ResourceSync specifies additional synchronization capabilities• Source decides which capabilities to support based on local

and community requirements


Follow Your Nose Discovery

• ResourceSync specifies mechanisms for Destinations to determine whether and how a Source supports the framework• Based on conventions for web discovery and documents

that detail the level of support


Follow Your Nose Document Navigation

• ResourceSync specifies mechanisms to navigate the various documents involved in the framework• Based on following typed links


Status

• Core specification (pull) is ANSI/NISO Z39.99-2014

• Notification (push), Archives specifications in beta

• Growing implementation interest motivated by the need to move beyond metadata (OAI-PMH)

• Tools will be listed at http://www.openarchives.org/rs/toc


Specifications: http://www.openarchives.org/rs

Herbert Van de SompelLos Alamos National Laboratory@hvdsomp - [email protected]

ResourceSyncA Quick Overview

The ResourceSync editors are: Martin Klein (LANL), Robert Sanderson (LANL), Herbert Van de Sompel (LANL), Simeon Warner (Cornell U.), Graham Klyne (Oxford U.), Bernhard Haslhofer (U. of Vienna), Michael L. Nelson (Old Dominion U.), Carl Lagoze (U. of Michigan)

Google Group: https://groups.google.com/d/forum/resourcesync

ResourceSync Quick Overview

Technology

Transcript of ResourceSync Quick Overview