Indo-US Workshop, June23-25, 2003

Building Digital Libraries for Communities using Kepler

Framework

M. Zubair

Old Dominion University

Indo-US Workshop, June23-25, 2003

Outline

Introduction

Kepler

Related tools

Conclusion

Indo-US Workshop, June23-25, 2003

Model and AssumptionsCommunity: group of individuals with similar interest, for example researchers working in the area of bio-informatics.

Different communities would use different metadata for describing their content.

Communities can be organized in a hierarchical structure, for example

Researchers working in the area of computer science Researchers working in Digital Libraries

Researchers working in Video Digital Libraries

Collections and associated services like searching, publishing, can be built at different levels of the hierarchy. Services for specialized communities (lower level) works on richer metadata specific to a community

Availability of technical expertise (in terms of software installation, etc.) and financial resources varies at different levels. At the specialized level one can not afford to have software that are complex to install and maintain.

Indo-US Workshop, June23-25, 2003

OAI-PMH and Communities

Use of OAI-PMH to facilitate building services at different levels.

For example, archives at the individual level (lowest level) can be harvested using OAI-PMH to build services at the next level.

Indo-US Workshop, June23-25, 2003

Community Building

CommunalRegistration

Service

CommunalService Provider

GroupServiceProvider

GroupServiceProvider

GroupServiceProvider

GroupRegistration

Service

OAICompliantRepository

PublishingTool

OAICompliantRepository

PublishingTool

KeplerArchivelet

GroupDigital Library

Indo-US Workshop, June23-25, 2003

Needs

Need for a framework and tools for defining, describing and supporting the publication/dissemination requirements of an individual belonging to a community.

Need for tools and software to create an out-of-the-box solution for deploying communal digital libraries for diverse communities.

Indo-US Workshop, June23-25, 2003

Challenges

What are the resource publication and dissemination needs of a diverse sample of communities

How much do they overlap/differ from each other, and what are the corresponding software requirements?

How can we configure search and authoring tools dynamically to reflect the richer metadata set that defines a community?

How do we maintain the simple self-installable software concept in the face of complex, hierarchical communities that differ in procedures for author and resource discovery roles?

How to maintain freshness of documents and metadata from archive lets.

Indo-US Workshop, June23-25, 2003

Kepler Framework and P2P

RegistrationService

ServiceProvider

Broker

Bob Carol

Where is "QoSIssues ..." paper

by John Doe ?Carol has it

Retireving "QoS Issues ...."paper by John Doe

Indo-US Workshop, June23-25, 2003

Issues Security and NAT (Network Address Translation)

In many instances, an archivelet is behind a firewall or NAT proxy, which makes it difficult for the service provider to harvest the archivelet.

Availability Archivelet in home environment (often up and down), use of dynamic

IP address. Freshness

Large number of archivelets with sparse changes. This doesn’t fit well with OAI-PMH’s “poll”-based mode.

Full text vs. Metadata As the archivelet is not up all the time, it is desirable to harvest full-text

documents as well to improve the availability of full-text to end-users.

Indo-US Workshop, June23-25, 2003

Architecture – Current Status

Indo-US Workshop, June23-25, 2003

Features Supports multiple metadata formats (OLAC,

OAI_DC). And working on adding support for complex objects.

Extensible: seamlessly adding support for more metadata formats.

Flexibility: API Specifications among various modules.

Optimizing the harvesting: the harvester gets notified by the archivelets when there are updates. Thus it can avoid needless harvesting.

Indo-US Workshop, June23-25, 2003

Metadata EditorsDC & OLAC

Indo-US Workshop, June23-25, 2003

Feature.. continued Allows the group manager to override the default

constraints in the metadata schema. He can:- Specify which elements are mandatory and which are

optional- Define regular expressions for validating the elements- Control the enumeration options for the elements by

adding more options or removing existing options- Defining the structure of the complex objects All these settings are downloaded by the archivelets and

the constraints are enforced by the metadata editors.

Indo-US Workshop, June23-25, 2003

Overriding the schema

Indo-US Workshop, June23-25, 2003

Server-Side Kepler Best for users behind firewalls and NAT. No need for software download. Just a web browser and an Internet connection The user creates a virtual archivelet on the group server. The user uses the web browser to publish metadata and

upload full-text documents. All the metadata and full-text are stored on the

server. The user still has a base URL that he can give to and

OAI compliant harvester to harvest his collection.

Indo-US Workshop, June23-25, 2003

Server-Side Kepler Home

Page

Create Your Own Online

Archivelet

Indo-US Workshop, June23-25, 2003

Online Archivelet with options-List-Publish-Import-Export-Settings-Help

Publishing Interface – DC

Editor

Bookmarkable Metadata Link

Indo-US Workshop, June23-25, 2003

Related Tools

Indo-US Workshop, June23-25, 2003

Rapid Visual OAI Tool (RVOT)

which can help small organizations in making their collections OAI-PMH (OAI Protocol for Metadata Harvesting) compliant in a quick and convenient way. RVOT can be used to graphically construct an OAI-PMH repository from a collection of files. It includes:

•a metadata translation tool•a lightweight HTTP server •and an OAI-PMH request handler

The records in the original collection can be in any of the supported formats including RFC1807, MARC subset, and COSATI subset formats. RVOT helps to define a mapping visually from native format to Dublin Core (DC) format, once the mapping is defined, native metadata files are converted to DC format metadata files, and the repository becomes OAI-PMH compliant.

RVOT is self-contained, easy to install, and can be extended to support new metadata formats.

Indo-US Workshop, June23-25, 2003

RVOT

Indo-US Workshop, June23-25, 2003

RVOT

Indo-US Workshop, June23-25, 2003

RVOT

Indo-US Workshop, June23-25, 2003

RVOT

Indo-US Workshop, June23-25, 2003

RVOT

Indo-US Workshop, June23-25, 2003

RVOT

Indo-US Workshop, June23-25, 2003

Conclusion

Individual focus tools can help in opening up the vast amount of untapped content.

Feasibility of the Kepler approach ?

URLs:

Kepler: http://kepler.cs.odu.edu/

RVOT: http://rvot.sourceforge.net/

ODU DL: http://dlib.cs.odu.edu/