Planning for Flexible Integration via Service-Oriented Architecture (SOA)

APSR Forum – The Well-Integrated RepositorySydney, Australia

February 2006

Sandy PayetteCo-Director, Fedora Project

Researcher, Cornell Information Science

Flexible integration and soa

• What are the motivating contexts?– Signs of change: e-research, e-science, advanced scholarly

pub– Anticipation of next moves (e.g., semantic web; web 2.0)

• How do we position for the future?– Goals– Functional requirements placed upon repositories

• What do we mean by flexible integration?– SOA and Service frameworks– Understanding why repositories play a central role in

service frameworks

• Example SOA work at Cornell

First… motivating contexts

D is c o nne c te d ne tw o rk s :fo rm a l public a tio n ne tw o rks o c ia l ne tw o rk (a c to rs )

H ybr i d ne tw o r kdo c um e nts ( fo r m al and i nfo r m al )datas e r vi c e sac to r s

D ata

Ac to r

F o r m al d o c u m en t

I n f o r m al d o c u m en t

D ata s e ts

W eb s er v ic e

The Future: Rich Scholarly Information Networks

• E-research, e-scholarship• Scholarly workbench• Advanced scholarly communication/publication

Already signs of change…

• Grid computing in sciences– Share computing resources– Share services and distributed virtual file systems– Examples

• Enabling Grids for E-Science (http://public.eu-egee.org/)• National Virtual Observatory (http://www.us-vo.org/)

• Humanities computing– Hyperlinked historical documentary editions– New Forms of Digital Scholarship

• Rossetti archive (http://www.rossettiarchive.org/)• Perseus (www.perseus.tufts)• Pompeii Forum (http://pompeii.virginia.edu)

– Tibetan and Himalayan Digital Library (thdl.org)

What’s out ahead?

• Semantic Web• Web 2.0• Expectations of new generation of scholars

Web 2.0 concepts

http://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.html

Implications of web 2.0

• Key themes– Services (not packaged apps)– Architecture of participation– Remix/transform data sources– Harness collective intelligence

• Emergent Behavior– Upcoming generations of scholars will have a completely

different paradigm and expectations regarding technology– Collaborative classification (e.g., flickr)– Power of collective intelligence (amazon)– Alternative trust models (reputation – ebay; open-source)

• Plan now for technologies to be service-oriented, flexible, decomposed

Positioning repositories for the future…

1. Support the creation and publication of new forms of “information units”

2. Integrate with the processes (e.g., workflows) of research, collaboration, and scholarly communication

3. Enable knowledge integration: capture semantic and factual relationships among information entities

4. Promote information re-use and contextualization

5. Facilitate collaborative activity and capture information that is created as a byproduct of it

Well-integrated repository should…

Support the new “information unit”

• Documents• Text• Data• Simulations• Images• Video• Computations• Automated

Analyses

Data

Aggregations

Enabling technologies:Selected SOAs (Cornell University)

• Fedora Service Framework

• National Science Digital Library

• NSF Pathways (Cornell/LANL)

Fedora Service Framework (2005-07)

CoreRepository

Service

Fedora Serv ices

Apps

PreservationIntegrity

Exte rn alW orkflow

JHOVE

GDFR

FedoraWorkflow

Administrator

PRO AI

(OAI Pro v id e r)

DirectoryIngest

W e b-b ase dsubmission andbasic workflow

FederationPID

ResolutionPreservationM onitoring

EventNotification

FedoraSearch

O ther

Oth erService

Dialog Box Name

O KTex t:

Tex t

Tex t

Tex t

Tex t

Tex t

Cancel

H elp

Sample Text Here Sample Text Here Sample TextHere Sample Text Here Sample Text Here SampleText Here Sample Text Here Sample Text HereSample Text Here Sample Text Here

S am ple Tex t Here S am ple Tex t Here S am ple Tex t Here S am ple Tex t HereS am ple Tex t Here S am ple Tex t Here S am ple Tex t Here S am ple Tex t HereS am ple Tex t Here S am ple Tex t Here S am ple Tex t Here S am ple Tex t Here

FIRE ClientDirIngest PolicyBuilder

Fedora Digital Objects

• Flexible object model can support– Documents, articles, journals– Electronic Scholarly Texts– Digital Images– Complex multimedia publications – Datasets– Metadata– Learning objects– More…

• Create “networks” of objects– Define object relationships and other properties via RDF– Collection/member; part/whole; etc.

Network of Digital Objects in a Fedora Repository

S erv ice

hasM em ber

ha sP art

d iagra m

letter

hasM em ber

P ID -1

P ID -4

P ID -7P ID -2P ID -5

P ID -3

hasP art

P ID -9

am azo n e-co m m erce

Pathways ProjectChallenges – Phase 1

• Current situation – Heterogeneous repository systems– Heterogeneous object models (or no object model) – Multiple protocols and service APIs– Services lacking formal interface definitions

• Can these resources ever play nicely together?• Need common abstractions…

– Information model– Service model

Cornell University and Los Alamos National Lab(http://www.infosci.cornell.edu/pathways)

Pathways Vision: “Graphite” Information Model

Im ag e O b jectW e b r e so ur c e

G ra ph ite O v e rla y Fra g m e n t

L A N LR e p o s i t o r y

S erv ice-B

U R I-1 0

T yp eU R I-1

T yp eU R I-3

T yp eU R I-4

T ypeU R I-7T ypeU R I-8

arX iv F ed ora

T yp eU R I-6

T ypeU R I-2

U R I-1

U R I-4

U R I-7

U R I-9

Gr id da t a se t

U R I-2

D o cu m en t

T yp eU R I-5

U R I-8

U R I-6

U R I-5

U R I-3

Cornell/LANL Pathways Project

Most things can be represented as a graph of nodes and arcs.

Building Block: Repository Integration (Access-perspective)

Fedora RepositoryServ ice

Fedora Serv ices

Apps

PreservationIntegrity

Exte rn alW o rkflo w

JHOVE

G DF R

FedoraWorkflow

Administrator

PRO AI

(O AI Pro v id e r)

DirectoryIngest

Inte rope rable Acce ss(ontology -base d)

Se m antic Se rv ice Matching (OW L-S)

FederationPID

ResolutionPreservationM onitoring

EventNotification

FedoraSearch

O penURLAccessPoint

Oth erService

Pathways

I nterDisseminatorS ervice

aDO Re

arX iv

DS pace

Ope

nUR

LO

penU

RL

Ope

nUR

L

Dialog Box Name

O KTex t:

Tex t

Tex t

Tex t

Tex t

Tex t

Cancel

H elp

Sample Text Here Sample Text Here Sample TextHere Sample Text Here Sample Text Here SampleText Here Sample Text Here Sample Text HereSample Text Here Sample Text Here

Sam ple Tex t Here Sam ple Tex t Here S am ple Tex t Here Sam ple Tex t Here

Sam ple Tex t Here Sam ple Tex t Here S am ple Tex t Here Sam ple Tex t Here

Sam ple Tex t Here Sam ple Tex t Here S am ple Tex t Here Sam ple Tex t Here

FIRE ClientDirIngest PolicyBuilde rO penURL

client

Cornell CIS and LANL Library

Information Model:“Article” Example Core-1 Ontology

Knowledge Network

NSDL: Creating a Collaborative Knowledge Network

AnnotationsQuality AssessmentsRelationshipsReviews

NSDL: Information Network Overlay

Conclusions: Planning for integration

What basic changes can occur now?

• Think in terms of service frameworks• Expose repositories as services• Enable easy integration of repository with other

services• Support compound digital objects

– Local and remote content– Any media type– Provide a way to associate services with objects (dynamic

views)

• Provide ability to assert relationships among objects• Look toward ontology-based metadata

Ongoing Challenges

• Low barrier to entry – Simple protocols (e.g., like OAI)– Light-weight (REST vs. SOAP?)– Simple tools to create overlays– Note complexity in setting up Grid-based services

• Service matching (object-to-service)– Ontologies to expose objects with formats and semantics– OWL-S for semantic service description– Matching-making algorithms

• Security and Trust– Authentication and trust among repositories and services– Interoperability of authorization policy

• Preservation– Distributed and dynamic digital objects a reality