1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.
-
Upload
georgiana-fowler -
Category
Documents
-
view
222 -
download
1
Transcript of 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.
![Page 1: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/1.jpg)
1
The NSDL:A Case Study in Interoperability
William Y. ArmsCornell University
![Page 2: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/2.jpg)
2
The NSDL is a program of the National Science Foundation's Directorate for Education and Human Resources, Division of Undergraduate Education.
The NSDL Core Integration is a collaboration between the University Center for Atmospheric Research (Dave Fulker), Columbia University (Kate Wittenberg) and Cornell University (Bill Arms).
The ideas discussed in this talk do not represent the official views of the NSF or the Core Integration team.
Acknowledgement and Disclaimer
![Page 3: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/3.jpg)
3
Research Funding: Europe and USA
Europe
Grant is awarded to carry out the research plan specified in proposal
USA
Grant is awarded to carry out research in the area described in the proposal, but is not expected to follow the precise plan.
![Page 4: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/4.jpg)
4
New Initiatives during a Grant
Program Activity University
Gigabit testbed Mosaic Illinois
CSTR Lycos Carnegie Mellon
DLI-1 Google PageRank Stanford
DLI-2 Open Archives Initiative Cornell
Examples of significant partial funding that was not envisaged in the proposal.
![Page 5: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/5.jpg)
5
NSF-funded Research Programs
NSF
Solicitation
Proposals
Research
New ideas
New ideas
![Page 6: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/6.jpg)
6
The NSDL Program
NSF's objective
Build a comprehensive digital library for all aspects of science education
NSF's approach
Solicitation encouraged wide diversity of proposals divided into general categories
Best 60+ proposals funded -- more to follow
Grants allow projects flexibility
Result
A splendid set of projects
A challenge in interoperability!
![Page 7: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/7.jpg)
7
NSDL Collections Funded by the NSF (a) Focused collections
![Page 8: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/8.jpg)
8
![Page 9: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/9.jpg)
9
![Page 10: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/10.jpg)
10
![Page 11: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/11.jpg)
11
NSDL Collections Funded by the NSF (b) Aggregates and federations
![Page 12: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/12.jpg)
12
![Page 13: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/13.jpg)
13
![Page 14: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/14.jpg)
14
![Page 15: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/15.jpg)
15
NSDL Service Projects Funded by the NSF
![Page 16: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/16.jpg)
16
![Page 17: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/17.jpg)
17
![Page 18: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/18.jpg)
18
![Page 19: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/19.jpg)
19
NSDL Core Integration Team Funded by the NSF
![Page 20: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/20.jpg)
20
Responsibility without Authority
Core Integration
Budget $4-6 million
Staff 25 - 30
Management Diffuse How can a small team, without direct management control, create a very large-scale digital library?
![Page 21: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/21.jpg)
21
All branches of science, all levels of education, very broadly defined:
Five year targets
1,000,000 different users
10,000,000 digital objects
10,000 to 100,000 independent sites
How Big might the NSDL be?
![Page 22: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/22.jpg)
22
Collections The NSDL program funds only a fraction of the relevant collections.
![Page 23: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/23.jpg)
23
Every Collection is Different
![Page 24: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/24.jpg)
24
... to provide a coherent set of services across great diversity.
The Core Integration Task ...
![Page 25: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/25.jpg)
25
A Spectrum of Interoperability
![Page 26: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/26.jpg)
26
Approaches to interoperability
The conventional approach
Wise people develop standards: protocols, formats, etc.
Everybody implements the standards.
This creates an integrated, distributed system.
Unfortunately ...
Standards are expensive to adopt.
Concepts are continually changing.
Systems are continually changing.
Different people have different ideas
![Page 27: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/27.jpg)
27
Interoperability is about agreements
Technical agreements cover formats, protocols, security systems so that messages can be exchanged, etc. Content agreements cover the data and metadata, and include semantic agreements on the interpretation of the messages. Organizational agreements cover the ground rules for access, for changing collections and services, payment, authentication, etc.
The challenge is to create incentives for independent digital libraries to adopt agreements
![Page 28: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/28.jpg)
28
Function versus cost of acceptance
Function
Cost of acceptance
Many adopters
Few adopters
![Page 29: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/29.jpg)
29
Example: textual mark-up
Function
Cost of acceptance
SGML
ASCII
HTML
XML
![Page 30: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/30.jpg)
30
Example: security
Function
Cost of acceptance
Public key infrastructure
IP address
Login ID and password
![Page 31: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/31.jpg)
31
Levels of interoperability
Level Agreements Example
Federation Strict use of standards AACR, MARC(syntax, semantic, Z 39.50and business)
Harvesting Digital libraries expose Open Archivesmetadata; simple metadata harvesting
protocol and registry
Gathering Digital libraries do not Web crawlerscooperate; services must and search enginesseek out information
![Page 32: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/32.jpg)
32
Metadata StrategyMetadata is expensiveThe NSDL cannot afford to create it manually
![Page 33: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/33.jpg)
33
Metadata Strategy
• Support eight standard formats
• Collect all existing metadata in these formats
• Provide crosswalks to Dublin Core
• Expose records in the metadata repository for others to harvest
• Concentrate on collection-level metadata
• Use automatic generation to augment item-level metadata
![Page 34: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/34.jpg)
34
Users
Collections
Metadata repository
The Metadata Repository
Services
The metadata repository is a resource for service providers.
It holds information about every collection and item known to the NSDL.
![Page 35: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/35.jpg)
35
Services Strategy
![Page 36: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/36.jpg)
36
The Metadata Repository as a Resource
Records are exposed through Open Archives Initiative harvesting protocol.
Core Integration team will provide some services based on the metadata repository.
The architecture encourages others to build services.
![Page 37: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/37.jpg)
37
Example: Search Service
Portal
Portal
Portal
Search andDiscoveryServices Collections
SDLIP OAI
http
Metadata repository
James Allan, Bruce Croft (University of Massachusetts, Amherst)
![Page 38: 1 The NSDL: A Case Study in Interoperability William Y. Arms Cornell University.](https://reader036.fdocuments.net/reader036/viewer/2022062314/56649e355503460f94b2413d/html5/thumbnails/38.jpg)
38
Research Challenges:
Extending the Architecture to Support Greater Riches
Federations with rich sets of agreements (e.g., MARC, Z39.50)
Rich object models (e.g., interactive, dynamic, continuous time)
Language tools (e.g, thesaurus, gazetteer)
... and Lesser Riches
Web crawling
Automated quality control