1 Digital Libraries : Archaeology, Automation, ETDs, and Enhancements Edward A. Fox ([email protected])...

1

Digital Libraries : Archaeology, Automation, ETDs, and Enhancements

Edward A. Fox ([email protected])Virginia Tech, USA

IADLC 2005

The International AdvancedDigital Library Conference in Nagoya

August 25-26, 2005

2

Outline

• Acknowledgements

• Introduction: Life Cycle, Curric., 5S, Book

• ETANA-DL, 5S Description

• Theory and Automation

• Education: CS, ETDs

• Quality, Integration, and Automation

• Selected Links, Discussion

3

Acknowledgements: Students

• Pavel Calado, Yuxin Chen, Fernando Das Neves, Shahrooz Feizabadi, Robert France, Marcos Gonçalves, Nithiwat Kampanya, S.H. Kim, Aaron Krowne, Bing Liu, Ming Luo, Paul Mather, Fernando Das Neves, Unni. Ravindranathan, Ryan Richardson, Rao Shen, Ohm Sornil, Hussein Suleman, Ricardo Torres, Wensi Xi, Baoping Zhang, Qinwei Zhu, …

4

Acknowledgements: Faculty, Staff

• Lillian Cassel, Debra Dudley, Roger Ehrich, Joanne Eustis, Weiguo Fan, James Flanagan, C. Lee Giles, Eberhard Hilf, John Impagliazzo, Filip Jagodzinski, Rohit Kelapure, Neill Kipp, Douglas Knight, Deborah Knox, Aaron Krowne, Alberto Laender, Gail McMillan, Claudia Medeiros, Manuel Perez, Naren Ramakrishnan, Layne Watson, …

5

Other Collaborators (Selected)

• Brazil: FUA, UFMG, UNICAMP

• Case Western Reserve University

• Emory, Notre Dame, Oregon State

• Germany: Univ. Oldenburg

• Mexico: UDLA (Puebla), Monterrey

• College of NJ, Hofstra, Penn State, Villanova

• University of Arizona

• University of Florida, Univ. of Illinois

• University of Virginia

6

Acknowledgements - Mentors

• JCR Licklider – undergrad advisor (1969-71)– Author in 1965 of “Libraries of the Future”– Before, at ARPA, funded start of Internet

• Michael Kessler – BS thesis advisor– Project TIP (technical information project)– Defined bibliographic coupling

• Gerard Salton – graduate advisor (1978-83)– “Father of Information Retrieval”

Acknowledgements: Support

• ACM, Adobe, AOL, CAPES, CNI, CONACyT, DFG, IBM, Microsoft, NASA, NDLTD, NLM, NSF (IIS-9986089, 0086227, 0080748, 0325579; ITR-0325579; DUE-0121679, 0136690, 0121741, 0333601), OCLC, SOLINET, SUN, SURA, UNESCO, US Dept. Ed. (FIPSE), VTLS

8

Outline








9

Information Life Cycle

AuthoringModifying

OrganizingIndexing

StoringRetrieving

DistributingNetworking

Retention/ Mining

AccessingFiltering

UsingCreating

10

DL Curriculum FrameworkSemester 1:

DL collections:development/creation

Semester 2:DL services and

sustainability

CO

UR

SE

ST

RU

CT

UR

E

DigitizationStorage

Interchange

Digital objectsCompositesPackages

MetadataCataloging

Author submission

NamingRepositories

Archives

Spaces(conceptual,geographic,2/3D, VR)

Architectures(agents, buses,

wrappers/mediators)Interoperability

Services(searching,

linking, browsing, etc.)

Intellectual property rights mgmt.

PrivacyProtection (watermarking)

Archiving and preservation

Integrity

Architectures(agents, buses,

wrappers/mediators)Interoperability

CO

RE

DL

TO

PIC

S

DocumentsE-publishing

Markup

Info. NeedsRelevanceEvaluation

Effectiveness

ThesauriOntologies

ClassificationCategorization

Bibliographic information

BibliometricsCitations

RoutingFiltering

Community filtering

Search & search strategyInfo seeking behavior

User modelingFeedback

Info summarizationVisualization

Multimedia streams/structures

Capture/representationCompression/coding

Content-based analysis

Multimedia indexing

Multimediapresentation,

rendering

RE

LA

TE

DT

OP

ICS

11

5S LayersSocieties

Scenarios

Spaces

Structures

Streams

12

5S LayersSocieties

Scenarios

Spaces

Structures

Streams

Fire

Wood

Earth

Metal

Water

5 Elements

13

5Ss

Ss Examples Objectives

Streams Text; video; audio; image Describes properties of the DL content such as encoding and language for textual material or particular forms of multimedia data

Structures Collection; catalog; hypertext; document; metadata

Specifies organizational aspects of the DL content

Spaces Measure; measurable, topological, vector, probabilistic

Defines logical and presentational views of several DL components

Scenarios Searching, browsing, recommending

Details the behavior of DL services

Societies Service managers, learners, teachers, etc.

Defines managers, responsible for running DL services; actors, that use those services; and relationships among them

14

Informal 5S & DL Definitions

DLs are complex systems that

• help satisfy info needs of users (societies)

• provide info services (scenarios)

• organize info in usable ways (structures)

• present info in usable ways (spaces)

• communicate info with users (streams)

15

Hypotheses

• A formal theory for DLs can be built based on 5S.

• The formalization can serve as a basis for modeling and building high-quality DLs.

16

Research Questions1. Can we formally elaborate 5S?

2. How can we use 5S to formally describe digital libraries?

3. What are the fundamental relationships among the Ss and high-level DL concepts?

4. How can we allow digital librarians to easily express those relationships?

5. Which are the fundamental quality properties of a DL? Can we use the formalized DL framework to characterize those properties?

6. Where in the life cycle of digital libraries can key aspects of quality be measured and how?

17

Book Parts

• Ch. 1. Introduction (Motivation, Synopsis)

• Part 1 – The “Ss”

• Part 2 – Higher DL Constructs

• Part 3 – Advanced Topics

• Appendix

18

Book Parts and Chapters - 1

• Ch. 1. Introduction (Motivation, Synopsis)

• Part 1 – The “Ss”– Ch. 2: Streams

– Ch. 3: Structures

– Ch. 4: Spaces

– Ch. 5: Scenarios

– Ch. 6: Societies

19


• Part 2 – Higher DL Constructs– Ch. 7: Collections

– Ch. 8: Catalogs

– Ch. 9: Repositories and Archives

– Ch. 10: Services

– Ch. 11: Systems

– Ch. 12: Case Studies

20


• Part 3 – Advanced Topics– Ch. 13: Quality– Ch. 14: Integration– Ch. 15: How to build a digital library– Ch. 16: Research Challenges, Future Perspectives

• Appendix– A: Mathematical preliminaries– B: Formal Definitions: Ss – C: Formal Definitions: DL terms, Minimal DL– D: Formal Definitions: Archeological DL– E: Glossary of terms, mappings

21

Outline








23Map courtesy: www.enchantedlearning.com

Initial ETANA-DL Member Locations

Virginia Tech

Mississippi State University

Vanderbilt University

Canadian University College

Walla Walla College

Andrews University

CWRU

Willamette University

26

Lahav Website

27

Megiddo Opening Screen

28

Locus Screen: Pictures

View all

29

Area Screen

31

ETANA-DL Approach• Applying and extending Digital Library (DL)

techniques to solve key problems: making primary data available, data preservation, and interoperability

• Modeling archaeological information systems using 5S to better understand the domain and design the system and the supporting services

• Rapidly prototyping DLs that handle heterogeneous archaeological data using componentized frameworks:– eliciting requirements– refining metamodel and union schema– modeling sites– mapping– harvesting– providing useful services

32

ETANA-DL Website

33

Marking – writingnotes for

a specific user

Marking Items

34Marked Items Display

Sender, Date,Object OAI ID

SenderComments

Options:View Record,

Add record to Items Of Interest,Re-mark item (Redirect),

Unmark item (Remove item from list)

35Discussions Page

Discussions about an

object

View/Post messages, create new

threads

36Recommendations

Items recommendedon the basis of

similar interests

37

ETANA-DL Multi-dimensional Browsing

3 new sites

2 new types of artifacts

38

ETANA-DL Visual Browsing Service

Visual BrowseBy site

39

Visual Browsing Nimrin: Topographical Drawings

Full site North west quadrant

Square:N40/W20

40

Visual Browsing Nimrin : Square information

Square:N40/W20

Locus: 86

Loci layout

41

Visual Browsing Nimrin : locus sheet

42

Visual Browsing Bab edh-Dhra'

Cemetery

Pottery # 25

43

Visual Browsing Bab edh-Dhra'

Cemetery

Pottery # 25

44

ETANA Societies

1. Historic and pre-historic societies (being studied)2. Archaeologists (in academic institutes, fieldwork

settings, or local and national governmental bodies)

3. Project directors4. Technical staff (consisting of photographers,

technical illustrators, and their assistants)5. Field staff (responsible for the actual work of

excavation)6. Camp staff (e.g., camp managers, registrars, tool

stewards)7. General public (e.g., educators, learners, citizens)

45

ETANA Societies

• Social issues1. Who owns the finds?

2. Where should they be preserved?

3. What nationality and ethnicity do they represent?

4. Who has publication rights?

5. What interactions took place between those at the site studied, and others? What theories are proposed by whom about this?

46

ETANA Scenarios1. Life in the site in former times2. Digital recording: the planning stage and the excavation stage 3. Planning stage: remote sensing, fieldwalking, field surveys, building

surveys, consulting historical and other documentary sources, and managing the sites and monuments

4. Excavation1. Detailed information is recorded, including for each layer of soil, and for

features such as pole holes, pits, and ditches. 2. Data about each artifact is recorded together with information about its

exact find spot. 3. Numerous environmental and other samples are taken for laboratory

analysis, and the location and purpose of each is carefully recorded. 4. Large numbers of photographs are taken, both general views of the

progress of excavation and detailed shots showing the contexts of finds. 5. Organization and storage of material6. Analysis and hypotheses generation and testing7. Publications, museum displays8. Information services for the general public

47

ETANA Spaces

1. Geographic distribution of found artifacts2. Temporal dimension (as inferred by

archaeologists) 3. Metric or vector spaces

1. used to support retrieval operations, and to calculate distance (and similarity)

2. used to browse / constrain searches spatially

4. 3D models of the past, used to reconstruct and visualize archaeological ruins

5. 2D interfaces for human-computer interaction

48

ETANA Structures

1. Site Organization1. Region, site, partition, sub-partition, locus,

…

2. Temporal orderings (ages, periods)

3. Taxonomies1. for bones, seeds, building materials, …

4. Stratigraphic relationships1. above, beneath, coexistent

49

ETANA Streams

1. successive photos and drawings of excavation sites, loci, unearthed artifacts

2. audio and video recordings of excavation activities and discussions

3. textual reports

4. 3D models used to reconstruct and visualize archaeological ruins.

50

Outline








51

5S and DL formal definitions and compositions (April 2004 TOIS)

5S

structures (d.10)streams (d.9) spaces (d.18) scenarios (d.21) societies (d. 24)

structural metadataspecification(d.25)

descriptive metadataspecification(d.26)

repository(d. 33)

collection (d. 31)

(d.34)indexingservice

structured stream (d.29)

digitalobject (d.30)

metadata catalog (d.32)

browsingservice

(d.37)

searchingservice (d.35)

digital library(minimal) (d. 38)

services (d.22)

sequence (d. 3)

graph (d. 6)function (d. 2)

measurable(d.12), measure(d.13), probability (d.14), vector (d.15), topological (d.16) spaces

event (d.10)state (d. 18)

hypertext(d.36)

sequence (d. 3)

transmission(d.23)

relation (d. 1) language (d.5)

grammar (d. 7)

tuple (d. 4)*

52

Streams

text

audio

image

video digitalobject

Repository

CollectionCatalog

describes

stores

is_version_of/ cites/links_to

Index

Service

Scenario

event

extends

reuses

ServiceManager

Actor

operationexecutes

participates_in

recipient

runs

Scenarios

Societies

inherits_from/includes

association

uses

Topological

ProbabilisticMetric

Measurable

Measure

describes

employsproduces

employsproduces

employs

produces

Structures

Spaces

Vector

contains

metadata specifications

is_a is_a

precedes

happens_before

is_a

redefinesinvokes

contains

contains

53

Browsing Collaborating Customizing Filtering Providing access Recommending Requesting Searching Visualizing

Annotating Classifying Clustering Evaluating Extracting Indexing

Measuring Publicizing

Rating Reviewing (peer)

Surveying Translating

(language)

Conserving Converting

Copying/Replicating Emulating Renewing

Translating (format)

Acquiring Cataloging

Crawling (focused) Describing Digitizing

Federating Harvesting Purchasing Submitting

Preservational Creational

Add Value

Repository-Building

Information Satisfaction

Services

Infrastructure Services

54

SearchingBrowsing

queryanchor

Society

actor

Collection, {digital object}

Recommending Filtering Binding Visualizing Expanding query

user model query/category {digital object}

{digital object} {digital object}

binder

InformationSatisfaction Services

space query’

fundamental

Rating Training

Infrastructure

Services (Add_Value)

composite

Requesting

handle

p pp

e e e{(digital object, actor, rate) }

p

e

e

p p p p p

e e

classifier

e ee e

e

p

e

Indexing

Index

p

e

transformer

e

55

The XML Log Format

Log

SessionId MachineInfo StatementTransaction Timestamp

SessionInfo RegisterInfo StatementEvent Timestamp

Action

Search Browse StoreSysInfoUpdate

SearchBy QueryString CatalogCollection PresentationInfo

StatusInfo

Timeout

56

5S Modeling -> SystemsDomain Concepts (theory)

DLArchitecture

instance of

ModelingLanguage(Meta-Model)

Model

used to compose instance of

abstracted from

represented by

interpreted as

represented by

interpreted as

instance of

instance of

Running

DL DL

Actors

“Real”World

“real” worldobject

Q

57

Tools/Applications

5S MetaModel

5SGraphDL

Expert

DL Designer

5SL DL

Model

5SLGen

Practitioner

Researcher

TailoredDL

Teacher

componentpool

ODLSearch,ODLBrowse,ODLRate,ODLReview,

…….

Logging ModuleXMLLog

58

Digital Object

RepositoryCollection Minimal DL

Metadata Catalog

Descriptive Metadata

Specification

A Minimal DL in the 5S Framework

Structural Metadata

Specification

Streams Structures Spaces Scenarios Societies

indexing

browsing searching

services

hypertext

Structured Stream

59

Streams Structures Spaces Scenarios Societies

indexing

browsing searching

services

hypertext

Structured Stream

Descriptive Metadata

specification

SpaTemOrg

StraDia

Arch Descriptive Metadata specification

ArchDO

ArchObj

ArchColl

Arch Metadata catalog

ArchDColl ArchDR Minimal ArchDL

A Minimal ArchDL in the 5S Framework

60

Overview of 5SGraph

Workspace

(instance model)

Structured

toolbox

(metamodel)

62

Outline








63

Computing and Information Technology Interactive Digital Educational Library (CITIDEL)

• Domain: computing / information technology

• Genre: one-stop-shopping for teachers & learners: courseware (CSTC, JERIC), leading DLs (ACM, IEEE-CS, DB&LP, CiteSeer), PlanetMath.org, NCSTRL (technical reports), …

• Submission & Collection: sub/partner collections www.citidel.org

64

Annotations

OAI Data

Harvester

EDUCATORS

ADMINISTRATORS LEARNERS

Multilingual Searching

Revising Annotating Filtering Browsing Administering

Filtering Profiles User Profiles

Union Metadata

OAI Data

Provider

Remote and Peer Digital Libraries (eg. NSDL -CIS)

PORTALS

SERVICES

REPOSITORIES

Digital library architecture for localand interoperable CITIDEL services

CITIDEL -> NSDL

• A collection project in the

• National STEM (science, technolgy, engineering, and mathematics) education Digital Library – NSDL

• National Science Digital Library

• www.nsdl.org

• (Next slides courtesy Lee Zia, NSF)

67

NSDL ProgramTracks

• Core Integration: coordinate a distributed alliance of resource collection and service providers; and ensure reliable and extensible access to and usability of the resulting network of learning environments and resources

• Collections: aggregate and actively manage a subset of the digital library’s content within a coherent theme / specialty

• Services: increase the impact, reach, efficiency, and value of the digital library in its fully operational form

• Targeted (Applied) Research: have immediate impact on one or more of the other three tracks

• Pathways: large efforts across broad ranges of areas or approaches or users

70

NSDL Information ArchitectureEssentially as developed by the Technical Infrastructure Workgroup

referenceditems &

collections

referenceditems &

collections

Special Databases

NSDLServicesNSDL

ServicesOther NSDLServices

CI Services

annotation

CI Services

discussion

CI Services

personalization

CI Services

authentication

CI Services

browsing

Core Services:information retrieval

Core Collection-Building Services

harvesting

Core Collection-Building Services

protocols

Core Services:metadata gathering

Portals &ClientsPortals &

ClientsPortals &Clients

Usage Enhancement

Collection Building

User Interfaces

NSDLCollections

NSDLCollections

NSDLCollections

CoreNSDL“Bus”

71

Digital Libraries in Education

• Analytical Survey, ed. Leonid Kalinichenko• © 2003, www.iite-unesco.org, [email protected]• Transforming the Way to Learn• DLs of Educational Resources & Services• Integrated/Virtual Learning Environment• Educational Metadata• Current DLEs: US (NSDL, DLESE, CITIDEL,

NDLTD), Europe (Scholnet, Cyclades), UK (Distributed National Electronic Resource)

A Digital Library Case Study

• Domain: graduate education, research

• Genre:ETDs=electronic theses & dissertations

• Submission: http://etd.vt.edu

• Collection: http://www.theses.org

Project: Networked Digital Library of Theses & Dissertations (NDLTD) http://www.ndltd.org

Student Gets CommitteeSignatures and Submits ETD

Signed

Grad School

Library Catalogs ETD, Access isOpened to the New Research

WWW

NDLTD

78

OCLC SRU Interface

80

ETD Union Search Mirror Site in China (CALIS)(http://ndltd.calis.edu.cn – popular site!)

82

Board of Directors• Suzie Allard (ETD 2004, U. Kentucky)• Denise A. D. Bedford (World Bank)• Julia C. Blixrud (ARL, SPARC)• José Luis Borbinha (Natl Lib Portugal)• Alex Byrne (ETD 2005, ADT: Australia)• Tony Cargnelutti (ETD 2005, Australia)• Vinod Chachra (VTLS)• Susan Copeland (RGU, UK)• Jude Edminster (Bowling Green St. U.)• Scott Eldredge (Treasurer, ETD 2002, BYU)• Edward A. Fox (Exec Director,Virginia Tech)• John H. Hagen (West Virginia U.)• Thomas B. Hickey (OCLC)• Christine Jewell (U. Waterloo, Canada)

• Delphine Lewis (ProQuest)• Joan K. Lippincott (CNI)• Mike Looney (Adobe)• Gail McMillan (Secretary, Virginia Tech)• Joseph Moxley (ETD 2000, USF)• Eva Müller (U. Uppsala, Sweden)• Ana Pavani (PUC Rio, Brazil)• Axel Plathe (UNESCO, Paris)• Sharon Reeves (National Library Canada)• Peter Schirmbacher (ETD 2003, Humboldt)• Hussein Suleman (U.Cape Town, S. Africa)• Shalini R. Urs (U. Mysore, India)• Eric F. Van de Velde (ETD 2001, Caltech)

83

Selected Projects / Sponsors

• Australia (ADT)• Brazil (BDT, IBICT)• Canada• Catalunya• Chile (Cybertesis)• Germany• India (Vidyanidhi)• Korea• OhioLINK: 79

colleges/univs

• Portugal (National Library)

• South Africa• UK (British Library,

JISC, Edinburgh, …)• UNESCO (especially

Latin America, Eastern Europe, Africa)

• Venezuela

84

Why ETD? Short Answer

• For Students:– Gain knowledge and skills for the Information Age– Richer communication (digital information, multimedia, …)

• For Universities: – Easy way to enter the digital library field and benefit

thereby

• For the World: – Global digital library – large, useful, many services

• General:– Save time and money– Increased visibility for all associated with research results

86

Outline








87

Describing Quality inDigital Libraries

• What’s a “good” digital Library?– Central Concept: Quality!– Hypotheses of this work:

• Formal theory can help to define “what’s a good digital library” by:

• New formalizations of quality indicators for DLs within our 5S framework

• Contextualizing these measures within the Information Life Cycle

88

AuthoringModifying

OrganizingIndexing

Storing

Archiving

NetworkingAccessing

Filtering

Creation

DistributionUtilization

Significance

Similarity

Pertinence

AccuracyCompletenessConformance

Seeking

SearchingBrowsingRecommending

Relevance

Timeliness

Accessibility

Accessibility

Inactive

Active

Discard

RetentionMining

Semi-Active

Preservability

Timeliness

Preservability

Describing

Quality and the Information Life Cycle

89

Formal Definition of DL Integration

• DLi=(Ri, DMi, Servi, Soci), 1 i n

– Ri is a network accessible repository

– DMi is a set of metadata catalogs for all collections

– Servi is a set of services

– Soci is a society

• UnionRep• UnionCat• UnionServices• UnionSociety

90

Formal Definition of DL Integration (Cont.)

• DL integration problem definition:

Given n individual libraries, integrate the n DLs to create a UnionDL.

91Repository1

DL1

Repository2

Union Catalog

Union Repository

Catalog1 Catalog2

Searching

Union DL DL2

archaeologists

Society

General Public

Society

ArchaeologistsGeneral Public

Union Society

ServiceBrowsingService

Union Service

Harvesting, Mapping,Searching, Browsing,

Clustering, Visualization

Architecture of a Union DL

92

Example of Union Service: CitiViz

93

Multidimensional Browsing: Percentages of Animal Bones Across Nimrin Cultural Phases

94

local schema global schema

95

Mapping recommendation

96

5S MetaModel

5SGraphDL

Expert

DL Designer

5SL DL

Model

5SLGen

Practitioner

Researcher

TailoredDL

Services

Teacher

componentpool

ODLSearch,ODLBrowse,ODLRate,ODLReview,

…….

Requirements (1) Analysis (2)

Implementation (4)

Design (3)

5SGraph 5SGen

Mapping Tool

5SSuite

97

5SGraph5S Archaeology

MetaModelArchDL Expert ArchDL Designer

ETANA-DLUnion Services

Descriptions

HarvestingMapping

SearchingBrowsing

…

Scenario Sub-model

VN Metadata Format

ETANA-DL Metadata Format

HD Metadata Format

Mapping Tool

Wrapper4VN Wrapper4HD

Inverted Files

Services DB

Index

Index

BrowseService

SearchService

Browse DB

OtherETANA-DL

Services

Web

Interface

XOAI

XOAI

VNCatalog

HDCatalog

UnionCatalog

5SGen

ComponentPool

Browsing…

98

Outline








99

Selected Links - http://fox.cs.vt.edu• CITIDEL (computing education resources)

– www.citidel.org• NCSTRL (computing technical reports)

– www.ncstrl.org• NDLTD (electronic theses and dissertations

worldwide)– www.ndltd.org and etdguide.org

• NSDL (National Science Digital Library)– www.nsdl.org

• OAI (Open Archives Initiative)– www.openarchives.org

• Virginia Tech Digital Library Research Laboratory (DLRL, www.dlib.vt.edu)– 5S, AmericanSouth.Org, CSTC, DL-in-a-box, ENVISION,

ETANA, MARIAN, NDLTD, NSDL, OAD, ODL, …)

100

Questions?Discussion?

Thank You!

1 Digital Libraries : Archaeology, Automation, ETDs, and Enhancements Edward A. Fox ([email protected])...

Documents

Transcript of 1 Digital Libraries : Archaeology, Automation, ETDs, and Enhancements Edward A. Fox ([email protected])...