DiGIR1 Distributed Databases and Applications John Wieczorek Museum of Vertebrate Zoology, UC...

DiGIR 1

Distributed Databases and Applications

John Wieczorek

Museum of Vertebrate Zoology, UC Berkeley

DiGIR 2

Distributed Databases Multiple sources of data …under local control, …with concepts in common …and a desire to deliver data as part of a

community.

DiGIR 3

Distributed Databases The Species Analyst (TSA) The Integrated Taxonomic Information System (ITIS) FishNet The Mammal Networked Information System (MaNIS) HerpNET The Ornithological Information System (ORNIS) …

DiGIR 4

Distributed Databases European Natural History Science Information

Network (ENHSIN) Biological Collection Access for Europe (

BioCASE) Australia Virtual Herbarium (AVH) Red Mundial de Información Sobre

Biodiversidad, Comisión Nacional para el Conocimiento y Uso de la Biodiversidad (REMIB, CONABIO)

…

DiGIR 5

Distributed Databases Mountain and Plains Spatio-Temporal Database-

Informatics (MaPSTeDI) Ocean Biogeographic Information System (OBIS) Pacific Basin Information Node, National Biological

Information Infrastructure (PBIN, NBII) Species Link, Centro de Referência em Informação

Ambiental (Species Link, CRIA) A Virtual Herbarium of the Chicago Region (vPlants) Spatial Analysis of Local Vegetation Inventories Across

Scales (SALVIAS) …

DiGIR 6

Distributed Databases Berkeley Natural History Museums (BNHM) Association of Biological Collections, UC Davis …

DiGIR 7

Distributed Databases LifeMapper Global Biodiversity Information Facility (GBIF)

DiGIR 8

Distributed vs. centralized Multiple sources of data …under local control, …with concepts in common …and a desire to deliver data as part of a

community

DiGIR 9

Distributed vs. centralized

In other words, distribute the headache rather than have one central migraine.

DiGIR 10

DiGIRDistributed Generic Information Retrieval

John Wieczorek, Stan Blum, Dave Vieglais, P.J. Schwartz

DiGIR 11

Project Rationale To avoid multiple incongruous

development efforts To pool resources and create a community

of experts To solve the problem of scalability

DiGIR 12

Project Goals To define a protocol for retrieving

structured data from multiple, heterogeneous databases across the Internet

To build a reference implementation of both provider and portal software using said protocol

DiGIR 13

Design Goals To use open protocols and standards, such

as HTTP and XML To decouple the protocol, software and

semantics To make new data provider installations as

easy as possible To have open source development and

GNU General Public Licensing

DiGIR 14

DiGIR ArchitectureUser InterfaceProtocolPortal EngineProvider

DiGIR 15

DiGIR ArchitectureProvider

DiGIR 16

DiGIR ArchitectureProviderRegistry

DiGIR 17

DiGIR ArchitecturePortal Engine

DiGIR 18

DiGIR ArchitecturePortal EngineRegistry

DiGIR 19

DiGIR ArchitectureUser Interface

DiGIR 20

DiGIR ArchitectureUser InterfaceProtocolPortal Engine

DiGIR 21

DiGIR ArchitectureUser InterfaceProtocolPortal EngineProtocolProvider

DiGIR 22

DiGIR ArchitectureUser InterfaceProtocolPortal EngineProtocolProvider

DiGIR 23

DiGIR ArchitectureUser InterfaceProtocolPortal Engine

DiGIR 24

DiGIR Component Summary

DiGIR 25

DiGIR Protocol Defines request and response message

formats for communication between provider, portal engine, and user interfaces Metadata requests Search requests Inventory requests

Remains unfettered by the structure of the data it transfers

DiGIR 26

Portal Engine The entry point for a “user” Can query a registry for

potential providers Can determine, based on

provider metadata, whether a provider should be queried

Can send requests to multiple providers

Communicates via protocol compliant messaging only

DiGIR 27

Portal Engine, continued Assembles responses

from providers Returns packaged results

to the “user” Logs activity

DiGIR 28

Provider Receives requests Retrieves data from database Sends results to requestor Supplies metadata to describe

data classification and availability

Logs requests

DiGIR 29

Registry Supports provider

“advertising” May be global and open May be private Need not be used at all Example: Universal

Description, Discovery and Integration (UDDI)

DiGIR 30

User Interfaces Must be able to assemble and

send a request document to a portal

Must be able to receive and interpret a response document from the portal

This is where the real fun is!

DiGIR 31

Example Network Configurations

DiGIR 32

BNHM Network Configuration

PHMAWorking

Database

OnlineDatabase

UCBGWorking

Database

DiGIRProvider

BNHMDiGIRPortal

UCJEPSWorking

Database

OnlineDatabase

UCMPWorking

Databases (4)

OnlineDatabase

EssigWorking

Database

OnlineDatabase

OnlineDatabase

BNHMPresentation

Layer

DiGIR 33

MaNIS Network Configuration

WorkingDatabase

OnlineDatabase

WorkingDatabase

DiGIRProvider

MaNISDiGIRPortal

WorkingDatabase

OnlineDatabase

WorkingDatabase

OnlineDatabase

WorkingDatabase

OnlineDatabase

OnlineDatabase

MaNISPresentation

Layer

DiGIRProvider

MaNISDiGIRPortal

MaNISPresentation

Layer

DiGIRProvider

MaNISDiGIRPortal

MaNISPresentation

Layer

DiGIRProvider

DiGIRProvider

DiGIR 34


LACMMS AccessDatabase

OnlineMS AccessDatabase

MVZSybase

Database

MaNISDiGIRPortal

TTUFoxPro

Database


UWBM4D-Mac

Database


CASSQL ServerDatabase

OnlineSQL ServerDatabase


MaNISDiGIRPortal

MaNISDiGIRPortal

MVZ-MaNISPresentation

Layer

LACM-MaNISPresentation

Layer

UWBM-MaNISPresentation

Layer

DiGIRProvider

DiGIRProvider

DiGIRProvider

DiGIRProvider

DiGIRProvider

DiGIR 35




MVZSybase

Database

MaNISDiGIRPortal

TTUFoxPro

Database


UWBM4D-Mac

Database





MaNISDiGIRPortal

MaNISDiGIRPortal


Layer


Layer


Layer

DiGIRProvider

DiGIRProvider

DiGIRProvider

DiGIRProvider

DiGIRProvider

DiGIR 36




MVZSybase

Database

MaNISDiGIRPortal

TTUFoxPro

Database


UWBM4D-Mac

Database





MaNISDiGIRPortal

MaNISDiGIRPortal


Layer


Layer


Layer

DiGIRProvider

DiGIRProvider

DiGIRProvider

DiGIRProvider

DiGIRProvider

DiGIR 37




MVZSybase

Database

MaNISDiGIRPortal

TTUFoxPro

Database


UWBM4D-Mac

Database





MaNISDiGIRPortal

MaNISDiGIRPortal


Layer


Layer


Layer

DiGIRProvider

DiGIRProvider

DiGIRProvider

DiGIRProvider

DiGIRProvider

DiGIR 38




MVZSybase

Database

MaNISDiGIRPortal

TTUFoxPro

Database


UWBM4D-Mac

Database





MaNISDiGIRPortal

MaNISDiGIRPortal


Layer


Layer


Layer

DiGIRProvider

DiGIRProvider

DiGIRProvider

DiGIRProvider

DiGIRProvider

DiGIR 39

Other Network Configurations

WorkingDatabase

OnlineDatabase

WorkingDatabase

DiGIRProvider

DiGIRProvider

DiGIRPortal

WorkingDatabase

OnlineDatabase

DiGIRProvider

WorkingDatabase

OnlineDatabase

DiGIRPortal

WorkingDatabase

OnlineDatabase

DiGIRProvider

DiGIRPortal

DiGIR 40

DiGing a little deeper

DiGIR 41

Provider Installation Web server (Apache, IIS, etc.) PHP: Hypertext Preprocessor

(PHP) Provider software (DiGIR)

Configuration tool Testing scripts Provider scripts Provider manual (DiGIR)

DiGIR 42

Provider Configuration Tool Provider metadata Resources Database connection Establishing table

relationships Concept to column (i.e.,

field, attribute) mapping

DiGIR 43

Portal Configuration Web server (Apache, IIS, etc.) Sun Java 2 (JDK 1.4) Tomcat (Apache) Portal software (DiGIR) Portal installation

documentation (DiGIR)

DiGIR 44

Portal Installation Engine configuration file

(finding providers) Presentation configuration

file (defining the Information Domain)

Presentation customization Engine start and stop scripts Presentation start and stop

scripts

DiGIR 45

Portal Demonstrations

DiGIR 46

DiGIR Project Information The DiGIR project is a collaborative effort DiGIR is currently established as an open

source development project on SourceForge (https://sourceforge.net/projects/digir).

Further documentation is available on the DiGIR web site (http://digir.net).

DiGIR1 Distributed Databases and Applications John Wieczorek Museum of Vertebrate Zoology, UC...

Documents

Transcript of DiGIR1 Distributed Databases and Applications John Wieczorek Museum of Vertebrate Zoology, UC...