Data-driven Applications with conStruct

Michael K. BergmanStructured Dynamics LLC

presented at

Semantic Technologies ’09 ConferenceSan Jose, CA

June 16, 2009

“BKN: Building Knowledge through Communities,

and Communities through Knowledge”

2

Bibliographic Knowledge NetworkSemTech ‘09

Presentation Outline

I. Overview of BKN

II. Demonstration of a BKN Node

III. More on Design

IV. Architecture and Web Services Framework

V. Collaboration Demonstration

VI. Design Benefits

VII. conStruct and Next Plans

VIII. Contact Info

I. Overview of BKN

4


Introduction BibKN or BKN: the Bibliographic Knowledge Network

Develop tools and services for virtual organizations (“VOs”): Scientific communities of varied sizes, interests Conference groups Departmental research groups Students, teachers and faculty alike

Select, filter and enhance bibliographic data for each VO: Multiple input citation, bibliographic and formats

Apply new and existing methods to these bibliographic collections Bibliometric analysis Machine learning Statistical visualization

Research, provide authoring tools for relevant structured data

First math and statistics then any knowledge community

5


The Project

Two-year effort

Began October 2008

Joint consortium effort of: University of California, Berkeley Harvard University Stanford University American Institute of Mathematics (AIM)

Director: Dr. Jim Pitman Professor of Statistics and Mathematics, UC Berkeley

Funded by NSF (Grant No. 0835851)

All software and data to be open source

6


Some Points for this Talk Four kinds of network “nodes”:

1. VO Nodes – collaboration portals2. Gateways – connections to existing external content3. Hubs – aggregate suppliers of useful datasets4. Individual dataset contributors and clients

Data formats and models: BibJSON: human readable, exchange format Ingest of various existing citation/biblio formats RDF: “canonical” internal data model

VO Nodes the emphasis of this talk: CMS: Drupal RDF triplestore: Virtuoso Full-text, faceted search: Solr

7


Preview of Benefits

Comprehensive toolset for structured data (semantic Web) conversion, use and exposure

Data-driven via ontologies; easily scoped, tailored

Naïve data formats and RDF APIs via RESTful Web services

Web services framework can mix-and-match: Standalone Integrated with any CMS External tools access

Web-wide user and dataset access and permissions

Available as a distro of the Drupal CMS

Open source and extensible

8


conStruct SCS

A ‘data-driven app’ (ontologies + structured data)

Datasets + permissions

Context-sensitive display templates

Use and exposure of linked data

Hosted by Drupal

II. Demonstration of a BKN Node

10


BibKN Welcome Screen

III. More on Design

12


Data Formats and Exchange

Ingest standard structured and bibliographic data: BibTeX Bibsonomy RePEc etc., etc.

Standard ‘naïve’ data format: BibJSON Attribute-value pair orientation Human readable Easily authored and edited

RDF as the internal, ‘canonical’ data model Also can be used for exchange

13


Data Formats and Exchange

14


Four Kinds of ‘Nodes’

VO Nodes – collaboration portals

Gateways – connections to existing external content

Hubs – aggregate suppliers of useful datasets

Individual dataset contributors and clients

The primary construct for data exchange and use is the dataset

15


Multiple Deployment Options

16


Illustrative BKN Network

17


The Backend Structure

Drupal: User interface and theming User and group management Content management system Many third-party tools and modules

Virtuoso: RDF triple store SPARQL and endpoints; linked data exposure Some structured data conversions

Solr: Full-text indexing Faceting and aggregating (counts) Innovative complete RDF search

18


The Backend Structure

IV. Architecture and Web Services Framework

20


Overall Architecture

21


structWSF (Web services framework) Standard functionality:

Data management: Create Read Update Delete

Browse and search Import/export Display templates

Dataset registration w/ permissions

Ontology registration

User authorization and access (IP-based initially)

RESTful design

22


structWSF Schema

V. Collaboration Demonstration

24


Topology of the Collaboration Demo

VI. Design Benefits

26


Comprehensive Toolset Standard functionality:

Data management: Create Read Update Delete

Browse and search Import/export Display templates

Dataset registration w/ permissions

Ontology registration

User authorization and access (via Drupal OG)

Drupal modules are lightweight wrappers over structWSF

27


Data-driven via Ontologies

Ontologies (OWL) guide: Instance records and attributes Data relationships Data presentation:

Trees/hierarchies Display templates

User interface, e.g.: Autocompletion Contextual dropdown list selections New record entry New attribute entry etc.

Addition or swap out of ontologies drives generic tools

28


RESTful structWSF

Independent, standalone piece




As described

29


Open Source Drupal Distro

Provided as the conStruct module

See conclusion

30


Re-cap of Benefits

Comprehensive toolset for structured data (semantic Web) conversion, use and exposure

Data-driven via ontologies; easily scoped, tailored




Available as a distro of the Drupal CMS

Open source and extensible

VII. conStruct and Next Plans

32


conStruct SCS conStruct SCS is a structured content system

Open source version of the BKN project that runs on Drupal

Drupal components: Drupal conStruct module (core functionality) structDisplay module (display templating) Required additional modules (e.g., Organic Groups,

WYSIWYG, etc.)

Structured Dynamics: structWSF (Web services framework)

Third parties: Virtuoso RDF triple store Solr full-text faceted search Smarty templating system

33


The conStruct System

34


conStruct Distribution Options

Initially: Downloads from multiple sites Download, Install & Configuration manual

Next option: Single-click Amazon EC2 AMI install

Planned: Complete, packaged Drupal distro

Hoped: Multiple CMS versions

35


conStruct Plans and Schedule

conStructSCS.com now open Online demo available

OpenStructs.org now open Also the distribution point for open source display templates,

converters and extractors

conStruct alpha distro to be available June 30

sturctWSF alpha distro to be available June 30

Additional conStruct supporting modules planned on an ongoing basis

VII. Contact Info

37


Contacts & InformationBibliographic Knowledge Network

Dr. Jim [email protected]

Nitin BorwankarProject [email protected]

Web Sitewww.BibKN.org

Structured Dynamics

Michael K. [email protected]: www.mkbergman.com

Web SitesconStructSCS.comwww.structureddynamics.comwww.umbel.org

Data-driven Applications with conStruct

Technology

Transcript of Data-driven Applications with conStruct