1

SLUBM: An Extended LUBM Benchmark for Stream Reasoning

Tu Ngoc Nguyen, Wolf Siberski

L3S Research Center, Universität Hannover, Germany{tunguyen, siberski}@l3s.de

2

1. Motivation

2. Benchmark

• Dataset

• Methodology

3. Tested Systems

4. Evaluation

• Settings and Results

5. Conclusion

2

Outline

3

Motivation

RDF Stream is everywhere

- social network, feed, financial market, network sensor

The need of processing heterogeneous and noisy RDF

- Stream-based reasoner

Application developers have to choose

- Best practice Benchmark

4

Benchmark

Extended Lehigh University Benchmark [LUBM]• Synthetic data, fixed list of 14 queries

Can be scaled to arbitrary sizes• Generate data of University domain

Familiar but not trivial ontology• University, Faculty, Professors, Students, Courses, …• Realistic structural properties• Artificial literal data “Professor1”, “GraduateStudent216“, “Course7“

• Simulate temporal University data

• Partition data by semesters

• RDF triples + time annotations

• e.g., (<GraduateStudent31, ub:takescourse, GraduateCourse1>, semester2)

• Predicate dynamic classification

• Three classes: dynamic, near-dynamic and static

• Examples:

• Dynamic: teaches, takes course

• Near-dynamic: has a member

• Static: has a degree from

5

Dataset

6

Methodology

System pipeline

7

Methodology

• Data Generator:

• Re-generate University -domain facts

• A semester counter for the loop

ub:Student

ub:GradStudent ub:Undergrad

ub:takescourse

ub:GradCourserdfs:subClassOfrdfs:subClassOf semester ++

8

Methodology

• RDF Handler:

• Parse RDF stream into RDF triples

• Annotate RDF with timestamp according to the semester counter

9

Methodology

out-dated facts need to be removed before adding new facts

•Rules for dynamic facts (with dynamic predicates):

• a time-to-last t△

• a produced fact will be removed

after t△

10

1.BaseVISor

• Forward chaining inference engine

• Based on Rete algorithm

2.Pellet

• OWL-DL reasoner

3.(Pellet)+Jena

• RDF Framework, supports triple-based abstraction

4.(Pellet)+OWLAPI

• RDF Framework, supports higher level of OWL abstraction syntax, the axioms

5.C-SPARQL

• language for continuous queries over streams of RDF data

• potential but not yet reasoning support

10

Tested Systems

11

• Intel(R) Xeon(R) E7520 1.87GHz processor

80GB memory

OpenJDK 1.6.0 24

Linux 2.6.x 64 bit

• 14 LUBM Queries

• 1 dynamic predicate: takecourses (approx. 10 percent of generated data are dynamic)

• Metrics: load time, query response time

11

Evaluation Settings

12

Evaluation ResultsBaseVISor Query time for LUBM queries for extended LUBM (1,0,5), which is LUBM(1,0) over 5 semesters

•Query 5: (type Person ?X) (memberOf ?X http://www.Department0.University0.edu)

•Query 6: (type Student ?X)•Query 13: (type Person ?X) (hasAlumnus http://www.University0.edu ?X)•Query 14: (type UndergraduateStudent ?X)

BaseVISor Query time for Query 14 for extended LUBM (1,0,5), (5,0,5), (10,0,5) and (50,0,5)

“UndergraduateStudent”

13

Evaluation ResultsQuery time for Query 14 for extended LUBM (10,0,5)

Load time for extended LUBM (5,05), (10,0,5), (20,0,5) and (50,0,5)

“UndergraduateStudent”

14

Evaluation ResultsQuery time for extended LUBM (1,0,5), (5,0,5), (10,0,5), (20,0,5) and (50,0,5) (for Query 14)

“UndergraduateStudent”

15

Evaluation Results

16

Identified strong need for a stream-based reasoning benchmark

• For stream-based application and stream-based reasoning developers

Extended LUBM towards a stream-based benchmark

• Other benchmarks can be extended similarly

Preliminary experiment with (adapted) stream-based reasoners

• BaseVISor shows potential performance

16

Conclusion