Post on 12-Jan-2016
description
1
Consensus building workshop
Conference track
2
Outline
• Introduction (ideas behind the track)
• Evaluation
• Discussion – interesting mappings
3
Conference track - Features
• Broadly understandable domain Conference Organisation
• Free exploration by participants within 10 ontologies
• No a priori reference alignment
• Participants: 6 research groups
4
Conference track - Dataset
OWL, tool Protege
5
Conference track - Participants
• 6 participants– Automs– Coma++– OWL-CtxMatch– Falcon– HMatch– RiMOM
6
Conference track - Goals
• Focus on interesting mappings and unclear mappings– Why should they be mapped?
• Arguments: against and for
– Which systems did discover them?– Differences in similarity measures
• Underlying techniques?
7
Outline…
• Introduction (ideas behind the track)
• Evaluation
• Discussion – interesting mappings
8
Evaluation
• Processing all mappings by hand
• Assessment based on personal judgement of organisers (consistency problem)
• Tags: TP, FP, interesting, ?, heterogenous mapping
• Types of errors and phenomena: – subsumption, inverse property, siblings,
lexical confusion
9
Evaluation…
• Subsumption mistaken for equivalence– Author,Paper_Author– Conference_Trip, Conference_part
• Inverse property– has_author,authorOf
• Siblings mistaken for equivalence– ProgramCommittee,Technical_commitee
• Lexical confusion error– program,Program_chair
• Relation – Class mapping– has_abstract,Abstract– Topic,coversTopic; read_paper,Paper
10
Evaluation…
• Some statistics as a side-effect of processing
11
Evaluation…
12
Evaluation…
• Distribution of similarity measures – for True Positive Mappings and – for False Positive Mappings
13
Evaluation…Coma++
0
50
100
150
200
250
300
350
400
450
500
0-20% 20-40% 40-60% 60-80% 80-100%
TP
FP
Coma++
0
50
100
150
200
250
300
350
400
450
500
0-20% 20-40% 40-60% 60-80% 80-100%
TP
FP
14
Evaluation…
Falcon
0
50
100
150
200
250
300
350
400
0-20% 20-40% 40-60% 60-80% 80-100%
TP
FP
15
Evaluation…
HMatch
0
50
100
150
200
250
300
350
400
0-20% 20-40% 40-60% 60-80% 80-100%
TP
FP
16
Evaluation…RiMOM
0
100
200
300
400
500
600
700
0-20% 20-40% 40-60% 60-80% 80-100%
TP
FP
RiMOM
0
100
200
300
400
500
600
700
0-20% 20-40% 40-60% 60-80% 80-100%
TP
FP
17
Record it!
18
Outline…
• Introduction (ideas behind the track)
• Evaluation
• Discussion – interesting mappings
19
Discussion
• Focus on interesting mappings and unclear mappings– Why should they be mapped?
• Arguments: against and for
– Which systems discover them?– Differences in similarity measures
• Underlying techniques?
20
Mapping 1
Element1 Element2 Relation
Person Confious:human =
Notes semantically same
System
Automs Coma++ OWL-CtxMatch
Falcon HMatch RiMOM
1.0 Iasted
1.0 Ekaw
No No No 0.7 confOf
0.63 Ekaw
0.81 Sigkdd
0.77 sofsem
1.0 PCS
21
Mapping 1confiousIasted
confOf
ekaw
sigkdd
sofsem
PCS
22
Mapping 2
Element1 Element2 Relation
OpenConf:Surname confious:last_name =
Notes Both are datatype properties, the former with People as domain, the latter with human as domain
System
Automs Coma++ OWL-CtxMatch
Falcon HMatch RiMOM
1.0 No No No 1.0 No
23
Mapping 3
Element1 Element2 Relation
sofsem:has_the_last_name confious:last_name =
Notes Both are datatype properties, the former with Person as domain, the latter with human as domain
System
Automs Coma++ OWL-CtxMatch
Falcon HMatch RiMOM
No 0.63 No No 0.8 No
24
Mapping 4
Element1 Element2 Relation
ekaw: PC_Member confOf:Member_PC =
Notes Change order of incompound names
System
Automs Coma++ OWL-CtxMatch
Falcon HMatch RiMOM
1.0 No No No No 0.53
25
Mapping 4
confOf
ekaw
26
Mapping 5
Element1 Element2 Relation
ekaw:Document confious:article =
Notes Semantically same?
System
Automs Coma++ OWL-CtxMatch
Falcon HMatch RiMOM
1.0 No No No No 1.0
27
Mapping 5
confious
ekaw
28
Mapping 8
Element1 Element2 Relation
cmt:Rejection OpenConf:Reject =
Notes Both relates to process of assessment.
But the former is a recommendation, the latter is a decision. So…
System
Automs Coma++ OWL-CtxMatch
Falcon HMatch RiMOM
No 0.29 No 0.94 No No
29
Mapping 8
cmt
OpenConf
30
Mapping 11
Element1 Element2 Relation
ekaw:Location Place =
Notes Semantically same? Both are at the highest level of hierarchy. But Location maybe more general than Place… what about City?
System
Automs Coma++ OWL-CtxMatch
Falcon HMatch RiMOM
No No No No iasted 0.8
sigkdd 0.8No
31
Mapping 11ekaw
iasted
Asserted conditions for iasted:Place
Location is domain of properties: locationOfLocation is range of properties: heldIn
iasted:Place is domain of properties: is_equipped_bysigkdd:Place is range of properties: can_stay_in
sigkdd
32
Mapping 12
Element1 Element2 Relation
sofsem:reviews ekaw:hasReview =
Notes DomainOf(hasReview)=Paper,rangeOf(hasReview)=Review
DomainOf(reviews)=Review,rangeOf(reviews)=Reviewed_contribution
Inverse property phenomena, useful?
System
Automs Coma++ OWL-CtxMatch
Falcon HMatch RiMOM
No No 1.0 No No No
33
• Call for contribution to our dataset
34
Thank you for your participation!