Consensus building workshop Conference track OAEI-2007 Ondřej Šváb The University of Economics...
-
Upload
jordan-dennis -
Category
Documents
-
view
214 -
download
0
Transcript of Consensus building workshop Conference track OAEI-2007 Ondřej Šváb The University of Economics...
Consensus building workshop
Conference trackOAEI-2007
Ondřej ŠvábThe University of Economics
The Department of Information and Knowledge Engineering
11/11/2007 Consensus Building Workshop within Conference track
2
Outline
• Introduction (ideas behind the track)
• Evaluation
• Discussion – ‘interesting’ mappings
11/11/2007 Consensus Building Workshop within Conference track
3
Features of conference track
• Broadly understandable domain of conference organisation
• Free exploration by participants within collection of 14 ontologies
• No a priori reference alignment
• participants: 6 research group
11/11/2007 Consensus Building Workshop within Conference track
5
Participants of conference track
• 6 participants– ASMOV– Falcon-AO– LILY– OLA– OntoDNA– SEMA
11/11/2007 Consensus Building Workshop within Conference track
6
Outline…
• Introduction (ideas behind the track)
• Evaluation
• Discussion – ‘interesting’ mappings
11/11/2007 Consensus Building Workshop within Conference track
7
Evaluation
• Manual evaluation as a precondition for Consensus Building Workshop
• Assessment based on personal judgement• Tags: TP, FP, interesting, heterogenous mapping, unclear, trivial• Types of errors and phenomena:
– Subsumption, inverse property, siblings
Subsumption mistaken for equivalence- Document, Presentation- Session, Tutorial
Inverse property- has_a_review, write_review- has_parts, isPartOf
Siblings mistaken for equivalence- Author, Reviewer- hasMail, first_name
11/11/2007 Consensus Building Workshop within Conference track
8
Evaluation...Precision and rrecall
0,0%
20,0%
40,0%
60,0%
80,0%
100,0%
ASMOV
Falco
nLil
yOLA
OntoD
NA
SEMA
systems
pre
cis
ion
/rre
ca
ll
Rrecall
Precision
Trivial/Subsumptions
0,0%
20,0%
40,0%
60,0%
80,0%
100,0%
ASMOV
Falco
nLil
yOLA
OntoD
NA
SEMA
systems
triv
ial/s
ub
su
mp
tio
ns
Trivial
Subsumptions
http://nb.vse.cz/~svabo/oaei2007/
ratioSubs shows ratio of the number of subsumption errors and the number of incorrect mappings, ratioTriv shows ratio of the number of so-called trivial mappings and the number of correct mappings
precision (P) is computed as ratio of the number of all correct mappings to the number of incorrect plus correct mappings, rrecall (rR) is computed as ratio of the number of all correct mappings to the number of all correct mappings. This is our
"relative" recall.
11/11/2007 Consensus Building Workshop within Conference track
9
Evaluation...• Some statistics as a side-effect of processing
http://nb.vse.cz/~svabo/oaei2007/
11/11/2007 Consensus Building Workshop within Conference track
10
Outline…
• Introduction (ideas behind the track)
• Evaluation
• Discussion – ‘interesting’ mappings
11/11/2007 Consensus Building Workshop within Conference track
12
Our goal
• Focus on ‘interesting’ (unclear) mappings
• Discussion:– Why should they be mapped?
• Arguments against and for
• Track argumentation process
11/11/2007 Consensus Building Workshop within Conference track
13
Our goal...
• Discuss mappings – generally– wrt. application usage
• Recall methodology of Grüninger and Fox– ‘Identification of motivating scenarios’– ‘Elaboration of informal competency questions’– But transport it into ontology matching field!
11/11/2007 Consensus Building Workshop within Conference track
14
Use-case
• Transformation application:– transfer of data of the conference from one
'software tool for organising conference' to other 'software tool for organising conference‘
– Mapping ontologies correspond with ‘sw tools’
11/11/2007 Consensus Building Workshop within Conference track
15
Informal competency questions
• After the transfer of data from one tool to another, will it be correct wrt.– the conflict of interests of authors vs. reviewers?– the assignment of papers to potential reviewers?– temporal aspect of the reviewing process?– temporal aspect of submission phase of papers/camera-ready
versions?– registration requirements of participants of conference and
members of commissions?– presentation of the contribution by at least one participant– ...
11/11/2007 Consensus Building Workshop within Conference track
16
Mapping 1
Element1 Element2 Relation
confious:
has_author
pcs:submited_by =
Notes Domain(has_author)=article, Range(has_author)=human
Domain(submited_by)=DOCUMENT, Range(submited_by)=PERSON
System Lily 1
11/11/2007 Consensus Building Workshop within Conference track
17
Informal competency questions
• After the transfer of data from one tool to another, will it be correct wrt.– the conflict of interests of authors vs. reviewers?– the assignment of papers to potential reviewers?– temporal aspect of the reviewing process?– temporal aspect of submission phase of papers/camera-ready
versions?– registration requirements of participants of conference and
members of commissions?– presentation of the contribution by at least one participant– ...
11/11/2007 Consensus Building Workshop within Conference track
18
Mapping 2
Element1 Element2 Relation
ekaw:
Possible_
Reviewer
cmt:Reviewer =
Notes Equivalence or subsumption?
System Lily 1
11/11/2007 Consensus Building Workshop within Conference track
19
Mapping 2
Possible_Reviewer is domain of:reviewerOfPaperPossible_Reviewer is range of:hasReviewerReviewWrittenBy
ekaw.owl
cmt.owlReviewer is domain of:adjustBidassignedByAdministratorassignedExternalReviewerhasBeenAssignedmarkConflictOfInterestreadPaperwriteReviewReviewer is range of:adjustedByassignedByReviewerassignedToassignReviewerreadByReviewerwrittenBy
11/11/2007 Consensus Building Workshop within Conference track
20
Informal competency questions
• After the transfer of data from one tool to another, will it be correct wrt.– the conflict of interests of authors vs. reviewers?– the assignment of papers to potential reviewers?– temporal aspect of the reviewing process?– temporal aspect of submission phase of papers/camera-ready
versions?– registration requirements of participants of conference and
members of commissions?– presentation of the contribution by at least one participant– ...
11/11/2007 Consensus Building Workshop within Conference track
21
Mapping 3
Element1 Element2 Relation
ekaw,conference:
Conference_
Participant
Pcs: ConferenceMember =
Notes Semantically same?
System ASMOV 1, OLA 0.7(0.625)
11/11/2007 Consensus Building Workshop within Conference track
22
Mapping 3a
Conference.owlPCS.owl
ConferenceMember is domain ofmember_of_conferenceConferenceMember is range ofhas_conference_member
11/11/2007 Consensus Building Workshop within Conference track
23
Mapping 3b
PCS.owl
ConferenceMember is domain ofmember_of_conferenceConferenceMember is range ofhas_conference_member
ekaw.owl
11/11/2007 Consensus Building Workshop within Conference track
24
Informal competency questions
• After the transfer of data from one tool to another, will it be correct wrt.– the conflict of interests of authors vs. reviewers?– the assignment of papers to potential reviewers?– temporal aspect of the reviewing process?– temporal aspect of submission phase of papers/camera-ready
versions?– registration requirements of participants of conference and
members of commissions?– presentation of the contribution by at least one participant– ...
11/11/2007 Consensus Building Workshop within Conference track
25
Mapping 4
Element1 Element2 Relation
sigkdd:
Program_Committee_Member
Pcs: ConferenceMember
=
Notes Semantically same?
System ASMOV 1
11/11/2007 Consensus Building Workshop within Conference track
26
Mapping 4
sigkdd.owlPCS.owl
ConferenceMember is domain ofmember_of_conferenceConferenceMember is range ofhas_conference_member
11/11/2007 Consensus Building Workshop within Conference track
27
Informal competency questions
• After the transfer of data from one tool to another, will it be correct wrt.– the conflict of interests of authors vs. reviewers?– the assignment of papers to potential reviewers?– temporal aspect of the reviewing process?– temporal aspect of submission phase of papers/camera-ready
versions?– registration requirements of participants of conference and
members of commissions?– presentation of the contribution by at least one participant– ...
11/11/2007 Consensus Building Workshop within Conference track
28
Mapping 5
Element1 Element2 Relation
micro:
begins
crs_dr:
has_startdate
=
Notes Domain(begins)=Activity or Conference, Range(begins)=dateTime
Domain(has_startdate)=event, Range(has_startdate)=dateTime
System ASMOV 1
11/11/2007 Consensus Building Workshop within Conference track
29
Mapping 5Micro.owl
Activity is domain ofbeginsendsexpectedAttendancehasTopicisPartOfActivity is range ofisIncluded
crs_dr.owl
event is domain ofhas_enddatehas_startdateexpectedAttendancehasTopicisPartOf
Conference is domain ofannualOrderbeginsendsisOrganizedByisPartOftakesPlaceConference is range ofisPartOforganizes
11/11/2007 Consensus Building Workshop within Conference track
30
Informal competency questions
• After the transfer of data from one tool to another, will it be correct wrt.– the conflict of interests of authors vs. reviewers?– the assignment of papers to potential reviewers?– temporal aspect of the reviewing process?– temporal aspect of submission phase of papers/camera-ready
versions?– registration requirements of participants of conference and
members of commissions?– presentation of the contribution by at least one participant– ...
11/11/2007 Consensus Building Workshop within Conference track
31
Mapping 6Element1 Element2 Relation
sigkdd:
Name_of_
conference
pcs:
Topic_of_conference
=
Notes Domain(Name_of...)=Conference, Range(Name_of...)=string
Domain(Topic_of...)=Conference, Range(Topic_of...)=string
Not fully specified in ontologies to distinguish?
System ASMOV 1, LILY 1, OLA 0.84
11/11/2007 Consensus Building Workshop within Conference track
32
Informal competency questions
• After the transfer of data from one tool to another, will it be correct wrt.– the conflict of interests of authors vs. reviewers?– the assignment of papers to potential reviewers?– temporal aspect of the reviewing process?– temporal aspect of submission phase of papers/camera-ready
versions?– registration requirements of participants of conference and
members of commissions?– presentation of the contribution by at least one participant– ...
11/11/2007 Consensus Building Workshop within Conference track
33
Mapping 7
Element1 Element2 Relation
ekaw:
Individual_
Presentation
iasted:
Presentation
=
Notes Subsumption or equivalence?
System SEMA 1
11/11/2007 Consensus Building Workshop within Conference track
34
Mapping 7ekaw.owl
Individ... is domain ofpresentationOfPaperendsexpectedAttendancehasTopicisPartOfIndivid... is range ofpaperPresentedAs
Iasted.owl
11/11/2007 Consensus Building Workshop within Conference track
35
Informal competency questions
• After the transfer of data from one tool to another, will it be correct wrt.– the conflict of interests of authors vs. reviewers?– the assignment of papers to potential reviewers?– temporal aspect of the reviewing process?– temporal aspect of submission phase of papers/camera-ready
versions?– registration requirements of participants of conference and
members of commissions?– presentation of the contribution by at least one participant– ...
11/11/2007 Consensus Building Workshop within Conference track
36
Mapping 8
Element1 Element2 Relation
sigkdd:
Start_of_
conference
iasted:
is_dated_on
=
Notes Domain(Start...)=Conference, Range(Start...)=dateTime
Domain(is_dated_on)=Activity, Range(is_dated_on)=dateTime
System ASMOV 1, OLA 0.52
11/11/2007 Consensus Building Workshop within Conference track
37
Informal competency questions
• After the transfer of data from one tool to another, will it be correct wrt.– the conflict of interests of authors vs. reviewers?– the assignment of papers to potential reviewers?– temporal aspect of the reviewing process?– temporal aspect of submission phase of papers/camera-ready
versions?– registration requirements of participants of conference and
members of commissions?– presentation of the contribution by at least one participant– ...
11/11/2007 Consensus Building Workshop within Conference track
38
Mapping 9
Element1 Element2 Relation
openconf:
is_Type
paperdyne:
isTypeOf
=
Notes Domain(is_Type)=Paper_File, Range(is_Type)=Format
Domain(isTypeOf)=not_specified,
Range(isTypeOf )=string
System Falcon 0.94, Lily 1, OntoDNA 1
11/11/2007 Consensus Building Workshop within Conference track
39
Mapping 9
isTypeOf has subproperties:isAuthorTypeOfisPaperTypeOf
Domain(isAuthorTypeOf)=Reviewer or AuthorRange(isAuthorTypeOf)={“invited”,”registred”}
Domain(isPaperTypeOf)=PaperRange(isPaperTypeOf)={“invited”,”regular”}
11/11/2007 Consensus Building Workshop within Conference track
40
Informal competency questions
• After the transfer of data from one tool to another, will it be correct wrt.– the conflict of interests of authors vs. reviewers?– the assignment of papers to potential reviewers?– temporal aspect of the reviewing process?– temporal aspect of submission phase of papers/camera-ready
versions?– registration requirements of participants of conference and
members of commissions?– presentation of the contribution by at least one participant– ...
11/11/2007 Consensus Building Workshop within Conference track
41
Mapping 10
Element1 Element2 Relation
micro:
Topic
edas:
Topic
=
Notes Equivalence? – instances, subclasses
System ASMOV 1, Falcon 1, Lily 1, OLA 1, OntoDNA 1
11/11/2007 Consensus Building Workshop within Conference track
42
Mapping 10
edas.owlmicro.owl
...
Topic is domain of:isIncludedTopic is range of:hasTopic
Topic is domain of:isTopicOfTopic is range of:hasTopic
Compilers,ForwardLookingIdea,Microarchitecture,Systems,are instances of Topic
11/11/2007 Consensus Building Workshop within Conference track
43
Informal competency questions
• After the transfer of data from one tool to another, will it be correct wrt.– the conflict of interests of authors vs. reviewers?– the assignment of papers to potential reviewers?– temporal aspect of the reviewing process?– temporal aspect of submission phase of papers/camera-ready
versions?– registration requirements of participants of conference and
members of commissions?– presentation of the contribution by at least one participant– ...
11/11/2007 Consensus Building Workshop within Conference track
44
Mapping 11
Element1 Element2 Relation
ekaw:
Presenter
paperdyne:
Speaker
=
Notes Semantically equivalent?
System SEMA 1
11/11/2007 Consensus Building Workshop within Conference track
45
Mapping 11
ekaw.owl paperdyne.owl
Paper spokenBy only Speaker
11/11/2007 Consensus Building Workshop within Conference track
46
Informal competency questions
• After the transfer of data from one tool to another, will it be correct wrt.– the conflict of interests of authors vs. reviewers?– the assignment of papers to potential reviewers?– temporal aspect of the reviewing process?– temporal aspect of submission phase of papers/camera-ready
versions?– registration requirements of participants of conference and
members of commissions?– presentation of the contribution by at least one participant– ...
11/11/2007 Consensus Building Workshop within Conference track
47
Mapping 12
Element1 Element2 Relation
cocus:
Preview
edas:
Review
=
Notes Lexical confusion...
System Lily 1, OntoDNA 0.86
11/11/2007 Consensus Building Workshop within Conference track
48
Informal competency questions
• After the transfer of data from one tool to another, will it be correct wrt.– the conflict of interests of authors vs. reviewers?– the assignment of papers to potential reviewers?– temporal aspect of the reviewing process?– temporal aspect of submission phase of papers/camera-ready
versions?– registration requirements of participants of conference and
members of commissions?– presentation of the contribution by at least one participant– ...
11/11/2007 Consensus Building Workshop within Conference track
49
Mapping 13
Element1 Element2 Relation
conference:
is_submitted_at
confious:
is_concerned_
conference
=
Notes Domain(is_sub)=Conference_contribution, Range(is_sub)=Conference_volume
Inverse property is is_submitted_at
Domain(is_concerned)=,
Range(is_concerned)=conference
System ASMOV 1
11/11/2007 Consensus Building Workshop within Conference track
50
Mapping 13
Usage for ‘is_concerned_conference’:
cocus.owl
conference.owl
Conference_volume is subclass of Conference Inverse property of ‘is_submitted_at’ is ‘has_contributions’
11/11/2007 Consensus Building Workshop within Conference track
51
Informal competency questions
• After the transfer of data from one tool to another, will it be correct wrt.– the conflict of interests of authors vs. reviewers?– the assignment of papers to potential reviewers?– temporal aspect of the reviewing process?– temporal aspect of submission phase of papers/camera-ready
versions?– registration requirements of participants of conference and
members of commissions?– presentation of the contribution by at least one participant– ...
11/11/2007 Consensus Building Workshop within Conference track
52
Mapping 14Element1 Element2 Relation
ekaw:
coversTopic
edas,confof:
hasTopic=
Notesconfof:
Domain(hasTopic)=
Working_event
Range(hasTopic)=
Topic
Domain(coversTopic)=Paper or Event, Range(coversTopic)=Research_Topic
Event has Conference as a subclass
Domain(hasTopic)=Conference or Paper,
Range(isTopic)=Topic
Topic has many subclasses with topics
System LILY 1, OLA 0.55 (0.47)
11/11/2007 Consensus Building Workshop within Conference track
53
• Call for contribution to our dataset