Attempting to Use Wikipedia Categories to Improve Retrieval

INEX Linked Data Ad Hoc Track 2012

David MasseyABI Kveik, 1st March 2013

Task Description

• 3.2M documents from English language Wikipedia

• 140 queries• Return a ranked list with 1000 documents for

each query• Use Linked Data

Document Collection

• Approx. 30% of the files describe deleted files, images, etc.

• XML-like documents - Regex• Missing documents• Each document consists of three parts:

1. Wikipedia article2. DBPedia properties3. Yago properties

<lodxml xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xml:lang='en' xmlns:xhtml='http://www.w3.org/1999/xhtml' encoding='UTF-8'><article title='73425'><wikipedia><paragraph><template type='Metadata'><arg></arg><tag name='id'>73425</tag><tag name='title'>The_Deer_Hunter</tag></template>

<template type='Other uses'><arg></arg><arg>Deer Hunter (disambiguation)</arg></template>

<infobox type='film'><tag name='name'>The Deer Hunter</tag>...

<tag name='director'><link><wikilink href='./f4/05/522346.xml'>Michael Cimino</wikilink><dbpedia href='http://dbpedia.org/resource/Michael_Cimino'></dbpedia><yago ref='Michael_Cimino'></yago></link></tag>

<dbpediaproperties><property name='http://dbpedia.org/ontology/thumbnail'><object name='http://upload.wikimedia.org/wikipedia/commons/thumb/5/57/The_Deer_Hunter_poster.jpg/200px-The_Deer_Hunter_poster.jpg'></object></property>

...

<yagoproperties><property name='hasDuration'><object name='10920.0#s'></object></property><property name='isCalled'><object name='A szarvasvad\u00e1sz'></object></property>

The_Deer_Hunter

guitar chord tuningguitar chord minorguitar classical flamencoguitar classical bachguitar origin Russiaguitar origin bluestango culture moviestango culture countriestango music composerstango music instrumentstango dance stylestango dance history

vietnam war movievietnam war factsvietnam food recipesvietnamese food blogvietnam travel national parkvietnam travel airportsbicycle sport racesbicycle sport disciplinesbicycle holiday naturebicycle benefits healthbicycle benefits environmentfemale rock singerssouth korean girl groupselectronic music genresdigital music notation formatsmusic conferencesintellectual property rights lobby

Queries

Two stage approach

1. Traditional retrieval2. Improve by:– Using links between documents– Using categories– Using Linked Data

Stage One

• Extract title, headings and categories from documents

• Index using Indri – Krovetz stemming, stopword list

• Weighted search – Title (10), Category (5), H2 (2), H3 (1)

• Smoothing (ask Michael)

Vietnam_War_Crimes_Working_GroupVietnam_War_in_filmOperation_Sunrise_(Vietnam_War)Vietnam_War_Story_IIBook:Vietnam_WarVietnam_during_World_War_IVietnam_War_casualtiesVietnam_War_Crimes_Working_Group_FilesPuerto_Ricans_Missing_in_Action_in_the_Vietnam_WarStar_Wars_Mini_Movie_AwardsVietnam_War_Memorial,_Hanoi17th_Parallel:_Vietnam_in_WarVietnam:_The_Camera_At_WarMarch_Against_the_Vietnam_War1960_in_the_Vietnam_War1961_in_the_Vietnam_WarList_of_Vietnam_War_flying_acesList_of_wars_involving_VietnamOutline_of_the_Vietnam_WarThe_War_Within:_America&apos;s_Battle_over_VietnamVietnam:_The_Ten_Thousand_Day_WarProtests_against_the_Vietnam_WarMatterhorn:_A_Novel_of_the_Vietnam_WarList_of_bombs_in_the_Vietnam_WarPuerto_Ricans_in_the_Vietnam_WarMilitary_history_of_Australia_during_the_Vietnam_War

Result After Stage One

Query: Vietnam War Movie

Stage Two

• Links between documents• Categories• Linked Data• …

Expand Query with Wordnet Synonyms

Original query: vietnam war movie

vietnam -> annamwar -> warfaremovie -> film flick pic picture

Expanded query: vietnam annam war warfare film flick movie pic picture

Calculate Text Similarity between Expanded Query and Category Name

Levenshtein distance: "The smallest number of insertions, deletions, and substitutions required to change one string or tree into another. " NIST (http://xlinux.nist.gov/dads/HTML/Levenshtein.html)

Original query: Vietnam War MovieExpanded query: Vietnam Annam War Warefare Film Flick Movie Pic Picture

vietnam annam war warfare film flick movie pic picture

vietnam 0 4 6 7 5 7 6 6 5

war 6 4 0 4 4 5 5 3 6

films 6 5 5 7 1 4 5 4 6

Category: Vietnam War Films = 1 + (0*0 + 0*0 + 1*1) = 2 / 3 = 0.66

0.66 Vietnam War films1 War films2.33 Star Wars films2.75 Star Wars fan films3 Fan films3 Punic Wars3 Star Wars3.66 Gulf War films3.75 World War I films5.5 Barbary Wars5.5 Boer Wars5.5 Civil wars5.5 Guild Wars5.5 Opium Wars5.66 Vietnam War books5.66 Vietnam War novels5.66 Vietnam War sites5.75 World War II media6.33 Flags of Vietnam6.33 Laws of war6.33 Media of Vietnam6.33 MTV Movie Awards6.8 Women in World War I7 Floods in Vietnam7 Songs of the Vietnam War7.33 Star Wars books7.33 Star Wars comics7.5 War crimes in Vietnam7.5 World War I games7.5 World War II comics

Threshold < 1

Categories ranked by similarity to expanded query

Query: Vietnam War Movie

vietnam annam war warfare film flick movie pic picture

star 5 4 2 5 4 5 5 4 5

wars 7 5 1 4 4 5 5 4 6

Problems

Category: Star Wars = 1 + (2*2 + 1*1) = 6 / 2 = 3

Frosker = Forsker?

Homonyms – Vietnam War Picture

Missing categories

Expanded Query: Vietnam Annam War Warefare Film Flick Movie Pic Picture

We_Were_SoldiersA_Better_Tomorrow_3The_War_(film)Faith_of_My_Fathers_(film)Combat_Shock The_Killing_Fields_(film)A_Bright_Shining_LieApocalypse_Now_ReduxFlight_of_the_IntruderDead_PresidentsR-PointThe_Last_HunterThere_Is_No_13Deceit_(2009_film)The_Ballad_of_Andy_CrockerSome_Kind_of_HeroThe_Deer_HunterA_Rumor_of_War_(miniseries)Platoon_(film)The_Crazy_World_of_Julius_VrooderThou_Shalt_Not_Kill..._Except1969_(film)A.W.O.L._(2006_film)The_Siege_of_Firebase_GloriaAlamo_BayRolling_Thunder_(film)

Result After Stage Two

Query: Vietnam War Movie

vietnam war movie '11-1------' '11--------'vietnam war facts '11--11111-' '11--11111-'vietnam food recipes '------1---' '------1---'vietnamese food blog '----------' '----------'vietnam travel national park '1111111111' '1111111111'vietnam travel airports '-1-111----' '-1-11-1---'guitar chord tuning '111111111-' '111111111-'guitar chord minor '11111--11-' '11111--11-'guitar classical flamenco '----1-----' '----1-----'guitar classical bach '1-11--11--' '1-11--11--'guitar origin Russia '----------' '----------'guitar origin blues '1-1-------' '1-1-------'tango culture movies '---1--1---' '---1--1---'tango culture countries '---1-1---1' '---1-1---1'tango music composers '-1--------' '---1-1---1'tango music instruments '----------' '----------'tango dance styles '11--------' '----------'tango dance history '111-------' '111-------'bicycle sport races '111-1--1--' '---1111-1-'bicycle sport disciplines '----1-----' '----1-----'bicycle holiday nature '----------' '----------'bicycle benefits health '------1---' '------1---'bicycle benefits environment '---------1' '---------1'female rock singers '1-------1-' '1-1--1--1-'south korean girl groups '----------' '111111-111'electronic music genres '1-1-1-----' '-1-1------'digital music notation formats '-111-1111-' '-111-1111-'music conferences '11111-1---' '--11111-1-'intellectual property rights lobby '111-1-1111' '111-1-1111'

Original Query Result Stage 1 Result Stage 2

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Stage One Stage Two Best

Precision and RecallP

R

Literature

Kaptein, R., Koolen, M., & Kamps, J. (2009, July). Using Wikipedia categories for ad hoc search. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval (pp. 824-825). ACM.

Vercoustre, A. M., Pehcevski, J., & Thom, J. (2008). Using wikipedia categories and links in entity ranking. Focused Access to XML Documents, 321-335.

Illustration

http://www.flickr.com/photos/pasukaru76/6196321318/