API's, Freebase, and the Collaborative Semantic web
-
Upload
dan-delany -
Category
Technology
-
view
9.043 -
download
2
description
Transcript of API's, Freebase, and the Collaborative Semantic web
![Page 1: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/1.jpg)
API’s, Freebase, and the Collaborative Semantic Web
Diana Tamabayeva & Dan Delany
![Page 3: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/3.jpg)
Object‐Oriented Representation
Structured data in object ‐ useful to
computers
“JohnSmithisafive‐foot‐tallmalewhoweighsone‐hundredeightypounds.He’s
forty‐twoyearsoldandhelovesdogs.”
String of characters ‐ useful to people
sortByHeight(users);
isADude(“JohnSmith”);
![Page 4: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/4.jpg)
Object‐Oriented Representation
classDog<Mammal@home,@age,@humandefgoHome
go(@home);end
classHuman<Mammal@home,@age,@pets@spouse,@friendsdefdivorce
@spouse=nil;end
classPlace@latlong,@elevation,@address,@namedefisBelowSeaLevel
@elevation<0;end
Home
HomeHuman
Pet
SpouseFriends
O‐O instance variables and methods represent structured semantic relationships between pieces of information.
![Page 5: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/5.jpg)
Structured Semantic Relationships Among Objects
We visualize this with a semantic graph.
![Page 6: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/6.jpg)
LAMP
The (hypertext) Web Today
WebServers
WWW
UserComputer
![Page 7: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/7.jpg)
ServersServers
The (semantic) Web Tomorrow?
UserComputer
Servers
DataAggregator/Visualizer
StructuredData(OWL/RDF)DocumentData(HTML/CSS)Hyperlinks(Oldwwwlinks)
![Page 8: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/8.jpg)
A Cloud that Talks To Itself.
vs.
![Page 9: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/9.jpg)
Why? Searchability.
With semantic graphs, you can perform semantic searches by traversing the graph:
“Where does the woman who lives at 2408 Walnut work?”
![Page 10: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/10.jpg)
Why? So You Can Mash it Up.
mapmash.googlepages.com/gaza.html
Mashups create meaning from data.housingmaps.com
twittermap.com
![Page 11: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/11.jpg)
Example: Image Analysis
Why? Context‐Aware AI
Today’s AI is limited by the domain‐specificity of its input data
Context‐Unaware: Blob Tracking
Blob2Blob1
Useful in it’s domain (in this case, touch detection), but ultimately has no ‘intelligence.’
![Page 12: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/12.jpg)
Example: Image Analysis
Why? Context‐Aware AI
Today’s AI is limited by the domain‐specificity of its input data
Context‐Aware: Mine the semantic graph for heuristics and clues
DanRachel Taifur
TreesSky
You’re in the mountains near Aspen, CO. It is the fall. Your friends Taifur and Rachel are atthe same location. Etc.
![Page 13: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/13.jpg)
Why? Context‐Aware AI
I have a dream for the Web [in which computers] become capable of analyzing all the data on the Web ‐ the content, links, and transactions between people and computers. A ‘Semantic Web’, which should make this possible, has yet to emerge, but when it does, the day‐to‐day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines.
The ‘intelligent agents’ people have touted for ages will finally materialize.
Tim Berners‐Lee, 1999
![Page 14: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/14.jpg)
14
Challenges ‐ Motivation/Critical Mass
Value comes from ubiquity, and ubiquity comes from value.How do we encourage adoption of technologythat does not yet provide value?
More importantly, how will it provide value to content producers? How is giving away your data’s meaning (your secret sauce) instead of presenting it alongside ads valuable?
Should semantic feeds be monetized?“Information As A Service”
![Page 15: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/15.jpg)
Challenges ‐ Privacy/Security
• ReducedanonymityontheWeb• Increasedinvasionofprivacy• SoluSon:accessprivilegesmust
becontrolledbyinfrastructure‐levelsecurity
![Page 16: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/16.jpg)
The State of the Semantic Web
![Page 17: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/17.jpg)
Top‐Down: Content Structuring•SemanScsearchenginesgenerateandleverageaninternalsemanScdatabase•DatasSllreturnedasHTML,noAPInothelpingcreatethesemanScweb!
![Page 18: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/18.jpg)
Top‐Down: Dapper
WWW
RSS
CSV
OWL
RDF
tosemanScweb
•Dapper‐UIforseXnguprulestoscrapesemi‐structureddata(CSV,RSS,XML)fromanysetofHTMLdocuments!
![Page 19: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/19.jpg)
Top‐Down: Yahoo! Pipes
RSS OWL
RDF
tosemanScweb
•Yahoo!Pipes‐UIfortransforming,reformaXng,andcombiningdatafeedsintomoreusefuldatafeeds. JSON
JSON
CSV
RSS
![Page 20: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/20.jpg)
Bottom‐Up: Publishing Standards
RDF and OWL standards•W3C‐sponsoredstandardsfordefiningsemanScrelaSonshipsandresources•Powerful,butcomplexandhardforhumanstoread/create•Nomo@va@onfordeveloperstocreate•NoW3C‐sponsoreduniversalontologies
1999
Dan’sCar CarVehicle Color
isA hasA
HasMul*ple(4)
RDF Data Node OWL Ontology
make
year
color
<rdf:Description rdf:about="http://.../DansCar"> <car:color>Red</car:color> <car:make>Honda</make> <car:year>1999</make>
owl:Class rdf:ID="Car" rdfs:subClassOf rdf:resource="#Vehicle" rdfs:subClassOf [a owl:Restriction; owl:cardinality "4"^^xsd:nonNegativeInteger; owl:onProperty <#Wheel> ]
![Page 21: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/21.jpg)
API’s: I dream of a RESTful tomorrow
Thousands of content creators are already sharing structured data with API’s!
![Page 22: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/22.jpg)
Bottom‐Up: Knowledge Bases
•CollaboraSvea_empttobuildfactualknowledgebase•“Object‐OrientedWikipedia”
DomainExperts
![Page 23: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/23.jpg)
Bottom‐Up: Knowledge Bases
•Contentcontributedbyexpertsmanually,orbydatasetownersautomaScally.
DomainExperts
Seman@cAnalysis
![Page 24: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/24.jpg)
Freebase: The Everything Graph
![Page 25: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/25.jpg)
Metaweb Freebase Statistics
•FreebaseLaunch,March2007•LLCFounded,July2005
•25,379userstoday.•Growingby600‐800/month
•5.3milliontopicstoday•Growingby~15,000/month•Pulledfrompublicdata•Bycomparison,Wikipediahas2.64millionEnglisharScles
![Page 26: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/26.jpg)
The Freebase ApproachCrea@veCommonsALribu@onLicense
CasualCollaborators
NodeEditorGUIApp Developers
Data Modelers
Expert Users &Dataset Owners
OpenAPI(MQL)
WWW
WWW
Exis@ngDatasets
ContentPublishers
RDFAPI
![Page 27: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/27.jpg)
Freebase Data PolicyCrea@veCommonsALribu@onLicense
OpenAPI(MQL)RD
FAPI
•RequiresA_ribuSonofSource
[{ "album" : { "artist" : [], "name" : null, "release_date" : null }, "limit" : 25, "name" : null, "name~=" : "Love*", "type" : "/music/track" }]
MQLQuery:
Returns:[{ "album" : { "artist" : ["Massive Attack"], "name" : "Blue Lines", "release_date" : "1991-04-08" }, "name" : "One Love", "type" : "/music/track"},{"album" : { "artist" : [“Squirrel Nut Zippers"], "name" : "The Inevitable", "release_date" : "1995-03-17" }, "name" : "Anything But Love", "type" : "/music/track" }, { "album" : { "artist" : [ "PJ Harvey" ], "name" : "Dry", "release_date" : "1992-02-11" }, "name" : "Oh My Lover", "type" : "/music/track" },
DataFreedom!
![Page 28: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/28.jpg)
The Freebase ApproachCrea@veCommonsALribu@onLicense
CasualCollaborators
NodeEditorGUIApp Developers
Data Modelers
Expert Users &Dataset Owners
OpenAPI(MQL)
WWW
WWW
Exis@ngDatasets
ContentPublishers
RDFAPI
![Page 29: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/29.jpg)
Freebase Community
CasualCollaborators
App Developers
Data Modelers
Expert Users &Dataset Owners
AppDevelopersList
DataModelersList
IRCChat(opentoall)
DiscussionThreadsonIndividualTopics
•NoCentralForum•NoBacklogofMailingList•NoFriends•NoPrivateMessages
![Page 30: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/30.jpg)
“Asaprogrammer,IfeelthatI'mmosteffecSvewhenI'mcontribuSng
largedatasets…faciliSesbuiltintoFreebasethatletusersuploadlistsoftopicsarelimitedtospecificsituaSons.”‐ Shawn Simister
Freebase Community Tools
CasualCollaborators
App Developers
Data Modelers
Expert Users &Dataset Owners
“Acre,theFreebaseapplicaSondevelopmentplaqorm,letsanyonemashupFreebasedatausingJavascriptandhaveithostedforfree.”‐ Shawn Simister, developer
Employees
“Wehavebeenworkinghardrecentlytoprovide
bulkimporttoolsforFreebase.Whilesuchtoolsexistinternally,thereconciliaSonprocesshasthusfarbeentoocomplicatedforpublicrelease.”‐ Brian Culbertson, Metaweb Engineer
“[DataModelingishardbecausenewschemasareaslowprocess,andtheycanbreakusers’code.Let’shavea
“SloppyFreebase”thatallowsuserstoenterunstructureddataunSlnewschemasaredefined.]”‐Jack Alves, former Metaweb Director of Engineering
![Page 31: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/31.jpg)
“Sloppy” Data Modeling
“[DataModelingishardbecausenewschemasareaslowprocess,andtheycanbreakusers’code.Let’shavea
“SloppyFreebase”thatallowsuserstoenterunstructureddataunSlnewschemasaredefined.]”‐Jack Alves, former Metaweb Director of Engineering
Currently, Freebase users cannot submit data if there is not already a data structure + ontology built for that data type.
“Sloppy” data creation allows users to create their own data types, which will later be cleaned and standardized.
CasualCollaborators
Bob Jonesmajor: English
Eric Bradleystudying:
CS
John Greenfocus:
Sociology
User‐Generated, Semi‐structured “sloppy” data
Automated DataCleaner‐Upper
Bob Jonesarea of study:
English
Eric Bradleyarea of study:
CS
John Greenarea of study:
Sociology
Clean, structured data
OWL
RDF
tosemanScweb
![Page 32: API's, Freebase, and the Collaborative Semantic web](https://reader034.fdocuments.net/reader034/viewer/2022051411/54708739b4af9fa30a8b47a4/html5/thumbnails/32.jpg)
Conclusion
Wearehere. • Progress• RDF/OWL• Freebase‐5.3mil.topics!• Dapper,Yahoo!Pipes,other
dataabstractors• API’soutthewazoo
• Issues• AdopSon• Privacy/Security• IntellectualProperty• We’regoodatmaking
content,butwesuckatminingitanddescribingit.