Future of Search | Yury Lifshits, Yahoo! Research
-
Upload
yury-lifshits -
Category
Technology
-
view
2.675 -
download
0
description
Transcript of Future of Search | Yury Lifshits, Yahoo! Research
Yury LifshitsYahoo! Researchhttp://yury.name
Future of Search
St. Petersburg | Helsinki
December 2008
Outline
Structured Search Yahoo! Work in Search
SearchMonkey BOSS
Research Agenda
Structured Search:work in progress
Structured Search =Bring structured data to search users
M.K. Bergman. The Deep Web: Surfacing Hidden Value. 2001.
Value Proposition
Coverage Real-time data Semi-private data
Structured queries Ordering and filtering results Straight-to-answers
User Interface: Query Search assist: Yahoo! Selector: LinkedIn, VKontakte.ru Multiple search buttons: Gmail Search tabs: Yahoo / Google
User Interface: Results
Federated page Facets Search transfer / search form
K.P. Yee, K. Swearingen, K. Li, M. Hearst. Faceted metadata for image search and browsing. CHI 2003.
Fernando Diaz. Aggregation of News Content Into Web Results. WSDM 2009.
http://glue.yahoo.com http://au.alpha.yahoo.com
Data Supply Chain
Atomic fact Flight, Event, Patent
Data aggregatorUS Patents, Amadeus/Sabre flights, Upcoming.com
Domain searchExpedia, Spock
General purpose searchYahoo!, Google, Yandex, Baidu
Getting structured data
Entity extraction Markup Feeds Search API (OpenSearch)
OR
Do a search transfer
Give Us Your Data For …
Traffic via search transferFirefox search box
Better presentation in search SearchMonkey
Hosted searchBOSS Custom
Showing your adsYahoo Local + AT&T
Yahoo! Work in Search
Slides by:Paul Tarjan, Chief Technical Monkey
Full version http://www.slideshare.net/ptarjan/searchmonkey-presentation
an open platform for using structured data to build more useful and relevant search results
Before After
What is SearchMonkey?
Enhanced Result: Zagat
Key/Value Pairsor Abstract
LinksImage
Infobar: Wikipedia Preview
Summary Blob
Creating an Infobar
Infobar advantages Annotate someone else’s site Use links and images from other domains
• Mash up info from multiple sites• Affiliate / coupon links? Hmmm…
Can act on *, all websites• But these apps can be annoying if poorly
designed
Key design principles Put something useful in the summary Be creative with the HTML
How to get data to SearchMonkey?
Humans see:• name• picture of a person• current job• industry, …
Computers see:an undifferentiatedblob of HTML
Can we make computers smarter?
How does it work?
Acme.com’sdatabase
Index
RDF/Microformat Markup
site owners/publishers share structured data with Yahoo!. 1
consumers customize their search experience with Enhanced Results or Infobars
3
site owners & third-party developers build SearchMonkey apps.2
DataRSS feed
Web Services
Page Extraction
Acme.com’s Web Pages
SearchMonkey Resources
Main: http://developer.yahoo.com/
searchmonkey
Lists and forums: searchmonkey-
[email protected] http://suggestions.yahoo.com/
searchmonkey
Vik Singh (Architect)Graham Mudd (Senior PMM)
BOSS = Build your Own Search Service
Open Yahoo’s core search features via web services to let 3rd parties revolutionize Search
Unrestricted
What
Unrestricted:
• Unlimited queries• Blend, re-order, discard• Full presentation control• Non-search apps OK
Monetization: Free or CPM or Ads
What
Barriers to entry are massive• $300M, top talent, a prayer to get to basic parity
No monopoly over great ideas
Search anywhere• Improve Vertical Quality w/ Web comprehensiveness• Fragment the market, foster more players, choice, competition
Yahoo extends advertising reach, 3rd parties revenue share
Why
Why
Traditional Search Distribution
+ BOSS Distribution
Tracks
API
A self-service, web services model for developers and start-ups to quickly build and deploy new search experiences.
• UIUC• CMU• Stanford• Purdue• IIT Bombay• MIT• UMass
CUSTOM
Working with 3rd parties to build a more relevant, brand/site specific web search experience.
This option is jointly built by Yahoo! and select partners.
ACADEMIC
Working with the following universities to allow for wide-scale research in the search field:
Interested in Custom? Email us [email protected]
http://boss.yahooapis.com/ysearch/{vert}/v1/{q}
{vert} := {web, news, images, spelling}
@ requiredappid
@ optional (Y!OS compliant)start, count, lang, region, format, callback, sites
BOSS API v1
Python (v2.5+) library
BOSS Search SDK plus …
SQL for remixing arbitrary XML/JSON sources
Loosely Functional programming paradigm
BOSS Mashup Framework
Ported enhanced version of BMF to GAE platform
http://zooie.wordpress.com/2008/08/04/yahoo-boss-google-app-engine-integrated/
Easiest way to deploy a BOSS application online
BMF + Google App Engine
http://www.4hoursearch.com
http://123people.com
Mashable! Contest for BOSS search engineshttp://mashable.com/boss/
Examples
BOSS Custom for TechCrunch
TechCrunch Network Search
CrunchBase + Posts + Web Sort by time / relevance Enhanced results Domain-specific facets Yahoo! sponsored search Real-time indexing Special results
Research Agenda
Structured Search
Analysis of search demand Intent classification General search vs. vertical
Incentives in data supply Push & real-time indexing Search user interface
One box vs. multi-box General vs. vertical
Deciding search transfer When? To whom?
Key Scientific ChallengesDraft: http://research.yahoo.com/ksc
1. Search intent2. Quality metrics3. Web mining4. Multilingual IR5. Nextgen search
Synthesized result pages
6. World knowledge
A.Z. Broder. Taxonomy of web search. SIGIR 2002.
More Problems
Discovery search
Web search vs. asking people
Event search
Thanks for your attention!
Yury Lifshits http://yury.name [email protected]