Using Solr in Online Travel Shopping to Improve User Experience
-
Upload
lucidworks-archived -
Category
Technology
-
view
3.974 -
download
4
Transcript of Using Solr in Online Travel Shopping to Improve User Experience
![Page 1: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/1.jpg)
Using Solr in Online Travel to Improve User Experience
Sudhakar Karegowdra, Esteban Donato Travelocity, May 25TH 2011
{ sudhakar.karegowdra, esteban.donato}@travelocity.com
![Page 2: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/2.jpg)
What We Will Cover § Travelocity § Speakers Background § Merchandising & Solr
• Challenges • Solution • Sizing and performance data • Take Away
§ Location Resolution & Solr • Challenges • Solution • Sizing and performance data • Take Away
§ Q&A 3
![Page 3: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/3.jpg)
§ First Online Travel Agency(OTA) Launched in 1996 § Grown to 3,000 employees and is one of the largest
travel agencies worldwide § Headquartered in Dallas/Fort Worth with satellite
offices in San Francisco, New York, London, Singapore, Bangalore, Buenos Aires to name a few
§ In 2004, the Roaming Gnome became the centerpiece of marketing efforts and has become an international pop icon
§ Owned by Sabre Holdings - sister companies include Travelocity Business, IgoUgo.com, lastminute.com, Zuji among others
4
![Page 4: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/4.jpg)
Speakers Background
§ Esteban Donato • Lead Architect Travelocity.com
§ My experience – 10 + years – Solr 2 years – Analyzing Mahout and
Carrot2 for document clustering engine.
§ Topic : Location Resolution
5
§ Sudhakar Karegowdra • Principal Architect
Travelocity.com § My experience
– 13 + years – Solr/ Lucene 3 years – Implementing Hadoop,
Pig and Hive for Data warehouse.
§ Topic : Merchandising
![Page 5: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/5.jpg)
6
Merchandising By Sudhakar Karegowdra
![Page 6: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/6.jpg)
The Challenge § Market Drivers
• Build Landing Pages with Faceted Navigation • Enable Content Segmentation and delivery • Support Roll out of Promotions • Roll up Data to a higher level
§ E.g., All 5 star hotels in California to bring all the 5 Star hotels from SFO,LAX, SAN etc.,
• Faster time to market new Ideas • Rapidly scale to accommodate global brands
with disparate data sources
7
![Page 7: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/7.jpg)
The Challenge § Traditional Database approach
• Higher time to market • Specialized skill set to design and optimize
database structures and queries • Aggregation of data and changing of structures
quite complex • Building Faceted navigation capabilities needs
complex logic leading to high maintenance cost
8
![Page 8: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/8.jpg)
Solution - Overview § Data from various sources aggregated and
ingested into Solr • Core per Locale and Product Type
§ Wrapper service to combine some data across
product cores and manage configuration rules
§ Solr’s built in Search and Faceting to power the navigation
9
![Page 9: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/9.jpg)
Solution – Architecture View
10
Solr Master (Multi Core)
Oracle
Offer Management
Tool ETL
Services/Business Logic
UI Widgets Mobile
Deals Products ……
Solr Slaves (Multi Core)
![Page 10: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/10.jpg)
Solution - Achievements § Millions of unique Long Tail Landing Pages
§ E.g., http://www.travelocity.com/hotel-d4980-nevada-las-vegas-hotels_5-star_business-center_green
§ Faster search across products § E.g., Beach Deals under $500
§ Segmented Content delivery through tagging § Scaled well to distribute the content to different
brands, partners and advertisers § Opened up for other innovative applications
§ Deals on Map, Deals on Mobile, Wizards etc.,
11
![Page 11: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/11.jpg)
Solution – Road Ahead § Migration to Solr 3.1
• Geo spatial search • CSV out put format
§ Query boosting by Search pattern § Near Real time Updates § Deal and user behavior mining in Hadoop –
MapReduce and Solr to Serve the Content § Move Slaves to Cloud
12
![Page 12: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/12.jpg)
Sizing & Performance § Index Stats
§ Number of Cores : 25 § Number of Documents : ~ 1 Million Records
§ Response § Requests : 70 tps § Average response time : 0.005 seconds (5 ms)
§ Software Versions § Solr Version 1.4.0
– filterCache size : 30000
§ Tomcat – 5.5.9 § JDK1.6
13
![Page 13: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/13.jpg)
Take Away § Semi Structured Storage in Solr helps
aggregate disparate sources easily Remember Dynamic fields
§ Multiple Cores to manage multiple locale data
§ Solr is a great enabler of “Innovations”
14
![Page 14: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/14.jpg)
15
Location Resolution By Esteban Donato
![Page 15: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/15.jpg)
The Challenge § How to develop a global location resolution
service? § Flexibility to changes § General enough to cover everyone needs § Multi language § Performance and scalability § Configurable by site
16
![Page 16: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/16.jpg)
Architecture of the solution
17
Location DB
Solr Master
Solr Slave
Management Tool
Auto-complete Resolution
Batch Job
§ Remote Streaming indexing § CSV format
§ Master/Slave architecture § Multi-core: each core represents a language § SolrJ client binary format § Solr response cache
![Page 17: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/17.jpg)
Auto-complete § System has to suggest options as the users
type their desired location § Examples “san” => San Francisco, “veg” =>
Las Vegas § Relevancy: not all the locations are equally
important. “par” => “Paris, France”; “Parana, Argentina”
§ Users can search by various fields: location code, location name, city code, city name, state/province code, state province name, country code, country name.
18
![Page 18: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/18.jpg)
Solr schema <dynamicField name="RANK*" type="int" required="false" indexed="true" stored="true" />
<field name="GLS_FULL_SEARCH" type="glsSearchField" required="false" indexed="true" stored="false" multiValued="true" />
<fieldType name="glsSearchField" class="solr.TextField" positionIncrementGap="100“>
<analyzer>
<tokenizer class="solr.PatternTokenizerFactory" pattern="[/\-\t ]+" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.TrimFilterFactory" />
<filter class="solr.ISOLatin1AccentFilterFactory" />
<filter class="solr.RemoveDuplicatesTokenFilterFactory" />
<filter class="solr.PatternReplaceFilterFactory" pattern="[,.]" replacement="" replace="all"/>
</analyzer>
</fieldType>
19
![Page 19: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/19.jpg)
Resolution § System has to resolve the location requested
by the users. § Contemplates aliases. Big Apple => New York § Contemplates ambiguities. § Contemplates misspellings. Lomdon => London
§ NGramDistance algorithm. § How to combine distance with relevancy § Error suggesting the correct location when it is a prefix.
Lond => London
20
![Page 20: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/20.jpg)
Spellchecker configuration <fieldType name=" spellcheckType " class="solr.TextField" positionIncrementGap="100“>
<analyzer>
<tokenizer class="solr.KeywordTokenizerFactory” />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.TrimFilterFactory" />
<filter class="solr.ISOLatin1AccentFilterFactory" />
<filter class="solr.RemoveDuplicatesTokenFilterFactory" />
<filter class="solr.PatternReplaceFilterFactory" pattern="[,.]" replacement="" replace="all"/>
</analyzer>
</fieldType>
21
![Page 21: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/21.jpg)
Sizing & Performance § 4 cores with ~ 500,000 documents indexed
each § Response times
• Auto-complete: 15ms, 20 TPS • Resolution: 10ms, 2 TPS
§ Cache configuration • queryResultCache: maxSize=1024 • documentCache, maxSize=1024 • fieldValueCache & filterCache disabled
22
![Page 22: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/22.jpg)
Wrap Up § Performance always as top priority § Develop simple but robust services § Provide a simple API
23
![Page 23: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/23.jpg)
Q&A
24
![Page 24: Using Solr in Online Travel Shopping to Improve User Experience](https://reader034.fdocuments.net/reader034/viewer/2022042714/555097f2b4c90595208b46f1/html5/thumbnails/24.jpg)
Contact § Esteban Donato
• [email protected] • Twitter: @eddonato
§ Sudhakar Karegowdra • [email protected] • Twitter: @skaregowdra https://www.facebook.com/travelocity Twitter: @travelocity and @RoamingGnome
25