Enhancing Search Using Lucenesymposiumna.s3.amazonaws.com/2012/Developer-Enhancing Searc… ·...

Amsterdam Las Vegas Melbourne

Enhancing Search Using Lucene

Las Vegas, October 22nd, 2012

Scott Rogers, Sean MacLean

CX Interactive


Introductions

• Scott Rogers – Director, Application Development

• Sean MacLean – Senior Application Developer

• CX Interactive – Digital agency, specializing in web based solutions

• I cannot do anything about my Canadian accent, eh!


Business Problem

• Need to allow searching over full text of

documents (in a variety of formats)

• Need to allow filtering on meta data (data

about the data)

• Replacing an older system with poor data–

garbage in, garbage out


User Requirements

• Capital markets firms

– Investment dealer (includes investment banking, sales & trading, advisory and product management)

– Publish multiple documents daily

– Documents may be relevant that day (e.g. market reports - date based), or later (text or meta data searching)


Technology Problem

• Many document formats not fully indexed by ‘out of the box’ Lucene

• Meta data can be searchable, filterable is harder

• Needed to connect documents to Sitecore content item

• How do we build a system that doesn’t require code modifications as content & meta data is added, changed or removed?


Design Overview


Design Overview

Design Ideals


Design Overview

Configurable

Design Ideals


Design Overview

Configurable Flexible

Design Ideals


Design Overview


Extensible

Design Ideals


Design Overview


Extensible Puppies, Rainbows

and Unicorns

Design Ideals


Solution Overview

Content Browser

Search Manager

Configuration / Sublayouts


Solution Overview

Content Browser

Search Manager


Something to display

items


Solution Overview

Content Browser

Search Manager


Something to search

items


Solution Overview

Content Browser

Search Manager


Some way to configure


Solution Details

Update Panels

JQuery

Dynamic Row Controls and ‘Lazy’ Loading

Content Browser


Solution Details

Sitecore Search API (Lucene)

Custom Indexer (Tika)

Index Search Context

Search Manager


Solution Details

Sitecore Content Tree

Templates

Layouts and Devices

Configuration


Search Manager Overview

Input: Query Text + IFilters

Output: Search Result Collection

Uses the Sitecore Search Api (Lucene) and Custom file crawler to provide context based search

IKVM wrapped Java-based Apache Tika providing linkage of parsed documents back to content items


Search Manager Overview

Sitecore

Pdfs

Cxi IndexCrawler

Lucene Search Manager Content

Query + Filters

Search Results Collection

IFilters SearchResults

Search Context Query Search


Search Manager Details

• Search (String query, List<IFilter> filters)




• SearchResultCollection GetSearchResults()





• SearchManager Class extensible for complex

data



• Search(String query, List<IFilter> filters)


• SearchManager Class extensible for complex data

• CxiIndexCrawler dynamically parses and links file content to its owner content item.


Content Browser Overview

Custom UserControl that extends UpdatePanel and is Content and Filter ‘Agnostic’

Large update panel used, along with JQuery Ajax calls for the previews and slide downs to link components together

Allowed the main browser to point at a datasource, and add any number of facets without changes to underlying code



Browser (filters)

Facets

Update Panel



• Search Results ListView

• Facets placeholder

• Pager control

• Options such as tabs using pre-assigned

filters and sorted results



• A ‘summary’ row user control injected for each

result

• Content ‘slide down’ via Ajax where view=content

• Mouse over ‘pop ups’ via Ajax where

view=preview

• Href=LinkManager(item.ID) + “?view=”


Content Browser Content

Ajax …?view=content


Content Browser Preview

Ajax

…?view=preview


Content Browser Facets

• 1-N ‘Facets’ (sublayouts) can be added

• Facets use FindControl(“Browser”) and

Browser.Filters.Add(self)

• Each Facet operates with an update panel

and is a ‘stand alone’ control.


Content Browser Facets


Configuration – Overview

• Add a Content Browser and point it at a

collection of similar content items

• Add ‘Facet’ filters at that map content item

fields to ‘lookup’ items

• Map ‘summary’, preview and content views


Browser Configuration

Model View Controller



Content (/templates/Publication)

•Default Redirect

•Preview Device •Content Device




Configuration (/…/PublicationBrowser)

•Browser •SummaryRow

•Facet A •Facet B •Facet C


•Default Redirect





Configuration (/…/PublicationBrowser)

Browser (/home/Publications.aspx)

•Browser

•MultiList •DropDown •SingleLine •Textbox


•Default Redirect



•Browser •SummaryRow

•Facet A •Facet B •Facet C


“Model” - Device Layouts

• Needed to keep the browser content agnostic

• Wanted content items to ‘display themselves’ upon request

• Assigned a standard markup container to a preview and content device layout – Allows mouse over to call Ajax

• Summary list uses a dynamically loaded row or custom .ascx


“Model” - Device Layouts

http://sitecorecxi/Publications.aspx?id={C4148482-...-204FE082D00F}

<a

href=http://sitecorecxi/.../Publications/CoverageListResearch/2011/Augus

t/ABH_T_08112011.aspx?view=preview

…

http://sitecorecxi/.../Publications/CoverageListResearch/2011/August/ABH_T_08112011.aspx?view=preview

http://sitecorecxi/.../Publications/CoverageListResearch/2011/August/ABH_T_08112011.aspx?view=preview


“Controller” - Browser Config


“Controller” - Facet Config


“View” – Public Page Config


Demo

Demo


Source Code

Key Classes


Solution Summary

• Started with Shared Source Module

• First iteration: added full text search on

PDFs

• Second iteration: added facets, IFilters,

searching on multiple document types

• Extended Configurability


Future Enhancements

• Reflection class loading technique


Future Enhancements


• Dynamic facet loading with sort order


Future Enhancements



• Default search using a custom Lucene

‘path’ term with sorting


Future Enhancements



• Default search using a custom Lucene

‘path’ term with sorting

• Integrate new Lucene apis and pre-filter


Future Enhancements


• Dynamic Facet loading with sort order

• Default search using a custom Lucene ‘path’ term with sorting

• Integrate new Lucene apis and pre-filter

• Configurable sortable fields


Questions


Feedback Appreciated!

Please take a moment to provide session feedback via the mobile site.

http://www.sitecore.net/SymNA

Enhancing Search Using Lucenesymposiumna.s3.amazonaws.com/2012/Developer-Enhancing Searc… ·...

Documents

Transcript of Enhancing Search Using Lucenesymposiumna.s3.amazonaws.com/2012/Developer-Enhancing Searc… ·...