Tracy L. ThompsonExecutive Director

NELLCOtracy.thompson@yale.edu

603-357-3385

Next Generation Discovery

An Alternative to Traditional Federated Searching

Universal Search Solution Goal• Build, in a collaborative environment, a

one-box search application that will allow legal researchers to find relevant content (as defined by collection development standards), both print and electronic, with a single search query and without reference to the OPAC or A-Z list as starting points.

Federated Search and NELLCO

• 2004 Reference Interest Group – BU• Why do our users need federated

search?– Our users go to Westlaw, Lexis and

Google (the ‘anchors’)

The Information ‘Mall’

Federated Search and NELLCO• 2004 Reference Interest Group – BU• Why do our users need federated

search?– Our users go to Westlaw, Lexis and

Google (the ‘anchors’)– Miss out on ‘boutique’ resources

• Current discovery (out)mode is through OPACs and A-Z lists

– We need to steer researchers to relevant resources

• Enhance scholarship• Justify budgets

Fed Search Paradigm

• Real-time search across multiple databases

• User enters search string• Search is broadcast to ALL associated

databases, regardless of content• Search string is translated into native

search language• Search results are delivered as they are

retrieved

Federated Search

Problems with the Paradigm

• Excessive/unnecessary vendor server traffic – scattershot approach

• Search may not translate as expected• Difficulty ranking – first-in-time delivery• Difficulty de-duping• Skewed usage statistics

An Alternative Approach

• FPLC committee began exploring enterprise search technology– Developed for fortune 500 companies– Intended for searching proprietary content– Examples

• Google’s GSA• Thunderstone’s Enterprise Search Appliance• Vivisimo • Endeca

Enterprise Search Paradigm

• Host creates local index of content• Searcher enters search string• Search is run against the index• Search results are displayed• Results link to content, not index

Enterprise Search

The Vision• Enterprise search applied across ALL

electronic content sources– OPAC– Third-party (paid) resources– Free web– Internal datasets

• Keep the librarian involved in collection development/selection

• Harness NELLCO’s leverage/vendor relationships?

Publisher Benefits?• More relevant exposure for their content • Reduced server traffic• Representative usage stats

Money• 2007 – Institute of Museum and Library

Services (IMLS) National Leadership Grant - $374k

• 1 to 1 matching required• 2 year grant period - 12/07-11/09• Project total @$850k

Technical Expertise• Open source solution• Development Partner – IndexData –

Danish-based library solution developer

Human Resources• Project Coordinator – Carl Grant• NELLCO ED and Admin. Assist.• Project Committee –

– Chaired by Roberta Woods (FPLC)– Members appointed by Directors– 10% time commitment over two years– 5 members represent library types

• Grad. intern from Syracuse• Participating member libraries• Participating vendor/publishers

The Plan• Secure vendor and member

commitments • Develop best practices – cookbooks• Develop interfaces

– one box GUI (but advanced search will be supported)

– Admin. interfaces– Design by committee

The Plan• Develop authentication scheme

– Two layers• Authenticate the user to the index (NELLCO)

– Identifies user for correct resource ‘menu’

• Authenticate the user to the resource (Vendor or library)

– Identifies user as authorized user of content

The Plan• Build the index – OAI-PMH protocol

– OPACs– Local content– Free web resources vetted by members– Subscription/fee-based resources

• 20 vendors during proof of concept phase• 5 currently committed• Commitments required by 4/1/08

The Plan• Launch beta by end of 08• Evaluate• Troubleshoot and modify• Evaluate• Grant period ends 11/09• Proof of concept = sustainability?

Thank you!

Questions?

tracy.thompson@yale.edu603-357-3385