Federated Search: Context, Capability, & Consequence Brenda Reeb, University of Rochester John...
-
Upload
merilyn-taylor -
Category
Documents
-
view
225 -
download
1
Transcript of Federated Search: Context, Capability, & Consequence Brenda Reeb, University of Rochester John...
Federated Search: Context, Capability, & Consequence
• Brenda Reeb, University of Rochester
• John D’Ignazio, Syracuse University School of Information Studies
• John Law, ProQuest
LITA National Forum 2005, San Jose, California
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
2
Today
• What we did and why we did it – Brenda• How we did it – John D’Ignazio• What we found – John Law• Capability and Consequences – John
Law• Questions – Audience!
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
3
Reasons why• Animal, mineral, or
vegetable?• Bridge from
federated search results to native resource – or not?
• Find information vs. analyze information – transparent technology and resources Context
How we did it
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
5
Inserting a new technology into a complex system
Questions multiply!• How does it work with mass of database offerings?• How does it relate to the existing web implementation?• How do you value the additional system responses? • How does it relate to Google?
How to answer?• Keep building…• Or,
Take it to the user?
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
6
Inspiration: A Formative User Study
• comfort with usability testing to evaluate library web design
• contextual inquiry: put tech development in context of user’s information tasks
• compare user work practice in library’s web environment1. w/ federated search2. without
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
7
Elements of Contextual Inquiry
An interview technique:– Master-apprentice model BUT
user is master!– Think aloud protocol…or in our case Think
After“Tell me about the paper you last wrote.” User then dives away from description toward concrete actions taken to reach goal
– Establish easy manner to elicit practices and user relationships with technology
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
8
Participants• University of Rochester
Brenda Reeb -- usability program coordinator
Nancy Foster -- anthropologist/researcher
Nora Dimmock -- multimedia center
• ProQuestJohn Law -- director, product
management
• EndeavorMike Visser -- product manager
• Association for Research Libraries - ARL
John D’Ignazio -- usability consultant
• Subjects - 9– 3 first year
– 6 upper years
– 3 social sciences
– 3 humanities
– 2 sciences
– 1 undeclared
• Solicited from library usability email roster – Did you write a paper last
semester?
• Received $25 gift certificate
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
9
Test SetupModerator/Apprentice- sits next to subject- guides, prompts subject through
task according to script What are you doing now?,Why
did you do X instead of Y?, What would you like to do right now? Show me.
Have you ever used federated search…let’s try.
Note-taker - records interaction,- concentrates on user
attitudes/approach
Subject and subject’s actions simultaneously recorded
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
10
Morae Software Suite• Event recording software, made
by TechSmith
• Flexible, easy, low cost
• 3 modules:– Morae recorder: capture audio,
video and user actions– Morae remote viewer: observe
a test from a remote location– Morae manager: analyze the
results and create visual presentations
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
11
Room for Improvement• Test Subjects:
– not chosen randomly; one third were library employees
• Moderators: – hard to play apprentice while guiding progress
• Test Setup: – not native work environment– recreating past actions may lead to idealization or omissions– forced switchover to federated search
• Fitness of Method– adopted from HW/SW development community; libraries may need
different formative/evaluative methods
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
12
What we found
• Observations• New capabilities for federated search • Consequences• Conclusions
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
13
Observations
• Students’ search strategies are well considered (albeit more literal than informed)
• Users value their time highly and are unwilling to “fish around” long for appropriate resources
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
14
Video snippet #1
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
15
Observations (con’t)
• Everybody wants a summary/abstract– Give me enough information to decide
which items are worth pursuing– Where are my search terms in my results?
• Nobody likes complex paths to full text
• Users expect that search results are presented in order of most relevant
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
16
Video snippet #2
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
17
Observations (con’t)
• “Search all” usually not appealing in searches of library resources
• Little known distinction between different databases
• Frequently accessed target e-resources via course pages
• Context adds to value perception: “Faculty-recommended resources”
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
18
Video snippet #3
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
19
Even more observations
• Trusted resources, whether appropriate for the research project or not, win over a “search all” federated view of library resources
• Resources discovered “in context” are considered as trusted and credible (otherwise, little perceived distinction between resources)
• Fluid interaction among tools and resources (e.g. Google, Amazon, NoodleBib, etc.)
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
20
New capabilities for Federated Search
• Provide evaluative information in search results. Why pick this one?– abstract/summary, subject term highlight, etc.
• Provide direct access to full text – no confusing intermediate pages (e.g. link resolver
page, citation page, etc.– predictors of full text availability (i.e. don’t show a
full text link unless full text is known to be available
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
21
New capabilities (con’t)
• Conundrum: – Based on current technology, “Search all”
federated searching of premium scholarly resources falls short of satisfying user needs
– Web search engines’ superiority in relevancy sorting of search results is found more satisfactory although lacking in scholarly integrity
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
22
Consequences
• Students return to past trusted resources. Is this wrong?– This is compensatory behavior:
• When resources are not available in the context of their work
• When it’s unclear where to go for quick, reliable answers to research questions
• Student focus is course-centric– The “home page” is the course page
– Learning management systems and student personalized portals will become “information central” as these solutions evolve
– Library resources need to be available in these environments• Selective content presented in context
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
23
Conclusions• Context adds value and trust
– Resources associated with a course were perceived as superior
• Clearly, students are engaged in their research– an appropriate combination of resources (and tools) need to
be available in-context
• Users differentiate between authoritative scholarly resources and general web search results, however: – the latter is easier to deal with since it provides a
predictable quality of search results, while – the former is inconsistently presented and difficult to
discover in a manner consistent with user needs, expectations and current practices
Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005
24
From the user’s perspective:
How do finch hormones impact birdsong?
Finch HormonesLab ReportDecember 2004