Post on 19-Jan-2016
Visualization in Visualization in Text Information Text Information
RetrievalRetrieval
Ben HoustonExocortex Technologies
www.exocortex.org
Zack JacobsonCAC
The Starting Goal
The Original Project GoalCan we come up with a graphical way of representing search
results in a way that is superior to text only displays?
Other VITA project members:Els Goyette Olivier DagenaisSarah Rosser
A Text IR Interaction Model
InterfaceIR Search
Proxy
DocumentCollection
Query(s)
Results
Browsing
User
A Quantification of User Needs
Specific resource.– Has a particular book, web page in mind.
Specific information.– Needs a book on a particular subject matter which
contains particular information.
Specific knowledge.– Needs to know about an unfamiliar subject matter.
A Quantification of User Needs
IR is good at these tasks.IMHO Visualization would be
an unneeded hindrance.
Maybe this is an opportunity here.There is a lot of information to shift through.
Specific resource.
Specific information.
Specific knowledge.
Formalizing Knowledge Search
There is a hypothetical set of relevant documents which the user would like: Dr
The user attempts to get the set Dr through initially guess and refining a series of: q1, q2, … qn.
We can think of it as iterative evolutionary hill climber.– Serial sub goals of finding qn+1 such that P(Dr|qn+1) > P(Dr |qn)
Thus… How can we help the user maximize P(Dr|q) as quickly as possible?
Don’t forget… popular IR problems.
Difficulty in formulating effective queries.– Average number of terms per query is about 1.5.
Words do not have a 1:1 mapping to semantic concepts.Determining the relevance ranking of an individual document.– Going past just words.
How do you deal with 1 billion documents?– Did you know its more than doubling every year?– Databases/indices of + 500 GB each.
Our efforts
1st Try: Bar charts. (Even 3D bar charts!)– Naïve first attempts – we won’t mention those.
2nd Try: Concept-document clustering in a information space.– Two prototypes: NetViz & AutoViz, more should be
developed.
The Major “Neat” Features
Focus on concrete representation of the query.Use data-mining techniques before visualization.Visual summaries.An active model for interaction.Bridging the gaps between “serial” queries.Widening / narrowing to get context.
Location, Color, Size, Shape
Show each concept ina meaningful spatial relationships.
Show the specific results positioned in relation to the concepts.
Display Intra-result Structure
Clustering on implicit/latent trends
Visual Document Summaries
Instead of
Lets show intra-documentconcept co-occurrence
Exploring within a result set
Highlighting and extracting subsets.
Each document has a probability distribution amount the different clusters.
1)|( iC
diCP
Exploring outside a result set
(Slightly Hypothetical)
Present three things to the user1. Where the user is. (The City)2. What is at the location the user is at. (The Sights)3. What are related/nearby places. (The Highways)
There is a mockup of this available on my website:www.exocortex.org/~ben/trendanalysis2.html
Bridging Serial Queries
Instead of requiring a user to judge each query as a separate entity why not let a user see what changes in the results as they refine their query?– Currently we do serial searching with backtracking.– A potentiator for for non-serial methods of
exploration in a (Bayesian) “concept space” network.
P(Dr|qn+1) > P(Dr |qn) P(Dr|f(qn+1,qn)) > P(Dr |f(qn))
Widening / Narrowing Scope
Allowing for interactive narrowing or widening of the display by filtering on document relevance.
Browser Integration… of course
Spawning of browsers. Seamless browserintegration. (hypothetical)
Results and Predictions
Extracting / presenting intra-result set structure is extremely effective. There is value breaking free from serial queries.Provide landmarks and easy exploratory interaction models. More worked Needed. ??? The current browser interface is really limiting.The underlying engine is more critical than visualizations.Visual document summaries need more work.Overactive (hyperactive) interfaces are hard to learn.
“Ad hoc” Results Ben’s Future of Text IR
• Visualization is usually a fix for insufficient data-mining / algorithm techniques (in text IR).
• Intra-result set clustering works in text only displays too. It will be integrated into existing text search engines.
• The metaphor of exploring information space it become more popular.
Hmm… Sturgeon tastes good.
Want to try it? Download the prototypes!NetViz http://www.exocortex.org/netviz
AutoViz http://www.exocortex.org/autoviz
Comments? Email me!ben@exocortex.org