How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query...
Transcript of How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query...
![Page 1: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/1.jpg)
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
HOW TO BUILD AN AWESOME SEARCH APP Stu McLean, Ph.D., Principal Consultant, Public Sector - Civilian, MarkLogic Ganesh Vaideeswaran, Senior Director, Development, MarkLogic
![Page 2: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/2.jpg)
SLIDE: 2
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Agenda Components of a search application
Deep dive
![Page 3: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/3.jpg)
SLIDE: 3
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
![Page 4: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/4.jpg)
SLIDE: 5
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
![Page 5: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/5.jpg)
SLIDE: 6
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
![Page 6: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/6.jpg)
SLIDE: 7
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
![Page 7: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/7.jpg)
SLIDE: 8
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
![Page 8: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/8.jpg)
SLIDE: 9
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
![Page 9: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/9.jpg)
SLIDE: 10
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
![Page 10: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/10.jpg)
SLIDE: 11
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
![Page 11: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/11.jpg)
SLIDE: 12
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
![Page 12: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/12.jpg)
SLIDE: 13
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
![Page 13: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/13.jpg)
SLIDE: 14
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
![Page 14: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/14.jpg)
SLIDE: 15
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
![Page 15: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/15.jpg)
SLIDE: 16
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
![Page 16: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/16.jpg)
SLIDE: 17
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Awesome Search for the Enterprise
WHAT MARKLOGIC PROVIDES
360° VIEW
![Page 17: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/17.jpg)
SLIDE: 19
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
NBCUniversal – SNL40 App
Smart Content
Enriched and targeted content
Constantly tuned recommendations
> 5,550 sketches
![Page 18: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/18.jpg)
SLIDE: 20
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Talent Kristen Wiig
Acted in
Episode 4 Anne Hathaway and
Killers
Part of Played
Character Maharelle Sister
Season 34
Segment The Lawrence Welk Show
Aired on
Date 10/4/08
Era
Acted in
Includes
Part of Has
Characteristic Tiny hands
NBCUniversal – SNL40 App
Smart Content: Enriched and targeted content Constantly tuned recommendations
![Page 19: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/19.jpg)
SLIDE: 21
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Fairfax County – Mapping Crime Data
Data Variety
500,000 GIS data points
Service Calls
Source Variety
800 GIS layers
Structured & Unstructured
![Page 20: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/20.jpg)
SLIDE: 23
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Mitchell1 – Streamlined Auto Repair Information
MITCHELL1 PRODEMAND
REPAIR INFO
EXPERT ADVICE
REPAIR ORDERS
Structured & Unstructured Data
Sophisticated Search
Multi-device Delivery
![Page 21: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/21.jpg)
SLIDE: 25
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Needs for an Awesome Search App
Relevance
Snippets
Highlight
Auto Complete
Spell Suggest
Synonyms
QUERY
Structured & Unstructured
Multi-Model approach
DATA
Alerts
Transactions
Security
Real-time
Languages
Configurability
Development Platform
PLATFORM
Context
Presentation
![Page 22: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/22.jpg)
SLIDE: 26
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Aboutness – The Key to Successful Search What is this document about?
What is this submitted query about?
What is the user’s activity about?
What are the responses about?
![Page 23: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/23.jpg)
SLIDE: 27
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Implementing Aboutness in MarkLogic
Data What is this document about?
![Page 24: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/24.jpg)
SLIDE: 28
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Implementing Aboutness in MarkLogic
Data
![Page 25: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/25.jpg)
SLIDE: 29
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Implementing Aboutness in MarkLogic
Data
Query What is this query about?
![Page 26: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/26.jpg)
SLIDE: 30
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Implementing Aboutness in MarkLogic
Data
Query
![Page 27: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/27.jpg)
SLIDE: 31
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Implementing Aboutness in MarkLogic
Data
Query
Context What is the user’s activity about?
![Page 28: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/28.jpg)
SLIDE: 32
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Implementing Aboutness in MarkLogic
Data
Query
Context
![Page 29: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/29.jpg)
SLIDE: 33
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Implementing Aboutness in MarkLogic
Data
Query
Context
Results What are the responses about?
![Page 30: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/30.jpg)
SLIDE: 34
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Implementing Aboutness in MarkLogic
Data
Query
Context
Results
![Page 31: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/31.jpg)
SLIDE: 35
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Implementing Aboutness in an Awesome Search App Data
Categorization, Envelope Structure
Query
Query as a Dialogue
Context
Building an Information Resource
Results
Relevant Response
![Page 32: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/32.jpg)
SLIDE: 36
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Implementing Aboutness in an Awesome Search App Data Categorization, Envelope Structure
Query
Query as a Dialogue
Context
Building an Information Resource
Results
Relevant Response
![Page 33: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/33.jpg)
SLIDE: 37
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Domain Knowledge ‒ Tools for Establishing Aboutness Taxonomies – vocabulary of the organization
Ontologies – Tie your documents to relevant data and documents
Thesaurus
Organizational units, people, products
Public vocabularies – Geographic, DBPedia
![Page 34: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/34.jpg)
SLIDE: 38
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Terms to look for in your documents Taxonomies
![Page 35: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/35.jpg)
SLIDE: 39
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Taxonomies and Reverse Queries Categorization ‒ Finding Aboutness
Public taxonomies
Getty – art and architecture
World Bank – humanitarian projects
Library of Congress – subject headings
National Library of Medicine – medical headings
Domain taxonomies
Best Buy
![Page 36: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/36.jpg)
SLIDE: 40
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Where does Aboutness reside?
title:
abstract:
glossary:
Body
Expert searchers compose carefully crafted searches with domain knowledge
The rest of us hope for the best
Categorizing Your Documents
![Page 37: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/37.jpg)
SLIDE: 41
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Turn a Taxonomy Entry into a Query <cts:or-query xmlns:cts="http://marklogic.com/cts"> <cts:near-query distance="5"> <cts:near-query distance="5"> <cts:element-word-query weight="10"> <cts:element xmlns:_1="http://marklogic.com/solutions/obi/source">_1:title</cts:element> <cts:text xml:lang="en">Non-Government</cts:text> <cts:option>case-insensitive</cts:option> <cts:option>punctuation-insensitive</cts:option> <cts:option>stemmed</cts:option> </cts:element-word-query> <cts:element-word-query weight="10"> <cts:element xmlns:_1="http://marklogic.com/solutions/obi/source">_1:title</cts:element> <cts:text xml:lang="en">Bond</cts:text> <cts:option>case-insensitive</cts:option> <cts:option>punctuation-insensitive</cts:option> <cts:option>stemmed</cts:option> </cts:element-word-query> </cts:near-query>
Implementing Categorization
![Page 38: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/38.jpg)
SLIDE: 42 © COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Reverse Query Categorization
Reverse Queries
![Page 39: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/39.jpg)
SLIDE: 43 © COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Reverse Query Categorization
Reverse Queries
Ingest mydoc.xml 1 MB document Save as
/tmp/mydoc.xml in collection tmp
![Page 40: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/40.jpg)
SLIDE: 44 © COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Reverse Query Categorization
Reverse Queries
Ingest mydoc.xml 1 MB document Save as
/tmp/mydoc.xml in collection tmp
In-memory copy of abridged document (10% = 100K)
![Page 41: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/41.jpg)
SLIDE: 45 © COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Reverse Query Categorization
Reverse Queries
Ingest mydoc.xml 1 MB document Save as
/tmp/mydoc.xml in collection tmp
In-memory copy of abridged document (10% = 100K)
![Page 42: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/42.jpg)
SLIDE: 46 © COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Reverse Query Categorization
Reverse Queries
Ingest mydoc.xml 1 MB document Save as
/tmp/mydoc.xml in collection tmp
In-memory copy of abridged document (10% = 100K)
cts:search(fn:doc(), cts:and-query(( cts:collection-query("topicQueries"), cts:reverse-query($reducedDoc) )), "unfiltered" )
![Page 43: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/43.jpg)
SLIDE: 47 © COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Reverse Query Categorization ‒ Scoring
Reverse Queries
/tmp/mydoc.xml
![Page 44: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/44.jpg)
SLIDE: 48 © COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Reverse Query Categorization ‒ Scoring
Reverse Queries
/tmp/mydoc.xml
for $q in $reverseSearchResults let $score := cts:fitness( cts:search( fn:doc(), cts:and-query(( cts:document-query($myURI), cts:query($q/query) )), ("score-logtf","unfiltered") )
![Page 45: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/45.jpg)
SLIDE: 49
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Envelope Pattern ‒ Know Your Data Homogenized Variant Data Types
Remain Schema-Agnostic, Load As-Is
Annotate Your Data
![Page 46: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/46.jpg)
SLIDE: 50 © COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Building the Envelope
Ingested Content
Metadata
Category Annotation
![Page 47: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/47.jpg)
SLIDE: 51
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Envelope ‒ Know Your Data Standardize ingested documents of multiple types Common names may be different (e.g. Title, Heading, Subject)
Normalize your searchable elements into a common format
Multiple versions of the same document (e.g. Translations)
Preserve artifacts
Indexed search on metadata (facets)
Full-text search on content
![Page 48: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/48.jpg)
SLIDE: 52
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Implementing Aboutness in an Awesome Search App Data
Categorization, Envelope Structure
Query Query as a Dialogue
Context
Building an Information Resource
Results
Relevant Response
![Page 49: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/49.jpg)
SLIDE: 53
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
What is the Query About? Establish Aboutness Through Dialogue
Type-ahead
Match Lexicons
Match Search History (user, global)
Background Search
Phrase Identification
Object Matching
Boost Queries
![Page 50: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/50.jpg)
SLIDE: 54
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
What is the Query About? Establishing Aboutness Through Dialogue (Cont’d)
Correction and Confirmation
Spell Check
Thesaurus Expansion
Semantic Expansion
Drill Down
Follow Up Search Box
Facets
Constraints on the Fly
![Page 51: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/51.jpg)
declare function get( $context as map:map, $params as map:map ) as document-node()? { let $suggestList := ("person", "country", "organization", "project", "region", "topic") let $partial := xdmp:url-decode(map:get($params, "partial-q")) let $suggestions := map:new(( for $s in $suggestList let $term := fn:concat($partial,"*") return map:entry($s,search:suggest( $term, $options, if ($s eq "person") then 20 else 3 ))) return document {xdmp:to-json($suggestions)} };
Extending Suggest REST API extension Search multiple lexicons for
each request Type-ahead must be fast Resist temptation to put
multiple suggest calls in the app
![Page 52: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/52.jpg)
SLIDE: 56
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Query Expansion ‒ Background Search Finding phrases when there are no quotes
Approach 1 SPARQL - Match phrases in metadata, taxonomies
Put the matched text into a weighted custom constraint (e.g. phrase: )
OR the phrase: constraint with the full query
Approach 2 Build NEAR queries using pairs of nearby terms
OR the phrase and boost the weight on the NEAR query if matched
![Page 53: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/53.jpg)
SLIDE: 57
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Example – Constraints on the Fly Selected documents share common elements
Elements not known in advance but can be used as in-app facets
Set as constraints in the submitted options
![Page 54: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/54.jpg)
SLIDE: 58
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Example – Selecting from Feature Fields
![Page 55: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/55.jpg)
SLIDE: 59
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Example – Selecting from Feature Fields
![Page 56: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/56.jpg)
SLIDE: 60
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Implementing Aboutness in an Awesome Search App Data
Categorization, Envelope Structure
Query
Query as a Dialogue
Context Building an Information Resource
Results
Relevant Response
![Page 57: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/57.jpg)
SLIDE: 61
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Aboutness – The Context Behind The Query Document
LexisNexis WestLaw Medline USPTO
![Page 58: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/58.jpg)
SLIDE: 62
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Aboutness – The Context Behind The Query Document Facts
LexisNexis WestLaw Medline USPTO
WikiPedia CIA World Factbook
![Page 59: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/59.jpg)
SLIDE: 63
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Aboutness – The Context Behind The Query Document Facts Information
LexisNexis WestLaw Medline USPTO
WikiPedia CIA World Factbook
Google Bing
![Page 60: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/60.jpg)
SLIDE: 64 © COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Reference Data – Link or Load External Data Real Time Read
![Page 61: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/61.jpg)
SLIDE: 65 © COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Reference Data – Link or Load External Data Ingest Retrieval Embedded Data
![Page 62: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/62.jpg)
SLIDE: 66 © COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Reference Data – Link or Load External Data Copied Reference Data Embedded Links
additional-query SPARQL
![Page 63: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/63.jpg)
SLIDE: 67
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Implementing Aboutness in an Awesome Search App Data
Categorization, Envelope Structure
Query
Query as a Dialogue
Context
Building an Information Resource
Results Relevant Response
![Page 64: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/64.jpg)
SLIDE: 68
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Planning Your Data For Results When to index metadata
Facets
Sortable
Lexicon dropdowns
Document joins
When not to index
Wildcarding
Search:search constraints
![Page 65: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/65.jpg)
SLIDE: 69
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Results ‒ Focus On Relevance Boost the Document: Quality
Boost the Element: Term Weighting
Boost the Term: Query Weighting
![Page 66: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/66.jpg)
xdmp:document-insert( "/example.xml", <a>aaa</a>, xdmp:default-permissions(), xdmp:default-collections(), 10 ) xdmp:document-load( "http://myCompany.com/file.xml", <options xmlns="xdmp:document-load"> <uri>/documents/myFile.xml</uri> <repair>none</repair> <permissions>{xdmp:default-permissions()}</permissions> < <format>xml</format> <quality>10</quality> </options>)
Boost the Document Set quality xdmp:document-insert xdmp:document-load
![Page 67: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/67.jpg)
SLIDE: 71
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
In the database configuration
Select Word Query from the menu
Under the includes tab add your ranked element with its weight
Boost the Element
![Page 68: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/68.jpg)
SLIDE: 72
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Boost the Term With no Query Weighting the MarkLogic Companies document appears first
![Page 69: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/69.jpg)
SLIDE: 73
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
The Boost-Query A conditional secondary query that increases the weight of results matching both
criteria
The boost-query pushes the MarkLogic World document to the top
![Page 70: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/70.jpg)
SLIDE: 74
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Aboutness – The Key to Successful Search What is this document about?
What is this query about?
What is the user’s activity about?
What are the responses about?
![Page 71: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/71.jpg)
SLIDE: 75
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Rendering Awesome Apps
![Page 72: How to Build an Awesome Search App - MarkLogic · Data Categorization, Envelope Structure Query Query as a Dialogue Context Building an Information Resource Results Relevant Response](https://reader034.fdocuments.net/reader034/viewer/2022051800/5ad1af677f8b9a05208be7a1/html5/thumbnails/72.jpg)
SLIDE: 76
© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Q&A