WEX Overview (Part II) - IBM...WEX Overview (Part II) Eric Smith – Watson Explorer Solution...
Transcript of WEX Overview (Part II) - IBM...WEX Overview (Part II) Eric Smith – Watson Explorer Solution...
WEX Overview(Part II)Eric Smith – Watson Explorer Solution Architect
Topics
• New Insights from Unstructured Data
• Tailoring WEX to your Environment
• Connecting WEX to other Analytic Tools
2
What is Unstructured Data?
3
News Articles Email Social Media
4
WEX
5
6
Watson Knowledge Studio• Cloud based, machine learning
solution for developing new domain knowledge for Watson tools
• Information stored in knowledge
graphs
• Runtime environment for fine tuning
and refining annotations
• Leverage in Watson Explorer or
Alchemy Language applications
IBM Confidential
Watson Knowledge Studio Clip
8
NHTSA Demo with WKS Annotator
9
Ontolection Trainer
• Provides a true machine learning
capability for creating an ontology
• ML algorithms are executed against a text corpus of data
• Output is leveraged within a
search collection to enable query expansion.
• Enhances Natural Language
Querying
10
Tailoring WEX to Your Environment
11
Analytic Components
Content Miner
Content Analytics
Admin Console
Analytics Infrastructure
Control Monitor
Configuration
Security Scheduler Logging
Websphere(Embedded or Enterprise)
360
Admin
360 Info
App
Foundational Infrastructure
Control Monitor ConfigurationSecurity Scheduler Logging
Crawlers
Content
Preparation(Text Analytics)
1
Indexer
2
Advanced
Analytics
& Search(runtime)
CrawlersConversion
Pipeline
1
Indexer
2
Search(runtime)
3
360 Info App User
Business Analysts &
Data Scientists
Data Sources
Foundational Components
4
4
3
UIMA
Domain Expert
Content
Analytics
Studio
Integrations with
REST
API
Annotator Admin
Console for
Foundational
Components
IBM Master Data
Mgmt
IBM
Counter
Fraud
BigInsights
BI Reporting
IBM Product
Integrations
Websphere(Embedded or Enterprise)
WEX Advanced Edition
WEX Functional Architecture
On- Premise
Connector
Admin UI
Document Text
ExtractionIndexing
Application
Builder
On Premise
Annotation
Watson CloudWatson
Services
WEX Conversion Pipeline
Watson
Services
13
14
Development EnvironmentLink to System Requirements: http://www-
01.ibm.com/support/docview.wss?uid=swg27045727
DE
V U
sers
Number of Server(s) CPUs Cores For Memory (GB) Storage
Each Server Each Server Each server
WEX FC Development
-18 64 3 TB
• Up to 3 to 6 TB of data
• No High-availability -failover
• RHEL Linux• On-Premise
• Application Builder
• WAS Liberty Profile
• Result aggregation
• Display rendering
• *Can be a VM
WEX AC
(Content
Analytics)
Developme
nt Server
WEX FC
Developme
nt Server
• NLP Annotation• Content Mining• *Can be a VM
64-bit x86, IBM POWER7, IBM POWER8, or IBM Z System
64-bit (AMD64 or Intel 64) x86 system
Normal flow (Primary)
Data replication (DI)
Fail-over flow
Normal flow (Secondary)
Annotator Flow
Normal flow (Primary)
Data replication
Load B
ala
ncer
• 15TB of data (10% structured)
• Projected index size:• Structured (1.5TB)• Unstructured (2 TB)
• High-availability - failover
• 8 Queries/second
• Indexing• Query service
Engine
Layer
• Crawling• Connectors
• Indexing failover
Crawl/Index Layer
• Clustering• Federated Search
• Result aggregation• Display rendering• App Builder
Application
Layer
Type of Server CPUs Cores For Memory (GB) Storage
Each Server Each Server Each server
Application 8 32 200 GB
Engine 16 64 2 TB
Crawler 16 32 1TB
16
WEX EE Production Environment – 3 Tier Architecture
HW
Load B
ala
ncer
Number of Server(s) CPUs Cores For Memory (GB) Storage
Each Server Each Server Each server
Application -6
Engine- 6
16
32
128
128
500 GB
3 TB
Data – 6 32 64 3 TB
Normal flow (Primary)
Data replication (DI)
Fail-over flow
Normal flow (Secondary)
• Up to 14 TB of data
• High-availability -failover
• 7 Queries/second• RHEL Linux
• Indexing• Query
Routing• Search
Results
Engine
Layer
• Crawling and Indexing
• Data Refreshing
Data Layer
• User Interface• Integration
Layer
Application
Layer
64-bit (AMD64 or Intel 64) x86
system
64-bit (AMD64 or Intel 64) x86 system
64-bit (AMD64 or Intel 64) x86 system (Can be
VMs)
Questions to Ask
• What is the use case?
– Search, Analytics, Both?
• How much data?
• What kind of data?
• Data Growth?
• Usage?
• Interface Options?
17
Connecting WEX to Other Analytic Tools
18
Streams
Product Integrations
19
MDM* InfoSphere
BigInsights*StreamsFileNet P8
WebSphere Portal
I2 Analyst Notebook
Cognos
SPSS
© 2017 International Business Machines Corporation 20
Natural Language Understanding – Augmented Indexes
Extract metadata automatically to improve exploration without building any annotators!
© 2017 International Business Machines Corporation 21
Watson Discovery Service – Runtime Integration
Bring curated news and blog content that has been augmented by Alchemy in context into a Watson Explorer application
Questions?
22