Unity: Because the Sum is Greater than the Parts
-
Upload
inside-analysis -
Category
Technology
-
view
421 -
download
1
Transcript of Unity: Because the Sum is Greater than the Parts
Twitter Tag: #briefr
The Briefing Room
! Reveal the essential characteristics of enterprise software, good and bad
! Provide a forum for detailed analysis of today’s innovative technologies
! Give vendors a chance to explain their product to savvy analysts
! Allow audience members to pose serious questions... and get answers!
Mission
Twitter Tag: #briefr
The Briefing Room
December: Innovators
January: Big Data
February: Analytics
March: Data in Motion
Twitter Tag: #briefr
The Briefing Room
Innovators
! Necessity is the mother of invention, and the need for unified access is driving innovation
! Big Data, Cloud and Social Media are the horses in front of the carriage
! Users want easier, faster and better system management Ph
oto
by G
eorg
e P.
Lan
dow
(w
ww
.vic
tori
anw
eb.o
rg)
This image may be used without prior permission for any scholarly or educational purpose
Twitter Tag: #briefr
The Briefing Room
Analyst: Krish Krishnan
Mr. Krishnan is a recognized expert in the strategy, architecture and implementation of high performance data warehousing solutions and unstructured data. He co-authored “Building the Unstructured Data Warehouse” along with Bill Inmon. He also has written eBooks, over 150 articles, viewpoints and case studies in Big Data, Business Intelligence, Data Warehousing, Data Warehouse Appliances and High Performance Architectures. Mr. Krishnan currently is promoting the next generation of data warehousing, focusing on Big Data, Semantic Technologies, Crowdsourcing, Analytics, and Platform Engineering. He leads the Data Warehouse Appliance and Architectures Expert Channel at BEyeNetwork.com and publishes with www.beyenetwork.com\kkrishnan.
Twitter Tag: #briefr
The Briefing Room
! Teradata is known for its analytic data solutions with a focus on integrated data warehousing, big data analytics and business applications.
! It offers a broad suite of technology platforms and solutions, and a wide range of data management applications and data mining capabilities.
! Teradata is now offering an integrated portfolio that turns a multi-system environment into an Analytical Ecosystem.
Teradata
Twitter Tag: #briefr
The Briefing Room
Imad Birouty is the Program Marketing Manager for the Teradata Analytical Ecosystem, which includes features such as Teradata Unity and Teradata’s Dual Active Solution. Imad is also responsible for Teradata’s Data Mart Consolidation program, the Value of Integrated Data, and Total Cost of Ownership. Prior to this, Imad led the Product Management team responsible for the NCR Platform including the processing nodes, BYNET Interconnect, Server Management, and Operating Systems. He set product strategy and direction and was responsible for seven major platform releases spanning 8 years.
Imad Birouty
11 12/4/12
Teradata Unified Data Architecture
AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
INTEGRATED DATA WAREHOUSE
Teradata Analytical Ecosystem
12 12/4/12
Teradata Unified Data Architecture
AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
INTEGRATED DATA WAREHOUSE
DATA LAB
Teradata Analytical Ecosystem
13 12/4/12
Teradata Unified Data Architecture
AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
DATA MARTS
TEST/ DEV
ANALYTICAL ARCHIVE
INTEGRATED DATA WAREHOUSE
DATA LAB
Teradata Analytical Ecosystem
14 12/4/12
Teradata Unified Data Architecture
AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
DATA MARTS
TEST/ DEV
ANALYTICAL ARCHIVE
INTEGRATED DATA WAREHOUSE
DATA LAB
INDEPENDENT DATA MART
Teradata Analytical Ecosystem
15 12/4/12
Teradata Unified Data Architecture
AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
DUAL SYSTEMS
DATA MARTS
TEST/ DEV
ANALYTICAL ARCHIVE
INTEGRATED DATA WAREHOUSE
DATA LAB
INDEPENDENT DATA MART
Teradata Analytical Ecosystem
16 12/4/12
Teradata Unified Data Architecture
AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
DUAL SYSTEMS
DATA MARTS
TEST/ DEV
ANALYTICAL ARCHIVE
INTEGRATED DATA WAREHOUSE
VIEWPOINT SUPPORT
INDEPENDENT DATA MART
UNITY
Teradata Analytical Ecosystem
DATA LAB
17 12/4/12
Teradata Unified Data Architecture
AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
DUAL SYSTEMS
DATA MARTS
TEST/ DEV
ANALYTICAL ARCHIVE
INTEGRATED DATA WAREHOUSE
DATA MINING BUSINESS INTELLIGENCE APPLICATIONS VIEWPOINT SUPPORT
INDEPENDENT DATA MART
Teradata Analytical Ecosystem
DATA LAB
Business Analysts Marketing Front-Line Workers
Operational Systems Customers / Partners Executives
18 12/4/12
• Meet service level goals > The right data, in the right place, at the right time, for the
right users > Track application readiness
• Resource optimization > Match application profiles to the right Teradata platform > Availability, performance, cost
Teradata’s Analytical Ecosystem Benefits
• Investment protection > Seamless application portability between
Teradata platforms > Platform repurposing
• One source support for Teradata products and services
19 12/4/12
Teradata Unity
Teradata Unity
Unity Director
Query routing and database synchronization between Teradata Systems
Unity Loader
High volume, intelligent data loading to multiple Teradata systems
Unity Ecosystem Manager End-to-end monitoring across data warehouse components
Unity Data Mover Data movement between Teradata systems
Integrated portfolio that turns a multi-system environment into an orchestrated Analytical Ecosystem
20 12/4/12
Unity Director Provides Query Management Reads
• Two methods for Read queries > User session routing > Individual query routing
21 12/4/12
Unity Director Provides Query Management Reads
• Two methods for Read queries > User session routing > Individual query routing
Controlled Routing (Named, Preferred)
Great for session and workload control across systems, especially when high availability failover is required
22 12/4/12
Unity Director Provides Query Management Reads
• Two methods for Read queries > User session routing > Individual query routing
Controlled Routing (Named, Preferred)
Great for session and workload control across systems, especially when high availability failover is required
Automatic Routing (Automatic)
Great when systems contain different data and/or for fast setup where session control isn’t required
23 12/4/12
Unity Director Provides Query Management Writes
• Three methods for Write operations > Default, Create Preferred, Create Balanced > Can be used in combination
• Writes to all systems where objects reside
• Great for keeping systems in sync
Default (writes)
24 12/4/12
Unity Director Provides Query Management Writes
• Three methods for Write operations > Default, Create Preferred, Create Balanced > Can be used in combination
• Writes to all systems where objects reside
• Great for keeping systems in sync
Default (writes)
Create Preferred (writes)
• Creates to only one system • Subsequent writes to created objects on just one system, such as temporary tables
• Prevents wasted resources on other systems
25 12/4/12
Unity Director Provides Query Management Writes
• Three methods for Write operations > Default, Create Preferred, Create Balanced > Can be used in combination
• Writes to all systems where objects reside
• Great for keeping systems in sync
Default (writes)
Create Preferred (writes)
Create Balanced (writes)
• Creates to only one system • Subsequent writes to created objects on just one system, such as temporary tables
• Prevents wasted resources on other systems
• Creates to only one system • Designated system for creates alternates among systems for different sessions based on a round-robin algorithm
26 12/4/12
Unity Loader
• Bulk loads sent to all participating systems; updates are asynchronous • More than two sites are supported with additional Unity Loader and
Director servers
Unity Loader
Site 1
27 12/4/12
Unity Loader
Unity Loader
Unity Loader
• Bulk loads sent to all participating systems; updates are asynchronous • More than two sites are supported with additional Unity Loader and
Director servers
Unity Loader
Site 1
28 12/4/12
Unity Loader
Unity Loader
Unity Loader
• Bulk loads sent to all participating systems; updates are asynchronous • More than two sites are supported with additional Unity Loader and
Director servers
Unity Loader
Site 1 ü
ü
ü
29 12/4/12
Unity Loader
Unity Loader
Unity Loader
• Bulk loads sent to all participating systems; updates are asynchronous • More than two sites are supported with additional Unity Loader and
Director servers
Unity Loader
Unity Loader
Site 1
Site 2
ü
ü
ü
30 12/4/12
Unity Loader
Unity Loader
Unity Loader
Unity Loader
Unity Loader
• Bulk loads sent to all participating systems; updates are asynchronous • More than two sites are supported with additional Unity Loader and
Director servers
Unity Loader
Unity Loader
Site 1
Site 2
ü
ü
ü
31 12/4/12
Unity Loader
Unity Loader
Unity Loader
Unity Loader
Unity Loader
• Bulk loads sent to all participating systems; updates are asynchronous • More than two sites are supported with additional Unity Loader and
Director servers
Unity Loader
Unity Loader
Site 1
Site 2
ü
ü
ü
Copyright © 2011 Teradata Corporation. 33
AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
DISCOVERY PLATFORM
CAPTURE | STORE | REFINE
INTEGRATED DATA WAREHOUSE
UNIFIED DATA ARCHITECTURE
Big Data Analytics
Big Data Management
Copyright © 2011 Teradata Corporation. 34
Business Analyst
Discovery Platform
Discovery
Discovery Platform Data Sources
Structured Data
Multi-Structured Data
Non relational Data
OLTP DBMS’s
Users
Data Scientist
SQL
MapReduce
Statistical Functions
Discovery Tools
• Structured and multi-structured data
• Doesn’t require extensive data modeling
• Doesn’t balance the books
• Data completeness can be good enough
• No stringent SLA’s
• Fraud patterns • Customer behavior • Digital marketing
optimization • Machine data/process
optimization
Copyright © 2011 Teradata Corporation. 35
Hadoop Use Cases Validated with Customers
• Land/source operational data - Only one extract from source system
• History or long term storage - Low cost storage
• Preprocess data - Sessionize data, remove XML tags
• Transformations - Structured and semi-structured
• Exploration - Investigate value of new data sources
• Batch scoring • Single subject reporting
Many define as Analytics
36 12/4/12
Teradata Unified Data Architecture
AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
DISCOVERY PLATFORM
CAPTURE | STORE | REFINE
INTEGRATED DATA WAREHOUSE
LANGUAGES MATH & STATS DATA MINING BUSINESS INTELLIGENCE APPLICATIONS
Engineers
Data Scientists Business Analysts Marketing Front-Line Workers
Operational Systems Customers / Partners Executives
37 12/4/12
Teradata Unified Data Architecture
AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
LANGUAGES MATH & STATS DATA MINING BUSINESS INTELLIGENCE APPLICATIONS
INTEGRATED DATA WAREHOUSE
DISCOVERY PLATFORM
Engineers
Data Scientists Business Analysts Marketing Front-Line Workers
Operational Systems Customers / Partners Executives
38 12/4/12
Teradata Unified Data Architecture
AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
LANGUAGES MATH & STATS DATA MINING BUSINESS INTELLIGENCE APPLICATIONS
INTEGRATED DATA WAREHOUSE
DISCOVERY PLATFORM
Engineers
Data Scientists Business Analysts Marketing Front-Line Workers
Operational Systems Customers / Partners Executives
39 12/4/12
INTRODUCING TERADATA ASTER BIG ANALYTICS APPLIANCE 3H Industry’s First Big Analytics Appliance with Aster and Hadoop in the same cabinet
Unified System Management and Analytics
Unified Solution for Big Data Storage, Management, and Analytics
Available NOW
Copyright © 2011 Teradata Corporation. 40
Aster SQL-H™ A Business User’s Bridge to Analyze Hadoop Data
Aster SQL-H Gives Analysts and Data Scientists a Better Way to Analyze Data Stored in Hadoop • Allow standard ANSI SQL access to
Hadoop data
• Leverage existing BI tool and enable self service
• Enable 50+ prebuilt SQL-MapReduce Apps and IDE
Hadoop Layer: HDFS
Pig
Hive
Hadoop MR
Aster: SQL-H
HCatalog
Dat
a
Dat
a Fi
ltering
NEW
41 12/4/12
Teradata Unified Data Architecture
AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
DISCOVERY PLATFORM
CAPTURE | STORE | REFINE
INTEGRATED DATA WAREHOUSE
LANGUAGES MATH & STATS DATA MINING BUSINESS INTELLIGENCE APPLICATIONS VIEWPOINT SUPPORT
Engineers
Data Scientists Business Analysts Marketing Front-Line Workers
Operational Systems Customers / Partners Executives
42 12/4/12
Common Management Console for Aster, Teradata and Apache
Hadoop Aster Specific Portlets • Aster Node
Monitoring • Aster Completed
Processes
Trend/ Visualization Portlets • Capacity
Heat Map • Metrics Graph • Metrics Analysis
Query Portlets • Query Monitor
Admin Portlets • Teradata System • Roles Manager
Other Portlets • System Health • Canary queries • Aster Data
Alerting
Teradata Viewpoint Integration Easier, Faster, and Better System Management
Available now with Teradata and Aster. Hadoop availability early 2013
43 12/4/12
• Customers that want to improve supportability ensure that Teradata Vital Infrastructure is activated in their production environment
• Teradata Vital Infrastructure is embedded proactive support software available on each Teradata platform
Improve Supportability with Teradata Vital Infrastructure
Regularly Collects Asset Data
Constantly Monitors Events
Identifies Risk Data Patterns/Trends
Detects Fault Alerts
Records Fault Events
Automatic Incident Reports Are Created
Alert Notifications Are Sent and Tracked
62–70% of Incidents Discovered Through TVI
44 12/4/12
Teradata Unified Data Architecture
AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
DUAL SYSTEMS
DATA MARTS
TEST/ DEV
ANALYTICAL ARCHIVE
INTEGRATED DATA WAREHOUSE
DATA MINING BUSINESS INTELLIGENCE APPLICATIONS VIEWPOINT SUPPORT
INDEPENDENT DATA MART
Teradata Analytical Ecosystem
DATA LAB
Business Analysts Marketing Front-Line Workers
Operational Systems Customers / Partners Executives
45 12/4/12
Teradata Unified Data Architecture
AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
DUAL SYSTEMS
DATA MARTS
TEST/ DEV
ANALYTICAL ARCHIVE
LANGUAGES MATH & STATS DATA MINING BUSINESS INTELLIGENCE APPLICATIONS VIEWPOINT SUPPORT
INDEPENDENT DATA MART
DISCOVERY PLATFORM
INTEGRATED DATA WAREHOUSE
DATA LAB
CAPTURE | STORE | REFINE
Engineers
Data Scientists Business Analysts Marketing Front-Line Workers
Operational Systems Customers / Partners Executives
S
Next Generation Data Warehouse
Designing & Building Scalable Data Warehouse Infrastructures BriefR
December 4th 2012
Data Workloads & Processing
@2012 Copyright Sixth Sense Advisors
Transac1onal Systems ODS
Enterprise Datawarehouse
Datamarts & Analy1cal Databases
Datamarts & Analy1cal Databases
Datamarts & Analy1cal Databases
Transac1onal Systems ODS
Transac1onal Systems ODS
Reports
Dashboards
Analy1c Models
Other Applica1ons
Data Transforma1on
Workload Challenge
S Unknowns S Do not know what data we are
geCng S Do not know formats or
volumes of data
@2012 Copyright Sixth Sense Advisors
S Requirements S Assigning the appropriate systems
and processes to manage workloads S Creates an interchangeable
infrastructure S Support Homogenous /
Heterogeneous plaIorms S Need to process data in a
reasonable 1me S Need a fault tolerant architecture as
large chunks of data cannot be reprocessed if and when a failure occurs
S Need automa1c resource management
S Issues S Long running jobs S Missed SLA’s S Performance & Response Issues S Scalability S Fault Tolerance
BeneGits
S Requirements S Assigning the appropriate systems and processes to manage
workloads S Creates an interchangeable infrastructure S Support Homogenous / Heterogeneous plaIorms S Need to process data in a reasonable 1me S Need a fault tolerant architecture as large chunks of data cannot
be reprocessed if and when a failure occurs S Need automa1c resource management
@2012 Copyright Sixth Sense Advisors
Questions
@2012 Copyright Sixth Sense Advisors
S What are the key drivers behind the “unity” architecture?
S What is the biggest benefit of this applica1on?
S Will Teradata evolve this fabric in the next genera1ons?
S What releases of the ecosystem are supported?
S Will this include integra1on of other Teradata management tools in the future?
S How can Systems Administrators learn this architecture? What addi1onal skills are needed?
Thank You
@2012 Copyright Sixth Sense Advisors
Krish Krishnan [email protected] Twi]er Handle: @datagenius
Twitter Tag: #briefr
The Briefing Room
Upcoming Topics
This month: Innovators
January: Big Data
February: Analytics
2013 Editorial Calendar www.insideanalysis.com