Klarna Tech Talk - Mind the Data!
-
Upload
jeffrey-t-pollock -
Category
Technology
-
view
394 -
download
1
Transcript of Klarna Tech Talk - Mind the Data!
Klarna Tech Talk:Mind the Data!Jeff PollockInfoSphere Information Integration & Governance
© 2013 International Business Machines Corporation 2
IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.
Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.
The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here
© 2013 International Business Machines Corporation
Accelerated Pace of Change
Data
Commerce
Shanghai
© 2013 International Business Machines Corporation
Capabilities
Turn information into insights
Deepen Engagement with customers, partners and employees
Enable the agile business
Accelerate product and service innovation
Deliver enterprise mobility
Optimize IT and business infrastructure
Manage risk, security and compliance
Driven by Technology Innovations
1. IDC 2015 Market Opportunity 2. IDC & Other External Sources 2015 Opportunity 3. IDC 2015 Market Opportunity, excluding services 4. IBM internal analysis based on IDC data
Mobile Enterprise $36B2
Cloud and Optimized Workloads
$73B4
Security Intelligence $38B3
Big Data Analytics $18B1
Categories Markets
© 2013 International Business Machines Corporation
Data-driven Technology = Data-driven Business
Volume Variety Velocity Veracity
Data at ScaleTerabytes to
petabytes of data
Data in Many FormsStructured, unstructured,
text, multimedia
Data in MotionAnalysis of streaming data to enable decisions within
fractions of a second.
Data UncertaintyManaging the reliability and predictability of inherently
imprecise data types.
© 2013 International Business Machines Corporation
© 2013 International Business Machines Corporation
Data Fabric Helps Clients Build Smarter Businesses
Human Resources
CustomerService
Sales
Marketing
Finance
Logistics
Technology & ProductDevelopment
© 2013 International Business Machines Corporation
Type 1: Normal Batch
• Traditional high-volume runtime
Type 2: Micro-Batch
• Regular intervals for near-real time
Type 3: ELT (SQL or MapReduce)
• Push processing to the data
Type 1: Relational Views, SQL
• Models and Nicknames give layered views into
federated data
Type 2: XML Views, XQuery
• XML documents and XQuery query semantics for
data retreival
Type 3: Services APIs, SOA
• Data Services layer for SOA
Type 1: Consolidation
• Bring data from many sources and make it available to users in one
Type 2: Distribution
• Workload balancing
Type 3: Peer to Peer
• Continuous bi-directional synchronization
CDC Based / Always On
• Receive low-latency log-based replication from CDC and process through complex transformation with guaranteed delivery
Type 1: Message Broker
Advanced Enterprise Service Bus (ESB) with the
reliability of MQ
Type 2: B2B Appliance
• XML hardware acceleration for high-speed any-to-
any transformations
• Trading partner connectivity and support for key
industry standards like HL7, X12, and EDIFACT
Trigger a Data Integration Job
Launch data integration/ cleansing tasks through web service bindings
Complex Queue Transformation
• Guaranteed delivery from/to message queues for complex, heterogenous requirements
Queue-based Delivery
• Deliver log-based replicated changes to a message queue with guaranteed delivery
Big Data Processing (Hadoop, MPP DW), Metadata (Management, Lineage, Impact) and Governance (Glossary, Quality, Discovery etc)
Batch
Replication
Federation
Messaging
Shop for Data Information Catalog
Universal place to begin looking for the business data you need
Canonical Data Services
• Always available application services with composite data views
Human-centric Data Stewardship and Data Curation
• Workflow managed enrichment of business data
Type 1: Simple Search
• Traditional keyword search
Type 2: Advanced Search
• Category and taxonomy search
Type 2: Faceted Navigation
• Active browse by value or schema
Type 1: Federated
• Access data via central registry
Type 2: Hub and Spoke
• Maintain source of truth
Type 3: Hybrid
• Leverages reference data, data by registry and local trusted data
Customer 360
• Total customer visibility with business data view and social data view – including Web sources
Search & Browse
Master Data
Data Fabric Access and Provision Data Seamlessly
Big Data Fabric
© 2013 International Business Machines Corporation 9
Smart Data is Integrated and Governed Data
Best Performance
Self Service Integration
Rapid Discovery
Single Platform
Unified Blueprints
Cloud Ready
Industry Focused Fastest ROI
Trusted Governance Confidence
Agile Frameworks Lowest TCO
© 2013 International Business Machines Corporation 10
Dynamic & LinearInstantly get better performance as hardware resources are added to any topology
On Demand CapacityAdd a new servers to scale out through simple configuration
Data PartitionedIn true MPP fashion (like PureData or Hadoop) data persisted in the data integration platform is stored in parallel to scale out the I/O.
Hadoop IntegratedPush all or part of the process out to PureData or Hadoop to take advantage of it’s scalability in ELT fashion.
Disk
CPU
Memory
Sequential
Disk
CPU
Shared Memory
CPUCPU CPU
4-way Parallel 64-way Parallel
Uniprocessor SMP System MPP Clustered
AnySourceData
Transform
Cleanse Enrich
Supported with DataStage MPP Grid or Hadoop clusters
Go Native – and Bring the Processing to the Data
© 2013 International Business Machines Corporation 11
3x 77%80%
Organizations with IIG outperform their
competitors
OutperformCompetitors
Organizations rated their decision making as good or excellent
Transform the Front Office Experience
EstablishTrusted Information
Organizations establish high or very high level of
trust in data
Value from Integration and Governance
© 2013 International Business Machines Corporation
Mind the Data (before it gets a mind of it’s own!)
Data
© 2013 International Business Machines Corporation
Technology Innovation :1st Integrated Information Server Platform1st Data Governance Capabilities Built-in1st Mainstream Big Data ETL Platform1st Unified Data Domain MDM Solution1st Integrated Data Lifecycle & Archiving
10 of the top 10 global banks
25 of the world’s leading telecoms
17 of the top 20 Chinese financial services firms
5 of the top 6 global insurance providers
Top 3 global automobile manufacturers
3 of the top 5 global retailers
Proven Results – IBM Integration & Governance
Market Leadership:#1 ETL Market Share (IDC)#1 MDM Market Share (Gartner)#1 Data Quality Vision (Gartner)#1 in Integration Capabilities (Gartner)#1 Lifecycle Management (IDC)
Get Started with IBM Big Data
© 2013 International Business Machines Corporation
Get Started with Big Data – A Reference Architecture
© 2013 International Business Machines Corporation
IBM’s Big Data PortfolioSmarter Analytics / Business Analytics & Optimization
Information Server
MDM, Optim
Guardium
SPSS
Cognos
BigInsights
Data Explorer
Industry Solutions• Financial Analytics
• Risk Analytics
• Threat & Fraud
• Workplace Analytics
• Customer Analytics
Business Intelligence
Performance Management
Content Management
Information Management Foundation
Client Services
Volume Variety Velocity Veracity
CONSULTING and IMPLEMENTATION SERVICES
Performance Management
Content Analytics
Decision Management
Risk Analytics
Business Intelligence and Predictive Analytics
ANALYTICS
Information Integration and Governance
BIG DATA PLATFORMContent
ManagementData
WarehouseStream
ComputingHadoop System
SECURITY, SYSTEMS, STORAGE AND CLOUD
Sales Marketing Finance Risk IT Operations HR
SOLUTIONS
Watson and Industry Solutions
CONSULTING and IMPLEMENTATION SERVICES
Performance Management
Content Analytics
Decision Management
Risk Analytics
Business Intelligence and Predictive Analytics
ANALYTICSPerformance Management
Content Analytics
Decision Management
Risk Analytics
Business Intelligence and Predictive Analytics
ANALYTICS
Information Integration and Governance
BIG DATA PLATFORMContent
ManagementData
WarehouseStream
ComputingHadoop System
SECURITY, SYSTEMS, STORAGE AND CLOUD
Sales Marketing Finance Risk IT Operations HR
SOLUTIONS
Watson and Industry Solutions
Sales Marketing Finance Risk IT Operations HR
SOLUTIONS
Watson and Industry Solutions
© 2013 International Business Machines Corporation
Big Data Exploration Enhanced 360o Viewof the Customer
Operations Analysis Data Warehouse Augmentation
Security/IntelligenceExtension Understand confidence
Determine risk Establish master record Extent to all sources
Automatic data protection Mask sensitive information
High volume data integration Automatic data protection
High volume data integration Agile big data archiving and retrieval
Building Confidence with Top 5 Big Data Use Cases
© 2013 International Business Machines Corporation
Automotive manufacturer to build out global data warehouse
Need• Consolidate existing DW projects globally • Deliver real-time operational reporting • Integrate and gain new insights across all
data sources
Benefits• Single infrastructure to consolidate
structured, semi-structured and unstructured data
• Proven, enterprise-class capabilities that can be deployed quickly and are simpler to manage
Data Warehouse Augmentation
© 2013 International Business Machines Corporation
Financial service provider enables customer-centric cross selling
Need:
• Federated views of data on 20 million customers
• View data across 160 siloed systems
Benefit
• Empowered agents
• Improved cross-selling to high value clients
Enhanced 360o Viewof the Customer
© 2013 International Business Machines Corporation
IBM Big Data Industry Momentum
© 2013 International Business Machines Corporation