WEBINAR: Architectures for Digital Transformation and Next-Generation Systems of Engagement
-
Upload
aerospike-inc -
Category
Technology
-
view
305 -
download
0
Transcript of WEBINAR: Architectures for Digital Transformation and Next-Generation Systems of Engagement
1© 2016 Aerospike Inc. All rights reserved. [ ]
Architectures for Digital Transformation
and Next-Generation Systems of
Engagement
June 21, 2016
2© 2016 Aerospike Inc. All rights reserved. [ ]
Noel Yuhanna - Principal Analyst,
Forrester Research
Brian Bulkowski, CTO and CO-
Founder, Aerospike
Speakers
3© 2016 Aerospike Inc. All rights reserved. [ ]
Response time: Hours, Weeks
TB to PB
Read Intensive
TRANSACTIONS
(OLTP)
Response time: Seconds
Gigabytes of data
Balanced Reads/Writes
ANALYTICS (OLAP)
STRUCTURED
DATA
Response time: Seconds
Terabytes of data
Read Intensive
BIG DATA ANALYTICS
System of engagement
Response time: < 5 ms
1-100 TB
Balanced Reads/Writes
24x7x365 Availability
UNSTRUCTURE
D DATA
NoSQL
Database Landscape
5© 2016 Aerospike Inc. All rights reserved. [ ]
LEGACY DATABASE
(Mainframe)
XDR
Decisioning Engine
DATA WAREHOUSE/
DATA LAKE
LEGACY RDBMS
HDFS BASED
BUSINESS
TRANSACTIONS
Web views
( Payments )
( Mobile Queries )
( Recommendation )
( And More )
High Performance NoSQL
“REAL-TIME BIG DATA”
“DECISIONING”
500Business Trans per sec
5000 Calculations per sec
X = 2.5 M Database Transactions per sec
Operational Scale
6© 2016 Aerospike Inc. All rights reserved. [ ]
CREDIT CARD
PROCESSING SYSTEM
FRAUD DETECTION &
PROTECTION APP
ACCOUNT
BEHAVIOR
ACCOUNT
STATISTICS
STATIC DATA
RULE 1 – PASSED ✔
RULE 2 – PASSED ✔
RULE 3 – FAILED ✗
HISTORICAL
DATA
RULES
RULE 1
RULE 2
RULE 3
…
• Challenge
– Implement new algorithmsas new patterns develop
– Every transaction requires hundreds of DB reads/writes
– High, constant ingest rates
– Scale-up as algorithms and business grow
– Overall SLA 750 ms
• Need to scale reliably
– 10 100 TB
– 10B 100 B objects
– 200k I Million+ TPS
• Selected NoSQL
– Built for Flash
– Predictable Low latency at High Throughput
– Immediate consistency
– Cross data center (XDR) for high availability
Fraud Prevention
7© 2016 Aerospike Inc. All rights reserved. [ ]
■ Challenge
■ DB2 stores positions for 10 Million customers
■ Value-at-risk calculations in minutes, not hours
■ Consistent view of trade state across all applications
■ Must update stock prices, show balances on 300
positions, process 250M transactions, 2 M updates/day
■ Cache uneconomical – 150 servers growing to 1000
■ Need to scale reliably
■ 3 13 TB
■ 100 400 Million objects
■ 200k I Million TPS
■ Selected NoSQL
■ Flash
■ Predictable Low latency at High Throughput
■ Immediate consistency
■ Cross data center (XDR) support
■ 10 Server Cluster
IBM DB2(MAINFRAME)
Read/Write
Start of Day
Data Loading
End of Day
Reconciliation
QueryREAL-TIME
DATA FEED
ACCOUNT
POSITIONS
XDR
Fin Serv – Positions System of Record
8© 2016 Aerospike Inc. All rights reserved. [ ]
Challenge• Per-account routing rules win edge systems
• Traffic shaping to implement account policies
• Accessible using provisioning applications
(self-serve and through support personnel)
Need for Extremely High Availability,
Reliably, Low latency• TBs of data
• 10-100M objects
• 10-200K TPS
Selected NoSQL• Clustered system
• Predictable low latency at high throughput
• Highly-available and reliable on failure
• Cross data center (XDR) support
SOURCE
DEVICE/USER DESTINATIONReal-Time
Auth. QoS Billing
Request Execute
Request
Real-Time ChecksConfig Module App
Update Device
User Setting
Hot-Standby
XDR
Telco – Real-Time Billing and Charging Systems
9© 2016 Aerospike Inc. All rights reserved. [ ]
Challenge• Billions of users & cookies across the internet
• Accessible using provisioning applications
(self-serve and through support personnel)
• Real-time algorithms used for targeting, offers.
Need for Extremely High Availability,
Reliably, Low latency• 10’s TBs of data
• 1B ~ 10B objects
• 1M ~ 10M TPS
Selected NoSQL• Clustered HA system
• Predictable low latency at high throughput
• Highly-available and reliable on failure
• Cross data center (XDR) support
AdTech – Targeting, Bidding, Programmatic
INTERNET
AD EXCHANGE
BIDDING
APPLICATION
SEARCHES
VISITS
TIME ON PAGE
AUDIENCE
HISTORICAL
DATA
BEHAVIOR
MODELS
MACHINE
LEARNING
10© 2016 Aerospike Inc. All rights reserved. [ ]
Aerospike – Enabling Your Digital Transformation
Powered by High
Performance NoSQL
Aerospike – The Next Generation
Operational Database
NEXT GENERATION ARCHITECTURE
• No cache required – simpler architecture
• Patented and Designed for Flash– Log structured File System
• Record Oriented, Schema Free NoSQL KV Store
PREDICTABLE PERFORMANCE
• DRAM or Hybrid DRAM/Flash for Persistence
• Stable, Low Latency and high throughput under any condition
• Deployable on Bare Metal, virtualized, containerized, or Cloud
DYNAMIC CLUSTERING
• Highest Uptime & Availability of any NoSQL (5 nines plus)
• Automatic DB Cluster formation, healing and dynamic sharding
• Master based synchronous replication – immediate consistency
• Cross Data Center Replication (XDR)
SMART APPS
• Machine Learning, Spark & Hadoop Integration
• Broad language support (C/C++, Java, C#, Python, Go, Node.js, PHP)
• Accelerated development – schema free
• Rich APIs, Geospatial and Secondary Indexes
TCO
• Optimized for Flash and DRAM
• Demonstrated 10:1 price performance savings
• Full Utilization ->10x reduction in servers deployed
• Huge operational efficiency – “Set it and Forget it”
$
12© 2016 Aerospike Inc. All rights reserved. [ ]
Architecture
1) No Hotspots
– Distributed Hash Table simplifies data partitioning
2) Smart Client – 1 hop to data, no load balancers
3) Shared Nothing Architecture,
– every node is identical
4) Smart Cluster, Zero Touch
– auto-failover, rebalancing, rack aware, rolling upgrades
5) Transactions and long-running tasks prioritized in real-time
6) XDR – sync replication across data centers ensures
– Zero Downtime
13© 2016 Aerospike Inc. All rights reserved. [ ]
OTHER DATABASE
OS FILE SYSTEM
PAGE CACHE
BLOCK INTERFACE
SSD HDD
BLOCK INTERFACE
SSD SSD
OPEN NVM
SSD
OTHER
DATABASE
AEROSPIKE FLASH OPTIMIZED
DATABASE
AEROSPIKE
HYBRID MEMORY SYSTEM™
• Direct device access
• Large Block Writes
• Indexes in DRAM
• Highly Parallelized
• Log-structured FS “copy-on-write”
• Fast restart with shared memory
Flash Optimized High Performance