A Pragmatic Reference Architecture for The Internet of Things
Pragmatic Architecture
description
Transcript of Pragmatic Architecture
![Page 1: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/1.jpg)
Pragmatic ArchitectureSimplify, Clarify, Economize
-Rajkumar Chandrasekaran
![Page 2: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/2.jpg)
• Sr. Dev Mgr & Innovation lead @ eBay
• 13+ years of IT experience
• Architecture (predominantly JSE/JEE, MicroSoft sometimes)
• Passion – Multiple technologies, Operational Excellence, Scalability
• Past time – Actionscript, Flex, Javascript (jQuery)
Who is your host
![Page 3: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/3.jpg)
Why Pragmatic Architecture
• process 70+ billion db operations a day
• ensure 99.94% availability
• contain 15000+ app servers & 1000+ database instances
• take on 8 billion url hits a day
• maintain 20+ TB worth of log data every day
Can you Imagine a system that can
“This is real and is happening and primarily due to the practical, realistic approaches taken by highly distinguished architects, engineers, management”
![Page 4: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/4.jpg)
• Takeaways
• Elements of Architecture
• A peek at what SOA is & is Not
• Arch Phases
• Real life problem
• Q & A
Agenda
![Page 5: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/5.jpg)
Takeaways
If you are a developer Programming for ‘ilities’ (or “abilities”) Understand newer/wider/deeper ways of architecting Simpler ways to embed Architectural vision into code
If you are an Architect Revisit some of the approaches towards building systems Sneak preview at highly scalable, highly available systems
If you are in Management Greater ROI, Faster returns Understand how great architecture can turn the business around
If you are none of the above Get all of the above
![Page 6: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/6.jpg)
Quiz Time
Q: Whose responsibility is Architecture?
a) Developersb) Architectsc) Managementd) Security
![Page 7: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/7.jpg)
Approaches to Architecture
![Page 8: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/8.jpg)
None
No realization
Zero effort in streamlining
High maintenance cost
Approaches to Architecture
![Page 9: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/9.jpg)
Pure
Sometimes very extensive
Misses the big picture, Long-term goals
High maintenance cost
Approaches to Architecture
![Page 10: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/10.jpg)
Pragmatic
![Page 11: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/11.jpg)
Pragmatic Architecture
More realistic approach Fine balance between Architectural Principles
and Product Goals Predominantly Incremental with long-term
vision Enables ‘Fail-fast’ Earns Trust & Respect from stakeholders
![Page 12: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/12.jpg)
Approaches to Architecture
No Pragmatic Pure0
5
10
15
20
Total Cost of Ownership (TCO)
TCO
![Page 13: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/13.jpg)
Architecture Evolution
![Page 14: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/14.jpg)
Architecture Basics– a quick look
![Page 15: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/15.jpg)
Quiz Time
Q: Which of the following makes sense?
a) Structure is part of Designb) Design is part of Architecturec) Architecture is part of Systemd) System is part of Architecture
![Page 16: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/16.jpg)
Structure Design
Framework
Architecture
Foun
datio
nRequirements
![Page 17: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/17.jpg)
Components Design Patterns
Frameworks / Libraries
Application Architecture
Foun
datio
nRequirements
![Page 18: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/18.jpg)
Services / Integration
Tiers / Layers
Software / Hardware
System Architecture
Foun
datio
nRequirements
![Page 19: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/19.jpg)
Enterprise Architecture
![Page 20: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/20.jpg)
Enterprise Architecture
System Architecture
Application Architecture
![Page 21: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/21.jpg)
Service Oriented Architectures
![Page 22: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/22.jpg)
Quiz Time
Q: Where does Service Orientation paradigm start from?
a) Web servicesb) Classc) Methodd) Variable
![Page 23: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/23.jpg)
public class SOA{
public boolean amISOA = true;
public boolean amISOA2(){
return (!false || !true) && (!(false || true ) || true);
}}
![Page 24: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/24.jpg)
public class SOA{
public boolean amISOA = true;
public boolean amISOA2(){
return (!false || !true) && (!(false || true ) || true);
}}
![Page 25: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/25.jpg)
Demystifying SOA
SOA is just a web service SOA is only for very large applications SOA is only used in multi-org integrations SOA increases scope/estimate drastically SOA is always a Request-Response sync call SOA reduces flexibility in all situations
Misconceptions :
![Page 26: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/26.jpg)
(Back to) Arch Evolution
Reality
Goals
Functional & Non functional Requirements/ Long term vision / Architecture Principles /
Constraints
![Page 27: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/27.jpg)
Architecture Phases – Phase I
Product Goal
Understand
Refine
Challenge
Vision
Influence
“Great listening skills at this stage helps in getting the long term vision right”
![Page 28: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/28.jpg)
Architecture Phases – Phase II
Product Goal
Domain View
Deployment View“Get the big rocks first”
Architecture Vision
Collaboration View
* Domain view – sub domains optional* Deployment view at a high level
![Page 29: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/29.jpg)
Architecture Phases – Phase III
Product Goal Further decomposition
Architecture Principles
“The devil is in the detail”
Architecture VisionInterface Definition
Domain CompositionTechnology Selection
Framework Selection
etc, etc and etc
![Page 30: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/30.jpg)
Architecture Principles
Non-Runtime Runtime
General Guideline
-Identify top 3 from Runtime Abilities
-Identify top 3 from Non-Runtime Abilities
![Page 31: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/31.jpg)
Architecture Phases – Phase IVAcross layers: Interaction requirements (SLA, fault tolerance rate etc)
Within each layer: Optimize, Choose the right technology, Callout non-obvious limitations
![Page 32: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/32.jpg)
Prioritize, Prioritize and Prioritize
Understand which of the abilities are most important to realize your goal
![Page 33: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/33.jpg)
Problem 1 – Huge DB read/writes
Requirements
8 Billion url hits/day
appox 1 B active users
99.94% available
$1, 736 trade per second
10’s of TB of add’l data to
be stored
Lightening search
![Page 34: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/34.jpg)
Solution – Huge DB read/writes
Phase I
• Understand multiple domains that need access and Prioritize the need• Abstract out Data access elements in to a separate layer (if that is not the case already)
Phase II
• Understand the current infrastructure, data center location, network limitations• Callout inter-domain SLAs and expectations
![Page 35: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/35.jpg)
Solution – Huge DB read/writes
Phase III
• Identify data sources that need to be highly available/readable• Ensure Domain App servers reside alongside ‘most-needed’ data sources• Partition DB significantly
Phase IV
• Optimize each data source for read/write• Single Write & Multiple reads• Data dependant routing for data spread across physical hosts
![Page 36: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/36.jpg)
Solution – Huge DB read/writes
Item Transaction
Product
User
Account Feedback
Domain-wise data split
![Page 37: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/37.jpg)
ItemHost N
Image Processing
Selling Summary Update
User Metrics
ITEM.NEW
Solution – Huge DB read/writes
Data dependant Routing & Data push to down stream (synch/asynch) systems
![Page 38: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/38.jpg)
Solution – Huge DB read/writes
DR & Failovers
Consumer
Primary Failover
![Page 39: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/39.jpg)
Solution – Huge DB read/writes
‘Divide & conquer’ Indexing & Search
![Page 40: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/40.jpg)
Solution – Huge DB read/writes
Surprises!NO Foreign Keys
NO Joins
Multiple Physical tables are mapped to one Logical table
Multiple Logical tables mapped to one physical table
![Page 41: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/41.jpg)
Quiz Time
Q: Why would someone translate multiple logical hosts to single physical host
a) Extensibility / Flexibilityb) Just for func) Scalabilityd) They got nothing better to do
![Page 42: Pragmatic Architecture](https://reader036.fdocuments.net/reader036/viewer/2022062520/568161b7550346895dd184bd/html5/thumbnails/42.jpg)
Questions
• Twitter / Facebook / Orkut / Skype : rajbits