ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?
-
Upload
avisi-bv -
Category
Technology
-
view
498 -
download
2
description
Transcript of ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?
![Page 1: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/1.jpg)
Space-Based Architecture:Linear scalability? High throughput? Low latency?
Pedro Rodrigues – [email protected]
![Page 2: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/2.jpg)
Space-Based Architecture (SBA) intro: ▫ Main concepts & a bit of history
SBA in practice▫ Lessons learned
SBA live▫ Linear scalability: demo
Agenda
2AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 3: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/3.jpg)
“Space-Based Architecture (SBA) is a software architecture pattern for achieving linear scalability of stateful, high-performance applications using the tuple space paradigm.”
@wikipedia
What is SBA?
3AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 4: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/4.jpg)
“scalability: scalability is the ability to grow an application to meet growing demand, without changing the code, and without sacrificing the data affinity and service levels demanded by your users.”
“linear scalability: the ability to grow as much as needed, at a fixed price per capacity unit”
@The Scalability Revolution: From Dead End to Open Road
Linear scalability
4AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 5: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/5.jpg)
Distributed memory model for interprocess coordination/communication.
Memory contents: tuples ▫ e.g. (“Pedro”, “Sogyo”, 5)
Memory is associative ▫ retrieval through template matching (e.g.
(“Pedro”, ?Employer, ?YearsOfService))
Tuple Space
5AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 6: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/6.jpg)
Operations available:▫ in – atomic read and remove▫ rd – atomic read ▫ out – write tuple into space▫ eval – create new processes to evaluate tuples
and write the results to the space
Tuple Space
6AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 7: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/7.jpg)
1983 – Linda programming model, Tuple Space.– David Gelernter and Nicholas Carriero at Yale
University
1998 – Jini, JavaSpaces– Sun Microsystems
2000 – GigaSpaces provides first commercial implementation of JavaSpaces
2000-2003 – SBA is born @GigaSpaces 2013 – GigaSpaces XAP 9.6.0
SBA: A bit of history
7AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 8: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/8.jpg)
Linear scalability using Tuple Space: how?
8AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 9: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/9.jpg)
SBA: processing unit
9AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
Processing Unit
SpaceData
Messages
Service A
Service B Service
C
NotifyTake Write
![Page 10: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/10.jpg)
Processing Unit
SpaceData
Messages
Service A
Service B Service
C
Processing Unit
SpaceData
Messages
Service A
Service B Service
C
Processing Unit
SpaceData
Messages
Service A
Service B Service
C
SBA: linear scalability processing unit replication + sharding
10AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
ClientApplication
Space Proxy
Processing Unit
SpaceData
Messages
Service A
Service B Service
C
Processing Unit
SpaceData
Messages
Service A
Service B Service
C
![Page 11: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/11.jpg)
SBA: fault toleranceprimary and backup processing units
11AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
Primary Processing Unit
SpaceData
Messages
Service A
Service B Service
C
Backup Processing Unit
SpaceData
Messages
Service A
Service B Service
C
![Page 12: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/12.jpg)
SBA: deployment/run environment
12AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
@wiki.gigaspaces.com
![Page 13: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/13.jpg)
SBA in practice
13AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 14: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/14.jpg)
Disclaimer
▫ For secrecy reasons, no detailed information about the project can be provided.
SBA in practice: a project
14AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 15: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/15.jpg)
Client:▫ Dutch governamental organisation
Goal:▫ Proof of concept of a system that extracts
information in “real time” from data coming in from a large set of disparate data sources.
Constraints:▫ Number of data sources and the amount of
incoming data is due to increase at a large pace in the near future.
SBA in practice: a project
15AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 16: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/16.jpg)
Team:▫ @full-time▫ 1 architect
▫ 1 project manager
▫ 3 experienced developers with (almost) no previous experience with SBA
▫ @part-time▫ 1 consultant with experience in SBA
Duration:▫ 7 months
SBA in practice: a project
16AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 17: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/17.jpg)
End Result:▫ A working PoC ▫ … and guidelines to guarantee high-performance
of the system to be built upon the deliverables of the PoC.
SBA in practice: a project
17AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 18: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/18.jpg)
SBA is not difficult to grasp Involve people with SBA experience Keep learning Partitioning data is key
▫ do it badly and you’ll pay the price
Size matters (when objects reach the millions)▫ E.g. Calendar vs long, 432 bytes vs 8 bytes
Perform (automatic?) integration tests in a cluster from the start
The project: lessons learned
18AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 19: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/19.jpg)
Only write to disk if necessary If you have to write to disk, do it asynchronously
(and in batches) Garbage Collection remains a challenge.
The project: lessons learned II
19AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 20: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/20.jpg)
Linear scalability: a demo
SBA
20AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 21: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/21.jpg)
SBA: demo
21AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
Processing Unit
Space
MessageProcessor
Take WriteClient
Application
Space Proxy
1. Writes a maximum of 30000 messages per second into the space.
2. Reads the number of messages processed after 10 seconds.
Takes every message written into the space and writes back a new object. Can process a maximum of 1000 messages per second.
![Page 22: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/22.jpg)
SBA offers a simplified programming model to build high-performance distributed systems
SBA simplifies administration/monitoring SBA itself does not guarantee high-performance
▫ Linear scalability, low latency, high throughput can be achieved but only with care
Beware:▫ It remains a challenge to introduce SBA in an
organisation used to work mainly with tier based solutions
Conclusion
22AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 23: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/23.jpg)
Give SBA a try if you’re dealing with data flood “write once, scale anywhere” seems more than a
slogan
Take Home Message
23AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 24: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/24.jpg)
?
24AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 2013
![Page 25: ASAS 2013 - Space-based architecture: Linear scalability? High throughput? Low latency?](https://reader034.fdocuments.net/reader034/viewer/2022051208/5479c7d4b4795963098b4883/html5/thumbnails/25.jpg)
AGILE SOFTWARE ARCHITECTURE SYMPOSIUM 201325
Contact
Pedro Rodrigues
030 - 220 22 16
Web: www.sogyo.nl
Blog: www.software–innovators.nl