HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect
-
Upload
spagoworld -
Category
Technology
-
view
392 -
download
0
Transcript of HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect
![Page 1: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/1.jpg)
© 2016 MapR Technologies 1© 2016 MapR Technologies 1MapR Confidential
© 2016 MapR Technologies
Fabian WilckensEMEA Solutions Architect
THINK DIFFERENTLYStream-based Microservice Architecture for Next-Generation Applications
![Page 2: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/2.jpg)
© 2016 MapR Technologies 2© 2016 MapR Technologies 2MapR Confidential
![Page 3: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/3.jpg)
© 2016 MapR Technologies 3© 2016 MapR Technologies 3MapR Confidential
![Page 4: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/4.jpg)
© 2016 MapR Technologies 4© 2016 MapR Technologies 4MapR Confidential
![Page 5: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/5.jpg)
© 2016 MapR Technologies 5© 2016 MapR Technologies 5MapR Confidential
The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.
- Martin Fowler
MICROSERVICE ARCHITECTURAL STYLESINGLE APPLICATION
SUITE OF SMALL SERVICESCOMMUNICATING WITH LIGHTWEIGHT MECHANISMS
BUILT AROUND BUSINESS CAPABILITIESFULLY AUTOMATED
DEPLOYMENTBARE MINIMUM OF CENTRALIZED MANAGEMENTWRITTEN IN DIFFERENT PROGRAMMING LANGUAGES
DIFFERENT DATA STORAGE TECHNOLOGIES
![Page 6: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/6.jpg)
© 2016 MapR Technologies 6© 2016 MapR Technologies 6MapR Confidential
WAIT A SECOND
SOA?SERVICE ORIENTED ARCHITECTURE
IS THERE A DIFFERENCE?
![Page 7: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/7.jpg)
© 2016 MapR Technologies 7© 2016 MapR Technologies 7MapR Confidential
Integrates different applications as a set of services
Architect a single application as a set of services
APP 1 APP 2
APP 4 APP 5
ENTERPRISE SERVICE BUS
APP 2
LOGISTICSSERVICE
CUSTOMERSERVICE
BILLINGSERVICE
USER INTERFACE SERVICE
MICROSERVICE-STYLE
CLASSIC SOA-STYLE
![Page 8: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/8.jpg)
© 2016 MapR Technologies 8© 2016 MapR Technologies 8MapR Confidential
A few years back …
![Page 9: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/9.jpg)
© 2016 MapR Technologies 9© 2016 MapR Technologies 9MapR Confidential
App Server
WAR/EAR
Backend
Logistics
Ordering
User Interface
Easy to develop? Easy to deploy? Easy to scale?
![Page 10: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/10.jpg)
© 2016 MapR Technologies 10© 2016 MapR Technologies 10MapR Confidential
CHALLENGES AHEAD
![Page 11: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/11.jpg)
© 2016 MapR Technologies 11© 2016 MapR Technologies 11MapR Confidential
LARGE CODE & BIG DATAHARD TO UNDERSTAND & CHANGE
OVERLOADED IDE
OVERLOADED WEB CONTAINERS
DEVELOPMENT SLOWS DOWN
EVEN SMALL APPLICATION CHANGES REQUIRE FULL REBUILD, TEST AND DEPLOYMENT
IMPACT ANALYSIS IS A HUGE EFFORT AND TAKES TIME
DIFFICULT TO FREQUENTLY CHANGE & DEPLOY
![Page 12: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/12.jpg)
© 2016 MapR Technologies 12© 2016 MapR Technologies 12MapR Confidential
OTHER CHALLENGESLONG-TERM COMMITMENT TO TECHNOLOGY STACK
RE-WRITE = COMPLETE RE-WRITE
MODULE DEPENDENCIES FORCE RE-WRITE
FAILURE IN MONOLITHIC APPLICATIONS BRINGS IT DOWN
![Page 13: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/13.jpg)
© 2016 MapR Technologies 13© 2016 MapR Technologies 13MapR Confidential
HORIZONTAL SCALING
App ServerWAR/EAR
Backend
Logistics
Ordering
User Interface
App ServerWAR/EAR
Backend
Logistics
Ordering
User Interface
App ServerWAR/EAR
Backend
Logistics
Ordering
User Interface
![Page 14: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/14.jpg)
© 2016 MapR Technologies 14© 2016 MapR Technologies 14MapR Confidential
VERTICAL SCALING
App ServerWAR/EAR
Backend
Logistics
Ordering
User Interface
App Server
WAR/EAR
Backend
Logistics
Ordering
User Interface
![Page 15: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/15.jpg)
© 2016 MapR Technologies 15© 2016 MapR Technologies 15MapR Confidential
DATA SCALING
App ServerWAR/EAR
Backend
Logistics
Ordering
User Interface
App ServerWAR/EAR
Backend
Logistics
Ordering
User Interface
App ServerWAR/EAR
Backend
Logistics
Ordering
User Interface
App ServerWAR/EAR
Backend
Logistics
Ordering
User Interface
![Page 16: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/16.jpg)
© 2016 MapR Technologies 16© 2016 MapR Technologies 16MapR Confidential
![Page 17: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/17.jpg)
© 2016 MapR Technologies 17© 2016 MapR Technologies 17MapR Confidential
MICROSERVICES SCALING
Container
Ordering
Container
Ordering
Container
Ordering
Ordering Database
Container
User Interface
Container
Logistics
LogisticsDatabase
Container
Backend
BackendDatabase
![Page 18: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/18.jpg)
© 2016 MapR Technologies 18© 2016 MapR Technologies 18MapR Confidential
MICROSERVICES TEAM SCALING
Container
Ordering
Container
Logistics
Container
Backend
![Page 19: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/19.jpg)
© 2016 MapR Technologies 19© 2016 MapR Technologies 19MapR Confidential
LET’S TALK BENEFITS (PROS)ONLY MAKE APIS TALK TO EACH OTHER
ADD MODULES BASED UPON BUSINESS REQUIREMENTS
CHANGE MODULES INDEPENDENTLY
DIFFERENT TEAMS WORK ON THEIR OWN PROBLEMS
FASTER TIME-TO-MARKET FOR NEW SERVICES
CAN STILL INTEGRATE WITH SOA
SERVICES/TEAMS SCALE INDEPENDENTLY
…
![Page 20: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/20.jpg)
© 2016 MapR Technologies 20© 2016 MapR Technologies 20MapR Confidential
![Page 21: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/21.jpg)
© 2016 MapR Technologies 21© 2016 MapR Technologies 21MapR Confidential
![Page 22: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/22.jpg)
© 2016 MapR Technologies 22© 2016 MapR Technologies 22MapR Confidential
The Challenge with Data Pipelines
Filtering & Aggregation
Alerting Processing
![Page 23: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/23.jpg)
© 2016 MapR Technologies 23© 2016 MapR Technologies 23MapR Confidential
Streams Simplify Data Movement
Filtering & Aggregation
Alerting Processing
StreamsReliable publish/subscribe transport between sources and destinations.
![Page 24: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/24.jpg)
© 2016 MapR Technologies 24© 2016 MapR Technologies 24MapR Confidential
Legacy Systems: Message QueuesIBM MQ, TIBCO, RabbitMQ
OrdersFront End
Order Processing
Order Processing
Usage/Requirements●Tight, transactional conversations between systems
●1:1 or Few:Few
●Low data rates
●Mission-critical delivery
Approach
●Queue-oriented design
●Each message replicated to N output queues
●Messages popped when read
●Scale-up, master/slave
Doesn’t Do
●High message rates (>100K/s)
●Slow consumers
●Queue replay/rewind
![Page 25: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/25.jpg)
© 2016 MapR Technologies 25© 2016 MapR Technologies 25MapR Confidential
Evolving “big data” Event Streams: Distributed LogsKafka, Hydra, DistributedLog
Usage/Requirements●High throughput data transferred from decoupled systems
●Many->1
●1->Many
●Different speeds
Approach
●Log-oriented design
●Write messages to log files
●Consumers pull messages at their own pace
●Scale-out
Doesn’t Do
●Global applications
●Message persistence
●Integrated analytics (data movement required)
DB_Changes
Stream Processing
Search/EDW
DB
![Page 26: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/26.jpg)
© 2016 MapR Technologies 26© 2016 MapR Technologies 26MapR Confidential
Introducing MapR StreamsGlobal Publish/Subscribe Event Streaming
Producers Publish Billions of
messages/sec to a topic
ConsumersReliable delivery to all
consumers. Immediately
GlobalTie together geo-dispersed
clusters. Worldwide
![Page 27: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/27.jpg)
© 2016 MapR Technologies 27© 2016 MapR Technologies 27MapR Confidential
MapR Converged Data Platform
Tenant 2 Tenant 3
Tenant 1
![Page 28: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/28.jpg)
© 2016 MapR Technologies 28© 2016 MapR Technologies 28MapR Confidential
Container
Ordering
Container
User Interface
Container
Logistics
Container
Backend
![Page 29: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/29.jpg)
© 2016 MapR Technologies 29© 2016 MapR Technologies 29MapR Confidential
Global Applications
Container
Ordering
Container
Backend
Container
Logistics
Container
User Interface
Container
User Interface
Container
User Interface
![Page 30: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/30.jpg)
© 2016 MapR Technologies 30© 2016 MapR Technologies 30MapR Confidential
Are Microservices here to stay?
We don’t know …Monolithic Apps are dead
![Page 31: HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect](https://reader031.fdocuments.net/reader031/viewer/2022030304/58778b991a28abc85f8b7509/html5/thumbnails/31.jpg)
© 2016 MapR Technologies 31© 2016 MapR Technologies 31MapR Confidential
Content & Pictures• http://enterprise.vnews.com/wp-content/uploads/2015/07/EP-SeanSmithCover-ns-vn-0728151.jpeg• http://www.rodmatthews.com.au/_assets/images/blog/Positional%20Images/Complexity%20Small%20
Version.jpg• http://www.coliquio-insights.de/wp-content/uploads/2015/02/5141328136_16d8f73ee7_b.jpg