Dwp from monolithtomicroservices
-
Upload
andrew-rea -
Category
Technology
-
view
40 -
download
0
Transcript of Dwp from monolithtomicroservices
Department for Work & Pensions 2
From Monolith to Microservices by Andrew Rea
Know your MonolithAPI CONUSMERS
APPLICATION
DATABASE
API
BUSINESS LOGIC
DATA ACCESS
PRESENTATION
APIAPI
DATABASE
Service 1 Service 2 Service “n”
DB DB DB
Service 1 Service 2 Service “n”
Access
Postgres
MSSQL
WORD DOCUMENT
EXCEL DOCUMENT
VBA
APPLICATION
UI Business Logic
Data Access Database
Logging Reports
Open Monolith Database Monolith Distributed Monolith
Paper Monolith Office MonolithClosed Monolith
Department for Work & Pensions 3
From Monolith to Microservices by Andrew Rea
That’s another fine mess you’ve gotten me into
APPLICATION
DATABASE
CODE
CODECODE
CODE
TABLE
TABLETABLE
TABLE
STRONG CODE COUPLING STRONG DB COUPLING DATABASE COUPLING NO DOMAIN BOUNDARIES IMPLICIT RELATIONSHIPS
Department for Work & Pensions 4
From Monolith to Microservices by Andrew Rea
That’s another fine mess you’ve gotten me into
APPLICATION
DATABASE
CODE
CODECODE
CODE
TABLE
TABLETABLE
TABLE
£ £ £INCREASES COST
INCREASES IMPLEMENTATION TIME
DECREASES VELOCITY
INCREASES RISK
INCREASES DEPLOYMENT SIZE
INCREASES COMPLEXITY
INCREASES CODE FEAR
INCREASES BUILD TIME
INCREASES TEST TIME
Department for Work & Pensions 5
From Monolith to Microservices by Andrew Rea
Microservices?APPLICATION
DATABASE
SERVICE
SERVICESERVICE
SERVICE
TABLE TABLE
TABLE
TABLE
NOT YET BUT…THIS IS MANAGEABLE
Department for Work & Pensions 6
From Monolith to Microservices by Andrew Rea
APPLICATION
DATABASE
SERVICE
SERVICESERVICE
SERVICE
TABLE TABLE
TABLE
TABLE
Microservices?NOT YET BUT… THIS IS MANAGEABLE
LOOSE COUPLING HIGH COHESION ENCAPSULATION DATABASE DECOUPLING DOMAIN BOUNDARIES EXPLICIT RELATIONSHIPS SERVICE INTEGRATION SERVICE COMPOSITION
Department for Work & Pensions 7
From Monolith to Microservices by Andrew Rea
What are Microservices?
SMALL
HOW SMALL?
INTEROPERABLE
HOW?
INDEPENDENT
HOW?
A REASON TO RE-STRUCTURE YOUR TEAMS
WHY? RESILIENT
HOW?
Department for Work & Pensions 8
From Monolith to Microservices by Andrew Rea
How do you get there?
GOOD DESIGN PRACTICES& PRINCIPLES
Department for Work & Pensions 9
From Monolith to Microservices by Andrew Rea
SEPARATE PROCESS PER SERVICE
APPLICATIONS
DATABASES
TABLE
TABLE
EASIER TO ADOPT DIFFERENT TECH
INDEPENDENTLY HOSTED
INDEPENDENTLY DEPLOYABLEINDEPENDENTLY SCALABLE
SERVICE SERVICE
SERVICE
SERVICE
GRAPHKEY / VALUEDOC
Department for Work & Pensions 10
From Monolith to Microservices by Andrew Rea
INDEPENDENT - HOW?
DEVELOP INDEPENDENTLY
DEPLOY INDEPENDENTLY
BUILD INDEPENDENTLY
OPERATE INDEPENDENTLYMONITOR INDEPENDENTLYSURVIVE INDEPENDENTLY
VERSION INDEPENDENTLYPROCESS
CONTINUOUSINTEGRATION
CODE REPOSITORY
DATA STORES
CONTAINER IMAGE
SERVICEDISCOVERY
DNS (A-RECORD)
QUEUES
PROVISIONINGSCRIPTS
CERITIFICATES
LOGICAL Encapsulation
Boundary
REQUIREMENTS
LOAD BALANCERPOOL
MONITORINGGROUP
SERVICE
PHYSICAL Encapsulation
BoundaryVERSION
Department for Work & Pensions 11
From Monolith to Microservices by Andrew Rea
INTEROPERABLE - HOW?
COMMON TRANSPORT PROTOCOL
RESTFUL API
PUBLISH AND SUBSCRIBE CAPABILITY
EVENT STREAM
STRUCTURED LOGGING IN COMMON FORMAT
SERVICE
RESTFUL API OVER HTTPS
COMMANDS QUERIES EVENTS
STD OUTMETRICS STD ERROR
LOG AGENT LOG AGGREGATOR
Department for Work & Pensions 12
From Monolith to Microservices by Andrew Rea
SEPARATE SERVICES BY BUSINESS CAPABILITY
SERVICE
DOMAIN 1
DOMAIN 2
DOMAIN 3
SERVICE 1
DOMAIN 1
SERVICE 2
DOMAIN 2
SERVICE 3
DOMAIN 3
Monolith Microservices
Department for Work & Pensions 13
From Monolith to Microservices by Andrew Rea
HOW DO THEY SCALE?
SERVICE
DOMAIN 1
DOMAIN 2
DOMAIN 3
SERVICEDOMAIN 1
SERVICEDOMAIN 2
SERVICEDOMAIN 3
SERVICE
DOMAIN 1
DOMAIN 2
DOMAIN 3
SERVICE
DOMAIN 1
DOMAIN 2
DOMAIN 3
SERVICEDOMAIN 3
SERVICEDOMAIN 1
SERVICEDOMAIN 3
SERVICEDOMAIN 3
SERVICEDOMAIN 3
SERVICEDOMAIN 2
SERVICEDOMAIN 2
SERVICEDOMAIN 1
Monolith Microservices
SAVES MONEY
Department for Work & Pensions 14
From Monolith to Microservices by Andrew Rea
DOES THIS HELP SECURITY?WE CAN DEPLOY SEPARATE INSTANCES OF THE SAME SERVICE IN DIFFERENT SECURITY ZONES TO MATCH
THE SENSITIVITY OF THE DATA WHICH THE SERVICE INSTANCE WILL BE HANDLING.
TRUST 1 TRUST 2
SERVICESEARCH
SERVICESEARCH
SERVICESEARCH
SERVICESEARCH
SERVICESEARCH
SERVICEBOOKING
SERVICEBOOKING
SERVICEBOOKING
SERVICEAVAILABILITY
SERVICEAVAILABILITY
SERVICEAVAILABILITY
SERVICEAVAILABILITY
DMZ
SERVICESEARCH
SERVICEBOOKING
SERVICEAVAILABILITY
Department for Work & Pensions 15
From Monolith to Microservices by Andrew Rea
SEPARATE BUILD PIPELINES
SERVICE 1
SERVICE 2
SERVICE 3
COMMIT FUNCTIONAL NON-FUNCTIONAL SMOKE TESTSRELEASE
SERVICE 1
SERVICE 1
SERVICE 1
SERVICE 1
INDEPENDENT BUILD PIPELINESINDEPENDENT BUILD SCRIPTSINDEPENDENT PROVISION SCRIPTSISOLATED TEST ENVIRONMENTS
Department for Work & Pensions 16
From Monolith to Microservices by Andrew Rea
SEPARATE SOURCE CONTROL
SERVICE
SERVICE
SERVICE
SERVICE
SERVICE
SERVICE SERVICE SERVICE
SERVICE SERVICE
GIVE EACH SERVICE ITS OWN SOURCE REPOSITORY
Department for Work & Pensions 17
From Monolith to Microservices by Andrew Rea
SEPARATE SOURCE CONTROL
GIVE EACH SERVICE ITS OWN SOURCE REPOSITORYTEAMTEAM
TEAM
SERVICE
SERVICE
SERVICE
SERVICE
SERVICE
SERVICE SERVICE SERVICE
SERVICE SERVICE
WhoOwnsThis?
Department for Work & Pensions 18
From Monolith to Microservices by Andrew Rea
A REASON TO RESTRUCTURE YOUR TEAMS
SERVICEAVAILABILITY
SERVICEBOOKING
SERVICESEARCH
SERVICEPROVIDERS
SERVICESEARCH
SERVICEPROVIDERS
SERVICEAVAILABILITY
SERVICEBOOKING
SERVICE (NOT COMPONENT) TEAMSFEATURE TEAMS
OR BOTH?BOTH CAN PLAY A VALUABLE PART IN A MICROSERVICES ARCHITECTUREBOTH HAVE THE SAME TEAM STRUCTURE
PULL REQUESTS ANYONE?
Department for Work & Pensions 19
From Monolith to Microservices by Andrew Rea
HOW SMALL (OR LARGE)?
2000 LINES OF CODE?
TWO PIZZA TEAM SIZE?
CAN BE RE-WRITTEN IN TWO WEEKS?
MODELS A SINGLE CONTEXT?
HOW ABOUT ALL OF THE ABOVE?What thought processes do the above questions trigger about your design?
TOO BIG? MAKE SMALLER, TOO SMALL? MAKE BIGGER
Department for Work & Pensions 20
From Monolith to Microservices by Andrew Rea
HOW DO THEY COMMUNICATE
CALL
CALL
CALL
SERVICEDOMAIN 1
SUBSCRIBE
SUBSCRIBE
SUBSCRIBE
CALL
CALL
CALL
OR BOTH
DIRECT INVOCATION EVENT DRIVEN
SERVICEDOMAIN 1
SERVICEDOMAIN 2
SERVICEDOMAIN 3
SERVICEDOMAIN 4
SERVICEDOMAIN 2
SERVICEDOMAIN 3
SERVICEDOMAIN 4
Department for Work & Pensions 21
From Monolith to Microservices by Andrew Rea
WHAT ABOUT MONITORING?
CALL
CALL
CALL
META ENDPOINTSMESSAGE CORRELATIONSYNTHETIC MONITORING
HEALTH CHECK
METRICS
INFORMATION
STRUCTURED LOGGINGBUSINESS APPLICATION INFRASTRUCTURENETWORK
LOCAL -> AGENT -> REMOTE
Department for Work & Pensions 22
From Monolith to Microservices by Andrew Rea
HOW DO CONSUMING APPS GET STABILITY?
SERVICE
DOMAIN 1
Stable Next Latest
CONSUMER A
DEVELOPERS
VERSIONS WITH DIFFERENT LEVELS OF STABILITY
Consuming “Stable” in Production
Testing integration with “Next”
Researching “Latest”
Department for Work & Pensions 23
From Monolith to Microservices by Andrew Rea
HOW TO VERSION THE SERVICES?
<MAJOR>.<MINOR>.<PATCH>-<TAG>
https://medium.com/javascript-scene/software-versions-are-broken-3d2dc0da0783#.wcyli1rt2
http://semver.org
<BREAKING>.<FEATURE>.<FIX>-<TAG>
EXAMPLES
v1.0.1 v2.0.1-alpha v2.0.1-beta v2.1.0-latest
Department for Work & Pensions 24
From Monolith to Microservices by Andrew Rea
RESILIENT?DOWNSTREAM
SYSTEMS
CIRCUIT BREAKER
CONFIRMATION BULK HEADS
HEALTH CHECKSALIVE READY
EXTERNAL PERSISTENCE
ACKNOWLEDGEMENT
STATE EVENTS
STATELESS
AUTO-RECOVERY
Department for Work & Pensions 25
From Monolith to Microservices by Andrew Rea
RESILIENT?DOWNSTREAM
SYSTEM
DOWNSTREAMSYSTEM
TEST HARNESS
SWITCH OUT REALSYSTEM WITH A HARNESS RANDOM
STATUS CODE TEST HARNESS
SLOW RESPONSE
TEST HARNESS
HUGE RESPONSE
TEST HARNESS
EMPTY RESPONSE
TEST HARNESS
BAD GATEWAY
RESPONSE TEST
NO RESPONSE TEST HARNESS
HOW WILL THE SERVICE RESPONDTO ABNORMAL SYSTEM DEPENDENCIES?
LETS FIND OUT BEFORE WE GO LIVE!
Department for Work & Pensions 26
From Monolith to Microservices by Andrew Rea
RESILIENT?
SERVICE TESTING UNIT TESTING INTEGRATION TESTINGACCEPTANCE TESTING
MUTATION TESTING FUZZING TESTINGSTATIC CODE ANALYSIS
PERFORMANCE TESTING LOAD TESTING STRESS TESTING IMPULSE TESTING LONGEVITY TESTING
FAILURE TESTING
SECURITY TESTING
SMOKE TESTING
RELEASE TESTING ROLLBACK TESTING
FUNCTIONAL
CROSS FUNCTIONAL
Department for Work & Pensions 27
From Monolith to Microservices by Andrew Rea
WHEN SHOULD THE TEAM DEPLOY?
WHEN CAN THE TEAM DEPLOY?
Department for Work & Pensions 28
From Monolith to Microservices by Andrew Rea
A LOT OF THIS IS NOT NEW
GOOD DESIGN PRACTICES & PRINCIPLES
BUT THEY DON’T COME FOR FREE!
Department for Work & Pensions 29
From Monolith to Microservices by Andrew Rea
CHEAT SHEET
EXTERNAL
INTERNAL
AGGREGATIONCORRELATIONVISUALIZATION
REPORTING
API
LOGG
ING
DATAM
ESSA
GING
APPLICATION
INTEGRATION
OPERATION
HTTPSXML JSON
SOAP REST
THROTTLING AUTHENTICATION AUTHORIZATION
CIRCUIT BREAKER
INSPECTION ROUTING
CIRC
UIT
BREA
KER
BULK HEADS
CIRCUIT BREAKERBULK HEADS BU
LK H
EADS
RELIA
BLE
MES
SAG
ING
DURA
BLE
MES
SAG
ING
HEALTH
DEPLOYMENT SERVICE REGISTRATION
CONFIGURATION READINESS CHECKS
STRU
CTUR
ED LO
GG
ING
TIM
E SE
RIES
BUSI
NES
SAP
PLIC
ATIO
NIN
FRAS
TRUC
TURE
NET
WO
RK
ERRO
R CA
TALO
G
EVEN
T CA
TALO
G
INFORMATION DISCOVERABLE API
BACK UP
SECURITYSECURITY SECURITY
TEST HARNESSES
CONTINUOUSINTEGRATION
CODE REPOSITORY
DATA STORES
Application Identity
PROTECTIVE MONITORING
COMMANDS QUERIES
EVENTS
DOMAIN MODEL
DOMAIN SERVICES
VERSIONING
DOMAIN EVENTS
CORR
ELAT
ION
ID
SERVICE DEPENDENCIES
SUBSCRIBING SERVICES
HORIZONTAL SCALING
FAILOVER
RECOVERY
PUBLISH
EVENT LOG
SUBSCRIBE
SECURITY
KEY SECURITY
ENCRYPTION IN TRANSIT
ENCRYPTION AT REST
SECURE DEVELOPMENT, DEPLOYMENT & OPERATIONAL PRACTICES AND PRINCIPLES
CONTAINER IMAGE
CONSISTENCY BOUNDARY
API GATEWAY
SERVICEFABRIC
SERVICEREGISTRY
LOAD BALANCER
WRITE STO
RE
READ STORE(S)
REVERSE PROXY
INSP
ECTI
ONAPPLICATION FIREWALLS
INSPECTIO
N
ENCAPSULATION BOUNDARY
CONTAINERREGISTRY
HEXAGANOL ARHCICTECTURE
Version 1 Microservices Infograhic by Andrew Rea
EXTERNALMONITORING
MONITORING
MONITORING
?SERVICE
DISCOVERY
DATA STORE ISO
LATION AM
QPM
QTT
HTTP
othe
r…
GRAPH
DOCUMENTKEY/VALUE
RELATIONAL
COLUMN
FLAT FILE