Guerrilla SOA

39
Guerrilla SOA How to fight back when a vendor takes control of your enterprise Dr. Jim Webber Dr. Jim Webber Service-Oriented Systems Practice Lead Service-Oriented Systems Practice Lead ThoughtWorks ThoughtWorks http://jim.webber.name http://jim.webber.name

Transcript of Guerrilla SOA

Page 1: Guerrilla SOA

Guerrilla SOAHow to fight back when a vendor takes control of your enterprise

Dr. Jim WebberDr. Jim WebberService-Oriented Systems Practice LeadService-Oriented Systems Practice LeadThoughtWorksThoughtWorkshttp://jim.webber.namehttp://jim.webber.name

Page 2: Guerrilla SOA

Fundamental PremiseFundamental Premise

There are two things money cannot buy:There are two things money cannot buy:

• LoveLove(Lennon/McCartney)(Lennon/McCartney)

• An SOAAn SOA(Webber)(Webber)

Page 3: Guerrilla SOA

RoadmapRoadmap

Enterprise Application Integration ApproachesEnterprise Application Integration Approaches

Enterprise Architecture, now and futureEnterprise Architecture, now and future

The Appealing Rationale for ESB...The Appealing Rationale for ESB...

Enterprise ArchitectureEnterprise Architecture

Realising SOA with Web ServicesRealising SOA with Web Services

What this means for youWhat this means for you

ConclusionsConclusions

Q&AQ&A

Page 4: Guerrilla SOA

Enterprise Application Integration Enterprise Application Integration ApproachesApproaches

Data integrationData integrationExtract, transform, route, inject dataExtract, transform, route, inject data

Application levelApplication levelRe-use application APIs, or I/O mechanismsRe-use application APIs, or I/O mechanisms

EAI implementationEAI implementationQueues etcQueues etc

Business domain tierBusiness domain tierIntegration at the object level, as typified by CORBA, Integration at the object level, as typified by CORBA, DCOM etcDCOM etc

User interface User interface Screen scraping, revamping, etc.Screen scraping, revamping, etc.

Last resort, when an application offers no other hooksLast resort, when an application offers no other hooks

Page 5: Guerrilla SOA

To ESB or not to ESB, that is the To ESB or not to ESB, that is the questionquestion

Product vendors are keen to provide product Product vendors are keen to provide product solution for everythingsolution for everything

Or to supply “consultantware” solutionsOr to supply “consultantware” solutions

The Enterprise Service Bus is the latest The Enterprise Service Bus is the latest incarnation of EAI technology that supports a incarnation of EAI technology that supports a number of useful functions:number of useful functions:

Transformations; adapters; choreography; reliability; Transformations; adapters; choreography; reliability; security etcsecurity etc

Seems like a good idea...Seems like a good idea...

Page 6: Guerrilla SOA

Today’s Enterprise ArchitectureToday’s Enterprise ArchitectureAccounting Marketing

SupportProduct Development

Page 7: Guerrilla SOA

How did we get here?How did we get here?

Tactical decisionsTactical decisions

Time and technology pressuresTime and technology pressures

Path of least resistance for individual Path of least resistance for individual applicationsapplications

This is the thin end of the wedge, technical debt This is the thin end of the wedge, technical debt can only increase from herecan only increase from here

Help!Help!

Page 8: Guerrilla SOA

Vendor Solutions AppearVendor Solutions Appear

Business needs to Business needs to competecompete

IT needs to be responsiveIT needs to be responsive

SOA gives IT a business SOA gives IT a business process focusprocess focus

Web Services are the Web Services are the most sensible way to most sensible way to implement SOAimplement SOA

More proprietary More proprietary middleware is the middleware is the answer!answer!

2 + 2 = 52 + 2 = 5

See: http://www.capeclear.com/technology/index.shtmlSee: http://www.capeclear.com/technology/index.shtml

Page 9: Guerrilla SOA

Accounting Marketing

SupportProduct Development

Integration Two Years LaterIntegration Two Years Later

Enterprise Service Bus

Page 10: Guerrilla SOA

Skeletons in the Closet...Skeletons in the Closet...

Enterprise Service Bus

Page 11: Guerrilla SOA

The Appealing Rationale for ESB...The Appealing Rationale for ESB...

Perceived single framework for all integration Perceived single framework for all integration needsneeds

Perceived simple connectivity between systemsPerceived simple connectivity between systems

Some features for security, reliable delivery, etc.Some features for security, reliable delivery, etc.

All you have to do is agree to lock yourself into a All you have to do is agree to lock yourself into a ESB and all this can be yours...ESB and all this can be yours...

Page 12: Guerrilla SOA

...And the Reality...And the Reality

The mess is swept under the carpetThe mess is swept under the carpetThe spaghetti is still there, but it’s hidden inside a vendor boxThe spaghetti is still there, but it’s hidden inside a vendor box

But the spaghetti is worse with an ESBBut the spaghetti is worse with an ESBMixing business rules, transformations, QoS etc with connectorsMixing business rules, transformations, QoS etc with connectors

What if I wanted to remove or replace my current ESB What if I wanted to remove or replace my current ESB platform?platform?

Vendor lock-in of the whole network!Vendor lock-in of the whole network!

ESBs are proprietary, so no guarantees that the messages ESBs are proprietary, so no guarantees that the messages transmitted across the bus are actually based on any open transmitted across the bus are actually based on any open protocolprotocol

Held to ransom by the ESB vendor!Held to ransom by the ESB vendor!Cannot easily replace one ESB with anotherCannot easily replace one ESB with another

Can only easily integrate systems for which the ESB vendor Can only easily integrate systems for which the ESB vendor provides specific adaptorsprovides specific adaptors

Or invest your money into extending their productOr invest your money into extending their product

Page 13: Guerrilla SOA

Intelligent Networks, Dumb Idea?Intelligent Networks, Dumb Idea?

Isn't this precisely what we're trying to get away Isn't this precisely what we're trying to get away from?from?

Integration should happen on the wire by default, Integration should happen on the wire by default, not inside some servernot inside some server

The ESB approach eschews the dumb network, The ESB approach eschews the dumb network, smart endpoint notion that underpins scalable, smart endpoint notion that underpins scalable, robust systemsrobust systems

ESB vendors are the new telcos – telling us that ESB vendors are the new telcos – telling us that smarts in the network is for our own goodsmarts in the network is for our own good

But let’s see how ESBs play out over the longer But let’s see how ESBs play out over the longer termterm

Page 14: Guerrilla SOA

Integration five years from nowIntegration five years from nowAccounting Marketing

SupportProduct Development

ResearchIT

Enterprise Service Bus

Page 15: Guerrilla SOA

Accounting Marketing

SupportProduct Development

ResearchIT

Integration ten years from nowIntegration ten years from now

ESB

Page 16: Guerrilla SOA

How did this happen?How did this happen?

Same old story:Same old story:Tactical decisionsTactical decisions

Time and technology pressuresTime and technology pressures

Path of least resistance for individual applicationsPath of least resistance for individual applications

Centralised ownership of the ESB sometimes is Centralised ownership of the ESB sometimes is an inhibitoran inhibitor

Too much effort to get on the bus, technically, Too much effort to get on the bus, technically, politicallypolitically

Individuals always mean to redress hacked Individuals always mean to redress hacked integrationsintegrations

But seldom do – it’s too hard when systems are liveBut seldom do – it’s too hard when systems are live

Page 17: Guerrilla SOA

Spaghetti is a fact of lifeSpaghetti is a fact of life

Businesses changeBusinesses change

Processes changeProcesses change

Applications changeApplications change

Integration changesIntegration changes

Need an enterprise computing strategy that:Need an enterprise computing strategy that:Reflects the changing structure of the business;Reflects the changing structure of the business;

Is spaghetti-friendly;Is spaghetti-friendly;

Commoditised;Commoditised;

Robust, secure, dependable, etc.Robust, secure, dependable, etc.

Page 18: Guerrilla SOA

Business-Led IntegrationBusiness-Led Integration

ESBs integrate with whatever existing systems exposeESBs integrate with whatever existing systems exposeGreen screen, web pages, CORBA objects, XML, etcGreen screen, web pages, CORBA objects, XML, etc

Integration happens at a low levelIntegration happens at a low levelMapping of bits and bytes of one variety onto bits and bytes of Mapping of bits and bytes of one variety onto bits and bytes of another formatanother format

This makes it hard to engage business in such projectsThis makes it hard to engage business in such projectsWithout business benefit no software has valueWithout business benefit no software has value

Integration is currently opaque to the businessIntegration is currently opaque to the business

Business must be involved in integration projects – not Business must be involved in integration projects – not just initiate themjust initiate them

The integration domain must use the same vocabulary as the The integration domain must use the same vocabulary as the business domainbusiness domain

Page 19: Guerrilla SOA

Spaghetti-Oriented ArchitectureSpaghetti-Oriented Architecture

Fighting against spaghetti is usually Fighting against spaghetti is usually unsuccessfulunsuccessful

This does not mean integration should be undertaken This does not mean integration should be undertaken without diligence!without diligence!

SOA is an approach which is spaghetti-agnosticSOA is an approach which is spaghetti-agnostic

Services are designed for integration with any Services are designed for integration with any consumerconsumer

Integration is decentralisedIntegration is decentralised

Result:Result:Loosely coupled, re-usable servicesLoosely coupled, re-usable services

Focus on business-meaningful process orchestrationFocus on business-meaningful process orchestration

Page 20: Guerrilla SOA

SOA and Web Services ApproachSOA and Web Services Approach

Applications (or subsets of applications) are Applications (or subsets of applications) are identified as being service-amenableidentified as being service-amenable

Or (sub) processes are identified for which there is no Or (sub) processes are identified for which there is no existing application/serviceexisting application/service

Web Services infrastructure is layered on top of Web Services infrastructure is layered on top of the application, exposing a SOAP interface to the the application, exposing a SOAP interface to the rest of the networkrest of the network

Business meaningful message exchangesBusiness meaningful message exchanges

Other services consume the functionality via Other services consume the functionality via SOAP message exchangesSOAP message exchanges

Traditional integration infrastructure is kept within Traditional integration infrastructure is kept within the Web Service implementation, if used at allthe Web Service implementation, if used at all

Page 21: Guerrilla SOA

Building the Service-Oriented Building the Service-Oriented EnterpriseEnterprise

SOAP becomes the ubiquitous transfer mechanism SOAP becomes the ubiquitous transfer mechanism across the enterprise (or Internet!)across the enterprise (or Internet!)

In effect, SOAP messages are the “EAI backbone” In effect, SOAP messages are the “EAI backbone” The underlying transport protocols are arbitraryThe underlying transport protocols are arbitrary

Applications understand SOAP messages nativelyApplications understand SOAP messages nativelyTrue end to end integration, but maintains loose couplingTrue end to end integration, but maintains loose coupling

In this context, existing ESB/EAI software becomes a In this context, existing ESB/EAI software becomes a toolkit for implementing individual Web Servicestoolkit for implementing individual Web Services

But integration happens at the SOAP levelBut integration happens at the SOAP levelCan commoditise what’s underneathCan commoditise what’s underneath

Page 22: Guerrilla SOA

The QoS functionality that a The QoS functionality that a Web Service requires is Web Service requires is implemented on a per-service implemented on a per-service basisbasis

Not “one size fits all”Not “one size fits all”

Implement only those QoS Implement only those QoS protocols that the service protocols that the service currently needscurrently needs

Push the integration Push the integration functionality to the edgesfunctionality to the edges

SOAP + WS-Addressing SOAP + WS-Addressing becomes the “bus”becomes the “bus”

Incremental and autonomousIncremental and autonomousDeliver high business-value Deliver high business-value services first!services first!

Non-Repudiation

Reliable Delivery

Security

Transactions

Decentralised IntegrationDecentralised Integration

Page 23: Guerrilla SOA

Application Integration...Application Integration...

EAI/ESB frameworks are fine for application integrationEAI/ESB frameworks are fine for application integrationA framework for development of (distributed) applicationsA framework for development of (distributed) applications

Think of the EAI toolkit as a container for your applicationThink of the EAI toolkit as a container for your applicationApplication versus enterprise frameworkApplication versus enterprise framework

Bus

Choreography/Rules/Routing/Transformations

Adapter Adapter Adapter

Application Domain

Page 24: Guerrilla SOA

...and Composite Business ...and Composite Business ProcessesProcesses

Processes across the enterprise consume and Processes across the enterprise consume and coordinate lower-level applicationscoordinate lower-level applications

Exposed via standards-based servicesExposed via standards-based services

Bus

Choreography/Rules/Routing/Transformations

Adapter Adapter Adapter

Application Domain

Gateway

Enterprise Process Domain

SOAP Messaging,

WS-*

Page 25: Guerrilla SOA

Web Services Standards: The State Web Services Standards: The State of Playof Play

Not all of the WS-* QoS protocols are ready yetNot all of the WS-* QoS protocols are ready yetGenerally in late stages of standardisation and early Generally in late stages of standardisation and early adoptionadoption

WS-Security is ready; WS-ReliableExchange WS-Security is ready; WS-ReliableExchange (WS-RX) in standardisation; WS-AT/BA ready on (WS-RX) in standardisation; WS-AT/BA ready on Java platform; WS-AT on .NetJava platform; WS-AT on .Net

Web Services toolkits for the rest will be Web Services toolkits for the rest will be available in the next year or soavailable in the next year or so

WCF, BizTalk, Axis2, XFire, etcWCF, BizTalk, Axis2, XFire, etc

Metadata is the key!Metadata is the key!

Page 26: Guerrilla SOA

Metadata, Metadata, Metadata…Metadata, Metadata, Metadata…

<endpoint…>

</endpoint>

<mex…>

</mex>

<wsdl…>

<policy…>

</policy>

</wsdl>

Page 27: Guerrilla SOA

Policy and ContractPolicy and Contract

<wsdl…>

<policy…>

</policy>

</wsdl>

<wsdl…>

<policy…>

<security-policy>

</security-policy>

<transaction-policy>

</transaction-policy>

<reliability-policy>

</reliability-policy>

</policy>

</wsdl>

Page 28: Guerrilla SOA

Web

Ser

vice

s C

lien

t S

tack

(W

CF

)Proxy GenerationProxy Generation

<wsdl…>

<policy…>

<security-policy>

</security-policy>

<transaction-policy>

</transaction-policy>

<reliability-policy>

</reliability-policy>

</policy>

</wsdl>

Proxy API

Consumer Implementation

Security Handler

Tx Handler

RM Handler

Page 29: Guerrilla SOA

Web

Ser

vice

s C

lien

t S

tack

(W

CF

)End-to-End MessagingEnd-to-End Messaging

Proxy API

Consumer Implementation

Security Handler

Tx Handler

RM Handler

Web

Ser

vice

s C

lien

t S

tack

(W

CF

)

Proxy API

Service Implementation

Security Handler

Tx Handler

RM Handler

Transport

Page 30: Guerrilla SOA

WS-SecurityWS-SecurityMature, widely implemented, and interoperableMature, widely implemented, and interoperable

WS-I Basic Security Profile tooWS-I Basic Security Profile too

Gives end-to-end securityGives end-to-end securityPrivacy, integrityPrivacy, integrityCan be used for authentication, non-repudiationCan be used for authentication, non-repudiation

http tcp jms

logical point-to-point SOAP message transfer

resources

message processing

Web Service logic

transport

resources

message processing

Web Service logic

transport

intermediate intermediate

Page 31: Guerrilla SOA

Reliable SOAP MessagingReliable SOAP Messaging

Non-interoperable implementations from two Non-interoperable implementations from two competing standards todaycompeting standards today

WS-ReliabilityWS-ReliabilityWS-ReliableMessagingWS-ReliableMessaging

Single, unified standard (WS-ReliableMessaging) Single, unified standard (WS-ReliableMessaging) being developed (by WS-RC committee)being developed (by WS-RC committee)

Not yet widely implementedNot yet widely implemented

Cannot achieve interoperable RM at the SOAP Cannot achieve interoperable RM at the SOAP level todaylevel todayInstead, use existing, mature, transportsInstead, use existing, mature, transports

MQ, JMS, MSMQ, etcMQ, JMS, MSMQ, etc

Page 32: Guerrilla SOA

Reliable Transport AlternativesReliable Transport Alternatives

SOAP messages are transport agnosticSOAP messages are transport agnosticChange the transport, change the bindingChange the transport, change the binding

Route messages over a reliable channel Route messages over a reliable channel E.g. MQ, MSMQ, JMS etcE.g. MQ, MSMQ, JMS etc

ServiceService

TCPQueued Transport

Page 33: Guerrilla SOA

Coordinating ServicesCoordinating ServicesProcesses need end-to-end guaranteesProcesses need end-to-end guarantees

Manual coordination necessary todayManual coordination necessary todayGeneric coordination soon!Generic coordination soon!

Service

Service

ServiceProcess

Business process message

Business process fault-handling message

Business process fault message

Process-specific coordination

logic

Process-specific fault handlers

Page 34: Guerrilla SOA

““WS-Fabric”WS-Fabric”

Service

Ad

min

istr

ativ

e d

omai

n

Service

Service

Service

Administrative domain

Service

Service Adm

inistrative dom

ain

network

SOAP messaging is the communica tion channel for. ( ) applica tions The ESB if it exis ts is pushed to the

.endpoints

Page 35: Guerrilla SOA

Same Old ArchitectsSame Old Architects

Business people and application architects design Business people and application architects design business-centric workflows which consume servicesbusiness-centric workflows which consume services

Re-using the functionality already deployed into the service Re-using the functionality already deployed into the service ecosystemecosystem

Service architects and developers build servicesService architects and developers build servicesUsing WS toolkits like WCF and AxisUsing WS toolkits like WCF and Axis

Enterprise architects influence QoS at the SOAP level...Enterprise architects influence QoS at the SOAP level...Using the WS-* specsUsing the WS-* specs

...and at the transport level...and at the transport levelExisting investments can form the underlay for SOAExisting investments can form the underlay for SOA

And undertake necessary governance rolesAnd undertake necessary governance roles

Page 36: Guerrilla SOA

ESB or SOA?ESB or SOA?

Investing in proprietary integration systems now Investing in proprietary integration systems now is investing in future legacyis investing in future legacy

ESB is not the solutionESB is not the solutionIt’s oh-so 1990’s integration glueIt’s oh-so 1990’s integration glue

SOA is the solutionSOA is the solutionBecause it focuses on supporting business processesBecause it focuses on supporting business processes

Web Services are a robust and commoditised Web Services are a robust and commoditised platform for SOA deliveryplatform for SOA delivery

Page 37: Guerrilla SOA

ConclusionsConclusions

SOA is the right integration architecture going forwardSOA is the right integration architecture going forwardSOA can be implemented incrementallySOA can be implemented incrementally

Drive SOA from a business perspectiveDrive SOA from a business perspectiveMost valuable processes/applications/services firstMost valuable processes/applications/services first

Commoditisation across the boardCommoditisation across the boardServers, developers, networking, re-use existing software, etcServers, developers, networking, re-use existing software, etc

Migrating towards a successful SOA is not always easyMigrating towards a successful SOA is not always easyLearning to build dependable SOAs can be difficultLearning to build dependable SOAs can be difficult

ESBs and Wizards cannot help – you need service-savvy geeks ESBs and Wizards cannot help – you need service-savvy geeks and process-aware business peopleand process-aware business people

No centralised integration middleware neededNo centralised integration middleware neededMetadata, metadata, metadata!Metadata, metadata, metadata!

It looks like It looks like you’re trying to you’re trying to build an SOA...build an SOA...

Page 38: Guerrilla SOA

Questions?Questions?

http://jim.webber.namehttp://jim.webber.name

Page 39: Guerrilla SOA

RemindersReminders

Fill in your evaluation formsFill in your evaluation formsWith heaps of praise!With heaps of praise!

Tech Ed Party this evening @ Home Nightclub Tech Ed Party this evening @ Home Nightclub