Intelligent integration with WSO2 ESB & WSO2 CEP

Intelligent Integration with WSO2 ESB and WSO2 CEP

Sriskandarajah Suhothayan (Suho)Technical Lead

WSO2 Inc.

Connected Business !

Connected Business !

To identity threats &

opportunities

What to integrate ?

In the line of Connected Business there will be many software systems

•Enterprise Resource Planning systems (ERP) •Management Information Systems (MIS) •Decision Support Systems (DSS) •Data stores • Legacy systems

Issues in integration

But integrating a multitude of complex software applications is not an easy task … • Disparate Systems, Services, Protocols • Diverse and dynamic business requirements • No single vendor/solution • On premise/Cloud solutions • Different QoS requirements • Not everyone adhere to standards

How could we integrate ?


Point to point connections ?

• Scalability, maintainability, troubleshooting nightmares.


Solution … Bus !

Integration Platform Contains …

• WSO2 ESB – Connects anything with every thing

• WSO2 DSS, WSO2 GReg – Expose data as a service, Repository and governance

• WSO2 MB – Guaranteed messaging

• WSO2 IS – Security, SSO, Role based access control

Integration

!!!!! Cloud On-premise ! Hybrid

Job Done :)

Oops !

http://islamgreatreligion.files.wordpress.com/2012/11/gaza-16-nov.jpg

http://islamgreatreligion.files.wordpress.com/2012/11/gaza-16-nov.jpg

The Prevention … ?

• Monitor ! • Monitor ! • Monitor !

The next mile ... !

Monitor and … • Take reactive action • Predict future & take preventive measures • Auto recovery at failure • Optimum resource utilisation • Ensure security & QoS • Alerts

Is it working as expected ?

• The load of the system

! It can be

The solution should be

• A realtime system • Capable of processing data on the fly • Notify alerts in various forms • Enable enterprise integration

What is ?

How ESB integrates with CEP?

How CEP integrates with ESB?

Patterns

• Monitoring Proxy service/API : QoS (E.g Response time) • In Message Store & forward : Managing the Queue • Monitoring Routing : Scaling, Throttling, Blacklisting

Monitoring QoS on Proxy Services & APIs

Monitoring response time

define stream ResponseTimeStream ( !! arrivalTime long, departureTime long, !! correlationID string, serviceID string, … ) ;!!from ResponseTimeStream!! [departureTime - arrivalTime > 2 min]!select correlationID, serviceID!insert into NotificationStream ;

Monitoring QoS on Proxy Services & APIs

Tuning Message Store and Forward

• Monitoring the Message Queue

Tuning Message Store and Forward

define stream QueueStatStream ( queueID string, size long ) ;!!define partition QueuePartition QueueStatStream.queueID;!!from a1 = QueueStatStream, ! a2 = QueueStatStream[size >a1. size]+!within 2 min!select a1. queueID as queueID, !! a1. size as firstSize,!! a2[last]. size as lastSize!insert into AdjustForwardingProcessStream!partition by QueuePartition;

Monitoring ESB routing and Scaling

Monitoring ESB routing and Scaling

define stream RequestStream ( ! !! correlationID string, serviceID string, !! endpointID string, … ) ;!!from RequestStream!select count(correlationID) as requestCount, !! serviceID, true as scaleUp!group by endpointID!having up requestCount > 90!insert into ScaleUpStream ;

Intelligent Scaling on

Throttling & Blacklisting users

Throttling & Blacklisting usersdefine stream RequestStream ( correlationID string, serviceID string, !

! ! userID string, tear string, requestTime long, … ) ;!!define table BlacklistedUserTable(userID string,!! ! ! ! time long,requestCount long); !


! ! userID string, tear string, requestTime long, … ) ;!!define table BlacklistedUserTable(userID string,!! ! ! ! time long,requestCount long); !!from RequestStream[tear==‘BRONZE’]#window.time(1 min)!

select userID, requestTime as time, !

! count(correlationID) as requestCount!

group by userID!

having up requestCount > 5!insert into BlacklistedUserTable ;!!


! ! userID string, tear string, requestTime long, … ) ;!!define table BlacklistedUserTable(userID string,!! ! ! ! time long,requestCount long); !!from RequestStream[tear==‘BRONZE’]#window.time(1 min)!

select userID, requestTime as time, !

! count(correlationID) as requestCount!

group by userID!

having up requestCount > 5!insert into BlacklistedUserTable ;!!from RequestStream[not(userID in BlacklistedUserTable. userID)] !select *!insert into ApprovedRequestStream ;

Patterns

• Monitoring Proxy service/API : QoS (E.g Response time) • In Message Store & forward : Managing the Queue • Monitoring Routing : Scaling, Throttling, Blacklisting !

• In Load balancing : Fix out of order arrivals • Rules : Temporal

Patterns

• Monitoring Proxy service/API : QoS (E.g Response time) • In Message Store & forward : Managing the Queue • Monitoring Routing : Scaling, Throttling, Blacklisting !

• In Load balancing : Fix out of order arrivals • Rules : Temporal !

• For Business Decisions : KPIs, Usage patterns, Waiting for the right time.. to trigger

• Monitoring for Security : Detect Attacks, Credit card fraud, Login patterns, Blacklisting

Waiting for the right time … to pull trigger

Waiting for the right time … to pull triggerdefine stream StockOrderRequest ( symbol string, !

! ! ! ! action string ) ;!

define stream StockStream ( symbol string, price double, … ) ;!

!define partition Stocks StockStream.symbol, ! ! !

! ! ! StockOrderRequest.symbol ;!

!from a1 = StockOrderRequest[action==‘buy’],!

! b1 = StockStream[price<a1.price]+,!

! b2 = StockStream[price>b1.price]!select a1.symbol as symbol, !! b1[0].price as firstPrice, !! b2.price as orderPrice!insert into PurchaseOrder;!

partition by Stocks

Fixing Security Vulnerabilities

• Identify Dos attacks • Analyse login patterns • Detect credit card fraud • Blacklisting


define stream PasswordChangeStream (userId string, time long); !!define stream TransactionStream (userId string, ! amount double, … ); !!from every (a1 = PasswordChangeStream) -> ! a2 = TransactionStream[amount >10000] !within 1 hour!select a1.userId as userId, a2. amount as amount!insert into PotentialFraud ;

!Better three hours too soon, than one minute too late. !!! William Shakespeare

Thank You

Questions ?

Intelligent integration with WSO2 ESB & WSO2 CEP

Data & Analytics

Transcript of Intelligent integration with WSO2 ESB & WSO2 CEP