Design and Implementation of a Service

Monitoring Console within a Service

Oriented Architecture Framework Zeroth Review

19-Jan-2012

Arvind Krishnaa J

Final Year, CSE ‘A’, 31508104017

Guided by

Dr. Chitra Babu,

HOD/CSE

Service Oriented Architecture

• Exposes an operation or a resource of some use

• Service contract specified through WSDL

Service Provider

• A repository of service listings

• Dynamically searchable Directory

• Finds a service contract from the directory

• Requests the service provider for an utility.

Consumer

Client-Server Interaction

Provider

eBay’s SOA Framework Develop a Service or Client using eBay’s Eclipse IDE

If it’s a service register it with the Asset Repository

If it’s a client, search in AR for the service it calls, by looking up WSDL

MetricsQueryService monitors service metrics

Service Provider and Service

Invoker Frameworks

Service Provider Framework Service Invoker Framework

Pluggable data

formats through

Serialization

factory

SOA Metrics Console (SMC)

Application Servers log events through TIBCO messages

CAL : Centralized Application Logging framework

All CAL logs aggregated, and OLAP cubes are constructed

Cube data pushed onto relational database using OLAP2db scripts

OLAP2db

Interface

GUI built using

in-house V4

framework.

Turmeric Monitoring Console

1. Deployed as a .war file in local server.

2. Monitors services/consumers created using Turmeric SOA plugin.

3. eBay’s CAL APIs log the metrics of services in the bundled MySQL database.

4. GUI built using Google Web Toolkit (GWT)

SMC vs. TMC

SMC Dashboard

1. Several operations which derive useful inferences from metrics.

2. Highly customizable reports.

3. Metrics stored in enterprise OLAP cubes.

1. Data loss (~60%) happening at OLAP2db layer.

2. V4 technology getting outdated rapidly.

3. Unacceptable slow page load time.

Turmeric Monitoring Console

1. Built on the highly scalable and browser independent GWT framework.

2. Faster response time from the UI layer.

3. OPEN SOURCE!

1. Only raw metrics are reported.

2. As of now, no facility to scale it up to enterprise level.

3. Monitors only services running on local server.

Objectives

1. Investigate the exact reasons for the data loss at the OLAP2db interface, and fine-tune or rewrite the scripts entirely.

2. Quantitative analysis between the merits of TMC and SMC.

3. If TMC is chosen, then the feature set of SMC needs to be mapped, and then implemented completely in TMC.

4. If SMC is preferred, then in addition to minimizing data loss, the response time of the dashboard needs to be improved.

5. Other persistence storage mechanisms, such as the distributed Cassandra NoSQL database to be explored.

Features and Functionality

Core Functionality

• Monitoring the metrics of services, such as call count, error count, response time and consumer traffic.

• Store the metrics efficiently on a distributed persistent storage.

Performance

• Average response time of the front-end should be within 2-3 seconds.

• UI designed using a stable and scalable framework.

Scalability

• Performance does not degrade irrespective of the number of metric logged.

• Capacity to handle any type of load, across several pools (groups of machines).

Reliability

• Continuous availability of the service (expect to maintain the current availability, which is around 99.94%)

• Metrics displayed are real-time and accurate, with minimum or no data loss.

Scope and Goal

Tune OLAP2db scripts to

minimize data loss

Create a service and a consumer using the

SOA plugin, and analyze its metrics

Quantitatively assert which alternative is

better? – To continue with SMC or integrate TMC

Tune SMC’s V4 Presentation Layer to decrease load time

Add missing features of SMC to TMC, including

an alternate storage mechanism

Come up with the design of an optimized monitoring console for

any SOA framework

Project Timeline

0 2 4 6 8 10 12 14 16

Analysis of Turmeric Code

Create a service and consumerusing Turmeric SOA plugin

Practical comparison of TMC anc SMC

Study of OLAP2db scripts

Tuning of OLAP2db scripts

Feasibiltiy study of TMC vs SMC

Enhance SMC to improve performance(or)

Expand TMC code to log more metrics

One-to-one feature mapping of TMC to SMC, withimprovements if needed

Final Report Preparation

Project Timeline

Duration (in weeks)

References and Bibliography [1] Leonard Richardson, Sam Ruby, “RESTful Web Services”, Second Edition, O’Reilly Media,

2009.

[2] eBay Open Source Project, “Turmeric SOA platform”

http://www.ebayopensource.org/index.php/Turmeric/HomePage

[3] eBay Open Source Project, Documentation of Turmeric SOA platform

https://www.ebayopensource.org/wiki/display/TURMERICDOC110GA/Turmeric+Doc

umentation+Overview

[4] eBay Open Source Project, Turmeric Source Code

http://www.github.com/ebayopensource

[5] Internal eBay documentation

(a) Internal wiki pages

(b) CAL Architecture document

(c) SOA platform Architecture document