W w w. P o r t S i g h t. c o m 1 Universal Web Service for Providing Enterprise Data Petr PALAS...

1

w w w. P o r t S i g h t . c o m

Universal Web Service for Providing Enterprise Data

Petr PALAS

PortSight Software Architect

[email protected]

www.PortSight.com

2


Quick Survey

Do you use Enterprise Information Portal in your company?

3


Agenda

• Motivation for Universal WS Tier

• Architecture Overview

• Issues & Solutions

• Advanced features

4


Why Universal Web Service?

5


Motivation for Universal WS (1)

General issues:• There’s a need to share data with

partners and customers• Users want to have 360° view of data.• Users want to access data from

Enterprise Portals and mobile devices.• Data come from various sources (ERP,

CRM, CMS...) that have different interfaces.

6


Ad Hoc Web Services (1)

• Building ad hoc WS on top of each system: – much effort, various interfaces– cannot provide joined view– more complicated management– coding necessary every time the data structure

is changed– fast development of new WS standards

7


Ad Hoc Web Services (2)

CRM ERP CMS

PortalApplications Partners

WS 1 WS 2 WS 3 WS 4

8


Motivation for Universal WS (2)

Universal WS

CRM ERP CMS

PortalApplications Partners

9


EAI versus Information Integration (1)

• Enterprise Application Integration – sharing data between applications

User Interface 1

User Interface 2

User Interface 3

10


EAI versus Information Integration (2)

• Information Integration – joined view of all data in one user/application/WS interface– BEA Liquid Data, IBM Xperanto, Nimble

Integration Suite– and others... (SQL Server 2003?)

Unified User/Application/ WS Interface

11


Architecture

12


Architecture Overview (1)

• Architecture Requirements:– Data from various sources (structured,

semi-structured and unstructured)– Unified interface for reading, searching

and modifying data– Unified security and metadata

management– Standard-based– Reasonable response time

13


Architecture Overview (2)

ERP CMS CRM Files Web Services

Unified Application and Web Services Interface

Portal Business Partners Mobile Devices

E-Mails

Data Aggregation Tier(Data adapters)

Integrated Services Tier(Metadata, caching, security, distributed search)

XML

14


First Idea (1)

• Describe data source – Source type

– Connection parameters

• Store name and schema of the table

• Publish it through WS

15


IntegratedServices Tier

First Idea (2) - Overview

Metadata:• Data source• Table name• Table schema

Data in XML

16


First Idea (3) – Data WS Interface

• SELECT(Table, Conditions) as XML• UPDATE(Table, UpdatedData)• INSERT(Table, InsertedData)• DELETE(Table, Conditions)• SEARCH(Table, Conditions,

SearchConditions) as XML• GetMetadata(Table) as XML containing XML

Schema

17


Issues & Solutions

18


Issue 1 – Data Joining (1)

Your distributor wants to see customers from Prague who ordered BMW car. But data are in RDBMS and XML. How to join them?

19



• Option 1: XQuery You can join semi-structured XML

data from various sources It still isn’t standard May be slow for large amounts of

data

20



Option 2: XML support in your RDBMS Your RDBMS must support XML also for

distributed queries

Optimized

Option 3: XML support in ADO.NET (or other data access library) Not so optimized as RDBMS

Not so flexible for semi-structured data

21



<CustomersOrders> <Customer> <CustID>5</CustID> ... <Order>

...<Units>3</Units>

<Order> </Customer></CustomersOrders>

XML ViewXML Query

<CustomersOrders>{ for $a in (<customers from Prag >) for $b in (<orders with ProductID=5>) where $a/CustID = $b/CustomerID return <customer>

{ $a/CustID, $b} </customer>}</CustomersOrders>

Database XML File

22


Issue 1 – Data Joining (5)Query Optimization:

Proprietary Query

Optimizer finds the best way

SQL Query using JOINs

RDBMS

XML Query Joins

SQL Query Query to Mail Server

CRM Mail

23


Issue 2 – Exploring Data

Solution: Query Builder with proprietary XML-based language and user interface

• Can be mapped to XQuery or SQL (optimization)

<customers> <where>City=“Prag”</where> <fields>

<field>CustomerName</field> </fields> <orders>...</orders></customers>

Your partners/employees don’t know your data structures.

24


Issue 3 - Security• WS Tier – emerging standards (WS-Security)

• Data source security – security rules for each user/group/role (“Distributors can read and update customers from their country.”)

• In future: XACML – allows other systems to inherit security rules from the original system (data source)

25


Issue 4 – Data Structure Changes

• The data structure may change -> use versioning:

<Customers.1>– It can be used for backward

compatibility

26


Issue 5 – Namespaces and Matching• There may be tables or attributes with the

same name – each table takes its namespace to sent XML data:

<Siebel:Customers.1> X <SAP:Customers.1>

• Concordance engine – matching records:Siebel:CustName = ‘John Smith’ X SAP:CustomerName = ‘J. Smith’

27


Issue 6 – Data Modification

• You can send all changed data/data to be deleted back to the server and it will map them to appropriate data sources.

• Microsoft XML Updategrams – proprietary

28


Issue 7 – Process Oriented Services (1)

• Placing order needn’t be only inserting a new record (data tier)

• But: – You have metadata -> you can generate specific

classes for the data source– Developer can then inherit and implement

special functionality (call CRM system API etc.)

29


Metadata

Issue 7 – Process Oriented Services (2)

Data Sources

Data Adapters

Universal Data Access Class

Universal Web Service

Data Storage andRetrieval

Data Tier

Business Tier

Web Services Tier

Code Generation

30


Issue 7 - Process Oriented Services (3)

Option 1: You can convert and automatically redirect the data to appropriate WS

Option 2: You add information about related WS to your metadata so that user knows about them and calls them directly

Booking a ticket – you may want to call Ticket WS that exists concurrently with you Universal Data WS

31


Final Design

32


Final Universal Data WS Interface• SELECTPREDEFINED(QueryID) as XML• SELECT(ProprietaryQuery) as XML• SELECTXQUERY(XQuery) as XML• UPDATE(DataToBeUpdated)• INSERT(DataToBeInserted)• DELETE(DataToBeDeleted)• SEARCH(ProprietaryQuery,SearchExpression

) as XML• GETMETADATA(EntityName) as XML

containing XML Schema, relationships and optionally XACML descriptors for required data

33


Further Development

• The system can be extended with:– Caching capabilities– WSRP/WSIA interface– Provisioning/Auditing system, Undo method– Components for UI – easy implementation to

your client applications– XSL-based system for viewing data and

converting them to other formats (like HTML, standardized XML medical data, etc.).

– XForms interface for editing.

34


Summary

• You have seen:– Concept of Universal Web Service– Proposed Architecture– Issues & solutions– Pros and Cons of the Universal WS

The vision behind: Unified, metadata driven access to all enterprise data and complete view of them anywhere, anytime and from any device or platform.

35


Recommended and used sources:

• www.portsight.com/technology • www.bea.com -> BEA Liquid Data• www.ibm.com -> XPeranto, Information

Integration• www.oasis-open.org -> WS-Security,

XACML, WSRP, WSIA• www.nimble.com -> Nimble Integration Suite• tiberi.us - great intro to XQuery• xqueryservices.com – XQuery .NET library

36


Questions & Answers

37


Thank You For Your Time!

W w w. P o r t S i g h t. c o m 1 Universal Web Service for Providing Enterprise Data Petr PALAS...

Documents

Transcript of W w w. P o r t S i g h t. c o m 1 Universal Web Service for Providing Enterprise Data Petr PALAS...