Authors: Alexey Lastovetsky, Xin Zuo, Peng Zhao Speaker: Xin Zuo

Authors: Authors: Alexey Lastovetsky, Xin Zuo, Peng ZhaoAlexey Lastovetsky, Xin Zuo, Peng Zhao

Speaker: Speaker: Xin ZuoXin Zuo

Heterogeneous Computing Laboratory (HCL)Heterogeneous Computing Laboratory (HCL)

School of Computer Science and InformaticsSchool of Computer Science and Informatics

University College DublinUniversity College Dublin

IrelandIreland

A Non-Intrusive and Incremental Approach to Enabling Direct A Non-Intrusive and Incremental Approach to Enabling Direct Communications in RPC-based Grid Programming SystemsCommunications in RPC-based Grid Programming Systems

HCL Group HCL Group http://hcl.ucd.ie First International Workshop on Workflow systems in e-Science - WSES06


OutlinesOutlines

An Overview of Our Approach

Enabling direct communication in NetSolve

Architecture of the Software Component

Testing Evaluation

Conclusions/Further work

Overview of Our ApproachOverview of Our Approach

About High performance Grid programmingAbout High performance Grid programming

- It has reached a certain level of maturity, examples: NetSolve and Ninf

- Need further development in terms of functionality and quality.

- Addition of a new feature to a Grid programming system:

Traditionally, achieved by changing the code of the system.

- Two serious disadvantages:

1. May introduce bugs which cause system run instable or even crash.

2. To replace the old version, can have very high organizational overhead

and sometimes be simply unrealistic on the Grid.

• The Features of our Software ComponentThe Features of our Software Component:

- Non-intrusiveness

- Increment


Overview of Our ApproachOverview of Our Approach

Non-intrusivenessNon-intrusiveness

The original system does not change and the new features are provided by a supplementary software component working on the top of the system.

• IncrementIncrement

Our supplementary software component does not have to be installed on all computers to enable applications with the new features.

To demonstrate the feasibility of our approachTo demonstrate the feasibility of our approach

- We choose NetSolve

- One particular feature: Direct Communication between remote tasks

- Exact Implementation and Experiments


Enabling direct communication in NetSolveEnabling direct communication in NetSolve

NetSolve is positioned as a programming system for high performance distributed computing on global networks based on GridRPC.

- http://icl.cs.utk.edu/netsolve/

- NetSolve (GridSolve), is a client-server system that enables users to solve complex scientific problems remotely

Unnecessary bridge communications: In NetSolve, output data of remote tasks are typically sent back to the client upon completion of each remote task even if the data are only needed as input for some other remote tasks.



Enabling direct communication



Using Our software component in NetSolve

- Client side: The only thing for client programmers to do is to install wrapper API and Job Name Service on the client side, then compile the client program with the wrapper library. The principle is quite easy: the programmer just replaces the input/output arguments with handlers as the input/output data.

- Procedure developers: The procedure programmers should do nothing to enable direct communications. They develop their own procedures as usual. The supplementary software component has no effect on both existing procedures and newly added procedures.

- Server administrator: To enable direct communication control on server side, the server administrator needs to register the software component as a new problem file to NetSolve.


Architecture of the Software ComponentArchitecture of the Software Component

About software component: Consists of three parts

- Client API & Argument Parser: Provide a uniform interface for the client to make remote procedure calls. Despite the modification on the remote side, the wrapper API allows the calls to be made in the same manner. The only difference is in the arguments that can be not only variables storing real data but also handlers.

- Server Connector: responsible for interacting with clients and other Server Connectors to enable direct communications.

- Job Name Service (JNS): responsible for registration of procedure upon its invocation during RPC call. Other procedures may send requests to the JNS to search for registered procedure. JNS is set up on the client side automatically.

HCL Group http://hcl.ucd.ie HCL Group http://hcl.ucd.ie First International Workshop on Workflow systems in e-Science - WSES06

Architecture of the Software ComponentArchitecture of the Software Component

Architecture of the supplementary software component enabling direct communications in NetSolve


Testing EvaluationTesting Evaluation

Selected computational task:

- Matrix Multiplication.

- We used 8 remote servers to perform 8 matrix multiplications.

- The client, agent and servers all were in the same Ethernet segment.

Experiment environment:

The interconnecting network is based on 100 Mbit Ethernet with a switch enabling parallel communications between computers at School of Computer Science and Informatics in University College Dublin.



(a) (b) * bridge communication x direct communication

(a) Time elapsed for both communication types when all communication links have the same bandwidth, 100Mb per sec. (b) Speedup due to the use of direct

communications for the homogeneous communication network.



(c) (d)■ bridge communication ▲ direct communication

(c) Time elapsed for both communication types when communication client and servers is at the rate of 10 Mb per sec, and communication between servers is

at the rate of 100 Mbit per sec. (d) Speedup due to the use of direct communications for the heterogeneous communication network.


Conclusions/Further workConclusions/Further work

Conclusions:

- we have presented an approach to reducing unnecessary bridge communications in RPC-based Grid programming systems, which is non-intrusive and incremental.

- The experimental results have shown that the performance of Grid applications can be significantly improved by using our supplementary software component.

Further work:

- Apply to different RPC-based Grid programming systems, ex: Ninf.

- Evaluations on real-world applications, we have planned : (i) parallel simulated annealing using genetic crossover in protein tertiary structure prediction system, (ii) image processing using sequential algorithms, and (iii) the matrix chain product problem in general scientific computations.


The ENDThe END


http://www.cs.ucd.ie

http://hcl.ucd.ie

Thank you!

Authors: Alexey Lastovetsky, Xin Zuo, Peng Zhao Speaker: Xin Zuo

Documents

Transcript of Authors: Alexey Lastovetsky, Xin Zuo, Peng Zhao Speaker: Xin Zuo