EScience and Particle Physics Roger Barlow eScience showcase May 1 st 2007.
Session: Application Deployment on Grids/eScience
description
Transcript of Session: Application Deployment on Grids/eScience
Session: Application Deployment on Session: Application Deployment on
Grids/eScienceGrids/eScience
The 28The 28thth Open Grid Forum Open Grid Forum
München, GermanyMünchen, Germany
| 2
Agenda
The presentation of “Application Deployment on Grids/eScience” from Lanzhou University. (25min.)
Question and Answer. (10min.)
Discussing with the topic of “How to make the developments and the Operations on Grids/eScience easier?” (20min.)
Dr. YI YANGDr. YI YANG
Lanzhou University, ChinaLanzhou University, China
[email protected]@lzu.edu.cn
Application Deployment on Application Deployment on
Grids/eScienceGrids/eScience
| 4
Outlines
Background
Our team and Project
Application Deployment Interface Framework (ADIF)
Current work
Further work
Publications
| 5
Background
Grids technologies are developed sophisticatedTo be the infrastructure of eScience Too many Grids Different countries
Globus, U.S.AEGEE, EUCNGrid, ChinaCGSP, China……
Does this met the objectives of the initial Grid?
Different purposes
Bioinformatic
Computer Chemistry
Environment
Physics
……
Different specifications
OGSI
Webservices
WSRF
Different imple-mentation of WSRF
……
| 6
Background
Objectives of Grid The usage is as convenient as the electricity to a
general customer. Standard, open, general-purpose protocols and
interfaces. A hardware and software integration. Sharing the resources on the internet. Co-operations among distributed areas and
organizations.
?
?
√√ !
?
| 7
Background
Current Grid Integration the software and hardware. Sharing the resources as much as more. To be as the infrastructures of many applications. BUT
The sharing resources is flat organized.
The usage of applications is always for special purposes.
The usage of applications is difficult for a general customer.
Why?
| 8
Background
Current development mode of Grid Applications Developed by the developers of Grid. Developers must familiar with both the
specifications of Grid and the domainknowledge of special purposes.
The development and maintenance is hard.
Applications tightly coupled with itsrelated Grid.
The leverage between applications andGrid is not clear.
The migration of applications to other gridsseems unable.
Applications
Gird
| 9
background
Idea The “abstraction” is always an important methodology to improve
computer science.
Why we don’t abstract the grid application develop interface (ADVI) to higher levels so that it can be used to develop applications by a domain user or specialist.
COMPUTERabstract Assembly
Language
abstractC/C++/Java
GridArchitecture
abstractADVI
abstract??
| 10
Background
GridApplications
Grid platform
DeployMiddleware
DeployMiddleware
DeployMiddleware
The “deployment” of grid applications on Grid Abstract the ADVI Separate the development of
Grid and Grid applications Make them loosely coupled Application Deployment Interface
Framework (ADIF)
| 11
Our Team
A single research group, which are composed of three departments of two Universities. ICCM and SC Lab in Lanzhou University GCD in Tsinghua University
Group Leader : Prof. Li Lian, [email protected]
Team Member : Prof. Ruishen Zhang, 2 lectors, 3 Ph.Ds, 8 Masters.
| 12
Project
Basic Research on Grid Specifications Supported by National Natural Science Foundation of China. 12 teams in total who are in charge of different parts of the whole
specifications respectively It is hoped to become the National Grid Specifications in
China after finished Started from Jan. 2008 and will end in Jun. 2010. Funding is 3 million chinese yuan
| 13
| 14
Application Deployment Interface Framework (ADIF)
Abstract the Application Deployment interface from Application Development interface
It is XML documents, and according to BNF.
Developers just deed to generate these documents by web wizards.
Reference from WSRF 、 WSDL 、 JSDL etc.
| 15
The Conception of Application Deploy
Grid platform and grid application can be developed independently.
Here, the basic architecture is the grid platform, and applications in different areas will be built on this basic architecture through the standard, uniform interfaces, which make their surface minimal, seeming like a ‘sandglass’.
| 16
The Advantages of Application Deploy
The grid middleware will be the core of gird applications and the shell of the grid middleware
The grid application will provide an easier and more custom user experiences
The standardizing of grid platform and grid application will be promoted rapidly.
It enhances the scalability and extensibility of grid applications
It also provides the possibility of deploying several applications on alternative grid platforms, especially the heterogeneous grid platforms.
| 17
Current Work
ADIF v1.0 A general specification Four sub specifications
Three levels Grid application level Grid application
deployment level Grid architecture level
Four interfaces Resource management Workflow management Logging management Notification managements
| 18
The general specification V1.0
A general introduction to ADIF
Terminologies and Definitions
Introduction about four sub specifications and their relationship.
| 19
Application Deployment Resource Management V1.0
The grid is to integrate the Internet into a giant super-computer to achieve the comprehensive sharing.
Unfortunately, collections of heterogeneously and geographically distributed resources in grid system are assembled loosely and disorderly.
Only limited resources are useful for individual users or communities.
It is intractable to hunt for the proper resources from such huge system according to user’s requirements.
The specification for discovering intended resources, from a common pool collecting all kinds of resources, becomes more crucial for many users.
| 20
Application Deployment Resource Management V1.0
At present, the Resource Management includes: Resource Deploy Here, resources deploying is allowed, which makes
individuation requirements customizing possible. Actually, only a document, called deployList, in XML format, is necessary.
Resource Undeploy Under specified circumstances, some resources in User Resource List MAY become obsolete. In order to make sure the practicability of the User Resource List, users are authorized to give these resources away absolutely according to their wishes.
Resource Destroy Usually, some resources should be discarded from the User Resource List when they have become unavailable. For security, under this operation, the unavailable resources are merely deleted from user logic view, and never destroyed in the grid system.
| 21
Application Deployment Resource Management V1.0
Grid Resources RDD User Resourcest
Example of Resource Deploy
| 22
Application Deployment Workflow Management V1.0
In order to support complex research experiments, distributed resources need to be orchestrated while the application workflow operations being managed under Grid environments.
The grid workflow management system thus should provide sets of tools to facilitate building grid application services by orchestrating web services.
From grid perspective, workflow is a coarsely granular unit which is supervised and controlled by the job management system of grid. The focus of workflow is mainly what to do and how to do.
At present, the user not only regards the logic of the whole processes, but also involves different requirements towards heterogeneous environments to submit their computational jobs.
| 23
Application Deployment Workflow Management V1.0
Therefore, a set of standard interfaces is proposed as following: Definition for the process control structure; Definition for the workflow designer, which addresses some conventions
on the structure of the designer; The Requirement Document Definition, which provides an XML
grammar document to implement the users’ requirements about computational jobs, in order to identify an environment to support reliable executions towards workflow.
| 24
Application Deployment Workflow Management V1.0
| 25
Application Deployment Logging Management V1.0
Logging management plays an important part in the grid platform. The logging can absolutely record all the information of each operation in the grid platform, which can provide all the necessary information for any possible analyzing of the grid administrator.From the perspective of users, the information, which should be acquainted, is not only the state information of grid platform, but also the record of the information of personal behaviors in the grid platform. So it is important to manage and maintain the records of the user’s behaviors in the grid system. However, for each user, what they concern about just is their own relevant information among the grid platform. So the interfaces to extend and shield the logging of grid SHOULD be provided.
| 26
Application Deployment Logging Management V1.0
Therefore, in this document the map relationship of logging, between the application system and the grid, is defined. What’s more, the flexible logging definition is provided, and the personnel operation for the logging model is supported. the logging for Job Result, the logging for submitting the Atomic Job, the logging for submitting the Workflow Job, the logging for packing the Application for gird platform, the logging for Destroying Resource in User Resource List, the logging for user Login in platform the logging for user Logout from grid platform
| 27
Application Deployment Logging Management V1.0
Example of logging deployment
User logging Mng.RDD
User logging
| 28
Application Deployment Notification Mechanism V1.0
There are multitudinous resources in Grid System, but the resources, meeting users’ requirements, MAY be only a small part.
So it turns necessary that the resources COULD be subscribed according to users’ requirements.
However, the resources among the Grid System are changing frequently and dynamically
It turns more urgent to provide an interaction pattern between the user and the grid, which is used to inform the users the current statuses of the grid resource.
| 29
Application Deployment Notification Mechanism V1.0
| 30
Current Results
ADIF documents V1.0 ADIF v1.0 ADIF-WorkflowManagement v1.0 ADIF-ResourceManagement v1.0 ADIF-LoggingManagement v1.0 ADIF-NotificationMechanism v1.0
A reference implementation on CGSP ( China Grid Support Platform)
| 31
Future Works
Improve the ADIF
Make the ADIF more implacable
Deploy eSoc on CGSP using ADIF
Interoperation between CGSP and CNGrid using ADIF
| 32
Publications
Jiazao Lin, Wanbing Shi, Yi Yang , Lian Li , A novel model for grid resources management based on user customization, ChinaGrid Annual Conference, 2008-9-12 [EI检索 ].
Wanbing Shi, Yi Yang, Chunyan Zhang, Lian Li , A Study on ADIF Based on e-Science Deployment Architecture, GCC2007 IEEE Computer Society,2007.8 [EI检索 ].
Lian Li, Ruishen Zhang, Yongwei Wu, Yi Yang, deploy eSOC on CGSP, Tech. Report. Lanzhou University.
Jiazao Lin, Zhili Zhao, Shoubo Li, Huajian Zhang, Lifen Li, Lian Li, Caihong Li , Application-objected Workflow Management System based on Abstract Service , The 8th international conference on grid and cooperative computing (GCC2009) , 2009 , 8(已发表)Jiazao Lin, Zhili Zhao, Lei Liu, Huarong Sun, Shoubo Li, Caihong Li, Li Liu, Lian Li, MDMF: A Comprehensive Framework for Managing Large-scale Heterogeneous Data in eSoC Collaborative Environment , SWS2009, 2009,pp:138-143
Chenyang Zhao, Shoubo Li, Zhiwei Liu, Li Liu, Lian Li. A Publish/Subscribe System based on Distributed Broker Management in e-Science environment. SWS2009, 2009, pp:20-25
Shoubo Li, Jiazao Lin, Chenyang Zhao, Zhili Zhao, Li Liu, Lian Li. GCP:A Load-balancing Tool in e-Science Environment. SWS2009, 2009. pp:90-95.
| 33
“Although we can see the limit steps we have made, but we still have a lot of works to do.”
Thank YouQuestions & Answers
| 35
How to make the developments and the Operations on Grids/eScience easier?
Is the idea of ADIF correct?
Does the ADIF useful?
IF it does, How to improve? Suggestions and Advices are welcome
Any Other sub specification?
……
Thank You