CNGrid Software ProgressCNGrid Software Progress

Zhiwei Xu zxu@ict.ac.cn www.cngrid.org

Institute of Computing Technology Software TeamChinese Academy of Sciences China National Grid

EU Grid At Asia WorkshopJune 23, 2005, Beijing

Contents

• CNGrid Software Objectives

• Approach and Roadmap

• Status Software Development and Applications Research Focus and Techniques

• Ideas for EU cooperation

CNGrid Software Objectives

• Support applications in four areas

• Connect distributed resources into single system images: eliminate silos

• Mask resource heterogeneity and distribution

• Automate common requirements

• Reduce lifecycle cost of distributed applications, thus enabling sharing and cooperation

CNGrid Software

Distributed Resources and Services

App Scope of CNGrid Software

ScienceResearch

Manufacturing Resources and Environment

Services Sector

Connectivity, Transparency, Automation

SPARC

Oracle

Solaris

IA-32

MySql

Linux Power4

WebSphere

AIX

VLIW

GIS

HP-UXAMD

SQL

Windows

MatLabPDESolver

Simulator Analyzer

Data MinerSingle System Image Single System Image

Application Grids

CNGrid Software provides automated common supports

WebBrowser

C/SClient

OtherClient

GOS API and Utilities

Vega GOS Constructs and Services

▪ Resource Info ▪ Resource Mgmt ▪ Jobs ▪ User ▪ Monitoring ▪ Accounting ▪ Data

GOS Kernel （ Apache, OMII, GT4, ）

Effective Resources

Virtual Resources

HPC Storage Database Software Files

PhysicalResources

GT Services ， Web Services

CNGrid Software

OGSAPlatform

Layer

Grid PortalWeb Style

Grid PortalC/S Style

OtherStyle

Grid

Security

Grid PortalGriDaEnGrishield

Vega GOS

GOS Constructs

GR

GSML PageClient

Effective

Virtual

Physical

Internet

GSML PageGSML Page

Grip1 Grip2 Grip3 Grip4

Grid Operating System（ GOS Kernel, Core, Libraries, Utilities）

Beijing Node

GSGR

Shanghai Node

GSGR

Xi’an Node

GS

Server PhysicalResource

GR Grid Router GS Grid Switch

Agora 1 Agora 2

MapperComposer

Composing Mapping

DawningDagger

EffectiveResource

VirtualResource

Grishield: CNGrid Security• End-to-End

From user log-on to physical resource execution Details are hidden from user/developer

• Based on WS-Security Cert based authentication; Token based authorization & AC; signature

Web uCert

Portal/Server

uid/pass

Grip Container

Agora pCertpCertpCert

Phy Svc Phy SVC Phy SVC Phy SVC

pCertuTK

pCertuTK

pCertuTK

pCertuTK

User Res AA

uTKuTKuTK

Other Client

pCert

uCert

GridDaEn: Grid Data Engine System level service of GOS developed by NUDT Provide uniform data operations over global namespace

Browser

Grid Portal Engine

Grip Container

Agora Service

DRBService

Grid Application

DRBService

DRBService

DRBService

Grid Portal

GridDaEn: Grid Data Engine

• Global logical view  Utilize a uniform three-level naming scheme that shields users fr

om low-level heterogeneous storage resources Provide global logical view of data resources in multiple domain

s for users• Uniform access 

Provide a set of uniform APIs and SDKs to access and manage geographically distributed data resources.    

• Federated services  A distributed structure: distributed DRB (Data Request Broker) a

nd distributed MDIS (Metadata Information Server) Several DRBs combined to provide federated services Distributed data replication and caching

Grid Data Engine

Vega GOS

Agora Service Engine

Java J2SE, J2EE/Microsoft Windows

GOS Hosting Environments

GOS Core Services

Service BUS

User Mgmt. EngineAgora

AccountVerification

Global UserAuthentication

Resource Mgmt. EngineService AddressTrans.

PT

Policy Engine

Resource Authority

Grip Engine

Grip Container

Grip Lifecycle Mgmt.

Core Libraries (Core API)

GOS System Services

File Services Database ServicesData BUS

Certificate/CA Mgmt.

System Mgmt. Services

Metadata Services

Information Integrator

Inforamtion BUS

TransformationStabilizer

Data Integrator

Computing Services

DDS

Batch Services

Legacy Resource

IntegratorMonitor

Logging Audit

Config

System Libraries (System API)

Portal Engine GSML Interpreter Abacus Compiler Workflow Engine

Grid PortalGSML

BrowserAbacus Studio

Vinca Workshop

Build-in Utilities

...

Tomcat(Apache)

WebSphere(IBM)

WebLogic(BEA)

.NET(Microsoft)

GT4(Globus)

User Def.Apps.

Service Router Service Registry Service Adapter

GOS Applications

ExtendedSystem

Services

Vega GOS and OGSA V1.0

• Vega is an implementation of (part of) OGSA• Vega would like to contribute to OGSA

After implementation and testing (running codes) Loose coupling Partner with other groups Focus on 4 key issues and aim at minimal common requirements

• Naming, Process/States, VO, Programming

• Vega complements existing grid projects Focus on implementation architecture, not protocols/services

• Use computer systems approach, not middleware or network Utilize existing software

• At Vega GOS kernel level– Apache; OMII, GT4; Commercial

• As services• At Vega GOS application level

Naming in OGSA and Vega GOS

• Vega matches OGSA 3-level naming convention OGSA Human-Oriented Abstract

Address Vega (EVP)Effective Virtual Physical

• As the primary way for virtualization• OGSA Naming specification must include

Precise definitions and axioms Syntax and semantics (rough consensus)

• Who provides, uses, and maintains such names• Scoping and name/address space• Lifecycle and exception handling• Mapping, resolution, binding• Provision for resources

PT(V1→E1)

Layered Resource Naming And Mapping

PRes1 P2 P3

http://hostname:port/suffixes

Router1

P4

Router2

Service Container A Service Container B

V2 V3 V4

ERes1 E2 E3

PT(V2→E1) PT(V3→E2)PT(V2→E2) PT(V4→E3)

Agora1

VRes1

vres://router_name:res_v_name

Agora2

eres://an:reneres://agora_name:res_e_name

Effective resource

Virtual resource

Physical resource

Top

Lay

er(A

gora

)O

verl

aps

Bot

tom

L

ayer

(Rou

ter)

VO in OGSA and Vega GOS

• There is no precise definition of VO in OGSA

• Agora is a concrete example of VO (community)

• Agora has a precise definition, and it holds Subjects, objects, context/policies information Agora-related system services

• Agora is persistent and “static”

• Application programmer knows the agora concept, but agora does not appear in app codes

Inner Structure of Agora

Tomcat+AxisAgora Access Control Mechanism

Authorization Engine

Resource Mgmt. ClientUser Mgmt. Client

UserLogin

ResourceAuthorization

ResourceMgmt.

Interface

UserMgmt.

Interface

Resource Mgmt. Service User Mgmt. Service

RoleProxyUserName profileERes MappingVRes PT

Tomcat+Axis Tomcat+Axis Tomcat+Axis

AAA Client

AuthorizationAuthorityService

AC PolicyMgmt.

ResourceSelection

Process/States in OGSA and Vega GOS

• There is no process concept in OGSA 1.0• Grip is distributed process in grids environment

A runtime construct representing a subject (a grid user running a grid application) to access and utilize objects (grid resources and services)

• Classification of “states” Session related Application logic specific Grid system related Resource related Service specific

Grip

Physical Service

Grip

createAgora Service

grip

uid/pass

Proxy, Profile

bindERes name

VRes name, Token, PT

invoke

Physical Service Physical Service

getResult

grip

grip

grip

Network of Resource Routers

authentication

•resource selection•resource authorization

resource locating

service invocation

•return•cache

close

•uCert,Profile

•uCert Profile

VResToken

PT

•uCert Profile

VResToken

PT PResRet

•uCert Profile

VResToken

PTPRes

Core and Kernel

Put It Together

Web

GripUser, App Logic

Address Space, States

AgoraPolicies:

Security and Selection

Phy Svc Phy SVC Phy SVC Phy SVC

Other Client

System ServicesResource Services

UI and Utility Tools

Common Supports

not per-service or per-application codes

Follow the E2E and KISS principles Loose coupling; Hide details, reduce coding; Try to minimize abstractions

4 abstractions: User, (Effective) Service, Grip, Agora5 API “functions”

GSML : Grid Service Markup Language

Grid User A

Grid User B

Main Constructs of the language: Pipes are software components consuming various resources (include services). At runtime, pipes are independent, concurrent, event-driven processes (or threads).The only way for interacting with pipes is sending events to or intercepting events from them.

•A new programming language for end users

•XML-based, descriptive rather than imperative

•Event-driven model

•Component-based design

•Focus on interaction

GSML Software Suite: A WYSIWYG Composer

Edit Area

EventProperties

ResourceRepositories

Resource Information Monitor

E-learning Collaboration

GSML: Demo Applications

Digital library

GSML:A Simple Example

<head><title>untitled</title></head><body><row><cell>

<pipe id="pService" type="WSInvoker"> <eventset> <event source="System" type="pageload"/> <target destination=“pService" type="WSInvoke">

<para> <name>wsdlLocation</name> <value><![CDATA[http://www.webservicex.com/stockquote.asmx?WSDL]]></value> </para> <para> <name>portName</name> <value>StockQuoteSoap</value> </para> ….

</target></eventset></pipe>

</cell></row></body>

Aviation and Space Simulation Computing

Biological Computing - Genome Sequence Tracing

Geological Computing - Underground Water Evaluation

CNGrid Software Roadmap in 2005

• 2004.11 2.0 preview Sample Apps

• 2005.2 2.0 alpha

• 2005.4 2.0 beta CNGrid Apps

• 2005.6 2.0 CNGrid Deploy

• 2005.7- 2.0 on OMII and GT4

• 2005.11.30-12.3 CI6016 & GCC 2005 Exhibit www.ict.ac.cn/ci6016

Suggestions for EU Cooperation

• Infrastructure Projects Connect China National Grid to EU grids

• CNGrid Software to connect resources and applications

• Research Projects Net-centric OS

• Architecture

• Key OS abstractions and constructs (Naming/Virtualization, VO, Grip)

• Exception handling

• Optimization Programming Environment

• Language and tools

• Debugging