© 2006 Open Grid Forum Storage Networking Community Group.

© 2006 Open Grid Forum

Storage Networking Community Group

2© 2006 Open Grid Forum

OGF IPR Policies Apply

• “I acknowledge that participation in this meeting is subject to the OGF Intellectual Property Policy.”• Intellectual Property Notices Note Well: All statements related to the activities of the OGF and addressed to

the OGF are subject to all provisions of Appendix B of GFD-C.1, which grants to the OGF and its participants certain licenses and rights in such statements. Such statements include verbal statements in OGF meetings, as well as written and electronic communications made at any time or place, which are addressed to:

• the OGF plenary session, • any OGF working group or portion thereof, • the OGF Board of Directors, the GFSG, or any member thereof on behalf of the OGF, • the ADCOM, or any member thereof on behalf of the ADCOM, • any OGF mailing list, including any group list, or any other list functioning under OGF auspices, • the OGF Editor or the document authoring and review process

• Statements made outside of a OGF meeting, mailing list or other function, that are clearly not intended to be input to an OGF activity, group or function, are not subject to these provisions.

• Excerpt from Appendix B of GFD-C.1: ”Where the OGF knows of rights, or claimed rights, the OGF secretariat shall attempt to obtain from the claimant of such rights, a written assurance that upon approval by the GFSG of the relevant OGF document(s), any party will be able to obtain the right to implement, use and distribute the technology or works when implementing, using or distributing technology based upon the specific specification(s) under openly specified, reasonable, non-discriminatory terms. The working group or research group proposing the use of the technology with respect to which the proprietary rights are claimed may assist the OGF secretariat in this effort. The results of this procedure shall not affect advancement of document, except that the GFSG may defer approval where a delay may facilitate the obtaining of such assurances. The results will, however, be recorded by the OGF Secretariat, and made available. The GFSG may also direct that a summary of the results be included in any GFD published containing the specification.”

• OGF Intellectual Property Policies are adapted from the IETF Intellectual Property Policies that support the Internet Standards Process.


Agenda topics

• Overview of SN-CG Activities• Presentation and discussion about

Storage GME project• SNIA Grid Taskforce Update


SN-CG Progress Report

• Development of Storage GME project• Basic project objectives defined

• Enterprise Grid Panel Organization• Follow-up on initial SNW-Orlando event

• Coordination with other Data Area projects• Requires further engagement with Data Area groups

• Collaboration with SNIA Grid Taskforce• “Storage Grid” terms definition• SGME work• Cross-education


Storage Grid Management Entity


Storage GME

• Definition of Basic Storage GME• Based on Grid Management Entity concept

developed by EGA• See EGA Reference Model

• Applied to data storage elements in Grid• Based on simple use cases (TBD)

• Work scope would include• S-GME Specification development• S-GME prototyping

• In full collaboration with SNIA GTf


Storage GME Project Initial Objectives

• Proof of concept• Initial implementation experience• Guide for further work

• Interoperable management layer• Simple user-case scenarios


S-GME Longer Term Objectives

• Typical Data Center Requirements:• Discovery• Asset Management• Replica Management• Security• Scripting• Location Transparency• Configuration Management


Storage GME Project – Selected Ingredients

• Grid Execution Engine • Condor

• Open Source• Interesting Stork project (data provisioning)

• Application:• Video Rendering Application• Scenarios to supply storage resource to rendering jobs

• S-GME Code• TBD


1. Simple case•Discover storage and available capacities•Provision and setup xTB of scratch/temporary capacity for given task per compute node•Provide location (exported mountpoint)•scheduled task runs to completion•De-provision capacity

2. Additional capability•Apply a performance SLA•Monitoring and rating of storage performance•Rank performance of each storage container used•Report performance•Decision: do not use slow container for next task

3. Advance capability•Determine that the data stored in temporary area is important (auditing, compliance, etc.) through an additional SLA•Retain data after or during task completion via,

•Remote copy•Cdp•Backup•Vtl•snap•Etc.

S-GME Project Scenario


2. Simple Use CaseProvision temporary storage

/pop

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

Storage GME


Storage GME

2. Additional CapabilityPerformance monitoring and reporting

/pop

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt


Storage GME

3. Advanced CapabilityReplicate data for retention purposes

/pop

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/pop

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/home

/coke /pepsi

/a /soda.txt

/coke /pepsi

/a /soda.txt


Update from SNIA Grid Taskforce

Ken Wood


Context

• Understanding how decoupling applications from the compute platform – Grid Computing• Location transparency• Computing is the execution platform of

applications

• Decoupling data from the storage platform• Location transparency• Storage is the containers of data

Sto

rag

e G

rid

s, D

ece

mb

er

20

06

© 2

00

6 S

tora

ge N

etw

ork

ing

Ind

ust

ry A

ssoci

ati

on. A

ll R

ights

Rese

rved

.


Network

Virtualization

Information

Enterprise GME

Network

N-GME

Compute Virtualization

Computing

Application

Storage Virtualization

Storage

Data

Storage GMECompute GME

D-GME

A-GME

GME: Grid Management Entity, as defined in EGA Reference Model v1.0

Context

Sto

rag

e G

rid

s, D

ece

mb

er

20

06

© 2

00

6 S

tora

ge N

etw

ork

ing

Ind

ust

ry A

ssoci

ati

on. A

ll R

ights

Rese

rved

.


Terms

• Storage for Grids• Storage which meets the needs of Grid Computing as it may not

have any internal grid architecture. SNIA should recommend that the term "Grid Storage" not be used to avoid confusion.

• Grid-based Storage• is a class or category of storage solutions that have leveraged

technologies from grid computing to create a grid-like internal architecture (horizontally scalable) to provide a storage capability. This type of storage may not expose its internal Grid-oriented capabilities to external management platforms for configuration and management, but is typically done in a proprietary fashion.

• Grid-managed Storage• is a storage architecture that both has an internal grid architecture

(horizontally scalable) and provides interfaces to this internal architecture in an open/standardized fashion such as to allow (Grid) management facilities to configure, provision, and otherwise define a needed storage service as required by the requesting entity.

Sto

rag

e G

rid

s, D

ece

mb

er

20

06

© 2

00

6 S

tora

ge N

etw

ork

ing

Ind

ust

ry A

ssoci

ati

on. A

ll R

ights

Rese

rved

.


Terms

• An enterprise grid is a collection of interconnected (networked) grid components under the control of a grid management entity

• The Grid Management Entity (GME) is that logical entity that manages –

• The grid components.• The relationships between various grid components.• Their life cycles.

• It is responsible for ensuring that the various grid components meet their goals, i.e. that the high level services and applications (eg ERP, BI, CRM etc.), deployed on the enterprise grid, meet their service level objectives within their operational constraints.

Source: EGA Reference Model V1.0

Sto

rag

e G

rid

s, D

ece

mb

er

20

06

© 2

00

6 S

tora

ge N

etw

ork

ing

Ind

ust

ry A

ssoci

ati

on. A

ll R

ights

Rese

rved

.


Terms

• Storage GME, S-GME• is an instantiation of a GME specifically for

managing storage and works in concert with the Enterprise or Global GME which manages all federated instantiations in a Grid, i.e. Compute GME, Network GME…

Sto

rag

e G

rid

s, D

ece

mb

er

20

06

© 2

00

6 S

tora

ge N

etw

ork

ing

Ind

ust

ry A

ssoci

ati

on. A

ll R

ights

Rese

rved

.


• Storage for Grid• Includes storage in many forms such

as DAS, SAN, NAS (Gateways and Filers), file servers, ftp servers and other storage servers

• Managed locally•

Examples

Non-Grid Consumer

Grid Computing

Sto

rag

e G

rid

s, D

ece

mb

er

20

06

© 2

00

6 S

tora

ge N

etw

ork

ing

Ind

ust

ry A

ssoci

ati

on. A

ll R

ights

Rese

rved

.


Examples

• Grid-based Storage• Grid Technologies, concepts, and architectures relating to

storage• Redundant Array of Independent Nodes• Clustered file systems• Distributed/Parallel file systems• File/Namespace virtualization• Storage virtualization• File Area Networks• Others?

RAIN

Clustered file systems

Distributed/parallel file systems

StorageVirtualization

File/NamespaceVirtualization

Sto

rag

e G

rid

s, D

ece

mb

er

20

06

© 2

00

6 S

tora

ge N

etw

ork

ing

Ind

ust

ry A

ssoci

ati

on. A

ll R

ights

Rese

rved

.


Examples

• Grid-managed Storage

InterfaceInterface

InterfaceInterface

InterfaceInterface

InterfaceInterface

InterfaceInterface

Storage GMEStorage GME

Enterprise or GlobalEnterprise or GlobalGMEGME

Sto

rag

e G

rid

s, D

ece

mb

er

20

06

© 2

00

6 S

tora

ge N

etw

ork

ing

Ind

ust

ry A

ssoci

ati

on. A

ll R

ights

Rese

rved

.


Examples

• Grid-managed Storage - Reference Model Proposal

Storage GridStorage Grid

SCSICIFSNFSHTTPFTP

GridFTP

WebServices

Enterprise Management

System


System

Grid ServicesGrid Services

Virtual ObjectsFile systems

LUNs / Blocks

Application or DB

level objects

XML documents

Object Storage

Blobs

Service LevelsAvailability

Performance

Replication

Information MgmtSecurity

ILM


Inter-node high-bandwidth network

Intra-node interconnect fabric

Grid Management(Single System Image)

Service Oriented Interfaces / Operations

PhysicalLayerPhysicalLayer

Control LayerControl Layer

Policies, Intelligence(Configuration, QoS,

Data Consistency, etc.)

Storage Grid Node

Storage Grid Node

Storage Grid Node

Gri

d M

etad

ata

Rep

osi

tory

Storage GridStorage GridStorage GridStorage Grid

SCSICIFSNFSHTTPFTP

GridFTP

WebServices


System


System


System


System



LUNs / Blocks

Application or DB

level objects

XML documents

Object Storage

Blobs


Performance

Replication


ILM

Grid ServicesGrid ServicesGrid ServicesGrid ServicesGrid ServicesGrid Services


LUNs / Blocks

Application or DB

level objects

XML documents

Object Storage

Blobs


Performance

Replication


ILM


Inter-node high-bandwidth network

Intra-node interconnect fabric

Grid Management(Single System Image)

Service Oriented Interfaces / Operations

PhysicalLayerPhysicalLayer

Control LayerControl Layer

Policies, Intelligence(Configuration, QoS,

Data Consistency, etc.)

Storage Grid Node

Storage Grid Node

Storage Grid Node

Gri

d M

etad

ata

Rep

osi

tory

Enterprise Enterprise GMEGME

Sto

rag

e G

rid

s, D

ece

mb

er

20

06

© 2

00

6 S

tora

ge N

etw

ork

ing

Ind

ust

ry A

ssoci

ati

on. A

ll R

ights

Rese

rved

.


SN-CG Next Steps


Next Steps

• Continuation of S-GME Project• Development Framework (IP Friendly)

• Provide SMI-S Education (OGF20)

• Organize OGSA/Grid education for SNIA

• Coordinate activities with other Data Area groups• Feedback on S-GME project / links with other

OGF (Data) Projects• GFS, GSM, Data WG…


Thank You

Don’t forget to register on [email protected] !


Full Copyright Notice

Copyright (C) Open Grid Forum (applicable years). All Rights Reserved.

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works.

The limited permissions granted above are perpetual and will not be revoked by the OGF or its successors or assignees.

© 2006 Open Grid Forum Storage Networking Community Group.

Documents

Transcript of © 2006 Open Grid Forum Storage Networking Community Group.