A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros...
-
Upload
jason-mclaughlin -
Category
Documents
-
view
217 -
download
2
Transcript of A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros...
![Page 1: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/1.jpg)
Managed by
A Lightweight Platform for Integration of Resource Limited Devices into Pervasive
Grids
Stavros Isaiadis and Vladimir Getov
University of Westminsterhttp://www.coregrid.net
![Page 2: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/2.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 2
• Current Grid Systems• Motivation• Project Overview• Research Issues• Operational Overview• Conclusion
Outline
![Page 3: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/3.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 3
Current Grid Systems
• Grids were initially developed to support large (mainly) scientific data-centric applications
• Resources were in abundance and Grids orchestrated sharing in large scale and/or harnessing of idle resources
• Current Grid systems assume:– Plenty of resources– Relatively reliable and static networks
• Traditionally, Grid systems are not concerned with the platform’s footprint: platform is “heavy” and cannot be deployed in resource limited devices
![Page 4: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/4.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 4
Grids and “Small” Devices
• But networks have evolved. The “edge” of networks increasingly consists of resource limited and/or mobile devices
• The Grid, however, hasn’t yet allowed the integration of these devices
• While access to the Grid and job submission is generally supported (e.g. through portals), contribution of resources is very difficult
• Resource constraints make installation of Grid middleware prohibiting
![Page 5: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/5.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 5
• Current Grid Systems
• Motivation• Project Overview• Research Issues• Operational Overview• Conclusion
Outline
![Page 6: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/6.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 6
But Why Integrate Resource Limited Devices?
• Future Grid systems should be truly pervasive and ubiquitous
• Mobile and pervasive computing attract a big part of current research and industry funds and hold a big share of the market pie
• Small and/or mobile devices increasingly offer functionality not available in traditional Grid nodes –multimedia equipment, sensors, global positioning systems etc.
![Page 7: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/7.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 7
• Current Grid Systems• Motivation
• Project Overview• Research Issues• Operational Overview• Conclusion
Outline
![Page 8: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/8.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 8
A Lightweight Grid Platform
A lightweight platform will provide the foundations for our project:
• Should be based on the CoreGRID component model
• Will consist of a small well defined set of core components
• Extra functionality through pluggable components
• Small footprint and communication overhead so that it won’t drain resources
![Page 9: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/9.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 9
• Current Grid Systems• Motivation• Project Overview
• Research Issues• Operational Overview• Conclusion
Outline
![Page 10: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/10.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 10
Dealing with Device Dynamicity
• Increased dynamicity is not acceptable in complex and heavily loaded Grid systems
• Solution: – Hide devices behind a proxy and delegate
finer control (scheduling, monitoring, recovery etc) to a local community system
– Devices now form a virtual “cluster” and are presented as a single entity to the Grid.
![Page 11: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/11.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 11
Virtual Aggregator Services
• Similar services (that implement the same interface) are aggregated and published through a single interface at the proxy
• The aggregator service implementation orchestrates access to the services it represents (by coordinating the various components like schedulers, indexes etc.)
• A single consistent interface to all aggregated resources is presented regardless of the internal state of the “cluster”
![Page 12: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/12.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 12
Making an Unstable Environment Virtually Stable
• When aggregating services that provide controlled access to raw resources (like cpu cycles, memory, storage etc.), we don’t present the total aggregate but only a fraction
• This way we can mask internal failures by reallocating resources from within the cluster without notifying (and thus placing extra burden on) higher level Grid components
![Page 13: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/13.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 13
Dealing with Failures
• Even behind a cluster, small devices still suffer from a high number of failures and we must provide the best possible quality of service
• Solution:– A monitoring and failure recovery framework
to predict, detect, diagnose and recover from failures
– Forecasting components to support the community scheduling system providing recommendations on the resource usage plans
![Page 14: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/14.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 14
The Monitoring Framework
• A set of small monitoring agents installed in the devices. They collect state information and forward them to the proxy
• A set of monitoring components at the proxy that collect and analyze the information– A monitoring “gateway” makes all decisions
and orchestrates the other components– A forecasting component provides predictions
on the best possible usage plan (according to usage and failure history)
– Failure recovery components decide on the recovery plans based on predefined scenarios
![Page 15: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/15.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 15
• Current Grid Systems• Motivation• Project Overview• Research Issues
• Operational Overview• Conclusion
Outline
![Page 16: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/16.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 16
Component Prototypes
• Currently using Globus Toolkit 3.2 middleware
• Will port to GT4 in the very near future
• Prototyping components using Java
• Prototypes are quite restrictive at this stage but they will:Help us get a better insightHelp us identify possible obstaclesProvide the skeleton for the functional final
platform
![Page 17: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/17.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 17
Operational Overview: Registration of Services
AGGREGATED SERVICE INTERFACES
RESOURCE META-DATA REPOSITORY
Add/Update Interface
Update Index
Discovery and Interface
IdentificationRegistration
LIVE RESOURCE INDEX
![Page 18: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/18.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 18
Operational Overview: Job SubmissionJob Submission
Forward
ConsultApplication Tasks
App/ Code
Job Requirements
Distribution
AGGREGATED SERVICE INTERFACES
SERVICE IMPLEMENTATION
LIVE RESOURCE INDEX
TASK EXTRACTION ENGINE
COMMUNITY SCHEDULING SYSTEM
![Page 19: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/19.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 19
Operational Overview: Failure Recovery
JOB META-DATA REPOSITORY
Event notification + status info
diagnosisforward
update
Event notification + status info
FORECASTING COMPONENT
LIVE RESOURCE INDEX
MONITORING GATEWAY
FAILURE DIAGNOSIS
FAILURE RECOVERY
plan
PLAN DEPLOYMENT
![Page 20: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/20.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 20
• Current Grid Systems• Motivation• Project Overview• Research Issues• Operational Overview
• Conclusion
Outline
![Page 21: A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster .](https://reader030.fdocuments.net/reader030/viewer/2022032604/56649e6b5503460f94b69782/html5/thumbnails/21.jpg)
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 21
Conclusion
• The lightweight platform could provide the foundations for truly ubiquitous Grids
• Coupled with a lightweight monitoring framework, it is a big first step towards full and efficient integration of resource limited devices into the Grid
• The early prototyping phase will help us gain a better insight on the issues that might arise and will provide the skeleton for the final deliverable platform