Grid Portal Development for Sensing Data Retrieval and Processing Diego Arias, Mariana Mendoza,...
-
Upload
cuthbert-mason -
Category
Documents
-
view
216 -
download
1
Transcript of Grid Portal Development for Sensing Data Retrieval and Processing Diego Arias, Mariana Mendoza,...
Grid Portal Development for Sensing Data Retrieval and Processing
Diego Arias, Mariana Mendoza, Fernando Cintron, Kennie Cruz, and Wilson Rivera
Parallel and Distributed Computing LaboratoryUniversity of Puerto Rico at Mayaguez
GCE06-SC06 – Tampa, FLNovember 12-13, 2006
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 2
Agenda
Introduction CASA: Collaborative and Adaptive Sensing of the Atmosphere
– NSF: EEC-031374; UMass, UPRM, OU, CSU
WALSAIP: Wide Area Large Scale Automated Information Processing– NSF: CISE-CNS-0424546; UPRM
Remarks and conclusions
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 3
Introduction: The PDC Laboratory
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 4
Introduction: PDCLab Research
Optimized e-Resource Administration
Automated Composition Signal Processing Operators
Adaptive Provisioning and Orchestration of Grid Services
PDC Portal
WALSAIPPortal
Information Dispersal STB Portal
CenSSIS Portal
Semi-Analytical Inversion Model
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 5
Introduction: PDCLab Grid Testbed
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 6
CentOS 4.2 Globus Toolkit 4.0.1.
– OpenPBS
– Torque
– PosgreSQL
– Apache Ant version 1.6.5
– Java SDK version 1.5
– Jakarta Tomcat version 5.5.9.
Introduction: PDCLab Grid Testbed
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 7
Introduction: PDCLab Grid Testbed
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 8
Agenda
Introduction CASA: Collaborative and Adaptive Sensing of the Atmosphere
– NSF: EEC-031374; UMass, UPRM, OU, CSU
WALSAIP: Wide Area Large Scale Automated Information Processing– NSF: CISE-CNS-0424546; UPRM
Remarks and conclusions
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 9
10,000 ft
tornado
wind
earth surface
snow3.
05 k
m
0 40 80 120 160 200 240RANGE (km)
Horz. Scale: 1” = 50 kmVert. Scale: 1” -=- 2 km
5.4
km
1 km
2 km
4 km
gap
CASA: Collaborative and Adaptive Sensing
gap - earth curvature prevents 72% of the troposphere below 1 km from being observed.
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 10
Academic Institutions/Researchers
Government
Media
Service/Request
Data
Service/RequestData
CASA: Student Testbed (STB)
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 11
CASA: STB Grid Portal Interface
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 12
CASA: STB Grid Portal Interface
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 13
CASA: Information Dispersal Algorithm
IDA - Dispersal Operation
0
100
200
300
400
500
600
700
800
900
1000
1100
1200
1300
1400
1500
0 100 200 300 400 500 600 700 800 900 1000
Data Size (MB)
Ela
ps
ed
Tim
e (
Se
c)
IDA-OLD
8MMA
8MAP
8FRL
16MMA
16MAP
16FRL
IDA - Dispersal Operation
0
100
200
300
400
500
600
700
800
900
1000
0 100 200 300 400 500 600 700 800 900 1000
Data Size (MB)
Ela
ps
ed
Tim
e (
Se
c)
8MMA
8MAP
8FRL
16MMA
16MAP
16FRL
Data File F
6
2
31
7 1112
10
5
9
8
4
IDA (12,4)
Data Retrieval
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 14
CASA: Data Management Portlets
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 15
CASA: Weather Information Portlets
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 16
CASA: Job Submission
Client
GridPortal
Resources
GSI
JOB MANAGER
GateKeeper
Resource Manager
Process
Process
Process
1 2 3 4 5 6 7 8 9 10
98 98 98 97 98 97 9794 93 93
2 2 1 1 1 1 1 0 0 00
10
20
30
40
50
60
70
80
90
100
CP
U U
sag
e (
%)
Netcdf to JPG - CPU Usage
Grid Server
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 17
Agenda
Introduction CASA: Collaborative and Adaptive Sensing of the Atmosphere
– NSF: EEC-031374; UMass, UPRM, OU, CSU
WALSAIP: Wide Area Large Scale Automated Information Processing– NSF: CISE-CNS-0424546; UPRM
Remarks and conclusions
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 18
WALSAIP: Signal Based Information Processing System
Physical Environment
Physical Environment
Observables
Signals
Data
Sensors Effectors
SignalProcessing
Information Processing
InformationKnowledgeProcessing
Knowledge
Decision System
Intelligence
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 19
SAS: Sensor Array Structures
DRS: Data Representation System
CSPS: Computational Signal Processing System
SDP: Signal Data Post-processing
SCS: Signal Conditioning System
RDS: Raw Data Server
CDS: Computed Data Server
IRS: Information Rendering System
DRSSCSSAS
RDS CDS
CSPS SDP
IRS
IRS
IRS
Pre-processing StagePost-processing Stage
Processing Stage
SAS
INTERNET
WALSAIP: Conceptual Framework Design
Net-CentricMIMO
Services
Net-CentricMIMO
Services
DRSSCSSAS
CSPS
SDP
IRS
IRS
IRS
SDP
IRS
IRS
IRS
DRS
SCS
SAS
RDS DBMS
RDS DBMS
CDS DBMS
CIP over a Cyberspace Infrastructure
CSPS
CIP: Computational and Information Processing
Net-CentricMIMO
Services
Net-CentricMIMO
Services
DRSSCSSAS
CSPS
SDP
IRS
IRS
IRS
SDP
IRS
IRS
IRS
DRS
SCS
SAS
RDS DBMS
RDS DBMS
CDS DBMS
CIP over a Cyberspace Infrastructure
CSPS
CIP: Computational and Information Processing
Net-CentricMIMO
Services
Net-CentricMIMO
Services
DRSSCSSAS
CSPS
SDP
IRS
IRS
IRS
SDP
IRS
IRS
IRS
DRS
SCS
SAS
RDS DBMS
RDS DBMS
CDS DBMS
CIP over a Cyberspace Infrastructure
CSPS
CIP: Computational and Information Processing
Net-CentricMIMO
Services
Net-CentricMIMO
Services
DRSSCSSAS
CSPS
SDP
IRS
IRS
IRS
SDP
IRS
IRS
IRS
DRS
SCS
SAS
RDS DBMS
RDS DBMS
CDS DBMS
CIP over a Cyberspace Infrastructure
CSPS
CIP: Computational and Information Processing
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 24
WALSAIP’s Jobos Bay NERR Environmental Surveillance and Monitoring Testbed
@ Jobos Bay NERR @ UPRM
INTERNETINTERNET
RF-Link
MEP-510
MEP-410
RS-232Base Station
MBR410
Local Host
Data Base on Host
Database Querying and Processing
Engine
Sensing-Data
SensorNode1
SensorNode5
Cooked-data
1-Jun022006-21:02:00-humidity-data.txt
1-Jun022006-21:02:00-humidity-meta.txt
Remote User
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 25
WALSAIP: Grid Portal Interface
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 26
WALSAIP: Grid Portal Interface
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 27
WALSAIP: Grid Portal Interface
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 28
Agenda
Introduction CASA: Collaborative and Adaptive Sensing of the Atmosphere
– NSF: EEC-031374; UMass, UPRM, OU, CSU
WALSAIP: Wide Area Large Scale Automated Information Processing– NSF: CISE-CNS-0424546; UPRM
Remarks and conclusions
Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 29
Remarks and Conclusions
Grid portals for large scale applications – Sensing of the Atmosphere (CASA)
– Environmental Surveillance and Monitoring (WALSAIP)
Grid services provided via portlets– Distributed data management
– Information dispersal
– Weather services
– Multiple job submission
– Service orchestration
Future work – Remote instrumentation via grid portals
– Integration to community grid infrastructures (Teragrid, etc….)