CS780-3 Lecture Notes In Courtesy of Mengjun Xie
description
Transcript of CS780-3 Lecture Notes In Courtesy of Mengjun Xie
![Page 1: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/1.jpg)
CS780-3 Lecture Notes
In Courtesy of Mengjun Xie
![Page 2: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/2.jpg)
AboutNEESgrid will link earthquake researchers across the U.S. with leading-edge computing resources and research equipment, allowing collaborative teams (including remote participants) to plan, perform, and publish their experiments. NEESgrid, the systems integration component of the NEES project, uses the newest and fastest communications technologies to tie the NEES network together.
![Page 3: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/3.jpg)
NEES Resources
Field Equipment
Laboratory Equipment
Remote Users
Remote Users: (K-12 Faculty and Students)
Instrumented Structures and Sites
Leading Edge Computation
Curated Data Repository
Laboratory Equipment
Global Connections
(FY 2005 – FY 2014)
(Faculty, Students, Practitioners) Simulation
Tools Repository
![Page 4: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/4.jpg)
NEES
Network for Earthquake Engineering Simulation NEES is a distributed array of experimental sites, grid-
based data repositories, tool archives, and computational resources, all seamlessly linked (hopefully!)
NEES has four components: The consortium, which will run NEES after 2004 The consortium development (CD) builds the consortium The experimental sites, which provide data and content The systems integration (SI) effort, termed NEESgrid
Network drivers include telepresence, curated repositories, scalable HPC, experimental-numerical coupling, short- and long-term QoS issues.
![Page 5: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/5.jpg)
NEES Network Stakeholders
Experimental Facilities Shake tables, centrifuges, wave tanks, field
sites
Resource providers Computers, software, storage, networks
End users Researchers, practicing engineers, students, …
Operational facilities NCSA/NEESgrid NEES Consortium in 2004
![Page 6: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/6.jpg)
NEESgrid?
A coordinated and secure architecture/environment A modular and extensible environment with a customizable user interfaceProvides common tools that allow leveraging resources and experiencesGoal: the Cyber Infrastructure that will facilitate this next generation of experimentation/simulation in earthquake engineering
“A Distributed Virtual Laboratory for Advanced Earthquake Experimentation and Simulation.”
-- B. F. Spencer, Jr. (PI & Project Director)
![Page 7: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/7.jpg)
NEESgrid?
Through the NEESgrid, researchers can: perform tele-observation and tele-operation
of experiments; publish to and make use of a curated data
repository using standardized markup; access computational resources and open-
source analytical tools; access collaborative tools for experiment
planning, execution, analysis, and publication.
![Page 8: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/8.jpg)
Telepresence
Telepresence means the capability to participate remotely in experimental trials. There are two main categories:
Tele-observation: the ability to observe the experiment and capture trial data from a remote site
Tele-operation: the ability to interact with the experiment equipment from a remote site.
![Page 9: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/9.jpg)
Collaboration and Visualization
The tasks of Collaboration and Visualization project include:
prototype a Grid-based collaborative environment; integrate support for visualization tools into the
collaborative environment; adapt the Comprehensive Collaborative Framework
(CHEF) for collaborative visualization applications and services; and
produce appropriate final documentation of the collaboration and visualization components of NEESgrid.
CHEF is a flexible web-based environment for remote collaboration.
![Page 10: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/10.jpg)
Main Components
Tele-Control Services and APIs Tele-Observation and Data Visualization E-Notebook Streaming data services DAQ and related services Data and Metadata Services Remote Collaboration and Visualization tools
and services Core Grid Services, deployment efforts,
packaging Simulation Component
![Page 11: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/11.jpg)
Progress
The task of designing and creating this infrastructure has been awarded to the NCSA at UIUC.
The components of the NEESgrid system will be completed by September, 2004, when management and operation of the NEES system will be turned over to a consortium of earthquake engineer researchers and practitioners.
![Page 12: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/12.jpg)
PartnerNational Center for Supercomputing Applications
Randy Butler —Deployment, Operations, and SupportMark Marikos —ManagementJoe Futrelle —Data and Metadata
University of Illinois at Urbana-Champaign
Bill Spencer —ManagementDan Abrams —Community Building
Argonne National Laboratory
Nestor J. Zaluzec —TelepresenceIan Foster —System Configuration
Information Sciences Institute
Carl Kesselman —System Configuration
University of Southern California
Jean-Pierre Bardet —Integrated Demonstrations
School of Information, University of Michigan
Joseph Hardin —CollaborationTom Finholt —User Requirements
![Page 13: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/13.jpg)
System Architecture
![Page 14: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/14.jpg)
System Architecture
![Page 15: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/15.jpg)
System Architecture
![Page 16: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/16.jpg)
System Architecture
![Page 17: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/17.jpg)
System Architecture
![Page 18: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/18.jpg)
System Architecture
![Page 19: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/19.jpg)
System Architecture
![Page 20: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/20.jpg)
Pre-Experiment Collaboration
The remote user (RU) logs into the equipment site's NEESpop. After authentication, RU can access all granted NEESgrid resources through SSN.RU can, via CHEF, read and participate in discussions related to the equipment site, see a calendar, share documents, make announcements, and correspond with other researchers to plan an experiment.Through NEESpop, RU can also access and edit electronic lab notebooks hosted on the TPM.
![Page 21: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/21.jpg)
Monitoring Network Status
RU contacts CHEF and requests a summary of the system's health and status.The CHEF server contacts a central NCSA server that monitors NEESgrid.Standard grid services (GIIS at NCSA, GRIS for other services) are continually collecting information on system connectivity and the availability of grid services such as GridFTP.The grid information is summarized and sent to RU's browser.
![Page 22: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/22.jpg)
Setting Up the Experiment
Using a CHEF teamlet, the equipment specialist at the equipment site enters the metadata describing the configuration of the experimental setup.
The metadata repository on the NEESpop holds this configuration information, making it available for browsing by RUs.
The NEESpop metadata repository also sends configuration information to the DAQ system(s).
![Page 23: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/23.jpg)
Subscribing to Trial Data Streams
RU contacts CHEF to browse available NEESgrid resources and discover what data will be available for streaming.A CHEF teamlet uses GIIS and GRIS to query and report features and availability of NEESgrid resources, as well as details of NEES equipment and available data streams.RU contacts the NEESpop and requests a subscription to several data streams.The NEESpop negotiates with the NSDS to broadcast the selected data streams.
![Page 24: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/24.jpg)
Streaming Near-Real-Time Data
Instrument information is sent in real time from the DAQ system through a NSDS daemon to the NSDS.
Data is streamed by NSDS to the RU. An applet in the RU's browser displays the results.
Video services are managed through the TPM server and streamed to the RU via a separate video streaming server.
![Page 25: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/25.jpg)
Ending the Trial
The DAQ equipment moves the trial data to a data repository hosted on the NEESpop.
The data is associated with appropriate metadata information, such as experiment ID, channel ID, and sensor type.
![Page 26: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/26.jpg)
Analyzing the Trial Data
The trial data can now be browsed via CHEF.
The trial data and metadata can be downloaded to the RU via GridFTP service.
The RU can then use either standard software, such as MatLAB, or their own customized software to analyze the trial data.
![Page 27: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/27.jpg)
Login Authentication
![Page 28: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/28.jpg)
ChatCHEF Environment
![Page 29: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/29.jpg)
E-NotebookCHEF Environment
![Page 30: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/30.jpg)
Data ViewerCHEF Environment
![Page 31: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/31.jpg)
NTCP DataCHEF Environment
![Page 32: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/32.jpg)
NEESgrid Central RepositoryCHEF Environment
![Page 33: CS780-3 Lecture Notes In Courtesy of Mengjun Xie](https://reader035.fdocuments.net/reader035/viewer/2022062305/568148b5550346895db5cce4/html5/thumbnails/33.jpg)
Summary
Characterizations of NEES project: A very specific application A wide range of resource sharing:
Raw data, experimental results, equipment, computation/software tools
Centralized controls, monitoring, and management.
Does this follow the three principles of Grid? Not really: it is subject to central control, not open source
software based.
But it is a successful model of grid applications.A Global and general grid is an ideal model, and may become realistic after many successful NEES projects.