Post on 03-Feb-2022
Kincho H. LawStanford University
(NEESgrid Data/MetaData Efforts and Issues)
Kincho H. Law
Professor of Civil and Environmental Engineering
(Structural Engineering and Engineering Informatics)
Stanford University
February 28,2005
NSF EXCITED Workshop
(Exchanging CyberInfrastructure Themes in Engineering Design)
NEESgrid – A CyberInfrastructure for Earthquake Engineering Simulation
Kincho H. LawStanford University
• NEESgrid System Integration Development
(NEESgrid - Bill Spencer, Jr. of UIUC – PI)
(Consortium effort – NCSA, ANL, USC’s ISI, U. Michigan, UC Berkeley, Stanford, and others…)
• NEESgrid’s Data and Metadata Efforts
(Data and MetaData Working Group)
• More on Data Issues
• Current Status
Outline of Presentation
Kincho H. LawStanford University
George E. Brown Jr. Network for Earthquake Engineering Simulation (NEES)
Field Testing EquipmentUniversity of Texas, Austin
$2,937,036
Geotechnical CentrifugeUniversity of California, Davis
$4,614,294
Reconfigurable Reaction WallUniversity of California, Berkeley
$4,268,323
Fast Hybrid Testing LaboratoryUniversity of Colorado, Boulder
$1,983,553Multi-Axial Subassemblage Testing
SystemUniversity of Minnesota, Twin Cities
$6,472,049
Lifeline Testing FacilityCornell University
$2,072,716
Field Testing EquipmentUniversity of California, Los Angeles
$2,652,761
Three (two relocatable) Shake Tables
University of Nevada, Reno$4,398,450
System IntegrationUniversity of Illinois at
Urbana-Champaign$10,000,000Tsunami Wave Basin
Oregon State University$4,775,832
Dual (relocatable) Shake Tables and High Performance Actuators
State University of New York,University at Buffalo
$6,160,785$4,379,865
Consortium DevelopmentConsortium of Universities for
Research in Earthquake Engineering$1,999,907
Permanently Instrumented Field SitesBrigham Young University
$1,944,423
Modular Simulation LaboratoryUniversity of Illinois at
Urbana-Champaign$2,958,011
Multi-directional Testing FacilityLehigh University
$2,593,317
Large Uniaxial Shake TableUniversity of California, San Diego
$5,890,000
Geotechnical CentrifugeRensselaer Polytechnic Institute
$2,380,579
NEESit OperationsSDSC, University of
California, San Diego-$40,000,000
Kincho H. LawStanford University
NEES Resources
Field Equipment
Laboratory Equipment
Remote Users
Remote Users: (K-12 Faculty and Students)
High-Performance Network(s)
Instrumented Structures and Sites
Leading Edge Computation
Curated Data Repository
Laboratory Equipment
Global Connections
(FY 2005 – FY 2014)
(Faculty, Students, Practitioners) Simulation
Tools Repository
• Enabling collaboration (laboratory experiments, field tests, simulations)
• Enabling data sharing, archiving and access by researchers (and public)
Kincho H. LawStanford University
Kincho H. LawStanford University
Vision of NEES (Vision of NEES (CollaboratoryCollaboratory))
Kincho H. LawStanford University
NEESgrid!- CyberInfrastructure• To connect and facilitate experimentation/simulation (virtual
collaboratory) in earthquake engineering for US and the world
►Tele-Control Services and APIs
►Tele-Observation and Data Visualization
►E-Notebook
►Streaming data services
►DAQ and related services
►Data and Metadata Services
►Remote Collaboration and Visualization tools and services
►Core Grid Services, deployment efforts, packaging
►Simulation Component
Kincho H. LawStanford University
Data/Metadata Working Group Member
Jean-Pierre Bardet University of Southern CaliforniaJennifer Swift University of Southern CaliforniaAndrei Reinhorn State University of New York, Buffalo(Data Sharing and Archiving Committee, DSAC)Ken Ferschweiler Northwest Alliance for Computational
Science and Engineering, Oregon State Univ.Lelli Van Den Einde University of California at San Diego Gokhan Pekcan University of Nevada, Reno(Coordinator of the Task Group)Hank Ratzesberger University of California, Santa Barbara Chuck Severance University of Michigan Bill Spencer University of Illinois, Urbana-Champaign(PI of NEES’s System Integration Project)Jim Eng University of MichiganJun Peng Stanford UniversityKincho H. Law Stanford University
Objective: To define a common approach and tools to enhance the sharing, access and utilization of the NEESgrid data repository
Kincho H. LawStanford University
NEESdata
NEESpop
LocalRepositoryA
PI
CentralRepository
DataTeamlets
Data Acquisition
Workstation
AP
I
DataTeamlets
AP
I
Data/MDIngestTools
Data tools
Data viewers
Grid and Web Services
NEESgrid Data – Core Elements
Kincho H. LawStanford University
Overall Data Modeling Efforts
NEES
Site A Site CSite B
Equipment People
Experiments Trials
Equipment People
Experiments Trials
Data Data Data
TsnumaiSpecimen
Shake TableSpecimen
GeotechSpecimen
CentrifugeSpecimen
Units Sensors Descriptions
SiteSpecificationsDatabase
ProjectDescription
Domain Specificmodels
Common Elements
Data / Observations
Kincho H. LawStanford University
NEESgrid Experiment Data Flow
NEESGridData
Repository
ProjectBrowser
DataTurbine
DataIngestion
ExperimentControl
StreamingViewer
DAQC
D
SiteSpecific
ProjectRelated
ExperimentalSetup
ExperimentalElement
DataElement
Data Model
DAQDisk
StoredViewer
Kincho H. LawStanford University
Evaluate existing data model representations and tools– Representation: E-R diagram, UML, NEESML, RDF, OWL– Tools: Protégé 2000, Rational Rose, Dezign
Evaluate existing data models for usability– Project Description: OrSt Model, Science Ontology– Sensors/Common Elements: Berkeley CUREE/Kajima, SensorML– Specimen/Structure Models: CIS/2, IFCs, SAC Data– Others: COSMOS
Visit, interview and discuss with experimental site personnel– Conference calls, UN Reno, UC Berkeley, UIUC, U Minnesota
Develop prototype models for feedback
Validate and test data models with saved experimental data
Data Model Development Effort by Stanford TeamData Model Development Effort by Stanford Team
A knowledge engineering approach:
(Shake Table/Large Scale Structural Tests – December 1, 2003)
Kincho H. LawStanford University
Pre-ExperimentStage
ExperimentStage
Post-ExperimentStage
Simulations Simulations Simulations
Process: Activities
Data Model
InstrumentationSpecimen Units of Measure
Observations Pre-experiment and post-experiment data could be as valuable as the actual experiment itselfComputer simulations play a significant role towards the design of an experiment as well as for post-event investigations
Kincho H. LawStanford University
Overview of Reference Data ModelOverview of Reference Data Model• Shake table and large scale structural tests• Simple and extensible, minimum burden to researchers • But sufficiently representative for earthquake
engineering experimentations
Data model representations:
Object Model– Has only 20+ main classes– Modeling tool: Protégé 2000– Backend storage: OWL (Web Ontology Language)
Relational Model– Has only 20+ tables– Modeling tool: Dezign– Backend storage: Relational database such as MySQL
Kincho H. LawStanford University
Project
Task
EventGroup
Event
1 SensorSetup
DAQCableWaveFormSetup
OutputData
Sensor
Publication
Person
Organization
InfrastructureSetup
SepcimenSetup
PrimaryEquipment
Site
File
SoftwareSetup
DAQSystem
SetupFile
DAQSetup
Specimen
DAQChannel
RolePerson
WaveForm Software
Activity
ApparatusSetup Apparatus
DataElement
SiteInformation
Prototype Class RelationshipPrototype Class Relationship
Kincho H. LawStanford University
Protégé Interface: Object ModelProtégé Interface: Object Model
Only 20+ main classes
Kincho H. LawStanford University
OWL RepresentationOWL Representation
Kincho H. LawStanford University
Relational Data RepresentationRelational Data Representation
Kincho H. LawStanford University
Modeling of Specimen and SetupsModeling of Specimen and Setups
Universal modeling of specimen for all experiments is very difficult if not impossible
Goal is to provide ways to archive the data and information on the project and the experiment
Desirable formats and features: CAD drawings; sketched drawings and notes; photos; narrative description; electronic notebook; linkage of drawings, sensor locations to data, etc..
Kincho H. LawStanford University
DAQChannel
DAQSystemSensor
DAQCable PC and/orExternal A/D
Sampling
ownermanufacturerserialNumberotherInfo
ownermanufacturerserialNumberotherInfochannelidfiltersamplingRategainoffsetexcitationVoltageunit
ownermanufacturerserialNumberotherInfocableidlengthconnectorType
ownermanufacturerserialNumberotherInfosensoridtypecalibrationInfolastCalibrationDateoutputQuantityminRangemaxRange
Modeling DAQ (data and setup)Modeling DAQ (data and setup)
Kincho H. LawStanford University
Project Data Browsing
Kincho H. LawStanford University
Project Data Browsing
Kincho H. LawStanford University
Project Data Browsing
Kincho H. LawStanford University
Project Data Browsing
Kincho H. LawStanford University
Data Services : Report Generator AssistanceData Services : Report Generator Assistance
RelationalDatabase:
MySQL
Jena JDBC
DBAccess.javaOWLAccess.java
DBDataRetrieval.javaOWLDataRetrieval.java
ReportGen.java
XML-Based DocumentsOWL (Web Ontology
Language)
Structured DataStroage
Based on Data Model
Standard Library forData Access
Present Common Interfacefor Data Access: e.g.
Access an Object by its ID
Retrieve Related Data:e.g. Data Related to
Instrumentation
Engineering ReportGenerator
DisplayNEES.java
OWLMapping.java DBMapping.java
Java Servlet Applications
Kincho H. LawStanford University
Re: Prof. Andrei Reinhorn @ SUNY-Buffalo
Report Structure & Corresponding FunctionsReport Structure & Corresponding Functions
getSummary()getScope()getSetupOverview()getInfrastructure()getSpecimen()getLoading()getInstrumentation()getProcedure()getSchedule()getImplementation()getResults()getDataProcessing()getPredictions()getDiscussions()
Kincho H. LawStanford University
Project
Task
EventGroup
Event
1 SensorSetup
DAQCableWaveFormSetup
OutputData
Sensor
Publication
Person
Organization
InfrastructureSetup
SepcimenSetup
PrimaryEquipment
Site
File
SoftwareSetup
DAQSystem
SetupFile
DAQSetup
Specimen
DAQChannel
RolePerson
WaveForm Software
Activity
ApparatusSetup Apparatus
DataElement
SiteInformation getSummary()
getSetupOverview()getInfrastructure()
getSpecimen()
getInstrumentation()
getLoading()
Data Model & Corresponding Functions
Kincho H. LawStanford University
Front Page to Access Front Page to Access MiniMOSTMiniMOST ProjectProject
Kincho H. LawStanford University
Retrieve SummaryRetrieve Summary
Kincho H. LawStanford University
Scope and General PresentationScope and General Presentation
Kincho H. LawStanford University
Infrastructure SetupInfrastructure Setup
Kincho H. LawStanford University
InstrumentationInstrumentation
Kincho H. LawStanford University
More on Data Issues…..Current Developments and Tools
• Facilitate experimenters and researchers generating and ingesting data into repository
• Allow browsing (and limited searching) and retrieving data/information about a project or experiment
• Support “No data generated by NEES would be lost” policy
But …..
“A data repository is only as good as what can be done with the data.”
Kincho H. LawStanford University
Data Life Cycle
Data Production
Data Management
Data Consumption
• For how long?• What format? (standardization, usability)
• Heterogeneous data types (text, CAD, video, streams)• Raw, processed, derived, curated data• Non-proprietary data format and software (what to
archive?)• What to preserve? Everything? Selectively?
Need: data policies and short/long term plans for the cyberinfrastructure community
(not just the cyberinfrastructure but the community it serves!)
Kincho H. LawStanford University
Consumers for the NEESgrid Data Repository:
• Experimenters – about generating reports, analysis of results
• Practitioners – about specific results for specific experiment on specific component (beam column joint)
• Researchers – about prior works for developing new ideas (new, novel active damping/sensing devices)
• Colleagues in other fields – data interoperability and sharing (IRIS’s seismological data sets)
• Faculty – developing teaching materials• Students – discovering new knowledge, mining archived
data • K-12 audience – about earthquakes and our civil
infrastructure systems
Kincho H. LawStanford University
Preservation Planning
Administration
DataManagement
ArchivalStorage
AccessIngestProducer Consumer
Management
OAIS (Open Archive Information System) Functional Model
http://ssdoo.gsfc.nasa.gov/nost/isoas/
Current Focus and Deliverables
NEES Consortium
?
?
Kincho H. LawStanford University
Capture Data/MetadataThroughout Data
Lifecycle
Data Models
Experiment Prep
Data Analysis
Data Publishing
Data Curation
Data Discovery and Reuse
Experiment Management
Data Monitoring
Kincho H. LawStanford University
What have I learnt about NEESgrid (CI) data issues?
The amount of data (in a wide variety of formats) to be generated will be overwhelming – standardization is hard but some standardization is necessaryExpectations of what data models and repository can support are very high (and may not be easy to deal with in short term) Putting data in a computer (meaningfully) IS NOT an easy problem
A lot to be learnt among different disciplines – engineering, digital library, digital archive and preservation, library information science, information and communication technologies, ….Many related ongoing works: Skyserver, DSPACE, IRIS, GEON, BRIN, …..
Kincho H. LawStanford University
Strong leadership and management team (not only technology developers) – coordinate activitiesTrue collaborative efforts – among developers, researchers, disciplines involvedDedication of community participants from the domain of concernPolicies and guidelines need to be well defined by the community, implemented and enforcedNeed to have “buy-in” by the researchers (or users of the CI)CI is not a pure “academic research” activity with much at stakes by the community at large
What have I learnt about NEESgrid (CI) issues?
Kincho H. LawStanford University
Current Status
NEES: Officially opened Oct. 1, 2004 and Opening Ceremony at NSF Nov. 15, 2004NEES Organization: NEESr (Researchers), NEESInc(Management), NEESit (IT operations – SDSC)NEESgrid transition from NCSA to SDSC on Oct. 1, 2004NEESit to release interim version by end of March, 2005International collaborations/activities related to NEES:
KOCEF ($90 million effort in Korea) E-Defense (Japan)NCREE (Taiwan)European groups and others.
Kincho H. LawStanford University
Rapidly Deployable CI and Data Problem
Source: A Proposal on “Innovative Use of IT in Post-Disaster Investigation,” EERI, Feb 2004
Kincho H. LawStanford University
Acknowledgments
NEES’s Data and Metadata Task GroupNEES’s Data Sharing and Archiving CommitteeJoy Pauschke of NSF, Bill Spencer of UIUC, Chuck Severance of U. Michigan and Joe Futrelle of UIUC, Gokhan Pekcan of U. Nevada, Reno, Anke Kamrath, Lelli Van Den Einde of SDSC and many others …..Jun Peng, Stanford UniversityAny opinions expressed in this presentation are those of the presenter and do not necessarily reflect the views of the NEESgrid, NEESit, NEESInc, NSF and his collaborators.
Kincho H. LawStanford University