Developing Workflow from TERMS: techniques for electronic resource management
Electronic Data Management and Workflow
-
Upload
earthsoft -
Category
Technology
-
view
5.206 -
download
3
description
Transcript of Electronic Data Management and Workflow
Imagine the result
Electronic Data Management and Workflow
Introduction
Jane Kennedy
Project Chemistry & Data Quality
ARCADIS U.S., Inc.New Orleans
EDD Management and Data Workflow Overview
• ARCADIS data management approach• Planning & setup• Data acquisition• Laboratory EDD prep and submittal• EDD receipt and review• Data distribution• Questions
Electronic Data Management
• Why manage data electronically?
• Consistency
• Confidence
• Efficient access to information
ARCADIS Approach to Data Management
PLANNING
Plan data acquisition,
management , quality, and reporting systems
ACQUISITION
Acquire data efficiently
MANAGEMENT
Manage data with tools appropriate to output
requirements
VALIDATION
Evaluate data quality
compliance with project
plan
REPORTING
Provide streamlined reporting and data
accessibility to users
Increase productivity of technical staff
ARCADIS Data Management Systems
• EQuIS 3.2• Desktop or server
• EQuIS 5• Enterprise system• Automation of EDD
management• Internally developed system – Access platform
• Server• Microsoft Excel
• Desktop
• Involved variety of disciplines and stakeholders in selection• Data managers• Corporate IT• GIS personnel• Project teams• Senior management• Clients
• Evaluated overall system applications
EQuIS 5 Deployment
A data acquisition and management strategy defines:
•Coordination of appropriate resources
•Project quality assurance process
•Data deliverables
•Communication pathways to ensure data usability and accessibility
Planning
• Project staff• Provide information to
labs and DMs
Stakeholder Participation
Database
•Field Data
•3rd Party Info
•Lab Data
•Boring Logs
•Maps/Figures
•Data Tables
• Data managers• Receipt, import, query and export
• Data visualization and project Team• Communicate data export content requirements
to DMs
Project Planning Documents
• Work Plan, SAP, FSP, QAPP• Document performance requirements• Summarize project activities• Define project goals• Establish data quality objectives
• Permits• Corrective action goals• Risk standards (RECAP)
Data Management Plan
Data Management Plan• Based on project planning documents• Define personnel responsibilities• Set up work flow• Data acquisition strategy• Establish project nomenclature• Create reference values• Detail storage and archive
Sample Collection Planning
• Establish location and sample nomenclature• Identify locations• Monitoring wells • Soil depths• Sample matrix• Trip blanks, equipment blanks• Field duplicates
• Provide information to field team and data manager prior to sampling
Database Set Up
• Prepare chemistry and geology database• Use applicable project nomenclature• Acquire geographic coordinates• Provide project specific criteria to data manager
• Permit limits• Screening standards/RECAP limits• Corrective action goals
• Identify export requirements• Trend plots, contours, charts• GIS or CAD
Field Data Acquisition
• Establish field data requirements• Well construction• Geologic information• Water quality parameters• Water levels
• Identify data acquisition format• Manual entry onto forms with transfer later• Electronic acquisition with nightly upload
• Geographic coordinates• Survey or GPS
• Routes of data transfer• Who has
responsibilities?• What format?• Quality Control Review
prior to submittal to Data Manager
• Geographic coordinates• Get them to the Data
Manager in a timely manner
Uploading Field Data
Chain Of Custody Documentation
• Complete COC with as many EDD expectations as possible• Sample ID• Matrix• Sample type• Samples to be
used for site specific QC
Communicate Project Requirements to Laboratory
• Laboratory is a partner in the project success• Communication prior to sample collection is
crucial• Data quality requirements• Performance expectations• Deliverables • Communication tree
What the Laboratory Needs
• List of samples and performance criteria
• Sample nomenclature• Select EDD Format prior to sample
submission• DEQ format• Consultant format
• Send project reference values• Confirm lab has programming
completed to generate required EDD
Laboratory EDD Preparation
• Understand the requirements• Spend the time to develop the 4-file EDD NOW• Minimize manual entry• Report and EDD MUST match
• Rounding routines• Manual data entry peer review
• Data Checker (EDP) = zero defects• EDD requires specific naming convention• Follow the rules or it will get kicked back
• Contact client for direction• Don’t make anything up• Review reference values for
other options• Get email confirmation of
directions• Include additions changes in
email submission of EDD
Reference Values Missing
Potential EDD PitfallsSample Table
• Confirm sample ID and handwriting interpretations• Do not add any suffixes to the sample ID unless
directed by consultant• Sample Delivery Group (report number) must be
populated• Lab may be required to populate start and end depth
for soils• Sample receipt date must be populated
• Sample type is critical because some samples require listing parent samples
• Field duplicates – lab may need to use sample type of “N” to clear checker
• MB = Material Blank not method blank (LB)
Potential EDD PitfallsSample Table (continued)
Potential EDD PitfallsTest Table
• Subsample amounts must be populated• Lab name must be populated• Sample dates and times must match sample file• Percent moisture cannot be null for solids• Analysis type must be appropriate for dilutions,
re-analyses• Understand the use of T, D, N for total and
dissolved field• Caution with methods where multiple parameters
reported (e.g. Method 300 or 352.3)• Time format can cause problems
Potential PitfallsResult Table
• Do not add suffixes to the CAS number• Only 1 result is reportable = Yes (multiple dilutions)• If the detect flag is yes, the result value cannot be
null• Units fields cannot be null for populated fields
requiring a definition of value units• Subsample amount must be populated• Quality control data must be included in the
appropriate field
Additional Laboratory Challenges
• Can’t load project reference values
• Data Checker continues to show errors
• Do NOT try to re-write the export every time you have samples
• Make sure to save the export to laboratory system
• LIMS changes - CAUTION
EDD Submission
• Data checker (EDP) = zero defects• Document problem resolution• Follow EDD naming convention• Email EDD to appropriate venue(s)
DATABASE
Web
FTP
Lab
PDA
FieldEDD
LabEDD
SubmitterNotice
ManagerNotice
Data Receipt:• Variety of information received in Electronic
Data Deliverable (EDD) for import into the database
EDP
• EDDs received by project data manager or via direct upload system
• EDP confirms completeness and compliance (3.2 v 5)
• Do not correct the EDD• If it is not compliant, return
to lab
Consultant EDD Management
Database
Data Manager
•Field Data
•3rd Party Info
•Lab Data
•Boring Logs
•Maps/Figures
•Data Tables
EDD Upload to Project Database
• Manage EDD upload schedule• Group work in batches
• Tracking is critical• Report and EDD may not arrive
at same time• Verification / validation of
conformance
Consultant Database Updates
• Verify sample IDs• Laboratory data flag definitions• Run update queries to move qualifiers• Add location codes to link samples• Field information
• Field parameters• Parent samples for field dups
• Perform project specific queries
Confirmation of Content
• Export data to crosstab format• Review a percentage of data against lab reports
• Define percentage in project plan process • If deficiencies are identified return to lab for
correction• Rounding Routines• Significant Figures• Data Flags
Track Changes to the Database
• Identify fields in the database to document changes by data managers• Historical data source• Laboratory Flagging changes• Special data qualifier definitions• Validated, Verified, or Neither• Validator comments/reason for qualification
• Updates of project information
• Screening tools are available• Critical to have appropriate QC
information• Populate data fields appropriately• Understand the limitations• Qualified personnel review of
screening tool reports
Electronic Data Quality Screening
‘My’ Format AND LEADMS
• Create LEADMS EDD export from data system
• If managing 2 EDDs (client specific and LEADMS)• Ensure any changes in the
desktop version are transferred to LEADMS
• Perform confirmation QC to verify
DATABASE
Web
FTP
Charts,Graphs
‘Pretty’ Reports
Tabular data,Crosstabs
Exports for data analysis
and visualization
Exports to Data Users
Database
Data Manager
•Field Data
•3rd Party Info
•Lab Data
•Boring Logs
•Maps/Figures
•Data Tables
Data Exporting
• Quality in = Quality out
• Exports can be automated, scheduled or team coordinated
• Provide information to the DM to yield efficiency
• Database monitors incoming data and generate reports based on arrival of:
– New data• Anytime there is new data for facility 123, build
Report A and email it to me and all my group.– New detections
• Anytime there is new Arsenic data greater than 15 ppb for facility 123, build Report B and email it to just me and my client.
– Date• Build Report C for facility 123 and email it to the
entire group on the 15th of every month.
.
Data Export Automation
Data Export to Visualization Programs
Thank You
Jane Kennedy
ARCADIS U.S., Inc.Phone: (504) 832-4174
Cell: (225) 205-8256Email: