Make it so - Recent Proceedingsproceedings.esri.com/library/userconf/petrol17/papers/... ·...
Transcript of Make it so - Recent Proceedingsproceedings.esri.com/library/userconf/petrol17/papers/... ·...
Make it so:Pushing UPDM to Production Against ArcGIS 10.5 and APR
ESRI Petroleum User Group (PUG) – Houston, TX
April 13, 2017
Anthony HermanSupervisor GIS, Drafting, and Technical
Records
Express-Platte Pipelines – Technical Services
—
ENBRIDGE| CELL: 403-919-8479 |[email protected] |
Peter VeenstraPrincipal GIS Technologist
Oil and Gas, TRC
KANSAS CITY| CELL: 816-820-7841 |
Express-Platte Pipeline System
Express-Platte Pipeline System
HCAContract role vacant
GIS Analyst
EP – Geospatial Data Management (GDM) Team
Anthony Herman Supervisor GIS
Jeff AndersonGIS Analyst
John FinneganGIS Coordinator
Stephen NingGIS Specialist
CP
ILI
Risk Run
CIS/DOC
Geotechnical
DIGS
Integrity
UAT
Application Development
Planning
IT apps and management
systems
RASCI
Projects
10X thinking
Field support User
support
Project support
Engineering
Operations
LMS
Lands
Business Development
Investigation
Unsecured and unmanaged data; notable example being an early interview with KM staff who moved over to Spectra Energy Liquids: why the portable hard-drives? It s GIS data the stuff on the server is old . One employee had ¼ million files equating to 5 times the size of server based geodatabases.
Local, desktop ArcGIS licenses were the norm, as was keeping data locally on workstations.
Express-Platte s history of multiple ownerships created a vast disconnect between geospatial data and technical records.
No data model was used to manage the data resulting in multiple attributes being replicated across featureclasses.
Multiple versions of the truth for multiple datasets, i.e.: digs, ILI, and centrelines.
Data was disconnected and misaligned, i.e.: coatings were spatially independent the pipeline.
Spectra Energy Due Diligence of GIS Data
GIS Production Databases | Spatial Data >
Integrity Databases and supporting data > Cathodic Protection >
Lands > EHS >
Facility Records >
Engineering >
Drafting >
DA
TAD
ATA
DA
TA
DA
TA
DA
TA
DA
TA
DA
TA
DA
TA
Geodatabases – no data model structure, un-versioned, unmanaged User data – 200 GB of unmanaged folders / 4 TB Ext HD s of unknown data*
Non-spatialized databases User data – unmanaged folders / duplications / TVC Non-spatialized databases User data – unmanaged folders / duplications / TVC
Non-spatialized databases Unmanaged spatial data
Un-spatialized pipeline attribute database
Access Database User data – unmanaged folders / duplications / TVC
Drafting Database User data – unmanaged business processes Records Project - Facility Records Database
History – Data Map – 2013 - Present
Pipe Facilities CP Cartopac Base
DRAS
Shapefiles FGDB GDB KML
Cambio Excel Data ILI DIGS CPDM Excel Data ILI DIGS
LMS PAD EHS
FRD Prebish
Eng DB Excel Data ILI DIGSIRTH
MeridianOverdue Projects
Current Projects
xlsx csv ASA
ESRI ArcGIS | CARTOPAC | XMAP | SHEETCUTTER | XTOOLS | EDGE LIQUIDS | PORTAL
DRAS AND MS EXCEL CPDM
WEB WEB
WEB (FILE SERVER BACKEND)
MS ACCESS
MERIDIAN
GIS
INTE
GR
ITY
CP
LAN
DS
EHS
REC
OR
DS
ENG
INEE
RIN
G
DR
AFT
ING
Non-spatialized databases Unmanaged spatial data
History – Data to Technology Map
Pipe Facilities CP Cartopac Base
DRAS
Shapefiles FGDB GDB KML
Cambio Excel Data ILI DIGS CPDM Excel Data ILI DIGS
LMS PAD EHS
FRD Prebish
Eng DB Excel Data ILI DIGSIRTH
MeridianOverdue Projects
Current Projects
xlsx csv ASA
Lands
ASA
Cartopac
Integrity
Legacy Transformation
Spatial Data
GIS Production Databases
Eng EHS
Out of the chaos
Working from a philosophy centered around pipeline integrity data, a plan was struck to deal with past failures.
GDI
-A Geospatial Data Infrastructure – GDI – is the construct of creating a hardware and software ecosystem that permits
geospatial data to be modelled, maintained, and served up to a user base, through intuitive, dynamic, and scalable portals, to
elucidate complex geospatial data and queries. -
GDI is the framework for Geospatial Operational Excellence-
Express-Platte s (EP) GDI mission is to model EP s real-world pipeline assets. EP s GDI is based on ESRI s Utility and Pipeline
Data Model (UPDM) and will leverage ESRI s ArcGIS 10.5 Enterprise, ArcGIS Event Editor, ArcGIS Pro, Advanced Pipeline
Referencing (APR) tools, and Portal. -
GOE and GDI
Express-Platte Pipeline System
Geospatial Operational Excellence
Steps to achieve Geospatial Operational Excellence
contemporary data model and tools-
data verification | consolidation | accuracy/precision-
referential integrity-
single version of the truth -
documentation-
scope and governance-
production---
As stated the goal is for GDI to virtually model all product carrying pipeline; linking those geospatial features to
the record of truth.
GOE Measures of Success
GOE: Data Model Why ESRI s UPDM and APR toolset
EP is an ESRI shop: UPDM was chosen to stay in-line
with the current ESRI development focus and
ArcGIS Pipeline Referencing (APR) hooks into UPDM.
Expectation: ESRI will handle the core data
management functionality decreasing SEL s reliance
on 3rd party tools.
Established: ESRI APR is based on ESRI s established Roads and Highways apps.
Vision: ESRI is planning to release OOTB templates,
reports, functionalities based on UPDM and APR.
Cost: reducing reliance on 3rd party vendors eases management of IT/GIS
infrastructure and lowers licensing and maintenance
costs.
Future: UPDM is built to allow growth, able to handle
gathering, transmission, and distribution pipeline
networks.
SDE
Recap – current data environment
Geodatabases
\\FileServer\Geodata
DRAS
Cambio CPDM
DIGS
LMS
PAD
EHS
FRD
Eng DBILI
IRTH
SQL Server 2008
Shapefiles FGDB GDB KML
Excel Dataxlsx csv ASA
CartopacCP BasePipe Facilities
\\FileServer\SE Liquids
Multiple \\FileServers
SQL Server 2012
GOE: data verification | consolidation | accuracy/precision
Geodatabases
\\FileServer\Geodata
DRAS
Cambio CPDM
DIGS
LMS
PAD
EHS
FRD
Eng DBILI
IRTH
Shapefiles FGDB GDB KML
Excel Dataxlsx csv ASA
CartopacCP BasePipe Facilities
\\FileServer\SE Liquids
Multiple \\FileServers
GDI.GDIUS GDI.GDICAN
Imperial Metric
SDE
FME
ModelBuilder
QA
Referential Integrity
Implementation of an architecture which ensures entries into the GDI database maintain the integrity of the data.
-Referential Integrity is architected through the GDI data model, ESRI and SQL
versioning, and permission group authentications, and ESRI APR toolset.-
Goals being to eliminate data duplication, increase data accuracy, and link data to other data to elucidate complex interdependencies.
GOE: referential integrity
TraceableVerifiableComplete
GOE: single version of the truth
The Source
\\FileServer
GOE: documentation
GDI DocumentationGeospatial Data Management (GDM)
Documentation
Enterprise Architect File
Data Dictionary
Schema Change Log
ESRI Documentation
Governance
Process Documents
Procedures
Projects
PHMSA Readiness
User Support
NPMS
Data Use Agreements
Workflows
SharePoint
Document Repositories
Apr 14 Jul 14 Oct 14 Jan 15 Apr 15 Jul 15 Oct 15 Jan 16 Apr 16 Jul 16 Oct 16 Jan 17
GOE: GDI Development
Dec 15
Phase 1 DeliverableJan 15
RFP
Jan 14
Budget Approval
Jan 17
Budget Approval
Feb 17
Phase 2 DeliverableApr 14
Review Deliverable
Mar 14
3rd Party Review
Mar 17
ProductionApr 16 - Aug 16
Integrity Design
Mar 15 - Aug 15
Data model Design
Aug 16 - Dec 16
Data Migration
May 14
SOW
Jan 15
Budget Approval
Mar 16
Phase 2 Kickoff
Jun 14 - Dec 14
SEL internal work
Jan 16
Budget Approval
Feb 15
Phase 1 Kickoff
Sep 15 - Dec 15
Data Migration
Jeff Anderson
Stephen Ning
Integrity Team
GOE: team
John FinneganGIS Coordinator
UPDM / APR / 10.5 Enterprise
Rick Morley
Pete Veenstra
Tim Monnig
Shawn McCormackIT Support
TRC
Robert Gardner
Hari Adduri
Additional Contractors
IT apps and management
systems
Tom Coolidge
Jeff Allen
Anjali Bhangay
APR TeamTom
DeWitt
William Isley
ESRI
Nathan Easley
Terry Delong Anthony Herman
Supervisor GIS
Event Editor
APR
ArcGIS Portal
ArcGIS Pro
ArcMap
ESRI 10.5 Enterprise*
GMAF
Lands
EHS
Geocortex
GDI SQL 2012
hosting GDIUS and GDICAN
HCA
GDM & other FileServers
P_Integrity
P_Pipeline
FRDIntegrity
other
ENG
ASA
GOE: GDI Production Architecture (projected)
*As part of the early adopter program we are using evaluation licensing to support GDI until we can stand up the 10.5 ArcGIS environment.
SheetCutter
Pipe viewer
Project Investment
Project Comps – Natural Gas (NGT) and Liquids (LT) Transmission Pipeline Operators
Bu
rnB
urn
Bu
rn
2017 | Phase 32014 | Phase 0 2015 | Phase 1 2016 | Phase 2
$23,500
Budget $62,000
$243,490
Budget $250,000
$252,762
Budget $235,000 Budget $115,000
$5,000
$890,000* $2,000,000 $930,000
Data model config and data migration, development of toolsets.
Database migration to new toolset and configuration of tools; add other data feeds.
Data model development and data migration, development of toolsets.
1 year 2 years 1 year
SE Transmission US NGT Operator US NGT & LT Operator
$3,800,000
Data model change and data migration.
4 years
US LT Operator
$247,000
Data migration and conversion.
2 years
European Offshore LT Operator
$519,752
Data model development, data migration, and implementation of new toolsets.
3 years
SE Liquids
*Does not include internal costs
GOE: GDI Investment
ESRI UPDM and ArcGIS for Pipeline Referencing
• Migration Plan
• Source Data
• Building the Centerline
• Source Data after the Source Data
• What is APR going to be used for again?
• Configuring APR (the lessons)
• Operational Excellence
• Migration Plan• Create UPDM Model (Release, Re-visit and Revise)
• Create a Lexicon Update the Data Dictionary
• Create a Mapping File (Source Data to UPDM, Existing Pipeline DM to UPDM)
• Create a Change Log (who changed, what, how and why?)
• Oh wait … new release …
• Create the Centerline
• Synchronize with the previous GIS data
• Add the dig sheet information
ESRI UPDM and ArcGIS for Pipeline Referencing
• Source Data• Multiple sources of the truth
• Created a MASTER Weld List of ID’s
ESRI UPDM and ArcGIS for Pipeline Referencing
ILI Vendor 1
ILI Vendor 2
ILI Vendor 3
Existing GIS Welds
XYZM
XYZM
XYZM
XYZM
Express-Platte Pipeline System
• Building the Centerline
ESRI UPDM and ArcGIS for Pipeline Referencing
• Source Data after the Source Data• When using ILI to create a centerline
calculate measured distance from XY distance not joint lengths
• Then calibrate to the welds and KEEP the offset distance
• Not all ILI’s called out the welds the same• But bend apex’s were not stored• Had to synchronize with the GIS bend apexs
to build the ‘real’ centerline• And we needed to manage this in two
geodatabases – CAN/US• This was a manual task but it would sure be
nice to have a ‘align’ function in APR and have the Geodatabase allow two LRS systems in a single geodatabase
ESRI UPDM and ArcGIS for Pipeline Referencing
• What is APR going to be used for again?
• Managing the centerline – engineering stationing and continuous measure stationing
• Replacing pipe (not often) – adding engineering equations
• Using the ‘event’ editor to load dig data –excavation, defect locations, pipe-replacements, coating condition and replacement
• That was the plan … not fully flushed out
ESRI UPDM and ArcGIS for Pipeline Referencing
• Configuring APR (the lessons)
• APR requires specification of M-Unit of Measure.
• Geodatabase does not require this.
• Configuring the M-Tolerance in the GeoDB, loading data
• Record and document your settings (you will do them over again)
ESRI UPDM and ArcGIS for Pipeline Referencing
• Learning APR• Build 10 engineering series (0-10, 11-20, 21-30 … etc.)
• Map the APR required tables and fields to your (UPDM) tables and fields
• Operational Excellence• Worked with champion
• Had a manager
• The approach was focused on making this the system of record
ESRI UPDM and ArcGIS for Pipeline Referencing
Make it so:Pushing UPDM to Production Against ArcGIS
10.5 and APR
Anthony HermanSupervisor GIS, Drafting, and Technical
Records
Express-Platte Pipelines – Technical Services
—
ENBRIDGE| CELL: 403-919-8479 |[email protected] |
Peter VeenstraPrincipal GIS Technologist
Oil and Gas, TRC
KANSAS CITY| CELL: 816-820-7841 |