Producing Standardized Data Using a Master FME Workspace
20
Producing Standardized Data Using a Master FME Workspace Victor Rutherford GIS Data Analyst, Developer April 25, 2012
-
Upload
safe-software -
Category
Technology
-
view
2.452 -
download
0
description
Presented by Victor Rutherford of Nexen Inc Abstract: We have successfully implemented Corporate-wide standards for our spatial data through the use of a Standard Data Model, some custom technical tools and a Generic Master FME script at the heart of it all. Nexen Inc. had a large amount of spatial data being consumed by various departments, and an effort was made to try to standardize it to help manage it all which initially missed the mark. The initial attempt led to various FME scripts with varying styles of design by multiple authors of varying degrees of expertise in FME. Management regrouped and succeeded with a different approach: One Generic Master FME script flexible enough to manage most of the data-manipulation scenarios and which also implemented the Spatial Data Standards which were desired into each data load and update. This also tremendously improved maintenance of the ETL environment.
Transcript of Producing Standardized Data Using a Master FME Workspace
Producing Standardized Data Using a Master FME WorkspaceVictor RutherfordGIS Data Analyst, Developer
April 25, 2012
Spatial Database
GIS
DB
Geomatics
Geology
Engineering
Cartography
Vendor Vendor
Vendor Vendor
AnalysisGeophysics
GIS DB
Loading and Updating Data
- Unstandardized
- Difficult to maintain
- Confusing to use
GIS DB
Master FME Workbench
(Generic)(Custom) Standardized
Magic Formula?
1 23
Standard Data Model Data Modeling Tool+ Generic FME Workbench
PSDM PSDMManager
MasterFME
Workspace
+ +
Attribute-mappings+ Lookup table (domains)
The Right Person
Domain Expert /Data Modeling
ETL/FME
Corporate Standards Steward
QA
(Generic)Data Modeling Tool
FME Rocks!
I understand rocks, but what’s an FME?
Master FME Workspace
(Look Ma, No hands!)
Master FME Workspace
Master FME Workspace
Master FME Workspace
Master FME Workspace
Source
Target
ID
ID
Delete?
CRC
Match = No Change
Mismatch = Update
Insert
Master FME Workspace
Master FME Workspace
Easier to Maintain
Master FME Workspace
Harder to Maintain
GISDB
GIS DB
Master FME Workbench
(Generic)(Custom) StandardizedGIS DB
Thank You!
Questions?
For more information: Victor Rutherford [email protected] [email protected] Nexen Inc. www.nexeninc.com VRGS Ltd. www.vrgs.ca
Source
Target
ID
ID
No ID/PK?
CRC
Match = No Change
Mismatch = Insert Mismatch = Delete?
Published Parameters
Source
PSDM Parameters
Output
Custom Transformers?
