Data Exchange Needs Using ETL Tools. Overview Client Project Issue Methodology Schedule & Budget...
-
Upload
garey-higgins -
Category
Documents
-
view
220 -
download
0
Transcript of Data Exchange Needs Using ETL Tools. Overview Client Project Issue Methodology Schedule & Budget...
GIS Project Manager: Kelly Dell
Overview
• Client
• Project Issue
• Methodology
• Schedule & Budget
• Results & Recommendations
• Challenges
• Acknowledgments
• Questions
Project Issue & Primary Goal
• Current data conversion process
o Each time data is updated it needs to be redone manually
o Need to streamline the process
o Extract Transform Load (ETL) Tools• Safe Software Feature Manipulation Engine (FME)• Esri ArcGIS Geoprocessing Tools (Data Interoperability Extension)
Methodology
Standard Methodological
Approach:
Identify objectives,
issues & goals
Research topic and best
practices
Identify scheduling,
budget, risks
Communicate & Conduct Meetings
Gather information
for evaluation, reporting and
decision making
Report findings
• Streamlined data conversion process
• User know-how
• ETL tools capabilities and limitations
• ETL tool decision making
• Final Report and recommendations
Goals & Deliverables
Extract Transform Load
ETL - Traditional
ETL is used to migrate data from one database to another, to form data marts and data warehouses and also to convert databases from one format or type to another.
Spatial ETL – Safe Software Feature Manipulation Engine - FME
Provides the data processing functionality of traditional ETL, but with a primary focus on the ability to manage spatial data (the geometry, attributes, and coordinate systems of spatial datasets)
SEP
OCT
NOV
DEC
JAN
FEB
MAR
APR
MAY
JUNPlanned Value (PV)
100
1500
2100
3900
6830
9180
11330
16580
18950
21700
Earned Value (EV)
Actual Cost (AC)
$2,500
$7,500
$12,500
Earned Value Management (EVM)
FME Server• FME Desktop is the authoring
tool for FME Server
• Any spatial ETL workflow created in FME Desktop can be published to FME Server
• FME Server is a scalable processing environment that allows multiple users via the web
• Workflows can be shared and automated in this environment
“Safe Software sets the standard for spatial data interoperability, so it was an obvious choice to use their FME technology for Esri’s ArcGIS Data Interoperability Extension.”
– Jack Dangermond, Esri President
ArcGIS Desktop Data Interoperability Spatial
ETL
Evaluation• Architecture• ETL Functionality• Ease-of-Use • Reusability• Debugging• Real-time• Connectivity• General ETL tool characteristics
Challenges• GIS Expertise, knowledge and training
• Assumed licenses acquired and ready to use
• Data availability and ability to work on data • Client Representation
Recommendations• Streamlined data conversion process
• User know-how
• ETL tools capabilities and limitations
• ETL tool decision making
Acknowledgements• Client
o Laura Kingston
• Project Thesis Adviser o Mike Wallace
• Technical Advisingo Colin Bissello Jonathan Lepageo Allison Hughes
• FME Software Acquisitiono Gerry McCoubreyo Craig Vernon