Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles
-
Upload
safe-software -
Category
Technology
-
view
1.984 -
download
8
description
Transcript of Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles
![Page 1: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/1.jpg)
Using FME to Convert TIGER Spatial Data From Oracle Spatial to ESRI Shapefiles
Mei-Leng Freeman, Tim Puzycki, Jay E. Spurlin Spatial Products Software Branch Geography Division, Bureau of the Census
![Page 2: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/2.jpg)
2
Introduction
• Public products – TIGER/Line Shapefiles – Partnership Program Shapefiles
• Internal products – Data Access and Dissemination Staff
(DADS) shapefiles for use in the American Fact Finder websites (internal and external)
– Live Individual Shapefile Request and Delivery System (LISRDS)
– Quality Control (QC) shapefile for various projects
![Page 3: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/3.jpg)
3
Shapefile Creation: Legacy Approach
• Legacy TIGER – Utilized for 1990 and 2000 Decennial Censuses – In-house database format – Binary files – Partitioned by county
• Legacy Shapefile Generation (shpGen) – Java shapefile classes – Perl driver scripts – Directly read from a TIGER partition – Legacy TIGER C API – Directly wrote to a shapefile – ASCII Job configuration files (user defined tags)
![Page 4: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/4.jpg)
4
Modernized Approach
• MAF/TIGER – The Legacy Master Address File (MAF) and
Topologically Integrated Geographic Encoding and Reference (TIGER) datasets were combined.
– Oracle Spatial and Oracle Topology Data Model v10.2
– Seamless, national data set • Current shpGen
– Perl driver scripts – Pre-generated FME mapping files (.FME) – FME Engine handles reading from Oracle Spatial
and writing to ESRI shapefiles
![Page 5: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/5.jpg)
5
Initial Process
• Considerations for automation – Automate everything possible. – Use COTS software, wherever possible. – Ensure consistency of data after conversion. – Provide ease of maintenance. – Make source code reusable. – Fairly stable source data structure. – Fairly stable target shapefile formats.
![Page 6: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/6.jpg)
6
FME Mapping File Usage
• Manually create FME mapping files (.FME) – Previous FME mapping files used as
templates. – A standardized set of macros is used.
• The FME mapping files are released as part of the software installation.
• Automated production control system drives the process and provides any dynamic inputs.
![Page 7: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/7.jpg)
7
Oracle Wallet Support
• Stores database credentials: – Username – Password – Database
• Avoids providing Username and Password to command line interface
![Page 8: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/8.jpg)
8
Example FME Mapping File
![Page 9: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/9.jpg)
9
Example FME Mapping File
![Page 10: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/10.jpg)
Manually Created FME Mapping Files
• TIGER/Line Shapefiles – Produced from “Benchmarks” (snapshots) of
the MAF/TIGER Database. – Public TIGER/Line shapefiles are posted on
the world wide web. • Partnership Shapefiles
– MAF/TIGER Partnership Software (MTPS)
• LISRDS Shapefiles – On-demand access to MAF/TIGER data – Data source may be “live” or a snapshot
10
![Page 11: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/11.jpg)
Current process
• A wider set of shapefile requirements – Mapping performance issues
• Chained line representation of area boundaries • Un-duplication of lines in partitioned layers
– Visual representation issues • Partial features • ‘Balance of’ features
– Connect with Oracle Wallet
11
![Page 12: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/12.jpg)
FME Mapping Files from FME Workbench
• DADS/AFF Shapefiles – Both .FME and .FMW files are archived with
the source code. – The .FME files are released to production,
and used by FME Engine. – A similar procedure (though not yet as
automated, and not making use of the geoprocessing tools) is used by another branch for the generalized and projected shapefiles.
– Samples on the following slides:
12
![Page 13: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/13.jpg)
13
![Page 14: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/14.jpg)
14
![Page 15: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/15.jpg)
15
![Page 16: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/16.jpg)
16
![Page 17: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/17.jpg)
17
![Page 18: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/18.jpg)
18
![Page 19: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/19.jpg)
19
![Page 20: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/20.jpg)
20
![Page 21: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/21.jpg)
Example: DADS/AFF Line Shapefiles
21
State 09
State 05
State 02
State 13
![Page 22: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/22.jpg)
Example: DADS/AFF Line Shapefiles
22
State 09
State 05
State 02
State 13
![Page 23: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/23.jpg)
23
![Page 24: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/24.jpg)
24
Minnesota (State 27)
Iowa (State 19)
![Page 25: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/25.jpg)
25
Workbench File Example: County Subdivisions
![Page 26: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/26.jpg)
26
Example: Connect with Oracle Wallet
![Page 27: Using FME to Convert TIGER Spatial Data From Oracle Spatial To ESRI Shapefiles](https://reader034.fdocuments.net/reader034/viewer/2022042614/557dadb7d8b42a351d8b4a32/html5/thumbnails/27.jpg)
Thank You!
• Questions? • For more information:
– Jay E. Spurlin • [email protected]
– Bureau of the Census • 4600 Silver Hill Road, Stop 7400 • Washington, DC 20233
• Products made using FME Engine – TIGER/Line Shapefiles
• http://www.census.gov/geo/www/tiger/index.html – American FactFinder (map base data)
• http://www.factfinder.census.gov/home/saff/main.html?_lang=en