GIS Utility Data: The Good The Bad & The UGLY

Alan Jacobson, PE

704-651-5538

GIS a Tool for Managing & Analyzing Public Utilities

Most common use of GIS for utilities?Producing and Maintaining a set of System Maps (A task which is

perhaps only marginally easier in GIS than in earlier CAD based solutions).

Using GIS to manage or analyze a utility requires much more thought about the GIS design and potentially much more data acquisition than simple map makingResearch is required to determine how the data will interact with desired third party software packages for asset management, hydraulic modeling, and work order management.Planning for other common or occasional analyses and including any required data is best performed when implementing the GIS design well prior to starting the data gathering and data input processes.

Building or maintaining GIS data to meet complex needs may require:

Knowledge of the utility’s actual behavior

Knowing what information is available or could be made available for each asset

Understanding Data processes used in one or more third party software packages

Understanding of the advanced uses that may be needed at later dates

Know Your Data History

Many GIS Datasets were originally built or converted from data that was specifically used to create a map at only one scale.Each feature included may have been intentionally spatially altered and or scaled hundreds of times its true size so that it was clearly visible at that scale.Data from these sources frequently create GIS conflicts with other features and or normal construction standards.More recently GPS technologies have created an easy method of field capture of GIS features. But even with GPS some features may be closer together vertically or horizontally than can be determined by the GPS equipment being used.

Accuracy

After a GIS is built to “make maps” or is built without consideration of required accuracies don’t be surprised to find out that many uses require a specialized GIS conversion or repair process or in the very worst cases a complete rebuild of the entire dataset

GIS does not stand for “Get It Surveyed” nor does Accurate imply that a Survey is needed

Knowing precisely where key features are located can be very helpful

GIS Utility Locations: PrecisionAerial OrthophotosSub Meter GPSRTK GPSSurvey Levels

36” RCP@ 0.045 %

100 Ft

0.045 Ft

Rules & Exceptions

Edge-Junction cardinality ruleMultiple Features at same XY location

Lines & Point Features in Simple Edge GIS Networks

1

2

4 5

Nodes (Pipe Ends) = Pipes Connected +1 – Loops Completed

For Large Water System Networks: Nodes/Pipes < 1

7

3

4 5

6

1 6 2

3

AssumptionsTo Be or Not to Bea Point FeatureCorrect Answer Depends on the use:Distribution MainTreatment PlantPump StationModelingDesignWork Order

CAD Drawings & System Maps

CAD Drawings and System Maps

CAD Drawings and System Maps After Conversion

From CAD Drawings and System Mapsto Comprehensive Water, Sanitary, & Storm Sewer GIS complete with Asset Management & System Hydraulic Modeling

1440 Miles of Water Main = 130,000 Water Features

1470 Miles of Sanitary Sewer = 70,000 Sewer Features

844 Miles Stormwater =102,000 Storm Features

90,000 Water Customer Locations

CAD Drawings and System Maps Can Produce Ugly but Accurate GIS Data

50 Feet

CAD Symbols were drawn for a known plotting scale on paper. Placement required exaggerating the actual planned or as-built location of the features to make the symbols more legible

Improperly handled CAD Symbols can be the dominant linear GIS feature and if they contain hatch patterns can result in an explosion of lines

Frequently CAD line work was removed to make the symbol more legible

CAD Drawings and System Maps - Symbols

20 Ft

15.5 Ft

2.7 Ft

Removed

or Trimmed

Why do I have hundreds of thousands of short pipes?Our city has 250 miles of Water Main, Why do I have 650 miles of pipe in GIS?

Bad Information from Any Source Makes BAD GIS

28% Slopes

Perhaps the Surveyed Stormwater Grate Inlet is not 74 feet Deep

Negative Slopes

4111691169000Hydrant_2004_NAD83_New

6298210981981Hydrant

16234036100523923397Water Valve

021051621Water Manhole

10273048225273Meter

213121012Water Vault

011000Water Treatment Plant

072055Water Tank

010011Pump

03582-1357356Cap

040044Increaser

052025052Reducer

823633187827543632Fitting

Duplicate Features

Total Features

Incorrect Position

Required Pipe Splits

Snapped to Endpoint

Snapped to LineGIS Feature Class

CAD Drawings and System Maps

What visually looks like 1 feature is often a collection of features or a series of duplicate and near duplicate GIS features

TEE1600571263.8373029231237055.06892079

CROSS1601571254.5370789141237039.40684078

CROSS1518571254.5370789141237039.40684078

TEE1602571232.6050468931237038.98853678

TEE1519571232.6050468931237038.98853678

TEE1603571239.37419891236934.48370468

TEE1520571239.37419891236934.48370468

TEE1604571262.1310629211236928.55397668

TEE1521571262.1310629211236928.55397668

TEE1513571315.8027429711236255.69074405

TEE1597571315.8027429711236255.69074405

TEE1514571306.0696229621236255.27474405

TEE1598571306.0696229621236255.27474405

901599571279.0687909371236253.94252005

901515571279.0687909371236253.94252005

TEE1512571336.843126991235959.17925577

TEE1596571336.843126991235959.17925577

TEE1700570064.0892058051235691.00927152

TEE810570064.0892058051235691.00927152

TEE1511571415.6458150641235273.12869513

TEE1595571415.6458150641235273.12869513

TEE1594571945.2376235571234019.56248597

TEE1510571945.2376235571234019.56248597

TypeOBJECTIDYX

Fitting Duplicates

Line1732576878 7157721521245222 2300164Line803572862.5521844111245042.71436823Line2723572862.5521844111245042.71436823Hydrant2721572905.5105204511245038.80908823Hydrant801572905.5105204511245038.80908823Line802572873.4869684221245036.46592023Line2722572873.4869684221245036.46592023Hydrant1743576891.4074841641244888.17920009Hydrant2766576891.4074841641244888.17920009Line1763576898.9735641711244877.37036808Line2767576898.9735641711244877.37036808Line1735576908.650108181244797.925632Line2765576908.650108181244797.925632Hydrant1713574057.4134975241244567.26783979Hydrant2754574057.4134975241244567.26783979Line2755574063.245689531244553.27052778Line1714574063.245689531244553.27052778Line1712574065.9674815321244372.08294361Line2753574065.9674815321244372.08294361Line1034569643.8462614141244188.21657544Line73569643.8462614141244188.21657544Hydrant1033569649.2801174191244140.51891139Hydrant72569649.2801174191244140.51891139Line1036569713.4849174791243184.2063345Line75569713.4849174791243184.2063345Hydrant1035569707.5930774731243183.2243185Hydrant74569707.5930774731243183.2243185Hydrant42570214.9000859461242368.56524574Hydrant1011570214.9000859461242368.56524574Hydrant1031574775.4466501931241878.52774129Hydrant67574775.4466501931241878.52774129Line1030574760.2725061791241838.52338925Line52574760.2725061791241838.52338925Line1890564049.6950882041241202.04082866Line13564049.6950882041241202.04082866Line1032569229.9569810281241055.18489252Line68569229.9569810281241055.18489252Line1916566668.0556666421233927.10066188Line1566668.0556666421233927.10066188Line5567292.3664512241233794.62782976Line1917567292.3664512241233794.62782976

TypeOBJECTIDYXWater Valve Duplicates

Duplicate Features:

Pipes Not Split or

Connected

Pipes Not Split or

Connected

Does GIS agree with Construction drawings?

Intersecting & Non-Intersecting Pipes.

Pipes Not Split or

Connected

Does GIS agree with Construction drawings?

Intersecting & Non-Intersecting Pipes.

Pipes Not Split or Connected

No construction drawings

OK Pipes Not Split or

ConnectedPipe Split

@ Tee

Pipes Split @ Tee

After GIS compared to original construction drawings?

Intersecting & Non-Intersecting Pipes.

Pipe Split @Valve

Pipes Split @Tee

Pipe Split

@Valve

Hydrants GPSed by city fire fighters

Many original GIS positions were much better than the Newly GPSed locations

Not all of the GPS work included getting out of the fire truck

45 Ft

Move the New Hydrant to the old position or to the position visible on Orthos, Check Old Hydrant for missing Data, reconnect and reposition Pipes, Valves, and Fittings as required

Pipes not Split, Symbols not Rotated

Pipes not Split

Pipes not Split

Pipes not Split

Symbols not Rotated

Pipes not Split

Pipes Split Symbols Rotated

Pipes Split

Pipes Split

Pipes Split

Symbols Rotated

Symbols Rotated

For a wide number of tasks (including system maintenance, map production, and modeling) it is important to be able to quickly distinguish all mains verses all service laterals.

Shown in dark Purple is a single CAD Sewerline. It is labeled as 24” and further West as 8”.

All line work in GIS was trimmed at the map “grid” edgesEdge matching and Reconstruction required for all features that cross map boundaries: Common in CAD and GIS Data (ArcInfo LIBRARIAN tiles)

CAD Drawings and System Maps – Valves Not Connected to Pipes

CAD Drawings and System Maps – Everything as one Shape FileSeveral thousand CAD features exported as a single ESRI Shape file “The proposed

Sewer Interceptor”. Less than 1% of the features were related to the interceptor.

Arcs and Symbols From CAD to Shape File to GeoDatabaseShape files cannot store curves. When true curves from

CAD or ArcGIS are converted to shape files, a single line with an arc segment becomes a line with multiple straight segments and vertices along the line at roughly 1 vertex point per 2 degrees of arc. Any curved pipes or imported “point / line” symbols containing circles or arcs are subsequently made into fairly complex and difficult to edit GIS features. This is BAD!!

If receiving data from unknown sources look for signs of Bad processes

Somewhere along the conversion path processes that split multiple part line segment features into individual short line features.

Pipes, Arcs, and Symbols

How many waterlines are shown along Saddlebrook Dr?54

Water System Arcs and Symbols

How many waterlines are shown along GlenEagles Lane?118750181

CAD Drawing TechniquesCAD system-Valves, Fittings, and Hydrants were drawn once and then stored as a Symbol or “Block”

Unfortunately routines are often used in CAD which convert blocks back into the original symbol’s arcs, circles, and lines. For this Utility all Hydrants and Valves in CAD were present on the water main layer as lines and Arcs

Can Good CAD Information be Converted to BAD GIS?

Shown as Red Dots are pipe endpoints that should be snapped to Sewer ManholesRed Crosses represent other problem locations.

Missing Features?Take advantage of ESRI’s Edge-Junction Cardinality rule capabilities or other Rules and Tests that are available using ArcObjects

Missing Hydrant Laterals

Missing Water lines

Attributes in GIS based on the nearest CAD text string

Spatially tied pipe to even when that distance is thousands of feet away

Pipes labeled as ARVs

Pipes need attributes not a string of text that must be read and interpreted prior to use

Sewer and Water Shapes compared to Surveyed Water and Sewer Locations

Hydrants are in Wrong Locations 58 Feet from older Surveyed Data

Water lines shown over 30 feet from surveyed valve locations

Water lines Not shown in New GIS shape files

Is it Good if Its Missing?

With Diameter Percent

With Material Percent

With Date Percent

Date Range

257 2.0% 132 1.0% 964 7.5% 1979-2002

SourceDate

Modified Pipes Feet MilesCatawba County Waterlines 2/3/2003 12,789 4,228,916 800.93

Position Artifacts

Model Quality GIS Utility Data

Despite misconceptions, GIS Data can be Model Quality even when Key Features have Significant Positional errors (30 ft or more) and when lengths of connecting pipes or cables are unrealistic

IF:Correct number of features and attributes are presentConnectivity between features has been correctly established

Model Quality does not mean perfect. Model quality simply means that the results of using the data for modeling produces answers that are useful and accurate within desired limits.

The UGLY Truth of BAD Data

1.6 Ft Gap

Test the DataTesting Utility GIS layers for errors and or compatibility with

desired processes or software packages can determine if the system is:GoodBad - Unsuitable for desired uses

UGLY - but otherwise Ok for several usesAs long as you Don’t send Robbyto find the Hydrant @1,254,783.034

497,904.935

Fire Hydrant Not Found @

Coordinates, Do You want me to

Dig Deeper?

Questions?

Alan Jacobson, PE

704-651-5538 AlanJacobsonPE@Yahoo.com

Test your Utility Features