Deep Dive Into Spatial Data

Isaac KunenSenior Program ManagerMicrosoft SQL Server

BB24

Preface

This talk is for anyone who wants a better understanding of SQL Server’s spatial support I’ve culled a few of the most interesting topics from

the message boards, my blog, etc.

Caveat: This is a 400 level talk I will assume some familiarity with SQL Server

spatial, but...

Bonus: Lots of pretty pictures

An Aggressive Agenda

Precision and Robustness The Sphere is not the Plane Extending Spatial Index Usage A Look at Where We’re Headed

Precision and Robustness

Let's Intersect Some Lines

Demo!

What’s Going On?

The actual intersection point is (2/3, 2) How does the computer handle this? Possibility: rational number type

Prohibitively expensive Real answer: limit precision

But, the same value, calculated different ways, may have different results

Spatial Isn’t Special

declare @u float = 1000

declare @v float = -1

declare @w float = 1.0001

print 'equal'

else

print 'not equal'

select (@u * (@v + @w)) -- 0.099999999999989

select ((@u * @v) + (@u * @w)) -- 0.100000000000023

-- this one!

if (@u * (@v + @w)) = ((@u * @v) + (@u * @w)) (@u * (@v + @w)) ((@u * @v) + (@u * @w))

Upshot

We have to learn to deal with this For many operations, we don’t need to worry.

Region queries, computing distances, etc.

If we depend on an exact result...where @line.STIntersects(@point) = 1

...then we should ask a fuzzy query instead:where @line.STDistance(@point) < ε

Brief Interlude

Q: How do we tell what’s in/out of a polygon?A: Cast a ray to infinity and count the number of

times it crosses the boundary

Robustness: Small ErrorsBig Problems

How We Perform Calculations

1. Snap to an integer grid2. Perform calculations using exact integer

arithmetic3. Convert back to floats

This process will cause small errors (within tolerance) ...

...but avoids the big ones

Geometry on the Grid

f(...)

The Sphere is not the Plane

What is an Edge?

This is not just a philosophical question:

In any case, on the plane this is pretty clear Curves like these are called geodesics

Locally path minimizing curve

?

Edges on the Sphere

Great CirclesNice properties

Geodesics Usually unique... ...not always

?

?

?

Edges on an Ellipsoid

Geodesics? Ambiguous Unnatural (?)

Great Elliptic Arcs Usually unique Natural (?) Very nice

computationally Usually close to a

geodesic

Another Brief Interlude

Q: How do we tell what’s in/out of a polygon? We cannot cast a ray to infinity—there isn’t one

A: Have to use ring orientation instead

Let Me Disabuse You of a Few Notions

Nonsense (!) Clockwise/

Counter-Clockwise Interior/Exterior

rings Sense

Left-/Right-Hand side

Extending Spatial

So We Have This Spatial Library…

We now have this wonderful spatial library inside SQL Server But wait, there’s more!

We ship the library separately CLR type + unmanaged DLL Available for separate download/install Can code against it through C#, VB, etc.

We can use it to extend spatial in various interesting ways SQL CLR Sink/Builder

Sink API

IGeometrySink {

void SetSrid(int srid)

void BeginGeometry(OpenGisGeometryType type)

void BeginFigure(double x, double y, double? z, double?

m)

void AddLine(double x, double y, double? z, double? m)

void EndFigure()

void EndGeometry()

}

Using the API

0:

BeginGeometry(Polygon)BeginFigure(0,0,null,null)AddLine(0,10,null,null)AddLine(10,0,null,null)AddLine(0,0,null,null)EndFigure()BeginFigure(2,2,null,null)AddLine(2,8,null,null)AddLine(8,2,null,null)AddLine(2,2,null,null)EndFigure()EndGeometry()

Polygon:

SetSrid(0)

Sink/Builder

Three easy pieces: IGeometrySink interface that allows for the

description of any geometry instance Populate(IGeometrySink) method on

geometry takes a sink and calls methods to describe itself

GeometryBuilder is a special IGeometrySink that constructs whatever instance is described

These repeat for geography These can be pipelined

Creating a SwapXY Function

Demo!

SQL Server Spatial Tools

The sink/builder API is a very powerful way to extend our spatial functionality

We’re slowly adding new functionality in the SQL Server Spatial Tools project on CodePlex Free stuff!

Partial list of functionality: Densification Affine transforms Linear referencing Simple projections

Index Usage

A: Usually because the index isn’t being used.Q: How do I tell?A: SELECT * FROM T WHERE g.STIntersects(@x)

= 1

Q: Why is my Query so Slow?

NO INDEX

INDEX!

Spatial indexes can be forced if needed.

SELECT * FROM T WHERE g.STIntersects(@x) = 1

Hinting the Index

WITH(INDEX(T_g_idx))

Plan choice is cost-based QO uses various information, including cardinality

When can we estimate cardinality? Variables: never Literals: not in this case Parameters: yes, but cached, so first call matters

But Why Isn't My Index Used?

DECLARE @x geometry = 'POINT (0 0)'SELECT *FROM TWHERE T.g.STIntersects(@x) = 1

SELECT *FROM TWHERE T.g.STIntersects('POINT (0 0)') = 1

EXEC sp_executesql N'SELECT * FROM T WHERE T.g.STIntersects(@x) = 1', N'@x geometry', N'POINT (0 0)'

Futures

Where are we going?

2D Map components coming to Reporting Services Eliminating hemisphere limitation Improving indexing, both plan choice and perf Additional methods

3D Looking at model-independent strategies

Raster

More info/Pointers

Where to get More Information

Go see the VE deep dive: BB10, Thursday at 1:45

My main sources of inspiration for the talk: http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=1629&SiteID=

1 http://blogs.msdn.com/isaac/ http://www.codeplex.com/sqlspatialtools

Other places you should know: http://www.microsoft.com/sqlserver/2008/en/us/spatial-data.aspx http://www.sharpgis.net/page/SQL-Server-2008-Spatial-Tools.aspx http://www.codeplex.com/ProjNET http://www.geoquery2008.com/ http://johanneskebeck.spaces.live.com/ http://blogs.msdn.com/edkatibah/

Evals & Recordings

Please fill

out your

evaluation for

this session at:

This session will be available as a recording at:

www.microsoftpdc.com

Please use the microphones provided

Q&A

© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market

conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Encore 1: Calculating on the Globe

Gnomonic Projection

Flattens to a plane Projects from the

center Edges are projected

exactly to lines Now operate in the

plane! Distortion grows

with distance... ...so the useful area

is limited

Extending the Projection

Use more planes Start with half a

tetrahedron Introduce

“fake” vertices on the folds

Flatten the tetrahedron to a single plane

Now, operate on the plane!

Our Workflow

Select a projection If the object is small enough, use a single plane If it’s too big, choose a tetrahedron We use the same bounding cap we’ve

described Operate in the plane Project back to the sphere

But...

What if the object is larger than a hemisphere? We can project to a full tetrahedron We then flatten to two planes

The projection is not particularly difficult Have to work with two planes

Sorting out the interface during the inverse projection is harder... ...but it’s coming!

Encore 2: Envelopes

Are Bounding Boxes Nonsense?

Naive Algorithm:

declare @_p geography = 'POINT (0 45)'

declare @A geography = @_p.STBuffer(1000000)

declare @B geometry =

geometry::Parse(@A.ToString())

declare @C geometry = @B.STEnvelope()

declare @D geography =

geography::Parse(@C.ToString())

Can Bounding Boxes Work?

Naive bounding box clearly doesn’t work We can patch this up a bit What happens over the poles, though?

A bounding box of any sort seems to be rather unnatural—to me, at least If you come up with a natural, simple, general

definition, please let me know If you’re projecting the data, bounding boxes

make sense in the projected space

Bounding Circles

We really want a simple proxy for the object We give bounding circles: center point and angle Guaranteed to cover, but not to be tight

Appendix

/**

* This class implements a geometry sink that will swap the x and y coordinates of the instance.

* Note that this will invert the orientation of all rings in the instance.

* It directs its output to another sink, and can therefore be used in a pipeline if desired.

*/

public class SwapXYSink : IGeometrySink

{

private readonly IGeometrySink _target; // the target sink

public SwapXYSink(IGeometrySink target)

{

_target = target;

}

// Just pass through without change.

public void SetSrid(int srid)

{

_target.SetSrid(srid);

}

// Just pass through without change.

public void BeginGeometry(OpenGisGeometryType type)

{

_target.BeginGeometry(type);

}

// Each BeginFigure call will just swap the x and y coordinates.

public void BeginFigure(double x, double y, double? z, double? m)

{

_target.BeginFigure(y, x, z, m);

}

// Each AddLine call will just swap the x and y coordinates.

public void AddLine(double x, double y, double? z, double? m)

{

_target.AddLine(y, x, z, m);

}

// Just pass through without change.

public void EndFigure()

{

_target.EndFigure();

}

// Just pass through without change.

public void EndGeometry()

{

_target.EndGeometry();

}

}

Full SwapXY Sink Code

Creating a SwapXY Function

Create a pipeline, last sink first:

public static SqlGeometry SwapXY(SqlGeometry source)

{ SqlGeometryBuilder b = new SqlGeometryBuilder(); SwapXYSink swap = new SwapXYSink(b); // Could chain more sinks... source.Populate(swap); return b.ConstructedGeometry;}

This can now be registered and used in SQL Server