U.S. Department of the InteriorU.S. Geological Survey

Aparajithan Sampath, Don Moe, Jon Christopherson, Greg Strensaas.

ASPRS, April 2010.

Geometric Evaluation and Validation of Aerial and Satellite Data using Sioux Falls

Geometric Test Range

2

OutlineOutline Overview of Geometric test ranges Sioux Falls range

Location, dimensions etc Accuracy assessment of Sioux Falls range

Two methods for orthoimagery product validation Using automated image-to-image analysis for rapid and

repeatable assessment of accuracy

3

Geometric Test Ranges: Part of Data Provider CertificationGeometric Test Ranges: Part of Data Provider Certification USGS to develop validation ranges across US

Aerial imagery validation Assess & certify aerial product accuracy over approved range 5-6 needed to “go operational”

Satellite data validation Ranges may be used for one or the other, or both LiDAR data validation: In the near future

Goal was to have quantified imagery over range extents

Use image-to-image analysis for rapid and repeatable assessment of accuracy

(May be reducing the wall-to-wall requirement)

4

Range LocationsRange Locations

Sioux Falls

Rolla, MOPueblo, CO

?

?

?

5

Sioux Falls RangeSioux Falls Range Orthoimagery covers

Minnehaha and Lincoln counties

85 km (53 miles) N-S 54 km (34 mi) E-W

30cm, 15cm, and 7.5cm(12in, 6in, & 3in)

LiDAR coverage at 1.4m posting

Orthoimagery and LiDAR data collected jointly by the USGS, Minnehaha and Lincoln Counties

6

Range Data Types & Control PointsRange Data Types & Control Points12” Data over Minnehaha County

6” data over parts of Minnehaha and Lincoln counties

12” Data over Lincoln County

3” data over the city of Sioux Falls

RTK Survey points on photo-identifiable features (Red dots)

Sanborn survey points (Black dots)

3320 km² total of 30cm (1ft) data over Minnehaha and Lincoln Counties

760 km² of 15cm (6in) data over Sioux Falls and surrounding areas

115 km² of 7.5cm (3in) data over Sioux Falls

7

Accuracy Assessment of Ortho ImageryAccuracy Assessment of Ortho Imagery Task was to determine the suitability of available

imagery to be used as reference imagery GPS-RTK Survey

Pick photo identifiable points from orthophotos Identify base station locations Actual survey

Accuracy Assessment Compare coordinates from Orthophotos and RTK survey Accuracy Analyst ™ used to measure and compile results

into report.

8

Accuracy of SurveyingAccuracy of Surveying The RTK survey process was

tested by surveying known points near Joe Foss Field and EROS

Maximum error was 3.6 cmor 1.43 inches

Perfectly acceptable for assessing 3”, 6” and 12”data products

9

RTK SurveyRTK Survey A total of 112 points were

surveyed 56 over Sioux Falls

covered by 3” GSD data Another 13 in the 6” GSD

region Rest in the 1’ GSD region

over Minnehaha and Lincoln Counties

10

ResultsResults

Orthoproduct

RMSE CE 95

7.5cm (3”) 8.5 cm (3.4in)

1.12 pixels

20.4 cm (8.0in)

2.68 pixels

15cm (6”) 11.6 cm (4.5in)

0.76 pixels

29.6 cm (11.4in)

1.9 pixels

30cm (12”) 22.9 cm (9.0in)

0.75 pixels

56.1cm (22.1in)

1.84 pixels

11

RMSE and CE 95 PlotsRMSE and CE 95 PlotsRMSE and CE 95

0.280.38

0.750.67

0.95

1.84

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

0.25 0.5 1

GSD in feet

Err

or

in f

eet

RMSE

CE95

RMSE and CE 95

1.12

0.76 0.75

2.68

1.9 1.84

0

0.5

1

1.5

2

2.5

3

3.5

4

0.25 0.5 1

GSD in feet

Err

or

in P

ixel

s

CE 95

RMSE

12

Geometric Assessment of GeoEye-1 Over Sioux Falls Test RangeGeometric Assessment of GeoEye-1 Over Sioux Falls Test Range

GeoEye-1 was launched in September 2008 Resolution – 0.41m at nadir for Panchromatic band Data provided to USGS has been resampled to 0.5m 1.65m for multispectral (Not Assessed)

15 x 15 km Single point scene

13

Geometric Accuracy Validation: IAS - I2IGeometric Accuracy Validation: IAS - I2I Select uniformly random points over reference

imagery, and determine corresponding locations in the search image using cross correlation

Compare coordinates between search and reference images.

For high resolution data, randomly selected points may prove problematic due to:

Shadows Look angle may render some points invisible Too many features Too low contrast Amount of data

14

I2I MatchingI2I Matching

Photo-Identifiable point P

Moving search template that determines cross correlation matching measure

Reference image chip

Error in Search image

P’s estimated location in study image

P’s actual location in study image determined from Image matching

Search image chip

15

Geometric Accuracy Validation: Combining Control Points, Open Source ToolsGeometric Accuracy Validation: Combining Control Points, Open Source Tools

Combine Geospatial Data Abstraction Library (GDAL) and IAS-I2I and Control points

GDAL used to locate and cut image chips around the control points (Reference and search chips)

Error corrections for the control points incorporated Reference chips are actually more accurate than the

reference image I2I measurements carried out between reference chip

center and the search chip

16

AdvantagesAdvantages Errors in reference image are eliminated Reference image chips as accurate as the GPS survey Procedure reduces/eliminates the need to rectify,

resample and creates large mosaics of reference images (around 200 of them)

Technique can potentially validate images in different coordinate systems and resolution

Image reference chips can be reused against other images

17

ResultsResults

45 control points used to calculate statistics

Pixels Meters

Line Sample Line Sample

Mean 2.77 0.11 1.38 .05

Standard Deviation 5.62 4.28 2.81 2.14

RMSE 6.27 4.29 3.13 2.14

18

Going ForwardGoing Forward Instead of wall-to-wall imagery, collect image

chips over previously surveyed locations The locations can be photo-identifiable points Planned use of a UAV with an attached camera Plan will generate high accuracy image chips of

GSD 3cm-50cm Use Image chips from NAIP data to validate

lower resolution products If successful, this can significantly reduce costs

19

Going ForwardGoing Forward Small area vertical

photogrammetry Use camera mounted on a

UAV to take vertical photographs over area of interest (mostly check point locations)

Camera could be short focal length COTS

Collect 3-4 RTK points over the area

Georectify the image

20

AdvantagesAdvantages No need for Wall-to-Wall coverage Small image (<100ft * 100ft) size will make it

more manageable Images can be collected from carefully selected

locations High accuracy ranges can be built with reduced

costs and time

21

SummarySummary Sioux Falls range is ready as Geometric validation site for high

resolution satellites and Aerial images Visionmap A3 system was flown over the range in August. GeoEye-1 was analyzed using the range, as well as RapidEye.

WorldView-2 data will be analyzed in the near future. The combined use open source tools (GDAL, IAS-I2I) to handle

large datasets is promising and efforts to improve the I2I tool will be investigated

Two More Ranges in early development Rolla, MO Pueblo, CO

Both have some existing imagery But not high enough resolution Developing new ideas for obtaining image chips