MODERN MOBILE MAPPING

8/14/2019 MODERN MOBILE MAPPING

1/12

GPS - IMAGE PROCESSING IN MOBILE

ABSTRACT

Mobile Mapping Systems have been widely used to map transportation

infrastructure features for several years. First generation Mobile Mapping Systems,

however, could not provide positioning accuracy better than one meter, especially in

built-up urban areas. Thus, these data sets have been predominantly used to feed various

GIS systems, primarily concerned with infrastructure inventory and facility management.

With recent technological developments such as improving imaging sensors and,

more importantly, the introduction of ring-laser gyro inertial systems. The prototype-

positioning component of the system is based on a tightly integrated GPS/INS system,

and the imaging component comprises a single downlooking high-resolution, 1K by 1K,

digital camera.

The process of automatically identifying centerlines, extracting image features

and matching them is demonstrated on a variety of data sets, indicating clearly that the

algorithmic performance has reached a sufficient threshold such that human interaction is

no longer required, and consequently, the only limiting condition of the real-timeimplementation is the available computer processing power.

1


2/12

ABSTRACT

INTRODUCTION

SYSTEM CONCEPT

PERFORMANCE OF THE AUTOMATED IMAGE SEQUENCE PROCESSING

HARDWARE IMPLEMENTATION

COLOR SPACE TRANSFORMATION

CENTERLINE EXTRACTION

FEATURE POINT EXTRACTION

STRIP FORMATION

POSITIONING PERFORMANCE

SUMMARY AND CONCLUSION

2


3/12

INTRODUCTION

Direct georeferencing of imaging sensors by means of integrated GPS/INS has

been in the spotlight in the surveying/mapping and remote sensing communities since the

mid-nineties (He et al., 1994; Bossler and Toth, 1995; El-Sheimy and Schwarz, 1999;

Schwarz, 1995).

One reason is that the primary driving force behind this process is a need to

accommodate the new spatial data sensors, such as LIDAR or SAR (airborne systems).

The second reason is that a substantial cost decrease, a possibility of data

reduction automation, and a short turn-around time are the most attractive features

offered by this technology

The main features of the system are the high image capture rate, the online use of

navigation estimates, and the on-the-fly image and stereo data processing. From a

navigation standpoint, the post processing of GPS/INS data provides more accurate

orientation as a benefit of forward and backward trajectory processing and precisely

synchronized timing information.

The high-accuracy GPS/INS/CCD system designed for monitoring linear highway

features is based on the concept of tight sensor integration, combining post-processing

with real-time image processing. The two primary components of the mobile mapping

system currently being implemented are precise navigation and digital imaging; both

allow for flexible and optimal system design, leading potentially to near-real time overall

data processing.

3


4/12

SYSTEM CONCEPT

Mobile Mapping Systems are built on the concept of combining high-performance

georeferencing with electronic imaging on a moving platform. MMS systems have been

using image sequences for a long time since it is an essential part of the concept.

However, progress toward the automation of the image sequence processing has been

slow for two reasons.

First, economy; the actual feature extraction represents approximately less than

20% of the overall cost, and therefore, the financial motivation is weak.

Second is the varying image scale, which makes the object recognition task quite

difficult in the feature-rich object space.

Traditional MMS systems work with forward- or side-looking cameras, while our

system uses a down-looking camera. This way the image scale changes very slightly and

there is an almost constant scale along the vehicle trajectory. The object contents of the

images are rather simple and predictable, such as the line marks, the primary interest to

us, surface texture variations, cracks, potholes, skid marks, etc.

Figure 1 shows the generic model of the dedicated centerline mapping system.

HARDWARE IMPLEMENTATION

The prototype of the integrated GPS/INS/CCD system designed for precision

monitoring of the highway edge- and centerlines comprises two dual-frequency Trimble

4000SSI GPS receivers and a medium-accuracy and high-reliability strapdown Litton

4


5/12

LN-100 inertial navigation system, based on Zero-lockTM Laser Gyro (ZLGTM) and A-

4 accelerometer triad (0.8 nmi/h CEP, gyro bias 0.003/h, accelerometer bias 25 g) .

Estimation of errors in position, velocity, and attitude, as well as errors in inertial

and GPS measurements, is accomplished by a 21-state centralized Kalman filter that

processes GPS L1/L2 phase observable in double-differenced mode together with the

INS strapdown navigation solution.

The estimated standard deviations are at the level of 2-3 cm for position

coordinates, and 5-7 arcsec and ~10 arcsec for attitude and heading components,

respectively. The imaging component is built around the Basler A201 camera, Kodak 1K

by 1K colour CCD with 9.07 mm by 9.16 mm imaging area (9-micron pixel size) and 15

images per second acquisition rate (15 Hz), which allows for 60% image overlap at

normal highway speed.

Figure 2 shows the system design, including the sensors, dataflow, processing steps.

PERFORMANCE OF THE AUTOMATED IMAGE SEQUENCE PROCESSING

5


6/12

To assess the feasibility of automated line extraction with 3D positioning and

consequently its real-time realization, a rich set of the potential image processing

functions was developed in a standard C++ programming environment.

Figure 3 shows the overall dataflow and processing steps, which will be

illustrated in more detail later. In short, the real-time image processing is feasible due to a

simple sensor geometry and the limited complexity of the imagery collected.

COLOR SPACE TRANSFORMATION

Color images are preferred over monochromatic ones. Figure 4 illustrates various

cases, including an extreme situation where the yellow solid lines are hardly visible in the

B/W image. A simple histogram analysis in the RGB (Red, Green, Blue) color space

easily reveals two peaks in the red and green channels representing the yellow color of

the centerline.

6


7/12

Although there are many image processing algorithms working with various color

data (multichannel gray-scale imagery), the great majority of the core functions work

only on simple monochrome image data. Therefore, if possible, a color space conversion

is desirable, in other words, moving from the 3D color space into one dimension, a color

direction, which shows the best possible separation for the objects we want to distinguish.

Since the quality of the extracted centerlines still show visible differences, a

filtering process has been implemented to remove this dissimilarity. The output of the

median filter is converted to a binary image. End results show no significant difference

between the centerline segments extracted from the very different images as illustrated in

Figure 6.

7


8/12

CENTERLINE EXTRACTION

After the RGB to S transformation and filtering, the geometry of the centerlines is

extracted from the binary images For a raster line, its centerline is of primary interest.

Skeleton is more like a one-pixel-width line, while centerline can be used to express a

one-pixel-width line in both raster and vector lines.

The skeleton is then extracted by shrinking the raster line from its boundary in all

directions until the one-pixel-width eight-connected line remains. In the medial axis

transformation method (Montanvert, 1986), the discrete medial axis pixels are the local

maximum of a transformation value.

A robust recursive filtering technique can eliminate noise such as gaps, although

most of the gaps and grey-scale irregularities already have been removed during the color

space transformation, as well as provide segmentation for multiple centerlines such as

double solid lines. Figure 7 depicts the results of this processing step.

Once boundary points are extracted, a line-following routine can generate the boundary

lines, which are subject of further cleaning such as removing irregularities by applying

geometrical constraints. In the final step, the midpoints are computed and the centerline is

extracted as shown in Figure 8.

8


9/12

FEATURE POINT EXTRACTION

To achieve the highest accuracy possible, the 3-dimensional centerline positions must be

obtained from stereo imagery. Knowing the camera orientation, both interior and exterior,

and the matching (identical) entities between the 2-dimensional centerlines, the 3-

dimensional centerline position can be easily computed.

I denotes the smoothing operation on the grey level image I(x, y). Ix and Iy indicate the x

and y directional derivatives respectively. Figure 9 depicts feature points extracted

around the centerline region from overlapping images.

9


10/12


11/12

POSITIONING PERFORMANCE

The multisensor system calibration is a key task to achieving the ultimate

accuracy of the given sensors. System calibration is defined here as the determination of

spatial and rotational offsets between the sensors as well as imaging sensor calibration.

Continuous calibration of the INS system is provided by GPS and thus is very dependent

on GPS anomalies such as satellite signal obstructions, multipath, interference, etc. The

effective ground pixel size was about 2-4 mm. Figure 11 shows the calibration range, and

the road area with control points.

A comprehensive analysis of the system calibration and positioning performance

are available in (Grejner-Brzezinska and Toth, 1999, and 2000). Using these boresight

parameters, the comparison of ground coordinates obtained by the photogrammetric

methods from the directly oriented imagery to the GPS-measured ground truth delivered

the ultimate accuracy performance of the overall system. The control points used in this

test were GPS-measured with an accuracy of ~1.5 cm per coordinate and were located

about 18 m from the perspective center of the camera.

SUMMARY AND CONCLUSION

This paper introduced a concept of an all-digital mapping system designed for

precise mapping of highway linear features. The test results presented here indicate that

an integrated, land-based system supported by a medium to high quality strapdown INS

11


12/12

and dual frequency differential GPS offers the capability for automatic and direct sensor

orientation of the imaging sensor with high accuracy.

In addition, the concept of real-time extraction of highwaylinear features such as

centerlines was demonstrated. The overall system performance was extensively tested by

a prototype positioning module (for more details see Grejner-Brzezinska and Toth, 1999,

and 2000), while the feasibility of automated feature extraction was evaluated only by

simulations.

12

MODERN MOBILE MAPPING

Documents

Transcript of MODERN MOBILE MAPPING