8/20/2010

1

DATA ACQUISITION andPREPARATIONPREPARATION

GE517 Geographic Information SystemEngr. Ablao

On data input… refers to the process of converting both paper and digital geographic data into a format compatible and digital geographic data into a format compatible and useful to a GIS.Data input is the bottleneck of GIS operations.

data input is slow, expensive and prone to errorcost of data and its conversion is often up to 80 per cent of the total GIS costdata conversion requires careful planning and constant data conversion requires careful planning and constant managementthe GIS is only as good as the data that it has at its disposal

8/20/2010

2

Issues in data inputData sources are varied

t hi d d t l topographic and cadastral mapsaerial photography and satellite imageryfield sheets and census informationetc.

Source data is at different scales and map projectionGIS involves encoding of spatial and non-spatial datag p pAutomation of data conversion only partly successful, but could revolutionize the processMore and more spatial data are becoming available in digital form

Prior to data input

Definition of data requirementsq

Operational planning and estimates

Data preparation

Data Input

Editing

8/20/2010

3

Data Sources

MapsMaps

Data Sources

MapsMaps

8/20/2010

4

Data Sources

Census Census May be spatial in character May be spatial in character Census Census and and Survey Survey DataData

May be spatial in character May be spatial in character if each item has a spatial if each item has a spatial reference, allowing its reference, allowing its location on the Earth to be location on the Earth to be identifiedidentifiedUsually in tabular formatUsually in tabular formatyyExamples: population Examples: population census, employment data, census, employment data, agricultural census data, agricultural census data, marketing datamarketing data

Aerial PhotographsAerial PhotographsAerial Photographs and Aerial Photographs and

First method of remote sensingA ‘snapshot’ of the Earth at a particular instant in timeMay be used as a background or base map for other data in a GIS

g pg pSatellite ImagesSatellite Images

base map for other data in a GIS Provides spatial context and aids in interpretation

Versatile, relatively inexpensive and detailed source of data for GIS

8/20/2010

5

Data Sources

Ground/ Ground/ Using tapes transits Ground/ Ground/ Land Land Surveying Surveying DataData

Using tapes, transits, theodolites, total stations, etc.Used to collect field data such as coordinates, elevations, and distancesData collected are in analog format (written down in format (written down in paper) which still need to be transformed to digital format for use in GIS

Data Sources

GPS GPS Relatively new technique of GPS GPS (Global (Global Positioning Positioning Systems) Systems) DataData

Relatively new technique of field data collectionOriginally designed for real-time navigationCan store collected coordinates and associated attribute information, which attribute information, which may be downloaded directly into a GIS databaseAccuracy ranges from 100 meters to a few centimeters

8/20/2010

6

Categories of Geographic Data AcquisitionCategories of Geographic Data Acquisition

Primary – collected through first-hand observationPrimary collected through first hand observationSecondary – data collected by another individual or organization; most are published data

Primary Raster and Vector DataPrimary Raster and Vector Data

Raster DataRaster Datasatellite imagesscanned aerial photographs

Vector DataLand survey pointsGPS observation data

8/20/2010

7

Methods of Data Input

Methods of Data InputMethods of Data Input1. Raster Data Acquisition

ScanningPhotogrammetryRemote sensing

2. Vector Data AcquisitionManual digitizing Computer assisted digitizingComputer-assisted digitizingField surveyingGPS surveying

3. Attribute Data AcquisitionKeyboard entry

8/20/2010

8

Methods of data inputKeyboard entryManual or operator-assisted digitizingScanningPhotogrammetric methodsSatellite remote sensing systemsFi ld Field surveySatellite positioning systemsOther computer systems

1. Keyboard entryKeyboard entry is primarily used for entering tabular data into the GIS databasetabular data into the GIS database.Typical attribute data sets entered may be:

vegetation classespolygon identifierssoil typestopographic detail

h f f h d bThe form of this data may be:numericalpha-numericlogical

8/20/2010

9

2.a. Manual digitizingConventional digitizing is the manual process of converting geographic map data into digital form.data into digital form.The digitizing process is as follows:

Map is placed on a flat digitizing tablet and affixed using tape.Operator identifies control (tic) points which have known geographic locations. Usually four or more points are identified.Operator digitizes the control points by moving the cursor to each location and then activating the digitizer by pressing a button.then activating the digitizer by pressing a button.Software then performs calibration to enable any features digitized to be transformed into true geographic coordinates.Map features are then digitized by tracing their boundaries and activating the digitizer as required.

2.a. Manual digitizingDigitizing modes

Point mode - the operator activates the button each time they want a location recordedStream mode - a continuous stream of coordinates are recorded during the digitizing process with no need to activate a record button. The rate of sampling can be controlled by time or distance intervals.

Digitizing accuracyS f i i l i t l di iti t Sources of errors: original map errors, internal digitizer errors, operator errors and control point errors.Digitizing accuracy = 1 mm x map scalee.g., Digitizing accuracy = 1 mm x 50,000 = 50 m

8/20/2010

10

2.b. Operator-assisted digitizing

Also known as heads-up digitizing because the Also known as heads up digitizing because the operator works with his head up looking at the screen rather than with his head down following the cursor on a digitizing tablet.It is said to be 10 times faster than manual digitizingIt is said to be 10 times faster than manual digitizing.

2.b. Operator-assisted digitizingThe process is as follows:

A map is scanned into a computer system and resides in raster format.The map is displayed on the screen for the operator to use as a reference.The operator moves the cursor to a position at the start of a line or contour and activates the computer software.pThe computer software takes over and converts the raster data to vector by following the pixels until there is a break.When it stops, operator moves cursor to a new line.

8/20/2010

11

8/20/2010

12

3. Scanning

S i i Scanning is an automated process of converting from paper-based products to digital formatsformats.

8/20/2010

13

3. ScanningIn the scanning process:

i d h h i hi h h b f i a map is passed through a scanning system which has a number of scanning detection units;the detection units “detect” the reflected light emitted from features on the map;the reflected light is converted to a reflectance valuethe image can then be converted to vector format and edited

Scanner types:h hpass-through

drum (normally used for large maps)flat bed (normally used for small maps)aperture card

3. ScanningCharacteristics of scanners

d f l ldata format - gray scale or colorresolution - generally given in dots-per-inch (dpi)scan speed - a function of scanner memory and transfer rate to storage devicethresholding - ability to control the scanner’s sensitivity to various features and colors

Note on scanninggRequires expensive scannersRequires large hard disk spaceRequires powerful workstations

8/20/2010

14

4. Photogrammetry and remote sensing

Both are concerned with Both are concerned with collecting geographic data using remote means.Planimetric and topographic information are usually derived from aerial photographs.Land cover and other information are usually derived from satellite imagery.Scanned aerial photographs and remotely sensed data are in digital raster format already.

5. Field surveys

Traditionally, field measurements are made by Traditionally, field measurements are made by surveyors or field staff who use specialized equipment and procedures for gathering geographic data.Field measurements usually include:

measurements of distance and directionmeasurements of distance and directionmeasurements in both horizontal and vertical planes

8/20/2010

15

5. Field surveys

Measurements can be made with:Measurements can be made with:compasses, transits and theodolites (for direction)tapes, chains and distance meters (for distance)levels (for elevation)GPS (all of the above)GPS (all of the above)

6. Satellite-based positioning systems

GPS (U d S ) GLONASS (R ) d GPS (United States), GLONASS (Russia) and GNSS (Europe) are three civilian satellite positioning systems that are operational at present.Primarily developed for military applications, the American and Russian systems are subject to degradation for civilian usecivilian use.Receivers cost anywhere between US$ 1,000 to US$ 100,000 and give accuracy from 100 m to 1 cm (using sophisticated GPS data processing techniques).

8/20/2010

16

7. Electronic Data Transfer7. Electronic Data Transfer

Used when data is Used when data is already in digital formUsually followed by data conversion, particularly when the transferred data is in a different data is in a different format than what is required

Some local data that are available, include:

7. Electronic Data Transfer7. Electronic Data Transfer

Some local data that are available, include:Municipal boundaries, 1:250,000 (P70,000)100-m contours, 1:250,000 (P50,000)Barangay boundaries (from NSO)

8/20/2010

17

Data Editing

Data EditingData Editing

Errors and inaccuracies during data acquisition and Errors and inaccuracies during data acquisition and input translate into errors in the GISBefore further analyses are made, these errors should be corrected to prevent the errors from

ti t t d i f tipropagating to generated information

8/20/2010

18

Errors in data inputEntity error

missing entitiesincorrectly-placed entitiesdisordered entities

Attribute errorusing the wrong code for an attribute

i llimisspellingsEntity-attribute agreement (logical consistency) error

correct code is linked to the wrong entity

Entity errorsAll entities that should have been entered are present.p

No extra entities have been digitized.

The entities are in the right place and are of the correct shape and size.

All entities that are supposed to be connected to each other are.

All polygons have only a single label point to identify them.

All entities are within the outside boundary identified with registration marks.

8/20/2010

19

Entity errors Attribute errorsPseudo-nodesDangling node

undershootovershoot

Missing labels and too many l b l

Missing attributesIncorrect attribute values

Other problemsProjection changes

labelsSliver polygons“Weird” polygons

j gEdge matchingRubber sheeting

8/20/2010

20

Joining Adjacent LayersJoining Adjacent Layers

Needed when there are multiple map sheets to be Needed when there are multiple map sheets to be usedEnsures that all layers form a continuous geographic database when joined together

Data ConversionData ConversionAfter input and editing of individual datasets, it is p g ,usually necessary to process the data before integrating them all into a single GISProcess of converting data on one form to a more useful format for the specific GIS application p ppOne of the most tedious, time-consuming, and error-prone processes in GIS

8/20/2010

21

Raster to Vector ConversionRaster to Vector Conversion

VectorizationVectorizationConverting scanned raster images to vector features (point, line, or polygons) Results are visually problematic most of the time

Raster Line ThinningRaster Line Thinning

‘skeletonizing’skeletonizingProcess of reducing raster linear features into unit width

8/20/2010

22

Line SmoothingLine Smoothing

Employed to make the resulting Employed to make the resulting vectors more visually appealing

during raster to vector conversion, the l ll j d/ k d results are usually jagged/crooked

(especially for diagonal lines)

Vectorization MethodsVectorization Methods

1 Manual – user selects and picks out features to be 1. Manual user selects and picks out features to be converted

2. Automatic – entire raster image is converted by the computer software without user intervention

3. Semi-automatic – combination of manual point ppicking and computerized line tracing

– produces best results

8/20/2010

23

Raster to Vector ConversionRaster to Vector Conversion

Changing raster images into vector Changing raster images into vector graphicsMay be done manually, automatically,

i t ti llor semi-automaticallyMajor limiting factor is the map quality

Graphical Data EditingGraphical Data Editing

Cl i hi b i Cleaning graphics by removing data conversion errors

8/20/2010

24

Attribute Data TaggingAttribute Data Tagging

Adding attribute data (e g Adding attribute data (e.g., feature identifiers, feature codes, and contour labels) to the graphical data

Vector to Raster ConversionVector to Raster Conversion

RasterizationRasterizationprocess of converting vector data (points, lines and polygons) into raster data (series of cells each with a discrete value) Produces visually satisfactory resultsy yMay be problematic in terms of the attributes assigned to pixels

Most evident along edges/boundaries (partial cells)

8/20/2010

25

Rasterization of LinesRasterization of Lines

Data IntegrationData Integration

Combining data from various Combining data from various sources and in various formats to be able to extract more/better information

8/20/2010

26

Two types of spatial data integration:Two types of spatial data integration:

1. Horizontal Integration ‘tiling’; merging of adjacent data sets

Two types of spatial data integration:Two types of spatial data integration:

2. Vertical Integration –map overlay; stacking of data sets/layers

8/20/2010

27

Examples of Adjustments Required for Data IntegrationExamples of Adjustments Required for Data Integration

Mathematical Transformations – translation, scaling, rotation, or skewingRectification – rearrangement of the location of objects to correspond to a specific

(geodetic) reference systemRegistration – rearrangement of the location of objects of one set so they correspond with

those of another, without referring to a specific reference systemRubber Sheeting – data set/layer is differentially ‘stretched’ so that tic points on the layer g y y p y

are moved to approximate the location of the corresponding ground control points or corresponding tic points in another layer

Edge Matching – employed to properly connect or line-up corresponding features in adjacent map sheets to create a seamless model

translationtranslation differential differential scalingscaling

rotationrotationskewingskewing

ground controlground control

map locationsmap locations

Mathematical Transformations Rubber Sheeting

GIS fileGIS file

8/20/2010

28

Widescreen Test Pattern (16:9)

Aspect Ratio Test

(Should appear circular)

16x9

4x3