Andrew ZolnaiCambridge UKaiz@zolnai.ca

Shipping and weather data CLIWOC ships captains logs 1662-1855

A Question on Joining Look-up Tables

Image: Royal Museum Greenwich

Historic Climatology

• Ships captains logs data from 1662 to 1855• Ships and routing details and location• Weather recording of almost 120 parameters• Release 1.x (2004) and Release 2.x (2007)

Location Metadata• First added to Esri file geodatabase – The complete (weather and location) dataset has

almost 290,000 points, and is almost 250Mb database– The locations were extracted into a geodatabase (70

Mb), then into a compressed file geodatabase (10 Mb)• Post map package for Release 2.0 – layer files showing them in 25-year time-slices– and for Captains Cook and de la Perouse

• Posted as ArcGIS Online map service– Use time stamps to aggregate layers– But stock server has performance issues– So Filter data by nationality and decade– Limits web fetches to 1000s not 100,000s

(Visio Pro data model reverse-engineeredfrom Dbase extracts of File Geodatabase)

Climate Metadata

• CLIWOC posted all shipboard data it had – from British, Dutch, French and Spanish sources– with look-up tables for each different source

• Merged wind force and direction data almost doubles the number of feature classes as observed– Started with Wind Force– Then joined Wind Direction– Finally merged Force & Direction

• Map package and layer template

(Dbase extract 8 char limit:add prefix of CLIWOC21_Features_ to all except look-ups Lookup_{ }_WindDirection_)

(Dbase extract 8 char limit:add prefix of CLIWOC21_Features_ to all except look-ups Lookup_{ }_WindForce_)

Wind Direction from each Maritime Agency Look-up to Merged

Wind Force from each Maritime Agency Look-up to Merged

Question

• Joining four lookup tables for each Nationality / Maritime Agency created redundant attributes

• Performing a merge in Geoprocessing collapsed the redundancies into single computable columns

• Why does it almost double the feature count?

• GIS data Visio diagrams