Making Metadata Work For Your National Climate Data Center Asheville NC February 2006 Lynda Wayne US...

Making Metadata Work For Your

National Climate Data Center • Asheville NC • February 2006

Lynda Wayne US Federal Geographic Data Committee / GeoMaxim

Mike MoellerNOAA Coastal Services Center / Perot Systems

NCDC Making Metadata Work For You FGDC / NOAA

Objectives

After the workshop, students can:

effectively read and comprehend metadata

make the business case for metadata

discriminate between minimal and quality metadata

develop a metadata template that uses a range of mandatory, conditional, & optional CSDGM elements

select and access metadata software

locate and access online resources

What IS Metadata?

Data ‘reporting’ WHO created the data?

WHAT is the content of the data?

WHEN was it created?

WHERE is it geographically?

HOW was the data developed?

WHY was the data developed?

What IS Metadata?

supplemental information

abstract

time period

author

sources

(file) size

What IS Metadata?

entity

attributes

Let’s Make Metadata

Turn to your neighbor and document the following:

Titlename

Theme Keywordswork & interests areas

Supplemental Informationunique interest, event, or quality

Metadata as a Data Component

MetadataTitleOriginatorPurposeAbstractGeographic ExtentTime Period of ContentSources UsedProcess DescriptionPositional AccuracyEntities and Attributes…….

phenomenaphenomena map / image / databasemap / image / database metadatametadata

abstraction abstraction

content context

etadata

(context)

Metadata: Building Block of theNational Spatial Data Infrastructure

access

• search• download

• describe• publish

• assess• apply

Metadata: Building Block of theNational Spatial Data Infrastructure

…a set of actions and new ways of accessing, sharing and using geographic data that enables far more comprehensive analysis of data to help decision-makers choose the best course(s) of action.

National Spatial Data Infrastructure

ata (context)

access

Metadata: Building Block of theYour Own Spatial Data Infrastructure

etadata

(context)

My SDI

access

Metadata the Awful

Geospatial Data Producers perceive metadata as:

time consumingexcessivefor the benefits of others

Geospatial Managers perceive metadata as:

non-productivecostly (time, training, software)

derive classify

collect

planimetric imagery

analysis

alternativecommittee

review

charette

metadata?

Metadata the Magnificent

Metadata can support:data management

project management

If it is:considered a component of the data

created during data development

populated with rich content

derive classify

collect

planimetric imagery

analysis

alternativecommittee

review

charette

Data Management via Metadata

DataAccountability

Discovery

& Re-use

Maintenance

& Update

DataLiability

Data Management:Data Maintenance and Update

MaintenanceAre the data current?

• older than?• prior to significant change?

Are the data valid?• newer sources?• better methods?

UpdateContact informationDistribution policies, availability, pricingURLSISO 19115 Topic CategoriesNew derivations

Data Management:Data Discovery and Re-use

Marketing metadata in the past…

If you create metadata, other people can discover your data

If you create metadata, you can find your own data

New and improved message…

Data Management:Data Discovery and Re-use

Data discovery bythemes / attributes

geographic extent

temporal character

methods

contributors

quality

Discoverable data is usable data

Data Management:Data Accountability

Exercise in data comprehension:

What do you know?

What don’t you know?

What should you know?

Are you willing to associate yourself with the metadata record?

Repeatable process:methodsvariablesparameters

Defensible process:public participationdecision-making

GIS-savvy public

Metadata can be crafted to preserve individual accountability in team projects

Return on InvestmentCatalog assessment:

# times occurs as inhouse source# times occurs as external sourcesignificance of source contribution

To facilitate discovery:standardized titling schemespreferred citation w/in Use Constraints

Data Management:Data Liability

Metadata is declaration of:Purpose

originator’s application of the data? ontological pedigree

Use Constraintsappropriate and inappropriate utility of the data

Completenesswhat is missing from the data

Distribution Liabilityexplicit liability of the producer and assumed liability of

the consumer

What to do…

What not to do…

Project Management via Metadata

PROJECTPLANNING

DELIVERABLES

COMMON LANGUAGE

PROJECT MONITORING

Project Management:Project Planning

Project design document:description / intentgeo & temporal extentsource dataattribute requirements

Benefits:expectations outlinedintegrated into processprovide medium to record progress

Project Management:Project Monitoring

Monitor:

status

QA/QC measures

changes

Requires that the metadata be actively maintained and reviewed.

Project Management:Common Language for Multi-participants

Common reference for:descriptors & parameterskeywords, vocabularies & thesauricontact informationattributesdistribution information

If reviewed regularly by all participants,improved opportunity for coordinating:

source dataanalytical methodsnew information

Project Management:Contract Deliverables

Metadata is a data component therefore part of any data deliverable

Clearly specify your metadata requirement via a metadata:

classification/validation schema, e.g. ‘level’ of metadata

template

specification manual

Provide a sample record if possible

Conclusions

If created with utility in mind, metadata can serve multiple roles:

and this is only human utility of metadata…

archivist preserve lineage

accountant preserve investment

developer document process

manager track progress

contributors communicate

consumer locate, assess, access data

Data Management:Reality Check

Discussion Questions:

Which of these data and project management metadata applications are you most likely to use and why?

Are there additional metadata applications that would serve your organization?

Making Metadata Part of the Process

New tools enable us to better integrate metadata creation into the data development process

If metadata were collected throughout the data process…

1. more accurate (no guessing)

2. more details

3. better decision-making.

But what can I do to better incorporate metadata into the data development process?

Make the business case:preserve data investments

limit data liability

manage data resources

find new data resources

easier data transfer

more efficient data distribution

step one:build adminstrative support

Emphasize individual benefits:reduce workload..in the long term

field fewer data inquiries

document personal contributions

step two:build technical support

Support your staffinclude in job descriptions & performance measures

provide

staff support

training

step two:build technical support

Organizational templatesidentify pertinent fields

populate fixed fieldsstandardized language

distribution methods

standards used

build source and contact libraries

step three:build templates

step four:distribute the effort

• title

• abs

• pur

• tim

• key

• com

Attrib

• attr

Attrib

• com

• sou

• fie

• ana

lytica

es• a

Data Processing Data Planning Data Analysis

manager technicians analysts

Managers Can WRITE metadata

data planning fieldstitle abstract purposegeographic extent keywordsand many more…

and manage the effortmetadata coordination

metadata enforcement

step five:lead by example

Establish policies:mandate use of standards and templates

develop boilerplate metadata deliverable language for data contractors

require units to publish their metadata

publish metadata SOP to document policies and procedures

step six:policies and procedures

Implementation Issues

What are the roadblocks to metadata implementation within your organization?

Operational Content

To fully realize the operational value of metadata it is important to utilize standardized vocabularies and provide rich content.

The following content recommendations support:internal data accessproject monitoringISO transition geodata.gov participation

…see Handout ‘Metadata Quick Guide’

Operational Content:Internal Data Access

TitleMini abstracts

Minimum: where, what, when

Best Practice: who, why, resolution, filename, source

Supplemental InformationInclude internal referencing information

• project acronyms

• regional/disciplinary references

Operational Content:Internal Data Access

Theme and Place Keywords• internal and external thesauri/vocabularies

Process Contacts• record as part of Process Steps

Distribution: Resource Description• internal data/document references such publication

numbers, PINs or other cryptic references

Operational Content:Project Monitoring

Attribute Accuracy

Positional Accuracy

Process Steps

•Prescribe content (template)•Assign responsibilities•Actively maintain/review

Operational Content:ISO Transition

Theme KeywordTheme Keyword Thesaurus: ISO 19115 Topic Categories

Include one or more Topic Categories:boundaries oceans health transportation economy societyelevation farming locationbiota structure environmentutilitiesCommunication intelligenceMilitarygeoscientificInformation inlandWatersimageryBaseMapsEarthCover planningCadastre

climatologyMeteorologyAtmosphere

Operational Content:geodata.gov

Online Linkagerepeatable fieldused to determine content typemetadata, downloadable data, WMS, ArcIMS

Linkage Type Include

metadata server standard .html

downloadable data ftp:// or .zip, .tgz, .gz, .dxf, .eoo, etc.

WMS ‘getmap’ request w/ layer name, vers., image format, SRS

ArcIMS http://<server>/image/<service_name>

Writing Quality Metadata

Practice what I preach..

Using an actual data set, write a quality:Title:

Abstract:

We’ll use these later when creating metadata.

How Do I Create Metadata?

Steps to Success:

1. Understand the organization and content of the CSDGM

2. Learn to use the CSDGM Workbook (aka The Greenbook)

3. Select a metadata editor tool

4. Apply the CSDGM to your data

CSDGM Overview

The CSDGM documents:

digital geospatial datadigital geospatial data

imagery digital land surveysGIS files geographic databases

MANY fields will not be applicable to your data

Your missionYour mission is to understand the standard well-enough to model it for your own use.

CSDGM Overview

Standard is organized into seven primary sections

There are three ‘supporting’ sections that repeat within the primary sections:

citation• a standardized way to cite the data and source data

time period information• a standardized way to record dates

contacts• a standardized way to document contacts

CSDGM Overview

1. Identification InformationGeneral bibliographic information including:

title, originator, data contact, status, date, time period of content, abstract, purpose, keywords, geographic location

2. Data Quality InformationLineage and data assessments

sources, process methods, accuracy, data processing contact

CSDGM Overview

3. Spatial Data OrganizationData format:

vector, point, raster

4. Spatial Reference InformationCoordinate system parameters:

horizontal / vertical coordinate system, projection, datum

CSDGM Overview

5. Entity and Attribute InformationDatabase design

entities, attributes, domains, description of data values

6. Distribution InformationHow to acquire the data

distribution contact, available formats, online distribution website, costs

CSDGM Overview

7. Metadata Reference InformationGeneral information about the metadata record itself

metadata contact, metadata standard used, metadata creation date, metadata review date

CSDGM Overview

But that wasn’t enough for some people like Ted..

Extensions:added elements to the CSDGM

• Remote Sensing Extension

Profilescontent requirements for specific elements

may included added elements as well• Biological Profile

• Shoreline Profile

Mine the Metadata Exercise

Use your new understanding of the CSDGM to navigate through the sample metadata record and find answers to a set of questions.

CSDGM Workbook

AKA The Greenbook, it provides:background information about the development and purpose of the CSDGMexplanation of the organization of the CSDGMFAQs about specific elementssample values for specific elementsinstructions on creating Profiles and Extensionssample Metadataa glossary of terms

the CSDGM Workbook is your best metadata friend

Metadata Tools

How do I determine the best metadata tool?Does your organization provide a tool that meets specific data management objectives?Does your GIS software provide an internal metadata editor?Do you need a tool that supports one or more CSDGM Profiles or Extensions? Do you need to document data resource beyond those managed within your GIS? Do you need a tool that is easily distributed to partners? What features are most important to you?

auto-capture of information bundling of metadata with data creation and use of templates easy to use interface robust help systems and tutorials

Metadata Tools

What metadata editors are available?Freeware:

tkme/xtmeUSGS, supports all profiles/extensions, well-maintained

MetaVistUSFS, desktop, supports biological profile

MERMAidNOAA CSC, online utility supports biological and shoreline profiles

Metadata Tools

What metadata editors are available?Commercial

SMMS (Intergraph GeoMedia and stand-alone)

auto capture from multiple file types, contact/source libraries, data exploration utilities

ArcCatalogmetadata ‘synchronized’ w/data, ability to enclose

documents, metadata bundled with data export

Blue Angelhighly customizable, supports non-geospatial metadata

Metadata Tools

What other metadata tools are available?Parsing and Validation

Metadata Parser (mp)checks for conformance to CSDGM specified:

• conditionality (are all mandatory elements are populated?) • data formats • element domains

mp batchbatch processor shareware from Intergraph

Chew N Spit (cns)• Rectifies indentation• Inserts capitalization and underbars

FGDC ‘compliant’ metadata must pass mp!

Metadata Tools

What other metadata tools are available?Parsing and Validation

error2htmprovides user-friendly interpretation of mp error report

dbfmetaconverts dbase to CSDGM Entity Attribute elements

mq provides interface for editing large, diverse collections of

metadata

Metadata Tools

Formshardcopy or online

Metadata Validation Tools

mp - Metadata Parser • Checks for CSDGM syntax

– Element names– Mandatory elements– Element content (domains and logical consistency)

• Embeds tags for NSDI Clearinghouse Distribution

cns – Chew ‘N Spit • Rectifies indentation• Inserts capitalization and underbars

FGDC ‘compliant’ metadata must pass mp!

CSDGM Elements

Turn topage 35

in theGreen Book

CSDGM Elements

Calendar DatesYYYYMMDD

Time of DayHHMMSSSS

CoordinatesLat/Lon Decimal Degrees

Network addresses & file namesService://hostname:port/path/filename

CSDGM Production Rules

What’s Mandatory? What’s Not?

Mandatory if Applicable - must be provided if the data set exhibits the defined characteristic.

Optional - provided at the discretion of the data set producer.

Mandatory - must be provided.

MeaningData

ElementCompoundElement

10/06/95

Using the Graphics to Make Decisions - 1

All elements are colored yellow, so all are mandatory and must be reported.

M M / A O

CompoundElement 1

CompoundElement 1.1

DataElement 1.1.1

DataElement 1.1.2

DataElement 1.2

10/06/95

Compound Element 1 is mandatory.

Compound Element 1.1 is optional. The producer must decide if it wants to provide the element.

If yes, Data Elements 1.1.1 and 1.1.2 are mandatory.

If no, do not report Compound Element 1.1, Data Element 1.1.1 or 1.1.2, and skip to Data Element 1.2.

Data Element 1.2 is mandatory. M M / A O

CompoundElement 1

CompoundElement 1.1

DataElement 1.1.1

DataElement 1.1.2

DataElement 1.2

10/06/95

Compound Element 1 is mandatory.

Compound Element 1.1 is mandatory.

Data Element 1.1.1 is mandatory.

Data Element 1.1.2 is mandatory if applicable. The producer must decide if the data set has the characteristic defined by the data element. If yes, then Data Element 1.1.2 is mandatory. If no, then Data Element 1.1.2 is not reported.

Data Element 1.2 is optional. M M / A O

CompoundElement 1

CompoundElement 1.1

DataElement 1.1.1

DataElement 1.1.2

DataElement 1.2

10/06/95

Compound Element 1 is mandatory if applicable. If not applicable to the data set, do not report any elements. If applicable, it is mandatory and:

Compound Element 1.1 is mandatory.

Data Element 1.1.1 is mandatory if applicable. If not applicable, do not report it. If applicable, it is mandatory.

Data Element 1.1.2 is mandatory.

Data Element 1.2 is optional. M M / A O

CompoundElement 1

CompoundElement 1.1

DataElement 1.1.1

DataElement 1.1.2

DataElement 1.2

10/06/95

Section One: Identification

Elements you use to ‘shop’ for a data set of interest:

WHO ? title, originator, publication date

WHAT ? abstract, keywords

WHERE ? geographic extent, browse graphic

WHEN ? time period of content

HOW? access/use constraints, status

Section Eight: Citation Information

Traditional elements used to document publications. Used for the data set you are documenting and source data sets.

Originator responsible party

Publication Date date made available

Title what, when, where and more

Presentation Form GIS file, imagery, dbase, etc.

Online Linkages view, download, WMS, etc.

Section Nine: Time Period Information

Fields used to document various dates referenced through out the metadata including:

Identification Information: Data Citation Publication DateData Quality Information: Source Data Citation Publication DateData Quality Information: Source Time Period of ContentData Distribution Information: Data Availability

Options include:

Single DateRange of DatesMultiple Dates

Section Ten: Contact Information

Contact information for individuals referenced through out the metadata record including:

Identification Information: Data Set ContactData Quality Information: Process Step ContactData Distribution Information: Distribution ContactMetadata Information: Metadata Contact

Section Two: Data Quality

How was the data set developed?

• Source Information

• Data processing

What checks were made of the data set?

• Logical Consistency

• Positional Accuracy

• Attribute Accuracy

Section Three: Spatial Data Organization

Point, Line, or Vector?

Section Four: Spatial Reference Information

Coordinate System

Section Five: Entity & Attribute Information

Database Description

Section Six: Distribution Information

How can I get the data set?• Distribution Contact Information

• Available Formats (digital and hardcopy)

• Standard and Custom Order Processes

• Online option

• Offline media

• Distribution Liability

• Fees

Section Seven: Metadata Information

Elements you use to ‘shop’ for a data set of interest:

WHO wrote the metadata record

WHAT metadata standard was used?

WHEN was the metadata created/updated?

Is the metadata available?

metadata access and use constraints

Transition to ISO Metadata

Standardized Metadata

Content Standard for Digital Geospatial Metadata (CSDGM) Vers. 2.0

remains current US standard

International Organisation for Standardization (ISO)

standard completed

US adoption in progress

1. Content

CSDGM similar and new elements

Domains and conditionality

Essential or core metadata

2. Format

UML / XML

Multi-level metadata

What’s new with ISO metadata?

CSDGM Similar ElementssameMetadata Standard Name

new terminology and definitionsCSDGM = Originator

ISO = Responsible Party

w/ Role = originatorcustodianownerpublisher….

ISO Metadata: Content

New (non-CSDGM) ElementsInternational elementsData / Metadata Language

Extended elementsCSDGM = geographic and temporal extent

ISO = geographic, temporal, & vertical extent

Lacking elementsISO Topic Category – 19 standardized subject categories to facilitate data discovery

Domains and ConditionalityFewer mandatory elementsCore contains only 7 mandatory elements

Fixed domains and code lists in place of free text to control vocabulary and improve discoveryResponsible Party - Role and Topic Categories represented as codes

• Role = Originator (006)

• Topic Category = Transportation (018)

More optional elements

ISO Topic Categoriesenable discovery by topic within geodata.gov

enable notification of Data Community Stewards

facilitate translation of CSDGM metadata to ISO formatboundaries oceans health

transportation economy societyelevation farming locationbiota structure environmentutilitiesCommunication intelligenceMilitarygeoscientificInformation inlandWatersimageryBaseMapsEarthCover planningCadastreclimatologyMeteorologyAtmosphere

ISO Core Metadata22 basic metadata elements

comprise essential metadata for all geospatial datadigital data

maps and charts

Element conditionalitymandatory

conditional

optional

all national profiles of ISO must include the core

ISO Metadata: Core Elements

Dataset title Spatial representation type

Dataset reference date Reference system

Dataset responsible party Lineage statement

Geographic location On-line resource

Dataset language Metadata file identifier

Dataset character set Metadata standard name

Dataset topic category Metadata standard version

Spatial resolution Metadata language

Abstract Metadata character set

Distribution format Metadata point of contact

Additional extent info (vert / temp) Metadata date stamp

**mandatory

Building the national profile

Upgrade conditionality

Extend codelists

Establish codelists

core elements extensions

ISO Metadata: New Content

ISO Metadata: UML Model

ISO Metadata: 19139 XML Schema

the good news….

XML supports consistent and robust implementation via software tools

• Format and exchange- current implementation - txt, doc, db file, etc.

- exchange format is txt

• Internal metadata harvesting and other automated features

ISO Metadata: Implications

more good news….

Multi-level metadataData Series - wetlands

Dataset - Paducah wetlands

Feature Type - marsh

Feature Instance - Mandy’s Marsh

Attribute Type - salinity

Attribute Instance - actual % water salinity

the not-so-great news….

Standard can no longer be ‘read’ by most geospatial data developers

greater dependency on tools

lack of direct contact difficult for some metadata creators - analogous to moving from command line to GUI

Crosswalks drafted for:CSDGM Vers. 2.0

Dublin Core, others

Conversion software in development:FGDC sponsored effort with Intergraph /SMMS for standalone

ESRI has drafted for ArcCatalog

What about legacy metadata?

Now….1. Continue creation using CSDGM vers.2 !!!

2. Use ISO-poised softwareESRI ArcCatalog

Intergraph / SMMS

MetaD, FGDC/Polytechnic University Catalunya

3. Add Topic Categories as theme keywords with ‘ISO 19115 Category’ as keyword thesaurus

4. Monitor FGDC website for news, materials, and training opportunities

How do I prepare?

Soon…1. Educate and inform your organization that a

migration, and assistance, is in the future

2. Develop a metadata conversion plan using crosswalks and conversion software

3. Adopt ‘CSDGM vers. 3.0’ when finalized

How do I prepare?

Why Metadata?

Utilizing External Data….

Why metadata?

Be a part of the BIG picture

Federal Geographic Data Committee (FGDC)- coordinates national geospatial data development

National Spatial Data Infrastructure (NSDI)- creation and distribution of seamless national geospatial data

Global Spatial Data Infrastructure (GSDI)- creation and distribution of seamless global geospatial data

Why metadata?

NSDI Framework Data- cooperatively-designed data dictionaries for seven key

‘reference’ themes

• Elevation and Bathymetry

• Hydrography

• Geodetic Control

• Cadastral

• Transportation

• Governmental Units

• Orthoimagery

state / local national

Why metadata?

NSDI Clearinghouse Network- international network of metadata distribution ‘nodes’

• FGDC• EROS• ESRI• NRCS• NOAA CSC• ALASKA GDC

internationalNCDC

available metadata records

user portal gateways nodes

GeospatialOne-Stop

Other Metadata ServicesWMS • OARS • WAFS

Why metadata?

NSDI Geospatial One Stop

metadata and links to data data view and webmapping servicesdata acquisition plans - marketplace thematic communities

…..fast tracking the NSDI

National Geospatial Programs Office

Geospatial One-stop

National Map

access

coordinationnation

al •

te •

l • p

etadata • services • comm

policies • standards • partnerships

Who creates metadata?

OfficiallyFederal organizations producing digital geospatial data

Federally funded projects that produce digital geospatial data

Morally and practicallyAnybody that creates digital data

It’s the right thing to do!

Making Metadata Work For Your National Climate Data Center Asheville NC February 2006 Lynda Wayne US...

Documents

Transcript of Making Metadata Work For Your National Climate Data Center Asheville NC February 2006 Lynda Wayne US...

UNC Asheville

Asheville dwi-attorney

Data Sharing what works… what doesn’t Data Sharing what works… what doesn’t Lynda Wayne, GeoMaxim/FGDC Dr. Tim De Troye, SC GIS Coordinator.

OF ASHEVILLE

Lynda + James

2013 Annual Report - United Way of Asheville and Buncombe ... · A new partnership with Asheville Radio Group, Paramount KIA of Asheville and the Asheville Tourists for our School

The Asheville Project The Asheville Project

UNC Asheville Magazine

Asheville Horizon Art

Lynda Waltho

AAUW Asheville

Creating Geospatial Metadata for the Long-term Lynda Wayne Federal Geographic Data Committee Geospatial One-Stop GeoMaxim.

Creating Metadata in Support of Geospatial One-Stop and the NSDI Lynda Wayne Federal Geographic Data Committee Geospatial One-Stop GeoMaxim.

FGDC CAP Kickoff Meeting Denver, CO September 28-29, 2004 1 Introduction to Metadata Lynda Wayne GeoMaxim / FGDC.

NSGIC Metadata Cadre’ who’s in, what’s up NSGIC 2002 Annual Conference Lynda Wayne GeoMaxim/ FGDC Metadata Education Coordinator.

2015 Asheville Marathonashevillemarathon.com/wp-content/uploads/2017/10/AsheHalfOverall.pdf · 2015 Asheville Marathon. 2015 Asheville Marathon 156 William Bulfer Asheville NC 1702

041026 Asheville Catt

Lynda Juall Carpenito Lynda Juall CARPENITO CARPENITO Lynda … · 2020. 6. 4. · Lynda Juall Carpenito Lynda Juall Carpenito Diagnosi infermieristiche Applicazione alla pratica

Geographic Information Systems and Geospatial Metadata CSCI 373: Advanced Database UNC Asheville April 27, 2006 Lynda Wayne US Federal Geographic Data.

Sneak Preview: Sneak Preview: The New US Geospatial Metadata Standard GeoMaxim Federal Geographic Data Committee (FGDC) Lynda Wayne, GISP Sharon Shin.