Geog 582 Autumn 2016Assignment 5

Working with Metadata and XML

Assignment Data

● Data from Assignment 4● “WAGIC_ArcGIS.xlsx” from Assignment5Data folder.

Learning Objectives

● Understand why it is important to create and maintain metadata● Understand how metadata is developed and used in a GIS workflow● Practice creating and updating metadata● Understand different metadata standards

Deliverables: ● A complete answer sheet answering Questions 1 and 3 provided below. ● XML files from Question 2 saved into a zip file.

Introduction

Given the enormous amount of spatial data from many different sources, how do you find what you really need? Metadata provides a way to document a dataset so that potential users will be able to find it and evaluate it to see if it’s suitable for their needs. Metadata also improves data sharing between users both within and across organizations. Metadata is defined as data about data; when it was created, how it was created, what the accuracy is, who is responsible for it, can you share it, etc. GIS data without metadata can be problem-atic to use – what are the limitations due to accuracy, scale, or currency, what do the at-tributes mean? Successful metadata creation and implementation requires thought and planning, and can be time consuming, but it is an integral part of GIS data holdings.

In Assignment 4, you’ve created and populated a geodatabase for green infrastructure sys-tem in one of the WRIAs. In this assignment, after learning some basic, yet important, con-cepts of metadata, you will practice creating and updating metadata for feature datasets and associated data from Assignment 4 using ArcGIS applications.

XML stands for Extensible Markup Language. Since ArcGIS metadata is stored using XML, you should understand how to read metadata in XML. You will also learn about the basic structure of XML and practice working with it in this assignment.

1

Metadata Standards and Metadata Style

While metadata is vital to our understanding of data, it would be difficult to understand each other’s metadata if they are documented in different ways. Metadata standards are needed to solve this kind of problem.

Metadata standards provide a common set of terminology, definitions, and expected values for the documentation of geospatial data. Standards provide a guideline for what informa-tion should be documented, in what way, and how much documentation is needed to mini-mally comply with that standard.

Two of the most commonly used metadata standards are the US Federal Geographic Data Committee (FDGC) standard and the International Standards Organization (ISO) standard. The FGDC standard is called the Content Standard for Geospatial Metadata (CSDGM). It is a standard designed to provide a complete description of a data source. Because it is quite detailed, various states and regions have created their own metadata standards to simplify (reduce) the amount of information that should be recorded. For example, Washington State (Washington State Geographic Information Council – WAGIC) has developed a work-ing metadata standard using a subset of the FGDC CSDGM. This standard serves a minimum requirement for significant spatial data. The state encourages the use of full FGDC standard when it is practical. For more details on WA State metadata standard, see http://ofm.wa.-gov/ocio/policies/documents/161.11.pdf.

The International Standards Organization (ISO) standards include ISO standard 19115 and ISO 19139. The ISO 19115, Geographic Information – Metadata, designed for international use, attempts to satisfy the requirements of all existing metadata standards. It allows for ei-ther general or detailed descriptions of data sources, makes some allowances for describ-ing resources other than data, and has a small number of mandatory elements. The new ISO 19139 standard, Geographic Information—Metadata—Implementation, provides a schema that defines how ISO 19115 metadata should be stored. ISO member nations can create their own ISO 19115 profile as long as it includes the core elements. To find out more about ISO metadata standards, visit the ISO Web site.

Which metadata standard to choose largely depends on who will use the data. For example, if your data will be used within the U.S., you can choose either the FGDC CSDGM or the ISO metadata standard. If you think that your data may be shared with an international organi-zation, then you should choose the ISO metadata standard. Currently, U.S. federal govern-ment agencies provide metadata that complies with the FGDC standard. However, recently the FGDC has encouraged the transition to ISO standard if a government agency or an orga-nization is able to do so.

ArcGIS provides several predefined metadata styles that comply with different metadata standards. A metadata style is the window through which you view and edit your metadata – it determines which metadata elements to display and defines how their data values ap-pear. ArcGIS (10.2) supports the following metadata styles: FGDC CSDGM Metadata, IN-SPIRE Metadata Directive, ISO 19139 Metadata Implementation Specification, Item De-

2

scription, and North American Profile of ISO19115 2003. The metadata style descriptions below are quoted from ESRI’s Help.

● ISO 19139 Metadata Implementation Specification: This style lets you view and edit a complete metadata document that complies with ISO standard 19139,Geographic in-formation — Metadata — XML schema implementation, export metadata in this for-mat, and validate it using the standard's XML Schemas. Use this style to create meta-data that complies with ISO standard 19115, Geographic information — Metadata.

● North American Profile of ISO 19115 2003: This style lets you view and edit a com-plete metadata document that complies with North American Profile of ISO 19115:2003 – Geographic information – Metadata, export metadata in this format, and validate it using the ISO 19139 XML Schemas.

● INSPIRE Metadata Directive: This style lets you view and edit a complete ISO 19139 metadata document that adheres to the INSPIRE Implementing Rules, export meta-data in the ISO 19139 format, and validate it using the ISO 19139 XML Schemas.

● FGDC CSDGM Metadata: This style lets you view and edit metadata following the FGDC Content Standard for Digital Geospatial Metadata (CSDGM) guidelines, export metadata in this standard's XML format, and validate it using the CSDGM XML DTD.

● Item Description is another style that is supported by ArcGIS. This style is the default metadata style, but it doesn’t adhere to specific metadata standards. This style is de-signed to facilitate providing information that is used by ArcGIS, and lets you view and edit a simple metadata document.

Question 1: What are the most commonly used metadata standards? What are major metadata styles that are currently supported by ArcGIS? What is ArcGIS metadata for-mat? What are the relationships between ArcGIS metadata format and different metadata styles?

Part I: Working with Metadata in ArcGIS

In ArcGIS you work with metadata mostly in ArcCatalog. You view and edit an item's meta-data in the Description tab, either in ArcCatalog or by opening the Item Description win-dow from other ArcGIS for Desktop applications. The information available for viewing and editing in ArcCatalog is determined by the current metadata style. Since you will use FGDC format for you metadata, illustrations and discussions will be focused on FGDC when appli-cable. This section introduces a general workflow to working with metadata in ArcGIS, fol-lowed by detailed explanations of each of the steps in the workflow.

The general workflow suggested by ESRI includes the following steps:

1. Choose a metadata style that’s appropriate for the standard you need to follow.2. View the item's ArcGIS metadata. Many properties of the item are added automati-

cally.

3

3. If the item has existing metadata created before ArcGIS 10.0, upgrade the item's metadata to the ArcGIS metadata format.

4. Create or edit the item's metadata in the ArcGIS metadata Editor. ● Add whatever content is appropriate for your organization and for the item.● After editing the information on a page, make sure no errors are listed at the

top.5. Save your changes and stop editing the item's metadata.6. Publish your metadata if it is needed (not applicable for this assignment).

Choosing Metadata Style

To create or edit FGDC metadata, the first step is to choose the FGDC metadata style. Item Description is the default metadata style that lets you see and edit basic metadata proper-ties/contents for an item, such as summary (abstract), description (purpose), credits, key-words/tags, extent, scale, and use limitations.

. If you want to view more information about the item or describe it in more detail than you can with the default Item Description style, or your metadata needs to be comply with a metadata standard. In that case, you need to choose a different metadata style to get access to the complete set of metadata.

● To choose the FGDC metadata style, in ArcCatalog, click Customize > ArcCatalog Options → click the Metadata tab → select FGDC CSDGM Metadata from the drop-down menu → Click OK.

4

Note: if you are in the Description tab when you choose a new metadata style, you will not see the results of the change immediately. You will need to click another tab in ArcCatalog or Item Description window, then click the Description tab for the new metadata style to take effect.

Upgrading Existing FGDC Metadata

Beginning with ArcGIS 10.0, metadata content is stored in a new internal format, known as ArcGIS Metadata. Metadata created in the current release of ArcGIS is ArcGIS metadata. To manage FGDC style metadata in the current release of ArcGIS, you must first upgrade the metadata using the Upgrade Metadata tool. When you view an item with existing FGDC-format metadata you will be notified immediately that it must be upgraded to the ArcGIS metadata format before it can be used in the Description tab (as shown be-low).

5

Click Yes to upgrade the existing FGDC metadata content right away. The Upgrade Metadata tool opens and the Upgrade Type is automatically set to FGDC_TO_ARCGIS. This process copies all existing FGDC content to the appropriate ArcGIS Metadata elements.

Viewing Metadata Content

By default, metadata is automatically updated when you view it in the Description tab. Ar-cGIS gets the item’s properties and records them in the appropriate metadata elements. When you copy a data or import feature classes to a geodatabase, their metadata are copied as well. If an item doesn’t have metadata, ArcGIS will create it once the item is added to and viewed in ArcCatalog. The spatial information of your data (e.g., extents and the coordinate system) and count of features are created automatically by ArcCatalog. If any of this infor-mation changes as you edit or update your data (for example, by projecting it to another co-ordinate system), ArcCatalog will automatically update it.

When ArcGIS is configured to use the FGDC CSDGM Metadata style, the ArcGIS Metadata heading appears at the bottom of the item's brief description. Metadata content created and managed using the Description tab is displayed under the ArcGIS Metadata heading, such as any item properties added automatically by ArcGIS, and any FGDC metadata content that corresponds to the basic item description. Any FGDC metadata created in a version earlier than ArcGIS 10.0 is displayed under the FGDC Metadata (read-only) heading.

6

An item's metadata can contain large amount of detailed information. It is divided into sec-tions in the display of the Description tab that corresponds to those in the ArcGIS metadata editor. If a green asterisk (*) appears next to a metadata element's name or value such as the Title and Content Type in the figure below, it means the value is automatically updated in the metadata by ArcGIS according to the item's intrinsic properties.

The “read-only” FGDC metadata is not automatically updated by the current version of Ar-cGIS unless you upgrade it to ArcGIS metadata using the Upgrade Metadata tool illustrated above.

7

Creating and Editing Metadata using FGDC Format

You create (if no metadata exists) and edit an item’s metadata in the ArcGIS metdata editor. Clicking the Edit button in the Descriptions tab will open the editor.

As shown in the figure above, the editor’s table of contents shows a list of pages that can be used to edit the metadata content under three headings (Overview, Metadata, and Re-source). Each page defines different groups of metadata elements. These metadata ele-

8

ments are the information that you need to enter or edit based on the metadata standard that you would follow.

The table of contents gives you an overall idea of whether an item's metadata meets rules of a particular metadata style. If there are any problems with the information provided on a page, the page will have a red X   in the table of contents (e.g., Citation, Constrains, and Quality in the figure above). Pages with no errors have a green check mark   instead. A plain page   in the table of contents has no rules concerning the information it lets you manage.

When there is a problem with an element (value) on a certain page, it will have a red back-ground. Typically such warning happens when a mandatory value has not been provided, or a wrong type of value may have been provided. For example, in the example below, Sum-mary and Description are required metadata elements that need to be provided. To make it easier to identify a problem or an error in the metadata editor, ArcGIS lists all errors that occur on a page at the top of that page. If you click the error in the list, the page will scroll to the place where you can fix the problem. If you are not sure what information to provide in a metadata element, you can hover the pointer over its input control to see the text ap-pearing at the bottom of the editor explaining what is expected.

To get a complete set of standard-compliant metadata, the best practice is reviewing each element that has red background in the page that has red x, and providing necessary infor-mation, until all the red x’s become green marks. However, since some metadata elements may not apply to an item, or you simply are not able to obtain required information, it is not uncommon that some elements may be left blank. You can always come back to update the metadata when the relevant information is available.

Exporting Metadata to XML

9

Metadata in ArcGIS is stored in XML. After creating your metadata content, if you want to share information about your item with others through a site, you must export it to an FGDC-compliant XML file that can be published to the site. In this assignment, you will not publish your metadata to any of the sites, but it is important to know how to export meta-data to XML files for the purpose of data sharing. There are many programs that can be used to view XML files, such as IE or Notepad ++ (open source application that can be downloaded from http://notepad-plus-plus.org/). A text editor like Notepad ++ makes it easier to edit an XML file.

In ArcCatalog, use the following steps to export metadata to FGDC XML file.

● Click the Export button   in the Description tab to open the Export Metadata dia-log box.

● In the dialog box, select the following parameters:✓ Source Metadata: automatically set to the item's location.✓ Translator: automatically set to the ARCGIS2FGDC.xml translator✓ Output file: provide name and path

The figure blow provides an example.

Part II: Creating and Editing Metadata for Watershed Characterization Geo-database

In this part of the assignment, you will use knowledge gained from Part I and your experi-ence to create and edit metadata for feature classes and/or raster datasets or tables that you created in Assignment 4. You will follow the FGDC working metadata standard devel-oped by the Washington State. If metadata already exists with the current data, you need to make sure that it meets the FGDC metadata requirements. If an item does not have meta-data, you need to create the metadata as per the standard.

The full document of the WAGIC working metadata standard can be found here http://ofm.wa.gov/ocio/policies/documents/161.11.pdf. This document explains the purpose and scope of the metadata standard, and provides a list of the minimum metadata elements needed. Read the document to understand this standard. Since the orders of the metadata elements in the document is not the same as those in the ArcGIS metadata editor and some names are slightly different, they are reordered and adjusted to match their counter parts

10

in the editor. A table was created to provide a side-by-side comparison of the elements in the ArcGIS metadata editor and WAGIC’s working standard. This table (WAGIC_ArcGIS.xlsx) can be found in the Assignment 5 data folder. It will help you identify the metadata ele-ments needed for the geodatabase. Note that the numbers after the element names from the WAGIC standard correspond to the Element No. in the working standard document.

The WAGIC working standard only provides a minimum requirement for metadata. You should enter more detailed information if it is available. On the other hand, since some metadata elements may not apply to your data, or you simply are not able to obtain re-quired information, it is natural that you may not even be able to provide some of the mini-mum requirements. In any case, the most important thing is to use your best knowledge to provide sets of metadata as complete as possible.

ESRI’s illustrated guide provides detailed examples of editing various metadata elements. Use it as a reference for your metadata creation and editing.

Question 2: Export the metadata of your data to XML files and examine the contents. For deliverables, please put all XML files into one zip file and submit.

Part III: Exploring XML Metadata

As you are aware, XML can be used to represent data content , structure and format for a variety of computing applications, one of those being GIS, and in particular ArcGIS. A num-ber of ArcGIS applications make use of XML to structure data such as in database schema definitions (as in the first three assignments) and in structured metadata (as in this assign-ment). Therefore, a GIS analyst should at least have some basic knowledge of XML files and how to read metadata in XML. These will be the focus of this part of the assignment.

XML Structure

XML is made up of Elements which have Attributes. In general, elements represent objects (or records), and attributes represent properties of that object. Attributes have values just like fields in the attribute table in ArcGIS. In fact, it can be helpful to think of elements as the rows in the attribute table and attributes as columns. For example, imagine you have a road layer that contains roads with the attributes OBJECTID, type, length, and speedLimit.

11

Two of the records in this imaginary layer could look like this if you open the layer’s at-tribute table in ArcMap:

482 Highway 10012 60

483 Rural 2223 25

To express these records in XML, we would need two road elements each with four at-tributes—something like this:

<road OBJECTID=“482” type=“Highway” length=“10012” speedLimit=“60”></road>

<road OBJECTID=“483” type=“Rural” length=“2223” speedLimit=“25”></road>

Notice that each element consists of an opening tag that looks like <elementName> and a closing tag that looks like </elementName> and every attribute is described by Attribute-Name= “AttributeValue”. (NOTE: it is also possible to declare elements with just one tag in-stead of separate opening and closing tags, in this case the ‘/’ simply comes before the end of the opening tag like so <elementName ….Attributes /> ). Elements can also be nested within other elements. For instance, we could declare a <country> element that contained <state> elements. How would that look?Attributes can also be declared within elements (between the opening and closing tags) in a similar way to the way elements are declared. For example the XML for our road objects above could be re-written as:

<road><OBJECTID>482</OBJECTID><type>Highway</type><length>10012</length><speedLimit>60</speedLimit>

</road><road>

<OBJECTID>483</OBJECTID><type>Rural</type><length>2223</length><speedLimit>25</speedLimit>

</road>

Here attributes have their own opening and closing tags and their data values come be-tween them.

12

Question 3: Choose one of the metadata XML files that you created in Part II (Question 2) of this assignment. Open the XML file and examine the contents. Cut and paste one complete (including opening tag, closing tag and everything in between) XML element into your an-swer sheet. Then describe what they are and list their attributes and values. Using a table might be the best way to convey this information, but it is not required.

13