Data Structure: Data Modeling or XML? Metatopia 2007 Washington, D.C. November 6, 2007 David C. Hay...

Data Structure:Data Modeling or XML?

Metatopia 2007Washington, D.C.November 6, 2007David C. Hay

Essential Strategies, Inc.13 Hilshire Grove Lane, Houston, TX 77055 (713) 464-8316 [email protected] www.essentialstrategies.com

Copyright (c) 2007, Essential Strategies, Inc. 2 /34/34

Agenda

Case Study: The Justice Department Four ways to look at data The Conceptual Entity/Relationship Model The XML Version An Examination An Alternative Model The Revised XML The Federal Data Reference Model Conclusions


Agenda

Case Study: The Justice Department Four ways to look at data The Conceptual Entity/Relationship Model The XML Version An Examination An Alternative Model The Revised XML The Federal Data Reference Model Conclusions


The heart of the problem . . .

1. U.S. Department of Justice. “Building Exchange Content Using the Global Justice XML Data Model: A User Guide for Practitioners and Developers”. June, 2005. p. v. (Available at: http://it.ojp.gov/topic.jsp?topic_id=201.)

“Accurate and germane sharing of informa-tion across jurisdictions is a critical issue for justice and public safety. Although there has been significant progress in the field of infor-mation technology, the lack of standards for exchanging justice data has not only been a major obstacle to, but also the principal rea-son for, the high costs involved with justice information exchange.”1


According to the Justice Department . . .

Sharing requires standards for exchanging data.

For this reason, they have developed the Global XML Data Model – in XML.

But sharing involves more than data exchange.

It means agreeing on semantics. This means understanding the meaning of

the data being exchanged . . . Not just the form and syntax.

This calls for This calls for something more...something more...


Agenda

Case Study: The Justice Department Four ways to look at data The Conceptual Entity/Relationship Model The XML Version An examination An alternative Model The Revised XML The Federal Data Reference Model Conclusions


Four ways to look at data

ExternalSchema 1

ExternalSchema 2

External Schema 3

ConceptualSchema

Internal Schema

Internal Schema

PhysicalSchema Physical

Schema

Logical Schema(Relnl.)

Logical Schema(XML)



ExternalSchema 1

ExternalSchema 2

External Schema 3

External Schema In particular viewer’s

terms Overlapping May be inconsistent,

at least in use of words.

Difficult to reconcile different views



ConceptualSchema

Conceptual Schema

Reconciles different external schema into one view.Reflects logical structure of data



InternalSchema(Relnl.)

InternalSchema(XML)

LogicalSchema(Relnl.)

LogicalSchema(XML)

Internal Schema Logical Schema –

design in terms of a DBMS: tables and columns, object classes, etc.

Physical Schema -- design of physical medium: cylinders, tablespaces, etc.


Schema



Schema

In terms of the Architecture Framework . . .

ExternalSchema 1

ExternalSchema 2

External Schema 3

ConceptualSchema


Business owners’ views

Architect’s view

LogicalSchema(XML)

Designers’ views

Builders’ views


Agenda



The Conceptual Model (Schema) . . .

ExternalSchema 1

ExternalSchema 2

External Schema 3

ConceptualSchema

Logical Schema(Relnl.)

Logical Schema(XML)


Schema


The Conceptual Entity / Relationship Model . . .

Addresses the semantics of the organization. Consists of assertions about the nature of the

enterprise. Is graphic, so it can be discussed with the

business community.


WORKBOOK(DOCUMENT)* AUTHOR* LAST AUTHOR# CREATED* LAST SAVED* COMPANY* VERSION NUMBER* .* WEB REFERENCE* DOWNLOAD COMPONENT

WORKSHEET# NAME

COLUMN# COLUMN NUMBER* AUTOFIT WIDTH (INDICATOR)* WIDTHo HIDDEN (INDICATOR)o SPAN (INDICATOR)

ROW# ROW NUMBER

CELL* INDEX* HREF* STYLE ID* DATA VALUE* ROW NUMBER* COLUMN NUMBER* COLUMN AUTOFIT WIDTH* COLUMN WIDTH

TABLE* EXPANDED COLUMN COUNT* EXPANDED ROW COUNT* FULL COLUMN (INDICATOR)* FULL ROW (INDICATOR)

DATUM* TYPE

part of

composed of

in

composed of

in

composed of

located in

the locationof

part of

composed of

in

home of

For example . . .


Note the semantics . . .

Each

<first entity>

m ust be[or]m ay be

<re la tionsh ip nam e>

one and only one[o r]one or m ore

<second entity>

primarily about

BO O K

T O P IC

primarily about

BOOK TOPICaddressed in

Each BOOK must be primarily about one and only one TOPIC.

Each TOPIC may be addressed in one or more BOOKS.

For example . . .


Ok, it’s true . . .

Many data modelers don’t care about semantics.

They only care about database design (the technical solution).

Just as XML advocates are promoting a technical solution.

But semantics comes first!


Agenda



What is XML?


XML is a kind of internal (Logical) schema. . .

ExternalSchema 1

ExternalSchema 2

External Schema 3

ConceptualSchema


LogicalSchema(XML)


Schema

Designers’ view


XML: The 1 Minute tutorial . . .

XML is a language for data communications. It is based on tags defined by the creator.

For example: <product>

<product_name>BlackBerry</product name></product>

Notes: Each tag provides a label for that

which follows it. Each tag must be accompanied by

an end tag (</…) Tags are defined by a community within which

communications are to take place. “All we have to do is to agree on the tags…”


XML Schema: The 1 minute tutorial . . .

XML Schema is an XML document that defines the tags toconfigure other documents.

The “tags” are predefined in a WW3 namespace: Xs:schema

xmins:xs=“http://www.w3.org/2001/XMLSchema” Some of the tags include:

<xs:element> <xs:complexType> <xs:sequence> <xs:simpleContent> <xs:attribute>

Key attributes of <xs:attribute> and <xs:element>: Name=“Chuck” Type=“xs:string” Use=“required” MinOccurs=“0” MaxOccurs=“unlimited”

The XML Schema version of our model . . .

<?xml version="1.0" encoding="UTF-8"?><?xml version="1.0" encoding="UTF-8"?><xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified"><xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">

<xs:element name="<xs:element name="WorkbookWorkbook">"><xs:complexType><xs:complexType>

<xs:sequence><xs:sequence><xs:element name="<xs:element name="WorksheetWorksheet" maxOccurs="unbounded">" maxOccurs="unbounded">

<xs:complexType><xs:complexType><xs:sequence><xs:sequence>

<xs:element name="<xs:element name="TableTable" minOccurs="0">" minOccurs="0"><xs:complexType><xs:complexType>

<xs:sequence><xs:sequence> <xs:attribute name="<xs:attribute name="ExpandedColumnCountExpandedColumnCount" type="xs:byte" use="required"/>" type="xs:byte" use="required"/>

<xs:attribute name="<xs:attribute name="ExpandedRowCountExpandedRowCount" type="xs:byte" use="required"/>" type="xs:byte" use="required"/> <xs:attribute name="<xs:attribute name="FullColumnsFullColumns" type="xs:boolean" use="required"/>" type="xs:boolean" use="required"/> <xs:attribute name="<xs:attribute name="FullRowsFullRows" type="xs:boolean" use="required"/>" type="xs:boolean" use="required"/>

<xs:element name="<xs:element name="ColumnColumn" maxOccurs="unbounded">" maxOccurs="unbounded"><xs:complexType><xs:complexType>

<xs:attribute name="<xs:attribute name="AutoFitWidthAutoFitWidth" type="xs:boolean" " type="xs:boolean" use="required"/>use="required"/>

<xs:attribute name="<xs:attribute name="WidthWidth" type="xs:decimal" " type="xs:decimal" use="required"/>use="required"/>

<xs:attribute name="<xs:attribute name="HiddenHidden" type="xs:boolean"/>" type="xs:boolean"/><xs:attribute name="<xs:attribute name="SpanSpan" type="xs:boolean"/>" type="xs:boolean"/>

</xs:complexType></xs:complexType></xs:element></xs:element><xs:element name="<xs:element name="RowRow" maxOccurs="unbounded">" maxOccurs="unbounded">


<xs:element name="<xs:element name="CellCell" maxOccurs="unbounded">" maxOccurs="unbounded"><xs:complexType><xs:complexType>

<xs:sequence><xs:sequence><xs:attribute name="<xs:attribute name="StyleIDStyleID" type="xs:string"/>" type="xs:string"/><xs:attribute name="<xs:attribute name="IndexIndex" type="xs:byte"/>" type="xs:byte"/><xs:attribute name="<xs:attribute name="HRefHRef" type="xs:string"/>" type="xs:string"/>


Of course one page isn’t enough . . .

<xs:element name="<xs:element name="DataData" minOccurs="0">" minOccurs="0"> <xs:complexType><xs:complexType> <xs:simpleContent><xs:simpleContent>

<xs:restriction base="xs:string"><xs:restriction base="xs:string"> <xs:attribute name="<xs:attribute name="TypeType" type="xs:string" " type="xs:string" use="required"/>use="required"/> </xs:restriction></xs:restriction>

</xs:simpleContent></xs:simpleContent></xs:complexType></xs:complexType>

</xs:element></xs:element></xs:sequence></xs:sequence>

</xs:complexType></xs:complexType></xs:element></xs:element>

</xs:sequence></xs:sequence></xs:complexType></xs:complexType>



</xs:sequence></xs:sequence><xs:attribute name="<xs:attribute name="NameName" type="xs:string" use="required"/>" type="xs:string" use="required"/>



</xs:element></xs:element></xs:schema></xs:schema>


Agenda


<?xml version="1.0" encoding="UTF-8"?><?xml version="1.0" encoding="UTF-8"?><xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified"><xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">

<xs:element name="<xs:element name="WorkbookWorkbook">"><xs:complexType><xs:complexType>

<xs:sequence><xs:sequence><xs:element name="<xs:element name="WorksheetWorksheet" maxOccurs="unbounded">" maxOccurs="unbounded">


<xs:element name="<xs:element name="TableTable" minOccurs="0">" minOccurs="0"><xs:complexType><xs:complexType>

<xs:sequence><xs:sequence> <xs:attribute name="<xs:attribute name="ExpandedColumnCountExpandedColumnCount" type="xs:byte" use="required"/>" type="xs:byte" use="required"/>

<xs:attribute name="<xs:attribute name="ExpandedRowCountExpandedRowCount" type="xs:byte" use="required"/>" type="xs:byte" use="required"/> <xs:attribute name="<xs:attribute name="FullColumnsFullColumns" type="xs:boolean" use="required"/>" type="xs:boolean" use="required"/> <xs:attribute name="<xs:attribute name="FullRowsFullRows" type="xs:boolean" use="required"/>" type="xs:boolean" use="required"/>

<xs:element name="<xs:element name="ColumnColumn" maxOccurs="unbounded">" maxOccurs="unbounded"><xs:complexType><xs:complexType>

<xs:attribute name="<xs:attribute name="AutoFitWidthAutoFitWidth" type="xs:boolean" " type="xs:boolean" use="required"/>use="required"/>

<xs:attribute name="<xs:attribute name="WidthWidth" type="xs:decimal" " type="xs:decimal" use="required"/>use="required"/>

<xs:attribute name="<xs:attribute name="HiddenHidden" type="xs:boolean"/>" type="xs:boolean"/><xs:attribute name="<xs:attribute name="SpanSpan" type="xs:boolean"/>" type="xs:boolean"/>

</xs:complexType></xs:complexType></xs:element></xs:element><xs:element name="<xs:element name="RowRow" maxOccurs="unbounded">" maxOccurs="unbounded">


<xs:element name="<xs:element name="CellCell" maxOccurs="unbounded">" maxOccurs="unbounded"><xs:complexType><xs:complexType>

<xs:sequence><xs:sequence><xs:attribute name="<xs:attribute name="StyleIDStyleID" type="xs:string"/>" type="xs:string"/><xs:attribute name="<xs:attribute name="IndexIndex" type="xs:byte"/>" type="xs:byte"/><xs:attribute name="<xs:attribute name="HRefHRef" type="xs:string"/>" type="xs:string"/>

Ok, Let’s look at that again . . .

Entity classes

Relationshipcardinality(Default:

must be 1)

Attributes

Attribute optionalit

y


<xs:element name="<xs:element name="DataData" minOccurs="0">" minOccurs="0"> <xs:complexType><xs:complexType>






</xs:element></xs:element></xs:sequence></xs:sequence><xs:attribute name="<xs:attribute name="NameName" type="xs:string" use="required"/>" type="xs:string" use="required"/>



</xs:element></xs:element></xs:schema></xs:schema>

And, of course, the rest of it . . .

You may not have noticed, but this is an attribute of

WORKSHEET


XML Spy does provide a graphic . . .

<Element>

<Complex type> <Sequence>

Optional element

Must be one or more elements

<Attribute>


But wait! There’s more to the model!

In the data model, these

were attributes of WORKBOOK


More about Styles . . .


Not to mention Worksheet Options, etc. . .


Please forgive me for not Please forgive me for not showing you the six pages showing you the six pages of resulting XML . . .of resulting XML . . .


EXCEL WORKBOOK* WINDOW HEIGHT* WINDOW WIDTH* WINDOW TOP X* WINDOW TOP Y* PROTECT STRUCTURE INDICATOR* PROTECT WINDOWS INDICATOR

WORKBOOK (DOCUMENT)* AUTHOR* LAST AUTHOR# CREATED* LAST SAVED* COMPANY* VERSION NUMBER* .* WEB REFERENCE* DOWNLOAD COMPONENT

STYLE* IDo NAME

WORKSHEET# NAME


ROW# ROW NUMBER


BORDER* POSITION* LINE STYLE* WEIGHT


DATUM* TYPE

SORTING

SORT

DOCUMENTPROPERTY(DESCRIPTION)* AUTHOR* LAST AUTHOR* CREATED* LAST SAVED (DATE)* VERSION (NUMBER)

BORDER GROUPING

ALIGNMENT* SS:VERTICALo SS:WRAP TEXT

NUMBER FORMATo SS:FORMAT

FONTo COLORo UNDERLINEo FAMILYo BOLD

PROTECTIONINTERIOR

part of

composed of

in

composed of

in

composed of

located in

the location of

in

to describe

part of

composed of

part of

composed of

part of

composed of

part of

composed of

part of

composed of

in

home of

of

describedby

What does this mean to the data model?




STYLE* IDo NAME

WORKSHEET# NAME


ROW# ROW NUMBER



WORKSHEET OPTIONo SELECTEDo PROTECT OBJECTSo PROTECT SCENARIOS


DATUM* TYPE

PANEo NUMBERo ACTIVE ROW

SORTING

SORT

OFFICE DOCUMENTSETTINGS* HREF

DOWNLOADCOMPONENT

DOCUMENT PROPERTY(DESCRIPTION)* AUTHOR* LAST AUTHOR* CREATED* LAST SAVED (DATE)* VERSION (NUMBER)




PROTECTION

PANE GROUP

PRINT* VALID PRINTER* PAPER SIZE INDEX* HORIZONTAL RESOLUTION* VERTICAL RESOLUTION

INTERIORBORDER GROUPING

part of

composed of

in

composed of

in

composed of

locaed in

the location of

in

to describe

part of

composed of

part of

composedof

part of

composed of

to support

supported via

part of

composed of

part of

composed of

part of

composed of

in

home of

the source of

derived from

the source of

derived from

of

described by

Including some very strange things . . .


The data model was inferred from the XML

Is it right?

2. My thanks to Peter Aiken for this example.2. My thanks to Peter Aiken for this example.

2




STYLE* IDo NAME

WORKSHEET# NAME


ROW# ROW NUMBER





DATUM* TYPE


SORTING

SORT

OFFICE DOCUMENTSETTINGS* HREF

DOWNLOADCOMPONENT

DOCUMENT PROPERTY(DESCRIPTION)* AUTHOR* LAST AUTHOR* CREATED* LAST SAVED (DATE)* VERSION (NUMBER)




PROTECTION

PANE GROUP

PRINT* VALID PRINTER* PAPER SIZE INDEX* HORIZONTAL RESOLUTION* VERTICAL RESOLUTION

INTERIORBORDER GROUPING

part of

composed of

in

composed of

in

composed of

located in

the location of

in

to describe

part of

composed ofpart of

composedof

part of

composed of

to support

supported via

part of

composed of

part of

composed of

part of

composed of

in

home of

the source of

derived from

the source of

derived from

of

described by

Let’s look at the model again . . .

Is not an EXCEL workbook a Is not an EXCEL workbook a workbook? (Sub-type?)workbook? (Sub-type?)

What is this What is this relationship?relationship?

Which Which columncolumnfor thisfor thiscell?cell?

Attributes Attributes of authors,of authors,company?company?

What is What is this?this?

… … or or this?this?

Attributes Attributes of of

WORKBOOK?WORKBOOK?

More authors?More authors?


XML limitations . . .

XML is fundamentally hierarchical . . . Cannot have multiple parents

Can describe a transaction, but . . . Assumes validity of data

Can impose cardinality rules Can impose syntactic rules No rules based on the meaning of the data.

Cannot describe semantics of relationships Cannot be presented to normal human

beings.


In Fairness . . .

While E/R modeling can more effectively portray the meaning of the data, it too is limited in its ability to portray business rules.

New tools (such as XML Spy) are making it possible to deal with XML graphically.


Agenda



WORKBOOK (DOCUMENT)* LAST AUTHOR# CREATED* LAST SAVED* COMPANY* VERSION NUMBER* .* WEB REFERENCE* DOWNLOAD COMPONENT

WORKSHEET# NAME

COLUMN# COLUMN NUMBER* AUTOFIT WIDTH* WIDTH

ROW# ROW NUMBER

CELL* INDEX* HREF* DATA VALUE* ROW NUMBER* COLUMN NUMBER* COLUMN AUTOFIT WIDTH* COLUMN WIDTH



AUTHORSHIP# SEQUENCE NUMBER* PRIMARY INDICATOR

COMPANY# IDENTIFIER* COMPANY NAME

PERSON# IDENTIFIER* FIRST NAMEo MIDDLE INITIAL* LAST NAMEo PRIMARY INDICATOR

STYLE ELEMENT# NAME* DESCRIPTION* VALUE

STYLE ELEMENTVALUE# SEQUENCE NUMBER* VALUE


PRINT...

TABLE* EXPANDED ROW COUNT* EXPANDED COLUMN COUNT* FULL ROWS* FULL COLUMNS

PANE GROUP

part of

composedof

in

composedof

in

composed of

locaed in

thelocationof

part of

composed of

to support

supportedvia

the location of

located in

of

created via

in therole of

by

published for

sponsor of

for

describedby

an example of

embodied in

Add network...Add network...

With Data Modeling, we have an alternative . . .

...and ...and a sub-type?a sub-type?

...many-to-many......many-to-many...

...referenc...referencee

entityentityclasses…classes…

...collapsed ...collapsed entities...entities...

Border positionBorder positionNumber formatNumber formatFont ColorFont ColorVerticalVertical alignment alignmentetc.etc.


Agenda



To convert a data model to XML . . .

Identify a hierarchy. NOTE: Several may be available.

Establish one direction for many-to-many relationships.

Move intersect attributes to new “many” side.

Inherit attributes from parents to children entity classes.


WORKBOOK (DOCUMENT)* AUTHOR* LAST AUTHOR# CREATED* LAST SAVED* COMPANY* VERSION NUMBER* .* WEB REFERENCE* DOWNLOAD COMPONENT* ..* WINDOW HEIGHT* WINDOW WIDTH* WINDOW TOP X* WINDOW TOP Y* PROTECT STRUCTURE INDICATORo PROTECT WINDOWS INDICATOR

WORKSHEET# NAME* EXPANDED COLUMN COUNT* EXPANDED ROW COUNT* FULL COLUMN INDICATOR* FULL ROW INDICATOR


ROW# ROW NUMBER









PANE GROUP PRINT...

part of

composed of

located in

thelocationof

part of

composed of

to support

supportedvia

thelocationof

located in

of

created via

in therole of

by

published for

sponsor of

for

describedby

an example of

embodied in

in

composed of

in

composed of

Here’s one version . . .



WORKBOOK (DOCUMENT)* LAST AUTHOR# CREATED* LAST SAVED* COMPANY* VERSION NUMBER* .* WEB REFERENCE* DOWNLOAD COMPONENT

WORKSHEET# NAME* EXPANDED ROW COUNT* EXPANDED COLUMN COUNT* FULL ROWS* FULL COLUMNS

CELL* INDEX* HREF* DATA VALUE* ROW NUMBER* COLUMN NUMBER* COLUMN AUTOFIT WIDTH* COLUMN WIDTH





COLUMN# COLUMN NUMBER* AUTOFIT WIDTH (INDICATOR)* WIDTH...

ROW# ROW NUMBER

DATUM* TYPE

PANE GROUP PRINT...

part of

composed of

part of

composed of

to support

supported via

aurhor of

created by

describedby

adescriptorof

composed of

part of

in

home of

for

to define

for

to define

And create a constrained model . . .


The Resulting XML (the graphic version, at least) . . .


More about the Worksheet . . .


WORKBOOK (DOCUMENT)* AUTHOR* LAST AUTHOR# CREATED* LAST SAVED* COMPANY* VERSION NUMBER* .* WEB REFERENCE* DOWNLOAD COMPONENT* ..* WINDOW HEIGHT* WINDOW WIDTH* WINDOW TOP X* WINDOW TOP Y* PROTECT STRUCTURE INDICATORo PROTECT WINDOWS INDICATOR

WORKSHEET# NAME* EXPANDED COLUMN COUNT* EXPANDED ROW COUNT* FULL COLUMN INDICATOR* FULL ROW INDICATOR


ROW# ROW NUMBER









PANE GROUP PRINT...

part of

composed of

located in

thelocationof

part of

composed of

to support

supportedvia

the location of

located in

of

created via

in therole of

by

published for

sponsor of

for

describedby

an example of

embodied in

composed of

in

in

composed of

Here’s a second version . . .

Only primary Only primary author is author is includedincluded




WORKSHEET# NAME



DATUM* TYPE



STYLEELEMENT# NAME* DESCRIPTION* VALUE

PRINT...

PANE GROUP

part of

composedof

part of

composed of

to support

supported via

described by

a descriptorof

composedof

part of

author of

prepared by

in

home of

part of

composed of

And create a constrained model . . .


The Resulting XML . . .


With Worksheet Details . . .


Agenda



Federal Enterprise Architecture . . .

3. CIO Council, Office of Management and Budget.3. CIO Council, Office of Management and Budget.

33


Data Sharing

Data Context

Query Points and Exchange Packages (XML)


Taxonomies (Categories)(Function/data Usage)


Data Description

Data Elements(E/R Model)


Specifically, the Data Reference Model . . .


Data Sharing

Data Context





Data Description



But you must But you must understand understand when you are when you are notnot doing doing thisthis but but thisthis..


Agenda



Conclusions . . .

XML is very good for data communications. English syntax is convenient. XML Schema is very powerful for describing

the structure of transactions It is widely accepted. Graphic tools make it more manageable.

Semantic data modeling is better for analyzing data structure. Graphic nature makes it suitable for

discussing semantic issues. Two dimensional format makes it possible to

describe networks.


More significantly. . .

But XML is fundamentally a technological design.

...while conceptual data modeling is fundamen-tally a way to describe the business problem.

It’s important to It’s important to understand the understand the

difference.difference.


It’s better to start with the Data Model

(More Semantics)

And then derive the XML script from that

(Less Semantics)


Questions?

Data Structure: Data Modeling or XML? Metatopia 2007 Washington, D.C. November 6, 2007 David C. Hay...

Documents

Transcript of Data Structure: Data Modeling or XML? Metatopia 2007 Washington, D.C. November 6, 2007 David C. Hay...