Generating XML from Relational Data ...

IBM DB2 Information Management Technical Conference

© IBM Corporation 2005

IBM GLOBAL SERVICES

F15

Guogen (Gene) Zhang

Generating XML from Relational Data Using SQL with Ease

®

Orlando, FLSept. 12-16, 2005



Outline• XML basics and motivation for SQL/XML

publishing functions

• XML publishing functions by example

• Advanced topics

• Preview of future SQL/XML features

• Summary



XML Basics I

<Emp name="Jack Lee"> <BIRTHDAY>1960-10-28</BIRTHDAY> <department>Shipping</department></Emp>

Element name Attribute name Attribute value

Start tag

End tag Element contentfor “department”

}

Nested elementsas content of “Emp”



XML Basics II

<hr:Emp xmlns:hr="http://www.example.com/hr.xsd" name="Jack Lee"> <hr:BIRTHDAY>1960-10-28</hr:BIRTHDAY> <hr:department>Shipping</hr:department></hr:Emp>

PrefixNamespace prefix

Namespace name (URI)

QName

Local part

Namespace declaration

NCName



Motivation for XML Publishing Functions

• The world wants XML.

• Majority of DB2 data is still relational.

• Generating XML from relational data makes DB2 an XML data source.

• That's what XML constructor functions are for.

• And that is the first step towards total XML support inside engine.



Advantages of using SQL/XML Functions

• Easy to extend existing applications with XML using SQL– Works for non-Unicode tables

• Lightweight Web server/applications– Submit SQL queries and return results to Web/XML clients

• Less application development efforts• High-performance

– Better performance for tables in Unicode

• Conforming to the SQL/XML standard



XML Publishing Functions• Scalar functions (constructors)

– XMLELEMENT, XMLATTRIBUTES

– XMLNAMESPACES (DB2 LUW V8.2)

– XMLFOREST

– XMLCONCAT

• Aggregate function: XMLAGG• Transient XML data type – internal use only• Cast function: XML2CLOB (DB2 LUW 8.2:

XMLSERIALIZE)• Available in DB2 V8 (z/OS New Function Mode)



XML Functions by Example– 1. simple element

<Emp>Jack Lee</Emp>

SELECT XML2CLOB( XMLELEMENT(NAME "Emp", e.fname ||' '|| e.lname ) ) AS "result" FROM employees e WHERE ...;



XML Functions by Example– 1’. simple element

<Emp name="Jack Lee"></Emp>

SELECT XML2CLOB( XMLELEMENT(NAME "Emp", XMLATTRIBUTES(e.fname ||' '|| e.lname AS "name") ) ) AS "result" FROM employees e WHERE ...;



XML Functions by Example- 1’. simple element null values

<Emp></Emp>

SELECT XML2CLOB( XMLELEMENT(NAME "Emp", XMLATTRIBUTES(e.fname ||' '|| e.lname AS "name") ) ) AS "result" FROM employees e WHERE ...;

Use CASE expression to avoid empty elements



XML Functions by Example – 2. namespace

<doc:Emp xmlns:doc="http://www.ibm.com/emp.xsd"> Jack Lee</doc:Emp>

SELECT XML2CLOB( XMLELEMENT(NAME "doc:Emp", XMLNAMESPACES ('http://www.ibm.com/emp.xsd' AS "doc" ), e.fname ||' '|| e.lname) ) AS "result" FROM employees e WHERE ...;



XML Functions by Example – 2’ namespace - illegal

<doc:Emp xmlns:doc="http://www.ibm.com/emp.xsd"> Jack Lee</doc:Emp>

SELECT XML2CLOB( XMLELEMENT(NAME "doc:Emp", XMLATTRIBUTES ('http://www.ibm.com/emp.xsd' AS "xmlns:doc" ), e.fname ||' '|| e.lname) ) AS "result" FROM employees e WHERE ...;



XML Functions by Example – 3. nested elements


SELECT XML2CLOB( XMLELEMENT(NAME "Emp", XMLATTRIBUTES(e.fname ||' '|| e.lname AS "name") XMLELEMENT(NAME BIRTHDAY, e.birthday), XMLELEMENT(NAME “department”, e.dept) ) ) AS "result" FROM employees e WHERE ...;



XML Functions by Example – 3’. alternative


SELECT XML2CLOB( XMLELEMENT(NAME "Emp", XMLATTRIBUTES(e.fname ||' '|| e.lname AS "name") XMLFOREST(e.birthday, e.dept as “department”) ) ) AS "result" FROM employees e WHERE ...;



XML Functions by Example – 4. mixed content

<Emp>Employee <name>Jack Lee</name>was hired on <hiredate>2000-05-24</hiredate></Emp>

SELECT XML2CLOB( XMLELEMENT(NAME "Emp", ‘Employee ’, XMLELEMENT(NAME “name”, e.fname ||' '|| e.lname), ‘ was hired on ’, XMLELEMENT(NAME “hiredate”, e.hire) ) ) AS "result"FROM employees e WHERE ...;



XML Functions by Example – 5. concat

<Emp name="Jack Lee"></Emp><department>Shipping</department>

SELECT XML2CLOB( XMLCONCAT( XMLELEMENT(NAME "Emp", XMLATTRIBUTES(e.fname ||' '|| e.lname AS "name") ), XMLELEMENT(NAME “department”, e.dept) ) ) AS "result"FROM employees e WHERE ...;



XML Functions by Example – 6. grouping

<Department name="Shipping"> <emp>Oppenheimer</emp> <emp>Martin</emp> <emp>Lee</emp></Department>

SELECT XML2CLOB( XMLELEMENT(NAME "Department", XMLATTRIBUTES (e.dept AS "name" ), XMLAGG(XMLELEMENT(NAME "emp", e.lname) ) ) ) AS "dept_list"FROM employees e GROUP BY dept;



XML Functions by Example – 6’. grouping with order

<Department name="Shipping"> <emp>Lee</emp> <emp>Martin</emp> <emp>Oppenheimer</emp></Department>

SELECT XML2CLOB( XMLELEMENT(NAME "Department", XMLATTRIBUTES (e.dept AS "name" ), XMLAGG(XMLELEMENT(NAME "emp", e.lname) ORDER BY e.lname ) ) ) AS "dept_list"FROM employees e GROUP BY dept;



XML Functions by Example – 7. complex query

SELECT XML2CLOB( XMLELEMENT( NAME "Dept", XMLATTRIBUTES ( D.DEPTNO AS "deptno", D.DEPTNAME AS "name" ), ( SELECT XMLAGG ( XMLELEMENT (NAME "Proj", XMLATTRIBUTES (P.PROJNO AS "projno", P.PROJNAME AS "name"), ( SELECT XMLAGG ( XMLELEMENT (NAME "Emp", XMLATTRIBUTES(E.EMPNO as "empno"), E.FIRSTNME || ' ' || E.LASTNAME ) ) FROM DSN8810.EMPPROJACT EP, DSN8810.EMP E WHERE EP.PROJNO = P.PROJNO AND EP.EMPNO = E.EMPNO ) ) ) FROM DSN8810.PROJ P WHERE P.DEPTNO = D.DEPTNO ) ) ) FROM DSN8810.DEPT D WHERE D.DEPTNO = 'D01';



XML Functions by Example – 7. result

<Dept deptno="D01" name="DEVELOPMENT CENTER"> <Proj projno="AD3100" name="ADMIN SERVICES"> <Emp empno="000010">CHRISTINE HAAS</Emp> </Proj> <Proj projno="MA2100" name="WELD LINE AUTOMATION"> <Emp empno="000010">CHRISTINE HAAS</Emp> <Emp empno="000110">VINCENZO LUCCHESI</Emp> </Proj></Dept>



XML Functions by Example – 8. HTML table



XML Functions by Example – 8. HTML file

<TABLE border="1"><CAPTION>Department-Employee Table</CAPTION><TR><TH>Dept No</TH><TH>Dept Name</TH><TH>Emp No</TH> <TH>Emp Name</TH><TH>Phone</TH></TR><TR><TD rowspan="6">A00</TD> <TD rowspan="6">SPIFFY COMPUTER SERVICE DIV.</TD></TR><TR><TD>000010</TD><TD>CHRISTINE HAAS</TD><TD>3978</TD></TR><TR><TD>000110</TD><TD>VINCENZO LUCCHESI</TD><TD>3490</TD></TR><TR><TD>000120</TD><TD>SEAN O'CONNELL</TD><TD>2167</TD></TR><TR><TD>200010</TD><TD>DIAN HEMMINGER</TD><TD>3978</TD></TR><TR><TD>200120</TD><TD>GREG ORLANDO</TD><TD>2167</TD></TR><TR><TD rowspan="5">C01</TD> <TD rowspan="5">INFORMATION CENTER</TD></TR><TR><TD>000030</TD><TD>SALLY KWAN</TD><TD>4738</TD></TR><TR><TD>000130</TD><TD>DOLORES QUINTANA</TD><TD>4578</TD></TR><TR><TD>000140</TD><TD>HEATHER NICHOLLS</TD><TD>1793</TD></TR><TR><TD>200140</TD><TD>KIM NATZ</TD><TD>1793</TD></TR></TABLE>



XML Functions by example – 8. the query for HTML table

SELECT VARCHAR( XML2CLOB( XMLElement(NAME "TABLE", XMLATTRIBUTES('1' as "border"), XMLElement(NAME CAPTION, 'Department-Employee Table'), XMLElement(NAME TR, XMLFOREST('Dept No‘ as TH, 'Dept Name‘ as TH, 'Emp No‘ as TH, 'Emp Name‘ as TH, 'Phone‘ as TH) ), XMLAGG( XMLCONCAT( XMLELEMENT(NAME TR, XMLELEMENT(NAME TD, XMLATTRIBUTES( X.CNT+1 as "rowspan"), D.DEPTNO), XMLELEMENT(NAME TD, XMLATTRIBUTES( X.CNT+1 as "rowspan"), D.DEPTNAME) ), ( SELECT XMLAGG(XMLElement(NAME TR, XMLForest(EMPNO as TD, FIRSTNME || ' ' || LASTNAME as TD, PHONENO as TD) ) ) FROM DSN8810.EMP E WHERE E.WORKDEPT = D.DEPTNO ) ) ) ) ) )FROM DSN8810.DEPT D, (SELECT WORKDEPT, COUNT(*) FROM DSN8810.EMP GROUP BY WORKDEPT) X(DEPTNO, CNT)WHERE D.DEPTNO = X.DEPTNO AND D.DEPTNO IN ('A00', 'C01');

}

}

TableHeader

Dept

} Emp



XML Functions by Example-- 9 SOAP message with empty body

<env:Envelope xmlns:env="http://www.w3.org/2002/12/soap-envelope"> <env:Header> <m:reservation xmlns:m= “http://travelcompany.example.org/reservation” env:role=“http://www.w3.org/2002/12/soap-envelope/role/next” env:mustUnderstand="true"> <m:reference>uuid:093a2da1-q345-739r-ba5d-pqff98fe8j7d</m:reference> <m:dateAndTime>2001-11-29T13:20:00.000-05:00</m:dateAndTime> </m:reservation> <n:passenger xmlns:n=“http://mycompany.example.com/employees” env:role=“http://www.w3.org/2002/12/soap-envelope/role/next” env:mustUnderstand="true"> <n:name>Jack Lee</n:name> </n:passenger> </env:Header> <env:Body>… </env:Body></env:Envelope>



XML Functions by Example- 9 code for SOAP msg w/ empty body

XMLELEMENT(NAME “env:Envelope”, XMLNAMESPACES(‘http://www.w3.org/2002/12/soap-envelope’ AS “env”), XMLELEMENT(NAME “env:Header”, XMLELEMENT(NAME “m:reservation”, XMLNAMESPACES(‘http://travelcompany.example.org/reservation’ AS “m”, ‘http://www.w3.org/2002/12/soap-envelope/role/next’ AS “role”), XMLATTRIBUTES(‘true’ AS “env:mustUnderstand”), XMLELEMENT(NAME “m:reference”, ‘uuid:093a2da1-q345-739r-ba5d-pqff98fe8j7d’), XMLELEMENT(NAME “m:dateAndTime”, ‘2001-11-29T13:20:00.000-05:00’) ), XMLELEMENT(NAME “n:passenger”, XMLNAMESPACES( ‘http://mycompany.example.com/employees’ AS “n”, ‘http://www.w3.org/2002/12/soap-envelope/role/next’ AS “role”), XMLATTRIBUTES(‘true’ AS “env:mustUnderstand”), XMLELEMENT(NAME “n:name”, ‘Jack Lee’) ) ), XMLELEMENT(NAME “env:Body”, ‘…’))



XML Functions by Example – 10. recursive document

Base: One level – no recursion

SELECT XML2CLOB(XMLELEMENT(NAME "PartList", XMLAGG(XMLELEMENT(NAME "Part", XMLATTRIBUTES(P.PartID as "PartID"), XMLELEMENT(NAME "PartName", P.PartName) ) ORDER BY P.PartID ) ) )FROM Part P WHERE P.ParentPartID = 0;

Base: One level – no recursion

Part (PartID, PartName, ParentPartID)

<PartList> <Part PartID=“1”> <PartName>Car</PartName> </Part></PartList>




SELECT XML2CLOB(XMLELEMENT(NAME "PartList", XMLAGG(XMLELEMENT(NAME "Part", XMLATTRIBUTES(P.PartID as "PartID"), XMLELEMENT(NAME "PartName", P.PartName), (SELECT XMLAGG(XMLELEMENT(NAME "Part ", XMLATTRIBUTES(S.PartID as "PartID"), XMLELEMENT(NAME "PartName", S.PartName) ) ORDER BY S.PartID ) FROM Part S WHERE S.ParentPartID = P.PartID ) ) ORDER BY P.PartID) ) ) FROM Part P WHERE P.ParentPartID = 0;

Two level recursion: <PartList> <Part PartID=“1”> <PartName>Car</PartName> <Part PartID=“2”> <PartName>Body</PartName> </Part> … </Part></PartList>




Three level recursion:

SELECT XML2CLOB(XMLELEMENT(NAME "PartList", XMLAGG(XMLELEMENT(NAME "Part", XMLATTRIBUTES(P.PartID as "PartID"), XMLELEMENT(NAME "PartName", P.PartName), (SELECT XMLAGG(XMLELEMENT(NAME "Part ", XMLATTRIBUTES(S.PartID as "PartID"), XMLELEMENT(NAME "PartName", S.PartName), (SELECT XMLAGG(XMLELEMENT(NAME "Part", XMLATTRIBUTES(SS.PartID as "PartID"), XMLELEMENT(NAME "PartName", SS.PartName) ) ORDER BY SS.PartID ) FROM Part SS WHERE SS.ParentPartID = S.PartID ) ) ORDER BY S.PartID ) FROM Part S WHERE S.ParentPartID = P.PartID ) ) ORDER BY P.PartID) ) ) FROM Part P WHERE P.ParentPartID = 0;




XML Functions by Example – 10. recursive document sample result

<PartList> <Part PartID=“1”> <PartName>Car</PartName> <Part PartID=“2”> <PartName>Body</PartName> <Part PartID=“3”> <PartName>Frame</PartName> </Part> <Part PartID=“4”> <PartName>Doors</PartName> </Part> </Part> <Part PartID=“5”> <PartName>Wheels</PartName> </Part> </Part></PartList>




SQL to XML Mappings

• SQL <identifier>s to XML names– Escaping for column names SQL: "Order Items" => XML: "Order_x0020_Items" SQL: "XMLDoc" => XML: "_x0078_MLDoc" SQL: "dept:id" => XML: "dept_x003A_id“

• SQL values to XML valuesSQL: 'Shipping & Receiving’

=> XML: 'Shipping & Receiving'SQL: 'X < 5' => XML: 'X < 5'SQL: '2002-02-26-10.01.01.123456'

=> XML: '2002-02-26T10:01:01.123456'



XML Functions in Views

• Need to use XML2CLOB

• Using CLOB result for XML in other XML constructors will lead to surprising result.

• “<Emp>Jack Lee</Emp>” will be become “<Emp>Jack Lee</Emp>” if it’s used in XMLELEMENT as content.



Namespace Scoping

• Namespaces declared by XMLNAMESPACES are syntactically scoped within the XMLELEMENT/XMLFOREST function, e.g. XMLELEMENT(…, NAMESPACES(…as “ns”) …).

• Namespace prefixes need to be declared in-scope except for these pre-defined namespace prefixes: xml, xs, xsd, xsi, sqlxml

startend



Major Optimization Techniques (DB2 z/OS)• Flatten nested XML scalar functions into one function and a tagging

template - avoid data copy

• Two phase processing for XML:– Map SQL values to XML values and build in-memory records before

tagging

– Apply tagging templates to generate XML string (Serialize)

• Materialized XML values in workfile contains only the handles to XML values in memory

• Optimize XMLAGG by internal quicksort for ORDER BY, and sharing tagging template for its argument

• Serialized XML text has no superfluous namespace declarations

• XML2CLOB materializes CLOB in-memory only.



Composition by DB2 SQL/XML Publishing functions in V8

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Tim

e (s

ec)

1M 2M 3M 4M 6M

Doc size

Elapsed

CPU

Z990 5 processors, 10GB,z/OS 1.4,DB2 for z/OS V8



Composition: XML Extender v.s. Publishing V8

0

5

10

15

20

25

30

1M 2M 3M 4M 5.8M

COMP Elapsed SQL/XML Elasped

XML Extender for DB2 for z/OS V8

IXM4C40

IXMLC13



Preview of Future SQL/XML Features

DB2 Engine

21

3

4

56

7

8 9Application Textual XML

Relational

XML

1

2

3

4

5

6

7

8

9

Bind in XML

Store as XML

Shred into SQL

Retrieve XML

Publish XML

Bind out XML

XML to XML

XML to SQL

SQL to XML

0 Data model



Summary

• SQL/XML publishing functions in DB2 V8

• Performance characteristics (z/OS)

• Preview of some future SQL/XML features



Thanks!

Guogen (Gene) Zhang, [email protected]

Generating XML from Relational Data ...

Documents

Transcript of Generating XML from Relational Data ...