Achieving Adaptivity for OLAP-XML Federations

Achieving Adaptivity for OLAP-XML Federations

Torben Bach PedersenAalborg University

Joint work with Dennis Pedersen, TARGIT

Torben Bach Pedersen · DOLAP 2003 · 04/22/232

Overview• Background: OLAP-XML federations• New challenges

– XML data changes– Slow or unreliable XML sources– Schema changes in data sources– Other challenges

• Integration in TARGIT architecture• Other applications of the techniques• Conclusion and future work• Related work


Data Warehousing & OLAPMultidimensional analysis: TARGIT Analysis


OLAP• Good for complex ad hoc queries

– Simple: natural, graphical queries– Fast: pre-aggregation

• A number of problems with physical integration– Short-term and varying data needs

• Population, product info, ...– Dynamical data

• Stock quotes, competitor pricing, ...– Data with limited access

• Competitor product info, public databases, ...


OLAP-XML Federations

OLAP-server

Client

Cube

Traditional OLAP architecture:


OLAP-XML Federations• Logical integration of XML data

– External dimensions– External measures

• Data combined at query time

Federation

Client

XMLCube


OLAP-XML Federations• Logical integration of XML data

– External dimensions– External measures

• Data combined at query time• Transparent for users• Flexible: many XML sources• Quick: running in a few mins• Data is always fresh• Performance often comparable to

physical integration

Federation

Client

XMLCube Cube


XPath Queries for Fetching XML<Books>

<Book><Title>1984</Title><Author>Orwell</Author>

</Book><Book>

<Title>Of Mice and Men</Title>

<Author>Steinbeck</Author></Book>

</Books>

/Books/Book[Author=”Steinbeck”]/Title

Federation

Client

XMLCube Cube

XPath

Dimension value


Old And New TARGIT Architecture


New Challenges• Our previous work focused on basic aspects

– Flexibility– General performance– Implementation

• New: what can go wrong? – need for adaptivity– XML data changes– XML sources slow or unreliable– Schema changes (XML, OLAP, federation)

• We often have no control over the XML sources• A solution has broad interest: views over XML sources


XML Data Changes• Basic federation

– XML data is integrated at query time => XML data changes handled automatically

• However, XML data is cached for performance– Cache timeout value ensures fresh data (set

manually or automatically)– 0 cache timeout => always fetch from source

• Only few current XML databases inform about changes– Xyleme allows users to subscribe to changes– Only delta should be transferred


ICE: Information and Content Exchange• Protocol proposed by W3C for automatically informing

about and requesting changes– Supported by major vendors– Push: subscribe to changes and keep cache up-to-date– Pull: request changes from source at query time


Slow and Unreliable XML Sources• Overload, maintenance, HW breakdown, attacks

– Often we no influence on this• Incremental presentation for user

– What if source is too slow or no reply at all? • Inform user that the system is not working…?• Specification of alternative sources

– Several queries per external dimension/measure– Increased fault tolerance, also better performance

Source Server Client


Slow and Unreliable XML Sources• Start several queries and use the fastest

– Always uses the fastest, but heavy load on sources– Use first response time as indicator for total time

• Start one query at a time

• Minimal load on sources, but slower

Fed Fed

Fed Fed?

Fed123

123

312


Slow and Unreliable XML SourcesAlternative sources of lower

quality: better than no data?

Alternatives– Expired cache data– Google, Xyleme, The

WayBack Machine– Backup-disk, tape– Etc.

Source Speed Quality

Local cache Fastest Fresh

Original source Fast? Freshest

Expired cache Fastest Old

Backup source Fast/slow Very old


Slow and Unreliable XML SourcesIn practice?Sources with equal priority chosen at random


Result: Algorithm for Fetching XML Data


Experiments• 1st experiment: fetching a 137 KB dimension

– Start 8 queries, when first 3 respond, (cancel) last 5, when fastest query finish, (cancel) remaining 2

– Fast reply = good indication of overall speed• 2nd experiment: search local cache, then Google cache


Schema Changes In XML SourcesHow to synchronize XML views after schema change?

(solution described in separate paper)

Bibliography

Publisher

PName Book

Author

AName

Title Price

/Bibliography/Author[AName=”Orwell”]/Book/Title

Bibliography

Publisher

PName

Book

Author

AName

Title Price


Additional Challenges• Changes to federation schema

– Cache may be invalidated– Discard affected cache results (unproblematic)

• OLAP data changes– Cache may be invalidated– Less frequent than XML data changes => cache will

often have expired anyway• OLAP schema changes

– Federated schema may be invalidated– Rare and easy to detect (and correct)


Integrating Techniques - Architecture


Integrating Techniques – Query Processing• Query Evaluator splits query into XML+OLAP parts

and determines query plan based on cost• Execution Engine coordinates and executes plan• Cache Manager maintains cache, e.g., through ICE• XML Component interface fetches XML data, chooses

between available XML sources (Algorithm 1)• View Synchronizer handles schema changes• Metadata Manager manages info about external

dimensions and measures + XML component characteristics


Other ApplicationsAll XPath-based views on XML dataLinks to parts of XML documents• Web pages• Documents (DocBook)• Software applicationsand many more…

Automatic recreation of broken linksIncreased fault tolerance and performance using

alternative sources

?


Conclusion and Future Work• Operational problems in OLAP-XML federations• XML data changes• Slow and unreliable XML sources

– Using several sources (Algorithm 1)– Experiment with Algorithm1

• Techniques integrated into federation architecture• Schema evolution and other challenges• Future work

– TARGIT implementation and testing– Using techniques in other applications


Related Work• Data changes in XML/semistructured documents

– Xyleme + Zhuge • Schema changes in scientific documents

– Not XML• Adaptive/dynamic query optimization

– Telegraph project– We use once per source, rather than per tuple

• Does not consider one or more of: OLAP+XML concepts, schema changes, slow and unreliable sources

• Own previous OLAP-XML work is not adaptive

Achieving Adaptivity for OLAP-XML Federations

Documents

Transcript of Achieving Adaptivity for OLAP-XML Federations