Interprocedural Shape Analysis for Recursive Programs Noam Rinetzky Mooly Sagiv.
Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive...
-
Upload
june-booth -
Category
Documents
-
view
219 -
download
1
Transcript of Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive...
![Page 1: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/1.jpg)
Outline
• Logistics
• Review
• Recursive Plans– Datalog programs vs. conjunctive queries– Recursive Programs– Maximality
• Effect of Functional dependencies
• Effect of Binding pattern restrictions
• Construction of maximal recursive plans
![Page 2: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/2.jpg)
Logistics
• Site Descriptions due today– Effects of omitting (redundant) world relations in site description?
• Wrappers due +1 wk– What about sites that return redundant garbage?– Should wrappers filter?
• Project writeups due 6/11
![Page 3: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/3.jpg)
Course Topics by Week• Search & Constraint Satisfaction
• Knowledge Representation 1: Propositional Logic• Autonomous Spacecraft 1: Configuration Mgmt
• Autonomous Spacecraft 2: Reactive Planning• Information Integration 1: Knowledge Representation
• Information Integration 2: Planning & Execution• Supervised Learning & Datamining
• Reinforcement Learning
• Bayes Nets: Inference & Learning
• Review & Future Forecast
![Page 4: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/4.jpg)
Demos• Parallel Aggregation Engine
– www.metacrawler.com– Fixed set of wrappers– One step plans
• Search Broker– sb.cs.arizona.edu/sb– Uses first word to select source; routes query – No aggregation– One step plans
• Parallel Shopping Agent– jango.excite.com– Matches queries to best wrappers from set of 500– One step plans
![Page 5: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/5.jpg)
Motivation: Info Integration
• Want agent such that
• User says what she wants
• Softbot determines how & when to achieve it
• Example:– Show me all reviews of movies starring Marlon
Brando that are currently playing in Seattle
Ebert
IMDB Spot
ShowT
![Page 6: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/6.jpg)
Knowledge Representation
Propositional Logic
Relational Algebra
Datalog
First-Order Predicate Calculus
Bayes NetworksDescription
Logic(s)
![Page 7: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/7.jpg)
Relational Algebra• Union
• Intersection
• Subtraction
• Selection
• Projection
• Cartesian Product
• Join
![Page 8: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/8.jpg)
Name SSNJohn 999999999Tony 777777777
EmployeeSSN Dname999999999 Emily777777777 Dave777777777 Joe
Name SSN DnameJohn 999999999 EmilyTony 777777777 DaveTony 777777777 Joe
Example: Empl JOIN Dependents
Empl
Result of Join
where join conditionis Empl.SSN=Dependents.EmployeeSSN
Dependents
![Page 9: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/9.jpg)
9
Propositional. Logic vs First Order
Ontology
Syntax
Semantics
Inference
Facts: P, Q
Atomic sentencesConnectives
Truth Tables
NPC, but SAT algos work well
Objects (e.g. Dan)Properties (e.g. mother-of)Relations (e.g. female)Variables & quantificationSentences have structure: termsfemale(mother-of(X)))
Interpretations (Much more complicated)
Undecidable, but theorem proving works sometimesLook for tractable subsets
![Page 10: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/10.jpg)
Datalog Rules, Programs & QueriesA pure datalog rule (e.g. first-order horn clause with a positive literal)has the following form:
head :- atom1, atom2, …., atom,… where all the atoms are non-negated and relational.
BritishProduct(X) :- Product(X,Y,P) & Company(P, “UK”, SP)
A datalog program is a set of datalog rules.A program with a single rule is a conjunctive query.
We distinguish EDB predicates and IDB predicates• EDB’s are stored in the database, appear only in the bodies• IDB’s are intensionally defined, appear in both bodies and heads.
![Page 11: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/11.jpg)
The Meaning of Datalog Rules
Repeat the following until you cannot derive any new facts:Consider every assignment from the variables in the bodyto the constants in the database.
If each of the atoms in the body is made true by the assignment,
then
add the tuple for the head into the relation of the head.
Start with the facts in the EDB and iteratively derive facts for IDBs.
![Page 12: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/12.jpg)
Correspondence: Datalog ~ Relational Algebra
EmployeeName SSNJohn 999999999Tony 777777777DependentsEmployeeSSN Dname999999999 Emily777777777 Joe
ED(Name, SSN, Dname) :- Employee(Name, SSN) & Dependents(SSN, Dname)
EDName SSN DnameJohn 999999999 EmilyTony 777777777 Joe
Given: EDBs
Define: IDB
![Page 13: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/13.jpg)
IIIIS Representation I• World Ontology
– Defines predicates of relational schemata– E.g.,
• actor-in (Movie, Part, Name), • review-of (Movie, Part) • year-of (Movie, Year)• shows-in (Movie, City, Theatre)
– User uses this language to specify queries– You use language to specify content of info sites
![Page 14: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/14.jpg)
IIIIS Representation II: • Queries
Find-all (M, Review, brando, seattle)Such That actor-in(M, Part, brando) &
shows-in(M, seattle, T) &review-of(M, Review)
• Writen in Datalog:
query(M, R, Brando, Seattle) actor-in(M, Part, brando) & shows-in(M, seattle, T) & review-of(M, R)
![Page 15: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/15.jpg)
IIIIIS Representation III• Information Source Functionality
– Info Required? $ Binding Patterns
– Info Returned?
– Mapping to World Ontology
Source may be incomplete: (not )
IMDBActor($Actor, M) actor-in(M, Part, Actor)
Spot($M, Rev, Y) review-of(M, Rev) &year-of(M, Y)
Sidewalk($C, M, Th) shows-in(M, C, Th)
•For Example
[Rajaraman95]
![Page 16: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/16.jpg)
Unsafe Rules
IMDBActor($Actor, M) actor-in(M, Part, Actor)
Sidewalk($C, M, Th, Time) shows-in(M, C, Th)
• All variables on left-hand-side must appear on right-hand-side– Otherwise rule is “unsafe”
• Converse not necessary– RHS var (e.g. Part) is existentially quantified
![Page 17: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/17.jpg)
Two Questions
• How find a valid solution plans?– Search...– Search-free synthesis of maximal recursive plan
• How verify a plan answers query?1. Verify information content of plan
• Same as DB problem of rewriting queries using views• Show expansion of plan equivalent to query• Technique of query containment• P Q iff P Q and Q P
2. Verifying binding pattern constraints
![Page 18: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/18.jpg)
A Plan to Solve the Query
IMDBActor($Actor, M) actor-in(M, Part, Actor) Spot($M, Rev, Y) review-of(M, Rev) &
year-of(M, Y)Sidewalk($C, M, Th) shows-in(M, C, Th)
query(M, R, b, s) actor-in(M, Part, b) &shows-in(M, s, T) &review-of(M, R)
plan(M, R, b, s) IMDBActor(b, M) &Sidewalk(s, M, Th) &Spot(M, R, Y)
plan'(M, R, b, s) actor-in(M, P, A) & review-of(M, R) & year-of(M, Y) & shows-in(M, C, T)
: M -> M Part -> Pb -> As -> CR -> R
![Page 19: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/19.jpg)
How verify this plan answers query?1. Verify information content of plan
2. Verifying binding pattern constraints
IMDBActor($Actor, M) actor-in(M, Part, Actor) Spot($M, Rev, Y) review-of(M, Rev) &
year-of(M, Y)Sidewalk($C, M, Th) shows-in(M, C, Th)
plan(M, R, brando, seattle) IMDBActor(b, M) &Sidewalk(s, M, Th) &Spot(M, R, Y)
![Page 20: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/20.jpg)
Outline
• Logistics
• Review
• Recursive Plans– Datalog programs vs. conjunctive queries– Recursive Programs– Maximality
• Effect of Functional dependencies
• Effect of Binding pattern restrictions
• Construction of maximal recursive plans
![Page 21: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/21.jpg)
Problem: Want All the Reviews
• Many review sites
• None is complete
• Might wish to go to several review sources….
plan1(M, R, brando, seattle) IMDBActor(b, M) &Sidewalk(s, M, Th) &Spot(M, R, Y)
plan2(M, R, brando, seattle) IMDBActor(b, M) &Sidewalk(s, M, Th) &RevOf(M, R)
RevOf(M, R) => Spot(M, R, Y)RevOf(M, R) => Ebert(M, R, Y)RevOf(M, R) => ...
![Page 22: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/22.jpg)
Maximality
• Ideal: Plan Query– I.e. Plan Query Query Plan– With most real info sources, one can never guarentee completeness
• Soundness: Plan Query• Plan Pis Maximal if...
Pi if Pi Query then Pi P
![Page 23: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/23.jpg)
Datalog programs
• New predicates subroutines
plan(M, R, brando, seattle) IMDBActor(brando, M) &ShowsInCity(M, seattle) &RevOf(M, R)
ShowsInCity(M, A, C) => Sidewalk(s, M, Th) ShowsInCity(M, A, C) => MetroCinema(s, M, Th)ShowsInCity(M, A, C) => ...RevOf(M, R) => Spot(M, R, Y)RevOf(M, R) => Ebert(M, R, Y)RevOf(M, R) => ...
![Page 24: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/24.jpg)
Datalog Program Dataflow Diagram
Start
IMDB
Metro-cinema
Sidewalk
Ebert
Spot
End
Brando
Movie
Review
Movie
Movie
Movie
Mov
ie
ReviewMovie
Movie
Seattle
Seattle
Movie
Movie ReviewX
+
+
![Page 25: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/25.jpg)
Recursive Query PlansConjunctiveQuery Plan
RecursiveQuery Plan
Number ofqueriesdependson data
![Page 26: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/26.jpg)
Solution to Two Open Problems
Two open query planning problems can be solved usingrecursive query plans:
Functional dependencies in domain model1
2 Limitations on binding patterns of sources
address_server ($Name,Address,Phone)
Name must be provided to query for Address and Phone
White_pages(Name,Address,Phone)Phone AddressEvery phone number corresponds to a single address.
6.1 [Duschka & Levy]
![Page 27: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/27.jpg)
Example
Flight DatabaseSan Francisco Intl. Airport
Flight DatabaseUnited Airlines
Schedule of pilots and aircraftsFlight information
Pilot’s WorkSchedule
![Page 28: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/28.jpg)
World Relations & Source RelationsWorld Relations:
flight(Airline,Flight_no,From,To)Flight Information
Source Relations:
Schedule of pilots and aircrafts
schedule(Airline,Flight_no,Date,Pilot,Aircraft)
Flight database San Francisco International Airport
Flight database United Airlines
Pilot’s work schedule
sfo(Airline,Flight_no,To)
united(Flight_no,From,To)
ws(Date,From,To,Pilot,Aircraft)
![Page 29: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/29.jpg)
Source DescriptionsFlight database San Francisco International Airport
Flight database United Airlines
Pilot’s work schedule
sfo(Airline,Flight_no,To)
united(Flight_no,From,To)
ws(Date,From,To,Pilot,Aircraft)
=> flight(Airline,Flight_no,sfo,To)
=> flight(ua,Flight_no,From,To)
=> flight(Airline,Flight_no,From,To) &schedule(Airline,Flight_no,Date,Pilot,Aircraft)
![Page 30: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/30.jpg)
Functional Dependencies
flight(Airline,Flight_no,From,To
Airlines assign flight numbers to specific routes:Airline, Flight_no FromAirline, Flight_no To
schedule(Airline,Flight_no,Date,Pilot,Aircraft)
Pilots work for only one airline:Pilot Airline
Different airlines don’t own same aircraft:Aircraft Airline
![Page 31: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/31.jpg)
Example Query
Query:Which pilots work for the same airline as Mike?
q(Pilot) <=> schedule(Airline,Flight_no1,Date1,Pilot,Aircraft1) & schedule(Airline,Flight_no2,Date2,mike,Aircraft2)
Answers can be retrieved from Pilot’s Work Schedule:
ws(Date,From,To,Pilot,Aircraft) => flight(Airline,Flight_no,From,To) & schedule(Airline,Flight_no,Date,Pilot,Aircraft)
![Page 32: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/32.jpg)
More Answers because of FDsws(Date,From,To,Pilot,Aircraft)=> flight(Airline,Flight_no,From,To) & schedule(Airline,Flight_no,Date,Pilot,Aircraft)
Functional dependencies:
Pilot AirlineAircraft Airline
wsDate From To Pilot Aircraft08/28 sfo nrt mike #11108/29 nrt sfo ann #11109/03 sfo fra ann #22209/04 fra sfo john #222… … … … ...
same aircraft, therefore same airline
same aircraft, therefore same airline
same aircraft, therefore same airline
Therefore, all these pilotswork for the same airline
![Page 33: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/33.jpg)
Data Dependency
Query:Which pilots work for the same airline as Mike?
Query plan (not maximal):q(Pilot) <=> ws(Date1,From1,To1,mike,Aircraft1) & ws(Date2,From2,To2,Pilot2,Aircraft1) &
ws(Date3,From3,To3,Pilot2,Aircraft2) &ws(Date4,From4,To4,Pilot,Aircraft2)
Depending on the data in the sources,query plans with 1, 2, 4, 6, 8, … subgoals areneeded to extract all available information
![Page 34: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/34.jpg)
Maximal Query Plan
Recursive query plan (maximal):q(mike) => ws(Date,From,To,mike,Aircraft)a(Aircraft) => ws(Date,From,To,mike,Aircraft)q(Pilot) => ws(Date,From,To,Pilot,Aircraft) &
a(Aircraft)a(Aircraft) => ws(Date,From,To,Pilot,Aircraft) &
q(Pilot)
Is it possible to generate maximalrecursive query plans automatically?
![Page 35: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/35.jpg)
Limitations on Binding Patterns
Flight database San Francisco Intl. Airport
sfo ($Airline, Flight_no, To)
Given airline, source returns flight number anddestination airports.
Flight database United Airline
united (Flight_no, $From, To)
Given airport of departure, source returns flightnumbers and destination airports.
![Page 36: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/36.jpg)
q(City1,City2) <=> sfo ($ua,N1,City1) & united (N2,$City1,City2)
Data Dependency
Query:Which connections are served by United Airlines?q(From,To) <=> flight(ua,Flight_no,From,To)
Query plan (not maximal):q(City3,City4) <=> sfo ($ua,N1,City1) & united (N2,$City1,City2) &
united (N3,$City2,City3) &united (N4,$City3,City4)
Depending on the data in the sources,query plans with 1, 2, 3, … subgoals are
needed to extract all available information
![Page 37: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/37.jpg)
Maximal Query Plan
Recursive query plan (maximal):
q(sfo,To) => sfo ($ua,N,To)
q(sfo,To) => united (N,$sfo,To)
q(From,To) => q(A,From) & united (N,$From,To)
Is it possible to generate maximalrecursive query plans automatically?
![Page 38: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/38.jpg)
Dataflow ViewStart
SFO
United
End ua
sfo, T
o
From, Tosfo, To
sfo
+
q(sfo,To) <=> sfo ($ua,N,To)q(sfo,To) <=> united (N,$sfo,To)q(From,To) <=> q(A,From) & united (N,$From,To)
United
select
From
From, To
q
![Page 39: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/39.jpg)
Outline
• Logistics
• Review
• Recursive Plans– Datalog programs vs. conjunctive queries– Recursive Programs– Maximality
• Effect of Functional dependencies
• Effect of Binding pattern restrictions
• Construction of maximal recursive plans
![Page 40: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/40.jpg)
Overview of Construction
User query
Source descriptions
Functionaldependencies
Limitations onbinding patterns
Recursive query plan
Rectifieduser query
Inverse rules
Chase rules
Domain rules
Transitivity rule
![Page 41: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/41.jpg)
Skolem Functions
• First-order logic is great because and right?
• Can eliminate existenially quantified vars!– Replace with a skolem function
X president_of(usa, X)– president_of(usa, bill_clinton)– president_of(usa, f())
D T dog(D) has-tail(D, T) D dog(D) has-tail(D, g(D))– g is the “tail-choosing” function; it maps dogs to their tails– Skolem function takes all (only!) preceding vars as args
![Page 42: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/42.jpg)
Inverse RulesSource description
ws(Date,From,To,Pilot,Aircraft)=> flight(Airline,Flight_no,From,To) & schedule(Airline,Flight_no,Date,Pilot,Aircraft)
Inverse rules
flight(f(D,F,T,P,A),g(D,F,T,P,A),F,T) <= ws(D,F,T,P,A)schedule(f(D,F,T,P,A),g(D,F,T,P,A),D,P,A) <= ws(D,F,T,P,A)
variable Airline is replaced by a function term whosearguments are the variables in the source relation
![Page 43: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/43.jpg)
ExamplewsDate From To Pilot Aircraft08/28 sfo nrt mike #11108/29 nrt sfo ann #11109/03 sfo fra ann #22209/04 fra sfo john #222
flightAirline Flight_no From To
?1 ?2 sfo nrt?3 ?4 nrt sfo?5 ?6 sfo fra?7 ?8 fra sfo
scheduleAirline Flight_no Date Pilot Aircraft
?1 ?2 08/28 mike #111?3 ?4 08/29 ann #111?5 ?6 09/03 ann #222?7 ?8 09/04 john #222
InverseRules
![Page 44: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/44.jpg)
Handling Skolem Functions
• As a first cut can discard all inverse rules with skolem functions
• Only useful if doing functional dependencies
![Page 45: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/45.jpg)
The ChaseAirline Flight_noDate Pilot Aircraft
?1 ?2 08/28 mike #111?3 ?4 08/29 ann #111?5 ?6 09/03 ann #222?7 ?8 09/04 john #222
Airline Flight_noDate Pilot Aircraft
?1 ?2 08/28 mike #111?1 ?4 08/29 ann #111?5 ?6 09/03 ann #222?7 ?8 09/04 john #222
Pilot AirlineTherefore: ?5 = ?1
Aircraft AirlineTherefore: ?3 = ?1
Airline Flight_noDate Pilot Aircraft
?1 ?2 08/28 mike #111?1 ?4 08/29 ann #111?1 ?6 09/03 ann #222?7 ?8 09/04 john #222
Aircraft AirlineTherefore: ?7 = ?1
![Page 46: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/46.jpg)
Chase Rules
Chase rule for functional dependency “Aircraft Airline”:
e(Airline1,Airline2) <= schedule(Airline1,N1,D1,P1,Aircraft1) &schedule(Airline2,N2,D2,P2,Aircraft2) &e(Aircraft1,Aircraft2)
Chase rule for functional dependency “Pilot Airline”:
e(Airline1,Airline2) <= schedule(Airline1,N1,D1,Pilot1,A1) &schedule(Airline2,N2,D2,Pilot2,A2) &e(Pilot1,Pilot2)
![Page 47: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/47.jpg)
Domain RulesFlight database San Francisco Intl. Airport
sfo ($Airline, Flight_no, To)Given airline, source returns flight number anddestination airports.
Flight database United AirlineUnited (Flight_no, $From, To)
Given airport of departure, source returns flightnumbers and destination airports.
• Can’t use United source unless know originating arirport names
• Can use SFO (and United!) sources to “prime” the pump for the United source
![Page 48: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/48.jpg)
Priming the Pump
• Instead of– q(From, To) => United(FlightNum, $From, To)
• We’ll write– q(From, To) => AllPossibleAirports(From) &
United(FlightNum, $From, To)– AllPossibleAirports(Name) => …
• Must generate these domain rules automatically – Paper generates one domain predicate– You should generate one per type
![Page 49: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/49.jpg)
Generating Domain Rules
Given:Source1($A, $B, $C, X, Y, Z) => ….
WhereA has is of TypeA, B is of TypeB …
Generate the following rulesTypeX(X) <= TypeA(A) & TypeB(B) & TypeC(C) &
Source1(A, B, C, X, Y, Z)TypeY(Y) <= …...
![Page 50: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/50.jpg)
Maximality of Constructed Plan
Theorem:
Given a user query, source descriptions, functional dependencies, and limitations,
(i) rectified user query,(ii) inverse rules,
(iii) chase rules,(iv) domain rules, and the(v) transitivity rule
is a maximal query plan.
![Page 51: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/51.jpg)
Arithmetic Predicates <
• Optional
• Must ensure that all variables are bound before < is called
• Must add arithmetic to your engine– (along with join…)
• Can ignore < if your engine doesn’t support them
![Page 52: Outline Logistics Review Recursive Plans –Datalog programs vs. conjunctive queries –Recursive Programs –Maximality Effect of Functional dependencies Effect.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649e665503460f94b60e92/html5/thumbnails/52.jpg)
Conclusions• Conjunctive query plans are insufficient to handle functional
dependencies and limitations on binding patterns.
• Recursive query plans can produce maximal sets of answers, even in the presence of functional dependencies and limitations on binding patterns.
• Recursive query plans can be easily constructed in polynomial time.