Entity Ranking and Relationship Queries Using an Extended Graph Model
1 Extended SQL for Multimedia Queries select from where orderby
-
date post
21-Dec-2015 -
Category
Documents
-
view
220 -
download
0
Transcript of 1 Extended SQL for Multimedia Queries select from where orderby
![Page 1: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/1.jpg)
1
Extended SQL for Multimedia Queries
select <whats> from <whatr>where <preds>orderby <whato)groupby <whatg>
<whats> can be alphanum, images, functions<functions> can be numeric-valued, vector- valued (such as histograms), image-valued (such as extractors), aggregates,….<preds> can be hard constraints or similarity functions<whato) can be alphanum or similarities<whatg> can be alphanum or similarities
![Page 2: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/2.jpg)
2
Queries from Current System
select A.id, A.img, simfMRI(A.img, B.img)from fMRI A, fMRI Bwhere B.id = 1 orderby simfMRI(A.img,B.img)
select A.id, I.img, S.probfrom fMRI A, simfMRI Swhere S.img = A.imgorderby S.prob
fMRI simfMRIid img img prob
*Assumes simfMRI is a global comparison of 2 fMRI images.
similarities of all images to aselected one withid = 1
![Page 3: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/3.jpg)
3
Queries from Current System
select A.id, A.imgfrom fMRI A, fMRI Bwhere B.id = 1 and simfMRI(A.img, B.img)
select distinct A.id, A.imgfrom fMRI A, simfMRI Swhere S.fid = A.fid
fMRI simfMRIfid id img img prob
*Assumes simfMRI is a global comparison of 2 fMRI images.
similarities of all images to aselected one withid = 1
User query:
MystiQ query:
![Page 4: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/4.jpg)
4
Potential Queries from Very Simple and Beyond
select simfMRI(A.img,B.img)from fMRI A, fMRI Bwhere A.id = 12 and B.id = 37
Return the similarity between the fMRI images ofpatient 12 and patient 37.
fMRI simfMRIid img prob single similarity value
![Page 5: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/5.jpg)
5
Find pairs of patients in the database having the same age and similar fMRI images.
select A.id, B.id, P.age, simfMRI(A.img, B.img)from patient P, patient Q, fMRI A, fMRI Bwhere P.ID = A.id and Q.ID = B.id and A.age = B.ageorderby simfMRI(A.img, B.img)
patient fMRI simfMRIid age id img img img prob
pairs of database images inorder of similarity
Q3
![Page 6: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/6.jpg)
6
Find pairs of patients in the database having the same age and similar fMRI images.
select A.id, B.id, P.age, from patient P, patient Q, fMRI A, fMRI Bwhere P.ID = A.id and Q.ID = B.id and A.age = B.age and simfMRI(A.img, B.img)
patient fMRI simfMRIid age fid1 fid2 prob
pairs of databaseimages
Q3
fid id img
User query:
select distinct A.id, B.id, P.age, from patient P, patient Q, fMRI A, fMRI B, simfMRIwhere P.ID = A.id and Q.ID = B.id and A.age = B.age and simfMRI.fid1 = A.fid and simfMRI.fid2=B.fid
MystiQ query:
Note: twopatients A.id,B.id may havemultiple similarimages. Theprobability is non-trivial.
![Page 7: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/7.jpg)
7
Given a query image called thisimg, return the id’sand similarities for database images that are similarenough.
select A.id, P.age, A.img, simfMRI(thisimg, A.img)where P.ID = A.id and simfMRI(thisimg,A.img) > threshorderby simfMRI(thisimg, A.img)
thisimg thresh patient fMRI simfMRIid age id img img prob
Q4
![Page 8: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/8.jpg)
8
Given a query image called thisimg, return the id’sand similarities for database images that are similarenough. select A.id, P.age, A.img
from patient P, fMRI Awhere P.ID = A.id and simfMRI(thisimg,A.img) > thresh and simfMRI(thisimg, A.img)
thisimg patient fMRI simfMRI
id age
Q4
User query:
MystiQ query:fid probfid id imgimg
select A.id, P.age, A.img, simfMRI(thisimg, A.img)from patient P, fMRI A, simFMRI Swhere P.ID = A.id and simfMRI(thisimg,A.img) > thresh and A.fid = S.fid
![Page 9: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/9.jpg)
9
Given a query image called thisimg, return the id’sand similarities for database images that are similarenough.
select A.id, P.age, A.imgfrom patient P, fMRI Awhere P.ID = A.id and and HIGH(simfMRI(thisimg,A.img)) [weight=7] and simfMRI(thisimg, A.img) [weight=0]
thisimg patient fMRI simfMRI
id age
Q4
Better user query:
fid probfid id imgimg
![Page 10: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/10.jpg)
10
Weights
I’m proposing extending the probabilistic predicatesand/or probabilistic tables with weightsE.g. FROM simfMRI S [weight=3.5]WHERE simfMRI(img1,img2) [weight=-0.5]
Default weight = 0.0Positive weight means the predicate is more importantNegative weight means it is less important.
![Page 11: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/11.jpg)
11
High/Low Predicates
I’m also proposing introducing two newprobabilistic predicates:
High(value) = more “true” the higher “value” is
Low(value) = more “true” the lower “value” is
These predicates make sense only within a query:Different values are compared and the higher (lower)receive higher probability.
![Page 12: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/12.jpg)
12
Given an image and a location, return id’s, images, and activation levels from database images that have high enough activation levels at that location in order bysimilarity at that particular location.
select A.id, A.img, activation(A.img,loc), simsignal(signal(A.img,loc), signal(thisimg,loc))from fMRI Awhere activation(A.img,loc) > threshorderby simsignal(signal(A.img,loc), signal(thisimg,loc)
thisimg loc thresh fMRI simsignal activation signal
signal prob(, ) (,)
activation is a function that returns the mean activation level at a given location.signal is a function that returns the signal at a given location.locations denote particular voxels.
Q5
fid id img
![Page 13: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/13.jpg)
13
Given an image and a location, return id’s, images, and activation levels from database images that have high enough activation levels at that location in order bysimilarity at that particular location.
select A.id, A.img, activation(A.img,loc)from fMRI A, thisimg Twhere activation(A.img,loc) > thresh and simsignal(signal(A.img,loc), signal(T,loc))
thisimg loc thresh fMRI simsignal activation signal
signal prob (, ) (,)
activation is a function that returns the mean activation level at a given location.signal is a function that returns the signal at a given location.locations denote particular voxels.
Q5
User query:
fid id img
![Page 14: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/14.jpg)
14
Given an image and a location, return id’s, images, and activation levels from database images that have high enough activation levels at that location in order bysimilarity at that particular location.
select A.id, A.img, activation(A.img,loc)from fMRI A, thisimg Twhere high(activation(A.img,loc)) [weight=10.0] and simsignal(signal(A.img,loc), signal(T,loc))
thisimg loc thresh fMRI simsignal activation signal
signal prob(, ) (,)
activation is a function that returns the mean activation level at a given location.signal is a function that returns the signal at a given location.locations denote particular voxels.
Q5
Better user query:
fid id img
![Page 15: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/15.jpg)
15
Return ids and images for images (A) that have thehighest activation level of all database images (B) at a given location. For each of these, return id’s andimages (C) of all database images whose activationis above a threshold at that location, ordered bytheir signal similarity to A at that location.
select A.id, A.img, C.id, C.imgfrom fMRI A, fMRI B, fMRI Cwhere activation(A.img,loc) = max(activation(B.img,loc)) and activation(C.img,loc) > threshorderby simsignal(signal(A.img,loc),signal(C.img,loc))
loc thresh fMRI simsignal activation signalid img signal prob (, ) (,)
Q6
![Page 16: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/16.jpg)
16
Return ids and images for images (A) that have thehighest activation level of all database images at a given location. For each of these, return id’s andimages (C) of all database images whose activationis above a threshold at that location, ordered bytheir signal similarity to A at that location.
select A.id, A.img, C.id, C.imgfrom fMRI A, fMRI Cwhere high(activation(A.img,loc)) [weight = 100.0]and high(activation(C.img,loc)) [weight = 0.0]and simsignal(signal(A.img,loc),signal(C.img,loc)) [weight=0.0]
loc thresh fMRI simsignal activation signalsignal prob (, ) (,)
Q6
User query:
fid id img
![Page 17: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/17.jpg)
17
Extract and return a neighborhood of a given size abouta given location from a given query image.
select nbd(thisimg, loc, size)from thisimg
thisimg loc size
Simple example of an extraction function.
Q7
![Page 18: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/18.jpg)
18
select A.id, A.img, I.loc, simnbd(nbd(A.img, I.loc, size), nbd(thisimg, thisloc, size)) as Sfrom fMRI A, intloc Iwhere A.img = I.img orderby S
Return id’s, images, and locations for images that havesimilar neighborhoods of a given size at that locationto neighborhoods of the query image thisimg at locationthisloc of the same given size.
thisimg thisloc fMRI intloc simnbd
img loc img nbd prob
The intloc relation contains the locations of interesting voxels in an image.This is not straightforward, as we are computing similarity of neighborhoods of images, which are obtained through function calls, not through relations.
thisimg
A
Q8
id img
![Page 19: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/19.jpg)
19
select A.id, A.img, I.locfrom fMRI A, intloc Iwhere A.fid = I.fid and simnbd(nbd(A.img, I.loc, size), nbd(thisimg, thisloc, size))
Return id’s, images, and locations for images that havesimilar neighborhoods of a given size at that locationto neighborhoods of the query image thisimg at locationthisloc of the same given size.
thisimg thisloc fMRI intloc simnbd
fid loc prob img nbd prob
The intloc relation contains the locations of interesting voxels in an image.This is not straightforward, as we are computing similarity of neighborhoods of images, which are obtained through function calls, not through relations.
thisimg
A
Q8
User query:
fid id img
![Page 20: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/20.jpg)
20
select A.id,A.img, I.locfrom fMRI A, SUR S, intloc I where A.img = I.img and simloc(I.loc, S.loc)
Return id’s, images and pattern locations of database imagesthat have an interesting pattern at a location that is similar toone of the locations of language sites in the samepatient’s SUR data.
fMRI SUR intloc simlocid img id loc img loc loc loc prob
Q9
![Page 21: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/21.jpg)
21
select A.id,A.img, I.locfrom fMRI A, SUR S, intloc I where l.fid = A.fid and simloc(I.loc, S.loc)
Return id’s, images and pattern locations of database imagesthat have an interesting pattern at a location that is similar toone of the locations of language sites in the samepatient’s SUR data.
fMRI SUR intloc simlocid img id loc loc loc prob
Q9
User query:
fid loc prob
![Page 22: 1 Extended SQL for Multimedia Queries select from where orderby](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649d6c5503460f94a4c777/html5/thumbnails/22.jpg)
22
select A.id,A.img, I.loc, S.locfrom fMRI A, SUR S, intloc I where A.img = I.img and A.id = S.id and simloc(I.loc, S.loc) and simpeaks(signal(A.img,I.loc),S.signal)
Return id’s, images and pattern locations of database imagesthat have an interesting pattern at a location that is similar toone of the locations of language sites in the samepatient’s SUR data and whose fMRI signal at the fMRI locationhas a similar peak pattern to the SUR signal at its location.
fMRI SUR intloc simloc simpeaks
id img id loc signal img loc loc loc prob signal signal prob
Q10 We don’t understand this query yet