MPEG - Query

download MPEG - Query

of 33

  • date post

    14-May-2017
  • Category

    Documents

  • view

    220
  • download

    0

Embed Size (px)

Transcript of MPEG - Query

  • 1

    SAMT08

    Semantic-driven multimedia retrieval with the MPEG Query Format

    Ruben Tous and Jaime Delgado

    Distributed Multimedia Applications Group (DMAG)Universitat Politcnica de Catalunya (UPC)

    Dept. Computer ArchitectureBarcelona (Spain)

  • Presentation outline

    The MPEG Query Format (MPQF)MPQF evaluation modelHow to extend MPQF to allow SPARQL-like queriesMPQF and metadata interoperability

    2

  • MPEG Query Format (MPQF)

  • 4

    MPEG Query Format (MPQF): IntroductionISO/IEC 15938-12 Query Format (MPQF)

    Recently reached the International Standard level

    MPQF: Provides a standardized interface for multimedia content retrieval systems (e.g.: MPEG-7 databases)

    Allows combining Data Retrieval like conditions (XPath, XQuery) with Information Retrieval like conditions (Query-by-Example, etc.)

    Is an XML application: One XML schema describing messages:Requester -> responder = Input Query Format Responder -> requester = Output Query FormatManagement messages (service properties, etc.)

    Part 12 of ISO/IEC 15938 (MPEG-7), but METADATA-NEUTRAL

  • 5

    Service Provider

    Requester

    MPQF

    MPQF

    MPQF

    MPQF Service N

    Service 2

    Service 1

    Responder

    Input Query FormatOutput Query Format

    Query Management InputQuery Management Output

    Client

    Query Splitter

    Resultlist Aggregator

  • MPEG Query Format (MPQF)

    Schema overview of the uppermost elements of MPQF

    6

    MpegQuery

    Query

    Management

    FetchResult

    Output

    Input

    Output

    Input

  • MPEG Query Format (MPQF)

    Input Query Format (IQF)

    7

    QFDeclaration

    ServiceSelection

    Input

    OutputDescription

    QueryCondition

    Path

    TargetMediaType

    Join

    Condition

  • 8

    MPEG Query Format (MPQF)

    Condition types

    IR-like criteriaQueryByFreeText

    QueryByDescription

    QueryByMedia

    TemporalQuery

    SpatialQuery

    Embedded XQuery expressions(metadata-neutral)

    XML query algebra(metadata-neutral)

    DR-like criteria

    QueryByROI

    DR-like criteriaBoolean connectors

  • MPEG Query Format (MPQF)

    Example input query condition tree (graphical view)

    9

    AND

  • 10

    Input Query Format (Example)

  • MPEG Query Format (MPQF)

    Output Query Format (OQF)

    11

    GlobalComment

    ResultItemOutput

    SystemMessage

    Comment

    TextResult

    Thumbnail

    MediaResource

    Description

    AggregationResult

  • 12

    Output Query Format (Example)

  • MPQF evaluation model

  • 14

    image2

    image1

    image3

    Multimedia Database: Simplistic view

  • 15

    Metadata DB

    image2

    i3.1 i3.2

    image1

    Content DB

    Multimedia Database: Complex view (the one in the Query Format)

    image4

    image1

  • 16

    =

    XML

    The Query Format assumes XML metadata

  • MPQF in depth

    The EvaluationPath element

    17

    //VideoSegment

    Lausanne

  • 18

    Metadata DB

    image2

    i3.1 i3.2

    image1

    Content DB

    Multimedia Database: Complex view (the one in the Query Format)

    image1

    Image/regionlocators//IDs

  • image2

    i.1 i3.2

    image1

    Content DB Metadata DB

    Imageregionlocators

    Independentmetadata

    Image Database: Practical view (ONE BIG XML TREE)

  • MPQF in depth

    Two evaluation styles in MPQF: Boolean and fuzzy-logic

    20

    //FileSize < 1000

    1=TRUE

    1=TRUE1=TRUE

    0.4

    0.56

    0.7

    QueryByMedia

    inputsequence

    outputsequence

  • How to extend MPQF to allow SPARQL-like queries

  • How to extend MPQF to allow SPARQL-like queries

    Knowledge base (KB) = Tbox (terminological component) +Abox (assertion component)Some relevant initiatives are choosing the RDF language for modeling metadata metadataTypical Semantic Web based multimedia KB (e.g. PhotoRDF):

    Tbox = RDFS/OWL ontologies Abox = RDF statements about the media objects

    22

    ABox

    Knowledge DB

    ps

    opsp

    s op

    o

    opsp

    s o

    opsp

    s o

    TBox(ontologies)

    o

    s

  • How to extend MPQF to allow SPARQL-like queries

    SPARQL. W3C Recommendation on 15th January 2008An SPARQL query consists of a basic graph pattern, expressed as a list of triple patterns

    23

    PREFIX dc: PREFIX ns: SELECT ?title ?priceWHERE { ?x ns:price ?price .FILTER (?price < 30.5)?x dc:title ?title . }

    ps

    opsp

    s o

    o

    opsp

    s o

  • How to extend MPQF to allow SPARQL-like queries

    Proposal 1/3: Extend the MPQF data model

    24

    video2

    v3.1 v3.2

    video1

    Content DB Metadata DB

    ABox

    Knowledge DB

    ps

    opsp

    s op

    s o

    opsp

    s o

    opsp

    s o

    TBox(ontologies)

    o

  • How to extend MPQF to allow SPARQL-like queries

    Proposal 2/3: Include a new query type: QueryBySPARQL

    25

    ASK { ?resource dc:title "Barcelona . }]]>

  • How to extend MPQF to allow SPARQL-like queries

    Visual representation of the execution of the new QueryBySPARQLquery type

    26

    video2

    v3.1 v3.2

    video1

    Content DB Metadata DB Knowledge DB

    ps

    opsp

    s o

    ps o

    o

    p

    sp

    p

    o

    ps o

    matchingRDFtriples

    EvaluationItem(EI)

  • AND

    PREFIX dc: ASK { ?resource dc:title "Barcelona . }

    1000//FileSize