1 Constraints for Multimedia Presentation Generation Joost Geurts, Multimedia and Human-Computer...
-
Upload
lynette-grant -
Category
Documents
-
view
220 -
download
4
Transcript of 1 Constraints for Multimedia Presentation Generation Joost Geurts, Multimedia and Human-Computer...
1
Constraints for Multimedia Presentation Generation
Joost Geurts, Multimedia and Human-Computer Interaction
CWI Amsterdam email: [email protected]
2
Talk overview
•Generating multimedia automatically
•Cuypers multimedia generation engine
•Multimedia and constraints–Quantitative constraints
–Qualitative constraints
•Cuypers demo
•Conclusion, future directions
3
Multimedia Presentation
•Multimedia Presentation– Image, Text, Video, Audio
–Based on Temporal and
Spatial Synchronization
•Multimedia Document–SMIL, SVG, HTML
–WYSIWYG
–Static Content
•Problem: Dynamic Content
4
Generating adaptive multimedia
•Content–Large multimedia database
•System profile–PC, PDA, WAP
•Network profile–Modem, Gigabit
•User profile–Language, Interests, Abilities, Preferences
Too costly to author manually
6
Cuypers multimedia generation engine
•Cuypers is based on–media independent presentation abstractions– transformation rules with built-in backtracking andconstraint solving
8
Communicative Devices
…rhetoric relations are than transformed into presentation independent communicative devices…
9
Automatic multimedia generation
•Designer does not specify complete presentation……but only specifies requirements
•System automatically finds a solution which meets requirements
•How should the requirements be specified?–Declarative constraints
10
Constraint satisfaction
•Constraints occur often in our daily lives–Agenda, Travelling, Shopping
•Constraint paradigm for Problem Solving–DeclarativeUsed for problems with:–Many variables–Large domains–Based on domain reduction paradigm
12
Traditional use of constraints
Quantitative constraints–Integer domain
–Reduction by arithmetic relations•Greater than (>)•Less than (<)•Equals (=)
–Example(x < y ; x [0..10], y [5..10] )
(x + y = z 3 , x = u + 1 ; x , y , z , u )
13
Solving a Constraint Satisfaction Problem
•Problem SEND
+ MORE = MONEY
•Modeling 1000 x S + 100 x E + 10 x N + D
+ 1000 x M + 100 x O + 10 x R + E= 10000 x M + 1000 x O + 100 x N + 10 x E + Y
•Domain reduction / Search•Solution
S=9, E=5, N=6, D=7, M=1, O=0, R=8, Y=2
14
Quantitative Constraints in Multimedia
…Communicative devices generate constraint-graph which the system tries to satisfy…
15
Drawbacks of quantitative constraints
•Too many (trivial) solutions that differ by:–1 pixel position, or–1 milliseconds in timing
•Not sufficiently expressive•cannot specify “no overlap” constraint
•Too low level•A.X2 B.X1
17
Solution: qualitative constraints
•For non-typical domains–Boolean, –Three valued logics, –Allen’s relation
•Advantages for Multimedia generation:–More intuitive–More expressive–Smaller domains
18
Domain Reduction Rules
•InverseA before B B after AA equal B B equal A
•TransitiveA before B , B before C A before CA overlaps B, B during C A overlap C or
A during C or A starts C
•EqualsA overlap C, A [o,d,s] C A overlap C
19
Qualitative Constraints
…Qualitative solutions translate automatically to lower level quantitative constraints…
20
New problem: What if constraints are insoluble?
•Combine Prolog unification and backtracking with constraint solving
•Use Prolog rules to generate constraints•Backtrack when constraints are insoluble
Solution: Constraint Logic Programming
21
Cuypers generation engine
•Multiple layers:
–Communicative devices
generate constraints
–Qualitative constraints
translate to quantitative
constraints
–Solution of both constraints
provides sufficient
information for final
presentation
22
Cuypers demo: scenario
•Client
•Server•Server
•Server
•Server•Client
User is interested in Rembrandt and wants to know about about the “chiaroscuro” technique
Query database
Generate constraints according to:–System profile–User profile–Network profile
Solve constraints / revise constraints
Generate SMIL presentationPlay presentation
23
Conclusions
•Quantitative constraintsare insufficient for automatic multimediapresentation generation. Also need
•Qualitative constraintsto allow intuitive and effectivehigh level specification, and
•Backtrackingfor revising specific constraintswhich otherwise cause the entire set to fail
24
Discussion
•Labeling– Choice of candidate variable– Choice of candidate value
•Transitive Reasoning Rule– Infer implicit relations– Redundant
•Allen’s Relations– Not very well suited for generating MM – Non interactive
25
Future directions
•Best-first instead of depth-first–Choose “best” among possible solutions–Needs evaluation criteria
•Improve knowledge management–Make design knowledge declarative and explicit
–Preserve metadata in final presentation–Use standardized and reusable profiles