Combining Media

8

Key Points

There are two models for combining elements of different media types: page-based and synchronization-based

Hypermedia is a page-based model that generalizes hypertext to include other media types. The World Wide Web is an example of a hypermedia system.

Key Points

Helper applications are used by Web browsers to display media types they cannot handle themselves. Plug-ins are used to extend the range of types that can be displayed within the browser environment.

The object element is provided for embedding arbitrary media in HTML 4 documents, but img and embed are more commonly used.

Introduction

There are two models currently in use for combining elements of different media types Paged-based Synchronization-based

Paged-based

Paged-based In a two-dimensional arrangement that resembles

the way text and images are laid out in books and magazines.

Time-based elements, such as video clips and sound, are embedded in the page as if they were images, occupying a fixed area; controls may be provided to start and stop playback.

Linking mechanism Linked paged-based multimedia productions are known as

hypermedia. World Wide Web

Synchronization-based

Makes time the central organizing principle

Presented as a sequence, like a slide show

Transitions, such as dissolves and wipes, may be used to go from one element in the sequence to the next.

Parallelism Several video clips may be shown at the same time,

perhaps overlaid against a s6tatic image, or a sound track may play during an animation.

Interactive

Scripting: author to write simple programs

Page-based multimedia Temporal organization can be added

Synchronization presentation linking

Reasons for Using Actions

Reason for associating actions with events To provide interactivity

Users can control the system’s behavior. Users can direct the flow of information that they receive.

Events that occur at specific times Allow time-based behavior and synchronization to be

introduced into systems Actions in timeline

Provide non-linearity

Multimedia Scripting

Scripting language Perform computations on the attributes of objects Create new objects Affecting their appearance and behavior Triggered by event

WWW Client-side Scripting

Server-side Enable an HTTP server to communicate with other resources,

such as databases, and incorporate data obtained from them into its response.

Web pages dynamically form time-varying data

Client-side Appearance and behavior of Web pages Allowing code has been downloaded from Internet to run on

your system. Scripts cannot

access any local resources such as files make any network connections Their interaction with server is limited to requesting new resources

and posting information form HTML forms. Secure but limited

Provide feedback to user

HTML and Hypermedia

Link: href attribute

Helper applications

Plug-in

OBJECT: HTML 4.0, preferred way

Netscape: EMBED

OBJECT

OBJECT has content. The content is displayed only user agent is unable to

display the object. Support arbitrary media PARAM, name and vale

Controller = false, autoplay=true, loop=true

Synchronization-based presentations

Pure hypermedia: no temporal structure

Timelines for authoring

Slide show packages Transitions: wipes, dissolves, ripples, page turns

Timeline-based multimedia More complex

Macromedia Director Cast, Cast members, Score Sprites can be animated using key frames. Sprite properties can be animated.

Limitations

Web browsers are not designed to fully support multimedia features

Limited coordination of media elements

The Problem

Lots of Bits Images, audio and video are beyond Internet design specs Results in space/ time constraints at:

the server the network( s) the client

Not All Bits are Equally Important Time between samples often more important than bits in sample, for example lip synchronization (but not always...)

Content may be Distributed Across Network Need to synchronize presentation

Objectives Add synchronization to the Web Allow interoperability Use declarative format, preferably text — thus XML

Solution

Use of a single timeline for all media

Creation of time-based multimedia delivery over the web

Synchronization

Coordination

What Is SMIL?

SMIL stands for Synchronized Multimedia Integration Language

SMIL is pronounced "smile"

SMIL is a language for describing audiovisual presentations

SMIL is easy to learn and understand

SMIL is an HTML-like language

SMIL is written in XML

SMIL presentations can be written using a text-editor

SMIL is a W3C standard SMIL 2.0 August 2001 http://www.w3.org/TR/smil20/

What Is SMIL?

A Simplified SMIL Example

<smil>

<body>

<seq repeatCount="indefinite">

<img src="image1.jpg" dur="3s" />

<img src="image2.jpg" dur="3s" />

</seq>

</body>

</smil>

What Can SMIL Do?

SMIL can be used to create Internet or Intranet presentations

SMIL can be used to create slide show presentations

SMIL has been described as the Internet answer to PowerPoint

SMIL presentations can display multiple file types (text, video, audio...)

SMIL presentations can display multiple files at the same time

SMIL presentations can display files from multiple web servers

SMIL presentations can contain links to other SMIL presentations

SMIL presentations can contain control buttons (stop, start, next, ...)

SMIL has functions for defining sequences and duration of elements

SMIL has functions for defining position and visibility of elements

SMIL Files

A SMIL file contains all the information necessary to describe a multimedia presentation

SMIL files are stored with the extension *.smil

A SMIL file contains the following: The layout of the presentation The timeline of the presentation The source of all multimedia elements

How to Play a SMIL File?

You will need a SMIL player. Different SMIL players can be found RealOne Platform by RealNetworks has full support

for SMIL 2.0 SMIL Player by InterObject supports the SMIL 2.0

Basic Profile

With Internet Explorer 5.5 or later, SMIL elements can be inserted into HTML files

This way any SMIL presentation can run over the Internet as standard HTML files

SMIL in HTML

Adding References Adding a "time" namespace definition to the <html> tag Adding an <?import> element to import the "time" namespace Adding a <style> element to define the class "time"

<html xmlns:time="urn:schemas-microsoft-com:time"> <head> <?import namespace="time" implementation="#default#time2"> <style>.time {behavior: url(#default#time2)}</style> </head><body> <time:seq repeatCount="indefinite"> <img class="time" src="image1.jpg" dur="3s" /> <img class="time" src="image2.jpg" dur="3s" /> </time:seq> </body> </html>

SMIL Timing

time formats: hh:mm:ss.f number [h|min|s|ms] wallclock (YYY-MM-DDThh:mm:ss+zone)

Duration The duration (dur="5s") attribute

When To Start? The begin (begin="2s") attribute

SMIL Sequence

<seq> - the most common SMIL element - defines a sequence

The <seq> element can have a number of attributes: begin, dur, repeatCount

<html xmlns:t="urn:schemas-microsoft-com:time"> <head> <?import namespace="t" implementation="#default#time2"> <style>.t {behavior: url(#default#time2)}</style> </head> <body><t:seq repeatCount="indefinite"> <h2 class="t" dur="1s"> I will display for one second</h2> <h2 class="t" dur="2s"> I will display for two seconds</h2> <h2 class="t" dur="3s"> I will display for three seconds</h2> </t:seq></body> </html>

SMIL in Parallel

Objects inside the <par> element will be played at the same time Attributes: begin, dur, endsync= "first"|"last"|id(clip), repeatCount

<html xmlns:t="urn:schemas-microsoft-com:time"> <head> <?import namespace="t" implementation="#default#time2"> <style>.t {behavior: url(#default#time2)}</style> </head> <body><par> <t:audio src="liar.wav" repeatCount="indefinite" type="wav" /> <t:seq repeatCount="indefinite"> <h2 class="t" dur="1s"> I will display for one second</h2> <h2 class="t" dur="2s"> I will display for two seconds</h2> </t:seq></par></body></html>

SMIL Transitions

IE 6 supports transitions based on the SMIL 2.0. Transitions are implemented with the element <transitionfilter> Attributes: type, begin, mode, from, to

Transition Filters fade, barnDoorWipe, barWipe, clockWipe, ellipseWipe,

fanWipe, irisWipe, pushWipe, slideWipe, snakeWipe, spiralWipe, starWipe

<t:transitionfilter targetelement="keyb" type="clockWipe" begin="keyb.begin" dur="2s" />

<img id="keyb" class="t" src="pic_keyb.jpg" dur="4s" width="128" height="107" />

SMIL Media Elements

<animation> <audio> <brush> <img> <param> <ref> <text> <textstream> <video>

Attributes: erase src type

<html xmlns:t="urn:schemas-microsoft-com:time"> <head> <?import namespace="t" implementation="#default#time2"> </head> <body><t:video src="http://www.ananova.com/about/vap_windows_check.wmv" repeatCount="indefinite" type="wmv" /></body> </html>

Reading List

Check: http://www.ccse.kfupm.edu.sa/sukairi/swe423/course_resources/8-Combining Media/