UNIT II . 1Multimedia Systems

Multimedia Building Blocks

TextSoundMIDIDigital AudioAudio File FormatsMIDI under Windows environmentAudio & Video Capture

UNIT II . 2Multimedia Systems

TextThere are three types of text that are used to produce pagesof documents.

Unformatted Text: enables pages to be created whichcomprise of strings of fixed-sized characters from alimited character set, also known as plaintext.

Formatted Text: enables pages and complete documents tobe created which comprise of strings of characters ofdifferent styles, size and shape with tables, graphics andimages inserted at appropriate points, also known asrichtext.

Hypertext: enables an integrated set of documents to becreated which have defined linkages between them.

UNIT II . 3Multimedia Systems

SoundThe most sensuous element of multimedia.How the power of sound is used, can make thedifference between an ordinary multimediapresentation & a professionally one.Misuse of sound can wreck the project.When something vibrates in the air by movingback & forth, it creates waves of pressure. Thesewaves spread like ripples from pebble tossed intoa still pool, and when they reach your eardrums,you experience the change of pressure, orvibrations, as sound.

UNIT II . 4Multimedia Systems

Sound (contd.)

Acoustics is the branch of physics that studiessound.Sound pressure levels (loudness or volume) aremeasured in decibels (dB), which is actually theratio between a chosen reference point on alogarithmic scale and the level that is actuallyexperienced.

UNIT II . 5Multimedia Systems

Use of sound

Use of sound in multimedia projects will notrequire highly specialized knowledge ofharmonics, intervals, sine waves, notation,octaves, or the physics of acoustics and vibration.Need to know the following:How to make sounds

How to record and edit sounds on your computer

How to incorporate sounds into multimedia work

UNIT II . 6Multimedia Systems

Multimedia System Sounds

In Windows, system sounds are WAV files, and they residein the Windows\Media subdirectory. Available system eventsounds include start.wav, chimes.wav, chord.wav, ding.wav,logoff.wav, notify.wav, recycle.wav, tada.wav, and theMicrosoft sound.wav which plays when Windows starts up.

After installing Microsoft Office, more choices are available:applause.wav, camera.wav, carbrake.wav, cashreg.wav,chimes.wav, clap.wav, driveby.wav, dumbroll.wav,explode.wav, glass.wav, gunshot.wav, laser.wav,projector.wav, ricochet.wav, type.wav, and whoosh.wav.

UNIT II . 7Multimedia Systems

Contd.

Newly created sound files can be added andinstalled in a manner that they play when systemevents occur: place the WAV sound files in~\Windows\Media directory and use theSound Control Panel to select them.Most sound used in a multimedia production iseither digitally recorded audio or MIDI (MusicalInstrument Digital Interface) music.

UNIT II . 8Multimedia Systems

Digital Audio

It is created when a sound wave is representedusing numbers – a process referred to as digitizing.Sound can be digitized from a microphone, asynthesizer, existing tape recordings, live radioand television broadcasts, and popular CDs, i.e.,any source, natural or prerecorded.Digitized sound is sampled sound. Every nthfraction of a second, a sample of sound is takenand stored as digital information in bits andbytes.

UNIT II . 9Multimedia Systems

Contd…

The quality of this digital recording depends uponhow often the samples are taken (sampling rate orfrequency, measured in kHz or thousands ofsamples per second) and how many numbers areused to represent the value of each sample(bitdepth, sample size, resolution, or dynamicrange). The more often a sample is taken and themore data is stored about that sample, the finerthe resolution and quality of the captured soundis, when it is played back.

UNIT II . 10Multimedia Systems

Contd…

Digital audio is said to be device independent,because the quality of audio is based on thequality of recording and not the device on whichend user will play the audio.The three sampling frequencies most often usedin multimedia are CD-quality 44.1 kHz, 22.05kHz, and 11.025 kHz. Sample sizes are either 8bits or 16 bits. The larger the sample size, thebetter the data describes the recorded sound.

UNIT II . 11Multimedia Systems

Quantization & ClippingThe process of rounding off the value of eachsample to the nearest integer is calledquantization.If the amplitude is greater than the intervalsavailable, clipping of the top and bottom of thewave occurs.Quantization can produce an unwantedbackground hissing noise, and clipping mayseverely distort the sound.

UNIT II . 12Multimedia Systems

Preparation of Digital Audio Files

Having analog source materials – music orsound effects that have been recorded on analogmedia, i.e., cassette tapes – the first step is todigitize the analog material by recording it ontocomputer-readable digital media. Generally, thisis done by playing sound from one device (suchas a tape recorder) into the computer, usingappropriate audio digitizing software.

UNIT II . 13Multimedia Systems

Contd…

One should focus on two crucial aspects ofpreparing digital audio files:Balancing the need for sound quality against file size.Higher quality means larger files, requiring longerdownload times on the Internet and more storagespace on a CD or DVD.

Setting proper recording levels to get a good, cleanrecording.

UNIT II . 14Multimedia Systems

File Size versus Quality

Sampling rate determines the frequency at whichsamples will be drawn for the recording.Sampling at higher rates more accuratelycaptures the high-frequency content of thesound.Audio resolution (such as 8- or 16-bit) determinesthe accuracy with which a sound can bedigitized. Using more bits for the sample sizeyields a recording that sounds more like itsoriginal.

UNIT II . 15Multimedia Systems

Contd…

Stereo recording are more lifelike and realisticbecause human beings have two ears.Mono recordings are fine, but tend to sound abit “flat” and uninteresting in comparison tostereo recordings.Logically, stereo sound files require twice asmuch storage space as mono files for the samelength of play time.

UNIT II . 16Multimedia Systems

Contd…Formulas for determining the size (in bytes) of a digitalrecording.For monophonic recording:

sampling rate duration of recording in sec. (bitresolution / 8) 1For stereo recording:

sampling rate duration of recording in sec. (bitresolution / 8) 2

(Note: sampling rate is measured in kHz, or thousand samples persec, so to convert from kHz to a whole number, multiply by 1000.Resolution is measured in bits per sample. Since there are 8 bits ina byte, therefore divide the bit resolution by 8.)

UNIT II . 17Multimedia Systems

Setting Proper Recording LevelsAny good piece of digital audio recording andediting s/w displays digital meter to let youknow how loud your sound is. Watch the metersclosely during recording, and you will never havea problem.Unlike analog meters, digital meters peak out.To avoid distortion, do not cross over the limit.If it happens, lower your volume and try again.Try to keep peak levels b/w -3 & -10. If you goover the peak, whether you can hear it or not,you introduce distortion into the recording.

UNIT II . 18Multimedia Systems

Editing Digital RecordingsAudacity is a free open source sound editing applicationfor Windows, Macintosh, and Linux OS. With this toolyou can create sound tracks and digital mixes.Basic sound editing operations:

Multiple TracksTrimmingSplicing & AssemblyVolume AdjustmentsFormat ConversionResampling or DownsamplingFade-ins and Fade-outsEqualizationTime StretchingDigital Signal Processing (DSP)Reversing Sounds

UNIT II . 19Multimedia Systems

Making MIDI Audio

MIDI (Musical Instrument Digital Interface) isthe quickest, easiest, and the most flexible toolfor composing your own original score – one ofthe most creative and rewarding aspects ofmaking a multimedia project; though creating anoriginal MIDI score is hard work – requiressomething about music, playing the piano, andlot of good ideas; beyond that it takes time andmusical skill to work with MIDI.

UNIT II . 20Multimedia Systems

Contd…

The process of creating MIDI is quite differentfrom digitizing existing audio. Digitized audio isthought as analogous to a bitmapped graphicimage (both use sampling of the original analogmedium to create a digital copy), while MIDI isthought analogous to structured or vectorgraphics (both involve instructions provided tos/w to be able to re-create the original one ondemand)

UNIT II . 21Multimedia Systems

Contd…

For digitized audio, the audio is played througha computer or device which can digitally recordthe sound, while to make MIDI scores,sequencer software and a sound synthesizer(built into the sound board on PCs) will beneeded. A MIDI keyboard is also useful forsimplifying the creation of musical scores.

UNIT II . 22Multimedia Systems

Contd…

Using sequencer s/w, you can record and editMIDI data. Rather than recording a note, thes/w creates data about each note as it is playedon a MIDI keyboard (or another MIDI device),e.g., which note it is, how much pressure wasused on the keyboard to play the note, how longit was sustained, and how long it takes for thenote to decay or fade away. This information,when played back through a MIDI device,allows the note to be reproduced exactly.

UNIT II . 23Multimedia Systems

Contd…

Because the quality of the play back dependsupon the end user’s MIDI device rather than therecording, MIDI is device dependent. Thesequencer software quantizes the score to adjustfor time inconsistencies, and it may also print aneatly penned copy of the score to paper.

UNIT II . 24Multimedia Systems

Contd…Advantage of structured data such as MIDI isthe ease with which the data can be edited. E.g.,you have a piece of music being played on ahonky-tonk piano, but your client wants thesound of saxophone. In this case if you had themusic in digitized audio, you would have to re-record and redigitize the music. But when it is inMIDI data, there is a value that designates theinstrument to be used for playing back themusic. So to change the instruments, just changethat value.

UNIT II . 25Multimedia Systems

General MIDIInstruments that can be synthesized areidentified by a General MIDI numbering systemthat ranges from 0 to 127 (see table in Book byTay Vaughan). Some MIDI devices offset thenumbers by one, using 1 to 128. Most of thesoftwares have a switch to accommodate thesedevices.

UNIT II . 26Multimedia Systems

Contd…

It is so easy to edit MIDI data, therefore, manyfine adjustments can be made to the music.Since MIDI is device dependent and the qualityof consumer MIDI playback h/w varies greatly,MIDI may be used as a production tool ratherthan a delivery medium.Till now, MIDI is the best way to create originalmusic for multimedia projects, so use MIDI toget the flexibility and required creative control.

UNIT II . 27Multimedia Systems

Components of a MIDI SystemSynthesizer: A sound generator. A goodsynthesizer often has a microprocessor,keyboard, control panels, memory, etc.Sequencer: A stand-alone unit or a s/w pgm for aPC. It is used to be a storage server for MIDIdata. Now a days it is a software music editor onthe computer. It has one or more MIDI INs andMIDI OUTs.Track: It is used in sequencer to organize therecordings. Tracks can be turned on or off onrecording or playing back.

UNIT II . 28Multimedia Systems

Contd…Channel: These are used to separate information in a MIDIsystem. There are 16 MIDI channels in one cable. Channelnumbers are coded into each MIDI message.

Timbre: The quality of sound, e.g., flute sound, cello sound,etc. Multitimbral – capable of playing many differentsounds at the same time (e.g., piano, brass, drums, etc.)

Pitch: Musical note that the instrument plays.

Voice: The portion of the synthesizer that produces sound.Synthesizers can have may (12, 20, 24, 36, etc.) voices. Eachvoice works independently and simultaneously to producesounds of different timbre and pitch.

Patch: The control settings that define a particular timbre.

UNIT II . 29Multimedia Systems

MIDI Connectors

These are 5-pin ports found on the back ofevery unit.MIDI IN: The connector via which the devicereceives all MIDI data.MIDI OUT: The connector through which thedevice transmits all the MIDI data it generatesitself.MIDI THROUGH: The connector by which thedevice echoes the data receives from MIDI IN.