LYU 0102 : XML for Interoperable LYU 0102 : XML for Interoperable Digital Video Library Digital Video Library

• Recent years, rapid increase in the usage oRecent years, rapid increase in the usage of multimedia information,f multimedia information,

• New approach: DIGITAL VIDEO LIBRARYNew approach: DIGITAL VIDEO LIBRARY• Automated video and audio indexingAutomated video and audio indexing• Navigation, visualizationNavigation, visualization• Search and retrievalSearch and retrieval•Video segmentation and summarization

Video InformationVideo Information

• Integration of speech , language, Integration of speech , language, and image processingand image processing

• Text processingText processing

• Audio processingAudio processing

• Image processingImage processing

• Video processingVideo processing

Digital Video LibraryDigital Video Library System Overview System Overview

Techniques to segment dataTechniques to segment data

Techniques we may applyTechniques we may apply

•VOCDVOCD

•Scene changesScene changes

•Text processingText processing

•Face detectionFace detection

•Storage as XML Storage as XML

Techniques to be Techniques to be discusseddiscussed

•VOCRVOCR

•Scene changesScene changes

•Storage and editing with XMLStorage and editing with XML

Video OCR for Digital NewsVideo OCR for Digital News

Detection of Text RegionDetection of Text Region

• Video news program comprises huge Video news program comprises huge numbers of framesnumbers of frames

• Roughly detect text regionRoughly detect text region

• Increase processing speedIncrease processing speed

• Reduce processing costReduce processing cost

Detection of Text RegionDetection of Text Region

• Typical text region can be Typical text region can be characterized as a horizontal characterized as a horizontal rectangular structurerectangular structure

• With clustered sharp edgesWith clustered sharp edges

• Regions of high contrast against the Regions of high contrast against the backgroundbackground

Image EnhancementImage Enhancement

• Sub-pixel Interpolation:Sub-pixel Interpolation:– To magnify the text area To magnify the text area – To increase the resolution of captionTo increase the resolution of caption

• Multi-frame Integration:Multi-frame Integration:– Video motion of non-caption areas, Video motion of non-caption areas,

caption relatively stablecaption relatively stable– To reduce the variability on backgroundTo reduce the variability on background

Character SegmentationCharacter Segmentation

• Vertical project profileVertical project profile

• Character segmentationCharacter segmentation

Character RecognitionCharacter Recognition

• Binarize the character image with Binarize the character image with thresholdthreshold

• Filter the binary image with Filter the binary image with morphological filtermorphological filter

• Filter the character image with Filter the character image with connected component filterconnected component filter

Post-ProcessingPost-Processing

• Further improve Further improve the recognition the recognition raterate1.1. Using the words of Using the words of

dictionary to refine dictionary to refine the characterthe character

2.2. Integrate the Integrate the recognition result recognition result of multiple framesof multiple frames

Scene changeScene change

• detection technique detection technique • effective method for segmenting a video effective method for segmenting a video

sequence into significant componentssequence into significant components

Existing MethodExisting Method

• Image difference method

•Histogram Difference Method

•Histogram Difference Method using DC Coefficient Image

• Our MethodOur Methodhistogram difference method with histogram difference method with

aa dynamic thresholddynamic threshold

Scene changeScene change

• grasp scene from the video for every grasp scene from the video for every 0.05 second0.05 second

• grasped scenes are 24-bit image, 8 grasped scenes are 24-bit image, 8 bits for each color (red R, green G, blue bits for each color (red R, green G, blue B)B)

• check each pixel with the most 2 check each pixel with the most 2 significant bitssignificant bits

• classify them classify them into 64 different classesinto 64 different classes• build a color histogrambuild a color histogram

Scene changeScene change• Compared the histogram with the Compared the histogram with the

pervious scenepervious scene • For each column of the histogram, For each column of the histogram,

calculate the difference calculate the difference • Sum all the differenceSum all the difference• If (total difference) > threshold If (total difference) > threshold

=> scene change=> scene change• Use the first frame as key frame Use the first frame as key frame

XMLXML

• Extensible Markup LanguageExtensible Markup Language

• Create its own mark-up language for Create its own mark-up language for describing the contentsdescribing the contents

• Look like a big databaseLook like a big database

AdvantagesAdvantages of using XML of using XML

• Platform and system independentPlatform and system independent

• Create your own tag Create your own tag

• Adopt UnicodeAdopt Unicode

• Universal formatUniversal format

• Easy to searchEasy to search

Design schemaDesign schema • Starts with

choosing a vocabulary

• Words and phrases that are able to describe extracted video information content and therefore can be used as tag name

• Show relationship between vocabulary entries

XML ParserXML Parser

• A parser is a A parser is a interface interface between an XML between an XML document and document and the application the application programprogram

• Document Object Document Object Model (DOM)Model (DOM)

How to present XMLHow to present XML

• Tree model Tree model becomes very becomes very similar to an XML similar to an XML schemaschema

• Represented as Represented as nodes that show nodes that show element/attribute element/attribute names or the text names or the text content and their content and their relative places relative places within the XMLwithin the XML

OUR TOOLOUR TOOL

OUR TOOLOUR TOOL

OUR TOOLOUR TOOL

OUR TOOLOUR TOOL

COMINGCOMING

• EXTRACT SECONDARY INFORMATIONEXTRACT SECONDARY INFORMATION

THE ENDTHE END