LYU 0602 Automatic PhotoHunt Generation1 Automatic PhotoHunt Generation Shum Hei Lung To Wan Chi...

LYU 0602 Automatic PhotoHunt Generation1

Automatic Automatic PhotoHunt PhotoHunt GenerationGenerationShum Hei Lung To Wan Chi

Supervisor: Prof. Michael R. Lyu


• Background• Objectives• Previous Work• Newly Developed Module

– Image Analysis – Image Warping– Object Appending

• Enhanced Module– Elimination– Game Engine

• Evaluation• Conclusion

AgendaAgenda


BackgroundBackground

• PhotoHunt is …– A Spot-the-difference game

– Classic yet evergreen

– Popular in electronic game centers all over the world

However…It is limited by man power


ObjectivesObjectives

• Develop real-time Image Generation Engine– Employ image processing techniques

– Mimic human behavior

• Develop PhotoHunt game– Make use of generation engine

– Implement more unique features


Objectives Objectives Image Generation EngineImage Generation Engine

• To generate an image for PhotoHunt game– Effects that may be applied:

• Elimination • Image Warping• Object Appending• Color Change

Definition of well generated image: • The effects should be

“NOT OBVIOUS YET DISCOVERABLE”


Previous WorkPrevious Work

ApplicationsApplications

• Semi Automatic PhotoHunt

• Game Engine

Automatic PhotoHunt Generation Automatic PhotoHunt Generation

Image Generation EngineImage Generation Engine

• Segmentation Module• Modification Module -Elimination -Color change• Smoothing module

Image Processing FoundationImage Processing Foundation


Gaussian Pyramid

Previous WorkPrevious WorkSegmentation Module Elimination Module Smoothing Module

• To detect and extract segment from the input image• Three Phases:

– Pyramid Segmentation

– Constraint Checking

– Reference image building

Game Engine


Previous WorkPrevious WorkSegmentation Module Elimination Module Smoothing Module Game Engine

• Direct Copy Algorithm

• Horizontal Gradient Algorithm

• Nearest Boundary Algorithm

• Enhanced Nearest Boundary Algorithm


Previous WorkPrevious WorkSegmentation Module Elimination Module Smoothing Module Game Engine

• To reduce noise and distortion

• To make the image more realistic

• Gaussian Filter (Neighbor size=3, sigma=1)


Previous WorkPrevious WorkSegmentation Module Elimination Module Smooth Module Game Engine


Last semesterLast semesterSegmentation

Module

Modification Modules

Smooth Image

Game Engine


This SemesterThis SemesterSegmentation

Module

Enhanced Elimination

Smooth Image

EnhancedGame Engine

Image Analysis Modules

Image Warping Color Change

Object Appending


Image Analysis Module



• Purpose– To extract useful information from the image in

order to assist the generation process

Segmentation Module Image Analysis Module

Color Change

Elimination

Image Warping


Image Analysis Module > Function 1Image Analysis Module > Function 1Screening out undesirable segmentScreening out undesirable segment

• Definition of undesirable segment– Regions that are wrongly segmented

• Cause of undesirable segment– The engine uses an optimized threshold to segment all

images

• AssumptionSegment & Surrounding havesimilar color and brightness

Undesirable Segment

RejectSegment

Come fromSame Object



Functions:

• Screening out undesirable segment

• Deciding modification to be applied

• Providing suggestion on replacement color


Image Analysis ModuleScreening out undesirable segmentScreening out undesirable segment

• Procedures: 1. Compute Mean of Object –(1)

2. Compute Mode of Background –(2)

3. Compare (1) and (2)

Object Mean : [66 72 72] T Bg Mode : [220 231 228] T Diff : 475>60 Accept

Object Mean : [66 72 72] T Bg Mode : [220 231 228] T

Diff : 284>60Accept

Object Mean : [217 92 76 ] T Bg Mode : [207 81 65] T

Diff : 32<60Object Mean : [3 83 148 ] T Bg Mode : [10 99 163] T

Diff : 39<60

Reject

Reject


Image Analysis Module > Function 2> Function 2Deciding modification to be appliedDeciding modification to be applied

Segment

Color Change

Image Warping

Elimination

Single Colored

Regular in shape

Any shape The property of image for the specified effect:


Image Analysis Module > Function 2> Function 2Deciding modification to be appliedDeciding modification to be applied

Object Neighbor Difference<Threshold1 Object Variance < Threshold2

Color Change

yesNo

Object occupied Area>70%

yesNo

Image Warping Elimination


Deciding modification to be appliedDeciding modification to be applied


Image Analysis Module > Function 3> Function 3 Suggestion on replacement coloruggestion on replacement color

Parameter Meaning

Background

Neighbor Difference

The complexity of background

Background Mode The major color of background

Background Mean The texture property of the background


Image WarpingImage WarpingModuleModule


Image WarpingImage Warping

• Produce Distortion by applying geometric transformation.


Image WarpingImage WarpingForward mapping algorithmForward mapping algorithm

Transformation1

Quadratic of y

Transformation2

linear of x

• Transformation Equation (General)

54322

12

0' bybxbxybybxby

xx '


• Transformation 1

• Transformation 2

Transformation EquationTransformation Equation

minX

Δy=0Mid pt

Δy=max

maxX

Δy=0

Δymax

1. Quadratic equation of root x=minX or maxX

2. Substituting (midptX, Δymax) into the equation to acquire the weight to control curvature

1. Flip the upper part, y>mid pt y

2. Enlarge the curve with a ratio proportional to distance between mid pt of y

maxY

minY

midptY


Object AppendingObject AppendingModuleModule


Object Appending Object Appending

• To append an object from our database to the original image

• Unable to carry out object recognition– Only generic objects are inserted to engine


Examples:Examples:


Enhanced Enhanced Elimination ModuleElimination Module


Elimination ModuleElimination Module

• Hybrid Elimination

Makes use of statistic data that came from the image analysis module

Information Needed:– Background Mode – Background Neighbor Difference– Background Mean


Hybrid Elimination AlgorithmHybrid Elimination Algorithm

• Check Background Neighbor Difference- To check whether the background is single colored

Case 1: Use the Background Mode to replace

Case 2: Apply texture from surrounding – Select the suitable surrounding region – Apply Direct copy Algorithm


Game EngineGame Engine


System OverviewSystem Overview

PHP S cript

U s e rs

Au tom at i cPh oto

Ge n e rati onEn gin e

P h o to S c o re

Gam eplayi n g

a ppl i cationwri tte n in

Flash

G a m e E n g i n e fo r W eb A p p lica tio n

im a g e s

o rig i n a l p h o to

w i th g en e ra ted p h o to

g e n e r at io n r e q u e s t

r e su l t in g p h o t o

p la y ing v iew

gen e

rat e

d p h

oto

s av e

d ph

o to s

p la yin g sc ore

p la y in g s co re

p la y in g sco re

p la y in g req u e st

u p lo a d ed im a g e s


System Interface System Interface • In later demo session


EvaluationEvaluation


Acceptance RateAcceptance Rate

Assume Acceptance rate= 75% = 988/1318 accepted imagesThus, x-axis is about 200 in upper limit and 4500 in lower limit.

Accepted Image varies Threhold

950

960

970

980

990

1000

1010

1020

1030

1040

1050

1060

100 120 140 160 180 220 240 260

Acc

ep

ted

Ima

ge

Lower Limit=5000

Lower limit=4500


Processing TimeProcessing Time

• Run one set of 1318 images for 17 times

• Average Processing time per set: – 00:15:50

• Average Processing time per image– 0.721 second


Image QualityImage Quality• Survey Result • Total Visitors : 91

• Total Hits: 1109• Total Images : 264• Received Survey : 121• Total Segments : 605

Satisfy 76%

Not Satisfy24%

Characteristics image shared for achieving good effects:

• Many objects within the image• Sharp Edge of object • Less noise• Maybe a cartoon


DemoDemo


ConclusionConclusion

• Developed the image generation engine

• Developed the game engine

• Carried out testing and analysis on the system

• Published the product to the public

• We are still watching the statistic and looking at feedback to improve our system


Q & AQ & A


The EndThe End

Thanks for your kind attention.

LYU 0602 Automatic PhotoHunt Generation1 Automatic PhotoHunt Generation Shum Hei Lung To Wan Chi...

Documents

Transcript of LYU 0602 Automatic PhotoHunt Generation1 Automatic PhotoHunt Generation Shum Hei Lung To Wan Chi...