Pyramid-Based Texture Analysis / SynthesisPresented By :Keren Kahanov

Texture SynthesisGeneration of much texture as desired so that any object can be coveredInput texture Output size Output texture

The output texture is synthesized image that match the texture appearance of the given digitized image

General TechniquesTexture mapping repeating the textureCan lead to unacceptable artifacts in visible seams , visible repetitionDistortion in map to 3-D often no natural map from planar texture image to the geometry/ topology of the surface. (There are some partial solutions , but no universal solution for mapping an image onto arbitrarily shaped surface.)

General Techniques (cont.)Create (paint) textures by hand directly onto the 3-d surface modelVery labor intensive Requires considerable artistic skillsComputer-synthesized texturesas much texture can generated as neededSome of the synthesis techniques produce textures that tile seamlessly.

Synthetic texturesThe distortion problem has been solved in two different ways:Some techniques work by synthesizing texture directly on the object surface.The second solution is to use solid textures (no mapping no distortion)

Heeger & Bergen Pyramid-Based Texture Analysis / Synthesis Our lecture today A technique for synthesizing an image (or solid texture) that matches the appearance of a given texture sample.The output image (at any size) looks like the original image in its color and texture properties.The texture is synthesized so that it tiles seamlessly.

Heeger & Bergen (cont.)First order statistics (histograms)Focuses on the synthesis of stochastic textures deterministic textures: characterized by a set of primitives and a placement rule (e.g., a tile floor ,bricks). stochastic textures : do not have easily identifiable primitives (e.g., granite, bark, sand).Many real-world textures have some mixture of these two (e.g. woven fabric, plowed fields).

Heeger & Bergens AlgorithmThe texture-matching algorithm relies on two main components:a self-inverting steerable pyramida histogram matching process.

Pyramids - ReminderReminder from image processing course by Dr. Hagit Hel-Or

Pyramids Reminder (cont.)

(1) Steerable Pyramida linear multi-scale, multi-orientation image decomposition developed in order to overcome the limitations of orthogonal separable wavelet decompositions The steerable pyramid, unlike wavelet, is non-orthogonal and overcompleteOvercomplete = the number of pixels in the pyramid is much greater than the number of pixels in the input image

Steerable Pyramid (cont.)The basis functions of the steerable pyramid are directional derivative operators, that come in different sizes and orientations translation-invariant (i.e., the subbands are aliasing-free, or equivariant with respect to translation)Translating the image the transform translates energy in each band is preserved (not in wavelet)

Aliasing

In steerable pyramid only the low-pass band is subsampledThis is done to minimize the amount of aliasing within each subband. Avoiding aliasing is critical because the algorithm treats each subband independently.Properly sampled Aliasing

Steerable Pyramid (cont.)rotation-invariant = steerable (i.e., the subbands are steerable, or equivariant with respect to rotation) rotating the image the transform rotatesA set of filters form a steerable basis if (1) they are rotated copies of each other, and (2) a copy of the filter at any orientation may be computed as a linear combination of the basis

Steerable filters example 2nd derivative of Gaussian Linear combination Same Linear combinationfilters:OriginalImage:outputs:

Steerable Pyramid (cont.)A self-inverting pyramid:it provides a complete representation of an imageoriginal image can be reconstructed by applying the same filters used to generate the pyramid

A steerable pyramid can capture the variation of a texture in both intensity and orientation

Steerable Pyramid Build

Needs 7 filters:Original image

Steerable Pyramid Build (cont.)Original image4 oriented filtersThe steerable pyramid

Steerable Pyramid- Fourier domainThe steerable pyramid performs a polar-separable decomposition in the frequency domain, thus allowing independent representation of scale and orientation4 3 2 1 11223344wavelet

(2) Histogram MatchingA generalization of histogram equalization.modify an image (im1) to have the same histogram as another image (im2).Using two lookup tables:the cumulative distribution function (cdf) of one image. The inverse cumulative distribution function (inverse cdf) of the other image

Cdf / inverse CdfThe cdf maps from the interval [0,256] to the interval [0,1]. (cdf = accumulated histogram) The inverse cdf maps back from [0,1] to [0,256]. constructed by resampling (with linear interpolation) the cdf so that its samples are evenly spaced on the [0,1] interval.Note! the values in the noise and target images need not lie in the range[0,256]

Histogram Matching Algorithm

Histogram matched

Image 1Image 2Image 1 histogram matched to image 2 histogram

Enough with all the talks about Pyramids & Histograms !!!

Its time to learn the real thing Heeger & Bergens Algorithm for Texture Analysis / Synthesis

Heeger & Bergens Algorithm

The Algorithm - stepsInput1 = sample texture imageInput2 = sample of uniform random noise ( noise image)Its dimensions = desired size of output imageMatch noise images histogram to the input samples histogram. ( modified noise image)The texture sample is decomposed into an oriented bandpass pyramid

The Algorithm steps (cont.)Loop for several iterations:Make pyramid from the modified noise image Histogram matching process is employed between corresponding subbands of the two pyramids. The noise pyramid is collapsed The histogram matching process is again applied to the new noise image and the texture sample After a few iterations, the output image, which began as random noise, begins to resemble the input texture.

When are two textures similar?FROM: Statistical Texture Synthesis Presented by:Zeev Litichever & Tal Hassner , Weizmann Institute

Textures appear visually similar ifthey have similar distributions of filter outputs*.(Filters : orientation and spatial-frequency)

Laplassian \ steerable pyramidFROM: Statistical Texture Synthesis Presented by:Zeev Litichever & Tal Hassner , Weizmann Institute

Using histograms for synthesisNoise image of output size:Apply filtersAnd create histogramsOutput image with similar distributions to sample.Iteratively coarse noise image to have sample histogram via histogram matching.Output pyramidFROM: Statistical Texture Synthesis Presented by:Zeev Litichever & Tal Hassner , Weizmann Institute

ResultsInput1 = texture imageInput2 = noise imageOutput

More Results

In each pair : left= input (original) , right =output (synthesized)

Edge HandlingProper edge handling in the convolution operations is important.synthesis pyramid: use circular convolution. Given that the synthesis starts with a random noise image, circular convolution guarantees that the resulting synthetic texture will tile seamlessly. (synthesis pyramid = noise pyramid)

Edge Handling (cont.)analysis pyramid: circular convolution would typically result in spuriously large filter responses at the image borders. This would, in turn, introduce artifacts in the synthesized texture. (analysis pyramid = original image pyramid) A reasonable border handler for the analysis pyramid is to pad the image with a reflected copy of itself. Reflecting at the border usually avoids spurious responses.

ColorHeeger and Bergens Algorithm can be applied for both :gray level texture imagescolor texture images

Color Space RGBRGB components of a typical image are not independentcannot directly apply the algorithm to each channel individually.

Looking for B ? B is on a vacation , in another dimension(helped to simplify the graph)correlation

Color Space TransformFirst, transform the color space of the input texture to independent channels

SVD algorithm :The decorrelation transform is :

Color Space Transform (cont.)GDiagonal matrixNo correlation in transformed color space

New Color Spaceeach independent channel can be processed separately.After processing, the channels are transformed back into RGB values, using yielding the synthesized texture.

Solid TexturesThe algorithm can also be used to make isotropic 3-d solid textures.Same as before, except for the choice of pyramid: use a 2-d Laplacian pyramid for analysis and a 3-d Laplacian pyramid for synthesis.

Solid Textures - examples

Texture Mixtures

Texture mixtures synthesized by choosing the color palette from one image and the pattern from a second image.

Texture Mixtures (cont.)

tiling different textures as input to Heeger and Bergens algorithmOutput: combine the properties of multiple textures.

Algorithm no. iterationsAn Implementation of Heeger and Bergens Texture Analysis/Synthesis Algorithm by Thomas F. El-Maraghi

The human observer would find it difficult, or even impossible, to discern any change after the 4th iteration.

No. of orientationThe effect of changing the number of oriented subbands in the pyramid on the synthesized texture.

the synthesized texture looks more like the original as the number of orientations is increased.

original1 orientation2 orientations4 orientations

Homogenous / Inhomogoneous input

Homogoneous input texture success

Inhomogoneous input texture produces blotchy synthetic texture failure

Failures

More Failures

summeryIn their paper entitled Pyramid-Based Texture Analysis/Synthesis , Heeger and Bergen present an algorithm for generating synthetic copies of stochastic textures. The algorithm is based on a self-inverting oriented bandpass pyramid and a histogram matching techniqueThe process is tantalizing both for the results it produces and for its intrinsic simplicity.Given a texture sample, either grayscale or color, the algorithm can synthesize as much of it as desired.

Upsides

FastSimpleNo parameters (requires only the target texture as input) You just crop a textured region from a digitized image and run a program to produce as much of that texture as you want.Produces tileable texturesWorks well on stochastic textures

Downsides

Doesnt work well other texturesFor example : inhomogeneous texture, quasi-periodic textures and on random mosaic textures

Not guaranteed to converge

ReferencesBased on articles :Pyramid-Based Texture Analysis/Synthesisby David J. Heeger & James R. BergenAn Implementation of Heeger and Bergens Texture Analysis/Synthesis Algorithm by Thomas F. El-MaraghiMore Referances:THE STEERABLE PYRAMID: A FLEXIBLE ARCHITECTURE FOR MULTI-SCALE DERIVATIVE COMPUTATION by Eero P Simoncelli & William T Freemanimage processing course ,Haifa University, by Dr. Hagit Hel-OrAdvanced Topics in Computer Vision course, Spring 2003, Weizmann Institute http://people.csail.mit.edu/billf/steer.htmlhttp://www.cns.nyu.edu/~eero/steerpyr/http://noobeed.com/nb_ex_image_histmatch.htmwww.dfanning.com/documents/programs.html#HISTOMATCH