18/05/15
1
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
CS380:IntroductiontoComputerGraphicsReconstruction&Resampling
Chapter17&18
MinH.KimKAISTSchoolofComputing
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
SUMMARYSampling(continuousàdiscrete)
2
18/05/15
2
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Aliasing• Scenemadeupofblackandwhitetriangles:jaggiesatboundaries– Jaggieswillcrawlduringmotion
• Iftrianglesaresmallenoughthenwegetrandomvaluesorweirdpatterns– Willflickerduringmotion
• Theheartoftheproblem:toomuchinformationinonepixel
3
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Anti-aliasing• Wecanalsomodelthisasanoptimizationproblem.• Theseapproachesleadto:
• whereissomefunctionthattellsushowstronglythecontinuousimagevalueatshouldinfluencethepixelvalue
4
I[i][j]← I(x , y)Ω∫∫ Fi , j(x , y)dxdy
Fi , j(x , y)
[x , y]t i,j
I[i][j]
I(xw , yw )
I(x , y)←
ΩFi , jxw = i , yw = j
Fi , j(x , y)
I(x , y)
Verticalsectionview
18/05/15
3
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
• Eventhatintegralisnotreallyreasonabletocompute• Instead,itisapproximatedbysomesumoftheform:wherekindexessomesetoflocationscalledthesamplelocations,so-calledover-sampling.
• Therendererfirstproducesa“highresolution”colorandz-buffer“image”,– wherewewillusethetermsampletorefertoeachofthesehighresolutionpixels.
Over-sampling
5
I[i][j]← 1
nI(xk , yk )
k=1
n
∑
(xk , yk )
apixel
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Overoperation• Tocompose,wecomputethecompositeimagecolors,,using
• Thatis,theamountofobservedbackgroundcoloratapixelisproportionaltothetransparencyoftheforegroundlayeratthatpixel.
• Likewise,alpha(coverage)forthecompositeimagecanbecomputedas:
6
If[i][j] over Ib[i][j]Ic[i][j]
Ic[i][j]← If [i][j]+Ib[i][j](1-α f [i][j])
αc[i][j]← α f [i][j]+α b[i][j](1-α f [i][j])
18/05/15
4
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Alphablendingerror• Thedifferencebetweentwo
• Thereforetheerroristhedifferencebetweentheintegralofaproductandaproductofintegrals.
• Itistheamountof“correlation(covariance)”betweenthedistributionofforegroundcoverageinsomepixelandthedistributionofthebackgrounddatawithinthatpixel.
7
error= Ib(x , y)Cb(x , y)C f (x , y)dxdyΩi , j∫∫ −
Ib(x , y)Cb(x , y)dxdy× C f (x , y)dxdyΩi , j∫∫Ωi , j
∫∫
cov(X ,Y )=E[XY ]−E[X ]E[Y ]
Realalphablending
Idealcontinuouscomposition
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
RECONSTRUCTION(DISCRETEààCONTINUOUS)
Chapter17
8
18/05/15
5
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Reconstruction• GivenadiscreteimageI[i][j],howdowecreateacontinuousimageI(x,y)?
• Itisthekeyproblemforresizingimagesandtexturemapping.– Howtogettexturecolorsthatfallinbetweentexels.
• Thisprocessiscalledreconstruction.
9
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Constantreconstruction• Arealvaluedimagecoordinateisassumedtohavethecoloroftheclosestdiscretepixel.Thismethodcanbedescribedbythefollowingpseudo-code:
• The(int)typecastroundsanumberp(pixelposition)tothenearestintegernotlargerthanp.
10
colorconstantReconstruction(floatx,floaty,colorimage[][]){inti=(int)(x+.5);intj=(int)(y+.5);returnimage[i][j];}
18/05/15
6
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Constantreconstruction• Theresultingcontinuousimageismadeupoflittlesquaresofconstantcolor.
• Eachpixelhasaninfluenceregionof1-by-1
11
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Bilinearproperties• Atintegercoordinates,wehaveI(x,y)=I[i][j];thereconstructedcontinuousimageIagreeswiththediscreteimageI.
• Inbetweenintegercoordinates,thecolorvaluesareblendedcontinuously.
• Eachpixelinthediscreteimageinfluences,toavaryingdegree,eachpointwithina2-by-2squareregionofthecontinuousimage.
• Thehorizontal/verticalorderingisirrelevant.• Coloroverasquareisbilinearfunctionof(x,y).
12
18/05/15
7
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Bilinearfunction• 1by1squarewithcoordinatesforsomefixed.
• wherearethefracxandfracyabove.
13
j < y < j +1i < x < i +1
i and j
I(i+ x f , j+ y f )←(1− y f ) (1− x f )I[i][j]+(x f )I[i+1][j]{ }+( y f ) (1− x f )I[i][j+1]+(x f )I[i+1][j+1]{ } ,
x f and yf
x f
y f
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Bilinearreconstruction• Cancreateasmootherlookingreconstructionusing
bilinearinterpolation.• Bilinearinterpolationisobtainedbyapplyinglinearinterpolationinboththehorizontalandverticaldirections.
14
colorbilinearReconstruction(floatx,floaty,colorimage[][]){intintx=(int)x;intinty=(int)y;floatfracx=x-intx;floatfracy=y-inty;colorcolorx1=(1-fracx)*image[intx][inty]+(fracx)*image[intx+1][inty];colorcolorx2=(1-fracx)*image[intx][inty+1]+(fracx)*image[intx+1][inty+1];colorcolorxy=(1-fracy)*colorx1+(fracy)*colorx2;return(colorxy);}
18/05/15
8
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Bilinearfunction• Rearrangingtheterms,
• Thisfunctionhastermsthatareconstant,linear,andbilineartermsinthevariables
15
I(i+ x f , j+ y f )← I[i][j]+ −I[i][j]+I[i+1][j]( )x f + −I[i][j]+I[i][j+1]( ) y f + I[i][j]-I[i][j+1]-I[i+1][j]+I[i+1][j+1]( )x f y f
(x f , yf )
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Bilinearbasisfunction• Rearrangethebilinearfunctiontoobtain:
• Forafixedposition,thecolorofthecontinuousreconstructionislinearinthediscretepixelvaluesofI:
16
I(i+ x f , j+ y f )← 1− x f − y f + x f y f( )I[i][j]+ x f − x f y f( )I[i+1][j]+ y f − x f y f( )I[i][j+1]+ x f y f( )I[i+1][j+1]
(x f , yf )
I(x , y)← Bi , j(x , y)I[i][j]
i , j∑
18/05/15
9
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Bilinearbasisfunction• TheseBarecalledbasisfunctions(tentfunctions)
• Theydescribehowmuchpixeli,jinfluencesthecontinuousimageat.
• In1D,wecandefineaunivariatehatfunction.
17
[x, y]t
Hi (x)
Hi(x)=x − i+1fori−1< x < i−x+ i+1fori < x < i+1
0else
⎧
⎨⎪
⎩⎪
∧
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
• In2D(bilinearfunction),letbeabivariatefunction:
• Thisiscalledatentfunction
Bilinearbasisfunction
18
Ti, j (x, y)
Ti , j(x , y)=Hi(x)Hj( y).
18/05/15
10
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Bilinearbasisfunction• Inconstantreconstruction,isaboxfunctionthatiszeroeverywhereexceptfortheunitsquaresurroundingthecoordinates(i,j),whereithasconstantvalue1.
19
I[i][j]
I(xw , yw )
I(x , y)←ΩFi , j
xw = i , yw = j
Bi, j (x, y)
I[i][j]← I(x , y)Ω∫∫ Fi , j(x , y)dxdy
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
RESAMPLING(RECONSTRUCTION+SAMPLING,DISCRETEààCONTINUOUSààDISCRETE)
Chapter18
20
18/05/15
11
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Resampling• Let’srevisittexturemapping• Westartwithadiscreteimageandendwithadiscreteimage.
• Themappingtechnicallyinvolvesbothareconstructionandsamplingstage.
• Inthiscontext,wewillexplainthetechniqueofmipmappingusedforanti-aliasedtexturemapping.
21
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Resamplingequation• Supposewestartwithatextureimage(discrete)T[k][l]andapplysome2DwarptothisimagetoobtainanoutputimageI[i][j].
22
T[k][l]← T(x , y)Ω∫∫ Fk ,l(x , y)dxdy
18/05/15
12
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Resamplingequation• Reconstructacontinuoustextureusingasetofbasisfunctions.
• Applythegeometricwrap(attheviewpoint)tothecontinuousimage.
23
T(xt , yt )Bk ,l(xt , yt )
texturecoordinates(asVV): v
wn
and 1wn
T(xt , yt )← Bk ,l(xt , yt )T[k][l]
k ,l∑
T '(xw , yw )←T(M(xw , yw ))(xt , yt )=M(xw , yw )
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Resamplingequation• Integrateagainstasetoffilters(e.g.,aboxfilter)toobtainthediscreteoutputimage.
24
Fk ,l(xw , yw )
I[i][j]← T '(xw , yw )Ω∫∫ Fi , j(xw , yw )dxwdyw
18/05/15
13
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Resamplingequation• Letthegeometrictransformbedescribedbyawarpmappingfunction,whichmapsfromcontinuouswindowtotexturecoordinates.
• Weobtain:
25
(xt , yt )=M(xw , yw )
I[i][j]← Fi, j (xw , yw )
Ω∫∫ Bk ,l M (xw , yw )( )k ,l∑ T[k][l]⎛
⎝⎜⎜
⎞
⎠⎟⎟dxwdyw
= T[k][l] Fi, j (xw , yw )
Ω∫∫ Bk ,l M (xw , yw )( )( )dxwdyw( )k ,l∑
I(x , y)← Bi , j(x , y)I[i][j]
i , j∑
(wecouldobtainanoutputpixelasalinearcombinationoftheinputtexturepixels.)
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Resamplingequation
• whereistheinversemappingistheJacobianmatrixofN(changesofvariables)(continuityofcompositionofFandN)
26
I[i][j]← det(D
M −1 ) Fi, j M −1(xt , yt )( )( )M (Ωi , j )∫∫ Bk ,lk ,l∑ (xt , yt )T[k][l]⎛
⎝⎜⎜
⎞
⎠⎟⎟dxtdyt
DN
ʹF =F !N
N =M−1
= det(DN ) ʹFi, j (xt , yt )( )M (Ωi , j )∫∫ Bk ,l
k ,l∑ (xt , yt )T[k][l]⎛
⎝⎜⎜
⎞
⎠⎟⎟dxtdyt
1( , ) ( , )w w t tx y M x y−=
• Instead,wecanrewritetheintegrationoverthetexturedomain,insteadofthewindowdomain.
I[i][j]← Fi, j (xw , yw )
Ω∫∫ Bk ,l M (xw , yw )( )k ,l∑ T[k][l]⎛
⎝⎜⎜
⎞
⎠⎟⎟dxwdyw
(xt , yt )=M(xw , yw )
18/05/15
14
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Resamplingequation
• Insummary,whenFisaboxfilter
• whereishowmuchtexelk,lblendstoobtainthevalueatcontinuoustexturelocationp,fromreconstructionstep.
27
= det(DN )
M (Ωi , j )∫∫ Bk ,lk ,l∑ (xt , yt )T[k][l]⎛
⎝⎜⎜
⎞
⎠⎟⎟dxtdyt
I[i][j]← Bk ,l M (xw , yw )( )
k ,l∑ T[k][l]⎛
⎝⎜⎜
⎞
⎠⎟⎟dxw dywΩ∫∫
Bk ,l (p)
(xt , yt )=M(xw , yw )
I[i][j]← Fi, j (xw , yw )
Ω∫∫ Bk ,l M (xw , yw )( )k ,l∑ T[k][l]⎛
⎝⎜⎜
⎞
⎠⎟⎟dxwdyw
= det(DN ) ʹFi, j (xt , yt )( )M (Ωi , j )∫∫ Bk ,l
k ,l∑ (xt , yt )T[k][l]⎛
⎝⎜⎜
⎞
⎠⎟⎟dxtdyt
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Summary• Thatis,weneedtointegrateovertheregiononthetexturedomain,andblendthatdatatogether.
• WhenourtransformationMeffectivelyshrinksthetexture,thenhasalargefootprintover
• IfMisblowingupthetexture,thenhasaverynarrowfootprintover
• Duringtexturemapping,Mcanalsodofunnierthings,likeshrinkinonedirectiononly.
28
M(Ωi , j )
M(Ωi , j )T(xt , yt )
M (Ωi, j )T (xt , yt )
(xt , yt )=M(xw , yw )
18/05/15
15
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Blowuptextureàsmallfootprint• Inthecasethatweareblowingupthetexture(narrowfootprintinwindows),thefilteringcomponenthasminimalimpactontheoutput.
• Inparticular,thefootprintofmaybesmallerthanapixelunitintexturespace,andthusthereisnotmuchdetailthatneedsblurring/averaging.
• Assuch,theintegrationstepcanbedropped,andtheresamplingcanbeimplementedas
29
M (Ωi, j )
I[i][j]← Bk ,l(xt , yt )
k ,l∑ T[k][l]⇐ det(DN )M(Ωi , j )
∫∫ Bk ,l(xt , yt )k ,l∑ T[k][l]⎛
⎝⎜⎜
⎞
⎠⎟⎟dxtdyt
(xt , yt )=M(i , j)
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Blowuptextureàsmallfootprint• WetellOpenGLtodothisusingthecallglTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR).
• Forasingletexturelookupinafragmentshader,thehardwareneedstofetch4texturepixelsandblendthemappropriately.
30
18/05/15
16
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Shrinkingtextureàlargefootprint• Inthecasethatatextureisgettingshrunkdown,then,toavoidaliasing,thefiltercomponentshouldnotbeignored.
• Unfortunately,theremaybenumeroustexturepixelsunderthefootprintof,andwemaynotbeabletodoourtexturelookupinconstanttime.
31
M (Ωi, j )
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Recap:MIPmapping• AnacronymoftheLatinphrase,multuminparvo(muchinlittle).
• Mipmapsareprecalculatedscaledversionsofanoriginaltexture.
32
18/05/15
17
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Recap:MIPmapping• Observethepatternofeachmipmapasitdecreasesinsize;itishalfthedimensionofthepreviousone.
• Thispatternisrepeateduntilthelastmipmap’sdimensionis1×1.
• Interpolatemulti-resolutiontexturesviatrilinearinterpolation
33
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Mipmappingindetail• Inmipmapping,onestartswithanoriginaltextureandthencreatesaseriesoflowerandlowerresolution(blurrier)texture.
• Eachsuccessivetextureistwiceasblurry.Andbecausetheyhavesuccessivelylessdetail,theycanberepresentedwith½thenumberofpixelsinboththehorizontalandverticaldirections.
34
T 0
T i
18/05/15
18
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Mipmappingindetail• Thiscollection,calledamipmap,isbuiltbeforetheanytrianglerenderingisdone.
• Thus,thesimplestwaytoconstructamipmapistoaveragetwo-by-twopixelblockstoproduceeachpixelinalowerresolutionimage.
• Duringtexturemapping,foreachtexturecoordinate,thehardwareestimateshowmuchshrinkingisgoingon.– Howbigisthepixelfootprintonthegeometry.
35
(xt , yt )
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Mipmappingindetail• Thisshrinkingfactoristhenusedtoselectfromanappropriateresolutiontexturefromthemipmap.Sincewepickasuitablylowresolutiontexture,additionalfilteringisnotneeded,andagain,wecanjustusereconstruction.
• Toavoidspatialortemporaldiscontinuitieswhere/wherethetexturemipmapswitchesbetweenlevels,wecanso-calledtrilinearinterpolation.Weusebilinearinterpolationtoreconstructonecolorfromandanotherreconstructionfrom.Thesetwocolorsarethenlinearlyinterpolated.Thisthirdinterpolationfactorisbasedonhowclosewearetochoosingleveliori+1
36
T i
T i T i+1
18/05/15
19
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Mipmappingindetail• MipmappingwithtrilinearinterpolationisspecifiedwiththecallglTexParameteri(GLTEXTURE2D,GLTEXTUREMINFILTER,GLLINEARMIPMAPLINEAR)
• TrilinearinterpolationrequiresOpenGLtofetch8texturepixelsandblendthemappropriatelyforeveryrequestedtextureaccess.
37
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Trilinearinterpolation• Trilinearinterpolation(3D):
38
p(x, y, z) = c0 + c1Δx + c2Δy + c3Δz + c4ΔxΔy + c5ΔxΔzc6ΔyΔz + c7ΔxΔyΔz
p0 + [0,1,n,n +1,n2,
n2 +1,n2 + n,n2 + n +1]ThedimensionofCLUTis(2n+1)x(2n+1)x(2n+1)
18/05/15
20
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Trilinearinterpolation• where
39
c1 = (p100 − p000 ) / (x1 − x0 )c2 = (p010 − p000 ) / (y1 − y0 )c3 = (p001 − p000 ) / (z1 − z0 )c4 = (p110 − p010 − p100 + p000 ) / [(x1 − x0 )(y1 − y0 )]c5 = (p101 − p001 − p100 + p000 ) / [(x1 − x0 )(z1 − z0 )]c6 = (p011 − p001 − p010 + p000 ) / [(y1 − y0 )(z1 − z0 )]c7 = (p111 − p011 − p101 − p110 + p100 + p001 + p010 − p000 ) /
[(x1 − x0 )(y1 − y0 )(z1 − z0 )].
Δx = x − x0Δy = y − y0Δz = z − z0c0 = p000
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Properties• Itiseasytoseethatmipmappingdoesnotdotheexactlycorrectcomputation.
• Firstofall,eachlowerresolutionimageinthemipmapisobtainedbyisotropicshrinking,equallyineverydirection.But,duringtexturemapping,someregionoftexturespacemaygetshrunkinonlyonedirection.
40
Nomipmapping
Mipmapping
Anisotropicmipmapping
18/05/15
21
MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012
Properties
41
M (Ωi, j )
• Evenforisotropicshrinking,thedatainthelowresolutionimageonlyrepresentsaveryspecific,dyadic,patternofpixelaveragesfromtheoriginalimage.
• Filteringcanbebetterapproximatedattheexpenseofmorefetchesfromvariouslevelsofthemipmaptoapproximatelycovertheareaonthetexture.
• ThisapproachisoftencalledansiotropicfilteringandcanbeabledinanAPIorusingthedrivercontrolpanel.
Top Related