Sub-Pixel Layout for Super-Resolution with Images in the ...hangzhao/posters/ECCV14_Poster.pdf ·...
1
Sub-Pixel Layout for Super-Resolution with Images in the Octic Group Boxin Shi 1,2 , Hang Zhao 1 , Moshe Ben-Ezra 1 , Sai-Kit Yeung 2 , Christy Fernandez-Cull 3 , R. Hamilton Shepard 3 , Christopher Barsi 1 , and Ramesh Raskar 1 1 2 3 This paper presents a novel super-resolution framework by exploring the properties of non-conventional pixel layouts and shapes. We show that recording multiple images, transformed in the octic group, with a sensor of asymmetric sub-pixel layout increases the spatial sampling compared to a conventional sensor with a rectilinear grid of pixels and hence increases the image resolution. The example above shows OTCCD (Orthogonal-Transfer CCD, MIT Lincoln Lab) images under 4 rotations can perform 4x Super-Resolution (SR). Idea illustration Take-home message: Result Contributions 23.47 10.71 4.67 1.56 Ground truth Image observed with a conventional sensor with a rectilinear grid of pixels SR results varying with sub-pixel layouts All full-rank sub-pixel layouts achieve SR reconstruction with designated magnification. Simulated images with various sub-pixel layouts for 4x SR. RMSE 24.39 1.75 16.57 1.70 22.92 3.87 4x SR Result OTCCD layout GT | LR | SR 8x SR Result LR | SR Key proposition rank: 64 Sub-pixel layouts for 8x SR. Real-simulation result for 4x (left) and 8x (right) SR Note how higher -rank layouts increase the resolution. 4x SR needs 4 images rotated by 90 o ; 8x SR needs all 8 images in the octic group. This is the gap among sub-pixels (treat as a dumb pixel). It also contributes to SR. o A novel view to the SR problem by using an asymmetric sub-pixel layout to form multiple images in the octic group o The theoretical bound of SR performance w.r.t. the number and distribution of sub-pixels o A sub-pixel layout selection algorithm to choose good layouts for well-posed SR o A simple yet effective SR reconstruction algorithm Potential hardware Given a group of pixels with poses in the octic group with each pixel containing sub-pixels, for a designated magnification factor , rank( ) is bounded as rank( ) is determined by number and distribution of sub-pixels; It evaluates the well- posedness of SR reconstruction. OTCCD + Dove prism + rotation mount “Rotation-asymmetric sub-pixel layout + rotated images” play an important role in increasing spatial sampling to break the limit of reconstruction-based super-resolution. Conventional CCD OTCCD images rotated by 0, 90, 180, and 270 degrees SR result An OTCCD pixel (with 4 sub-pixels) in the octic group rank: 4 rank: 7 rank: 12 rank: 16 rank: 16 Octic group (4 th order dihedral group) Original Original 90 o rotation 90 o rotation 180 o rotation 180 o rotation 270 o rotation 270 o rotation Reflection Reflection Reflection Reflection Reflection Reflection Reflection Reflection rank( ) min
Transcript of Sub-Pixel Layout for Super-Resolution with Images in the ...hangzhao/posters/ECCV14_Poster.pdf ·...
Sub-Pixel Layout for Super-Resolution with Images in the Octic GroupBoxin Shi1,2, Hang Zhao1, Moshe Ben-Ezra1, Sai-Kit Yeung2, Christy Fernandez-Cull3, R. Hamilton Shepard3, Christopher Barsi1, and Ramesh Raskar1
1 2 3
This paper presents a novel super-resolution framework by exploring the propertiesof non-conventional pixel layouts and shapes. We show that recording multipleimages, transformed in the octic group, with a sensor of asymmetric sub-pixel layoutincreases the spatial sampling compared to a conventional sensor with a rectilineargrid of pixels and hence increases the image resolution. The example above showsOTCCD (Orthogonal-Transfer CCD, MIT Lincoln Lab) images under 4 rotations canperform 4x Super-Resolution (SR).
Idea illustration
Take-home message:
Result
Contributions
23.47
10.71
4.67
1.56
Ground truth
Image observed witha conventional sensorwith a rectilinear grid of pixels
SR results varying with sub-pixel layouts
All full-rank sub-pixel layoutsachieve SR reconstructionwith designated magnification.
Simulated images with various sub-pixel layouts for 4x SR.
RMSE24.39 1.75
16.57 1.70
22.92 3.87
4x SR ResultOTCCD layoutGT | LR | SR
8x SR ResultLR | SR
Key proposition
rank: 64
Sub-pixel layouts for 8x SR.
Real-simulation result for 4x (left) and 8x (right) SR
Notehow higher-rank layoutsincrease theresolution.
4x SR needs 4 imagesrotated by 90o; 8x
SR needs all 8images in the
octic group.
This is the gap amongsub-pixels (treat as a dumb pixel). It also contributes to SR.
o A novel view to the SR problem by using an asymmetric sub-pixel layout to form multiple images in the octic group
o The theoretical bound of SR performance w.r.t. the number and distribution of sub-pixels
o A sub-pixel layout selection algorithm to choose good layouts for well-posed SRo A simple yet effective SR reconstruction algorithm
Potential hardware
Given a group of pixels with poses in the octic group with each pixel containing sub-pixels, for a designated magnification factor , rank( ) is bounded as
rank( ) is determined by number and distribution of sub-pixels; It evaluates the well-posedness of SR reconstruction.
OTCCD + Dove prism + rotation mount
“Rotation-asymmetric sub-pixel layout + rotated images” play an important role in increasing spatial sampling to break the limit of reconstruction-based super-resolution.
Conventional CCD OTCCD images rotated by 0, 90, 180, and 270 degrees SR result
An OTCCD pixel (with 4 sub-pixels) in the octic group
rank: 4
rank: 7
rank: 12
rank: 16
rank: 16
Octic group (4th order dihedral group)O
rigin
alO
rigin
al
90o
rota
tion
90o
rota
tion
180o
rota
tion
180o
rota
tion
270o
rota
tion
270o
rota
tion
ReflectionReflection ReflectionReflection ReflectionReflection ReflectionReflection
rank( ) min
