Distortion Correction ECE 6276 Project Review Team 5: Basit Memon Foti Kacani Jason Haedt Jin Joo...
-
Upload
cecily-hall -
Category
Documents
-
view
214 -
download
0
Transcript of Distortion Correction ECE 6276 Project Review Team 5: Basit Memon Foti Kacani Jason Haedt Jin Joo...
Distortion CorrectionECE 6276 Project Review
Team 5:Basit MemonFoti KacaniJason HaedtJin Joo LeePeter Karasev
2 ECE 6276 Final Project Team 5
7/14/2009
Outline
• Motivation
• Components
• Component Optimization
• Current Results
• Plans for Catapult C
• Schedule
3 ECE 6276 Final Project Team 5
7/14/2009
Objective
Given a distorted image with known size and known lens distortion parameter, generate an undistorted image.
4 ECE 6276 Final Project Team 5
7/14/2009
Motivation – Why?
• The formation of undistorted images can be described by a series of matrix multiplications
• Distortion makes it very difficult to calibrate a camera to measure geometry (depth, size, orientation, etc)
• Many applications in image processing and computer vision like structure estimation, image mosaicing, and ultimately vision-based control.
5 ECE 6276 Final Project Team 5
7/14/2009
Motivation contd..
Application: Measure motion and geometryProblem: Known geometry in the scene is warped, relationship between 3D and 2D points is nonlinear. Solution: Undo the distortion, so x2D = A * X3D
6 ECE 6276 Final Project Team 5
7/14/2009
Literature Review (I)
K.T. Gribbon, C.T. Johnston, and D.G. Bailey, “A Real-time FPGA Implementation of a Barrel Distortion Correction Algorithm with Bilinear Interpolation”
•Focus on reducing hardware complexity.
•Uses LUTs to store mapping data.
•No quantitative results provided.
•Logic resource utilization on RC-100 is 51%
7 ECE 6276 Final Project Team 5
7/14/2009
Literature Review (II)
Qiang, L.; Allinson, N.M.;” FPGA Implementation of Pipelined Architecture for Optical Imaging Distortion Correction”, Signal Processing Systems Design and Implementation, 2006. SIPS '06.
•Same algorithm as previous one
•Implementation on a Xilinx FPGA XCS3 1000-4 uses 75% of the hardware multipliers.
•Residual error of the undistorted image was 1.5% of the distorted image.
8 ECE 6276 Final Project Team 5
7/14/2009
Literature Review (III)
Hany Farid & Alin C. Popescu, “Blind Removal of Lens Distortion”, Journal of the Optical Society of America 2001
•For removal of distortion in absence of any calibration
data.•Uses polyspectral analysis to detect higher order correlations in frequency domain which are proportional to the distortion.
•Computationally intensive.
•No quantitative results.•Accuracy is not comparable to those based on known distortion parameters.
9 ECE 6276 Final Project Team 5
7/14/2009
Components to Achieve ObjectiveComponent 1: Matlab forward distortion function
– Verifies correctness of undistortion algorithm
Component 2: Data ordering test bench – Order the C++ input stream from MATLAB generated data
Component 3: Undistortion lookup table or lookup function– Initial prototype in MATLAB
Component 4: Least squares interpolation lookup function– Initial prototype in MATLAB– Compare different techniques such as LUT vs NEAREST NEIGHBOR
Component 5: Verification structure – Compare original C++ result to undisorted image in MATLAB
MATLAB C++ Catapult C MATLAB
10 ECE 6276 Final Project Team 5
7/14/2009
Component Optimization Questions• What size buffers do we need to compare against previous frames?
• For undistortion function can we compute them dynamically or do we need a pre-defined LUT?
• For finding fast least squares / linear system solver, compare speed cost vs. nearest neighbor and effect on output error
11 ECE 6276 Final Project Team 5
7/14/2009
Matlab Demo of Algorithm - Original
12 ECE 6276 Final Project Team 5
7/14/2009
Matlab Demo of Algorithm - Distorted
13 ECE 6276 Final Project Team 5
7/14/2009
Matlab Demo of Algorithm - Recovered
14 ECE 6276 Final Project Team 5
7/14/2009
Design Goals
• Limited Buffers
• 8 bits per sub-pixel (24 bits total)
• Resolution (up to 640x480)
• Concerned with geometry
• Area, Throughput, Latency
•Target a low-cost implementation that handles consumer video application pixel clocks of 165 MHz (Apprx 6 ns cycle time).
15 ECE 6276 Final Project Team 5
7/14/2009
Test Vectors
The Line Test
Original Image
Distorted Image
Recovered Image
16 ECE 6276 Final Project Team 5
7/14/2009
Plans for Catapult C Code
• Use C/C++ and Algorithmic C data types to describe synthesizable hardware• Architectures (type of hardware interface (streaming buffers) )• Constraints (Throughput, area, latency)•RTL generation and verification • Optimizations
-Pipelining-Parallelism-Loop Unrolling-Scheduling-Streaming buffers/ Read & Write
17 ECE 6276 Final Project Team 5
7/14/2009
Project Timeline
Tasks Owner Description Week 1 Week 2 Week 3Algorithm Research ALL Research different papers on barrel distortion
MATLAB Implementation PeterMock up distortion algorithm using MATLAB and use image
processing libraries for ease of use
MATLAB/C++ Testbench Jason Take MATLAB results and convert them into C++ structure to
be used in C++ testbenchC++ Code Porting Jin Joo Port MATLAB implementation of undistortion to C++
Presentation 1 ALL Initial project reportCatapult C Porting Foti,Memon Port to supported CATAPULT datatypes
Catapult C Synthesis Jason See results of CATAPULT algorithmPlans for Optimization Foti, Memon Look at ways to optimize CATAPULT code
Presentation 2 ALLCompleted Catapult Code, Synthesis Results, Steps for
OptimizationCatapult C Optimization Jin Joo Complete optimization suggestions
Catapult C Synthesis of Optimization Jason See results of CATAPULT algorithmDemo ALL Show CATAPULT demo and final HDL
Presentation 3 ALL Final report out
18 ECE 6276 Final Project Team 5
7/14/2009
Project Risks
Risk 1:
Slowness of interpolation plan: the key goal of undistortion is “geometrical accuracy” which hold even if noise is injected. To mitigate this risk we have a forward mapping algorithm that is fast compared to any interpolation method but at the cost of missing pixels near the edges.
Risk 2:
Not enough storage space to keep a lookup table of coordinates. To mitigate this risk we could compute coordinates on the fly at the cost of math operations.
Risk 3:
Cannot find least squares method for FPGA. To mitigate this risk we could do nearest neighbor interpolation.
19 ECE 6276 Final Project Team 5
7/14/2009
Current Status
•Straight lines are recovered with minimal injection of missing points or noise through our algorithm.
•Add more quantitative results on the geometrical accuracy of recovered images than presented in previous results.
•Generalize the coordinate mapping to make the implementation more robust to varying distortion models and hence supporting more cameras “on the fly.”
20 ECE 6276 Final Project Team 5
7/14/2009
References
• Yi Ma, Stefano Soatto, et al., “An Invitation to 3-D Vision”
•Richard Hartley, Andrew Zisserman, “Multiple View Geometry in Computer Vision”
•Edward M. Mikhail, James S. Bethel, J. Chris McGlone, “Introduction to Modern Photogrammetry”
21 ECE 6276 Final Project Team 5
7/14/2009
Questions?
?