Automatic Target Recognition Using Automatic Target Recognition Using Algebraic Functions of Views (AFoVs)Algebraic Functions of Views (AFoVs)

George Bebis and Wenjing LiGeorge Bebis and Wenjing LiComputer Vision LaboratoryComputer Vision LaboratoryDepartment of Computer ScienceDepartment of Computer ScienceUniversity of Nevada, RenoUniversity of Nevada, Reno

http://www.cs.unr.edu/CVLhttp://www.cs.unr.edu/CVL

The main goal of this project is to improve the The main goal of this project is to improve the performance of Automatic Target Recognition (ATR) by performance of Automatic Target Recognition (ATR) by developing a more powerful ATR frame work which can developing a more powerful ATR frame work which can handle changes in the appearance of a target more handle changes in the appearance of a target more efficiently and robustly. The new framework will be built efficiently and robustly. The new framework will be built around a hybrid model of appearance by integrating (1) around a hybrid model of appearance by integrating (1) Algebraic Functions of Views (AFoVs), a powerful Algebraic Functions of Views (AFoVs), a powerful mathematical model of geometric appearance, with (2) mathematical model of geometric appearance, with (2) eigenspace representations, a well known empirical eigenspace representations, a well known empirical model of appearance which has demonstrated model of appearance which has demonstrated significant capabilities in recognizing complex objects significant capabilities in recognizing complex objects under no occlusion. This project is sponsored by The under no occlusion. This project is sponsored by The office of Naval Research (ONR).office of Naval Research (ONR).

Main GoalMain Goal

ProblemProblem

We address the problem of 3D object recognition from We address the problem of 3D object recognition from 2D images assuming that: 2D images assuming that: – viewpoint is arbitraryviewpoint is arbitrary– 3D structure information is not available3D structure information is not available

Given some knowledge of how certain objects Given some knowledge of how certain objects may appear and an image of a scene possibly may appear and an image of a scene possibly containing those objects, report containing those objects, report which which objects objects are present in the scene and are present in the scene and where.where.

ObjectivesObjectives

Couple AFoVs with eigenspace representation for Couple AFoVs with eigenspace representation for enhanced hypothesis generation and verification enhanced hypothesis generation and verification

Integrate AFoVs with grouping for robust feature extraction Integrate AFoVs with grouping for robust feature extraction and efficient hypothesis generation and efficient hypothesis generation

Integrate AFoVs with indexing to bypass the Integrate AFoVs with indexing to bypass the correspondence problem and enable efficient searching correspondence problem and enable efficient searching

Integrate AFoVs with probabilistic hypothesis generation Integrate AFoVs with probabilistic hypothesis generation Integrate AFoVs with incremental learning Integrate AFoVs with incremental learning Design a methodology for choosing a sparse set of Design a methodology for choosing a sparse set of

reference viewsreference views Extend AFoVs to other types of imagery Extend AFoVs to other types of imagery

FrameworkFramework

Images from various viewpoints

Model groups

Convex grouping

Selection of referenceviews

Estimate the range of parameter values of AFoVs

Sample space of appearances

Compute index

Realistic appearances

New image

Image groups

Compute index

Index Structure

Estimate AFoVs parameters

Predict appearance

Verify predictions

Convex grouping

Training stage Recognition stage

Using SVD & IA

Using constraints

Access

Retrieve

Establish HypothesesRank them by probability

Coarse k-d tree

compute probabilities using Gaussian mixtures

1 2(mod , , , )el v v group

Experimental ResultsExperimental Results

3 models2 views/model

group size: 516 groups/object3300 sampled views/object (on average)

Experimental Results Experimental Results (cont’d)(cont’d)

reference views reference views

novel view novel view

Experimental Results Experimental Results (cont’d)(cont’d)

reference views reference views

novel viewnovel view

Experimental Results Experimental Results (cont’d)(cont’d)

reference views

novel view novel view

Work in ProgressWork in Progress

Employ a scheme for selecting “good” groups of features.Employ a scheme for selecting “good” groups of features. Devise a method for selecting the reference views.Devise a method for selecting the reference views. Reject unrealistic appearances from index table.Reject unrealistic appearances from index table. Employ improved indexing schemes (e.g., K-d trees).Employ improved indexing schemes (e.g., K-d trees). Represent object appearance more compactly.Represent object appearance more compactly.

– Reduces space requirements considerably.Reduces space requirements considerably. Develop a probabilistic hypothesis generation scheme. Develop a probabilistic hypothesis generation scheme.

– Use probabilistic models to represent geometric model Use probabilistic models to represent geometric model appearance.appearance.

Combine geometric and empirical models of appearance.Combine geometric and empirical models of appearance.– Can improve hypothesis generation and verification.Can improve hypothesis generation and verification.

ModelsModels

Verification ResultsVerification Results

Group 1, MSE=8.0339e-5 Group 2, MSE=4.3283e-5

Verification ResultsVerification Results

Group 1, MSE=2.5977e-5

Group 2, MSE=5.3829e-5

Group 3, MSE=5.9901e-5

Verification ResultsVerification Results

Group 1, MSE=3.9283e-5 Group 1 (Shift 4), MSE=3.3383e-5

Combine Geometric and Empirical Combine Geometric and Empirical Models of Appearance Models of Appearance

Current AFoVs framework predicts geometric appearance.Current AFoVs framework predicts geometric appearance. Extend AFoVs framework to predict empirical appearance.Extend AFoVs framework to predict empirical appearance. Integrate geometric with empirical appearance.Integrate geometric with empirical appearance.

– Improve both hypothesis generation and verification.Improve both hypothesis generation and verification.

Real ImagesReal Images

Related PublicationsRelated Publications

G. Bebis et al., “Genetic Object Recognition Using Combinations G. Bebis et al., “Genetic Object Recognition Using Combinations of Views”, of Views”, IEEE Transactions on Evolutionary Computing, IEEE Transactions on Evolutionary Computing, vol. 6, vol. 6, no. 2, pp. 132-146, 2002no. 2, pp. 132-146, 2002..

G. Bebis et al., “Indexing Based on Algebraic Functions of G. Bebis et al., “Indexing Based on Algebraic Functions of Views”, Views”, Computer Vision and Image Understanding (CVIU),Computer Vision and Image Understanding (CVIU), vol. vol. 72, no. 3, pp. 360-378, 1998.72, no. 3, pp. 360-378, 1998.

G. Bebis et al., “Learning Affine Transformations”, G. Bebis et al., “Learning Affine Transformations”, Pattern Pattern Recognition, Recognition, vol. 32, pp. 1783-1799, 1999.vol. 32, pp. 1783-1799, 1999.

G. Bebis et al., “Algebraic Functions of Views for Model-Based G. Bebis et al., “Algebraic Functions of Views for Model-Based Object Recognition”, Object Recognition”, International Conference on Computer International Conference on Computer Vision (ICCV), Vision (ICCV), pp. 634-639, 1998.pp. 634-639, 1998.

http://www.cs.unr.edu/CVLhttp://www.cs.unr.edu/CVL