1. Sparse Isotropic Hashing Ikuro Sato, Mitsuru Ambai, Koichiro Suzuki Denso IT Laboratory, Inc. {isato, manbai, ksuzuki}@d-itlab.co.jp 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 1/28 Presented at MIRU 2013, Japan. Peer reviewed paper available at http://www.am.sanken.osaka-u.ac.jp/CVA/

2. Introduction Proposed method Experiment 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 2/36 3. Practical issues of large-scale image retrieval ex) descriptor-matching approach millions of sums-of-product / query ? slow query image query image DB: ~108 images 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 3/28 4. Potential solution: descriptor binarization computational time of similarity real 512 bit 256 bit 128 bit 64 bit 32 bit binary codes1 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 4/28 5. Binarization by hash functions 1. supervised uses known point-to-point correspondences ex) Ambai et al, 2012. 2. unsupervised intends to preserve similarities among the original real vectors 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 5/28 6. Popular hash function ex) Random Proj. (Goemans et al, 1995) Very Sparse Rand. Proj. (Li et al, 2006) Sequential Proj. (Wang et al, 2010) 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. Iterative Quantization (Gong et al, 2011) Isotropic Hashing (Kong et al, 2012) this work state-of-the-art 6/28 7. Most related work: Isotropic Hashing (Kong et al, 2012) 1. orthonormality 2. isotropic variance 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 7/28 8. Most related work: Isotropic Hashing (Kong et al, 2012) 1. orthonormality 2. isotropic variance Robust to noise from spherically symmetric distribution. 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 8/28 9. Learning of Isotropic Hashing Lift and Projection (LP) algorithm isotropic orthogonal Gradient Flow algorithm omitted. intersection 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 9/28 1) PCA: 10. Under-constrained problem Its more natural to impose additional conditions to make the problem over-constrained. our motivation 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 10/28 11. Our contribution 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 11/28 12. Introduction Proposed method Experiment 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 12/36 13. Problem setup 1. rotational matrix 2. isotropic variance 3. sparsity 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 13/28 14. Condition-1: Special orthogonal group -1 1 0 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 14/28 15. Condition-2: Cost function for isotropic variance Exact solutions exist according to the Schur-Horn Theorem (AJM1954). 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 15/28 16. 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 16/28 17. Our optimization problem 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 17/28 18. Algorithm Sparse Isotropic Hashing (SIH) Repeat until convergence. endfor notations simplified 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 18/28 19. Illustration of the optimization process 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 19/28 20. Introduction Proposed method Experiment 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 20/36 21. Dataset etc. * M. Ambai and I. Sato, Fast binary coding of local descriptors based on supervised learning (MIRU2012). descriptor query set (u=1) training set (u=2, 3, 4) test set (u=5, 6) CARD (Ambai et al, 2011) without binarization 12896 50053 25238 # local descriptors 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 21/28 22. Evaluation criterion Mean Average Precision (mAP) expected value of area under Precision-Recall curve 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. precision recall 1.0 Average Precision 22/28 23. Methods compared All methods use 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 23/28 state-of-the-art 24. mAP for CARD 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 24/28 25. mAP for CARD 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 25/28 26. mAP for CARD almost on top 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 26/28 27. mAP for CARD 10% drop in mAP, 20x faster coding env.: VS2010, C program 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 27/28 28. Conclusion Isotropic Hashing (Kong et al, 2012): highly under-constrained 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 28/28