Nội dung

Beyond bags of features: Adding spatial information Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba Adding spatial information • Computing bags of features on sub-windows of the whole image • Using codebooks to vote for object position • Generative part-based models Spatial pyramid representation • Extension of a bag of features • Locally orderless representation at several levels of resolution level 0 Lazebnik, Schmid & Ponce (CVPR 2006) Spatial pyramid representation • Extension of a bag of features • Locally orderless representation at several levels of resolution level 0 level 1 Lazebnik, Schmid & Ponce (CVPR 2006) Spatial pyramid representation • Extension of a bag of features • Locally orderless representation at several levels of resolution level 0 level 1 Lazebnik, Schmid & Ponce (CVPR 2006) level 2 Scene category dataset Multi-class classification results (100 training images per class) Caltech101 dataset http://www.vision.caltech.edu/Image_Datasets/Caltech101/Caltech101.html Multi-class classification results (30 training images per class) Implicit shape models • Visual codebook is used to index votes for object position visual codeword with displacement vectors training image annotated with object localization info B. Leibe, A. Leonardis, and B. Schiele, Combined Object Categorization and Segmentation with an Implicit Shape Model, ECCV Workshop on Statistical Learning in Computer Vision 2004 Implicit shape models • Visual codebook is used to index votes for object position test image B. Leibe, A. Leonardis, and B. Schiele, Combined Object Categorization and Segmentation with an Implicit Shape Model, ECCV Workshop on Statistical Learning in Computer Vision 2004 Implicit shape models: Details B. Leibe, A. Leonardis, and B. Schiele, Combined Object Categorization and Segmentation with an Implicit Shape Model, ECCV Workshop on Statistical Learning in Computer Vision 2004 Generative part-based models R. Fergus, P. Perona and A. Zisserman, Object Class Recognition by Unsupervised Scale-Invariant Learning, CVPR 2003 Probabilistic model P (image | object) P(appearance, shape | object) max h P(appearance | h, object) p ( shape | h, object) p (h | object) Part Part h: assignment of features to parts descriptors locations Candidate parts Probabilistic model P (image | object) P(appearance, shape | object) max h P(appearance | h, object) p ( shape | h, object) p (h | object) h: assignment of features to parts Part 1 Part 3 Part 2 Probabilistic model P (image | object) P(appearance, shape | object) max h P(appearance | h, object) p ( shape | h, object) p (h | object) h: assignment of features to parts Part 1 Part 3 Part 2 Probabilistic model P (image | object) P(appearance, shape | object) max h P(appearance | h, object) p ( shape | h, object) p (h | object) Distribution over patch descriptors High-dimensional appearance space Probabilistic model P (image | object) P(appearance, shape | object) max h P(appearance | h, object) p ( shape | h, object) p (h | object) Distribution over joint part positions 2D image space Results: Faces Face shape model Recognition results Patch appearance model Results: Motorbikes and airplanes Representing people Summary: Adding spatial information • Spatial pyramids • Pro: simple extension of a bag of features, works very well • Con: no geometric invariance, no object localization • Implicit shape models • Pro: can localize object, maintain translation and possibly scale invariance • Con: need supervised training data (known object positions and possibly segmentation masks) • Generative part-based models • Pro: very nice conceptually, can be learned from unsegmented images • Con: combinatorial hypothesis search problem