Learning representations for visual recognition

Mots clé :

• transfer learning
• convolutional neural network
• architecture learning

In this dissertation, we propose methods and data driven machine learning solutions which address and benefit from the recent overwhelming growth of digital media content.
First, we consider the problem of improving the efficiency of image retrieval. We propose a coordinated local metric learning (CLML) approach which learns local Mahalanobis metrics, and integrates them in a global representation where the L2 distance can be used.  This allows for data visualization in a single view, and use of efficient L2-based retrieval methods. Our approach can be interpreted as learning a linear projection on top of an explicit high-dimensional embedding of a kernel. This interpretation allows for the use of existing frameworks for Mahalanobis metric learning for learning local metrics in a coordinated manner. Our experiments show that CLML improves over previous global and local metric learning approaches for the task of face retrieval.
Second, we present an approach to leverage the success of CNN models for visible spectrum face recognition to improve heterogeneous face recognition, eg., recognition of near-infrared images from visible spectrum training images. We explore different metric learning strategies over features from the intermediate layers of the networks, to reduce the discrepancies between the different modalities. In our experiments we found that the depth of the optimal features for a given modality, is positively correlated with the domain shift between the source domain (CNN training data) and the target domain. Experimental results show the that we can use CNNs trained on visible spectrum images to obtain results that improve over the state-of-the art for heterogeneous face recognition with near-infrared images and sketches. 
Third, we present convolutional neural fabrics for exploring the discrete and exponentially large CNN architecture space in an efficient and systematic manner. Instead of aiming to select a single optimal architecture, we propose a ``fabric'' that embeds an exponentially large number of architectures. The fabric consists of a 3D trellis that connects response maps at different layers, scales, and channels  with a sparse homogeneous  local connectivity pattern. The only hyper-parameters of the fabric (the number of channels and layers) are not critical for performance. The acyclic nature of the fabric allows us to use backpropagation for learning. Learning can thus efficiently configure the fabric to implement each one of exponentially many architectures and, more generally, ensembles of all of them.  While scaling linearly in terms of computation and memory requirements, the fabric leverages exponentially many chain-structured architectures in parallel by massively sharing weights between them. We present benchmark results competitive with the state of the art for image classification on MNIST and CIFAR10, and for semantic segmentation on the Part Labels dataset.

Directeurs:

• Mme Cordelia Schmid (directeur de recherche - INRIA Grenoble )
• Mr Jakob Verbeek (Chargé de recherche - INRIA Grenoble )

Raporteurs:

• Mr Frederic Jurie (Professeur - University of Caen )
• Tinne Tuytelaars (Professeur - Katholieke Universiteit Leuven )

Examinateurs:

• Mr Andrew Bagdanov (Associate Professeur - University of Florence )