Constrained dominant sets and its applications in computer vision

Abstract

In this thesis, we present new schemes which leverage a constrained clustering method to solve several computer vision tasks ranging from image retrieval, image segmentation and co-segmentation, to person re-identification. In the last decades clustering methods have played a vital role in computer vision applications; herein, we focus on the extension, reformulation, and integration of a well-known graph and game theoretic clustering method known as Dominant Sets. Thus, we have demonstrated the validity of the proposed methods with extensive experiments which are conducted on several benchmark datasets.

We first discuss `Dominant Sets for "Constrained" Image Segmentation,' DSCIS. In DSCIS, we present a unified model to tackle image segmentation and co-segmentation problem in both an interactive and unsupervised fashion; whereby, the proposed algorithm can deal naturally with several types of constraints and input modality, including scribbles, sloppy contours, and bounding boxes, and is able to robustly handle noisy annotations on the part of the user. Our method is based on some properties of a family of quadratic optimization problems related to dominant sets, a well-known graph-theoretic notion of a cluster which generalizes the concept of a maximal clique to edge-weighted graphs. In particular, we show that by properly controlling a regularization parameter which determines the structure and the scale of the underlying problem, we are in a position to extract groups of dominant-set clusters that are constrained to contain predefined elements.

Following, we present novel schemes for content-based image retrieval (CBIR) using constrained dominant sets (CDS). We present two different CBIR methods. The first method, `Multi-feature Fusion for Image Retrieval Using Constrained Dominant Sets,' MfFIR, fuse several hand-crafted and deep features to endow a representative similarity which better define the closeness of given query and gallery images; whereas, the second one, `Constrained Dominant Sets for Image Retrieval,' CDSIR, exploit a constrained diffusion process to produce a robust similarity between query and gallery images. In MfFIR, we propose a computationally efficient approach to fuse several hand-crafted and deep features, based on the probabilistic distribution of a given membership score of a constrained cluster in an unsupervised manner. Towards this end, we first introduce an incremental nearest neighbor (NN) selection method, whereby we dynamically select k-NN to the query. Next, we build several graphs from the obtained NN sets and employ constrained dominant sets (CDS) on each graph G to assign edge weights which consider the intrinsic manifold structure of the graph, and detect false matches to the query. Finally, we compute the positive-impact weight (PIW) based on the dispersive degree of the characteristics vector. As a result, we exploit the entropy of a cluster membership-score distribution. In addition, the final NN set bypasses a heuristic voting scheme. Our approach presents two main advantages. Firstly, compared to the state of the art methods, it can robustly quantify the effectiveness of features for a specific query, without any supervision. Secondly, by diffusing the pairwise similarity between the nearest neighbors, our model can easily avoid the inclusion of false-positive matches in the final shortlist. On the other hand, in CDSIR, we leverage constrained dominant sets to dynamically constrain a similarity diffusion process to provide context-sensitive similarities.

Finally, we present a Deep Constrained Dominant Sets (DCDS); in which, we able to optimize the constrained-clustering process in an end-to-end manner and leverage contextual information in the learning process. In this work, we integrate the well-known graph and game-theoretic method called CDS into a deep model and tackle the challenging computer vision problem of person re-identification.