Automatic Extraction of Overlapping Camera Clusters for 3D Reconstruction

Abstract

The purpose of this work is to automatically detect overlapping camera clusters for 3-dimensional (3D) reconstruction, using extensions of the dominant set technique. The two driving motives of the thesis were: first, to remove the number of constraints imposed by previous works while running the clustering algorithm, second, to integrate an image selection algorithm in order to enhance the 3D reconstruction performance further. The constraints imposed by previous works have been vanished after we have employed a version of dominant set clustering which allows overlapping. We have also intervened the bulky dense reconstruction phase by an efficient image selection method.

The methodology used, for extracting overlapping clusters of cameras, is the dominant sets approach which often converges in a very reasonable time. The replicator dynamics locate individual groups, and after each group extraction the similarity matrix is modified with the aim of destabilizing the located cluster under the dynamics, without affecting the other sets. The entire similarity matrix is always passed to the dynamics; there is no need to cut part of the located group from the graph. Doing so allows an object to be grouped in more than one class, which is our interest. Overlap is important in order to get a smooth (well-covered) reconstruction near cluster boundaries.

Experimental results show that the performance of the associated 3D reconstruction is much faster, due to the intervention of image selection algorithm, before the start of a computationally expensive dense reconstruction step. The inputs are list of camera parameters and point clouds found from the famous Bundler - Structure from Motion (SfM) algorithm. Then, our method selects and clusters the cameras, eventually the output is fed to the Patch based Multi View Stereo (PMVS) algorithm. The task of PMVS is producing the final dense reconstruction of the scene. So, in the 3D reconstruction pipeline, our work lies between SfM (which gives sparse 3D point clouds) and PMVS (which gives dense 3D point clouds of the object). Therefore, the outputs of SfM are clustered and selected by our work and then pass to PMVS. In addition to clustering, image selection is employed to cut out unnecessary camera redundancy. Processing near-duplicate images increases the computational time without improving the reconstruction quality.

Comparable results, with the current state-of-the-art of overlapping cluster extraction, have been found in this work. The performance of our method is better than the previous work while mantaining the quality precisely the same. We have tested our method on some bench mark camera-datasets and pretty good results are found.