From Images to 3D Model: Reconstructing Fresco Fragments

Abstract

Reconstructing the three-dimensional structure of an object starting from a sequence of RGB images (multi-view 3D reconstruction) is a well-known problem and a challenging task. The hardware needed for the acquisition, the size and shape of the object, and the environmental conditions greatly affect the accuracy of the reconstruction. Moreover, a multi-view 3D reconstruction involves many intermediate steps, from data acquisition to image pre-processing, up to mesh generation and its refinement. Each intermediate step significantly affects the successive ones. This work focuses on data-preprocessing, particularly in two directions: the former considers reconstructing a 3D object from curated and background-free images, investigating how object segmentation affects the final 3D reconstruction. The latter automates the former, considering a deep-learning approach to automatically detect and segment the object of interest in the sequence of images used to recover the 3D structure. The performance are assessed on a dataset of fresco fragments acquired within the H2020 project, RePAIR.