Black-Mark Removal and Semantic Motif Segmentation for Digital Restoration of Fresco Fragments

Abstract

Art restoration is a crucial process in the preservation and appreciation of cultural heritage. Among various types of art, ancient frescoes present unique challenges due to their delicate nature and susceptibility to deterioration. This thesis presents a study on the digital restoration of broken fragments of ancient frescoes from Pompeii, focusing on addressing common problems such as manually-added black marks and subsequent deterioration of the intact surfaces of the fragments, and the segmentation of patterns painted by the artists. This work is particularly important as it serves as a pre-processing step for potential fresco reconstruction using puzzle-solving algorithms. The proposed restoration process involves detecting the black lines and removing them by inpainting, visual enhancement, and segmenting the patterns. In particular, we investigated the use of off-the-shelf OpenCV functions and the YOLO (You Only Look Once) to detect manually-added black marks and conducted a comparative analysis of several inpainting methods, including Telea's Method, Fast Marching Method, Biharmonic Method, and Criminisi's Method for their removal. Then, semantic segmentation is applied to the fresco fragments, which aids in differentiating between various elements within the artworks, such as background, figures, and architectural features. The results provide valuable insights into the effectiveness of different inpainting techniques and semantic segmentation for fresco image restoration, paving the way for future research and development in digital art conservation and fresco reconstruction. Overall, this research contributes to the advancement of digital art conservation, with implications for both the restoration and potential reconstruction of damaged or fragmented frescoes.