Using Neural Network to detect security-sensitive elements in screenshots of web pages

Abstract

The goal of this work, carried out during the internship at Yarix s.r.l., is to develop an artificial intelligence (AI) module based on neural networks, this will be integrated on Yarix's proprietary cyber intelligence platform to detect security sensible elements, like forms or brand logos, in screenshots of web pages. The purpose of this module is to lighten the work of Yarix analysts, who will be able to go and examine the web pages of the results filtered by the AI module. The image dataset, required for the training of the AI module, was collected through scripts, pre-existing databases, and by hand, by following the needs of the company. After the implementation of the various neural network taken into consideration, we compare the performances of the different versions during both the training and testing phases. Based on previous results in terms of speed and accuracy, the best solution for Yarix's needs was a Single Shoot Detection with MobileNet model. By integrating this model into the Yarix platform and testing the AI module "in the wild", it has been noted that the work of the analysts has been considerably lightened, in particular concerning possible phishing attempts linked, for example, to login forms.