Cyber data intelligence tool: from raw data breaches to structured storing, performant searching and user-friendly visualisation

Abstract

The thesis aims to summarise and motivate the cyber intelligence project carried out during the internship period in a local company. The scenario in which the developed software runs will be presented. The technologies and criticalities found will be highlighted and the final product that can be used by the company for its own purposes will be shown.

It is possible to find everything you need by surfing the web, also a lot of stolen credentials. Everyday people upload on the web data which came from a breach due to lack of security in the network infrastructures, malware attacks or human errors. Cyber security companies exploit these data breaches to obtain information and implement security activities for their customers. By exploring the deep web, analysts are able to download a massive amount of unauthorised data that very often are raw or not well formed. Hence the need to develop a tool which is able to ingest and structure any kind of data into a database and link it to a data-visualiser in order to perform optimised targeted searches. The final step is to protect customers by alerting them of possible threats due to a leak of confidential and sensitive credentials.