Anomaly detection with time series forecasting

Abstract

Anomaly detection time series is a very large and complex field. In recent years, several techniques based on data science were designed in order to improve the efficiency of methods developed for this purpose. In this paper, we introduce Recurrent Neural Networks (RNNs) with LSTM units, ARIMA and Facebook Prophet library for dectecting the anomalies with time series forcasting. Due to the challenges in obtaining labeled anomaly datasets, an unsupervised approach is employed. Unsupervised anomaly detection is the process of finding outlying records in a given dataset without prior need for training. An anomaly could become normal during the data evolution, therefore it is necessary to maintain a dynamic system to adapt the changes. While LSTM and ARIMA are powerful methods for time series forecasting the future, the Prophet package works best with time series that have strong seasonal effects and several seasons of historical data. The Prophet is robust to missing data and shifts in the trend, and typically handles anomalies well. The resulting prediction errors are modeled to give anomaly scores. We also provide a quantitative comparison among approved techniques for voting the optimal choice of the problem. Our experiments, we implement with the practical datasets collected from real products using by thousands of users.