harnessing deep learning for anomaly detection in log data: a comprehensive study

With the increasing prevalence of online services, big data systems, and internet of things (iot) devices, detecting anomalies in large system logs has become a significant concern. this study presents a systematic literature review of automated log analysis for anomaly detection from january 2017 to october 2024. the study classifies existing approaches into five types: hybrid, supervised, unsupervised, semi-supervised, and self-supervised. each technique is analysed based on its assumptions, benefits, limitations, computational complexity, and performance in practical applications. additionally, it addresses the challenges and concerns associated with developing anomaly detection systems for real-life applications using deep neural networks. the survey's objective is not to perform a statistical analysis of the published methodologies but to classify them, highlight the key features of various deployed architectures, and focus on unresolved issues that require further investigation in this domain. the study offers valuable direction for researchers, emphasising the need for scalable, robust, and interpretable anomaly detection systems. this survey advances the understanding of current capabilities and highlights future directions for enhancing the reliability of complex systems.