DEVELOPMENT OF A SOFTWARE SYSTEM FOR ANOMALY DETECTION IN DATA USING NEURAL NETWORKS

Abstract

The paper presents the results of developing a software system for anomaly detection in data using neural networks, aimed at improving detection accuracy, adaptability to changes in data characteristics, and ensuring real-time data processing. The objective of the study is to develop an integrated software system for anomaly detection that provides efficient processing of heterogeneous data, increased accuracy of deviation detection, adaptation to changes in data (concept drift), and the ability to operate under real-time streaming conditions. The relevance of the research is determined by the rapid growth of heterogeneous data volumes and the need to create intelligent systems capable of timely detecting anomalies in complex information environments, particularly in cybersecurity, network traffic analysis, IoT, and financial systems. The paper analyzes modern approaches to anomaly detection based on neural networks, including autoencoders, recurrent, transformer-based, and generative models, identifying their advantages, limitations, and challenges of integration into applied software systems. An architecture of the software system is proposed, implementing a modular approach and including subsystems for data collection, preprocessing, model training, anomaly detection, and model adaptation, with primary emphasis placed on integrating the system’s algorithmic and architectural components. A distinctive feature of the system is the use of an ensemble approach to form an integrated anomaly score, as well as mechanisms for adapting to data changes (concept drift) based on distribution analysis. Algorithms for streaming data processing with minimized latency and ensured scalability were applied. An experimental study was conducted on datasets of various types, which allowed evaluating the universality and effectiveness of the proposed approach. The obtained results confirm an increase in anomaly detection accuracy and system robustness to changes in data characteristics compared to the use of individual models. A distinctive feature of the work is the development of an integrated software system architecture that combines different types of neural networks, implements adaptation mechanisms, and forms a generalized anomaly indicator. The practical significance of the work lies in the possibility of applying the developed system in real information systems for monitoring, analysis, and decision support.