A METHOD FOR DETECTING MALICIOUS DROPPERS BASED ON GRAPH ATTENTION AND API CALL PATTERNS

Abstract

The paper presents an improved method for detecting malicious droppers in computer systems, which is based on the construction of directed graphs of API calls and their further analysis using Graph Attention Networks (GAT). The proposed approach is focused on modern types of droppers that use polymorphism, metamorphism, various obfuscation techniques, dynamic loading of components and conditional code execution, which complicate their detection by traditional means.

Unlike signature methods, which depend on the presence of known malware samples, and heuristic approaches, which often suffer from a high number of false positives, the GAT-based model allows for the analysis of structural and contextual dependencies between API calls. Thanks to the mechanisms of attention, the network is able to determine the most informative vertices and edges in the graph, which reflect the hidden patterns of droppers' behavior, regardless of modifications to their machine code.

In the work, a detailed experimental analysis is performed using a corpus of Windows PE files, which includes both legitimate and malicious samples of various families. The method compared to basic machine learning models such as Random Forest, SVM and gradient boosting. The obtained results demonstrate significant advantages of the GAT-based approach in both classification accuracy and resistance to complex obfuscation techniques, which confirms its effectiveness for practical application in security systems.