REVIEW OF APPROACHES FOR REAL-TIME PERFORMANCE GUARANTEES IN EDGE MACHINE LEARNING SYSTEMS

Abstract

This article presents a structured and comprehensive review of approaches aimed at ensuring real-time performance guarantees in edge machine learning systems. It highlights the ongoing shift of artificial intelligence workloads from centralized cloud infrastructures toward edge environments, where data is generated and processed locally. While this transition improves latency, bandwidth usage, and data privacy, it also introduces critical challenges due to the limited computational resources of edge devices.
The core contribution of the study is the development of a multi-layered taxonomy that organizes existing techniques into four key domains: model-level optimization, system-level scheduling, runtime adaptation, and hardware-software co-design. Each layer addresses different aspects of the real-time constraint problem. Model-level techniques focus on reducing computational complexity through methods such as quantization, pruning, and efficient architecture design. System-level strategies ensure timely execution through scheduling algorithms and resource allocation mechanisms. Runtime adaptation introduces dynamic adjustments to maintain performance under changing conditions, while hardware-software co-design aligns algorithms with the capabilities of underlying hardware.
The article emphasizes that no single technique is sufficient to guarantee real-time performance. Instead, it underlines the importance of integrating multiple approaches to balance trade-offs between accuracy, latency, and energy efficiency. This holistic perspective is particularly relevant for safety-critical applications, where missing execution deadlines can lead to system failure. Overall, the study provides a clear conceptual framework and valuable insights for designing robust, real-time edge AI systems.