Author(s)

Ivy Osei, Kwame Mensah^*

Affiliation(s)

University College London, Gower St, London WC1E 6BT, UK. 

Corresponding Author

Kwame Mensah

ABSTRACT

This paper presents a novel framework for intelligent anomaly detection in distributed systems, leveraging deep learning techniques to enhance the identification of anomalies in real-time. As organizations increasingly depend on distributed architectures—such as cloud computing, microservices, and peer-to-peer networks—ensuring the reliability and security of these systems becomes crucial. Anomalies, which signify deviations from expected behavior, can indicate serious issues ranging from system malfunctions to security breaches. Traditional anomaly detection methods often struggle in distributed environments due to their reliance on predefined thresholds and assumptions about data distributions, leading to high rates of false positives and negatives. This study explores the potential of deep learning models, including Convolutional Neural Networks, Recurrent Neural Networks, and Autoencoders, to address these challenges. The proposed framework encompasses data collection, preprocessing, model selection, training, evaluation, and deployment, facilitating systematic anomaly detection while enabling continuous learning. The results indicate that deep learning models significantly outperform traditional methods, demonstrating their ability to capture complex patterns in high-dimensional data. Furthermore, the findings suggest that advancements in deep learning and hybrid approaches could further enhance anomaly detection capabilities across various domains, including finance, healthcare, and cybersecurity.

This research contributes to the field by providing a comprehensive methodology for intelligent anomaly detection tailored to the unique challenges of distributed systems, paving the way for more resilient and secure computing environments.

KEYWORDS

Anomaly detection; Deep learning; Distributed systems

CITE THIS PAPER

Ivy Osei, Kwame Mensah. Intelligent anomaly detection in distributed systems via deep learning. World Journal of Information and Knowledge Management. 2024, 2(3): 36-43. DOI: https://doi.org/10.61784/wjikm3024.

REFERENCES

[1] Hassan M U, Rehmani M H, Chen J. Anomaly detection in blockchain networks: A comprehensive survey. IEEE Communications Surveys & Tutorials, 2022, 25(1): 289-318.

[2] Jafarian T, Masdari M, Ghaffari A, et al. A survey and classification of the security anomaly detection mechanisms in software defined networks. Cluster Computing, 2021, 24: 1235-1253.

[3] Zhang X, Li P, Han X, et al. Enhancing Time Series Product Demand Forecasting with Hybrid Attention-Based Deep Learning Models. IEEE Access, 2024.

[4] Wang X, Wu Y C, Ji X, et al. Algorithmic discrimination: examining its types and regulatory measures with emphasis on US legal practices. Frontiers in Artificial Intelligence, 2024, 7: 1320277.

[5] Wang X, Wu Y C, Zhou M, et al. Beyond surveillance: privacy, ethics, and regulations in face recognition technology. Frontiers in big data, 2024, 7: 1337465.

[6] Said A M, Yahyaoui A, Abdellatif T. Efficient anomaly detection for smart hospital IoT systems. Sensors, 2021, 21(4): 1026.

[7] Douiba M, Benkirane S, Guezzaz A, et al. An improved anomaly detection model for IoT security using decision tree and gradient boosting. The Journal of Supercomputing, 2023, 79(3): 3392-3411.

[8] Liu Y, Hu X, Chen S. Multi-Material 3D Printing and Computational Design in Pharmaceutical Tablet Manufacturing. Journal of Computer Science and Artificial Intelligence, 2024.

[9] ALMahadin G, Aoudni Y, Shabaz M, et al. VANET network traffic anomaly detection using GRU-based deep learning model. IEEE Transactions on Consumer Electronics, 2023, 70(1): 4548-4555.

[10] Qiu L. DEEP LEARNING APPROACHES FOR BUILDING ENERGY CONSUMPTION PREDICTION. Frontiers in Environmental Research, 2024, 2(3): 11-17.

[11] Krichen M. Anomalies detection through smartphone sensors: A review. IEEE Sensors Journal, 2021, 21(6): 7207-7217.

[12] Wang M. AI Technologies in Modern Taxation: Applications, Challenges, and Strategic Directions. International Journal of Finance and Investment, 2024, 1(1): 42-46.

[13] Das T K, Adepu S, Zhou J. Anomaly detection in industrial control systems using logical analysis of data. Computers & Security, 2020, 96: 101935.

[14] Jacob V, Song F, Stiegler A, et al. Exathlon: A benchmark for explainable anomaly detection over time series, 2020. arXiv preprint arXiv:2010.05073.

[15] Li P, Ren S, Zhang Q, et al. Think4SCND: Reinforcement Learning with Thinking Model for Dynamic Supply Chain Network Design. IEEE Access, 2024.

[16] Zhang X, Chen S, Shao Z, et al. Enhanced Lithographic Hotspot Detection via Multi-Task Deep Learning with Synthetic Pattern Generation. IEEE Open Journal of the Computer Society, 2024.

[17] Novaes M P, Carvalho L F, Lloret J, et al. Long short-term memory and fuzzy logic for anomaly detection and mitigation in software-defined network environment. IEEE Access, 2020, 8: 83765-83781.

[18] Choi K, Yi J, Park C, et al. Deep learning for anomaly detection in time-series data: Review, analysis, and guidelines. IEEE access, 2021, 9: 120043-120065.

[19] Kardani-Moghaddam S, Buyya R, Ramamohanarao K. ADRL: A hybrid anomaly-aware deep reinforcement learning-based resource scaling in clouds. IEEE Transactions on Parallel and Distributed Systems, 2020, 32(3): 514-526.

[20] Baccari S, Hadded M, Ghazzai H, et al. Anomaly Detection in Connected and Autonomous Vehicles: A Survey, Analysis, and Research Challenges. IEEE Access, 2024.

[21] Liu Y, Ren S, Wang X, et al. Temporal Logical Attention Network for Log-Based Anomaly Detection in Distributed Systems. Sensors, 2024, 24(24): 7949.

[22] Ullah I, Mahmoud Q H. Design and development of RNN anomaly detection model for IoT networks. IEEE Access, 2022, 10: 62722-62750.

[23] Khaledian E, Pandey S, Kundu P, et al. Real-time synchrophasor data anomaly detection and classification using isolation forest, kmeans, and loop. IEEE Transactions on Smart Grid, 2020, 12(3): 2378-2388.

[24] Zamanzadeh Darban Z, Webb G I, Pan S, et al. Deep learning for time series anomaly detection: A survey. ACM Computing Surveys, 2024, 57(1): 1-42.