 |
A. M. Sokolov, A. A. Larionov, V. M. Vishnevsky, A. A. Mukhtarov Architecture of a Distributed Computing System with Tasks Containerization and Prioritization |
 |
|
Abstract.
This article describes an architecture of a distributed system, which can speed up the process of obtaining results for such tasks. The system comprises a backend server, control service (supervisor), a set of worker nodes and a database. To abstract from particular languages and tools required for computational algorithms, these algorithms are executed in Docker containers. The system supports several strategies for tasks prioritization to operate efficiently under heavy load introduced by multiple users. To make use of the system, the user only needs to build a Docker image with an encapsulated algorithm, describe the input dataset in a JSON file and upload them via web interface. The system can be deployed in any public cloud. In this article, we provide a detailed description of the system architecture and numerical results obtained from computations on various clouds and local platforms. We show the influence of different prioritization strategies on the duration of computations under a moderate workload.
Keywords:
parallel computing; container virtualization; cloud computing.
PP. 5-18.
DOI 10.14357/20718632230401
EDN HSOYNI References
1. Vishnevsky V. et al. Performance evaluation of the priority multi-server system mmap/ph/m/n using machine learning methods // Mathematics. 2021. Vol. 9, no. 24. 2. Arora R., Redondo C., Joshua G. Scalable software infrastructure for integrating supercomputing with volunteer computing and cloud computing // Communications in Computer and Information Science. 2019. Pp. 105–119. 3. Nouman Durrani M., Shamsi J. A. Volunteer computing: Requirements, challenges, and solutions // J. Netw. Comput. Appl. 2014. Vol. 39, no. 1. Pp. 369–380. 4. Mengistu T. M., Che D. Survey and taxonomy of volunteer computing // ACM Comput. Surv. 2019. Vol. 52, no 3. 5. Anderson D. P. BOINC: A Platform for Volunteer Computing // J. Grid Comput. 2020. Vol. 18, no 1. Pp. 99–122. 6. Nikitina N. et al. Volunteer Computing Project Si-Dock@home for Virtual Drug Screening Against SARSCoV-2 // IFIP Advances in Information and Communication Technology. 2021. Pp. 23–34. 7. Barranco J. et al. LHC@Home: A BOINC-based volunteer computing infrastructure for physics studies at CERN // Open Eng. 2017. Vol. 7, no. 1. Pp. 379–393. 8. Brundo R. and Nicola R. De. Blockchain-based decentralized cloud/fog solutions: Challenges, opportunities, and standards // IEEE Commun. Stand. Mag. 2018. Vol. 2, no 3. Pp. 22–28. 9. Antelmi A. et al. A Volunteer Computing Architecture for Computational Workflows on Decentralized Web // IEEE Access. 2022. Vol. 10. Pp. 98993–99010. 10. Agliamzanov R., Sit M. and Demir I. Hydrology@Home: A distributed volunteer computing framework for hydrological research and applications // J. Hydroinformatics. 2020. Vol. 22, no 2. Pp. 235–248. 11. Sukhoroslov O. and Putilina E. Cloud Services for Automation of Scientific and Engineering Computations Science // Business. Soc. 2018. Vol. 1, no 2. Pp. 6–9. 12. Abramson D., Giddy J. and Kotler L. High performance parametric modeling with Nimrod/G: Killer application for the global grid? // Proceedings of the International Parallel Processing Symposium, IPPS. 2000. Pp. 520–528. 13. Korenkov V. et al. The JINR distributed computing environment // EPJ Web of Conferences. 2019. Pp 03009. 14. Lamanna M. The LHC computing grid project at CERN // Nucl. Instruments Methods Phys. Res. Sect. A Accel. Spectrometers, Detect. Assoc. Equip. 2004. Vol. 534, no 1–2. Pp. 1–6.
|
