Abstract
The purpose of this work is to introduce Crescoware, a container-based gateway for HPC/AI applications in the ENEAGRID infrastructure. Crescoware is a web application created to facilitate the migration of CRESCO users’ workloads to Singularity containers, bringing all the necessary actions to work with containers within a visual environment capable of abstracting many complex command-line operations, typically executed over multiple terminal sessions. Among the features offered by Crescoware there are: a collaborative catalog of Singularity containers; a workspace for the creation of new containers, based on centralized build servers; and a web-based SSH client to test these containers. After reviewing the advantages offered by containers if compared to a traditional bare-metal approach, this work also focuses on how the problem of enabling users to create and launch Singularity containers has been dealt with, having in mind the goal of enabling a frictionless experience on ENEAGRID/CRESCO systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Huerta, E.A., et al.: Convergence of artificial intelligence and high performance computing on NSF-supported cyberinfrastructure. J. Big Data 7(1), 1–12 (2020). https://doi.org/10.1186/s40537-020-00361-2
Brayford, D., Vallecorsa, S., Atanasov, A., Baruffa, F., Riviera, W.: Deploying AI frameworks on secure HPC systems with containers. In: 2019 IEEE High Performance Extreme Computing Conference (HPEC), pp. 1–6 (2019)
Colonnelli, I., Cantalupo, B., Spampinato, C., Pennisi, M., Aldinucci, M.: Bringing AI pipelines onto cloud-HPC: setting a baseline for accuracy of COVID-19 diagnosis. ENEA CRESCO in the Fight Against COVID-19, pp. 66–73 (2021)
Iannone, F., et al.: CRESCO ENEA HPC clusters: a working example of a multifabric GPFS Spectrum Scale layout. In: 2019 International Conference on High Performance Computing & Simulation (HPCS), pp. 1051–1052. Dublin, Ireland (2019)
Bashari Rad, B., Bhatti, H., Ahmadi, M.: An introduction to Docker and analysis of its performance. Int. J. Comput. Sci. Network Secur. 173, 8 (2017)
Kurtzer, G.M., Sochat, V., Bauer, M.W.: Singularity: Scientific containers for mobility of compute. PLoS ONE 12(5), e0177459 (2017)
Mariano, A., et al.: Fast Access to Remote Objects 2.0 A renewed gateway to ENEAGRID distributed computing resources. Future Gener. Comput. Syst. 94(2019), 920–928 (2017)
OpenAFS homepage. http://www.openafs.org/. Accessed 25 Nov 2021
GPFS homepage. https://www.ibm.com/docs/en/spectrum-scale/4.2.0?topic=scale-overview-gpfs. Accessed 25 Nov 2021
Ceph homepage. https://ceph.com/en/. Accessed 25 Nov 2021
InfiniBand homepage. https://www.nvidia.com/en-us/networking/products/infiniband/. Accessed 25 Nov 2021
MediaWiki homepage. https://www.mediawiki.org/wiki/MediaWiki. Accessed 25 Nov 2021
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mariano, A., D’Amato, G., Formisano, G., Guarnieri, G., Santomauro, G., Migliori, S. (2022). Crescoware: A Container-Based Gateway for HPC and AI Applications in the ENEAGRID Infrastructure. In: Arai, K. (eds) Intelligent Computing. SAI 2022. Lecture Notes in Networks and Systems, vol 506. Springer, Cham. https://doi.org/10.1007/978-3-031-10461-9_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-10461-9_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-10460-2
Online ISBN: 978-3-031-10461-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)