Memory-based scheduling of scientific computing clusters

Détails

ID Serval
serval:BIB_CBFEE4CA0C49
Type
Article: article d'un périodique ou d'un magazine.
Collection
Publications
Institution
Titre
Memory-based scheduling of scientific computing clusters
Périodique
The Journal of Supercomputing
Auteur⸱e⸱s
Niemi T., Hameri A.-P.
ISSN
0920-8542
Statut éditorial
Publié
Date de publication
2012
Peer-reviewed
Oui
Volume
61
Numéro
3
Pages
520-544
Langue
anglais
Résumé
This study looks at how increased memory utilisation affects throughput and energy consumption in scientific computing, especially in high-energy physics. Our aim is to minimise energy consumed by a set of jobs without increasing the processing time. The earlier tests indicated that, especially in data analysis, throughput can increase over 100% and energy consumption decrease 50% by processing multiple jobs in parallel per CPU core. Since jobs are heterogeneous, it is not possible to find an optimum value for the number of parallel jobs. A better solution is based on memory utilisation, but finding an optimum memory threshold is not straightforward. Therefore, a fuzzy logic-based algorithm was developed that can dynamically adapt the memory threshold based on the overall load. In this way, it is possible to keep memory consumption stable with different workloads while achieving significantly higher throughput and energy-efficiency than using a traditional fixed number of jobs or fixed memory threshold approaches.
Mots-clé
Scientific computing, Scheduling, Throughput optimisation, Energy-efficiency, Green computing, High energy physics
Web of science
Création de la notice
06/11/2012 18:23
Dernière modification de la notice
20/08/2019 16:46
Données d'usage