Memory-based scheduling of scientific computing clusters

Details

Serval ID
serval:BIB_CBFEE4CA0C49
Type
Article: article from journal or magazin.
Collection
Publications
Institution
Title
Memory-based scheduling of scientific computing clusters
Journal
The Journal of Supercomputing
Author(s)
Niemi T., Hameri A.-P.
ISSN
0920-8542
Publication state
Published
Issued date
2012
Peer-reviewed
Oui
Volume
61
Number
3
Pages
520-544
Language
english
Abstract
This study looks at how increased memory utilisation affects throughput and energy consumption in scientific computing, especially in high-energy physics. Our aim is to minimise energy consumed by a set of jobs without increasing the processing time. The earlier tests indicated that, especially in data analysis, throughput can increase over 100% and energy consumption decrease 50% by processing multiple jobs in parallel per CPU core. Since jobs are heterogeneous, it is not possible to find an optimum value for the number of parallel jobs. A better solution is based on memory utilisation, but finding an optimum memory threshold is not straightforward. Therefore, a fuzzy logic-based algorithm was developed that can dynamically adapt the memory threshold based on the overall load. In this way, it is possible to keep memory consumption stable with different workloads while achieving significantly higher throughput and energy-efficiency than using a traditional fixed number of jobs or fixed memory threshold approaches.
Keywords
Scientific computing, Scheduling, Throughput optimisation, Energy-efficiency, Green computing, High energy physics
Web of science
Create date
06/11/2012 18:23
Last modification date
20/08/2019 16:46
Usage data