Enhanced Scheduling of AI Applications in Multi-Tenant Cloud Using Genetic Optimizations

Seokmin Kwon, Hyokyung Bahn

Research output: Contribution to journalArticlepeer-review

Abstract

The artificial intelligence (AI) industry is increasingly integrating with diverse sectors such as smart logistics, FinTech, entertainment, and cloud computing. This expansion has led to the coexistence of heterogeneous applications within multi-tenant systems, presenting significant scheduling challenges. This paper addresses these challenges by exploring the scheduling of various machine learning workloads in large-scale, multi-tenant cloud systems that utilize heterogeneous GPUs. Traditional scheduling strategies often struggle to achieve satisfactory results due to low GPU utilization in these complex environments. To address this issue, we propose a novel scheduling approach that employs a genetic optimization technique, implemented within a process-oriented discrete-event simulation framework, to effectively orchestrate various machine learning tasks. We evaluate our approach using workload traces from Alibaba’s MLaaS cluster with over 6000 heterogeneous GPUs. The results show that our scheduling improves GPU utilization by 12.8% compared to Round-Robin scheduling, demonstrating the effectiveness of the solution in optimizing cloud-based GPU scheduling.

Original languageEnglish
Article number4697
JournalApplied Sciences (Switzerland)
Volume14
Issue number11
DOIs
StatePublished - Jun 2024

Bibliographical note

Publisher Copyright:
© 2024 by the authors.

Keywords

  • artificial intelligence
  • cloud
  • genetic algorithm
  • machine learning
  • task scheduling

Fingerprint

Dive into the research topics of 'Enhanced Scheduling of AI Applications in Multi-Tenant Cloud Using Genetic Optimizations'. Together they form a unique fingerprint.

Cite this