EdgePipe: Tailoring Pipeline Parallelism with Deep Neural Networks for Volatile Wireless Edge Devices

Jinyi Yoon, Yeongsin Byeon, Jeewoon Kim, Hyungjune Lee

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

As intelligence recently moves to the edge to tackle the problems of privacy, scalability, and network bandwidth in the centralized intelligence, it is necessary to construct an efficient yet robust deep learning model viable at edge devices, which are usually volatile in wireless links and device functionality. The intensive computation burden for deep learning at the edge side necessitates some level of parallel processing via acceleration. We propose EdgePipe, a deep learning framework based on deep neural networks (DNNs) with a mixture of model parallelism and pipeline training for high resource utilization over volatile wireless edge devices. To tackle the volatility problem in wireless links and device functionality, a concept of super neuron is defined to be a group of neurons across adjacent layers, which is the basis of model partitioning at edge devices. The relatively loss-resilient neuron structure prevents the entire forward or backward training paths from being totally broken down due to only some intermittent link or device failure caused by one or few devices. Furthermore, we design a subsequent pipeline training mechanism based on the prior super-neuron-based model partitioning for fast convergence with more training data in a fixed timeline. The experimental results have demonstrated that EdgePipe outperforms several counterpart algorithms including PipeDream under the volatile wireless lossy or device malfunctioning environments, while preserving the low interlayer communication overhead.

Original languageEnglish
Pages (from-to)11633-11647
Number of pages15
JournalIEEE Internet of Things Journal
Volume9
Issue number14
DOIs
StatePublished - 15 Jul 2022

Bibliographical note

Publisher Copyright:
© 2014 IEEE.

Keywords

  • Distributed deep learning
  • edge device
  • model parallelism
  • pipeline parallelism
  • volatile wireless links

Fingerprint

Dive into the research topics of 'EdgePipe: Tailoring Pipeline Parallelism with Deep Neural Networks for Volatile Wireless Edge Devices'. Together they form a unique fingerprint.

Cite this