EdgePipe: Tailoring Pipeline Parallelism with Deep Neural Networks for Volatile Wireless Edge Devices

Jin Yi Yoon, Yeongsin Byeon, Jeewoon Kim, Hyung June Lee

Research output: Contribution to journalArticlepeer-review

Abstract

As intelligence recently moves to the edge to tackle the problems of privacy, scalability, and network bandwidth in the centralized intelligence, it is necessary to construct an efficient yet robust deep learning model viable at edge devices, which are usually volatile in wireless links and device functionality. The intensive computation burden for deep learning at the edge side necessitates some level of parallel processing via acceleration. We propose EdgePipe, a deep learning framework based on deep neural networks (DNNs) with a mixture of model parallelism and pipeline training for high resource utilization over volatile wireless edge devices. To tackle the volatility problem in wireless links and device functionality, a concept of super neuron is defined to be a group of neurons across adjacent layers, which is the basis of model partitioning at edge devices. The relatively loss-resilient neuron structure prevents the entire forward or backward training paths from being totally broken down due to only some intermittent link or device failure caused by one or few devices. Further, we design a subsequent pipeline training mechanism based on the prior super neuron-based model partitioning for fast convergence with more training data in a fixed timeline. The experimental results have demonstrated that EdgePipe outperforms several counterpart algorithms including PipeDream under the volatile wireless lossy or device malfunctioning environments, while preserving the low interlayer communication overhead.

Original languageEnglish
JournalIEEE Internet of Things Journal
DOIs
StateAccepted/In press - 2021

Keywords

  • Computational modeling
  • Deep learning
  • Distributed Deep Learning
  • Edge Device
  • Model Parallelism
  • Neurons
  • Parallel processing
  • Pipeline Parallelism.
  • Pipelines
  • Training
  • Volatile Wireless Links
  • Wireless communication

Fingerprint

Dive into the research topics of 'EdgePipe: Tailoring Pipeline Parallelism with Deep Neural Networks for Volatile Wireless Edge Devices'. Together they form a unique fingerprint.

Cite this