TrainWare: A memory optimized weight update architecture for on-device convolutional neural network training

Seungkyu Choi; Jaehyeong Sim; Myeonggu Kang; Lee Sup Kim

doi:10.1145/3218603.3218625

TrainWare: A memory optimized weight update architecture for on-device convolutional neural network training

Seungkyu Choi, Jaehyeong Sim, Myeonggu Kang, Lee Sup Kim

Department of Computer Science & Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

17 Scopus citations

Abstract

Training convolutional neural network on device has become essential where it allows applications to consider user's individual environment. Meanwhile, the weight update operation from the training process is the primary factor of high energy consumption due to its substantial memory accesses. We propose a dedicated weight update architecture with two key features: (1) a specialized local buffer for the DRAM access deduction (2) a novel dataflow and its suitable processing element array structure for weight gradient computation to optimize the energy consumed by internal memories. Our scheme achieves 14.3%-30.2% total energy reduction by drastically eliminating the memory accesses.

Original language	English
Title of host publication	ISLPED 2018 - Proceedings of the 2018 International Symposium on Low Power Electronics and Design
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Print)	9781450357043
DOIs	https://doi.org/10.1145/3218603.3218625
State	Published - 23 Jul 2018
Event	23rd IEEE/ACM International Symposium on Low Power Electronics and Design, ISLPED 2018 - Bellevue, United States Duration: 23 Jul 2018 → 25 Jul 2018

Publication series

Name	Proceedings of the International Symposium on Low Power Electronics and Design
ISSN (Print)	1533-4678

Conference

Conference	23rd IEEE/ACM International Symposium on Low Power Electronics and Design, ISLPED 2018
Country/Territory	United States
City	Bellevue
Period	23/07/18 → 25/07/18

Bibliographical note

Publisher Copyright:
© 2018 Association for Computing Machinery.

Keywords

CNN
Dataflow
On-device
Training architecture
Weight update

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1145/3218603.3218625

Cite this

Choi, S., Sim, J., Kang, M., & Kim, L. S. (2018). TrainWare: A memory optimized weight update architecture for on-device convolutional neural network training. In ISLPED 2018 - Proceedings of the 2018 International Symposium on Low Power Electronics and Design Article a19 (Proceedings of the International Symposium on Low Power Electronics and Design). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1145/3218603.3218625

Choi, Seungkyu ; Sim, Jaehyeong ; Kang, Myeonggu et al. / TrainWare : A memory optimized weight update architecture for on-device convolutional neural network training. ISLPED 2018 - Proceedings of the 2018 International Symposium on Low Power Electronics and Design. Institute of Electrical and Electronics Engineers Inc., 2018. (Proceedings of the International Symposium on Low Power Electronics and Design).

@inproceedings{2351aead1e1f4c409ee400da8174c4b6,

title = "TrainWare: A memory optimized weight update architecture for on-device convolutional neural network training",

abstract = "Training convolutional neural network on device has become essential where it allows applications to consider user's individual environment. Meanwhile, the weight update operation from the training process is the primary factor of high energy consumption due to its substantial memory accesses. We propose a dedicated weight update architecture with two key features: (1) a specialized local buffer for the DRAM access deduction (2) a novel dataflow and its suitable processing element array structure for weight gradient computation to optimize the energy consumed by internal memories. Our scheme achieves 14.3%-30.2% total energy reduction by drastically eliminating the memory accesses.",

keywords = "CNN, Dataflow, On-device, Training architecture, Weight update",

author = "Seungkyu Choi and Jaehyeong Sim and Myeonggu Kang and Kim, {Lee Sup}",

note = "Publisher Copyright: {\textcopyright} 2018 Association for Computing Machinery.; 23rd IEEE/ACM International Symposium on Low Power Electronics and Design, ISLPED 2018 ; Conference date: 23-07-2018 Through 25-07-2018",

year = "2018",

month = jul,

day = "23",

doi = "10.1145/3218603.3218625",

language = "English",

isbn = "9781450357043",

series = "Proceedings of the International Symposium on Low Power Electronics and Design",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "ISLPED 2018 - Proceedings of the 2018 International Symposium on Low Power Electronics and Design",

}

Choi, S, Sim, J, Kang, M & Kim, LS 2018, TrainWare: A memory optimized weight update architecture for on-device convolutional neural network training. in ISLPED 2018 - Proceedings of the 2018 International Symposium on Low Power Electronics and Design., a19, Proceedings of the International Symposium on Low Power Electronics and Design, Institute of Electrical and Electronics Engineers Inc., 23rd IEEE/ACM International Symposium on Low Power Electronics and Design, ISLPED 2018, Bellevue, United States, 23/07/18. https://doi.org/10.1145/3218603.3218625

TrainWare: A memory optimized weight update architecture for on-device convolutional neural network training. / Choi, Seungkyu; Sim, Jaehyeong; Kang, Myeonggu et al.
ISLPED 2018 - Proceedings of the 2018 International Symposium on Low Power Electronics and Design. Institute of Electrical and Electronics Engineers Inc., 2018. a19 (Proceedings of the International Symposium on Low Power Electronics and Design).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - TrainWare

T2 - 23rd IEEE/ACM International Symposium on Low Power Electronics and Design, ISLPED 2018

AU - Choi, Seungkyu

AU - Sim, Jaehyeong

AU - Kang, Myeonggu

AU - Kim, Lee Sup

PY - 2018/7/23

Y1 - 2018/7/23

N2 - Training convolutional neural network on device has become essential where it allows applications to consider user's individual environment. Meanwhile, the weight update operation from the training process is the primary factor of high energy consumption due to its substantial memory accesses. We propose a dedicated weight update architecture with two key features: (1) a specialized local buffer for the DRAM access deduction (2) a novel dataflow and its suitable processing element array structure for weight gradient computation to optimize the energy consumed by internal memories. Our scheme achieves 14.3%-30.2% total energy reduction by drastically eliminating the memory accesses.

AB - Training convolutional neural network on device has become essential where it allows applications to consider user's individual environment. Meanwhile, the weight update operation from the training process is the primary factor of high energy consumption due to its substantial memory accesses. We propose a dedicated weight update architecture with two key features: (1) a specialized local buffer for the DRAM access deduction (2) a novel dataflow and its suitable processing element array structure for weight gradient computation to optimize the energy consumed by internal memories. Our scheme achieves 14.3%-30.2% total energy reduction by drastically eliminating the memory accesses.

KW - CNN

KW - Dataflow

KW - On-device

KW - Training architecture

KW - Weight update

UR - http://www.scopus.com/inward/record.url?scp=85051489052&partnerID=8YFLogxK

U2 - 10.1145/3218603.3218625

DO - 10.1145/3218603.3218625

M3 - Conference contribution

AN - SCOPUS:85051489052

SN - 9781450357043

T3 - Proceedings of the International Symposium on Low Power Electronics and Design

BT - ISLPED 2018 - Proceedings of the 2018 International Symposium on Low Power Electronics and Design

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 23 July 2018 through 25 July 2018

ER -

Choi S, Sim J, Kang M, Kim LS. TrainWare: A memory optimized weight update architecture for on-device convolutional neural network training. In ISLPED 2018 - Proceedings of the 2018 International Symposium on Low Power Electronics and Design. Institute of Electrical and Electronics Engineers Inc. 2018. a19. (Proceedings of the International Symposium on Low Power Electronics and Design). doi: 10.1145/3218603.3218625

TrainWare: A memory optimized weight update architecture for on-device convolutional neural network training

Abstract

Publication series

Conference

Bibliographical note

Keywords

UN SDGs

Access to Document

Fingerprint

Cite this