Distributed PARAFAC Decomposition Method Based on In-memory Big Data System

Hye Kyung Yang; Hwan Seung Yong

doi:10.1007/978-3-030-18590-9_31

Distributed PARAFAC Decomposition Method Based on In-memory Big Data System

Hye Kyung Yang, Hwan Seung Yong

Department of Computer Science & Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

We propose IM-PARAFAC, a PARAFAC tensor decomposition method that enables rapid processing of large scalable tensors in Apache Spark for distributed in-memory big data management systems. We consider the memory overflow that occurs when processing large amounts of data because of running on in-memory. Therefore, the proposed method, IM-PARAFAC, is capable of dividing and decomposing large input tensors. It can handle large tensors even in small, distributed environments. The experimental results indicate that the proposed IM-PARAFAC enables handling of large tensors and reduces the execution time.

Original language	English
Title of host publication	Database Systems for Advanced Applications - DASFAA 2019 International Workshops
Subtitle of host publication	BDMS, BDQM, and GDMA, Proceedings
Editors	Guoliang Li, Yongxin Tong, Juggapong Natwichai, Joao Gama, Jun Yang
Publisher	Springer Verlag
Pages	292-295
Number of pages	4
ISBN (Print)	9783030185893
DOIs	https://doi.org/10.1007/978-3-030-18590-9_31
State	Published - 2019
Event	24th International Conference on Database Systems for Advanced Applications, DASFAA 2019 - Chiang Mai, Thailand Duration: 22 Apr 2019 → 25 Apr 2019

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11448 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th International Conference on Database Systems for Advanced Applications, DASFAA 2019
Country/Territory	Thailand
City	Chiang Mai
Period	22/04/19 → 25/04/19

Bibliographical note

Publisher Copyright:
© 2019, Springer Nature Switzerland AG.

Keywords

Apache Spark
Distributed PARAFAC decomposition

Access to Document

10.1007/978-3-030-18590-9_31

Cite this

Yang, H. K., & Yong, H. S. (2019). Distributed PARAFAC Decomposition Method Based on In-memory Big Data System. In G. Li, Y. Tong, J. Natwichai, J. Gama, & J. Yang (Eds.), Database Systems for Advanced Applications - DASFAA 2019 International Workshops: BDMS, BDQM, and GDMA, Proceedings (pp. 292-295). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11448 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-18590-9_31

Yang, Hye Kyung ; Yong, Hwan Seung. / Distributed PARAFAC Decomposition Method Based on In-memory Big Data System. Database Systems for Advanced Applications - DASFAA 2019 International Workshops: BDMS, BDQM, and GDMA, Proceedings. editor / Guoliang Li ; Yongxin Tong ; Juggapong Natwichai ; Joao Gama ; Jun Yang. Springer Verlag, 2019. pp. 292-295 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{1baf4d42ecf54e688df5002aeefa3aa6,

title = "Distributed PARAFAC Decomposition Method Based on In-memory Big Data System",

abstract = "We propose IM-PARAFAC, a PARAFAC tensor decomposition method that enables rapid processing of large scalable tensors in Apache Spark for distributed in-memory big data management systems. We consider the memory overflow that occurs when processing large amounts of data because of running on in-memory. Therefore, the proposed method, IM-PARAFAC, is capable of dividing and decomposing large input tensors. It can handle large tensors even in small, distributed environments. The experimental results indicate that the proposed IM-PARAFAC enables handling of large tensors and reduces the execution time.",

keywords = "Apache Spark, Distributed PARAFAC decomposition",

author = "Yang, {Hye Kyung} and Yong, {Hwan Seung}",

note = "Publisher Copyright: {\textcopyright} 2019, Springer Nature Switzerland AG.; 24th International Conference on Database Systems for Advanced Applications, DASFAA 2019 ; Conference date: 22-04-2019 Through 25-04-2019",

year = "2019",

doi = "10.1007/978-3-030-18590-9_31",

language = "English",

isbn = "9783030185893",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "292--295",

editor = "Guoliang Li and Yongxin Tong and Juggapong Natwichai and Joao Gama and Jun Yang",

booktitle = "Database Systems for Advanced Applications - DASFAA 2019 International Workshops",

}

Yang, HK & Yong, HS 2019, Distributed PARAFAC Decomposition Method Based on In-memory Big Data System. in G Li, Y Tong, J Natwichai, J Gama & J Yang (eds), Database Systems for Advanced Applications - DASFAA 2019 International Workshops: BDMS, BDQM, and GDMA, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11448 LNCS, Springer Verlag, pp. 292-295, 24th International Conference on Database Systems for Advanced Applications, DASFAA 2019, Chiang Mai, Thailand, 22/04/19. https://doi.org/10.1007/978-3-030-18590-9_31

Distributed PARAFAC Decomposition Method Based on In-memory Big Data System. / Yang, Hye Kyung; Yong, Hwan Seung.
Database Systems for Advanced Applications - DASFAA 2019 International Workshops: BDMS, BDQM, and GDMA, Proceedings. ed. / Guoliang Li; Yongxin Tong; Juggapong Natwichai; Joao Gama; Jun Yang. Springer Verlag, 2019. p. 292-295 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11448 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Distributed PARAFAC Decomposition Method Based on In-memory Big Data System

AU - Yang, Hye Kyung

AU - Yong, Hwan Seung

PY - 2019

Y1 - 2019

N2 - We propose IM-PARAFAC, a PARAFAC tensor decomposition method that enables rapid processing of large scalable tensors in Apache Spark for distributed in-memory big data management systems. We consider the memory overflow that occurs when processing large amounts of data because of running on in-memory. Therefore, the proposed method, IM-PARAFAC, is capable of dividing and decomposing large input tensors. It can handle large tensors even in small, distributed environments. The experimental results indicate that the proposed IM-PARAFAC enables handling of large tensors and reduces the execution time.

AB - We propose IM-PARAFAC, a PARAFAC tensor decomposition method that enables rapid processing of large scalable tensors in Apache Spark for distributed in-memory big data management systems. We consider the memory overflow that occurs when processing large amounts of data because of running on in-memory. Therefore, the proposed method, IM-PARAFAC, is capable of dividing and decomposing large input tensors. It can handle large tensors even in small, distributed environments. The experimental results indicate that the proposed IM-PARAFAC enables handling of large tensors and reduces the execution time.

KW - Apache Spark

KW - Distributed PARAFAC decomposition

UR - http://www.scopus.com/inward/record.url?scp=85065442131&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-18590-9_31

DO - 10.1007/978-3-030-18590-9_31

M3 - Conference contribution

AN - SCOPUS:85065442131

SN - 9783030185893

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 292

EP - 295

BT - Database Systems for Advanced Applications - DASFAA 2019 International Workshops

A2 - Li, Guoliang

A2 - Tong, Yongxin

A2 - Natwichai, Juggapong

A2 - Gama, Joao

A2 - Yang, Jun

PB - Springer Verlag

T2 - 24th International Conference on Database Systems for Advanced Applications, DASFAA 2019

Y2 - 22 April 2019 through 25 April 2019

ER -

Yang HK, Yong HS. Distributed PARAFAC Decomposition Method Based on In-memory Big Data System. In Li G, Tong Y, Natwichai J, Gama J, Yang J, editors, Database Systems for Advanced Applications - DASFAA 2019 International Workshops: BDMS, BDQM, and GDMA, Proceedings. Springer Verlag. 2019. p. 292-295. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-18590-9_31

Distributed PARAFAC Decomposition Method Based on In-memory Big Data System

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Fingerprint

Cite this