Distributed PARAFAC Decomposition Method Based on In-memory Big Data System

Hye Kyung Yang, Hwan Seung Yong

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

We propose IM-PARAFAC, a PARAFAC tensor decomposition method that enables rapid processing of large scalable tensors in Apache Spark for distributed in-memory big data management systems. We consider the memory overflow that occurs when processing large amounts of data because of running on in-memory. Therefore, the proposed method, IM-PARAFAC, is capable of dividing and decomposing large input tensors. It can handle large tensors even in small, distributed environments. The experimental results indicate that the proposed IM-PARAFAC enables handling of large tensors and reduces the execution time.

Original languageEnglish
Title of host publicationDatabase Systems for Advanced Applications - DASFAA 2019 International Workshops
Subtitle of host publicationBDMS, BDQM, and GDMA, Proceedings
EditorsGuoliang Li, Yongxin Tong, Juggapong Natwichai, Joao Gama, Jun Yang
PublisherSpringer Verlag
Pages292-295
Number of pages4
ISBN (Print)9783030185893
DOIs
StatePublished - 2019
Event24th International Conference on Database Systems for Advanced Applications, DASFAA 2019 - Chiang Mai, Thailand
Duration: 22 Apr 201925 Apr 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11448 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference24th International Conference on Database Systems for Advanced Applications, DASFAA 2019
Country/TerritoryThailand
CityChiang Mai
Period22/04/1925/04/19

Bibliographical note

Funding Information:
Acknowledgement. This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2016R1D1A1B03931529).

Publisher Copyright:
© 2019, Springer Nature Switzerland AG.

Keywords

  • Apache Spark
  • Distributed PARAFAC decomposition

Fingerprint

Dive into the research topics of 'Distributed PARAFAC Decomposition Method Based on In-memory Big Data System'. Together they form a unique fingerprint.

Cite this