Incremental PARAFAC decomposition for three-dimensional tensors using apache spark

Hye Kyung Yang, Hwan Seung Yong

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

Recent studies have focused on the use of tensor analysis for tensor decomposition because this method can identify more latent factor and patterns, compared to the matrix factorization approach. The existing tensor decomposition studies used static dataset in their analyses. However, in practice, data change and increase over time. Therefore, this paper proposes an incremental Parallel Factor Analysis (PARAFAC) tensor decomposition algorithm for three-dimensional tensors. The method of incremental tensor decomposition can reduce recalculation costs associated with the addition of new tensors. The proposed method is called InParTen; it performs distributed incremental PARAFAC tensor decomposition based on the Apache Spark framework. The proposed method decomposes only new tensors and then combines them with existing results without recalculating the complete tensors. In this study, it was assumed that the tensors grow with time as the majority of the dataset is added over a period. In this paper, the performance of InParTen was evaluated by comparing the obtained results for execution time and relative errors against existing tensor decomposition tools. Consequently, it has been observed that the method can reduce the recalculation cost of tensor decomposition.

Original languageEnglish
Title of host publicationWeb Engineering - 19th International Conference, ICWE 2019, Proceedings
EditorsIn-Young Ko, Maxim Bakaev, Flavius Frasincar
PublisherSpringer Verlag
Pages63-71
Number of pages9
ISBN (Print)9783030192730
DOIs
StatePublished - 2019
Event19th International Conference on Web Engineering, ICWE 2019 - Daejeon, Korea, Republic of
Duration: 11 Jun 201914 Jun 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11496 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference19th International Conference on Web Engineering, ICWE 2019
Country/TerritoryKorea, Republic of
CityDaejeon
Period11/06/1914/06/19

Keywords

  • Apache spark
  • Incremental tensor decomposition
  • PARAFAC decomposition

Fingerprint

Dive into the research topics of 'Incremental PARAFAC decomposition for three-dimensional tensors using apache spark'. Together they form a unique fingerprint.

Cite this