Video summarization by learning relationships between action and scene

Jungin Park; Jiyoung Lee; Sangryul Jeon; Kwanghoon Sohn

doi:10.1109/ICCVW.2019.00193

Video summarization by learning relationships between action and scene

Jungin Park
, Jiyoung Lee
, Sangryul Jeon
, Kwanghoon Sohn

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

9 Scopus citations

Abstract

We propose a novel deep architecture for video summarization in untrimmed videos that simultaneously recognizes action and scene classes for every video segments. Our networks accomplish this through a multi-task fusion approach based on two types of attention modules to explore semantic correlations between action and scene in the videos. The proposed networks consist of the feature embedding networks and attention inference networks to stochastically leverage the inferred action and scene feature representations. Additionally, we design a new center loss function that learns the feature representations by enforcing to minimize the intra-class variations and to maximize the inter-class variations. Our model achieves a score of 0.8409 for summarization and accuracy of 0.7294 for action and scene recognition on test set of CoVieW'19 dataset, which is ranked 3rd.

Original language	English
Title of host publication	Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1545-1552
Number of pages	8
ISBN (Electronic)	9781728150239
DOIs	https://doi.org/10.1109/ICCVW.2019.00193
State	Published - Oct 2019
Event	17th IEEE/CVF International Conference on Computer Vision Workshop, ICCVW 2019 - Seoul, Korea, Republic of Duration: 27 Oct 2019 → 28 Oct 2019

Publication series

Name	Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019

Conference

Conference	17th IEEE/CVF International Conference on Computer Vision Workshop, ICCVW 2019
Country/Territory	Korea, Republic of
City	Seoul
Period	27/10/19 → 28/10/19

Bibliographical note

Publisher Copyright:
© 2019 IEEE.

Keywords

Action recognition
Deep learning
Multi task learning
Scene recognition
Video summarization
Video understanding

Access to Document

10.1109/ICCVW.2019.00193

Cite this

Park, J., Lee, J., Jeon, S., & Sohn, K. (2019). Video summarization by learning relationships between action and scene. In Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019 (pp. 1545-1552). Article 9022169 (Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCVW.2019.00193

Park, Jungin ; Lee, Jiyoung ; Jeon, Sangryul et al. / Video summarization by learning relationships between action and scene. Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 1545-1552 (Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019).

@inproceedings{32c0d8490db74807a980340c3592db93,

title = "Video summarization by learning relationships between action and scene",

abstract = "We propose a novel deep architecture for video summarization in untrimmed videos that simultaneously recognizes action and scene classes for every video segments. Our networks accomplish this through a multi-task fusion approach based on two types of attention modules to explore semantic correlations between action and scene in the videos. The proposed networks consist of the feature embedding networks and attention inference networks to stochastically leverage the inferred action and scene feature representations. Additionally, we design a new center loss function that learns the feature representations by enforcing to minimize the intra-class variations and to maximize the inter-class variations. Our model achieves a score of 0.8409 for summarization and accuracy of 0.7294 for action and scene recognition on test set of CoVieW'19 dataset, which is ranked 3rd.",

keywords = "Action recognition, Deep learning, Multi task learning, Scene recognition, Video summarization, Video understanding",

author = "Jungin Park and Jiyoung Lee and Sangryul Jeon and Kwanghoon Sohn",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 17th IEEE/CVF International Conference on Computer Vision Workshop, ICCVW 2019 ; Conference date: 27-10-2019 Through 28-10-2019",

year = "2019",

month = oct,

doi = "10.1109/ICCVW.2019.00193",

language = "English",

series = "Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1545--1552",

booktitle = "Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019",

}

Park, J, Lee, J, Jeon, S & Sohn, K 2019, Video summarization by learning relationships between action and scene. in Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019., 9022169, Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019, Institute of Electrical and Electronics Engineers Inc., pp. 1545-1552, 17th IEEE/CVF International Conference on Computer Vision Workshop, ICCVW 2019, Seoul, Korea, Republic of, 27/10/19. https://doi.org/10.1109/ICCVW.2019.00193

Video summarization by learning relationships between action and scene. / Park, Jungin; Lee, Jiyoung; Jeon, Sangryul et al.
Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 1545-1552 9022169 (Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Video summarization by learning relationships between action and scene

AU - Park, Jungin

AU - Lee, Jiyoung

AU - Jeon, Sangryul

AU - Sohn, Kwanghoon

PY - 2019/10

Y1 - 2019/10

N2 - We propose a novel deep architecture for video summarization in untrimmed videos that simultaneously recognizes action and scene classes for every video segments. Our networks accomplish this through a multi-task fusion approach based on two types of attention modules to explore semantic correlations between action and scene in the videos. The proposed networks consist of the feature embedding networks and attention inference networks to stochastically leverage the inferred action and scene feature representations. Additionally, we design a new center loss function that learns the feature representations by enforcing to minimize the intra-class variations and to maximize the inter-class variations. Our model achieves a score of 0.8409 for summarization and accuracy of 0.7294 for action and scene recognition on test set of CoVieW'19 dataset, which is ranked 3rd.

AB - We propose a novel deep architecture for video summarization in untrimmed videos that simultaneously recognizes action and scene classes for every video segments. Our networks accomplish this through a multi-task fusion approach based on two types of attention modules to explore semantic correlations between action and scene in the videos. The proposed networks consist of the feature embedding networks and attention inference networks to stochastically leverage the inferred action and scene feature representations. Additionally, we design a new center loss function that learns the feature representations by enforcing to minimize the intra-class variations and to maximize the inter-class variations. Our model achieves a score of 0.8409 for summarization and accuracy of 0.7294 for action and scene recognition on test set of CoVieW'19 dataset, which is ranked 3rd.

KW - Action recognition

KW - Deep learning

KW - Multi task learning

KW - Scene recognition

KW - Video summarization

KW - Video understanding

UR - https://www.scopus.com/pages/publications/85082457559

U2 - 10.1109/ICCVW.2019.00193

DO - 10.1109/ICCVW.2019.00193

M3 - Conference contribution

AN - SCOPUS:85082457559

T3 - Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019

SP - 1545

EP - 1552

BT - Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 17th IEEE/CVF International Conference on Computer Vision Workshop, ICCVW 2019

Y2 - 27 October 2019 through 28 October 2019

ER -

Park J, Lee J, Jeon S, Sohn K. Video summarization by learning relationships between action and scene. In Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 1545-1552. 9022169. (Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019). doi: 10.1109/ICCVW.2019.00193

Video summarization by learning relationships between action and scene

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Fingerprint

Cite this