Graph Regularization Network with Semantic Affinity for Weakly-Supervised Temporal Action Localization

Jungin Park; Jiyoung Lee; Sangryul Jeon; Seungryong Kim; Kwanghoon Sohn

doi:10.1109/ICIP.2019.8803589

Graph Regularization Network with Semantic Affinity for Weakly-Supervised Temporal Action Localization

Jungin Park
, Jiyoung Lee
, Sangryul Jeon
, Seungryong Kim
, Kwanghoon Sohn

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

6 Scopus citations

Abstract

This paper presents a novel deep architecture for weakly-supervised temporal action localization that not only generates segment-level action responses but also propagates segment-level responses to the neighborhood in a form of graph Laplacian regularization. Specifically, our approach consists of two sub-modules; a class activation module to estimate the action score map over time through the action classifiers, and a graph regularization module to refine the estimated action score map by solving a quadratic programming problem with the predicted segment-level semantic affinities. Since these two modules are integrated with fully differentiable layers, the proposed networks can be jointly trained in an end-to-end manner. Experimental results on Thumos14 and ActivityNet1.2 demonstrate that the proposed method provides outstanding performances in weakly-supervised temporal action localization.

Original language	English
Title of host publication	2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings
Publisher	IEEE Computer Society
Pages	3701-3705
Number of pages	5
ISBN (Electronic)	9781538662496
DOIs	https://doi.org/10.1109/ICIP.2019.8803589
State	Published - Sep 2019
Event	26th IEEE International Conference on Image Processing, ICIP 2019 - Taipei, Taiwan, Province of China Duration: 22 Sep 2019 → 25 Sep 2019

Publication series

Name	Proceedings - International Conference on Image Processing, ICIP
Volume	2019-September
ISSN (Print)	1522-4880

Conference

Conference	26th IEEE International Conference on Image Processing, ICIP 2019
Country/Territory	Taiwan, Province of China
City	Taipei
Period	22/09/19 → 25/09/19

Bibliographical note

Publisher Copyright:
© 2019 IEEE.

Keywords

graph Laplacian regularization
semantic affinity
weakly-supervised temporal action localization

Access to Document

10.1109/ICIP.2019.8803589

Cite this

Park, J., Lee, J., Jeon, S., Kim, S., & Sohn, K. (2019). Graph Regularization Network with Semantic Affinity for Weakly-Supervised Temporal Action Localization. In 2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings (pp. 3701-3705). Article 8803589 (Proceedings - International Conference on Image Processing, ICIP; Vol. 2019-September). IEEE Computer Society. https://doi.org/10.1109/ICIP.2019.8803589

@inproceedings{0c7cb8f2563442fcb630bfb0e1d7d3c0,

title = "Graph Regularization Network with Semantic Affinity for Weakly-Supervised Temporal Action Localization",

abstract = "This paper presents a novel deep architecture for weakly-supervised temporal action localization that not only generates segment-level action responses but also propagates segment-level responses to the neighborhood in a form of graph Laplacian regularization. Specifically, our approach consists of two sub-modules; a class activation module to estimate the action score map over time through the action classifiers, and a graph regularization module to refine the estimated action score map by solving a quadratic programming problem with the predicted segment-level semantic affinities. Since these two modules are integrated with fully differentiable layers, the proposed networks can be jointly trained in an end-to-end manner. Experimental results on Thumos14 and ActivityNet1.2 demonstrate that the proposed method provides outstanding performances in weakly-supervised temporal action localization.",

keywords = "graph Laplacian regularization, semantic affinity, weakly-supervised temporal action localization",

author = "Jungin Park and Jiyoung Lee and Sangryul Jeon and Seungryong Kim and Kwanghoon Sohn",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 26th IEEE International Conference on Image Processing, ICIP 2019 ; Conference date: 22-09-2019 Through 25-09-2019",

year = "2019",

month = sep,

doi = "10.1109/ICIP.2019.8803589",

language = "English",

series = "Proceedings - International Conference on Image Processing, ICIP",

publisher = "IEEE Computer Society",

pages = "3701--3705",

booktitle = "2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings",

}

Park, J, Lee, J, Jeon, S, Kim, S & Sohn, K 2019, Graph Regularization Network with Semantic Affinity for Weakly-Supervised Temporal Action Localization. in 2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings., 8803589, Proceedings - International Conference on Image Processing, ICIP, vol. 2019-September, IEEE Computer Society, pp. 3701-3705, 26th IEEE International Conference on Image Processing, ICIP 2019, Taipei, Taiwan, Province of China, 22/09/19. https://doi.org/10.1109/ICIP.2019.8803589

Graph Regularization Network with Semantic Affinity for Weakly-Supervised Temporal Action Localization. / Park, Jungin; Lee, Jiyoung; Jeon, Sangryul et al.
2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings. IEEE Computer Society, 2019. p. 3701-3705 8803589 (Proceedings - International Conference on Image Processing, ICIP; Vol. 2019-September).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Graph Regularization Network with Semantic Affinity for Weakly-Supervised Temporal Action Localization

AU - Park, Jungin

AU - Lee, Jiyoung

AU - Jeon, Sangryul

AU - Kim, Seungryong

AU - Sohn, Kwanghoon

PY - 2019/9

Y1 - 2019/9

N2 - This paper presents a novel deep architecture for weakly-supervised temporal action localization that not only generates segment-level action responses but also propagates segment-level responses to the neighborhood in a form of graph Laplacian regularization. Specifically, our approach consists of two sub-modules; a class activation module to estimate the action score map over time through the action classifiers, and a graph regularization module to refine the estimated action score map by solving a quadratic programming problem with the predicted segment-level semantic affinities. Since these two modules are integrated with fully differentiable layers, the proposed networks can be jointly trained in an end-to-end manner. Experimental results on Thumos14 and ActivityNet1.2 demonstrate that the proposed method provides outstanding performances in weakly-supervised temporal action localization.

AB - This paper presents a novel deep architecture for weakly-supervised temporal action localization that not only generates segment-level action responses but also propagates segment-level responses to the neighborhood in a form of graph Laplacian regularization. Specifically, our approach consists of two sub-modules; a class activation module to estimate the action score map over time through the action classifiers, and a graph regularization module to refine the estimated action score map by solving a quadratic programming problem with the predicted segment-level semantic affinities. Since these two modules are integrated with fully differentiable layers, the proposed networks can be jointly trained in an end-to-end manner. Experimental results on Thumos14 and ActivityNet1.2 demonstrate that the proposed method provides outstanding performances in weakly-supervised temporal action localization.

KW - graph Laplacian regularization

KW - semantic affinity

KW - weakly-supervised temporal action localization

UR - https://www.scopus.com/pages/publications/85076823912

U2 - 10.1109/ICIP.2019.8803589

DO - 10.1109/ICIP.2019.8803589

M3 - Conference contribution

AN - SCOPUS:85076823912

T3 - Proceedings - International Conference on Image Processing, ICIP

SP - 3701

EP - 3705

BT - 2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings

PB - IEEE Computer Society

T2 - 26th IEEE International Conference on Image Processing, ICIP 2019

Y2 - 22 September 2019 through 25 September 2019

ER -

Park J, Lee J, Jeon S, Kim S, Sohn K. Graph Regularization Network with Semantic Affinity for Weakly-Supervised Temporal Action Localization. In 2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings. IEEE Computer Society. 2019. p. 3701-3705. 8803589. (Proceedings - International Conference on Image Processing, ICIP). doi: 10.1109/ICIP.2019.8803589

Graph Regularization Network with Semantic Affinity for Weakly-Supervised Temporal Action Localization

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Fingerprint

Cite this