Sunkyung Kim, Hyesong Choi, Dongbo Min

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


In multi-task learning (MTL) for visual scene understanding, it is crucial to transfer useful information between multiple tasks with minimal interferences. In this paper, we propose a novel architecture that effectively transfers informative features by applying the attention mechanism to the multi-scale features of the tasks. Since applying the attention module directly to all possible features in terms of scale and task requires a high complexity, we propose to apply the attention module sequentially for the task and scale. The cross-task attention module (CTAM) is first applied to facilitate the exchange of relevant information between the multiple task features of the same scale. The cross-scale attention module (CSAM) then aggregates useful information from feature maps at different resolutions in the same task. Also, we attempt to capture long range dependencies through the self-attention module in the feature extraction network. Extensive experiments demonstrate that our method achieves state-of-the-art performance on the NYUD-v2 and PASCAL-Context dataset. Our code is available at

Original languageEnglish
Title of host publication2022 IEEE International Conference on Image Processing, ICIP 2022 - Proceedings
PublisherIEEE Computer Society
Number of pages5
ISBN (Electronic)9781665496209
StatePublished - 2022
Event29th IEEE International Conference on Image Processing, ICIP 2022 - Bordeaux, France
Duration: 16 Oct 202219 Oct 2022

Publication series

NameProceedings - International Conference on Image Processing, ICIP
ISSN (Print)1522-4880


Conference29th IEEE International Conference on Image Processing, ICIP 2022

Bibliographical note

Funding Information:
* Corresponding author This work was supported by the Mid-Career Researcher Program through the NRF of Korea (NRF-2021R1A2C2011624). Sunkyung Kim is grateful for financial support from Hyundai Motor Chung Mong-Koo Foundation.

Publisher Copyright:
© 2022 IEEE.


  • cross attention
  • monocular depth estimation
  • Multi-task learning
  • self-attention
  • semantic segmentation


Dive into the research topics of 'SEQUENTIAL CROSS ATTENTION BASED MULTI-TASK LEARNING'. Together they form a unique fingerprint.

Cite this