TY - GEN
T1 - Pin the Memory
T2 - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
AU - Kim, Jin
AU - Lee, Jiyoung
AU - Park, Jungin
AU - Min, Dongbo
AU - Sohn, Kwanghoon
N1 - Funding Information:
This research was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (NRF2021R1A2C2006703), the Yonsei University Research Fund of 2021 (2021-22-0001), and the Mid-Career Researcher Program through the NRF of Korea (NRF-2021R1A2C2011624).
Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - The rise of deep neural networks has led to several break-throughs for semantic segmentation. In spite of this, a model trained on source domain often fails to work properly in new challenging domains, that is directly concerned with the generalization capability of the model. In this paper, we present a novel memory-guided domain generalization method for semantic segmentation based on meta-learning framework. Especially, our method abstracts the conceptual knowledge of semantic classes into categorical memory which is constant beyond the domains. Upon the meta-learning concept, we repeatedly train memory-guided networks and simulate virtual test to 1) learn how to memorize a domain-agnostic and distinct information of classes and 2) offer an externally settled memory as a class-guidance to reduce the ambiguity of representation in the test data of arbitrary unseen domain. To this end, we also propose memory divergence and feature cohesion losses, which encourage to learn memory reading and update processes for category-aware domain generalization. Extensive experiments for semantic segmentation demonstrate the superior generalization capability of our method over state-of-the-art works on various benchmarks.11https://github.com/Genie-Kim/PintheMemory
AB - The rise of deep neural networks has led to several break-throughs for semantic segmentation. In spite of this, a model trained on source domain often fails to work properly in new challenging domains, that is directly concerned with the generalization capability of the model. In this paper, we present a novel memory-guided domain generalization method for semantic segmentation based on meta-learning framework. Especially, our method abstracts the conceptual knowledge of semantic classes into categorical memory which is constant beyond the domains. Upon the meta-learning concept, we repeatedly train memory-guided networks and simulate virtual test to 1) learn how to memorize a domain-agnostic and distinct information of classes and 2) offer an externally settled memory as a class-guidance to reduce the ambiguity of representation in the test data of arbitrary unseen domain. To this end, we also propose memory divergence and feature cohesion losses, which encourage to learn memory reading and update processes for category-aware domain generalization. Extensive experiments for semantic segmentation demonstrate the superior generalization capability of our method over state-of-the-art works on various benchmarks.11https://github.com/Genie-Kim/PintheMemory
KW - grouping and shape analysis
KW - Scene analysis and understanding
KW - Segmentation
KW - Self-& semi-& meta- & unsupervised learning
UR - http://www.scopus.com/inward/record.url?scp=85136194322&partnerID=8YFLogxK
U2 - 10.1109/CVPR52688.2022.00431
DO - 10.1109/CVPR52688.2022.00431
M3 - Conference contribution
AN - SCOPUS:85136194322
T3 - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
SP - 4340
EP - 4350
BT - Proceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
PB - IEEE Computer Society
Y2 - 19 June 2022 through 24 June 2022
ER -