TY - JOUR
T1 - Discrete-Continuous Transformation Matching for Dense Semantic Correspondence
AU - Kim, Seungryong
AU - Min, Dongbo
AU - Lin, Stephen
AU - Sohn, Kwanghoon
N1 - Funding Information:
This research was supported by Next-Generation Information Computing Development Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT (NRF-2017M3C4A7069370). This work of S. Kim was supported (in part) by the Yonsei University Research Fund (Yonsei Frontier Lab. Young Researcher Supporting Program) of 2018. This work of D. Min was supported in part by the Young Researcher Program through the NRF under Grant NRF-2018R1C1B6004622.
Publisher Copyright:
© 1979-2012 IEEE.
PY - 2020/1/1
Y1 - 2020/1/1
N2 - Techniques for dense semantic correspondence have provided limited ability to deal with the geometric variations that commonly exist between semantically similar images. While variations due to scale and rotation have been examined, there is a lack of practical solutions for more complex deformations such as affine transformations because of the tremendous size of the associated solution space. To address this problem, we present a discrete-continuous transformation matching (DCTM) framework where dense affine transformation fields are inferred through a discrete label optimization in which the labels are iteratively updated via continuous regularization. In this way, our approach draws solutions from the continuous space of affine transformations in a manner that can be computed efficiently through constant-time edge-aware filtering and a proposed affine-varying CNN-based descriptor. Furthermore, leveraging correspondence consistency and confidence-guided filtering in each iteration facilitates the convergence of our method. Experimental results show that this model outperforms the state-of-the-art methods for dense semantic correspondence on various benchmarks and applications.
AB - Techniques for dense semantic correspondence have provided limited ability to deal with the geometric variations that commonly exist between semantically similar images. While variations due to scale and rotation have been examined, there is a lack of practical solutions for more complex deformations such as affine transformations because of the tremendous size of the associated solution space. To address this problem, we present a discrete-continuous transformation matching (DCTM) framework where dense affine transformation fields are inferred through a discrete label optimization in which the labels are iteratively updated via continuous regularization. In this way, our approach draws solutions from the continuous space of affine transformations in a manner that can be computed efficiently through constant-time edge-aware filtering and a proposed affine-varying CNN-based descriptor. Furthermore, leveraging correspondence consistency and confidence-guided filtering in each iteration facilitates the convergence of our method. Experimental results show that this model outperforms the state-of-the-art methods for dense semantic correspondence on various benchmarks and applications.
KW - Dense semantic correspondence
KW - continuous optimization
KW - discrete optimization
KW - interative inference
UR - http://www.scopus.com/inward/record.url?scp=85055701362&partnerID=8YFLogxK
U2 - 10.1109/TPAMI.2018.2878240
DO - 10.1109/TPAMI.2018.2878240
M3 - Article
C2 - 30371354
AN - SCOPUS:85055701362
SN - 0162-8828
VL - 42
SP - 59
EP - 73
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
JF - IEEE Transactions on Pattern Analysis and Machine Intelligence
IS - 1
M1 - 8510898
ER -