TY - JOUR
T1 - Feature Augmentation for Learning Confidence Measure in Stereo Matching
AU - Kim, Sunok
AU - Min, Dongbo
AU - Kim, Seungryong
AU - Sohn, Kwanghoon
N1 - Funding Information:
Manuscript received January 10, 2017; revised July 18, 2017; accepted August 29, 2017. Date of publication September 8, 2017; date of current version October 3, 2017. This work was supported by the Institute for Information and communications Technology Promotion funded by the Korea government under Grant 2016-0-00197. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Aljosa Smolic. (Corresponding author: Kwanghoon Sohn.) S. Kim, S. Kim, and K. Sohn are with the School of Electrical and Electronic Engineering, Yonsei University, Seoul 120-749, South Korea (e-mail: kso428@yonsei.ac.kr; srkim89@yonsei.ac.kr; khsohn@yonsei.ac.kr).
Publisher Copyright:
© 2017 IEEE.
PY - 2017/12
Y1 - 2017/12
N2 - Confidence estimation is essential for refining stereo matching results through a post-processing step. This problem has recently been studied using a learning-based approach, which demonstrates a substantial improvement on conventional simple non-learning based methods. However, the formulation of learning-based methods that individually estimates the confidence of each pixel disregards spatial coherency that might exist in the confidence map, thus providing a limited performance under challenging conditions. Our key observation is that the confidence features and resulting confidence maps are smoothly varying in the spatial domain, and highly correlated within the local regions of an image. We present a new approach that imposes spatial consistency on the confidence estimation. Specifically, a set of robust confidence features is extracted from each superpixel decomposed using the Gaussian mixture model, and then these features are concatenated with pixel-level confidence features. The features are then enhanced through adaptive filtering in the feature domain. In addition, the resulting confidence map, estimated using the confidence features with a random regression forest, is further improved through K-nearest neighbor based aggregation scheme on both pixel- and superpixel-level. To validate the proposed confidence estimation scheme, we employ cost modulation or ground control points based optimization in stereo matching. Experimental results demonstrate that the proposed method outperforms state-of-the-art approaches on various benchmarks including challenging outdoor scenes.
AB - Confidence estimation is essential for refining stereo matching results through a post-processing step. This problem has recently been studied using a learning-based approach, which demonstrates a substantial improvement on conventional simple non-learning based methods. However, the formulation of learning-based methods that individually estimates the confidence of each pixel disregards spatial coherency that might exist in the confidence map, thus providing a limited performance under challenging conditions. Our key observation is that the confidence features and resulting confidence maps are smoothly varying in the spatial domain, and highly correlated within the local regions of an image. We present a new approach that imposes spatial consistency on the confidence estimation. Specifically, a set of robust confidence features is extracted from each superpixel decomposed using the Gaussian mixture model, and then these features are concatenated with pixel-level confidence features. The features are then enhanced through adaptive filtering in the feature domain. In addition, the resulting confidence map, estimated using the confidence features with a random regression forest, is further improved through K-nearest neighbor based aggregation scheme on both pixel- and superpixel-level. To validate the proposed confidence estimation scheme, we employ cost modulation or ground control points based optimization in stereo matching. Experimental results demonstrate that the proposed method outperforms state-of-the-art approaches on various benchmarks including challenging outdoor scenes.
KW - Confidence measure
KW - confidence feature augmentation
KW - confidence map aggregation
KW - ground control point
KW - random regression forest
UR - http://www.scopus.com/inward/record.url?scp=85032381267&partnerID=8YFLogxK
U2 - 10.1109/TIP.2017.2750404
DO - 10.1109/TIP.2017.2750404
M3 - Article
C2 - 28910763
AN - SCOPUS:85032381267
SN - 1057-7149
VL - 26
SP - 6019
EP - 6033
JO - IEEE Transactions on Image Processing
JF - IEEE Transactions on Image Processing
IS - 12
M1 - 8030080
ER -