Feature Augmentation for Learning Confidence Measure in Stereo Matching

Sunok Kim, Dongbo Min, Seungryong Kim, Kwanghoon Sohn

Research output: Contribution to journalArticlepeer-review

34 Scopus citations


Confidence estimation is essential for refining stereo matching results through a post-processing step. This problem has recently been studied using a learning-based approach, which demonstrates a substantial improvement on conventional simple non-learning based methods. However, the formulation of learning-based methods that individually estimates the confidence of each pixel disregards spatial coherency that might exist in the confidence map, thus providing a limited performance under challenging conditions. Our key observation is that the confidence features and resulting confidence maps are smoothly varying in the spatial domain, and highly correlated within the local regions of an image. We present a new approach that imposes spatial consistency on the confidence estimation. Specifically, a set of robust confidence features is extracted from each superpixel decomposed using the Gaussian mixture model, and then these features are concatenated with pixel-level confidence features. The features are then enhanced through adaptive filtering in the feature domain. In addition, the resulting confidence map, estimated using the confidence features with a random regression forest, is further improved through K-nearest neighbor based aggregation scheme on both pixel- and superpixel-level. To validate the proposed confidence estimation scheme, we employ cost modulation or ground control points based optimization in stereo matching. Experimental results demonstrate that the proposed method outperforms state-of-the-art approaches on various benchmarks including challenging outdoor scenes.

Original languageEnglish
Article number8030080
Pages (from-to)6019-6033
Number of pages15
JournalIEEE Transactions on Image Processing
Issue number12
StatePublished - Dec 2017

Bibliographical note

Publisher Copyright:
© 2017 IEEE.


  • Confidence measure
  • confidence feature augmentation
  • confidence map aggregation
  • ground control point
  • random regression forest


Dive into the research topics of 'Feature Augmentation for Learning Confidence Measure in Stereo Matching'. Together they form a unique fingerprint.

Cite this