Stereo Confidence Estimation via Locally Adaptive Fusion and Knowledge Distillation

Sunok Kim, Seungryong Kim, Dongbo Min, Pascal Frossard, Kwanghoon Sohn

Research output: Contribution to journalArticlepeer-review

Abstract

Stereo confidence estimation aims to estimate the reliability of the estimated disparity by stereo matching. Different from the previous methods that exploit the limited input modality, we present a novel method that estimates confidence map of an initial disparity by making full use of tri-modal input, including matching cost, disparity, and color image through deep networks. The proposed network, termed as Locally Adaptive Fusion Networks (LAF-Net), learns locally-varying attention and scale maps to fuse the tri-modal confidence features. Moreover, we propose a knowledge distillation framework to learn more compact confidence estimation networks as student networks. By transferring the knowledge from LAF-Net as teacher networks, the student networks that solely take as input a disparity can achieve comparable performance. To transfer more informative knowledge, we also propose a module to learn the locally-varying temperature in a softmax function. We further extend this framework to a multiview scenario. Experimental results show that LAF-Net and its variations outperform the state-of-the-art stereo confidence methods on various benchmarks.

Original languageEnglish
Pages (from-to)1-13
Number of pages13
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
DOIs
StateAccepted/In press - 2022

Keywords

  • Color
  • Costs
  • deep learning
  • Estimation
  • Feature extraction
  • Image color analysis
  • knowledge distillation
  • Knowledge engineering
  • stereo confidence estimation
  • Stereo matching
  • Training

Fingerprint

Dive into the research topics of 'Stereo Confidence Estimation via Locally Adaptive Fusion and Knowledge Distillation'. Together they form a unique fingerprint.

Cite this