Cross-Scale Cost Aggregation for Stereo Matching

Kang Zhang, Yuqiang Fang, Dongbo Min, Lifeng Sun, Shiqiang Yang, Shuicheng Yan

Research output: Contribution to journalArticlepeer-review

48 Scopus citations


This paper proposes a generic framework that enables a multiscale interaction in the cost aggregation step of stereo matching algorithms. Inspired by the formulation of image filters, we first reformulate cost aggregation from a weighted least-squares (WLS) optimization perspective and show that different cost aggregation methods essentially differ in the choices of similarity kernels. Our key motivation is that while the human stereo vision system processes information at both coarse and fine scales interactively for the correspondence search, state-of-the-art approaches aggregate costs at the finest scale of the input stereo images only, ignoring inter-consistency across multiple scales. This motivation leads us to introduce an inter-scale regularizer into the WLS optimization objective to enforce the consistency of the cost volume among the neighboring scales. The new optimization objective with the inter-scale regularization is convex, and thus, it is easily and analytically solved. Minimizing this new objective leads to the proposed framework. Since the regularization term is independent of the similarity kernel, various cost aggregation approaches, including discrete and continuous parameterization methods, can be easily integrated into the proposed framework. We show that the cross-scale framework is important as it effectively and efficiently expands state-of-the-art cost aggregation methods and leads to significant improvements, when evaluated on Middlebury, Middlebury Third, KITTI, and New Tsukuba data sets.

Original languageEnglish
Article number7368908
Pages (from-to)965-976
Number of pages12
JournalIEEE Transactions on Circuits and Systems for Video Technology
Issue number5
StatePublished - May 2017

Bibliographical note

Publisher Copyright:
© 1991-2012 IEEE.


  • Cost aggregation
  • local stereo matching
  • multiscale


Dive into the research topics of 'Cross-Scale Cost Aggregation for Stereo Matching'. Together they form a unique fingerprint.

Cite this