Reliability-based multiview depth enhancement considering interview coherence

Jinwook Choi, Dongbo Min, Kwanghoon Sohn

Research output: Contribution to journalArticlepeer-review

25 Scopus citations


Color-plus-depth video format has been increasingly popular in 3-D video applications, such as auto-stereoscopic 3-D TV and freeview TV. The performance of these applications is heavily dependent on the quality of depth maps since intermediate views are synthesized using the corresponding depth maps. This paper presents a novel framework for obtaining high-quality multiview color-plus-depth video using a hybrid sensor, which consists of multiple color cameras and depth sensors. Given multiple high-resolution color images and low quality depth maps obtained from the color cameras and depth sensors, we improve the quality of the depth map corresponding to each color view by increasing its spatial resolution and enforcing interview coherence. Specifically, a new up-sampling method considering the interview coherence is proposed to enhance multiview depth maps. This approach can improve the performance of the existing up-sampling algorithms, such as joint bilateral up-sampling and weighted mode filtering, which have been developed to enhance a single-view depth map only. In addition, an adaptive approach of fusing multiple input low-resolution depth maps is proposed based on the reliability that considers camera geometry and depth validity. The proposed framework can be extended into the temporal domain for temporally consistent depth maps. Experimental results demonstrate that the proposed method provides better multiview depth quality than the conventional single-view-based methods. We also show that it provides comparable results, yet much more efficiently, to other fusion approaches that employ both depth sensors and stereo matching algorithm together. Moreover, it is shown that the proposed method significantly reduces bit rates required to compress the multiview color-plus-depth video.

Original languageEnglish
Article number6578545
Pages (from-to)603-616
Number of pages14
JournalIEEE Transactions on Circuits and Systems for Video Technology
Issue number4
StatePublished - Apr 2014


  • Color camera
  • Depth sensor
  • Depth up-sampling
  • Interview coherence
  • Reliability


Dive into the research topics of 'Reliability-based multiview depth enhancement considering interview coherence'. Together they form a unique fingerprint.

Cite this