Depth video enhancement based on weighted mode filtering

Dongbo Min, Jiangbo Lu, Minh N. Do

Research output: Contribution to journalArticlepeer-review

222 Scopus citations

Abstract

This paper presents a novel approach for depth video enhancement. Given a high-resolution color video and its corresponding low-quality depth video, we improve the quality of the depth video by increasing its resolution and suppressing noise. For that, a weighted mode filtering method is proposed based on a joint histogram. When the histogram is generated, the weight based on color similarity between reference and neighboring pixels on the color image is computed and then used for counting each bin on the joint histogram of the depth map. A final solution is determined by seeking a global mode on the histogram. We show that the proposed method provides the optimal solution with respect to L 1 norm minimization. For temporally consistent estimate on depth video, we extend this method into temporally neighboring frames. Simple optical flow estimation and patch similarity measure are used for obtaining the high-quality depth video in an efficient manner. Experimental results show that the proposed method has outstanding performance and is very efficient, compared with existing methods. We also show that the temporally consistent enhancement of depth video addresses a flickering problem and improves the accuracy of depth video.

Original languageEnglish
Article number5963715
Pages (from-to)1176-1190
Number of pages15
JournalIEEE Transactions on Image Processing
Volume21
Issue number3
DOIs
StatePublished - Mar 2012

Keywords

  • Depth enhancement
  • depth sensor
  • multiscale color measure (MCM)
  • temporal consistency
  • weighted mode filtering (WMF)

Fingerprint

Dive into the research topics of 'Depth video enhancement based on weighted mode filtering'. Together they form a unique fingerprint.

Cite this