2D-plus-depth based resolution and frame-rate up-conversion technique for depth video

Jinwook Choi, Dongbo Min, Kwanghoon Sohn

Research output: Contribution to journalArticlepeer-review

22 Scopus citations


We propose a novel framework for upconversion of depth video resolution in both spatial and time domains considering spatial and temporal coherences. Although the Time-of-Flight (TOF) sensor which is widely used in computer vision fields provides depth video in realtime, it also provides a low resolution and a low frame-rate depth video. We propose a cheaper solution that enhances depth video obtained from a TOF sensor by combining it with a Charge-coupled Device (CCD) camera in 3D contents which consist of 2D-plus-depth. Temporal fluctuation problems are also considered for temporally consistent framerate up-conversion. It is important to maintain temporal coherence in depth video, because temporal fluctuation problems may cause eye fatigue and increase bit rates on video coding. We propose a Motion Compensated Frame Interpolation (MCFI) using reliable and rich motion information from a CCD camera and 3-dimensional Joint Bilateral Up-sampling (3D JBU) extended into the temporal domain of depth video. Experimental results show that depth video obtained by the proposed method provides satisfactory quality.

Original languageEnglish
Article number5681132
Pages (from-to)2489-2497
Number of pages9
JournalIEEE Transactions on Consumer Electronics
Issue number4
StatePublished - Nov 2010

Bibliographical note

Funding Information:
1This research was supported by the MKE(The Ministry of Knowledge Economy), Korea, under the ITRC(Information Technology Research Center) support program supervised by NIPA(National IT Industry Promotion Agency) (NIPA-2010-C1090-1001-0006).


  • TOF sensor
  • depth video
  • frame-rate
  • resolution
  • up-conversion


Dive into the research topics of '2D-plus-depth based resolution and frame-rate up-conversion technique for depth video'. Together they form a unique fingerprint.

Cite this