Abstract
In this paper, we propose a novel super-resolution (SR) technique for multi-view 360-degree images in equirectangular projection (ERP) format. To the best of our knowledge, the proposed algorithm is the first study of multi-view 360-degree images in ERP. In multi-view SR (MV-SR), it is important to fuse the knowledge of features at different viewpoints, but the task is hardly achieved using a conventional CNN because conventional convolution is shift invariant. Thus, to solve the problem, we take a coarse-to-fine approach to exploit the correlation among multi-views in an ERP domain. First, we conduct depth-based warping on reference ERP to synthesize the image with the same viewpoint of the target low-resolution (LR) ERP. The non-linear distortion between the two ERP images can be remarkably reduced after the proposed warping. Second, we employ a flow estimator to refine the remaining flow between the warped reference image and the LR image. Our CNN architecture generates the SR at the end of the network by combining the features of LR-ERP and the warped reference ERP. It is demonstrated with experimental results that the proposed algorithm provides significantly improved quality of multi-view 360-degree images for SR as compared to the state-of-the-art in MV-SR.
| Original language | English |
|---|---|
| Title of host publication | 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2020 - Proceedings |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 1283-1286 |
| Number of pages | 4 |
| ISBN (Electronic) | 9789881476883 |
| State | Published - 7 Dec 2020 |
| Event | 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2020 - Virtual, Auckland, New Zealand Duration: 7 Dec 2020 → 10 Dec 2020 |
Publication series
| Name | 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2020 - Proceedings |
|---|
Conference
| Conference | 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2020 |
|---|---|
| Country/Territory | New Zealand |
| City | Virtual, Auckland |
| Period | 7/12/20 → 10/12/20 |
Bibliographical note
Funding Information:ACKNOWLEDGMENT This work was supported by Institute of Information and communications Technology Planning and Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2018-0-00765 Development of Compression and Transmission Technologies for Ultra High Quality Immersive Videos Supporting 6DoF) and the National Research Foundation of Korea (NRF) (No.NRF-2019R1C1C1010249)
Publisher Copyright:
© 2020 APSIPA.