Fast Multi-Type Tree Partitioning for Versatile Video Coding Using a Lightweight Neural Network

Sang Hyo Park, Je Won Kang

Research output: Contribution to journalArticlepeer-review

40 Scopus citations

Abstract

In this paper, we propose a fast decision scheme using a lightweight neural network (LNN) to avoid redundant block partitioning in versatile video coding (VVC). A more versatile block structure, named the multi-type tree (MTT) structure, which includes binary trees (BTs) and ternary trees (TTs), is adopted by VCC, in addition to the traditional quadtree structure. The MTT improved the coding efficiency compared with previous video coding standards. However, the new tree structures, mainly TT, significantly increased the complexity of the VVC encoder. Although widespread application of VVC has been inhibited, this problem has not yet been investigated thoroughly in the literature. In this study, we first determine the statistical characteristics of coded parameters that exhibit correlation with the TT and develop two useful types of features - explicit VVC features (EVFs) and derived VVC features (DVFs) - to facilitate the intra coding of VVC. These features can be obtained efficiently during the intra prediction before the determination of the best block partitioning during rate-distortion optimization in VVC encoding. Our LNN model decides whether to terminate the nested TT block structures subsequent to a quadtree based on the features. The experimental results confirm that the proposed method substantially decreases the encoding complexity of VVC with a slight coding loss under the All Intra configuration. Our code, models, and dataset are available at https://github.com/foriamweak/MTTPartitioning_LNN.

Original languageEnglish
Pages (from-to)4388-4399
Number of pages12
JournalIEEE Transactions on Multimedia
Volume23
DOIs
StatePublished - 2021

Bibliographical note

Funding Information:
This work was supported in part by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) under Grant NRF-2019R1C1C1010249, in part by the BK21 FOUR project (AI-driven Convergence Software Education Research Program) funded by the Ministry of Education, School of Computer Science and Engineering, Kyungpook National University, Korea under Grant 4199990214394, and in part by the Basic Science Research Program through the NRF funded by the Ministry of Education under Grant 2020R1I1A3072227.

Publisher Copyright:
© 1999-2012 IEEE.

Keywords

  • Block partitioning
  • Deep learning
  • Encoding complexity
  • Image compression
  • Intra prediction
  • Multi-type tree
  • Neural network
  • VVC
  • Video compression

Fingerprint

Dive into the research topics of 'Fast Multi-Type Tree Partitioning for Versatile Video Coding Using a Lightweight Neural Network'. Together they form a unique fingerprint.

Cite this