Novel distribution model of transformed coefficients in video coding using quad-tree structured block partitioning

Today’s video coding standard such as high efficiency video coding uses a full quad-tree structured block partitioning, so the underlying statistics of transformed coefficients becomes more complicated to estimate than the previous standards due to the coding structure. However, a statistical distribution of transformed residue is important for a design of a smart encoder. Thus, in this paper, we present a theoretic analysis of a distribution of transformed coefficients produced from an encoder using different transform sizes, and derive a probability density function (pdf) for the estimation. The proposed density model provides a more accurate distribution model than the conventional pdfs. Parameters are theoretically estimated, and rate-distortion model is established from the proposed pdf. We also apply the proposed method to a rate control problem to show the efficiency of the proposed density model. Our experimental results show that the proposed method is better capable of modeling the mixed sources of multiple-type transform coefficients occurred from the quad-tree coding structure of transform and provides an accurate estimate in rate control.