VVC Transform: Multi Transform Selection (MTS)
The article is the first in the series of articles about VVC transform. It describes Multi Transform Selection (MTS) as a novel type of transformation used in VVC and demonstrates its application in VQ Analyzer ver. 6.0 and DVK developed by ViCueSoft.
Versatile Video Coding (VVC / H266) as a new video compression standard
Versatile Video Coding (VVC / H266) standard was released as a new international video compression standard in July 2020. VVC became the successor of High Efficiency Video Codec (HEVC / H265) and it was declared that VVC will be more than 30% bitrate resource efficient due to increasing algorithm complexity, especially transform algorithms.
Transform is a type of data compression. The main goal of the transform stage is to map images or their residual data to the transform area. The choice of the transform depends on multiple factors: effective computational complexity, reversibility as well as data decorrelation and compactness (minimal inter-dependence and energy concentration in a small number of transform coefficients).
Two most popular transform categories to which all transform methods belong are image-based transforms and block-based transforms. In this article we will consider one of the new methods of block-based transforms. Block-based transform coding is used for the predicted residual block, where the low frequency components of the transform coefficients obtained after the transform are concentrated in the upper left corner of the block and the high frequency components are in the lower right corner.
In VVC, some new transform features were implemented in comparison with its predecessors. Multi Transform Selection is one of such features.
Multi Transform Selection (MTS)
Multi Transform Selection (MTS) is a new variant of cosine transformation that was added in VVC. In its predecessor HEVC, DCT-II was the only variant of this type. In comparison with HEVC, MTS has some new sine/cosine transform types: DST-7 and DCT-8. These three transform types definitions are shown below:
Compared with HEVC, in VVC more attention is paid to a more precise quantization of transform matrices so that their orthogonality is preserved. In order to keep the intermediate value of the transform coefficient within the 16bit range, after horizontal and vertical transform, all coefficients are 10bit.
Also, to better control MTS both intra/inter modes were specified. To enable MTS, it needs to be enabled on SPS and CU levels and the following conditions need to be met:
- CU both width and height are less or equal to 32,
- there are some non-zeroed coefficients (not only DC coefficient is not zeroed),
- the last coefficient is not DC (CBF is set to equal to 1),
- there are not so many coefficients (the last significant coefficient is not located inside the MTS zero-out region).
If MTS CU flag is equal to 0, DCT-II transform mode is applied for both directions and its maximum sizes for both width and height increase up to 64.
In the table below mode combinations for different blocks are shown:
In order to reduce сomputational complexity of large-size blocks DST-7 and DCT-8, the high-frequency coefficient zero operation is performed when the size of the transform block using DST-7 and DCT-8 (width or height, or width and height) reaches 32. Only 16x16 low-frequency coefficients in the upper left corner are retained.
Now, I will demonstrate which MTS mode were used for transformation of every block in our encoded demo-stream. To visualize it, we will use VQ Analyzer ver. 6.0.
Here we can also highlight all possible variants of MTS:
The example of MTS work is shown in pic 6.
Here you can find a demo-stream encoded with a full variety of allowed MTS indexes. To open and visualize it you can use VQ Analyzer ver. 6.0.
Such bitstreams are parts of Decoder Validation Kit (DVK) product. More information on DVK is available here.
This is all for today. In our next article we will continue telling you about VVC transform possibilities.