High Efficiency Video Coding (HEVC) Version 2 includes the Range Extensions (RExt) specifying the support for chroma formats other than 4:2:0 and bit depths beyond 10 bits per sample. All 4:4:4 RExt profiles specify the usage of Cross-Component Prediction (CCP). It was developed by the support of the Image and Video Coding group targeting the direct compression of R´G´B´ content. CCP improves the compression efficiency for all relevant RExt applications ranging from consumer operation points up to lossless coding.
In-Loop Luma-to-Chroma Prediction
External color space conversion resolves statistical dependencies among the color components resulting in improved compression efficiency for almost all relevant operation points. This is mostly achieved by representing the chroma signals as a residual signal relative to the luma component. However, local dependencies remain after the fixed color transform, e.g., when transform from R´G´B´ to Y´Cb Cr. It is also highly desirable to compress directly in R´G´B´, especially for novel applications such as screen content. Many solutions addressing those aspects rely on linear luma-to-chroma prediction operating in the coding loop of the hybrid video architecture.
Features of Cross-Component Prediction
CCP is a concrete form of the in-loop luma-to-chroma prediction approach that has a well-balanced trade-off between complexity and compression efficiency improvement. CCP inherits the following features:
- Linear prediction without offset, low-cost reconstruction rule
- Forward-adaptive signaling, limiting the decoder’s complexity
- Operating on prediction residuals, limiting the delay between parsing and reconstruction
- Adaptively at transform unit level, higher flexibility in combination with RQT
In general terms, linear prediction relies on the two parameters slope and offset. Due to the application on prediction residuals, the expectation or the mean is zero leading to the usage of the slope parameter only. This parameter is transmitted for each chroma transform block, with a slope parameter equal to zero means that CCP is disabled.
The above figure shows an example for CCP in context of the coding unit and the residual quadtree. The gray-shaded residual blocks of the luma component denote significance, i.e., non-zero valued residuals exist. The white-shaded residual blocks are insignificant. CCP may be used for the gray-shaded residual blocks only. In this concrete example, CCP is used for two blocks in the Cb component and one block in the Cr component, denote by the corresponding gray-shaded blocks. Each prediction use a different weight in this example.
Reconstruction, Syntax, and Rate-Distortion Optimization (RDO)
The additional CCP process resides between the residual reconstruction and the final reconstruction stages. Chroma residuals are reconstructed as hitherto from the bitstream. A specific amount of the luma residual is added to the chroma residual. This amount is specified by the slope parameter transmitted in the bitstream. Logically, the prediction is skipped for a slope parameter value equal to zero.
Because CCP is forward-adaptive, the best slope parameter and the best cost in terms of rate-distortion has to be derived at encoder’s side. A brute-force strategy may be expensive in terms of run time for software or logic for hardware. An example shortcut can be found in the HEVC reference software implementation. In that implementation, the rate-distortion cost of the case without CCP and the case with an estimated slope parameter are compared. The estimated slope parameter is derived by a simplified linear regression calculation.
Publications
- A. Khairat, T. Nguyen, M. Siekmann, D. Marpe, and T. Wiegand, Adaptive Cross-Component Prediction for 4:4:4 High Efficiency Video Coding, IEEE International Conference on Image Processing (ICIP 2014), Paris, Oct 27-30, 2014.
- T. Nguyen, A. Khairat, D. Marpe, M. Siekmann, and T. Wiegand, Extended Cross-Component Prediction in HEVC, Picture Coding Symposium, Cairns, Australia, 2015, pp. 164–168.