In recent years, several video coding standards such as H.261, H.263, MPEG-1, and MPEG-2 have been introduced, which mainly address the compression of generic video data for digital storage and communication services. These schemes utilize the statistics of the video signal without knowledge of the semantic content of the frames and can therefore robustly be used for arbitrary scenes.

Higher coding efficiency can be expected from model-based video coders when the semantic information of the scene can be exploited. This is often done by using 3-D models that describe the shape and texture of the objects in the scene. The 3-D object descriptions are encoded only once. Individual video frames are characterized by 3-D motion and deformation of theseobjects that can be transmitted at extremely small bit-rates making the scheme amenable for very low bit-rate applications.

However, this type of coder is restricted to scenes that can be composed of objects that are known by the decoder. One typical class of scenes are head-and-shoulder sequences frequently found in applications such as video-telephone or video-conferencing systems. For such head-and-shoulder scenes bit-rates of about 1 kbit/s with acceptable quality can be achieved. This has also motivated the recently determined synthetic and natural hybrid coding (SNHC) part of the MPEG-4 standard. SNHC allows the transmission of a 3-D face model that can be animated to generate different facial expressions.

On this webpage we demonstrate an extension of an H.263 video coder that utilizes information from a model-based coder. Instead of exclusively predicting the current frame of the video sequence from the previously decoded frame, prediction from the synthetic frame of the model-based coder is additionally allowed. The coder decides which prediction is efficient in terms of rate-distortion performance. Hence, the coding efficiency does not decrease below H.263 in the case the model-based coder cannot describe the current scene. On the other hand, if the objects in the scene are compliant to the 3-D models in the codec, a significant improvement in coding efficiency can be achieved.

Experimental Results

Experiments are conducted using two self-recorded natural CIF sequences Peter and Eckehard consisting of 200 and 100 frames, respectively. Both sequences are encoded at 8.33 Hz. Rate-distortion curves are generated by varying the DCT quantizer over values 10, 15, 20, 25, and 31. Bit-streams are generated that are decodable producing the same PSNR values as at the encoder. The data for the first INTRA frame and the initial 3-D model are excluded from the results thus simulating steady-state behavior. For the transmission of the face model description that is not changed during the sequence, the position of 316 control points and a texture map of size 450 x 512 pixels have to be encoded.

We first show rate-distortion curves for the proposed coder in comparison to the H.263 test model, TMN-10. The following abbreviations are used for the two cases:

• TMN-10: The result produced by the H.263 test model, TMN-10, using Annexes D, F, I, J, and T.
• MAC: Model-aided coder: H.263 extended by model-based prediction.