Layer-Aware FEC

Figure 1 shows an exemplary layered media stream with two media layers, base and enhancement layer, such as produced by SVC or MVC. The base layer (BL) represents the base quality, which is in both cases, SVC and MVC, a video stream with a resolution of 720p. An additional integration of the enhancement layer (EL) in the decoding process improves the video quality. With SVC, the quality increases to a resolution of 1080p. With MVC, decoding of the EL adds a second view for 3D television. As shown in Figure 1, there are dependencies between the media layers. The BL can be decoded independently from other media layers while the EL needs the BL for error-free decoding.

Forward Error Correction (FEC) is used for protecting data against channel impairments. FEC is typically applied in transmission systems on physical layer or upper system layers such as link or application layer. FEC algorithms implemented in today’s transmission systems, such as RaptorQ or LDPC, are optimized for transmission of single layer data streams and do not exploit the knowledge of the dependency structure in layered media codecs. There are two ways for generation of FEC data for layered media streams with today’s FEC codes which are illustrated in Figure 2. One is to simply treat all media layers as one data block and generate FEC data from that FEC source block (left part of the figure). This approach preserves the optimized correction capabilities of the FEC code. However, doing so requires the reception of all media layers which makes reception of media layers separately in a multicast manner or prioritizing of more important media layers impossible. The second approach is to generate for each media layer a separate FEC code (right part of the figure). The latter approach does not have the same limitations as the first approach but is still not optimal. This becomes obvious in situations where the base layer cannot be corrected. In such situations, the enhancement layer becomes useless for the media decoding process due to its dependencies to the base layer and with it all the associated protection data, although it has been successfully received.

The idea of the Layer-Aware FEC (LA-FEC) is to produce FEC data of a certain media layer over the source data of the media layer itself and all media layers this layer depends on. Thereby, FEC data additionally protects data it depends on. The LA-FEC process is illustrated in Figure 3. Since the base layer (BL) does not have any dependencies to other media layers, its FEC generation remains untouched and is generated from the source data of BL only. The FEC data of the enhancement layer (EL) is produced over the source data of both media layers as illustrated by the extended LA-FEC source block. After transmission of both media layers, all FEC coded data can be used for a combined error correction as illustrated in the following example.

To illustrate the principle of the LA-FEC approach, we apply a simple FEC algorithm which generates parity bits by XOR combinations of source symbols. Note that LA-FEC can be integrated in more efficient FEC algorithms as shown by the integration of LA-FEC in RaptorQ and LDPC below. Figure 4 compares the encoding and decoding process of Standard FEC on the left side and LA-FEC on the right side of the figure. The exemplary bit stream consists of two media layers, layer 0 and layer 1, where layer 1 depends on layer 0. Each layer consists of k=3 source bits and the simple FEC algorithm generates p=2 parity bits for each layer. As can be seen in the figure, LA-FEC integrates the source bits of layer 0 in the encoding process of the parity bits of layer 1. The resulting codewords have the same length of n=5 for standard FEC and LA-FEC and for both media layers. After FEC encoding, the codewords are transmitted over an erasure channel. In the outlined situation, the codeword of layer 0 is affected by three erasures which are highlighted by ‘E’, while the codeword of layer 1 remains error free. In case of standard FEC, there are not enough parity bits within layer 0 for successful FEC decoding. The source bits can therefore not be recovered. Although the layer 1 codeword is correctly received, it cannot be used due to the missing media coding dependencies on layer 0. Using LA-FEC, the parity bits of layer 1 can be used jointly with the parity bits of layer 0 for correcting layer 0. Since layer 1 is correctly received, there are in total four parity bits available for correction of the three source bits of layer 0. In the given example, both media layers can only be corrected with LA-FEC. It should be noted that if using the LA-FEC, the enhancement layer cannot be corrected independently of the base layer. Therefore, the improvement in base layer protection comes at the expense of reduced recovery probability of the enhancement layer. Nevertheless, in cases where the base layer is lost, the enhancement layer data cannot be used in the media decoding process anyway due to missing media dependencies within the media stream. Therefore, LA-FEC never performs worse than standard FEC in terms of media quality.