Hardware Architectures for Ultra Low Latency Video Coding

Besides the traditional video coding domains e.g. video broadcast services or video playback applications, there are many other application fields for video coding which demands completely different coding parameters than the before mentioned applications. In the traditional domains the most important parameter of the used video codec can be constricted to the best possible achieved PSNR for a given data rate. The next most interesting video coding parameter is the achieved coding latency, which describes the time / delay needed to process a video signal from the input of the encoder to the corresponding video signal at the output of the decoder. This parameter is most important for application where an interaction between the receiver side and the transmitter side takes place. In these applications latencies much lower than one frame period are required, which could be in the range of only several milliseconds. To achieve very low latencies we've designed completely hardwired video codec ip cores, which are fully compatible to H.264/AVC featuring special tools which provide nearly the same coding efficiency as their high latency counterparts. E.g. since coding of intra coded frames results in excessive bit rate changes when switching between intra and inter frames we've integrated the so called intra refresh mode, to achieve constant bitrates for low latency transmission of the coded video signals. The placement pattern of the intra encoded macroblocks depends on the used strategy. This pattern also defines the time after which a decoder receives one intra encoded macroblock for each macroblock position. Our completely hardwired ELLVC (Enhanced Low Latency Video Codec) IP core supports the refresh pattern which is shown in the following illustration. Intra encoded macroblocks are highlighted.

Our IP core ELLVC  achieves coding latencies down to 1ms, which has been evaluated using our ESGVC video codec evaluation board.