Why do we need Video Coding?
Videos conquer the world, streaming and social media platforms are becoming more and more popular. But what are the technical innovations enabling the boom? Newest technologies in video compression provide the basis for the distribution of videos worldwide. Some of the contributions to the most popular video compression standards have been developed at the Fraunhofer Heinrich Hertz Institute.
It is estimated that about 80% of the global privately used internet traffic consists of video data. Besides the entertainment and social media industry, video streaming plays a key role also in other areas of our daily life. For example, some online platforms offer complete lecture series as streams. Furthermore, videos can be used as educational material in medicine for example when broadcasting medical operations. Given these enormous amount of video data, there is a strong need for efficient video compression methods.
Typically, such a method consists of two parts, the encoder and the decoder. This is where the term “codec” stems from. Initially, the raw video data is given on the transmitter side. Examples for transmitters are Movie and TV productions, video conferences, security cameras or streaming platforms. The transmitter transforms the initial video data into a bitstream that is sent to a receiver, for example a TV or a mobile phone, via a transmission channel like the internet.
The key problem is that the capacity of modern transmission channels is limited. Thus, the worldwide distribution of video signals is possible only due to the existence of video coding technologies which significantly reduce the size of the transmitted data while maintaining visual quality.
Basic Principles of Video Coding
Classical video compression methods are based on two principles taken from information theory: the reduction of redundancy and the loss of irrelevant information. Video data is highly redundant: This means that, given already decoded information, not every current information is equally likely. This fact is exploited by coding more likely information with fewer bits than less likely one. For example, the time wise correlation of subsequent frames is used in order to transmit only its displacement with respect to previous video frames rather than transmitting the content of the object again.
Given the high demands on modern compression methods, lossy compression techniques are essential for modern video codecs. Here, the principle of removing irrelevant information is used, i.e., image data is removed which is invisible to a human observer. A typical example is the reduction of color information in the videos. Since the human perception of changes in color is much less sensitive than changes in brightness, one divides the image into luminance and chroma channels. Then, the resolution of the chroma channels can be largely decreased while maintaining overall good visual quality.
Video coding standards
A video coding standard is a fixed communication protocol that exactly specifies how a receiver generates an output video sequence out of a bitstream. Thus, video coding standards guarantee interoperability between multiple types of transmitters and receivers. A video compression standard specifies solely the bitstream processing at the receiver while the generation of the bitstream at the transmitter remains open. In this way, using a single standard, a transmitter can decide between different operational points for the tradeoff between the size of the transmitted bitstream and the quality of the received signal.
The H.26x series of video compression standards exists over 25 years and is extremely popular until the current day. Their worldwide distribution has contributed majorly to the broad adoption and growing popularity of video technologies. The H.264/AVC (“Advanced Video Coding”) standard is still one of the most commonly used video compression standards: every HDTV receiver, every Blu-Ray player and most internet videos are using the H.264/AVC standard. It is included into approximately one billion devices worldwide. Its direct successor H.265/HEVC (“High Efficiency Video Coding”) is on the market since 2013 and requires only half the data rate in comparison to H.264/AVC while maintaining image quality. In the last years, the involved companies, universities and research institutes (among others the Fraunhofer Heinrich Hertz Institute) have joined forces: the international committee JVET (“Joint Video Experts Team”) was founded by the ITU-T VCEG and the ISO/IEC MPEG in April 2018 in order to develop a new video compression standard. The new video compression standard called H.266/VVC (“Versatile Video Coding”) is to be completed in October 2020 and aims to reduce the required data rate even further while maintaining image quality.