The important growth of the amount of produced and consumed digital image and video data increasingly amplifies the need for semantic approaches in image processing. The knowledge of what is in the content is highly valuable as it enables a multitude of application scenarios. Creating the necessary metadata manually is thereby not an option due to the prohibitive costs of multiple hours of work needed for each individual hour of video. Automated methods for extraction, creation and deployment of semantic information are the focus of research in the Content-Aware Image Processing Group. For digitized or otherwise degraded content, full- and no-reference quality assessment and optimization tools with a high correlation to the human perception are provided to enhance the metadata quality.

The basic idea that knowledge of what is in the image is key, is applied to multiple scenarios within the research group. Focus lays on the reliable automatic creation of content metadata itself. Content-awareness is also used to increase efficiency of image and video coding.

With the amount of digital image and video data in archives exploding and an ever increasing addition of newly produced and digitalized content, the challenge of keeping the content easily accessible is becoming more and more difficult. It is highly uneconomic to screen and annotate all content in person. But without the knowledge of the substance of the archived content the assets are worthless and cannot be reused and marketed. Automated, reliable systems that categorize, classify, annotate and tag the content and are also able to detect duplicates are the only way to increase the efficiency of this cumbersome process insofar as to sustain values.

The compression of video content containing high-frequency textures using state-of-the-art codecs leads to very large bit rate assignments to these areas. However, we assume that the exact reproduction of such areas is subjectively not required and are looking for coding methods which result in substantially lower bit rate assignments. One such consists in inserting a texture analyzer at the encoder and a texture synthesizer at the decoder. In areas, where it is assumed that the viewer perceives the semantic meaning of the displayed texture rather than the specific details therein, the texture is correspondingly labeled by the texture analyzer. These are later synthesized and regenerated as an approximate version of the original texture based on corresponding metadata transmitted by the encoder. So far, the approach has yielded bit rate reductions of up to 41% when compared at the same visual quality to a reference H.264/AVC codec.

With more and more archives digitalizing image and video content in order to preserve cultural heritage, the amount of resources and time wasted due to diverse distortion sources during the process is ever increasing. Automated quality assessments systems within the digitalization process can quickly detect quality issues at both global and local level. The same technologies enable archives to better tag existing content whether it is suitable for repurposing and rank multiple copies of the same content based on quality-criteria.