With the establishment of the Learned Data Compression research group, the Video Communications and Applications (VCA) department at Fraunhofer HHI (Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute) is strengthening its activities in the field of AI-based image and video compression. The group is headed by Prof. Dr. Jona Ballé, an internationally recognized expert in learning-based compression and visual perception models. “We are very proud of Jona Ballé as one of the worldwide leading pioneers in the field of Nonlinear Transform Coding (NTC) joining our team”, says Dr. Detlev Marpe, head of the VCA department. “Her groundbreaking scientific work has contributed significantly to the development of the JPEG AI standard, which was finalized in 2025.”
“First of all, it's great to be back in Germany! I look forward to exploring Berlin, which I've always wanted to do,” says Jona Ballé. After several years as a Staff Research Scientist at Google Research and a subsequent professorship at NYU Tandon School of Engineering, she is now joining Fraunhofer HHI for her new role. Her scientific career began at RWTH Aachen University, where she completed her doctorate with summa cum laude honors and worked on innovative image compression methods early on.
The new VCA research group focuses on fundamentally rethinking the architecture of visual data representation. Instead of classic, conventionally designed compression methods, the group's research focuses on end-to-end optimized systems, in which each component is learned entirely from data. A key concept is Nonlinear Transform Coding developed by Dr. Ballé since 2016, which replaces traditional linear transformations such as discrete cosine transform (DCT) or wavelets with trainable neural networks. “We are striving for a paradigm shift towards end-to-end optimized systems,” she explains. Building on the success of NTC, the group will rethink established coding tools such as quantization and entropy coding, and challenge the fundamental assumptions of classical source coding.
A second research focus is on models of human vision. Machine learning opens up the possibility of developing more complex, biologically inspired perception models that more accurately predict how humans judge image quality. “Machine learning offers enormous potential for developing more complex models of visual perception—by imitating aspects of how humans learn to see,” says Prof. Ballé. These findings will be directly incorporated into new, qualitatively improved compression methods.
“I am even more excited to work with some of the best people in the field of video compression and to put some of the novel ideas I have been researching in recent years into practice,” emphasizes Prof. Ballé. With her expertise and the new group, Fraunhofer HHI is strengthening its position as a leading research location for image and video compression, boosting future applications in streaming, XR, communication technologies, and visual quality assurance.
Further information can be found here.