IMC Research Overview

Volumetric Video

Volumetric Video Studio (3rd Gen)

Since February 2023, our new volumetric capture studio is in operation in our new premises at Lanolinfabrik.

Read more

3D Human Body Reconstruction for eXtended Reality (XR)

New applications in media production especially for eXtended Reality (XR) require realistic 3D models of humans, which move and behave naturally. We developed a concept for 3D Human Body Reconstruction based on a 360° multi-view capture system for recording dynamic objects and the related processing pipeline to compute realistic moving 3D models of humans.

Read more

Volumetric Video Capture

We investigated several multi-view capture systems in the past years towards the goal to create a fully functional 360 volumetric capture studio.

Read more

Volumetric Video Workflow

Creating dynamic 3D models from humans is a challenging task. We developed a complete and automatic volumetric video workflow, which transforms several video streams into a coherent dynamically changing 3D model. The challenge is to reduce the amount of data by several orders of magnitude, while keeping the geometrical detail and texture quality.

Read more

Volumetric Video Production

Since 2016, when we started with the first capture systems for full 3D human body capture, several XR experiences have been produced in order to show the progress and capability of this new technology. The different productions cover a large range of genres demonstrating the capability of this new data format.

Read more

Augmented, Virtual and Mixed Reality

Dynamic Exposure Visualization of Air Quality Data with Augmented Reality

We are developing new ways to visualize environmental data such as air pollution or traffic for Augmented Reality applications. The objective is to provide the user with complex spatio-temporal information in a comprehensive way. We proudly present the world's first AR app that visualizes real-time environmental 3D sensor data, the Dynamic Exposure Visualization App (DEVA).

Read more

Interactive Visualization for Cultural Heritage

Digitization of cultural heritage artefacts is key to making them accessible anywhere at any time either for education or research. We investigate novel ways how to visualize and interact with such kind of data.

Read more

Interactive 3D Book Explorer

Touchless interaction in combination with 3D visualization offers the right technology for interactive exploration of three-dimensional data. A robust demonstrator was developed to show the potential of this technology.

Read more

Volumetric Data Visualization in Medicine

Complex three-dimensional medical data requires new approaches for visualization to better understand and interpret. Based on our expertise in 3D visualization, we developed an intuitive application for rendering of volumetric data.

Read more

Virtual Communication and Collaboration

Remote Assistance in Construction and Industry

Within the national funded German research project DigitalTWIN (BMWI) we have developed a mobile remote assistance system, which is able to connect a worker on the construction site with a remote expert, for example the architect in the office.

Read more

Immersive Video Communication

We are experts with many years of experience in the field of immersive video communication, working on concepts, systems and algorithm development for single and multi-user 2D/3D videocommunication scenarios supporting eye contact and correct gesture representation as well as 3D rendering on autostereoscopic displays.

Read more

Virtual Eye Contact

We developed a real-time multi-view video processing module for video conferencing systems offering direct eye contact between the local and remote conferees.

Read more

Hybrid Meetings and Collaboration

Due to our extensive experience in immersive video communication, we will further research and develop new approaches for future hybrid collaboration and communication technologies and services.

Read more

Deep Learning for Immersive Media and 3D Scene Processing

Foreground/Background Separation

Deep learning can be successfully applied to foreground/background separation tasks. In the context of our volumetric video pipeline, we investigate new approaches for object segmentation in order to increase robustness and quality of the segmentation.

Read more

Neural Representations and Rendering

We are investigating how analysis-by-synthesis based on neural geometry representations and neural rendering can be used in real-world multi-view reconstruction scenarios in the context of human body reconstruction and dynamic scenes. 

Read more

Multi-View Stereo for High-Accuracy Human Body Reconstruction

We investigate and optimize deep learning-based multi-view stereo networks Vis-MVSNet for depth map estimation in the context of professional volumetric video reconstruction.

Read more

Previous Research Topics

Pose and Gesture Analysis

We research and develop fast and reliable algorithms for detection and tracking of whole body movements as well as movements of specific body parts. Our solutions focus on body, head, eye, hand and finger tracking as well as posture, movement and gait analysis. Our algorithms are based on non-intrusive, marker less sensors which provide 2D and/or 3D video signals, e.g. colour, depth-from stereo, time of flight or structured light. These algorithms are designed to work with a variety of cameras from low cost, mobile to high-end devices finding a best compromise between performance and price, from offline behaviour analysis to real-time human-computer interaction.

Read more

Behaviour Analysis for Human-Computer Interaction

Based on our detection and tracking algorithms as well as additional novel input sources such as voice control, we create novel human computer interaction modules. This involves calibration, fusion and interpretation of various data in a specific context. These modules are used in various applications such as behaviour analysis for ambient assisted living solutions, interaction with applications and media content as well as novel video communication applications.

Read more