Volumetric Video Workflow

Since more than ten years, we research and develop a complete pipeline for the creation of dynamic 3D models. This effort resulted in a fully automatic volumetric video workflow as depicted in the figure below. The first blocks in blue refer to modules that perform processing of images from various video streams. The yellow blocks refer to modules that are dedicated to mesh processing. Finally, the green blocks assign components, which are dedicated to encoding and streaming of the volumetric assets to the target device, where real-time rendering is performed. This can be either a VR head set or any Augmented Reality device (glasses, tablet, smart phone). The main challenge in this workflow is to manage the huge amount of data in a clever and optimal way. In our capture studio, we acquire about 1.6 TeraByte of video in one minute. This is reduced to several hundreds of MegaByte, by keeping the geometrical detail and offering high-resolution texture. 

In the figure below, results of the different intermediate processing steps are shown.

Selected publications

  1. Oliver Schreer, Ingo Feldmann, Sylvain Renault, Marcus Zepp, Peter Eisert, Peter Kauff, Capture and 3D Video Processing of Volumetric VideoIEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, September 2019.

  2. Oliver Schreer, Ingo Feldmann, Thomas Ebner, Sylvain Renault, Christian Weissig, Danny Tatzelt, Peter Kauff, Advanced Volumetric Capture and ProcessingSMPTE Motion Imaging Journal, vol. 128, no. 5, May 2019, doi: 10.5594/JMI.2019.2906835, Invited Paper.