Since more than ten years, we research and develop a complete pipeline for the creation of dynamic 3D models. This effort resulted in a fully automatic volumetric video workflow as depicted in the figure below. The first blocks in blue refer to modules that perform processing of images from various video streams. The yellow blocks refer to modules that are dedicated to mesh processing. Finally, the green blocks assign components, which are dedicated to encoding and streaming of the volumetric assets to the target device, where real-time rendering is performed. This can be either a VR head set or any Augmented Reality device (glasses, tablet, smart phone). The main challenge in this workflow is to manage the huge amount of data in a clever and optimal way. In our capture studio, we acquire about 1.6 TeraByte of video in one minute. This is reduced to several hundreds of MegaByte, by keeping the geometrical detail and offering high-resolution texture.
Oliver Schreer, Ingo Feldmann, Sylvain Renault, Marcus Zepp, Peter Eisert, Peter Kauff, Capture and 3D Video Processing of Volumetric Video, IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, September 2019.
Oliver Schreer, Ingo Feldmann, Thomas Ebner, Sylvain Renault, Christian Weissig, Danny Tatzelt, Peter Kauff, Advanced Volumetric Capture and Processing, SMPTE Motion Imaging Journal, vol. 128, no. 5, May 2019, doi: 10.5594/JMI.2019.2906835, Invited Paper.