Immersive Multi-User and Multi-Party 3D Video-Converence System

Based on the European FP7 project 3DPresence we will present a multi-party, high-end 3D videoconferencing demonstrator prototype. We address the problem of transmitting the feeling of physical presence in real-time to multiple remote locations in a transparent and natural way.

Modern computer games pose high demands on CPU power and are usually only available on high end PCs and game consoles. In order to enable ubiquitous gaming in a home environment, hotel, or café, an interactive framework for remote gaming has been developed within the European project Games@Large.

In this system, 3D graphical applications like games are executed on a server while the visual output is streamed to clients like laptops, settop boxes, or mobile devices. Both 3D graphics and video streaming optimized for low delay transmission of synthesized content is supported.

We show a system for Virtual Clothing in which a user can view himself wearing virtual clothes in a virtual fitting room environment without the need to actually try them on.  A single camera captures a moving person and the elastic deformations and illumination of his clothes are estimated. An arbitrary texture is augmented onto the moving garment such that the person seems to wear the virtually textured piece of cloth.

The Stereoscopic Analyzer STAN is a system for the production of perfect stereo 3D. Developed by the Fraunhofer Heinrich Hertz Institute, Berlin, in co operation with KUK Film Production, Munich, STAN’s combination of hardware and software captures and analyses stereo pictures, enabling them to be processed in realtime. STAN combines real-time image analysis with intelligent automated tools and intuitive graphical user interfaces to assist camera operators and production staff in shooting the right stereo content for 3D postproduction and 3D live events.

Format conversion from stereo to display specific multiview formats is an essential post-production process for future 3D services allowing the reproduction of stereo content on a wide range of 3D displays. Dedicated conversion can be provided for most existing autostereoscopic displays.

3D television is seen as the next major step in home entertainment. The cinema and computer games industries have already shown that there is considerable public demand for 3D content. But in contrast to 3D cinema and 3D gaming, 3D-TV must be able to serve a multitude of different 3D displays ranging from conventional stereo reproduction to innovative auto-stereoscopic multi-view displays that allow for watching 3D-TV content without using special glasses. This requires the development of new 3D production techniques as well as flexible and generic 3D-TV distribution systems.

The HHI presents "Tomorrow's Cinema" as a cylindrical panorama projection. The 180° screen in combination with 3D-sound leads to a perfect immersive impression. Key component for the system is the CineCard, a hardware based solution for multi projection. Seven HD projectors, each with 3.000 ANSI-Lumen makes possible to provide a resolution of about 7.000 x2.000 pixel.
The 3D-sound is realized by an IOSONO-sound system, developed by Fraunhofer IDMT.
The most of the content is generated with a special panorama camera, the OmniCam also presented on site.

The HHI is currently working on a multi-camera rig that can be used for the capture of immersive, high-resolution video panoramas. The scaleable rig is designed for use with HD cameras; each individual view provides an opening angle of 30 degrees horizontally and 53.3 degrees vertically (portrait format). The presented version of the rig is equipped with six cameras and provides panoramas with about 6.000 x 2.000 pel. A full circle rig for up to 12 HD cameras allows up 360 degree capturing capability and a resolution of about 12.000 x 2.000 pel.

We show a 3D video transmission based on the joint ISO/IEC and ITU-T standard for  Multiview Video Coding (MVC).  Each view is transmitted over an individual RTP network flow. This system allows for unequal error protection, prioritized retransmission and aggregation of views in the network. In case of transmission errors or low channel capacity, the number of transmitted views can be reduced down to the base view which is MPEG4-AVC/H.264 backward compatible.