Research Topics of Immersive Media & Communication Group

3D Human Body Reconstruction for eXtended Reality (XR)

New applications in media and virtual reality require creation of realistic 3D models of humans, which move and behave naturally. Current computer-generated avatars do not offer the necessary level of realism. We research on 360° multi-view capture systems being able to acquire dynamic objects and to compute realistic and naturally moving 3D models.

Read more

Volumetric Video Production

Volumetric video will become the new data format for immersive eXtended Reality applications. Since 2016, when we started with the first capture systems for full body capture, our algorithms and workflows have been evolved significantly. Several XR experiences have been produced in order to show the progress and capybility of this new technology.

Read more

Real-time 3D scene reconstruction

Real-time low delay 3D scene reconstruction is increasingly important for many fields of application such as tools for content creation and post production, 3D videocommunication or photo-realistic rendering for mixed-reality applications. Our expertise in graphics card programming allows us to realize a variety of approaches that require low delay and real-time.

Read more

Immersive video communication

We are experts with many years of experience in the field of immersive video communication, working on concepts, systems and algorithm development since more than 15 years. This includes 2D videocommunication supporting direct eye contact, 3D rendering on autostereoscopic displays, single and multi-user videocommunication scenarios supporting eye contact and correct gesture representation as well as large size wall display video communication.

Read more

Virtual eye contact

We developed a real-time multi-view video processing module for video conferencing systems offering direct eye contact between the remote conferees. Based on at least stereo cameras mounted on top of a display the 3D shape of the person is computed. Depending on the viewing position of the local participant a virtual view of the remote participant is rendered in order to provide direct eye contact.

Read more

Pose and gesture analysis

We research and develop fast and reliable algorithms for detection and tracking of whole body movements as well as movements of specific body parts. Our solutions focus on body, head, eye, hand and finger tracking as well as posture, movement and gait analysis. Our algorithms are based on non-intrusive, marker less sensors which provide 2D and/or 3D video signals, e.g. colour, depth-from stereo, time of flight or structured light. These algorithms are designed to work with a variety of cameras from low cost, mobile to high-end devices finding a best compromise between performance and price, from offline behaviour analysis to real-time human-computer interaction.

Read more

Behaviour analysis for human-computer interaction

Based on our detection and tracking algorithms as well as additional novel input sources such as voice control, we create novel human computer interaction modules. This involves calibration, fusion and interpretation of various data in a specific context. These modules are used in various applications such as behaviour analysis for ambient assisted living solutions, interaction with applications and media content as well as novel video communication applications.

Read more

Contact-free human-computer interaction

We believe that in the near future traditional computer input devices like keyboard and mouse are going to be supplemented or replaced by more natural interfaces. Applying usability engineering methods and the user centred design approach, we develop novel concepts for contact-free human-computer interaction that allow users to control any application or system by using free-form hand, finger or whole body movements and gestures. If suitable and beneficial, we include additional input technologies such as voice control to improve user experiences in specific scenarios. Applying our interaction modules various solutions in the field of medical, automotive, digital signage, cultural heritage or education have been developed that prove the capability of these interaction concepts.

Read more

3D user interfaces

Regarding the enormous potential of virtual environments, the development of appropriate ready-to-use graphical components is the key to successfully make 3D multimedia contents accessible to a large public. We have a long experience in designing novel graphical interfaces and adding navigation metaphors. Furthermore, we propose techniques to display AR scenes onto stereoscopic, autostereoscopic and panoramic screens. Through our 3D framework, we develop sustained UI components and integrate them into awesome applications.