Low-Latency Processing of 3D Video Data
The flexible Controller System for autostereoscopic 3D displays developed by the Fraunhofer Heinrich Hertz Institute is based on a novel FPGA hardware platform with optimized implementations (IP cores) that enables processing of high definition 3D video data in real-time. This Fraunhofer HHI technology enables low latency, high data rate transmission and dynamic intermediate processing of digital video signals from PC and camera interfaces.
The Fraunhofer Heinrich Hertz Institute has developed this Controller System especially for 3D displays with glasses-free three dimensional rendering. Lens-based and barrier raster techniques are used to give each of the viewer’s eyes its own perspective, while control of individual pixels means that image data can be assigned dynamically to the position of the left or right eye.
The stereo camera integrated in the 3D display films the viewer. A separate downstream processing unit tracks the viewer’s eyes and reports the position of the viewer in front of the screen to the 3D Display Controller. The distribution of the individual pixels in the input signal (like that of a 3D content camera) has been re-computed and – in line with the viewer’s position and the raster separation model – rendered on the screen as an interleaved video signal. This is synchronized with the image written on the display panel. If the viewer changes his position, the image information is aligned to the positions of the subpixels that become visible from the position of the eyes.
A Full HD 3D picture running with a 60 Hz display refresh rate requires a data throughput rate of up to 20 gigabit a second. As traditional PC systems cannot handle such high data rates in real-time, the controller system uses a powerful FPGA-based hardware platform. The system can be integrated in the display or used as a set-top box.
Exploiting the possibilities of autostereoscopic displays and 3D processing with real-time capability opens the way to novel types of visualization, for instance in the fields of gaming, or production, automation and automotive technology. Various interfaces and 3D image formats enable the rendering of video streams from both computers and cameras.
Areas of Application
- Visualization of spatial constructions (e.g. in mechanical engineering)
- 3D views of inaccessible places
- 3D microscopy (e.g. for material analysis)
- Virtual production
- 3D displays for virtual prototyping
Benefits of the 3D Controller System
- 3D signal conditioning and processing for autostereoscopic 3D displays in real-time
- Freedom of movement through rapid adaption to the viewer’s position
- Integration of complex signal processing stages in a lean system
- Excellent response speeds to time-critical applications
- Cuts back on hardware and power consumption
- Standards-compliant display and camera interfaces
- Rendering of HDMI 1.4a compliant video data