December 3, 2014

The OmniCam-360 in action: Recording the Berlin Philharmonic’s Concert Marking the Fall of the Berlin Wall

The OmniCam-360 developed by the Fraunhofer Heinrich Hertz Institute HHI is a high resolution 360° panoramic camera unrivalled anywhere in the world. The Berlin Philharmonic Orchestra were among the very first to recognize the sheer potential packed by this camera. After a series of productions, the camera has now been used to record the concert given by the Berlin Philharmonic on the occasion of the 25th anniversary of the fall of the Berlin Wall. For the broadcasting of such live events a beta version of an app for a ‘Pan & Scan' system has now been released.

Fraunhofer HHI’s compact OmniCam consists of ten HD cameras mounted on a mirror rig. Fed-in single shots are corrected in real-time and stitched together in a parallax-free video panorama boasting a resolution of approx. 2,000 x 10,000 pixels. The OmniCam measures around 50 x 50 centimeter and weighs some 15 kilos. For the concert marking the fall of the Berlin Wall, the OmniCam was suspended behind the conductor Sir Simon Rattle to capture his field of vision without, however, obscuring him. 

The aim of the OmniCam project is to introduce the camera system alongside regular systems as used at live broadcasting events. A special app developed in collaboration with a consortium of international partners is used to stream shots from the OmniCam in HD quality to a tablet which can be used as either as a second screen or main screen from which footage can also be broadcast on TV. By wiping and/or zooming, viewers can focus on that part of the picture that interests them the most. In broadcasts of concerts this could be the section of the orchestra with their favorite instruments, or the conductor, or they could just let their gaze wander.

To prevent overload on the ‘Pan & Scan‘ data lines, only selected footage is sent and not the complete panorama. Accordingly, the panorama is subdivided into a number of small tiles. For quick downloading, adjoining tiles are pre-loaded in low resolution and only loaded in full resolution when the wipe movement comes in their direction.

At the same time great importance is attached to the highest quality of sound recording. In cooperation with Sennheiser and Neumann, digital microphones were used for the first time in a concert given by the Berlin Philharmonic with the microphone system optimized for play-back on a range of 3D audio systems.

The Digital Concert Hall (DCH)

Since 2008 concerts by the Berlin Philharmonic can be seen and heard by people all around the world without having to visit the German capital. Each season some 40 DCH concerts are live-streamed on the internet and a few days after the performance stored in what is now a pretty extensive archive.

“In collaborating with the Digital Concert Hall our aim is not merely to give subscribers access to the TV edited version of the performance but with the app to enable them to freely chose and open up their own angles and views of the event,“ says the graduate engineer Christian Weißig, Image Processing project manager at Fraunhofer HHI.

Prior to the concert commemorating the fall of the Berlin Wall, the Berlin Philharmonic and Fraunhofer HHI had already used the OmniCam in the 2012 MusicTANZ project “Carmen“.