Mobile Applications for 3D Video

Mobile 3D services have become popular recently, since high-quality stereoscopic display technology is available for handheld devices. As such devices are normally used by single users, the mobile 3D display technology is usually presenting two views for a fixed viewing position. Thus, no additional glasses are required and the underlying user interfaces and mobile applications generate stereoscopic images and videos. We investigate and provide stereoscopic solutions for different mobile devices, especially in the area of 3D video. Here, our focus in on compression efficiency for stereoscopic video content under the special requirements of mobile devices, such as limited hardware recourses.

Stereo Video Formats

  • CSV (Conventional Stereo Video): classical recording by stereoscopic cameras, which requires no additional processing, however causes the highest bit rate in 3D video coding
  • MRS (Mixed Resolution Stereo): sub-samples one view to save bit rate in compression process, requires up-sampling at the display side, overall perception is close to the sharp view, as the human brain can mask the blurring in the up-sampled view
  • V+D (Video plus Depth): requires depth information for one view and gives lower bit rates in compression, as depth data can be compressed more efficiently than video data, second view is synthesized at the display side by depth image-based rendering, enables baseline adaptation, in contrast to video-only formats

Coding Methods

  • Simulcast Coding (e.g. by MPEG-4 AVC/H.264): Each view is coded separately, can be applied to all formats, however requires the highest bit rate
  • Joint Stereo Coding (e.g. by MPEG MVC/H.264 or Stereo SEI Massage): Joint coding of both views, more efficient than simulcast coding, as dependencies between views are also exploited, can only be applied to video-only formats
  • V+D Coding (e.g. MPEG C part 3): Special container format for Simulcast coding for V+D, coding methods are the same as simulcast coding, only signaling is applied to identify the different components

Publications

[1] A. Gotchev, D. Strohmeier, K. Müller, G. Bozdagi Akar, and V. Petrov, “Source and Channel Coding Recipes for Mobile 3D Television”, Coding for Mobile Applications”, Proc. International Conference on Digital Signal Processing, (DSP 2011), Corfu, Greece, July 2011.

[2] F. Rhee, P. Merkle, and K. Müller, “A Generic 3D Video Codec Framework for Mobile Applications”, Proc. 3DSA 2011, International Conference on 3D Systems and Applications, Seoul, Korea, June 2011.

[3] H. Brust, A. Smolic, K. Müller, G. Tech, T. Wiegand, “Mixed Resolution Coding of Stereoscopic Video for Mobile Devices”, Proc. IEEE 3DTV Conference, Potsdam, Germany, Mai 2009.

[4] P. Merkle, Y. Wang, K. Müller, A. Smolic, and T. Wiegand, “Video plus Depth Compression for Mobile 3D Services”, Proc. IEEE 3DTV Conference, Potsdam, Germany, Mai 2009.

[5] P. Merkle, H. Brust, K. Dix, K. Müller, and T. Wiegand, “Stereo Video Compression for Mobile 3D Services”, Proc. IEEE 3DTV Conference, Potsdam, Germany, Mai 2009. [6] G. Tech, A. Smolic, H. Brust, P. Merkle, K. Dix, Y. Wang, K. Müller, and T. Wiegand, “Optimization and Comparison of Coding Algorithms”, Proc. IEEE 3DTV Conference, Potsdam, Germany, Mai 2009.