Interactive Media - Human Factors mUltimo3D

Project description
In this project, advanced human-computer interaction technologies are developed, implemented and tested with users. A 3D graphic user interface, where visualized data objects float in front of a 3D display, is combined with a range of input devices for multimodal interaction. Video-based systems allow users to interact via hand gestures, eye and gaze movements.

Objectives

• To develop an autostereoscopic 3D display (no stereo glasses required) which supports the natural link of accommodation and vergence of the human eyes. In particular, the optical design aims to allow direct manipulation of visualized data objects floating in front of the display.
• To develop user-friendly technologies for sensing the user's head location and gaze point as well as the hand position
• To develop a 3D operating system with intelligent interface agents supporting multimodal interaction with conventional input devices (keyboard, mouse and speech recognition) and video trackers (eye, gaze and hand tracking).
• To evaluate and optimize the interplay of the various modalities with users.

Project data
Supported by: Bundesministerium für Bildung und Forschung (BMBF)
Project period: closed

Publications

• J. Liu, S. Pastoor, K. Seifert and J. Hurtienne: Three-dimensional PC: toward novel forms of human-computer interaction". Proceedings SPIE Photonics East Critical Review Conference on Three-dimensional Video and Display, 5-8- November, Boston, 2000 (invited paper).
• K. Schenke and S. Pastoor: "3D-Teleworking-Arbeitsplatz der Zukunft", Heidelberger Klub, Tagungsband 2000.
• S. Pastoor, J. Liu and S. Renault: An Experimental Multimedia System Allowing 3-D Visualization and Eye-Controlled Interaction without User-Worn Devices. IEEE Transactions on Multimedia, Vol. 1, No. 1, pp. 41-52, March 1999.
• Sheng-Wen Shih, Yu-Te Wu (Chi Nan University Taiwan), Jin Liu: "A Calibration-Free Gaze Tracking Technique", ICPR 2000, Sep 3-8, 2000, Barcelona.
• J. Liu, D. Przewozny and S. Pastoor: "Layered Representation of Objects Based on Multiview Image Analysis." IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, June 2000, Volume 10, Nummer 4. pp. 518-529.
• S. Pastoor: "3-D Displays and Video Trackers Ease Computer Operation", EuroPhotonics, Photonics Tech Briefs, October/November 1999, pp36-39.
• S. Pastoor and J. Liu: Augenmaus & Co. - Entwicklung einer neuartigen Anwenderschnittstelle für Multimedia-Computer. In: M. Rütting and
  K. Seifert (Ed.) Blickbewegungen in der Mensch-Maschine-Systemtechnik Pro Universtate Verlag, 1999).
• S. Pastoor: 3D-Multimedia-Computer. Dokumentationsband zum Münchner Kreis Kongreß "Anwenderfreundliche Kommunikationssysteme", 16. Und 17. Juni 1999, Europäisches Patentamt München.
• J. Liu, D. Przewozny und S. Pastoor: Depth Layering of Objects for Multimedia Applications. WIAMIS'99, Workshop on Image Analysis for Multimedia Interactive Services Proceedings, 31. Mai - 1. Juni 1999, Berlin.
• S. Pastoor, J. Liu, E. Feddersen, V. Tietje: "Messverfahren zur Ermittlung der Position eines Objektes vor einem Bildschirm und Vorrichtung zur Durchführung des Verfahrens mit Sensorzeile". Az. 100 56 244.3, Patent angemeldet: 19.11.2000.
• J. Liu und S. Pastoor: "Videobasierte Blickrichtungsbestimmung mit initialer Referenzkalibrierung und selbstabgleichender, anwenderindividueller Rekalibrierung", Az: 199 53 835.2, Patent angemeldet: 30.10.99