Virtual worlds for digital diagnostics and cognitive rehabilitation

VR input and output technologies for the implementation of clinical diagnostic and rehabilitation components

VReha project page

Duration: December 2017 - November 2019

Sponsored by BMBF

Background and overall objectives

Immersive technologies such as Virtual Reality (VR) or Augmented Reality (AR) are currently experiencing rapid development. Almost every household will have VR and AR systems in the foreseeable future. This opens up completely new possibilities for diagnostic and therapeutic applications in medicine. The aim of the present project is to use these possibilities for diseases with impairment of cognitive functions (dementia, stroke).

In comparison to conventional methods, the diagnosis of cognitive impairments by using realistic tests is to become significantly more sensitive and precise; this way, restrictions relevant to everyday life can be better recorded ("ecological validity"). In connection with medical diagnostics, a better and standardised early detection of cognitive disorders (e. g. in the early stages of dementia) is expected. VR-based individual disorder profiles are created, which can be used to determine and implement VR-based outpatient therapy programs (see Figure 1). In this way, neuropsychological and cognitive disorders can be treated better and closer to everyday life.

In the VReha project, a configurable, app-based toolbox is to be developed as a prototype, which can be used to supplement existing procedures by means of VR-based hardware and be used for digital diagnostics and an adapted rehabilitation of cognitive impairments.

Motivation and goal of the HHI sub-project

New, user-friendly and convincing applications must be developed for a broad introduction of VR technologies in the field of diagnostics and rehabilitation. Decisive new and further developments are necessary, especially in the interaction in virtual scenes as well as in visualization.

In this subproject, Fraunhofer HHI will provide new basic technologies for VR input and visualization, which will be developed into a prototype VR application in cooperation with the partners of the consortium. This VR Tool-Box will then be the basis for the new digital diagnostic and rehabilitation procedures of the clinical partners.
Development will focus on the following two scientific and technical areas:

  1. new contactless interaction technologies and
  2. realistic visualization of self-representation.


Intuitive touchless control and navigation is required for easy, user-friendly interaction with virtual scene objects and content. Show gestures and body poses should allow the user to move naturally in the virtual scene in order to successfully perform the tasks of diagnostics and rehabilitation.
An essential problem of current VR representations is the missing visualization of the user himself. The aim is to develop new methods that capture the user's 3D geometry and generate a realistic 3D model of the person for visualization.

VReha project consortium

  • Fraunhofer HHI
  • Max-Planck-Institut für Kognitions- und Neurowissenschaften
  • Charité – Universitätsmedizin Berlin Humboldt-Universität zu Berlin, Klinik für Neurologie mit Experimenteller Neurologie
  • Universitätsklinikum Leipzig AöR
  • Klinik für Kognitive Neurologie




  • ICVR_Poster_VReha_HHI_2019.pdf 1 MB

    HHI Poster presented at the ICVR 2019 conference on the topic of natural and precise hand.object interaction in VR