June 8, 2021
The VR research project "NeuroHum" (Neurocognitively-guided modeling of virtual humans for enhanced realism in immersive media) started with its first project phase in early May. In the project, the "Vision and Imaging Technologies" department of the Fraunhofer Heinrich Hertz Institute (HHI), together with the "Mind-Body-Emotion" and "Neural Interactions and Dynamics" groups of the Department of Neurology at the Max Planck Institute (MPI) for Human Cognitive and Brain Sciences, is researching the perception of virtual humans and feeding the findings back into modeling and synthesis. The project is scheduled to run for four years.
Today, virtually depicted humans are an essential part of numerous application areas, for example movies, games or human-machine interfaces supported by computer graphics (CG). This is not only the case for traditional on-screen representations, but also for immersive interactive environments such as virtual (VR), augmented (AR), or mixed reality (MR). Creating realistic, believable, and emotive digital representations of people is still one of the most challenging tasks in CG, especially for real-time and interactive use cases. However, the mentioned features are essential for a variety of applications with socio-emotional components, such as trainings or therapies that involve other (virtual) humans or digital companions (e.g., for elderly people).
Current models of artificial humans are either hand-crafted or based on statistical assumptions. They do not systematically and objectively incorporate neurocognitive effects that virtual humans have on the user. More recently, we saw the introduction of photorealistic avatars reconstructed from real world footage or synthesized using novel generative deep learning approaches (neural animation, deep fakes). The key challenge in representing virtual humans is to avoid the "Uncanny Valley effect". This phenomenon describes that highly naturalistic virtual humans, which are still recognizable as artificial, can evoke uncomfortable feelings in viewers.