NeuroHum

The aim of this project is to enhance the perceptual and emotional realism in immersive media by neurocognitively-guided computer graphics. Particularly, together with the "Mind-Body-Emotion" and "Neural Interactions and Dynamics" groups of the Department of Neurology at the Max Planck Institute (MPI) for Human Cognitive and Brain Sciences we will investigate the perception of digital representations of humans and their emotions in order to develop new perception-driven hybrid methods for human modelling and animation. This project will lead to an improved realism of virtual humans by integrating human modeling (CG and immersive media) and neurocognitive response patterns (psychophysiology and cognitive neuroscience) in an unprecedented multi-dimensional and data-driven fashion. Besides these technological advances, an in-depth understanding of the complex interplay between naturalistic stimuli and human neurocognition promises to improve (1) the generalizability of effects found in labbased experiments, (2) the transfer of behavior learned in virtual training or therapy sessions to everyday life and - more generally - (3) our understanding of the mind, the brain and the body and their link to the experience of reality.

In short, to better understand the biology of the mind, we investigate the perceptual and emotional processing of realistic virtual humans in naturalistic experiments and by multimodally analyzing behavior, subjective experience, and neurophysiology. In turn, to systematically optimize the modeling, animation and rendering of virtual humans, we will make use of statistical and machine learning procedures that fuse technological with neurocognitive information.

In particular, our objectives are:

• We will analyze and relate relevant neurocognitive (i.e., neurophysiological and behavioral) features that covary with the perceived realness of virtual humans and their emotions
• We will develop new concepts for the mapping of this feature space onto the relevant dimensions in visual as well as computational, artificial spaces that underlie the generative approaches in the production of virtual humanss
• We will feedback this information into the model generation process and formulate specific loss functions based on the perceptual mode