As part of the European research project LUMINOUS, a project partner meeting was recently held, bringing together the international partner organizations to present their research results and further plan the demonstrators and use cases. The main topics of discussion were new approaches to creating realistic avatars and controlling them via large language models. The meeting was hosted by the Fraunhofer Heinrich Hertz Institute (HHI), which is one of the key research partners actively involved in developing the innovative XR platform.
The project brings together leading research institutions, universities, and companies from across Europe:
• DFKI – German Research Center for Artificial Intelligence GmbH (Germany)
• Ludus Tech SL (Spain)
• Mindesk SRL (Italy)
• MindMaze SA (Switzerland)
• Centre hospitalier universitaire vaudois (Switzerland)
• University College London (UK)
• Fraunhofer HHI (Germany)
• University of the Basque Country – EHU (Spain)
• Vicomtech (Spain)
• University College Dublin (Ireland)
• Hypercliq IKE (Greece)
• Ricoh International B.V. – German branch
Visionary XR technology with language intelligence
The aim of the project is to develop a multimodal, language-based XR platform that adapts flexibly to individual user needs and unknown environments. The focus is on the integration of large language models (LLMs), which act as “oracles” and describe new tasks on demand. These are communicated via voice interfaces or virtual avatars – for example, as coaches or teachers – and supplemented by visual aids and procedural steps.
The platform is intended for use in a wide variety of areas in the future: from rehabilitation and immersive learning environments to design reviews, entertainment, and sports. The focus is on intuitive, situation-specific communication.
Three use cases in focus
The technologies are evaluated on the basis of specific application scenarios:
• A virtual therapist for the treatment of aphasia
• An e-learning system with a virtual assistant for fire safety aspects
• An architectural design review with virtual tours
The meeting served not only as a forum for scientific exchange, but also for strategic coordination on the implementation of the demonstrators. The partners were enthusiastic about the progress made and the potential of the platform for future XR applications.
Further information on the project can be found on the official LUMINOUS website.
