We develop modern computer vision and AR-based technologies for medical applications. We apply state-of-the-art image and video analysis methods, e.g. for intraoperative assistance systems. Current work includes real-time 3D measurement for implant fitting, hyperspectral-based tissue analysis, videobased blood flow and vital sign monitoring as well as surgical instrument detection and phase recognition. Additionally, also human-machine interaction and human-in-the-loop plays an important role.
Current solutions cover the entire surgical pipeline.
Intraoperative AR/MR Tools
We work on interactive and multimodal-based augmented reality (AR) systems for computer-assisted surgery in the context of ear, nose and throat (ENT) treatment. The processing pipeline uses fully digital stereoscopic imaging devices, which support multispectral and white light imaging to generate high resolution image data, and consists of several AR and mixed-reality (MR) modules with the ability of remote communication.
Camera-based analysis of vital parameters and local blood flow
The extraction of heart rate and other vital parameters from surgical video recordings has attracted much attention. We develop tools for the extraction of global vital parameters, such as heart and respiratory rate, or even locally resolved perfusion intensity and blood flow from remote photoplethysmography (rPPG) signals. Applications range from intraoperative assistance during transplantation to patient monitoring for contactless intensive care services. Read more.
Hyperspectral Capturing and Intraoperative Tissue Analysis
We develop and investigate several hyperspectral camera setups to monitor the different optical behavior of tissue types in vivo. Especially, we address the automatic spectral examination of human tissues using multispectral imaging with promising potential for automatic visualization during surgery. The aim of this work is to collect and analyze behaviors of specific tissues and organs to use that knowledge for intraoperative guidance. For differentiation and visualization, different model-based as well as artificial methods are investigated to open up optical opportunities during surgery. Read more
Surgical Phase Recognition
We develop vision-based methods for automatic surgical phase recognition and workflow analysis from intraoperative video data. By combining temporal video analysis, instrument usage patterns, and contextual scene understanding, our approaches enable a reliable identification of surgical phases. Applications include intelligent operating room support, automated documentation, workflow optimization, and quality assessment, as well as the foundation for context-aware decision support and human-in-the-loop systems in surgery.
Projects
