Assistive Artificial Intelligence and Augmented Reality Tools for Microscopic Diagnostics in Dermatology and Microscopic Surgery
July 2023 - January 2025
Within the Fraunhofer PACT project “AI/AR Microscope”, we aim to develop explainable intelligent tools for microscopy in diagnostics and surgery, powered by artificial intelligence (AI) and augmented reality (AR).
To achieve this vision, we will develop a modular toolbox comprising AI models for the classification of (dermatoscopy, histopathology, and surgical) microscopy images, and latest explainable AI techniques that can help health practitioners to better understand the resulting AI decisions. Additionally, we will develop AI-incorporated AR software components capable of processing and visualizing the microscopic views in real-time (e.g., to aid during surgery with an overlay of tissue type and navigation information).
We expect that leveraging the great potential of novel AI and AR tools for real-time analysis and visualization of relevant information from digital microscopy data will eventually translate into better outcomes for patients. The planned modular toolbox will be presented in various demonstrators for technology evaluation.
The AI and VIT departments of Fraunhofer HHI are collaborating with Fraunhofer USA Center Mid-Atlantic CMA in this research project. Fraunhofer HHI has relevant expertise in AR and in the field of AI/ML for image-based diagnostics in medicine. Fraunhofer USA CMA has expertise in software testing, specifically toward the development and evaluation of reliable AI–based software and can therefore evaluate the robustness of the system.
Project Partners
- Fraunhofer USA Center Mid-Atlantic CMA