Trustworthy AI

Benchmarking and evaluations are essential components in the assessment of quantitative aspects of Trustworthy Artificial Intelligence. These activities serve as a stepping stone towards responsible deployment of AI applications. Given the fast-paced development of methods and the wide variety of implementations in the modern AI landscape, research at Fraunhofer HHI focuses both on specific application-level evaluation and generically applicable evaluation frameworks. Our activities prioritize the requirements of the conformity assessments prescribed for high-risk applications as defined by the European AI Act, especially in the healthcare domain.

To successfully implement AI applications, we need more than just good algorithms. It is also important to consider the needs of the people who use them. This is where Human-Centered AI comes in: Here we examine user requirements and adapt AI systems to make them understandable, reliable, and trustworthy. Together with partners from industry, healthcare, and the public sector, we develop user-friendly, interactive AI solutions that meet the demands of real-world practice.

For the trustworthy deployment of AI, it is essential to make uncertainties in predictions transparent. Our Applied Machine Learning Group is successfully researching methods for quantifying and disentangling different sources of uncertainty, such as aleatoric (data-related) and epistemic (model-related) uncertainty. This distinction enables better risk assessment and helps make AI systems more reliable and practical. In this way, we contribute to ensuring that AI applications are not only powerful but also transparent and robust when used in real-world scenarios.