Search in large image collections is usually done by using keywords such as “beach“, “flower“ or “landscape“. The manual assignment of such keywords to images for image search, is however very laborious and inaccurate. The Fraunhofer Heinrich Hertz Institute HHI has developed technologies within the scope of the research program, THESEUS to automatically analyze image content and assign keywords to images.

The approach for automatically assigning keywords to images is divided into three steps

• Extraction of metadata
• Training
• Classification

In a first step, features (e.g. edges, color and geometric shapes) suitable to describe image content are automatically extracted from an image. For each category (e.g. “beach“) to be learned by the system, a training set of images is required, which contains positive and negative samples of the category.

In the second step, the system is trained with the features extracted from the training set, in order to be able to distinguish the learned category from other categories. The system is then able to determine if a previously “unseen“ image belongs to the learned category or not, by analyzing its features.