Detection of Face Morphing Attacks

Identification tasks with identity cards, passports and visa are often performed automatically by biometric face recognition systems. Criminals can trick these systems, such that two people can use the same passport for authentication. This attack (morphing attack) is performed by fusing two face images to a synthetic face image that contains characteristics of both people. Using this image on a passport, both people are authenticated by a biometric face recognition system.

To generate a face morph, we align two face images, such that each facial feature (e.g. left corner of the right eye) is at the same position in both images. Since the haircut and the ears are difficult or even impossible to align and usually ignored by commercial facial recognition systems, we blend only the face region and stitch it into one of the aligned input images. The transitions are calculated individually for different frequency bands and an optimal stitch line is estimated.

In order to decide whether a face image is authentic or crated by a morphing algorithm, we trained deep convolutional neural networks on morphed and original face images. Since we focus on semantic image content, like highlights in the eyes or the shape and appearance of facial features to determine the authenticity of a face image, we applied several preprocessing steps on our training data to remove sensor or camera specific information. In addition to a decision about the authenticity of an image, we are also interested in a reason for decision. For this purpose, we analyze the regions in the face image that were relevant for the networks decision using the LRP Toolbox. This might also be useful when involving humans to the decision making. It helps to find suspicious regions in a face image that needs to be checked by a human as well as it helps to identify and classify artefacts that were created during a morphing process so that experts can be trained to identify them.

Publication

C. Seibold, W. Samek, A. Hilsmann, P. Eisert
Detection of Face Morphing Attacks by Deep Learning, Proc. 16th Int. Workshop on Digital Forensics and Watermarking (IWDW2017), Magdeburg, Germany, August 2017.

S. Lapuschkin, A. Binder, G. Montavon, K.R. Müller, W. Samek
Analyzing classifiers: Fisher vectors and deep neural networks, Proc. Conf. on Computer Vision and Pattern Recognition (CVPR), 2016.