How does an AI deepfake work?

Face Analysis

The first step in the deepfake process involves using the Face Analyser. This component of the software scans the source video or image to detect any faces present. It identifies facial features, contours, and landmarks that will be crucial for the subsequent steps.

Creating a Face Reference

Once a face is detected and analyzed, a Face Reference is created. This is essentially a snapshot or a detailed profile of the face, capturing all its unique features. This reference will be used to recognize the same face in other frames or videos.

Face Recognition

Now, when we have a target video or image where we want to superimpose the source face, the Face Recognition process begins. This involves comparing the faces in the target video with the Face Reference from the source. The comparison is based on the Face Distance metric. If the Face Distance between the target face and the Face Reference is below a certain threshold, it indicates that the faces are similar or identical. The lower the Face Distance, the more similar the two faces are.

Deepfake Generation

Once a match (or close match) is found using Face Recognition, the AI begins the process of superimposing the source face onto the target face. This involves aligning facial landmarks, adjusting for lighting and angle differences, and ensuring the face looks natural in the new setting.

Advanced deepfake algorithms will also consider facial expressions, emotions, and movements to ensure the result is seamless.

Addressing Face Bouncing

One challenge during the deepfake generation process is Face Bouncing. This refers to the flickering or bouncing between faces, especially if the Face Recognition process fails or is inconsistent across frames.

To address this, the software needs to have robust error-handling mechanisms. If the Face Recognition process detects a Face Distance that's above the threshold in certain frames, causing bouncing, the software might:

Re-analyze those frames to ensure accurate face detection.

Use data from adjacent frames to predict and correct the face in the problematic frame.

Apply smoothing algorithms to reduce noticeable flickers or jumps.

Finalization

After processing all frames and ensuring a consistent and realistic face swap, the deepfake video or image is finalized. It's then ready for viewing, sharing, or further editing.