Deepfake Video Detection Using Biologically Inspired Geometric Deep Learning

Deepfakes are synthetic media in which a person’s appearance is altered to look like another’s. In order to identify them we trace the face by employing a 3D manifold obtained from Google’s open-source MediaPipe library. It provides us with a local topology of the face that is invariant to any homeomorphic transformation across consecutive frames. We measure the local muscle movement in the face using a heatmap of correlations between pixels in the corresponding frames. These unique biological characteristics are projected to the canonical face mesh and used as features in a 3D Convolutional Neural Network to quantify  the authenticity of the face on the video. The proposed model is trained on publicly available databases of manipulated and real videos, including FaceForensics, FaceForensics++, and Deep Fakes Dataset, with more than 500 gigabytes of data available.  

The paper is accompanied by a web application with the frontend written in React JS, backend in Python using Flask with Nginx as the front-facing reverse proxy, and Gunicorn to serve the Flask app. The trained model will be integrated with the web application and deployed on Stony Brook University servers as a publicly available application that allows users to analyze their own videos and label them as fake or real.