Earlier this week there was news of a mother who allegedly used deepfake photos in a bid to get members of her daughter’s cheerleading team kicked off of the team.
It’s unclear how good those deepfakes were, but a new tool developed by scientists at the University at Buffalo can reportedly detect deepfakes with 94 percent effectiveness when viewing portrait-like photos.
It does this by examining the reflections in the eyes of the subject.
“The cornea is almost like a perfect semisphere and is very reflective,” explains lead author of the paper and Empire Innovation Professor of Computer Science and Engineering at the University at Buffalo, Siwei Lyu. “So, anything that is coming to the eye with a light emitting from those sources will have an image on the cornea.”
Most importantly, the two eyes will have similar reflections.
Where artificial intelligence stumbles is in generating those reflections accurately. The scientists behind the paper say that this could be due to the large number of photos used to generate a deepfake.
The tool developed by Lyu and his team maps a deepfake image and then examines the eyes, then the eyeballs and lastly the light reflected by each eyeball. The team will present its findings at the IEEE International Conference on Acoustics, Speech and Signal Processing to be held in June in Toronto, Canada.
The research paper linked to this study titled Exposing GAN-Generated Faces Using Inconsistent Corneal Specular Highlights is available here as a PDF. As an aside, a GAN is a generative adversary network which is often used in the creation of deepfake images.
As much as this tool fills us with hope, the way Lyu is spotting deepfakes can easily be circumvented. For instance, a person could just fix the reflections in the eyes during editing. GANs could be trained to apply similar reflections to eyes. The key is that the tool is only looking at the individual shape of the pixels in the image not the shape of the eyes or even what is being reflected.
While it’s not perfect it’s good to see Lyu continuing his work in trying to make it easier to spot deepfakes.
In 2020 he assisted Facebook in its deepfake detection challenge and he created the Deepfake-o-meter which can help folks identify if a video is fake.