BME researchers develop protection against the most sophisticated online scam

In a memorable scene from the 2006 film Mission Impossible 3, the voice of an arms dealer (Philippe Seymour Hoffman) is copied in a matter of minutes, and then the voice of the masked protagonist (Tom Cruise) is manipulated in real time by a computer to sound as if he were speaking. Back then, this might have seemed like crazy sci-fi to the average viewer:

Twenty years on, certain software programs can now produce fake live videos. Even an American president may be faked to say things he would never say, practically without any delay:

This form of deepfake technology is causing IT security professionals a major headache, because the more things we arrange online, the greater the scope for fraud. This is why the Deepfake Guard platform, jointly developed by researchers at the Department of Automation and Applied Informatics at BME and TC&C Kft., is so important, as 

it can detect voice and video-based deepfake manipulations in real time during live communication.

Think of situations such as opening a bank account or taking out insurance, where the service provider needs to identify the customer to prevent fraud. Attempts of abuse of this kind have already been documented in Hungary, some of them successful, carried out using tools available to anyone on the dark web. Obviously, some technological knowledge is required to use them, but the sellers act as professional service providers, offering customer support to criminals.

"It is important that the verification is real-time, so that the live video connection cannot be faked with artificial intelligence-generated sound and images. Here, AI is actually fighting against AI, as the deepfake detector is also an AI-based algorithm, which means it is constantly learning," said Bertalan Forstner, associate professor at BME, who worked on the project together with associate professor Bence Kővári and doctoral student Gergő Békési.

The managers of TC&C – with former BME teachers among them – approached the department two years ago, having realized that the biggest challenge for their main product, a call center application, would soon be to ensure that the person in front of the camera is who they say they are. They came to the right place with their idea. "We have been working with generative technologies for a long time. In fact, even the most complex domestic research in the field of handwritten signature authentication is also being conducted here, a knowledge the Hungarian police have already seen the benefits of," said Bertalan Forstner.

The result of the research is a system that monitors sound and image separately. The two reinforce each other, however, the software does not make decisions itself, but performs real-time analysis to assess risks and provide feedback to the operator. Its accuracy exceeds 98 percent in case of known methods.

The defense also includes solutions that function as a kind of captcha, such as asking the person being called to cover their mouth with their hand. One of the common weaknesses of deepfake generators is that in such cases, the mouth appears faintly on the back of the hand (as shown on the right side of the image pairs below) – and it is immediately obvious that who we see is only an avatar generated from previous recordings, behind which the real speaker is hiding.

This technology can be useful not only for banks, but also for any large company, even for internal communication. "There was a scam in Hong Kong where the the email address of a company's financial manager's assistant was obtained, invited to an online meeting by the scammers, and instructed to transfer large sums of money to certain accounts. Valuable business secrets can also be stolen this way, and compared to the potential profits, these generator technologies are relatively inexpensive," explained Gergő Békési.

In other words, 

anyone who uses live video-based tools beyond their private conversations will probably need protection sooner or later.

 According to BME experts, business users will very soon expect such security features.

The next year will be devoted to fine-tuning, partly with the aim of making the algorithm run on smaller hardware, even on a laptop, so that the product can be attractive to average business users. There are no reliable competitors on the market at the moment. Microsoft Azure came up with something similar about three months ago, but it is not a mature, comprehensive solution either, according to Gergő Békési.

gp
cover image: Perchance.org