AI-Powered Scams: How Deepfakes, Voice Cloning and Generative AI Are Changing Fraud
Artificial intelligence is changing the economics of fraud.
Many scams still rely on familiar techniques: impersonation, social engineering, urgency, deception and the exploitation of trust. What is changing is the technology available to criminals.
Generative artificial intelligence can produce convincing text, images, audio and video at speed. It can help criminals imitate trusted individuals, create synthetic identities, improve fraudulent communications and potentially operate some scams at greater scale.
The UK's National Assessment Centre has assessed that criminals will increasingly adopt generative AI technologies such as deepfakes, large language models and voice cloning to enable fraud. Importantly, the technology is currently being used largely to enhance existing fraud threats rather than invent completely new categories of fraud.
That distinction matters.
The fundamental psychology of many scams remains familiar. AI can simply make the deception more convincing.
 |
| Deepfakes, voice cloning and generative AI are making impersonation scams more convincing, scalable and difficult to detect. |
What is an AI-powered scam?
An AI-powered scam is fraud in which artificial intelligence is used to create, automate, personalise or strengthen some part of the deception.
The AI itself is not necessarily the fraud.
Instead, it may become another tool available to the offender.
For example, AI could potentially be used to:
- generate convincing phishing messages
- imitate someone's voice
- create realistic fake images or video
- construct synthetic online identities
- translate fraudulent messages into multiple languages
- personalise social-engineering approaches
- generate content for fake websites or investment promotions
- help scale communications across large numbers of potential victims
This can reduce some of the traditional barriers to producing convincing fraudulent material.
Deepfakes and the problem of digital trust
Deepfakes are artificially generated or manipulated images, audio or video designed to appear authentic.
Their significance for fraud goes beyond whether a fake video looks impressive.
Deepfakes challenge a basic assumption people have historically relied upon: seeing or hearing someone can help establish that the person is genuine.
That assumption is becoming less reliable.
A fraudster may potentially impersonate a company executive, celebrity, family member or another trusted individual using synthetic media.
The UK Government describes deepfakes as a rapidly evolving threat and estimates that around eight million deepfakes were shared in 2025, compared with approximately half a million in 2023.
Not every deepfake is fraudulent, of course. But the technology creates significant opportunities for impersonation and social engineering.
Voice cloning and impersonation fraud
Voice cloning is particularly concerning because telephone conversations often create an immediate sense of trust.
A familiar voice can make a request seem authentic.
AI systems can generate synthetic speech designed to imitate a particular person's voice. A criminal could then combine that capability with information gathered from social media, compromised accounts, previous communications or other sources.
Imagine receiving an urgent call apparently from a family member:
They have lost their phone.
They are stranded somewhere.
They need money immediately.
And the voice sounds exactly like them.
The UK's Stop! Think Fraud campaign specifically warns that criminals can use AI voice cloning to impersonate someone a victim knows or trusts and create an urgent request for payment.
The technology strengthens a technique fraudsters have used for years: creating emotional pressure before the victim has time to verify the story independently.
Deepfake fraud can also target businesses
Individuals are not the only potential targets.
AI-enabled impersonation creates serious risks for organisations, particularly where employees have authority to make payments or disclose sensitive information.
One widely reported case illustrates the potential scale.
The UK's National Assessment Centre describes a 2024 payment-diversion fraud in which generative AI was used to create deepfake recreations of company employees during a virtual meeting. A finance employee was deceived into transferring approximately £20 million to accounts controlled by criminals.
The lesson is bigger than that individual case.
Traditional business controls may assume that recognising a senior colleague's face or voice provides meaningful verification.
Increasingly, it may not.
Organisations therefore need processes that verify important instructions through independent channels rather than relying solely on what someone appears or sounds like during a call.
Generative AI and phishing
Phishing has traditionally contained clues that attentive recipients could sometimes identify: poor grammar, unusual phrasing, inconsistent formatting or generic language.
Generative AI can reduce some of those weaknesses.
Large language models can produce fluent, professional and contextually convincing messages quickly.
The National Assessment Centre assesses phishing as likely the most prevalent initial attack method used by organised crime groups against UK individuals and businesses. It also assesses that LLMs used alongside phishing tools are highly likely to be contributing to increased volume and sophistication.
That means advice such as “look for spelling mistakes” is no longer sufficient on its own.
A perfectly written message can still be fraudulent.
AI and romance fraud
Romance fraud depends heavily on emotional manipulation and the creation of a believable identity.
Synthetic media can potentially strengthen both.
A fraudster may use AI-generated photographs, altered video, synthetic voices or automated conversations to make a fictional identity appear more convincing.
This can make traditional verification techniques less reliable.
A video call, for example, may once have provided reassurance that an online identity corresponded to a real person. Advances in face-swapping and synthetic-video technology mean even that evidence requires greater caution.
The wider principle is important: digital familiarity should not automatically be treated as proof of identity.
Synthetic identities
Generative AI also intersects with another important area of financial crime: synthetic identity fraud.
A synthetic identity can combine genuine and fabricated information to create an apparently legitimate person.
AI-generated photographs, documents and supporting information can potentially make such identities more convincing.
Synthetic identities may then be used in attempts to obtain financial products, create accounts or bypass organisational controls.
The challenge for fraud teams is therefore moving beyond simple document inspection toward analysing combinations of identity, device, behavioural and network information.
Why urgency remains one of the biggest warning signs
Despite increasingly sophisticated technology, many scams continue to depend on an old psychological technique: urgency.
The message may say:
Transfer the money now.
Your account is under attack.
Your relative needs immediate help.
This investment opportunity expires today.
Do not tell anyone.
The purpose is to reduce the victim's opportunity to think, verify and seek advice.
The UK's Stop! Think Fraud guidance identifies false urgency as an important warning sign and advises people to be suspicious when someone attempts to rush them into a decision.
AI may improve the presentation of a scam, but the underlying manipulation can remain surprisingly familiar.
How individuals can respond to suspected AI impersonation
Trying to identify every deepfake visually or acoustically is unlikely to be a reliable long-term defence.
Verification is more important.
If someone contacts you unexpectedly asking for money, sensitive information or urgent action, independently verify the request.
For example, end the conversation and contact the person or organisation using a telephone number you already know to be genuine.
Do not rely on contact information supplied within the suspicious message itself.
For family impersonation scams, families may also consider agreeing on a private question or phrase that can be used when an unusual financial request needs verification.
Most importantly, resist pressure to act immediately.
A convincing voice is evidence of what you heard.
It is no longer necessarily proof of who was speaking.
How organisations need to adapt
Businesses face the same problem at a larger scale.
Payment processes, identity verification and internal authorisation procedures should increasingly assume that voices, images and video can potentially be manipulated.
Important transactions may therefore require multiple forms of verification.
Organisations can also analyse behavioural, device, transaction and network information to identify inconsistencies that would not necessarily be visible from an identity document or video call alone.
This is where fraud analytics and AI-assisted detection can become part of the defence against AI-enabled fraud.
The same technological development is therefore operating on both sides of the problem.
Criminals can use AI to improve deception.
Fraud teams can use AI and analytics to improve detection.
AI does not replace the fundamentals of fraud investigation
There is a temptation to treat AI-enabled fraud as an entirely new phenomenon.
In many cases, however, established investigative principles remain highly relevant.
Investigators still need to ask:
What actually happened?
What evidence supports the account?
How was the victim approached?
Where did the money go?
Which accounts, devices or identities are connected?
Can the information be independently verified?
Does the activity form part of a wider pattern?
Technology changes the evidence and the methods available to both sides, but it does not eliminate the need for structured investigation and critical thinking.
The emerging fraud intelligence challenge
The UK Government's current Fraud Strategy recognises generative AI as both an opportunity and a risk and highlights work to improve deepfake detection capabilities.
But deepfake detection alone will not solve the problem.
The broader challenge is determining what information can still be trusted when text, photographs, voices and video can all potentially be generated or manipulated.
For fraud investigators and analysts, that makes corroboration increasingly important.
A single piece of digital evidence may tell part of the story.
Fraud intelligence comes from connecting multiple sources, testing competing explanations and identifying patterns that are difficult for an offender to fabricate consistently.
The future of AI-enabled fraud
Artificial intelligence is unlikely to make traditional fraud techniques disappear.
It is more likely to make many of them faster, cheaper, more personalised and potentially more convincing.
Phishing may become better written.
Impersonation may become more realistic.
Synthetic identities may become harder to distinguish.
Social engineering may become increasingly personalised.
But defensive technology will evolve too.
AI, behavioural analytics, network analysis and improved identity verification can help organisations identify suspicious activity that conventional rules may miss.
The future of fraud may therefore involve an escalating contest between AI-assisted deception and AI-assisted detection.
The organisations—and individuals—best prepared for that environment will be those that understand an increasingly important principle:
Seeing is no longer necessarily believing. Verification matters more than ever.
Comments
Post a Comment