Preparing Your Business for AI-Driven Cyber Attacks
Research from Keepnet found that AI-generated phishing emails achieve a 54% click-through rate, compared with 12% for manually written ones. Meanwhile, IBM's breach research found that 16% of breaches now involve attackers using AI, with AI-generated phishing accounting for 37% of those cases and deepfake impersonation another 35%.
The UK's National Cyber Security Centre is blunt about where this is heading. Its assessment of AI's impact on cyber threat to 2027 concludes that AI will highly likely increase the volume and impact of cyber intrusions, and that a growing digital divide will open up between organisations keeping pace with AI-enabled threats and those that are not. The good news is that the NCSC also expects this to happen through the enhancement of existing techniques rather than entirely new attack types, which means the fundamentals still matter. They just need to be applied more rigorously, and faster.
What AI Actually Changes for Attackers
AI hasn't invented new categories of attack. It has removed the friction from existing ones.
Phishing no longer looks like phishing. Poor grammar, awkward phrasing and generic greetings were once reliable warning signs. Generative AI removes all of them, producing fluent, personalised emails in seconds by drawing on publicly available information about a company and its staff. The UK government's 2024 cyber security research found that phishing dominated the threat landscape, reported by 84% of businesses, and separate industry research found 46% of SMBs faced AI-generated or advanced phishing schemes in the past year alone.
Impersonation has moved beyond email. Voice cloning can now produce a convincing match from as little as three seconds of audio, which means a recording from a webinar, a voicemail greeting or a conference talk is enough raw material to impersonate a director on a phone call or voice note.
The gap between vulnerability and exploitation is shrinking. The NCSC expects AI to compress the time between a vulnerability being disclosed and being exploited, increasing the volume of attacks against systems that haven't been patched promptly.
The barrier to entry is falling. Less skilled attackers can now run campaigns that previously required real expertise, which widens the pool of people targeting small and mid-sized businesses specifically.
Preparing for this means working across three layers at once: technology, policy and people.
Technology: Harden the Foundations First
The most effective technical responses to AI-enhanced attacks are largely the same controls that defend against conventional ones, applied with less tolerance for gaps.
- Enforce phishing-resistant MFA. AI-generated lures are designed to harvest credentials and session tokens, so SMS codes and push notifications offer limited protection. FIDO2 security keys or passkeys cannot be relayed through a phishing proxy, and should be mandatory for administrators and anyone with access to finance or sensitive data.
- Use email security that analyses behaviour, not just content. Filters that look for known bad links or suspicious wording struggle against freshly generated, well-written messages. Modern tooling that flags unusual sender behaviour, new domains, and out-of-pattern requests catches what content scanning misses, and forms a core part of any phishing defence strategy.
- Shorten your patching window. If AI is shrinking the time between disclosure and exploitation, a monthly patch cycle is no longer fast enough for internet-facing systems. Prioritise critical and actively exploited vulnerabilities for rapid deployment, and use endpoint protection with automated detection and response to contain anything that slips through.
- Monitor for exposed credentials. Attackers use leaked credentials to make AI-written phishing and account takeover attempts far more convincing. Knowing which of your staff credentials are already circulating lets you force resets before they are used against you.
Policy: Remove the Reliance on Spotting Fakes
If a convincing fake can be produced in seconds, policy has to protect people who can't tell the difference.
- Make out-of-band verification mandatory. Any request to change bank details, approve an unusual payment or share sensitive information should be confirmed through a second channel, such as a call to a number already on file, regardless of how authentic the original message appears. This single rule neutralises most deepfake voice and email impersonation attempts.
- Set clear rules for AI tools your own staff use. Staff pasting confidential data into unapproved AI tools creates its own exposure. A short, practical acceptable use policy, covered in our guidance on using AI safely, sets out which tools are approved and what data must never be entered.
- Rehearse your response. AI-enabled attacks move quickly, so a documented and tested incident response plan matters more, not less. Knowing who decides, who contacts whom, and how to isolate affected systems should not be worked out during an incident.
Employee Awareness: Train for the Threats That Exist Now
Awareness training built around spotting typos and odd greetings is out of date. Effective training now focuses on behaviour: pausing on unusual requests, verifying through a second channel, and reporting quickly without fear of blame.
- Train on the realistic scenarios. That means AI-polished email, voice messages that sound like a senior colleague, and video calls where a participant may not be who they appear to be.
- Run regular simulations rather than annual sessions. Short, frequent exercises build habits far more effectively than a once-a-year course, and give you a measurable click and report rate to track.
- Make reporting easy and blame-free. Quick reporting of a suspicious message is often the difference between a contained incident and a breach. Our approach to reducing human cyber risk focuses on exactly this.
At a Glance
| Layer | What AI changes | Practical response |
|---|---|---|
| Technology | Phishing bypasses content filters; exploitation windows shrink | Phishing-resistant MFA, behavioural email security, faster patching |
| Policy | Voice and video can be convincingly faked | Mandatory out-of-band verification, AI acceptable use policy, tested response plan |
| People | Traditional red flags disappear | Behaviour-based training, regular simulations, blame-free reporting |
Frequently Asked Questions
What is an AI-driven cyber attack? It is a conventional attack, such as phishing, impersonation fraud or vulnerability exploitation, that uses AI to make it faster, more personalised or more convincing. The NCSC assesses that AI will mostly enhance existing techniques rather than create entirely new ones.
Are AI-generated phishing emails really more effective? Research from Keepnet found AI-generated phishing achieved a 54% click-through rate against 12% for manually crafted messages, largely because AI removes the spelling errors and awkward phrasing people were trained to look for.
Can deepfake voice and video really be used against a small business? Yes. Voice cloning needs only a few seconds of audio, and the targets are often finance or operations staff asked to action an urgent payment. Mandatory call-back verification using a known number is the most reliable defence.
Is multi-factor authentication still worth it? Absolutely, but the method matters. Phishing-resistant MFA, such as FIDO2 keys or passkeys, holds up against AI-driven credential theft, whereas SMS and push-based approaches can be bypassed through real-time phishing proxies.
What should a small business do first? Start with the controls that remove reliance on human judgement: phishing-resistant MFA for privileged accounts, mandatory verification for payment changes, and a faster patching process for internet-facing systems. Then build awareness training around current scenarios.
Can Protrona help us assess our readiness? Yes. Our security consultancy services include reviewing your email security, access controls, verification processes and staff awareness against current AI-enhanced threats, aligned with the NCSC's published AI and cyber security guidance.
Check how resilient your business is to AI-enhanced attacks
AI has raised the standard of what an attacker can produce, which means resilience now depends on controls and processes that don't rely on anyone spotting a fake. A short review of your technology, policies and training will show where the gaps are. Get in touch to arrange one.