What Happened
The UK NCSC issued a report warning that generative AI will 'almost certainly' increase the volume and impact of cyber attacks over the next two years, lowering the barrier for phishing, fraud, and malware development.[1] The guidance highlights risks from LLM misuse, data leakage, and AI-enabled social engineering and urges organizations of all sizes, including SMEs, to implement secure-by-design practices and governance around AI adoption.[1]
Why It Matters
The NCSC reports that generative AI will almost certainly increase the volume and impact of cyber attacks over the next two years, mainly by improving phishing, social engineering, reconnaissance, and malware-related activity. It also warns that AI lowers the barrier for less-skilled threat actors and may contribute to a broader ransomware threat. CyberSE.AI would treat this as a high-priority malicious AI use risk, with immediate value in executive advisory and adversarial testing to assess exposure to AI-enabled attack methods.
CyberSE Analysis
This signal maps to malicious AI use. Organizations using AI agents, LLM APIs, SaaS integrations, or sensitive data workflows should review whether this class of issue could create unauthorized tool execution, data leakage, weak approval gates, or unmanaged supply-chain exposure.
Recommended Actions
- Restrict AI agent tool permissions and production write paths.
- Review sensitive data access across prompts, logs, embeddings, memory, and SaaS integrations.
- Add human approval workflows for high-impact or state-changing actions.
- Run prompt injection and indirect prompt injection tests against affected workflows.
- Document the owner, control gap, and remediation deadline for this risk class.
Source
https://www.ncsc.gov.uk/report/the-cyber-threat-posed-by-artificial-intelligence