While all manner of legitimate organizations are attempting to understand how to best and safely use artificial intelligence to improve productivity, the Trustwave SpiderLabs' Technology Deep Dive: AI Cyber Arms Race takes a forward-looking view at how adversaries are gaining experience and capability in their ability to turn AI against us.
The report, a supplement to Trustwave SpiderLabs' 2025 Trustwave Risk Radar Report: Technology Sector, covers emerging attack types powered by AI and how AI is being used to enhance the effectiveness of phishing attacks. An additional report, Technology Industry Deep Dive: Dark Web-Powered Supply Chain Attacks, completes SpiderLabs' triple coverage of the tech industry.
Emerging AI Threats
Some of the emerging threats SpiderLabs studied include those targeting other AI systems, particularly large language models (LLMs) and multi-agent systems (MAS).
These include indirect prompt injection attacks that can stealthily manipulate LLMs through hidden instructions in shared content, leading to the exfiltration of sensitive data. Backdoored LLMs pose risks by embedding malicious triggers that can silently alter outputs, such as inserting harmful code.
Then, in agent-to-agent (A2A) communication, threat actors can abuse agent cards to elevate compromised agents, skewing task execution and leaking data. Finally, MAS frameworks are vulnerable when interacting with untrusted entities, as they inherently assume trust, exposing critical blind spots that attackers can exploit.
AI Using AI-themed Phishing Lures
Threat actors know organizations are requesting their workers use one of the many popular LLMs to make themselves more efficient, so they are utilizing this in their phishing campaigns. In one example cited, an attacker used AI to create an email purportedly from ChatGPT's creator asking for login credentials, or is directed to a fake webpage where credit card info is collected.
Defending Against AI Attackers
Fortunately, defenders aren't standing still. Trustwave MailMarshal email security solution is one example of AI-powered defense that can counter these AI-fueled threats. With modules like PageML, URLDeep, and D-Fence, MailMarshal utilizes machine learning to analyze email content, headers, and embedded links in real-time, flagging even sophisticated and evasive phishing attempts.
Please download this, and the other technology sector reports for all the details.
